dump_struct is one of the longest functions in the kdoc_parser class,
making it hard to read and reason about. Move the definition of the prefix
transformations out of the function, join them with the definition of
"attribute" (which was defined at the top of the file but only used here),
and reformat the code slightly for shorter line widths.
Just code movement in the end.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-5-corbet@lwn.net
A lot of the regular expressions in this file have extraneous backslashes
that may have been needed in Perl, but aren't helpful here. Take them out
to reduce slightly the visual noise.
Escaping of (){}[] has been left in place, even when unnecessary, for
visual clarity.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-4-corbet@lwn.net
The complex struct_members regex was defined far from its use; bring the
two together. Remove some extraneous backslashes while making the move.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-3-corbet@lwn.net
There were two locations duplicating the logic of stripping private members
and associated comments; coalesce them into one, and add some comments
describing what's going on.
Output change: we now no longer add extraneous white space around macro
definitions.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-2-corbet@lwn.net
The kerneldoc parsing phase gathers all of the information about the
declarations of interest, then passes it through to the output phase as a
dict that is an unstructured blob of information; this organization has its
origins in the Perl version of the program. It results in an interface
that is difficult to reason about, dozen-parameter function calls, and
other ills.
Introduce a new class (KdocItem) to carry this information between the
parser and the output modules, and, step by step, modify the system to use
this class in a more structured way. This could be taken further by
creating a subclass of KdocItem for each declaration type (function,
struct, ...), but that is probably more structure than we need.
The result is (I hope) clearer code, the removal of a bunch of boilerplate,
and no changes to the generated output.
Versions of Python prior to 3.7 do not guarantee to remember the insertion
order of dicts; since kernel-doc depends on that guarantee, running with
such older versions could result in output with reordered sections.
Python 3.9 is the minimum for the kernel as a whole, so this should not be
a problem, but put in a warning just in case somebody tries to use
something older.
Suggested-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
entry.sectcheck is just a duplicate of our list of sections that is only
passed to check_sections(); its main purpose seems to be to avoid checking
the special named sections. Rework check_sections() to not use that field
(which is then deleted), tocheck for the known sections directly, and
tighten up the logic in general.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
They are part of the interface, so use them directly. This allows the
removal of the transitional __dict__ hack in KdocItem.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Get rid of the excess "return" statements in dump_declaration(), along with
a line of never-executed dead code.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Each declaration type passes through the name in a unique field of the
"args" blob - even though we have always just passed the name separately.
Get rid of all the weird names and just use the common version.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Callers to output_declaration() always pass the parameter information from
self.entry; remove all of the boilerplate arguments and just get at that
information directly. Formalize its placement in the KdocItem class.
It would be nice to get rid of parameterlist as well, but that has the
effect of reordering the output of function parameters and struct fields to
match the order in the kerneldoc comment rather than in the declaration.
One could argue about which is more correct, but the ordering has been left
unchanged for now.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Callers of check_sections() join parameterlist into a single string, which
is then immediately split back into the original list. Rather than do all
that, just use parameterlist directly in check_sections().
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The code goes out of its way to create a special list of parameters in
entry.struct_actual that is just like entry.parameterlist, but with extra
junk. The only use of that information, in check_sections(), promptly
strips all the extra junk back out. Drop all that extra work and just use
parameterlist.
No output changes.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The section list always comes directly from the under-construction entry
and is used uniformly. Formalize section handling in the KdocItem class,
and have output_declaration() load the sections directly from the entry,
eliminating a lot of duplicated, verbose parameters.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Python dicts (as of 3.7) are guaranteed to remember the insertion order of
items, so we do not need a separate list for that purpose. Drop the
per-entry sectionlist variable and just rely on native dict ordering.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Add some comments to dump_enum to help the next person who has to figure
out what it is actually doing.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250703184403.274408-8-corbet@lwn.net
Add a set of comments to process_proto_function and reorganize the logic
slightly; no functional change.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250703184403.274408-6-corbet@lwn.net
process_proto_type() is using a complex regex and a "while True" loop to
split a declaration into chunks and, in the end, count brackets. Switch to
using a simpler regex to just do the split directly, and handle each chunk
as it comes. The result is, IMO, easier to understand and reason about.
The old algorithm would occasionally elide the space between function
parameters; see struct rng_alg->generate(), foe example. The only output
difference is to not elide that space, which is more correct.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250703184403.274408-5-corbet@lwn.net
Putting the floor under brcount does not change the output in any way, just
remove it.
Change the termination test from ==0 to <=0 to prevent infinite loops in
case somebody does something truly wacko in the code.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250703184403.274408-4-corbet@lwn.net
process_proto_type() and process_proto_function() reinventing the strip()
string method with a whole series of separate regexes; take all that out
and just use strip().
The previous implementation also (in process_proto_type()) removed C++
comments *after* the above dance, leaving trailing whitespace in that case;
now we do the stripping afterward. This results in exactly one output
change: the removal of a spurious space in the definition of
BACKLIGHT_POWER_REDUCED - see
https://docs.kernel.org/gpu/backlight.html#c.backlight_properties.
I note that we are putting semicolons after #define lines that really
shouldn't be there - a task for another day.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250703184403.274408-2-corbet@lwn.net
Since our output items contain their name, we don't need to pass it
separately.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
This class is intended to replace the unstructured dict used to accumulate
an entry to pass to an output module. For now, it remains unstructured,
but it works well enough that the output classes don't notice the
difference.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Add some comments to dump_enum to help the next person who has to figure
out what it is actually doing.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Add a set of comments to process_proto_function and reorganize the logic
slightly; no functional change.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
process_proto_type() is using a complex regex and a "while True" loop to
split a declaration into chunks and, in the end, count brackets. Switch to
using a simpler regex to just do the split directly, and handle each chunk
as it comes. The result is, IMO, easier to understand and reason about.
The old algorithm would occasionally elide the space between function
parameters; see struct rng_alg->generate(), foe example. The only output
difference is to not elide that space, which is more correct.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Now that "inline_*" are just ordinary parser states, split them into two
separate functions, getting rid of some nested conditional logic.
The original process_inline() would simply ignore lines that didn't match
any of the regexes (those lacking the initial " * " marker). I have
preserved that behavior, but we should perhaps emit a warning instead.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250627184000.132291-9-corbet@lwn.net
The processing of inline kerneldoc comments is a state like the rest, but
it was implemented as a set of separate substates. Just remove the
substate logic and make the inline states normal ones like the rest.
INLINE_ERROR was never actually used for anything, so just take it out.
No changes to the generated output.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250627184000.132291-8-corbet@lwn.net
Reorganize process_export() to eliminate duplicated code, don't look for
exports in states where we don't expect them, and don't bother with normal
state-machine processing if an export declaration has been found.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250627184000.132291-6-corbet@lwn.net
The type_param regex matches "@..." just fine, so the special-case branch
for that in dump_section() is never executed. Just remove it.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250627184000.132291-4-corbet@lwn.net
Rather than having other code mucking around with this bit of internal
state, encapsulate it internally. Accumulate the description as a list of
strings, joining them at the end, which is a more efficient way of building
the text.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250627184000.132291-3-corbet@lwn.net
Putting the floor under brcount does not change the output in any way, just
remove it.
Change the termination test from ==0 to <=0 to prevent infinite loops in
case somebody does something truly wacko in the code.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
process_proto_type() and process_proto_function() reinventing the strip()
string method with a whole series of separate regexes; take all that out
and just use strip().
The previous implementation also (in process_proto_type()) removed C++
comments *after* the above dance, leaving trailing whitespace in that case;
now we do the stripping afterward. This results in exactly one output
change: the removal of a spurious space in the definition of
BACKLIGHT_POWER_REDUCED - see
https://docs.kernel.org/gpu/backlight.html#c.backlight_properties.
I note that we are putting semicolons after #define lines that really
shouldn't be there - a task for another day.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Now that "inline_*" are just ordinary parser states, split them into two
separate functions, getting rid of some nested conditional logic.
The original process_inline() would simply ignore lines that didn't match
any of the regexes (those lacking the initial " * " marker). I have
preserved that behavior, but we should perhaps emit a warning instead.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The processing of inline kerneldoc comments is a state like the rest, but
it was implemented as a set of separate substates. Just remove the
substate logic and make the inline states normal ones like the rest.
INLINE_ERROR was never actually used for anything, so just take it out.
No changes to the generated output.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Reorganize process_export() to eliminate duplicated code, don't look for
exports in states where we don't expect them, and don't bother with normal
state-machine processing if an export declaration has been found.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The type_param regex matches "@..." just fine, so the special-case branch
for that in dump_section() is never executed. Just remove it.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Rather than having other code mucking around with this bit of internal
state, encapsulate it internally. Accumulate the description as a list of
strings, joining them at the end, which is a more efficient way of building
the text.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Move the last SPECIAL_SECTION special case into the proper handler
function, getting rid of more if/then/else logic. The leading-space
tracking was tightened up a bit in the move. Add some comments describing
what is going on.
No changes to the generated output.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250621203512.223189-10-corbet@lwn.net
Now that the function can actually fit into a human brain, add a few
comments. While I was at it, I switched to the trim_whitespace() helper
rather than open-coding it.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250621203512.223189-9-corbet@lwn.net
Separate out the end-of-comment logic into its own helper and remove the
duplicated code introduced earlier.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250621203512.223189-8-corbet@lwn.net
Move the recognition of this state to when we enter it, rather than when we
exit, eliminating some twisty logic along the way.
Some changes in output do result from this shift, generally for kerneldoc
comments that do not quite fit the format. See, for example,
struct irqdomain. As far as I can tell, the new behavior is more correct
in each case.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250621203512.223189-7-corbet@lwn.net
