Commit graph

16 commits

Author SHA1 Message Date
Mauro Carvalho Chehab
01dba1680c docs: kernel_include.py: move apply_range() and add a docstring
While not required, better to have caller functions at the end.
As apply_range() is now called by xref_text(), move it to be
before the latter.

No functional changes.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/a6ce0fd7c03a01338753fd81ed0c4631f78311d6.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:43 -06:00
Mauro Carvalho Chehab
4ad9cabc34 docs: kernel_include.py: append line numbers to better report errors
It is best to point to the original line of code that generated
an error than to point to the beginning of a directive.

Add support for it. It should be noticed that this won't work
for literal or code blocks, as Sphinx will ignore it, pointing
to the beginning of the directive. Yet, when the output is known
to be in ReST format, like on TOC, this makes the error a lot
more easier to be handled.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/a0953af8b71e64aaf2e0ba4593ad39e19587d50a.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:43 -06:00
Mauro Carvalho Chehab
e4d91787de docs: kernel_include.py: add support to generate a TOC table
When generate-cross-refs is used, instead of just implementing
the default of generating a literal block, we can also
generate a ReST file as a TOC.

The advantage is that, by being a ReST file, missing references
will point to the place inside the header file that has the
broken link.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/c0d32cd1ef94017e05984b0a38bd2516f7db21e2.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:43 -06:00
Mauro Carvalho Chehab
9be2a5c3c8 docs: kernel_include.py: move code and literal functions
Simplify run() even more by moving the code which handles
with code and literal blocks to their own functions.

No functional changes.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/78d08dfa3f08adabc30bf93b8a1cde4e19b7bd41.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:43 -06:00
Mauro Carvalho Chehab
67faed5d21 docs: kernel_include.py: remove range restriction for gen docs
Originally, parse-readers were generating an output where
the first two lines were setting a literal block.

The script now gets only the actual parsed data without that,
so it is now safe to allow start-line and end-line parameters
to be handled.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/5dff693860a6a3faade15c24abdc380f09db468d.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:43 -06:00
Mauro Carvalho Chehab
3f7f3d4941 docs: kernel_include.py: move range logic to a separate function
Cleanup run() function by moving the range logic to a separate
function.

Here, I ended checking the current Sphinx implementation, as it
has some extra logic for the range check.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/12fa2204a9e7e309ae4b8694a37ebad9327ca634.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:43 -06:00
Mauro Carvalho Chehab
012e00dda3 docs: kernel_include.py: move rawtext logic to separate functions
The run function is too complex. merge run() and _run() into
a single function and move the read logic to separate functions.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/04776a94c85b6c931c198a149f08b299c9f571a3.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:42 -06:00
Mauro Carvalho Chehab
39f5f2fa8c docs: kernel_include.py: generate warnings for broken refs
In the past, Sphinx used to warn about broken references. That's
basically the rationale for adding media uAPI files: to get
warnings about missed symbols.

This is not true anymore. So, we need to explicitly check them
after doctree-resolved event.

While here, move setup() to the end, to make it closer to
what we do on other extensions.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/73be9a198746421687e2eee916ccf8bf67980b7d.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:42 -06:00
Mauro Carvalho Chehab
0cb6aee358 docs: kernel_include.py: allow cross-reference generation
kernel_include extension was originally designed to be used by the
media comprehensive uAPI documentation, where, instead of simpler
kernel-doc markups, the uAPI documentation is enriched with a larger
text, with images, complex tables, graphs, etc.

There, we wanted to include the much simpler yet documented .h
file.

This extension is needed to include files from other parts of the
Kernel tree outside Documentation, because the original Sphinx
include tag doesn't allow going outside of the directory passed
via sphinx-build command line.

Yet, the cross-references themselves to the full documentation
were using a perl script to create cross-references against the
comprehensive documentation.

As the perl script is now converted to Phython and there is a
Python class producing an include-compatible output with cross
references, add two optional arguments to kernel_include.py:

1. :generate-cross-refs:

        If present, instead of reading the file, it calls ParseDataStructs()
        class, which converts C data structures into cross-references to
        be linked to ReST files containing a more comprehensive documentation;

        Don't use it together with :start-line: and/or :end-line:, as
        filtering input file line range is currently not supported.

2. :exception-file:

        Used together with :generate-cross-refs:. Points to a file containing
        rules to ignore C data structs or to use a different reference name,
        optionally using a different reference type.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/efc39c8e54a2056ae2fdb94d5006fcb19e227198.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:42 -06:00
Mauro Carvalho Chehab
bb97996500 docs: kernel_include.py: Update its coding style
With the help of tools like black, pylint, autopep8 and flake,
improve the code style in preparation for further changes.

No functional changes.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/f64c3af47fdfd632bb5f8eb88e3c7d94b0b84a66.1755872208.git.mchehab+huawei@kernel.org
2025-08-29 15:54:42 -06:00
Mauro Carvalho Chehab
fa5a06e94a docs: sphinx: add missing SPDX tags
Several Sphinx extensions and tools are missing SPDX tags.
Add them.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/1a62226c5fe524eb87bdb80b33bc7ec880a68880.1750585188.git.mchehab+huawei@kernel.org
2025-06-25 12:39:17 -06:00
Mauro Carvalho Chehab
8def404249 docs: extensions: don't use utf-8 syntax for descriptions
None of the descriptions at the Sphinx extensions are using
non-ascii characters, so no need to place them under u"""
notation.

Also, according with:
	https://docs.python.org/3/deprecations/pending-removal-in-3.16.html

the 'u' format code is scheduled to be removed in Python 3.16.

So, let's just use """.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/8a42f6be53464af4b866492a7e9ddf29c0429997.1739254187.git.mchehab+huawei@kernel.org
2025-02-13 11:33:32 -07:00
Akira Yokosawa
d43ddd5c91 docs: kernel_include.py: Cope with docutils 0.21
Running "make htmldocs" on a newly installed Sphinx 7.3.7 ends up in
a build error:

    Sphinx parallel build error:
    AttributeError: module 'docutils.nodes' has no attribute 'reprunicode'

docutils 0.21 has removed nodes.reprunicode, quote from release note [1]:

  * Removed objects:

    docutils.nodes.reprunicode, docutils.nodes.ensure_str()
        Python 2 compatibility hacks

Sphinx 7.3.0 supports docutils 0.21 [2]:

kernel_include.py, whose origin is misc.py of docutils, uses reprunicode.

Upstream docutils removed the offending line from the corresponding file
(docutils/docutils/parsers/rst/directives/misc.py) in January 2022.
Quoting the changelog [3]:

    Deprecate `nodes.reprunicode` and `nodes.ensure_str()`.

    Drop uses of the deprecated constructs (not required with Python 3).

Do the same for kernel_include.py.

Tested against:
  - Sphinx 2.4.5 (docutils 0.17.1)
  - Sphinx 3.4.3 (docutils 0.17.1)
  - Sphinx 5.3.0 (docutils 0.18.1)
  - Sphinx 6.2.1 (docutils 0.19)
  - Sphinx 7.2.6 (docutils 0.20.1)
  - Sphinx 7.3.7 (docutils 0.21.2)

Link: http://www.docutils.org/RELEASE-NOTES.html#release-0-21-2024-04-09 [1]
Link: https://www.sphinx-doc.org/en/master/changes.html#release-7-3-0-released-apr-16-2024 [2]
Link: https://github.com/docutils/docutils/commit/c8471ce47a24 [3]
Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
Cc: stable@vger.kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/faf5fa45-2a9d-4573-9d2e-3930bdc1ed65@gmail.com
2024-05-02 09:50:59 -06:00
Mauro Carvalho Chehab
bcf0a536bf docs: kernel_include.py: add sphinx build dependencies
The files included by kernel-include should be added as build
dependencies, in order for sphinx-build to rebuild the corresponding
docs if any changes at the included file happens.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Link: https://lore.kernel.org/r/d30fc4fa422a13b7e1623d690945c46b58a55e79.1648290305.git.mchehab@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2022-03-28 13:53:46 -06:00
Markus Heiser
b62b9d81a0 docs: sphinx-extensions: add metadata parallel-safe
The setup() function of a Sphinx-extension can return a dictionary. This
is treated by Sphinx as metadata of the extension [1].

With metadata "parallel_read_safe = True" a extension is marked as
save for "parallel reading of source". This is needed if you want
build in parallel with N processes. E.g.:

  make SPHINXOPTS=-j4 htmldocs

will no longer log warnings like:

  WARNING: the foobar extension does not declare if it is safe for
  parallel reading, assuming it isn't - please ask the extension author
  to check and make it explicit.

Add metadata to extensions:

* kernel-doc
* flat-table
* kernel-include

[1] http://www.sphinx-doc.org/en/stable/extdev/#extension-metadata

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Tested-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-01 08:19:02 -06:00
Markus Heiser
037763785a doc-rst: add kernel-include directive
The kernel-include directive is needed to include the auto generated rst
content from a build (pre-) process. E.g. the linux_tv Makefile
generates intermediate reST-files from header files. Since there is a O=
option:

  make O=dir [targets] Locate all output files in "dir"

We need to include intermediate reST files from arbitrary (O=/tmp/foo)
locations:

The 'kernel-include' reST-directive is a replacement for the 'include'
directive. The 'kernel-include' directive expand environment variables
in the path name and allows to include files from arbitrary locations.

.. hint::

  Including files from arbitrary locations (e.g. from '/etc') is a
  security risk for builders. This is why the 'include' directive from
  docutils *prohibit* pathnames pointing to locations *above* the
  filesystem tree where the reST document with the include directive is
  placed.

Substrings of the form $name or ${name} are replaced by the value of
environment variable name. Malformed variable names and references to
non-existing variables are left unchanged.

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-08 09:31:04 -03:00