[v4,0/3] dts: API docs generation

  The generation is done with Sphinx, which DPDK already uses, with
slightly modified configuration of the sidebar present in an if block.

Dependencies are installed using Poetry from the dts directory:

poetry install --with docs

After installing, enter the Poetry shell:

poetry shell

And then run the build:
ninja -C <meson_build_dir> dts-doc

Python3.10 is required to build the DTS API docs.

The patchset contains the .rst sources which Sphinx uses to generate the
html pages. These were first generated with the sphinx-apidoc utility
and modified to provide a better look. The documentation just doesn't
look that good without the modifications and there isn't enough
configuration options to achieve that without manual changes to the .rst
files. This introduces extra maintenance which involves adding new .rst
files when a new Python module is added or changing the .rst structure
if the Python directory/file structure is changed (moved, renamed
files). This small maintenance burden is outweighed by the flexibility
afforded by the ability to make manual changes to the .rst files.

We can merge this patch when:
1. We agree on the approach with manually modifying the files.
This approach is, in my opinion, much better than just generating the
.rst files every time,
2. Bruce sends his ack on the meson modifications. I believe we had a
positive reaction on the previous version, but not this one.
3. The link to DTS API docs that was added to doxy-api-index.md is
satisfactory. I think Thomas could check this?

v2:
Removed the use of sphinx-apidoc from meson in favor of adding the files
generated by it directly to the repository (and modifying them).

v3:
Rebase.

v4:
Rebase.

Juraj Linkeš (3):
  dts: add doc generation dependencies
  dts: add API doc sources
  dts: add API doc generation

 buildtools/call-sphinx-build.py               |  33 +-
 doc/api/doxy-api-index.md                     |   3 +
 doc/api/doxy-api.conf.in                      |   2 +
 doc/api/meson.build                           |  11 +-
 doc/guides/conf.py                            |  39 +-
 doc/guides/meson.build                        |   1 +
 doc/guides/tools/dts.rst                      |  34 +-
 dts/doc/conf_yaml_schema.json                 |   1 +
 dts/doc/framework.config.rst                  |  12 +
 dts/doc/framework.config.types.rst            |   6 +
 dts/doc/framework.exception.rst               |   6 +
 dts/doc/framework.logger.rst                  |   6 +
 ...ote_session.interactive_remote_session.rst |   6 +
 ...ework.remote_session.interactive_shell.rst |   6 +
 .../framework.remote_session.python_shell.rst |   6 +
 ...ramework.remote_session.remote_session.rst |   6 +
 dts/doc/framework.remote_session.rst          |  17 +
 .../framework.remote_session.ssh_session.rst  |   6 +
 ...framework.remote_session.testpmd_shell.rst |   6 +
 dts/doc/framework.runner.rst                  |   6 +
 dts/doc/framework.settings.rst                |   6 +
 dts/doc/framework.test_result.rst             |   6 +
 dts/doc/framework.test_suite.rst              |   6 +
 dts/doc/framework.testbed_model.cpu.rst       |   6 +
 .../framework.testbed_model.linux_session.rst |   6 +
 dts/doc/framework.testbed_model.node.rst      |   6 +
 .../framework.testbed_model.os_session.rst    |   6 +
 dts/doc/framework.testbed_model.port.rst      |   6 +
 .../framework.testbed_model.posix_session.rst |   6 +
 dts/doc/framework.testbed_model.rst           |  26 +
 dts/doc/framework.testbed_model.sut_node.rst  |   6 +
 dts/doc/framework.testbed_model.tg_node.rst   |   6 +
 ..._generator.capturing_traffic_generator.rst |   6 +
 ...mework.testbed_model.traffic_generator.rst |  14 +
 ....testbed_model.traffic_generator.scapy.rst |   6 +
 ...el.traffic_generator.traffic_generator.rst |   6 +
 ...framework.testbed_model.virtual_device.rst |   6 +
 dts/doc/framework.utils.rst                   |   6 +
 dts/doc/index.rst                             |  41 ++
 dts/doc/meson.build                           |  27 +
 dts/meson.build                               |  16 +
 dts/poetry.lock                               | 499 +++++++++++++++++-
 dts/pyproject.toml                            |   7 +
 meson.build                                   |   1 +
 44 files changed, 920 insertions(+), 20 deletions(-)
 create mode 120000 dts/doc/conf_yaml_schema.json
 create mode 100644 dts/doc/framework.config.rst
 create mode 100644 dts/doc/framework.config.types.rst
 create mode 100644 dts/doc/framework.exception.rst
 create mode 100644 dts/doc/framework.logger.rst
 create mode 100644 dts/doc/framework.remote_session.interactive_remote_session.rst
 create mode 100644 dts/doc/framework.remote_session.interactive_shell.rst
 create mode 100644 dts/doc/framework.remote_session.python_shell.rst
 create mode 100644 dts/doc/framework.remote_session.remote_session.rst
 create mode 100644 dts/doc/framework.remote_session.rst
 create mode 100644 dts/doc/framework.remote_session.ssh_session.rst
 create mode 100644 dts/doc/framework.remote_session.testpmd_shell.rst
 create mode 100644 dts/doc/framework.runner.rst
 create mode 100644 dts/doc/framework.settings.rst
 create mode 100644 dts/doc/framework.test_result.rst
 create mode 100644 dts/doc/framework.test_suite.rst
 create mode 100644 dts/doc/framework.testbed_model.cpu.rst
 create mode 100644 dts/doc/framework.testbed_model.linux_session.rst
 create mode 100644 dts/doc/framework.testbed_model.node.rst
 create mode 100644 dts/doc/framework.testbed_model.os_session.rst
 create mode 100644 dts/doc/framework.testbed_model.port.rst
 create mode 100644 dts/doc/framework.testbed_model.posix_session.rst
 create mode 100644 dts/doc/framework.testbed_model.rst
 create mode 100644 dts/doc/framework.testbed_model.sut_node.rst
 create mode 100644 dts/doc/framework.testbed_model.tg_node.rst
 create mode 100644 dts/doc/framework.testbed_model.traffic_generator.capturing_traffic_generator.rst
 create mode 100644 dts/doc/framework.testbed_model.traffic_generator.rst
 create mode 100644 dts/doc/framework.testbed_model.traffic_generator.scapy.rst
 create mode 100644 dts/doc/framework.testbed_model.traffic_generator.traffic_generator.rst
 create mode 100644 dts/doc/framework.testbed_model.virtual_device.rst
 create mode 100644 dts/doc/framework.utils.rst
 create mode 100644 dts/doc/index.rst
 create mode 100644 dts/doc/meson.build
 create mode 100644 dts/meson.build
  

<snip>
> The patchset contains the .rst sources which Sphinx uses to generate the
> html pages. These were first generated with the sphinx-apidoc utility
> and modified to provide a better look. The documentation just doesn't
> look that good without the modifications and there isn't enough
> configuration options to achieve that without manual changes to the .rst
> files. This introduces extra maintenance which involves adding new .rst
> files when a new Python module is added or changing the .rst structure
> if the Python directory/file structure is changed (moved, renamed
> files). This small maintenance burden is outweighed by the flexibility
> afforded by the ability to make manual changes to the .rst files.
>
> We can merge this patch when:
> 1. We agree on the approach with manually modifying the files.
> This approach is, in my opinion, much better than just generating the
> .rst files every time,

+1 for manually modifying .rst files. The .rst files are very simple,
and I think the added flexibility to change headers or tweak things as
needed is a big benefit over just auto-generating and not having as
much control. Additionally, if it just so happens that the
auto-generated file looks fine and the developer doesn't want to make
changes, they can still just generate it themselves and it fits right
in, so this approach still works with the other regardless.


> 2. Bruce sends his ack on the meson modifications. I believe we had a
> positive reaction on the previous version, but not this one.
> 3. The link to DTS API docs that was added to doxy-api-index.md is
> satisfactory. I think Thomas could check this?
>
<snip>
> --
> 2.34.1
>
  

On Mon, Apr 29, 2024 at 9:49 AM Jeremy Spewock <jspewock@iol.unh.edu> wrote:

> <snip>
> > The patchset contains the .rst sources which Sphinx uses to generate the
> > html pages. These were first generated with the sphinx-apidoc utility
> > and modified to provide a better look. The documentation just doesn't
> > look that good without the modifications and there isn't enough
> > configuration options to achieve that without manual changes to the .rst
> > files. This introduces extra maintenance which involves adding new .rst
> > files when a new Python module is added or changing the .rst structure
> > if the Python directory/file structure is changed (moved, renamed
> > files). This small maintenance burden is outweighed by the flexibility
> > afforded by the ability to make manual changes to the .rst files.
> >
> > We can merge this patch when:
> > 1. We agree on the approach with manually modifying the files.
> > This approach is, in my opinion, much better than just generating the
> > .rst files every time,
>
> +1 for manually modifying .rst files. The .rst files are very simple,
> and I think the added flexibility to change headers or tweak things as
> needed is a big benefit over just auto-generating and not having as
> much control. Additionally, if it just so happens that the
> auto-generated file looks fine and the developer doesn't want to make
> changes, they can still just generate it themselves and it fits right
> in, so this approach still works with the other regardless.
>
> +1