From patchwork Mon Nov 6 17:15:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Juraj_Linke=C5=A1?= X-Patchwork-Id: 367 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B06B7432BB; Mon, 6 Nov 2023 18:16:06 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 83A0240689; Mon, 6 Nov 2023 18:16:06 +0100 (CET) Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by mails.dpdk.org (Postfix) with ESMTP id EAD29402F2 for ; Mon, 6 Nov 2023 18:16:04 +0100 (CET) Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-5437269a661so11359557a12.0 for ; Mon, 06 Nov 2023 09:16:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1699290964; x=1699895764; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yF8Psogq88OG2hB9Ft0F1HqhdjlXBuqeOeBQf1u1u/M=; b=f6CW5j0bj9ZMsweLElaZ89JBRbJ0O9TUx+o/Qb3Ls2bd+4l2IOWQe+Hq+tnIWNF98V iDkHbbP9miw+pdDrhLFFQDgLpaKnhG3QIduo17qeCKoSIPXluKMzf4NWsYy9R40+vPQa GBsQejVvRPKRcnLMdV4NModTmtCRI9L9iKD/PGS2lkZOKLr2pQO6Ps6kZ+AriV3Fz5n1 3BdwZAAZEovVyQMcd+uujV9GlS2r7HzXhvabrud/YFE9OZ1LUeqqCvjE7S5A9IrYs0qr OPqh7cpDdv0SQRfONIRc/ps33PSR9LWkw37HFS0h3AA450A5bzAf+oqE0nMvPbXiQMAy vKQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699290964; x=1699895764; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yF8Psogq88OG2hB9Ft0F1HqhdjlXBuqeOeBQf1u1u/M=; b=GVSjQiZgv76uOH49ad9loc5ZAXmgylWGA7wUV62H6IsyyS0MELAzAxsNKb3fKGAu8C JLrPCrHY2lMfdnNXv8/sybrSJQIJGa9ALkMsCcPQD5pQTImsFAPT++WM2heeAw0E/Knp VkXxGf/Q+wgWYytz52jzWmchN4isRNUdQLTRqU3gJJJbuxMrbp1mUTLacB2wTAx2rB14 NpDZevUIugZvwlM3j5oh149L2DvJvwP7ItYfWpTJnKyU7he9UCM7p3HZMFS6ojzGWadu QheYwxk/UdveVLTGDnMEClm1ncuHZ++YtOQZeLN/9mFZvKN+7ow9ES9SPifBJaRWlHgX SV4Q== X-Gm-Message-State: AOJu0Yyp/RJz8XoUv88IxB4TcLjNRyKJhSn4LMaTNl2UgC3zxNU7Xl1u 4fsHP/Lf4gFMes0XrCdoltajyA== X-Google-Smtp-Source: AGHT+IH/jSgsVaxRteqxb0JPOVtHLrPNpMQOu3HyQcZi2rVhVm2a/1ibf0LgIgwd9P2Jvk8eRjdjyA== X-Received: by 2002:a17:907:2d27:b0:9a2:295a:9bbc with SMTP id gs39-20020a1709072d2700b009a2295a9bbcmr101378ejc.37.1699290964560; Mon, 06 Nov 2023 09:16:04 -0800 (PST) Received: from jlinkes-PT-Latitude-5530.. (ip-46.34.243.197.o2inet.sk. [46.34.243.197]) by smtp.gmail.com with ESMTPSA id s10-20020a170906354a00b009b947aacb4bsm47016eja.191.2023.11.06.09.16.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Nov 2023 09:16:04 -0800 (PST) From: =?utf-8?q?Juraj_Linke=C5=A1?= To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, bruce.richardson@intel.com, jspewock@iol.unh.edu, probb@iol.unh.edu, paul.szczepanek@arm.com, yoan.picchi@foss.arm.com Cc: dev@dpdk.org, =?utf-8?q?Juraj_Linke=C5=A1?= Subject: [PATCH v5 00/23] dts: add dts api docs Date: Mon, 6 Nov 2023 18:15:38 +0100 Message-Id: <20231106171601.160749-1-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230831100407.59865-1-juraj.linkes@pantheon.tech> References: <20230831100407.59865-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The commits can be split into groups. The first commit makes changes to the code. These code changes mainly change the structure of the code so that the actual API docs generation works. There are also some code changes which get reflected in the documentation, such as making functions/methods/attributes private or public. The second set of commits (2-21) deal with the actual docstring documentation (from which the API docs are generated). The format of docstrings is the Google format [0] with PEP257 [1] and some guidelines captured in the last commit of this group covering what the Google format doesn't. The docstring updates are split into many commits to make review possible. When accepted, these may be squashed (commits 4-21). The docstrings have been composed in anticipation of [2], adhering to maximum line length of 100. We don't have a tool for automatic docstring formatting, hence the usage of 100 right away to save time. NOTE: The logger.py module is not fully documented, as it's being refactored and the refactor will be submitted in the near future. Documenting it now seems unnecessary. The last two commits comprise the final group, enabling the actual generation of documentation. The generation is done with Sphinx, which DPDK already uses, with slightly modified configuration (the sidebar: unlimited depth and better collapsing - I need comment on this). The first two groups are the most important to merge, as future development can't proceed without them. The third group may be finished/accepted at a later date, as it's fairly independent. The build requires the same Python version and dependencies as DTS, because Sphinx imports the Python modules. The modules are imported individually, requiring the code refactoring mentioned above. Dependencies are installed using Poetry from the dts directory: poetry install --with docs After installing, enter the Poetry shell: poetry shell And then run the build: ninja -C dts-doc [0] https://google.github.io/styleguide/pyguide.html#s3.8.4-comments-in-classes [1] https://peps.python.org/pep-0257/ [2] https://patches.dpdk.org/project/dpdk/list/?series=29844 Juraj Linkeš (23): dts: code adjustments for doc generation dts: add docstring checker dts: add basic developer docs dts: exceptions docstring update dts: settings docstring update dts: logger and settings docstring update dts: dts runner and main docstring update dts: test suite docstring update dts: test result docstring update dts: config docstring update dts: remote session docstring update dts: interactive remote session docstring update dts: port and virtual device docstring update dts: cpu docstring update dts: os session docstring update dts: posix and linux sessions docstring update dts: node docstring update dts: sut and tg nodes docstring update dts: base traffic generators docstring update dts: scapy tg docstring update dts: test suites docstring update dts: add doc generation dependencies dts: add doc generation buildtools/call-sphinx-build.py | 29 +- doc/api/meson.build | 1 + doc/guides/conf.py | 34 +- doc/guides/meson.build | 1 + doc/guides/tools/dts.rst | 103 ++++ dts/doc/conf_yaml_schema.json | 1 + dts/doc/index.rst | 17 + dts/doc/meson.build | 49 ++ dts/framework/__init__.py | 12 +- dts/framework/config/__init__.py | 379 ++++++++++--- dts/framework/config/types.py | 132 +++++ dts/framework/dts.py | 161 +++++- dts/framework/exception.py | 156 +++--- dts/framework/logger.py | 72 ++- dts/framework/remote_session/__init__.py | 80 ++- .../interactive_remote_session.py | 36 +- .../remote_session/interactive_shell.py | 152 ++++++ dts/framework/remote_session/os_session.py | 284 ---------- dts/framework/remote_session/python_shell.py | 32 ++ .../remote_session/remote/__init__.py | 27 - .../remote/interactive_shell.py | 133 ----- .../remote_session/remote/python_shell.py | 12 - .../remote_session/remote/remote_session.py | 172 ------ .../remote_session/remote/testpmd_shell.py | 49 -- .../remote_session/remote_session.py | 232 ++++++++ .../{remote => }/ssh_session.py | 28 +- dts/framework/remote_session/testpmd_shell.py | 86 +++ dts/framework/settings.py | 188 +++++-- dts/framework/test_result.py | 296 +++++++--- dts/framework/test_suite.py | 230 ++++++-- dts/framework/testbed_model/__init__.py | 28 +- dts/framework/testbed_model/{hw => }/cpu.py | 209 +++++-- dts/framework/testbed_model/hw/__init__.py | 27 - dts/framework/testbed_model/hw/port.py | 60 --- .../testbed_model/hw/virtual_device.py | 16 - .../linux_session.py | 69 ++- dts/framework/testbed_model/node.py | 217 +++++--- dts/framework/testbed_model/os_session.py | 425 +++++++++++++++ dts/framework/testbed_model/port.py | 93 ++++ .../posix_session.py | 85 ++- dts/framework/testbed_model/sut_node.py | 227 +++++--- dts/framework/testbed_model/tg_node.py | 70 +-- .../testbed_model/traffic_generator.py | 72 --- .../traffic_generator/__init__.py | 44 ++ .../capturing_traffic_generator.py | 52 +- .../{ => traffic_generator}/scapy.py | 114 ++-- .../traffic_generator/traffic_generator.py | 87 +++ dts/framework/testbed_model/virtual_device.py | 29 + dts/framework/utils.py | 136 +++-- dts/main.py | 17 +- dts/meson.build | 16 + dts/poetry.lock | 509 +++++++++++++++++- dts/pyproject.toml | 13 +- dts/tests/TestSuite_hello_world.py | 16 +- dts/tests/TestSuite_os_udp.py | 16 +- dts/tests/TestSuite_smoke_tests.py | 53 +- meson.build | 1 + 57 files changed, 4181 insertions(+), 1704 deletions(-) create mode 120000 dts/doc/conf_yaml_schema.json create mode 100644 dts/doc/index.rst create mode 100644 dts/doc/meson.build create mode 100644 dts/framework/config/types.py rename dts/framework/remote_session/{remote => }/interactive_remote_session.py (76%) create mode 100644 dts/framework/remote_session/interactive_shell.py delete mode 100644 dts/framework/remote_session/os_session.py create mode 100644 dts/framework/remote_session/python_shell.py delete mode 100644 dts/framework/remote_session/remote/__init__.py delete mode 100644 dts/framework/remote_session/remote/interactive_shell.py delete mode 100644 dts/framework/remote_session/remote/python_shell.py delete mode 100644 dts/framework/remote_session/remote/remote_session.py delete mode 100644 dts/framework/remote_session/remote/testpmd_shell.py create mode 100644 dts/framework/remote_session/remote_session.py rename dts/framework/remote_session/{remote => }/ssh_session.py (83%) create mode 100644 dts/framework/remote_session/testpmd_shell.py rename dts/framework/testbed_model/{hw => }/cpu.py (50%) delete mode 100644 dts/framework/testbed_model/hw/__init__.py delete mode 100644 dts/framework/testbed_model/hw/port.py delete mode 100644 dts/framework/testbed_model/hw/virtual_device.py rename dts/framework/{remote_session => testbed_model}/linux_session.py (79%) create mode 100644 dts/framework/testbed_model/os_session.py create mode 100644 dts/framework/testbed_model/port.py rename dts/framework/{remote_session => testbed_model}/posix_session.py (74%) delete mode 100644 dts/framework/testbed_model/traffic_generator.py create mode 100644 dts/framework/testbed_model/traffic_generator/__init__.py rename dts/framework/testbed_model/{ => traffic_generator}/capturing_traffic_generator.py (66%) rename dts/framework/testbed_model/{ => traffic_generator}/scapy.py (71%) create mode 100644 dts/framework/testbed_model/traffic_generator/traffic_generator.py create mode 100644 dts/framework/testbed_model/virtual_device.py create mode 100644 dts/meson.build