From patchwork Wed Jan 3 12:54:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 135706 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DFEFD437DF; Wed, 3 Jan 2024 13:54:56 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 783CA402DD; Wed, 3 Jan 2024 13:54:56 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 682BB402DC for ; Wed, 3 Jan 2024 13:54:54 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7E6B1C15; Wed, 3 Jan 2024 04:55:39 -0800 (PST) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9B1D13F64C; Wed, 3 Jan 2024 04:54:52 -0800 (PST) From: Luca Vizzarro To: dev@dpdk.org Cc: Luca Vizzarro , Paul Szczepanek , Thomas Monjalon , Lijuan Tu , =?utf-8?q?Juraj_Linke=C5=A1?= Subject: [PATCH] dts: improve documentation Date: Wed, 3 Jan 2024 12:54:37 +0000 Message-Id: <20240103125438.182098-1-Luca.Vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Improve instructions for installing dependencies, configuring and launching the project. Finally, document the configuration schema by adding more comments to the example and documenting every property and definition. Reviewed-by: Paul Szczepanek Signed-off-by: Luca Vizzarro --- .mailmap | 1 + doc/guides/tools/dts.rst | 255 +++++++++++++++++++++++++++++++++------ dts/conf.yaml | 31 +++-- 3 files changed, 240 insertions(+), 47 deletions(-) diff --git a/.mailmap b/.mailmap index ab0742a382..6326e28d08 100644 --- a/.mailmap +++ b/.mailmap @@ -815,6 +815,7 @@ Louise Kilheeney Louis Luo Louis Peens Luca Boccassi +Luca Vizzarro Luc Pelletier Lukasz Bartosik Lukasz Czapnik diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst index 32c18ee472..31495cad51 100644 --- a/doc/guides/tools/dts.rst +++ b/doc/guides/tools/dts.rst @@ -91,7 +91,7 @@ Setting up DTS environment .. code-block:: console - poetry install + poetry install --no-root poetry shell #. **SSH Connection** @@ -189,72 +189,73 @@ Running DTS ----------- DTS needs to know which nodes to connect to and what hardware to use on those nodes. -Once that's configured, DTS needs a DPDK tarball and it's ready to run. +Once that's configured, DTS needs a DPDK tarball or a git ref ID and it's ready to run. Configuring DTS ~~~~~~~~~~~~~~~ -DTS configuration is split into nodes and executions and build targets within executions. -By default, DTS will try to use the ``dts/conf.yaml`` config file, -which is a template that illustrates what can be configured in DTS: - - .. literalinclude:: ../../../dts/conf.yaml - :language: yaml - :start-at: executions: - +DTS configuration is split into nodes and executions and build targets within executions, +and follows a defined schema as described in `Configuration Schema`_. +By default, DTS will try to use the ``dts/conf.yaml`` :ref:`config file `, +which is a template that illustrates what can be configured in DTS. The user must have :ref:`administrator privileges ` which don't require password authentication. -The other fields are mostly self-explanatory -and documented in more detail in ``dts/framework/config/conf_yaml_schema.json``. DTS Execution ~~~~~~~~~~~~~ -DTS is run with ``main.py`` located in the ``dts`` directory after entering Poetry shell:: +DTS is run with ``main.py`` located in the ``dts`` directory after entering Poetry shell: + +.. code-block:: console - usage: main.py [-h] [--config-file CONFIG_FILE] [--output-dir OUTPUT_DIR] [-t TIMEOUT] - [-v VERBOSE] [-s SKIP_SETUP] [--tarball TARBALL] - [--compile-timeout COMPILE_TIMEOUT] [--test-cases TEST_CASES] - [--re-run RE_RUN] + (dts-py3.10) $ ./main.py --help + usage: main.py [-h] [--config-file CONFIG_FILE] [--output-dir OUTPUT_DIR] [-t TIMEOUT] [-v VERBOSE] + [-s SKIP_SETUP] [--tarball TARBALL] [--compile-timeout COMPILE_TIMEOUT] + [--test-cases TEST_CASES] [--re-run RE_RUN] - Run DPDK test suites. All options may be specified with the environment variables provided in - brackets. Command line arguments have higher priority. + Run DPDK test suites. All options may be specified with the environment variables provided in brackets. + Command line arguments have higher priority. options: -h, --help show this help message and exit --config-file CONFIG_FILE - [DTS_CFG_FILE] configuration file that describes the test cases, SUTs - and targets. (default: conf.yaml) + [DTS_CFG_FILE] configuration file that describes the test cases, SUTs and targets. + (default: conf.yaml) --output-dir OUTPUT_DIR, --output OUTPUT_DIR - [DTS_OUTPUT_DIR] Output directory where dts logs and results are - saved. (default: output) + [DTS_OUTPUT_DIR] Output directory where dts logs and results are saved. (default: + output) -t TIMEOUT, --timeout TIMEOUT - [DTS_TIMEOUT] The default timeout for all DTS operations except for - compiling DPDK. (default: 15) + [DTS_TIMEOUT] The default timeout for all DTS operations except for compiling DPDK. + (default: 15) -v VERBOSE, --verbose VERBOSE - [DTS_VERBOSE] Set to 'Y' to enable verbose output, logging all - messages to the console. (default: N) + [DTS_VERBOSE] Set to 'Y' to enable verbose output, logging all messages to the + console. (default: N) -s SKIP_SETUP, --skip-setup SKIP_SETUP - [DTS_SKIP_SETUP] Set to 'Y' to skip all setup steps on SUT and TG - nodes. (default: N) - --tarball TARBALL, --snapshot TARBALL - [DTS_DPDK_TARBALL] Path to DPDK source code tarball which will be - used in testing. (default: dpdk.tar.xz) + [DTS_SKIP_SETUP] Set to 'Y' to skip all setup steps on SUT and TG nodes. (default: N) + --tarball TARBALL, --snapshot TARBALL, --git-ref TARBALL + [DTS_DPDK_TARBALL] Path to DPDK source code tarball or a git commit ID, tag ID or + tree ID to test. To test local changes, first commit them, then use the commit ID + with this option. (default: dpdk.tar.xz) --compile-timeout COMPILE_TIMEOUT [DTS_COMPILE_TIMEOUT] The timeout for compiling DPDK. (default: 1200) --test-cases TEST_CASES - [DTS_TESTCASES] Comma-separated list of test cases to execute. - Unknown test cases will be silently ignored. (default: ) + [DTS_TESTCASES] Comma-separated list of test cases to execute. Unknown test cases + will be silently ignored. (default: ) --re-run RE_RUN, --re_run RE_RUN - [DTS_RERUN] Re-run each test case the specified amount of times if a - test failure occurs (default: 0) + [DTS_RERUN] Re-run each test case the specified amount of times if a test failure + occurs (default: 0) The brackets contain the names of environment variables that set the same thing. -The minimum DTS needs is a config file and a DPDK tarball. +The minimum DTS needs is a config file and a DPDK tarball or git ref ID. You may pass those to DTS using the command line arguments or use the default paths. +Example command for running DTS with the template configuration and DPDK tag v23.07: + +.. code-block:: console + + (dts-py3.10) $ ./main.py --git-ref v23.07 DTS Results ~~~~~~~~~~~ @@ -335,3 +336,183 @@ There are three tools used in DTS to help with code checking, style and formatti These three tools are all used in ``devtools/dts-check-format.sh``, the DTS code check and format script. Refer to the script for usage: ``devtools/dts-check-format.sh -h``. + +Configuration Schema +-------------------- + +Definitions +~~~~~~~~~~~ + +_`Node name` + *string* – A unique identifier for a node. **Examples**: ``SUT1``, ``TG1``. + +_`ARCH` + *string* – The CPU architecture. **Supported values**: ``x86_64``, ``arm64``, ``ppc64le``. + +_`CPU` + *string* – The CPU microarchitecture. Use ``native`` for x86. **Supported values**: ``native``, ``armv8a``, ``dpaa2``, ``thunderx``, ``xgene1``. + +_`OS` + *string* – The operating system. **Supported values**: ``linux``. + +_`Compiler` + *string* – The compiler used for building DPDK. **Supported values**: ``gcc``, ``clang``, ``icc``, ``mscv``. + +_`Build target` + *object* – Build targets supported by DTS for building DPDK, described as: + + ==================== ========================================================================================= + ``arch`` See `ARCH`_ + ``os`` See `OS`_ + ``cpu`` See `CPU`_ + ``compiler`` See `Compiler`_ + ``compiler_wrapper`` *string* – Value prepended to the CC variable for the DPDK build. + + **Example**: ``ccache`` + ==================== ========================================================================================= + +_`hugepages` + *object* – hugepages described as: + + ==================== ================================================================ + ``amount`` *integer* – The amount of hugepages to configure. + + Hugepage size will be the system default. + ``force_first_numa`` (*optional*, defaults to ``false``) – If ``true``, it forces the + + configuration of hugepages on the first NUMA node. + ==================== ================================================================ + +_`Network port` + *object* – the NIC port described as: + + ====================== ================================================================================= + ``pci`` *string* – the local PCI address of the port. **Example**: ``0000:00:08.0`` + ``os_driver_for_dpdk`` | *string* – this port's device driver when using with DPDK + | When setting up the SUT, DTS will bind the network device to this driver + | for compatibility with DPDK. + + **Examples**: ``vfio-pci``, ``mlx5_core`` + ``os_driver`` | *string* – this port's device driver when **not** using with DPDK + | When tearing down the tests on the SUT, DTS will bind the network device + | *back* to this driver. This driver is meant to be the one that the SUT would + | normally use for this device, or whichever driver it is preferred to leave the + | device bound to after testing. + + **Examples**: ``i40e``, ``mlx5_core`` + ``peer_node`` *string* – the name of the peer node connected to this port. + ``peer_pci`` *string* – the PCI address of the peer node port. **Example**: ``000a:01:00.1`` + ====================== ================================================================================= + +_`Test suite` + *string* – name of the test suite to run. **Examples**: ``hello_world``, ``os_udp`` + +_`Test target` + *object* – selects specific test cases to run from a test suite. Object is described as follows: + + ========= ============================================================================================= + ``suite`` See `Test suite`_ + ``cases`` (*optional*) *array* of *string* – list of the selected test cases in the test suite to run. + + Unknown test cases will be silently ignored. + ========= ============================================================================================= + +Properties +~~~~~~~~~~ + +The configuration requires listing all the execution environments and nodes +involved in the testing. These can be defined with the following properties: + +``executions`` + *array* listing the execution environments. Each entry is described as per the following object: + + +----------------------------+-------------------------------------------------------------------+ + | ``build_targets`` | *array* of `Build target`_ | + +----------------------------+-------------------------------------------------------------------+ + | ``perf`` | *boolean* – Enable performance testing. | + +----------------------------+-------------------------------------------------------------------+ + | ``func`` | *boolean* – Enable functional testing. | + +----------------------------+-------------------------------------------------------------------+ + | ``test_suites`` | *array* of **one of** `Test suite`_ **or** `Test target`_ | + +----------------------------+-------------------------------------------------------------------+ + | ``skip_smoke_tests`` | (*optional*) *boolean* – Allows you to skip smoke testing | + | | if ``true``. | + +----------------------------+-------------------------------------------------------------------+ + | ``system_under_test_node`` | System under test node specified with: | + | +---------------+---------------------------------------------------+ + | | ``node_name`` | See `Node name`_ | + | +---------------+---------------------------------------------------+ + | | ``vdevs`` | (*optional*) *array* of *string* | + | | | | + | | | List of virtual devices passed with the ``--vdev``| + | | | argument to DPDK. **Example**: ``crypto_openssl`` | + +----------------------------+---------------+---------------------------------------------------+ + | ``traffic_generator_node`` | Node name for the traffic generator node. | + +----------------------------+-------------------------------------------------------------------+ + +``nodes`` + *array* listing the nodes. Each entry is described as per the following object: + + +-----------------------+---------------------------------------------------------------------------------------+ + | ``name`` | See `Node name`_ | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``hostname`` | *string* – The network hostname or IP address of this node. | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``user`` | *string* – The SSH user credential to use to login to this node. | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``password`` | (*optional*) *string* – The SSH password credential for this node. | + | | | + | | **NB**: Use only as last resort. SSH keys are **strongly** preferred. | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``arch`` | The architecture of this node. See `ARCH`_ for supported values. | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``os`` | The operating system of this node. See `OS`_ for supported values. | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``lcores`` | | (*optional*, defaults to 1) *string* – Comma-separated list of logical | + | | | cores to use. An empty string means use all lcores. | + | | | + | | **Example**: ``1,2,3,4,5,18-22`` | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``use_first_core`` | (*optional*, defaults to ``false``) *boolean* | + | | | + | | Indicates whether DPDK should use only the first physical core or not. | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``memory_channels`` | (*optional*, defaults to 1) *integer* | + | | | + | | The number of the memory channels to use. | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``hugepages`` | (*optional*) See `hugepages`_. If unset, hugepages won't be configured | + | | | + | | in favour of the system configuration. | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``ports`` | | *array* of `Network port`_ – Describe ports that are **directly** paired with other | + | | | nodes used in conjunction with this one. Both ends of the links must be | + | | | described. If there any inconsistencies DTS won't run. | + | | | + | | **Example**: port 1 of node ``SUT1`` is connected to port 1 of node ``TG1`` etc. | + +-----------------------+---------------------------------------------------------------------------------------+ + | ``traffic_generator`` | (*optional*) Traffic generator, if any, setup on this node described as: | + | +----------+----------------------------------------------------------------------------+ + | | ``type`` | *string* – **Supported values**: *SCAPY* | + +-----------------------+----------+----------------------------------------------------------------------------+ + +.. _configuration_schema_example: + +Example +~~~~~~~ + +The following example (which can be found in ``dts/conf.yaml``) sets up two nodes: + +* ``SUT1`` which is already setup with the DPDK build requirements and any other + required for execution; +* ``TG1`` which already has Scapy installed in the system. + +And they both have two network ports which are physically connected to each other. + +.. note:: + This example assumes that you have setup SSH keys in both the system under test + and traffic generator nodes. + +.. literalinclude:: ../../../dts/conf.yaml + :language: yaml + :start-at: executions: \ No newline at end of file diff --git a/dts/conf.yaml b/dts/conf.yaml index 37967daea0..2d6fa38a2c 100644 --- a/dts/conf.yaml +++ b/dts/conf.yaml @@ -1,65 +1,76 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright 2022-2023 The DPDK contributors +# Copyright 2023 Arm Limited executions: + # define one execution environment - build_targets: - arch: x86_64 os: linux cpu: native + # the combination of the following two makes CC="ccache gcc" compiler: gcc compiler_wrapper: ccache - perf: false - func: true - skip_smoke_tests: false # optional flag that allows you to skip smoke tests - test_suites: + perf: false # disable performance testing + func: true # enable functional testing + skip_smoke_tests: false + test_suites: # the following test suites will be run in their entirety - hello_world - os_udp + # The machine running the DPDK test executable system_under_test_node: node_name: "SUT 1" vdevs: # optional; if removed, vdevs won't be used in the execution - "crypto_openssl" + # Traffic generator node to use for this execution environment traffic_generator_node: "TG 1" nodes: + # Define a system under test node, having two network ports physically + # connected to the corresponding ports in TG 1 (the peer node) - name: "SUT 1" hostname: sut1.change.me.localhost user: dtsuser arch: x86_64 os: linux - lcores: "" - use_first_core: false - memory_channels: 4 + lcores: "" # use all the available logical cores + use_first_core: false # tells DPDK to use any physical core + memory_channels: 4 # tells DPDK to use 4 memory channels hugepages: # optional; if removed, will use system hugepage configuration amount: 256 force_first_numa: false ports: + # sets up the physical link between "SUT 1"@000:00:08.0 and "TG 1"@0000:00:08.0 - pci: "0000:00:08.0" os_driver_for_dpdk: vfio-pci # OS driver that DPDK will use - os_driver: i40e + os_driver: i40e # OS driver to bind when the tests are not running peer_node: "TG 1" peer_pci: "0000:00:08.0" + # sets up the physical link between "SUT 1"@000:00:08.1 and "TG 1"@0000:00:08.1 - pci: "0000:00:08.1" os_driver_for_dpdk: vfio-pci os_driver: i40e peer_node: "TG 1" peer_pci: "0000:00:08.1" + # Define a Scapy traffic generator node, having two network ports + # physically connected to the corresponding ports in SUT 1 (the peer node). - name: "TG 1" hostname: tg1.change.me.localhost user: dtsuser arch: x86_64 os: linux - lcores: "" ports: + # sets up the physical link between "TG 1"@000:00:08.0 and "SUT 1"@0000:00:08.0 - pci: "0000:00:08.0" os_driver_for_dpdk: rdma os_driver: rdma peer_node: "SUT 1" peer_pci: "0000:00:08.0" + # sets up the physical link between "SUT 1"@000:00:08.0 and "TG 1"@0000:00:08.0 - pci: "0000:00:08.1" os_driver_for_dpdk: rdma os_driver: rdma peer_node: "SUT 1" peer_pci: "0000:00:08.1" - use_first_core: false hugepages: # optional; if removed, will use system hugepage configuration amount: 256 force_first_numa: false