From patchwork Mon Mar 3 14:57:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 152195 X-Patchwork-Delegate: paul.szczepanek@arm.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CBF8E46331; Mon, 3 Mar 2025 15:57:25 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 48AFD402D2; Mon, 3 Mar 2025 15:57:21 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id CAD66400D7 for ; Mon, 3 Mar 2025 15:57:19 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 42101106F; Mon, 3 Mar 2025 06:57:33 -0800 (PST) Received: from localhost.localdomain (unknown [10.57.64.81]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 700D93F66E; Mon, 3 Mar 2025 06:57:18 -0800 (PST) From: Luca Vizzarro To: dev@dpdk.org Cc: Luca Vizzarro , Paul Szczepanek , Patrick Robb Subject: [PATCH v4 1/7] dts: add tests package to API docs Date: Mon, 3 Mar 2025 16:57:03 +0200 Message-ID: <20250303145709.126126-2-Luca.Vizzarro@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250303145709.126126-1-Luca.Vizzarro@arm.com> References: <20241108134532.130681-1-luca.vizzarro@arm.com> <20250303145709.126126-1-Luca.Vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Luca Vizzarro The test suites are not documented in the API as their respective documentation files are missing. Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Dean Marx --- doc/api/dts/index.rst | 1 + doc/api/dts/tests.TestSuite_blocklist.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_checksum_offload.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_dual_vlan.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_dynamic_config.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_dynamic_queue_conf.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_hello_world.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_l2fwd.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_mac_filter.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_mtu.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_pmd_buffer_scatter.rst | 8 ++++++++ ...ests.TestSuite_port_restart_config_persistency.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_promisc_support.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_queue_start_stop.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_smoke_tests.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_softnic.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_uni_pkt.rst | 8 ++++++++ doc/api/dts/tests.TestSuite_vlan.rst | 8 ++++++++ doc/api/dts/tests.rst | 11 +++++++++++ 19 files changed, 148 insertions(+) create mode 100644 doc/api/dts/tests.TestSuite_blocklist.rst create mode 100644 doc/api/dts/tests.TestSuite_checksum_offload.rst create mode 100644 doc/api/dts/tests.TestSuite_dual_vlan.rst create mode 100644 doc/api/dts/tests.TestSuite_dynamic_config.rst create mode 100644 doc/api/dts/tests.TestSuite_dynamic_queue_conf.rst create mode 100644 doc/api/dts/tests.TestSuite_hello_world.rst create mode 100644 doc/api/dts/tests.TestSuite_l2fwd.rst create mode 100644 doc/api/dts/tests.TestSuite_mac_filter.rst create mode 100644 doc/api/dts/tests.TestSuite_mtu.rst create mode 100644 doc/api/dts/tests.TestSuite_pmd_buffer_scatter.rst create mode 100644 doc/api/dts/tests.TestSuite_port_restart_config_persistency.rst create mode 100644 doc/api/dts/tests.TestSuite_promisc_support.rst create mode 100644 doc/api/dts/tests.TestSuite_queue_start_stop.rst create mode 100644 doc/api/dts/tests.TestSuite_smoke_tests.rst create mode 100644 doc/api/dts/tests.TestSuite_softnic.rst create mode 100644 doc/api/dts/tests.TestSuite_uni_pkt.rst create mode 100644 doc/api/dts/tests.TestSuite_vlan.rst create mode 100644 doc/api/dts/tests.rst diff --git a/doc/api/dts/index.rst b/doc/api/dts/index.rst index 33b05953d2..a11f395e11 100644 --- a/doc/api/dts/index.rst +++ b/doc/api/dts/index.rst @@ -14,6 +14,7 @@ Packages :includehidden: :maxdepth: 1 + tests framework.testbed_model framework.remote_session framework.params diff --git a/doc/api/dts/tests.TestSuite_blocklist.rst b/doc/api/dts/tests.TestSuite_blocklist.rst new file mode 100644 index 0000000000..8d032786a0 --- /dev/null +++ b/doc/api/dts/tests.TestSuite_blocklist.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +blocklist Test Suite +==================== + +.. automodule:: tests.TestSuite_blocklist + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_checksum_offload.rst b/doc/api/dts/tests.TestSuite_checksum_offload.rst new file mode 100644 index 0000000000..8eea87f2e9 --- /dev/null +++ b/doc/api/dts/tests.TestSuite_checksum_offload.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +checksum_offload Test Suite +=========================== + +.. automodule:: tests.TestSuite_checksum_offload + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_dual_vlan.rst b/doc/api/dts/tests.TestSuite_dual_vlan.rst new file mode 100644 index 0000000000..22d8c7195f --- /dev/null +++ b/doc/api/dts/tests.TestSuite_dual_vlan.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +dual_vlan Test Suite +==================== + +.. automodule:: tests.TestSuite_dual_vlan + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_dynamic_config.rst b/doc/api/dts/tests.TestSuite_dynamic_config.rst new file mode 100644 index 0000000000..268ef499b6 --- /dev/null +++ b/doc/api/dts/tests.TestSuite_dynamic_config.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +dynamic_config Test Suite +========================= + +.. automodule:: tests.TestSuite_dynamic_config + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_dynamic_queue_conf.rst b/doc/api/dts/tests.TestSuite_dynamic_queue_conf.rst new file mode 100644 index 0000000000..c142b9dc40 --- /dev/null +++ b/doc/api/dts/tests.TestSuite_dynamic_queue_conf.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +dynamic_queue_conf Test Suite +============================= + +.. automodule:: tests.TestSuite_dynamic_queue_conf + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_hello_world.rst b/doc/api/dts/tests.TestSuite_hello_world.rst new file mode 100644 index 0000000000..10db998987 --- /dev/null +++ b/doc/api/dts/tests.TestSuite_hello_world.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +hello_world Test Suite +====================== + +.. automodule:: tests.TestSuite_hello_world + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_l2fwd.rst b/doc/api/dts/tests.TestSuite_l2fwd.rst new file mode 100644 index 0000000000..5a0ca312e9 --- /dev/null +++ b/doc/api/dts/tests.TestSuite_l2fwd.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +l2fwd Test Suite +================ + +.. automodule:: tests.TestSuite_l2fwd + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_mac_filter.rst b/doc/api/dts/tests.TestSuite_mac_filter.rst new file mode 100644 index 0000000000..a65aaeac0b --- /dev/null +++ b/doc/api/dts/tests.TestSuite_mac_filter.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +mac_filter Test Suite +===================== + +.. automodule:: tests.TestSuite_mac_filter + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_mtu.rst b/doc/api/dts/tests.TestSuite_mtu.rst new file mode 100644 index 0000000000..6e8cb808da --- /dev/null +++ b/doc/api/dts/tests.TestSuite_mtu.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +mtu Test Suite +============== + +.. automodule:: tests.TestSuite_mtu + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_pmd_buffer_scatter.rst b/doc/api/dts/tests.TestSuite_pmd_buffer_scatter.rst new file mode 100644 index 0000000000..cdf30fd879 --- /dev/null +++ b/doc/api/dts/tests.TestSuite_pmd_buffer_scatter.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +pmd_buffer_scatter Test Suite +============================= + +.. automodule:: tests.TestSuite_pmd_buffer_scatter + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_port_restart_config_persistency.rst b/doc/api/dts/tests.TestSuite_port_restart_config_persistency.rst new file mode 100644 index 0000000000..2db4754a01 --- /dev/null +++ b/doc/api/dts/tests.TestSuite_port_restart_config_persistency.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +port_restart_config_persistency Test Suite +========================================== + +.. automodule:: tests.TestSuite_port_restart_config_persistency + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_promisc_support.rst b/doc/api/dts/tests.TestSuite_promisc_support.rst new file mode 100644 index 0000000000..496ee01f2f --- /dev/null +++ b/doc/api/dts/tests.TestSuite_promisc_support.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +promisc_support Test Suite +========================== + +.. automodule:: tests.TestSuite_promisc_support + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_queue_start_stop.rst b/doc/api/dts/tests.TestSuite_queue_start_stop.rst new file mode 100644 index 0000000000..87121676fb --- /dev/null +++ b/doc/api/dts/tests.TestSuite_queue_start_stop.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +queue_start_stop Test Suite +=========================== + +.. automodule:: tests.TestSuite_queue_start_stop + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_smoke_tests.rst b/doc/api/dts/tests.TestSuite_smoke_tests.rst new file mode 100644 index 0000000000..8c78c3025a --- /dev/null +++ b/doc/api/dts/tests.TestSuite_smoke_tests.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +smoke_tests Test Suite +====================== + +.. automodule:: tests.TestSuite_smoke_tests + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_softnic.rst b/doc/api/dts/tests.TestSuite_softnic.rst new file mode 100644 index 0000000000..a091f265ea --- /dev/null +++ b/doc/api/dts/tests.TestSuite_softnic.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +softnic Test Suite +================== + +.. automodule:: tests.TestSuite_softnic + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_uni_pkt.rst b/doc/api/dts/tests.TestSuite_uni_pkt.rst new file mode 100644 index 0000000000..95c5a5a28b --- /dev/null +++ b/doc/api/dts/tests.TestSuite_uni_pkt.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +uni_pkt Test Suite +================== + +.. automodule:: tests.TestSuite_uni_pkt + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.TestSuite_vlan.rst b/doc/api/dts/tests.TestSuite_vlan.rst new file mode 100644 index 0000000000..dfbe6196fd --- /dev/null +++ b/doc/api/dts/tests.TestSuite_vlan.rst @@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +vlan Test Suite +=============== + +.. automodule:: tests.TestSuite_vlan + :members: + :show-inheritance: diff --git a/doc/api/dts/tests.rst b/doc/api/dts/tests.rst new file mode 100644 index 0000000000..fdfa6c1f8d --- /dev/null +++ b/doc/api/dts/tests.rst @@ -0,0 +1,11 @@ +.. SPDX-License-Identifier: BSD-3-Clause + +Test Suites +=========== + +.. toctree:: + :glob: + :hidden: + :maxdepth: 1 + + tests.* From patchwork Mon Mar 3 14:57:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 152196 X-Patchwork-Delegate: paul.szczepanek@arm.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3E32C46331; Mon, 3 Mar 2025 15:57:32 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6BD14402E3; Mon, 3 Mar 2025 15:57:22 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id E06A8402AE for ; Mon, 3 Mar 2025 15:57:20 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5B122113E; Mon, 3 Mar 2025 06:57:34 -0800 (PST) Received: from localhost.localdomain (unknown [10.57.64.81]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 89A1D3F66E; Mon, 3 Mar 2025 06:57:19 -0800 (PST) From: Luca Vizzarro To: dev@dpdk.org Cc: Luca Vizzarro , Paul Szczepanek , Patrick Robb Subject: [PATCH v4 2/7] dts: amend test suites docstring Date: Mon, 3 Mar 2025 16:57:04 +0200 Message-ID: <20250303145709.126126-3-Luca.Vizzarro@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250303145709.126126-1-Luca.Vizzarro@arm.com> References: <20241108134532.130681-1-luca.vizzarro@arm.com> <20250303145709.126126-1-Luca.Vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Luca Vizzarro Amend the test suites docstring to conform to valid rst. Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Dean Marx --- dts/tests/TestSuite_dynamic_config.py | 10 +-- dts/tests/TestSuite_mac_filter.py | 42 +++++----- dts/tests/TestSuite_mtu.py | 114 +++++++++++++------------- 3 files changed, 83 insertions(+), 83 deletions(-) diff --git a/dts/tests/TestSuite_dynamic_config.py b/dts/tests/TestSuite_dynamic_config.py index a4bee2e90b..1fce31a0b5 100644 --- a/dts/tests/TestSuite_dynamic_config.py +++ b/dts/tests/TestSuite_dynamic_config.py @@ -36,13 +36,13 @@ class TestDynamicConfig(TestSuite): 1. Default mode: verify packets are received and forwarded. 2. Disable promiscuous mode: verify that packets are received - only for the packet with destination address matching the port address. + only for the packet with destination address matching the port address. 3. Disable promiscuous mode broadcast: verify that packets with destination - MAC address not matching the port are received and not forwarded, and verify - that broadcast packets are received and forwarded. + MAC address not matching the port are received and not forwarded, and verify + that broadcast packets are received and forwarded. 4. Disable promiscuous mode multicast: verify that packets with destination - MAC address not matching the port are received and not forwarded, and verify - that multicast packets are received and forwarded. + MAC address not matching the port are received and not forwarded, and verify + that multicast packets are received and forwarded. """ def send_packet_and_verify(self, should_receive: bool, mac_address: str) -> None: diff --git a/dts/tests/TestSuite_mac_filter.py b/dts/tests/TestSuite_mac_filter.py index e6c55d3ec6..9dbfec5da2 100644 --- a/dts/tests/TestSuite_mac_filter.py +++ b/dts/tests/TestSuite_mac_filter.py @@ -93,13 +93,13 @@ def test_add_remove_mac_addresses(self) -> None: the properties applied to the PMD at any given time. Test: - Start TestPMD without promiscuous mode. - Send a packet with the port's default mac address. (Should receive) - Send a packet with fake mac address. (Should not receive) - Add fake mac address to the PMD's address pool. - Send a packet with the fake mac address to the PMD. (Should receive) - Remove the fake mac address from the PMD's address pool. - Send a packet with the fake mac address to the PMD. (Should not receive) + * Start TestPMD without promiscuous mode. + * Send a packet with the port's default mac address. (Should receive) + * Send a packet with fake mac address. (Should not receive) + * Add fake mac address to the PMD's address pool. + * Send a packet with the fake mac address to the PMD. (Should receive) + * Remove the fake mac address from the PMD's address pool. + * Send a packet with the fake mac address to the PMD. (Should not receive) """ with TestPmdShell() as testpmd: testpmd.set_promisc(0, enable=False) @@ -127,15 +127,15 @@ def test_invalid_address(self) -> None: built-in hardware address, or exceed their address pools. Test: - Start TestPMD. - Attempt to add an invalid mac address. (Should fail) - Attempt to remove the device's hardware address with no additional addresses in the - address pool. (Should fail) - Add a fake mac address to the pool twice in succession. (Should not create any errors) - Attempt to remove the device's hardware address with other addresses in the address - pool. (Should fail) - Determine the device's mac address pool size, and fill the pool with fake addresses. - Attempt to add another fake mac address, overloading the address pool. (Should fail) + * Start TestPMD. + * Attempt to add an invalid mac address. (Should fail) + * Attempt to remove the device's hardware address with no additional addresses in the + address pool. (Should fail) + * Add a fake mac address to the pool twice in succession. (Should not create any errors) + * Attempt to remove the device's hardware address with other addresses in the address + pool. (Should fail) + * Determine the device's mac address pool size, and fill the pool with fake addresses. + * Attempt to add another fake mac address, overloading the address pool. (Should fail) """ with TestPmdShell() as testpmd: testpmd.start() @@ -185,11 +185,11 @@ def test_multicast_filter(self) -> None: to the PMD. Test: - Start TestPMD without promiscuous mode. - Add a fake multicast address to the PMD's multicast address pool. - Send a packet with the fake multicast address to the PMD. (Should receive) - Remove the fake multicast address from the PMDs multicast address filter. - Send a packet with the fake multicast address to the PMD. (Should not receive) + * Start TestPMD without promiscuous mode. + * Add a fake multicast address to the PMD's multicast address pool. + * Send a packet with the fake multicast address to the PMD. (Should receive) + * Remove the fake multicast address from the PMDs multicast address filter. + * Send a packet with the fake multicast address to the PMD. (Should not receive) """ with TestPmdShell() as testpmd: testpmd.start() diff --git a/dts/tests/TestSuite_mtu.py b/dts/tests/TestSuite_mtu.py index 63e570ba03..af6ab88501 100644 --- a/dts/tests/TestSuite_mtu.py +++ b/dts/tests/TestSuite_mtu.py @@ -48,8 +48,7 @@ def set_up_suite(self) -> None: """Set up the test suite. Setup: - Set traffic generator MTU lengths to a size greater than scope of all - test cases. + Set traffic generator MTU lengths to a size greater than scope of all test cases. """ self.topology.tg_port_egress.configure_mtu(JUMBO_MTU + 200) self.topology.tg_port_ingress.configure_mtu(JUMBO_MTU + 200) @@ -89,12 +88,13 @@ def assess_mtu_boundary(self, testpmd_shell: TestPmdShell, mtu: int) -> None: will not. First, start testpmd and update the MTU. Then ensure the new value appears - on port info for all ports. - Next, start packet capturing and send 3 different lengths of packet and verify - they are handled correctly. - # 1. VENDOR_AGNOSTIC_PADDING units smaller than the MTU specified. - # 2. Equal to the MTU specified. - # 3. VENDOR_AGNOSTIC_PADDING units larger than the MTU specified (should be fragmented). + on port info for all ports. Next, start packet capturing and send 3 different lengths of + packet and verify they are handled correctly: + + 1. VENDOR_AGNOSTIC_PADDING units smaller than the MTU specified. + 2. Equal to the MTU specified. + 3. VENDOR_AGNOSTIC_PADDING units larger than the MTU specified (should be fragmented). + Finally, stop packet capturing. Args: @@ -126,35 +126,35 @@ def test_runtime_mtu_updating_and_forwarding(self) -> None: """Verify runtime MTU adjustments and assess packet forwarding behavior. Test: - Start TestPMD in a paired topology. - Set port MTU to 1500. - Send packets of 1491, 1500 and 1509 bytes. - Verify the first two packets are forwarded and the last is dropped. - - Set port MTU to 2400. - Send packets of 1491, 1500 and 1509 bytes. - Verify all three packets are forwarded. - Send packets of 2391, 2400 and 2409 bytes. - Verify the first two packets are forwarded and the last is dropped. - - Set port MTU to 4800. - Send packets of 1491, 1500 and 1509 bytes. - Verify all three packets are forwarded. - Send packets of 4791, 4800 and 4809 bytes. - Verify the first two packets are forwarded and the last is dropped. - - Set port MTU to 9000. - Send packets of 1491, 1500 and 1509 bytes. - Verify all three packets are forwarded. - Send packets of 8991, 9000 and 9009 bytes. - Verify the first two packets are forwarded and the last is dropped. + * Start TestPMD in a paired topology. + * Set port MTU to 1500. + * Send packets of 1491, 1500 and 1509 bytes. + * Verify the first two packets are forwarded and the last is dropped. + + * Set port MTU to 2400. + * Send packets of 1491, 1500 and 1509 bytes. + * Verify all three packets are forwarded. + * Send packets of 2391, 2400 and 2409 bytes. + * Verify the first two packets are forwarded and the last is dropped. + + * Set port MTU to 4800. + * Send packets of 1491, 1500 and 1509 bytes. + * Verify all three packets are forwarded. + * Send packets of 4791, 4800 and 4809 bytes. + * Verify the first two packets are forwarded and the last is dropped. + + * Set port MTU to 9000. + * Send packets of 1491, 1500 and 1509 bytes. + * Verify all three packets are forwarded. + * Send packets of 8991, 9000 and 9009 bytes. + * Verify the first two packets are forwarded and the last is dropped. Verify: - Verifies the successful forwarding of packets via a search for an inserted payload. - If the payload is found, the packet was transmitted successfully. Otherwise, the packet - is considered dropped. + * Verifies the successful forwarding of packets via a search for an inserted payload. + If the payload is found, the packet was transmitted successfully. Otherwise, the + packet is considered dropped. - Verify that standard MTU packets forward, in addition to packets within the limits of - an MTU size set during runtime. + * Verify that standard MTU packets forward, in addition to packets within the limits of + an MTU size set during runtime. """ with TestPmdShell(tx_offloads=0x8000, mbuf_size=[JUMBO_MTU + 200]) as testpmd: testpmd.set_port_mtu_all(1500, verify=True) @@ -185,16 +185,16 @@ def test_cli_mtu_forwarding_for_std_packets(self) -> None: """Assesses packet forwarding of standard MTU packets after pre-runtime MTU adjustments. Test: - Start TestPMD with MTU size of 1518 bytes, set pre-runtime. - Send packets of size 1491, 1500 and 1509 bytes. - Verify the first two packets are forwarded and the last is dropped. + * Start TestPMD with MTU size of 1518 bytes, set pre-runtime. + * Send packets of size 1491, 1500 and 1509 bytes. + * Verify the first two packets are forwarded and the last is dropped. Verify: - Verifies the successful forwarding of packets via a search for an inserted payload. - If the payload is found, the packet was transmitted successfully. Otherwise, the packet - is considered dropped. + * Verifies the successful forwarding of packets via a search for an inserted payload. + If the payload is found, the packet was transmitted successfully. Otherwise, the + packet is considered dropped. - Verify the first two packets are forwarded and the last is dropped after pre-runtime - MTU modification. + * Verify the first two packets are forwarded and the last is dropped after pre-runtime + MTU modification. """ with TestPmdShell( tx_offloads=0x8000, @@ -215,14 +215,14 @@ def test_cli_jumbo_forwarding_for_jumbo_mtu(self) -> None: """Assess packet forwarding of packets within the bounds of a pre-runtime MTU adjustment. Test: - Start TestPMD with MTU size of 9018 bytes, set pre-runtime. - Send packets of size 8991, 9000 and 1509 bytes. + * Start TestPMD with MTU size of 9018 bytes, set pre-runtime. + * Send packets of size 8991, 9000 and 1509 bytes. Verify: - Verifies the successful forwarding of packets via a search for an inserted payload. - If the payload is found, the packet was transmitted successfully. Otherwise, the packet - is considered dropped. + * Verifies the successful forwarding of packets via a search for an inserted payload. + If the payload is found, the packet was transmitted successfully. Otherwise, the + packet is considered dropped. - Verify that all packets are forwarded after pre-runtime MTU modification. + * Verify that all packets are forwarded after pre-runtime MTU modification. """ with TestPmdShell( tx_offloads=0x8000, @@ -241,16 +241,16 @@ def test_cli_mtu_std_packets_for_jumbo_mtu(self) -> None: """Assess boundary of jumbo MTU value set pre-runtime. Test: - Start TestPMD with MTU size of 9018 bytes, set pre-runtime. - Send a packets of size 8991, 9000 and 9009 bytes. - Verify the first two packets are forwarded and the last is dropped. + * Start TestPMD with MTU size of 9018 bytes, set pre-runtime. + * Send a packets of size 8991, 9000 and 9009 bytes. + * Verify the first two packets are forwarded and the last is dropped. Verify: - Verifies the successful forwarding of packets via a search for an inserted payload. - If the payload is found, the packet was transmitted successfully. Otherwise, the packet - is considered dropped. + * Verifies the successful forwarding of packets via a search for an inserted payload. + If the payload is found, the packet was transmitted successfully. Otherwise, the + packet is considered dropped. - Verify the first two packets are forwarded and the last is dropped after pre-runtime - MTU modification. + * Verify the first two packets are forwarded and the last is dropped after pre-runtime + MTU modification. """ with TestPmdShell( tx_offloads=0x8000, From patchwork Mon Mar 3 14:57:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 152197 X-Patchwork-Delegate: paul.szczepanek@arm.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A848046331; Mon, 3 Mar 2025 15:57:38 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 94B01402ED; Mon, 3 Mar 2025 15:57:23 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 005A3402DF for ; Mon, 3 Mar 2025 15:57:21 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 71FD7106F; Mon, 3 Mar 2025 06:57:35 -0800 (PST) Received: from localhost.localdomain (unknown [10.57.64.81]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A23543F66E; Mon, 3 Mar 2025 06:57:20 -0800 (PST) From: Luca Vizzarro To: dev@dpdk.org Cc: Luca Vizzarro , Paul Szczepanek , Patrick Robb Subject: [PATCH v4 3/7] dts: fix smoke tests docstring Date: Mon, 3 Mar 2025 16:57:05 +0200 Message-ID: <20250303145709.126126-4-Luca.Vizzarro@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250303145709.126126-1-Luca.Vizzarro@arm.com> References: <20241108134532.130681-1-luca.vizzarro@arm.com> <20250303145709.126126-1-Luca.Vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Luca Vizzarro Sphinx autodoc complains of is_blocking being duplicated. This is the case as the parent already holds a docstring for this attribute. Remove the duplication. Fixes: 6ef07151aac4 ("dts: update docstrings") Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Dean Marx --- dts/tests/TestSuite_smoke_tests.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/dts/tests/TestSuite_smoke_tests.py b/dts/tests/TestSuite_smoke_tests.py index a8ea07595f..0474b82bd2 100644 --- a/dts/tests/TestSuite_smoke_tests.py +++ b/dts/tests/TestSuite_smoke_tests.py @@ -30,8 +30,6 @@ class TestSmokeTests(TestSuite): The infrastructure also needs to be tested, as that is also used by all other test suites. Attributes: - is_blocking: This test suite will block the execution of all other test suites - in the test run after it. nics_in_node: The NICs present on the SUT node. """ From patchwork Mon Mar 3 14:57:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 152198 X-Patchwork-Delegate: paul.szczepanek@arm.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A120146331; Mon, 3 Mar 2025 15:57:47 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6510540611; Mon, 3 Mar 2025 15:57:25 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id F1CB2402E4 for ; Mon, 3 Mar 2025 15:57:22 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8919E113E; Mon, 3 Mar 2025 06:57:36 -0800 (PST) Received: from localhost.localdomain (unknown [10.57.64.81]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B9D573F66E; Mon, 3 Mar 2025 06:57:21 -0800 (PST) From: Luca Vizzarro To: dev@dpdk.org Cc: Luca Vizzarro , Paul Szczepanek , Patrick Robb Subject: [PATCH v4 4/7] dts: update autodoc sorting order Date: Mon, 3 Mar 2025 16:57:06 +0200 Message-ID: <20250303145709.126126-5-Luca.Vizzarro@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250303145709.126126-1-Luca.Vizzarro@arm.com> References: <20241108134532.130681-1-luca.vizzarro@arm.com> <20250303145709.126126-1-Luca.Vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Luca Vizzarro The autodoc member sorting order default is set to alphabetical, which translates to autodoc sorting every member in modules, classes etc. This also brings some side effects, like sorting capabilities which can't be compared and result in errors. This change prevents autodoc from sorting, and keeping the order as the developer intended it. Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Dean Marx --- doc/guides/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/guides/conf.py b/doc/guides/conf.py index da139df1bd..565f5705d8 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -87,6 +87,7 @@ autodoc_typehints = 'both' autodoc_typehints_format = 'short' autodoc_typehints_description_target = 'documented' + autodoc_member_order = 'bysource' # DTS docstring options. add_module_names = False From patchwork Mon Mar 3 14:57:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 152199 X-Patchwork-Delegate: paul.szczepanek@arm.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 35D3B46331; Mon, 3 Mar 2025 15:57:53 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DDB3540647; Mon, 3 Mar 2025 15:57:26 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 2057640395 for ; Mon, 3 Mar 2025 15:57:24 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A5027106F; Mon, 3 Mar 2025 06:57:37 -0800 (PST) Received: from localhost.localdomain (unknown [10.57.64.81]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D21623F66E; Mon, 3 Mar 2025 06:57:22 -0800 (PST) From: Luca Vizzarro To: dev@dpdk.org Cc: Luca Vizzarro , Paul Szczepanek , Patrick Robb Subject: [PATCH v4 5/7] dts: run only one test run per execution Date: Mon, 3 Mar 2025 16:57:07 +0200 Message-ID: <20250303145709.126126-6-Luca.Vizzarro@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250303145709.126126-1-Luca.Vizzarro@arm.com> References: <20241108134532.130681-1-luca.vizzarro@arm.com> <20250303145709.126126-1-Luca.Vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Luca Vizzarro To aid configurability and flexibility both for the user and the developer, make DTS run only one test run per execution. Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Dean Marx --- doc/guides/tools/dts.rst | 18 ++++---- dts/.gitignore | 2 +- dts/framework/config/__init__.py | 73 +++++++++++++++----------------- dts/framework/config/test_run.py | 15 +++---- dts/framework/runner.py | 14 +++--- dts/framework/settings.py | 16 +++---- dts/test_run.example.yaml | 43 +++++++++++++++++++ dts/test_runs.example.yaml | 43 ------------------- dts/tests_config.example.yaml | 0 9 files changed, 107 insertions(+), 117 deletions(-) create mode 100644 dts/test_run.example.yaml delete mode 100644 dts/test_runs.example.yaml create mode 100644 dts/tests_config.example.yaml diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst index 097813e310..2affcf5d39 100644 --- a/doc/guides/tools/dts.rst +++ b/doc/guides/tools/dts.rst @@ -206,12 +206,12 @@ and then run the tests with the newly built binaries. Configuring DTS ~~~~~~~~~~~~~~~ -DTS configuration is split into nodes and test runs, +DTS configuration is split into nodes and a test run, and must respect the model definitions as documented in the DTS API docs under the ``config`` page. The root of the configuration is represented by the ``Configuration`` model. -By default, DTS will try to use the ``dts/test_runs.example.yaml`` -:ref:`config file `, +By default, DTS will try to use the ``dts/test_run.example.yaml`` +:ref:`config file `, and ``dts/nodes.example.yaml`` :ref:`config file ` which are templates that illustrate what can be configured in DTS. @@ -228,7 +228,7 @@ DTS is run with ``main.py`` located in the ``dts`` directory after entering Poet .. code-block:: console (dts-py3.10) $ ./main.py --help - usage: main.py [-h] [--test-runs-config-file FILE_PATH] [--nodes-config-file FILE_PATH] [--output-dir DIR_PATH] [-t SECONDS] [-v] + usage: main.py [-h] [--test-run-config-file FILE_PATH] [--nodes-config-file FILE_PATH] [--output-dir DIR_PATH] [-t SECONDS] [-v] [--dpdk-tree DIR_PATH | --tarball FILE_PATH] [--remote-source] [--precompiled-build-dir DIR_NAME] [--compile-timeout SECONDS] [--test-suite TEST_SUITE [TEST_CASES ...]] [--re-run N_TIMES] [--random-seed NUMBER] @@ -237,8 +237,8 @@ DTS is run with ``main.py`` located in the ``dts`` directory after entering Poet options: -h, --help show this help message and exit - --test-runs-config-file FILE_PATH - [DTS_TEST_RUNS_CFG_FILE] The configuration file that describes the test cases and DPDK build options. (default: test-runs.conf.yaml) + --test-run-config-file FILE_PATH + [DTS_TEST_RUN_CFG_FILE] The configuration file that describes the test cases and DPDK build options. (default: test-run.conf.yaml) --nodes-config-file FILE_PATH [DTS_NODES_CFG_FILE] The configuration file that describes the SUT and TG nodes. (default: nodes.conf.yaml) --output-dir DIR_PATH, --output DIR_PATH @@ -486,12 +486,12 @@ And they both have two network ports which are physically connected to each othe This example assumes that you have setup SSH keys in both the system under test and traffic generator nodes. -.. _test_runs_configuration_example: +.. _test_run_configuration_example: -``dts/test_runs.example.yaml`` +``dts/test_run.example.yaml`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. literalinclude:: ../../../dts/test_runs.example.yaml +.. literalinclude:: ../../../dts/test_run.example.yaml :language: yaml :start-at: # Define diff --git a/dts/.gitignore b/dts/.gitignore index d53a2f3b7e..96ee0b6ac4 100644 --- a/dts/.gitignore +++ b/dts/.gitignore @@ -1,4 +1,4 @@ # default configuration files for DTS nodes.yaml -test_runs.yaml +test_run.yaml diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py index c42eacb748..5495cfc5d5 100644 --- a/dts/framework/config/__init__.py +++ b/dts/framework/config/__init__.py @@ -13,7 +13,7 @@ The configuration files are split in: - * A list of test run which are represented by :class:`~.test_run.TestRunConfiguration` + * The test run which is represented by :class:`~.test_run.TestRunConfiguration` defining what tests are going to be run and how DPDK will be built. It also references the testbed where these tests and DPDK are going to be run, * A list of the nodes of the testbed which ar represented by :class:`~.node.NodeConfiguration`. @@ -40,16 +40,14 @@ from .node import NodeConfiguration from .test_run import TestRunConfiguration -TestRunsConfig = Annotated[list[TestRunConfiguration], Field(min_length=1)] - NodesConfig = Annotated[list[NodeConfiguration], Field(min_length=1)] class Configuration(FrozenModel): """DTS testbed and test configuration.""" - #: Test run configurations. - test_runs: TestRunsConfig + #: Test run configuration. + test_run: TestRunConfiguration #: Node configurations. nodes: NodesConfig @@ -68,40 +66,36 @@ def validate_node_names(self) -> Self: @model_validator(mode="after") def validate_port_links(self) -> Self: - """Validate that all the test runs' port links are valid.""" + """Validate that all of the test run's port links are valid.""" existing_port_links: dict[tuple[str, str], Literal[False] | tuple[str, str]] = { (node.name, port.name): False for node in self.nodes for port in node.ports } defined_port_links = [ - (test_run_idx, test_run, link_idx, link) - for test_run_idx, test_run in enumerate(self.test_runs) - for link_idx, link in enumerate(test_run.port_topology) + (link_idx, link) for link_idx, link in enumerate(self.test_run.port_topology) ] - for test_run_idx, test_run, link_idx, link in defined_port_links: + for link_idx, link in defined_port_links: sut_node_port_peer = existing_port_links.get( - (test_run.system_under_test_node, link.sut_port), None - ) - assert sut_node_port_peer is not None, ( - "Invalid SUT node port specified for link " - f"test_runs.{test_run_idx}.port_topology.{link_idx}." + (self.test_run.system_under_test_node, link.sut_port), None ) + assert ( + sut_node_port_peer is not None + ), f"Invalid SUT node port specified for link port_topology.{link_idx}." assert sut_node_port_peer is False or sut_node_port_peer == link.right, ( - f"The SUT node port for link test_runs.{test_run_idx}.port_topology.{link_idx} is " + f"The SUT node port for link port_topology.{link_idx} is " f"already linked to port {sut_node_port_peer[0]}.{sut_node_port_peer[1]}." ) tg_node_port_peer = existing_port_links.get( - (test_run.traffic_generator_node, link.tg_port), None - ) - assert tg_node_port_peer is not None, ( - "Invalid TG node port specified for link " - f"test_runs.{test_run_idx}.port_topology.{link_idx}." + (self.test_run.traffic_generator_node, link.tg_port), None ) + assert ( + tg_node_port_peer is not None + ), f"Invalid TG node port specified for link port_topology.{link_idx}." assert tg_node_port_peer is False or sut_node_port_peer == link.left, ( - f"The TG node port for link test_runs.{test_run_idx}.port_topology.{link_idx} is " + f"The TG node port for link port_topology.{link_idx} is " f"already linked to port {tg_node_port_peer[0]}.{tg_node_port_peer[1]}." ) @@ -111,24 +105,21 @@ def validate_port_links(self) -> Self: return self @model_validator(mode="after") - def validate_test_runs_against_nodes(self) -> Self: - """Validate the test runs to nodes associations.""" - for test_run_no, test_run in enumerate(self.test_runs): - sut_node_name = test_run.system_under_test_node - sut_node = next((n for n in self.nodes if n.name == sut_node_name), None) - - assert sut_node is not None, ( - f"Test run {test_run_no}.system_under_test_node " - f"({sut_node_name}) is not a valid node name." - ) + def validate_test_run_against_nodes(self) -> Self: + """Validate the test run against the supplied nodes.""" + sut_node_name = self.test_run.system_under_test_node + sut_node = next((n for n in self.nodes if n.name == sut_node_name), None) - tg_node_name = test_run.traffic_generator_node - tg_node = next((n for n in self.nodes if n.name == tg_node_name), None) + assert ( + sut_node is not None + ), f"The system_under_test_node {sut_node_name} is not a valid node name." - assert tg_node is not None, ( - f"Test run {test_run_no}.traffic_generator_name " - f"({tg_node_name}) is not a valid node name." - ) + tg_node_name = self.test_run.traffic_generator_node + tg_node = next((n for n in self.nodes if n.name == tg_node_name), None) + + assert ( + tg_node is not None + ), f"The traffic_generator_name {tg_node_name} is not a valid node name." return self @@ -160,10 +151,12 @@ def load_config(ctx: ValidationContext) -> Configuration: Raises: ConfigurationError: If the supplied configuration files are invalid. """ - test_runs = _load_and_parse_model(ctx["settings"].test_runs_config_path, TestRunsConfig, ctx) + test_run = _load_and_parse_model( + ctx["settings"].test_run_config_path, TestRunConfiguration, ctx + ) nodes = _load_and_parse_model(ctx["settings"].nodes_config_path, NodesConfig, ctx) try: - return Configuration.model_validate({"test_runs": test_runs, "nodes": nodes}, context=ctx) + return Configuration.model_validate({"test_run": test_run, "nodes": nodes}, context=ctx) except ValidationError as e: raise ConfigurationError("the configurations supplied are invalid") from e diff --git a/dts/framework/config/test_run.py b/dts/framework/config/test_run.py index 1b3045730d..c1e534e480 100644 --- a/dts/framework/config/test_run.py +++ b/dts/framework/config/test_run.py @@ -206,14 +206,13 @@ class TestSuiteConfig(FrozenModel): .. code:: yaml - test_runs: - - test_suites: - # As string representation: - - hello_world # test all of `hello_world`, or - - hello_world hello_world_single_core # test only `hello_world_single_core` - # or as model fields: - - test_suite: hello_world - test_cases: [hello_world_single_core] # without this field all test cases are run + test_suites: + # As string representation: + - hello_world # test all of `hello_world`, or + - hello_world hello_world_single_core # test only `hello_world_single_core` + # or as model fields: + - test_suite: hello_world + test_cases: [hello_world_single_core] # without this field all test cases are run """ #: The name of the test suite module without the starting ``TestSuite_``. diff --git a/dts/framework/runner.py b/dts/framework/runner.py index 801709a2aa..a0016d5d57 100644 --- a/dts/framework/runner.py +++ b/dts/framework/runner.py @@ -6,7 +6,7 @@ """Test suite runner module. -The module is responsible for preparing DTS and running the test runs. +The module is responsible for preparing DTS and running the test run. """ import os @@ -47,8 +47,8 @@ def __init__(self): def run(self) -> None: """Run DTS. - Prepare all the nodes ahead of the test runs execution, - which are subsequently run as configured. + Prepare all the nodes ahead of the test run execution, which is subsequently run as + configured. """ nodes: list[Node] = [] try: @@ -59,11 +59,9 @@ def run(self) -> None: for node_config in self._configuration.nodes: nodes.append(Node(node_config)) - # for all test run sections - for test_run_config in self._configuration.test_runs: - test_run_result = self._result.add_test_run(test_run_config) - test_run = TestRun(test_run_config, nodes, test_run_result) - test_run.spin() + test_run_result = self._result.add_test_run(self._configuration.test_run) + test_run = TestRun(self._configuration.test_run, nodes, test_run_result) + test_run.spin() except Exception as e: self._logger.exception("An unexpected error has occurred.") diff --git a/dts/framework/settings.py b/dts/framework/settings.py index cf82a7c18f..256afd5cfb 100644 --- a/dts/framework/settings.py +++ b/dts/framework/settings.py @@ -14,10 +14,10 @@ The command line arguments along with the supported environment variables are: -.. option:: --test-runs-config-file -.. envvar:: DTS_TEST_RUNS_CFG_FILE +.. option:: --test-run-config-file +.. envvar:: DTS_TEST_RUN_CFG_FILE - The path to the YAML configuration file of the test runs. + The path to the YAML configuration file of the test run. .. option:: --nodes-config-file .. envvar:: DTS_NODES_CFG_FILE @@ -125,7 +125,7 @@ class Settings: """ #: - test_runs_config_path: Path = Path(__file__).parent.parent.joinpath("test_runs.yaml") + test_run_config_path: Path = Path(__file__).parent.parent.joinpath("test_run.yaml") #: nodes_config_path: Path = Path(__file__).parent.parent.joinpath("nodes.yaml") #: @@ -323,14 +323,14 @@ def _get_parser() -> _DTSArgumentParser: ) action = parser.add_argument( - "--test-runs-config-file", - default=SETTINGS.test_runs_config_path, + "--test-run-config-file", + default=SETTINGS.test_run_config_path, type=Path, help="The configuration file that describes the test cases and DPDK build options.", metavar="FILE_PATH", - dest="test_runs_config_path", + dest="test_run_config_path", ) - _add_env_var_to_action(action, "TEST_RUNS_CFG_FILE") + _add_env_var_to_action(action, "TEST_RUN_CFG_FILE") action = parser.add_argument( "--nodes-config-file", diff --git a/dts/test_run.example.yaml b/dts/test_run.example.yaml new file mode 100644 index 0000000000..330a31bb18 --- /dev/null +++ b/dts/test_run.example.yaml @@ -0,0 +1,43 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2022-2023 The DPDK contributors +# Copyright 2023 Arm Limited + +# Define the test run environment +dpdk: + lcores: "" # use all available logical cores (Skips first core) + memory_channels: 4 # tells DPDK to use 4 memory channels + build: + dpdk_location: + # dpdk_tree: Commented out because `tarball` is defined. + tarball: dpdk-tarball.tar.xz + # Either `dpdk_tree` or `tarball` can be defined, but not both. + remote: false # Optional, defaults to false. If it's true, the `dpdk_tree` or `tarball` + # is located on the SUT node, instead of the execution host. + + # precompiled_build_dir: Commented out because `build_options` is defined. + build_options: + # the combination of the following two makes CC="ccache gcc" + compiler: gcc + compiler_wrapper: ccache # Optional. + # If `precompiled_build_dir` is defined, DPDK has been pre-built and the build directory is + # in a subdirectory of DPDK tree root directory. Otherwise, will be using the `build_options` + # to build the DPDK from source. Either `precompiled_build_dir` or `build_options` can be + # defined, but not both. +traffic_generator: + type: SCAPY +perf: false # disable performance testing +func: true # enable functional testing +skip_smoke_tests: false # optional +# by removing the `test_suites` field, this test run will run every test suite available +test_suites: # the following test suites will be run in their entirety + - hello_world +vdevs: # optional; if removed, vdevs won't be used in the execution + - "crypto_openssl" +# The machine running the DPDK test executable +system_under_test_node: "SUT 1" +# Traffic generator node to use for this execution environment +traffic_generator_node: "TG 1" +port_topology: + - sut.port-0 <-> tg.port-0 # explicit link. `sut` and `tg` are special identifiers that refer + # to the respective test run's configured nodes. + - port-1 <-> port-1 # implicit link, left side is always SUT, right side is always TG. diff --git a/dts/test_runs.example.yaml b/dts/test_runs.example.yaml deleted file mode 100644 index 4f225d1b82..0000000000 --- a/dts/test_runs.example.yaml +++ /dev/null @@ -1,43 +0,0 @@ -# SPDX-License-Identifier: BSD-3-Clause -# Copyright 2022-2023 The DPDK contributors -# Copyright 2023 Arm Limited - -# Define one test run environment -- dpdk: - lcores: "" # use all available logical cores (Skips first core) - memory_channels: 4 # tells DPDK to use 4 memory channels - build: - dpdk_location: - # dpdk_tree: Commented out because `tarball` is defined. - tarball: dpdk-tarball.tar.xz - # Either `dpdk_tree` or `tarball` can be defined, but not both. - remote: false # Optional, defaults to false. If it's true, the `dpdk_tree` or `tarball` - # is located on the SUT node, instead of the execution host. - - # precompiled_build_dir: Commented out because `build_options` is defined. - build_options: - # the combination of the following two makes CC="ccache gcc" - compiler: gcc - compiler_wrapper: ccache # Optional. - # If `precompiled_build_dir` is defined, DPDK has been pre-built and the build directory is - # in a subdirectory of DPDK tree root directory. Otherwise, will be using the `build_options` - # to build the DPDK from source. Either `precompiled_build_dir` or `build_options` can be - # defined, but not both. - traffic_generator: - type: SCAPY - perf: false # disable performance testing - func: true # enable functional testing - skip_smoke_tests: false # optional - # by removing the `test_suites` field, this test run will run every test suite available - test_suites: # the following test suites will be run in their entirety - - hello_world - vdevs: # optional; if removed, vdevs won't be used in the execution - - "crypto_openssl" - # The machine running the DPDK test executable - system_under_test_node: "SUT 1" - # Traffic generator node to use for this execution environment - traffic_generator_node: "TG 1" - port_topology: - - sut.port-0 <-> tg.port-0 # explicit link. `sut` and `tg` are special identifiers that refer - # to the respective test run's configured nodes. - - port-1 <-> port-1 # implicit link, left side is always SUT, right side is always TG. diff --git a/dts/tests_config.example.yaml b/dts/tests_config.example.yaml new file mode 100644 index 0000000000..e69de29bb2 From patchwork Mon Mar 3 14:57:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 152200 X-Patchwork-Delegate: paul.szczepanek@arm.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 95ED846331; Mon, 3 Mar 2025 15:57:59 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C837940608; Mon, 3 Mar 2025 15:57:27 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 2DB814060A for ; Mon, 3 Mar 2025 15:57:25 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BFFFB113E; Mon, 3 Mar 2025 06:57:38 -0800 (PST) Received: from localhost.localdomain (unknown [10.57.64.81]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id EE2F93F66E; Mon, 3 Mar 2025 06:57:23 -0800 (PST) From: Luca Vizzarro To: dev@dpdk.org Cc: Luca Vizzarro , Paul Szczepanek , Patrick Robb Subject: [PATCH v4 6/7] dts: add per-test-suite configuration Date: Mon, 3 Mar 2025 16:57:08 +0200 Message-ID: <20250303145709.126126-7-Luca.Vizzarro@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250303145709.126126-1-Luca.Vizzarro@arm.com> References: <20241108134532.130681-1-luca.vizzarro@arm.com> <20250303145709.126126-1-Luca.Vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Luca Vizzarro Allow test suites to be configured individually. Moreover enable them to implement their own custom configuration. Adds a new argument to the command line which enables the user to supply a YAML file to configure all of the requested test suites. This argument becomes mandatory only if any of the selected test suites' configuration does not provide defaults for at least one field. The configuration file is a simple mapping of test suite names to their corresponding configurations. The validation object is created dynamically, implementing only the selected test suites at runtime. This is needed so that any test suites that require manual configuration, won't complain if they are not selected. Bugzilla ID: 1375 Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Dean Marx --- doc/guides/tools/dts.rst | 9 ++++--- dts/framework/config/__init__.py | 39 +++++++++++++++++++++++++++--- dts/framework/config/test_run.py | 36 ++++++++++++++++++++++++--- dts/framework/runner.py | 7 +++++- dts/framework/settings.py | 17 +++++++++++++ dts/framework/test_run.py | 23 ++++++++++++------ dts/framework/test_suite.py | 25 +++++++++++++------ dts/tests/TestSuite_hello_world.py | 14 +++++++++-- dts/tests_config.example.yaml | 2 ++ 9 files changed, 144 insertions(+), 28 deletions(-) diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst index 2affcf5d39..fcc6d22036 100644 --- a/doc/guides/tools/dts.rst +++ b/doc/guides/tools/dts.rst @@ -228,9 +228,10 @@ DTS is run with ``main.py`` located in the ``dts`` directory after entering Poet .. code-block:: console (dts-py3.10) $ ./main.py --help - usage: main.py [-h] [--test-run-config-file FILE_PATH] [--nodes-config-file FILE_PATH] [--output-dir DIR_PATH] [-t SECONDS] [-v] - [--dpdk-tree DIR_PATH | --tarball FILE_PATH] [--remote-source] [--precompiled-build-dir DIR_NAME] - [--compile-timeout SECONDS] [--test-suite TEST_SUITE [TEST_CASES ...]] [--re-run N_TIMES] [--random-seed NUMBER] + usage: main.py [-h] [--test-run-config-file FILE_PATH] [--nodes-config-file FILE_PATH] [--tests-config-file FILE_PATH] + [--output-dir DIR_PATH] [-t SECONDS] [-v] [--dpdk-tree DIR_PATH | --tarball FILE_PATH] [--remote-source] + [--precompiled-build-dir DIR_NAME] [--compile-timeout SECONDS] [--test-suite TEST_SUITE [TEST_CASES ...]] + [--re-run N_TIMES] [--random-seed NUMBER] Run DPDK test suites. All options may be specified with the environment variables provided in brackets. Command line arguments have higher priority. @@ -241,6 +242,8 @@ DTS is run with ``main.py`` located in the ``dts`` directory after entering Poet [DTS_TEST_RUN_CFG_FILE] The configuration file that describes the test cases and DPDK build options. (default: test-run.conf.yaml) --nodes-config-file FILE_PATH [DTS_NODES_CFG_FILE] The configuration file that describes the SUT and TG nodes. (default: nodes.conf.yaml) + --tests-config-file FILE_PATH + [DTS_TESTS_CFG_FILE] Configuration file used to override variable values inside specific test suites. (default: None) --output-dir DIR_PATH, --output DIR_PATH [DTS_OUTPUT_DIR] Output directory where DTS logs and results are saved. (default: output) -t SECONDS, --timeout SECONDS diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py index 5495cfc5d5..129e6f3222 100644 --- a/dts/framework/config/__init__.py +++ b/dts/framework/config/__init__.py @@ -17,6 +17,7 @@ defining what tests are going to be run and how DPDK will be built. It also references the testbed where these tests and DPDK are going to be run, * A list of the nodes of the testbed which ar represented by :class:`~.node.NodeConfiguration`. + * A dictionary mapping test suite names to their corresponding configurations. The real-time information about testbed is supposed to be gathered at runtime. @@ -27,8 +28,9 @@ and makes it thread safe should we ever want to move in that direction. """ +import os from pathlib import Path -from typing import Annotated, Any, Literal, TypeVar, cast +from typing import TYPE_CHECKING, Annotated, Any, Literal, TypeVar, cast import yaml from pydantic import Field, TypeAdapter, ValidationError, model_validator @@ -38,7 +40,11 @@ from .common import FrozenModel, ValidationContext from .node import NodeConfiguration -from .test_run import TestRunConfiguration +from .test_run import TestRunConfiguration, create_test_suites_config_model + +# Import only if type checking or building docs, to prevent circular imports. +if TYPE_CHECKING or os.environ.get("DTS_DOC_BUILD"): + from framework.test_suite import BaseConfig NodesConfig = Annotated[list[NodeConfiguration], Field(min_length=1)] @@ -50,6 +56,8 @@ class Configuration(FrozenModel): test_run: TestRunConfiguration #: Node configurations. nodes: NodesConfig + #: Test suites custom configurations. + tests_config: dict[str, "BaseConfig"] @model_validator(mode="after") def validate_node_names(self) -> Self: @@ -127,7 +135,7 @@ def validate_test_run_against_nodes(self) -> Self: T = TypeVar("T") -def _load_and_parse_model(file_path: Path, model_type: T, ctx: ValidationContext) -> T: +def _load_and_parse_model(file_path: Path, model_type: type[T], ctx: ValidationContext) -> T: with open(file_path) as f: try: data = yaml.safe_load(f) @@ -154,9 +162,32 @@ def load_config(ctx: ValidationContext) -> Configuration: test_run = _load_and_parse_model( ctx["settings"].test_run_config_path, TestRunConfiguration, ctx ) + + TestSuitesConfiguration = create_test_suites_config_model(test_run.test_suites) + if ctx["settings"].tests_config_path: + tests_config = _load_and_parse_model( + ctx["settings"].tests_config_path, + TestSuitesConfiguration, + ctx, + ) + else: + try: + tests_config = TestSuitesConfiguration() + except ValidationError as e: + raise ConfigurationError( + "A test suites' configuration file is required for the given test run. " + "The following selected test suites require manual configuration: " + + ", ".join(str(error["loc"][0]) for error in e.errors()) + ) + nodes = _load_and_parse_model(ctx["settings"].nodes_config_path, NodesConfig, ctx) try: - return Configuration.model_validate({"test_run": test_run, "nodes": nodes}, context=ctx) + from framework.test_suite import BaseConfig as BaseConfig + + Configuration.model_rebuild() + return Configuration.model_validate( + {"test_run": test_run, "nodes": nodes, "tests_config": dict(tests_config)}, context=ctx + ) except ValidationError as e: raise ConfigurationError("the configurations supplied are invalid") from e diff --git a/dts/framework/config/test_run.py b/dts/framework/config/test_run.py index c1e534e480..688688e88e 100644 --- a/dts/framework/config/test_run.py +++ b/dts/framework/config/test_run.py @@ -18,7 +18,13 @@ from pathlib import Path, PurePath from typing import Annotated, Any, Literal, NamedTuple -from pydantic import Field, field_validator, model_validator +from pydantic import ( + BaseModel, + Field, + create_model, + field_validator, + model_validator, +) from typing_extensions import TYPE_CHECKING, Self from framework.exception import InternalError @@ -27,7 +33,7 @@ from .common import FrozenModel, load_fields_from_settings if TYPE_CHECKING: - from framework.test_suite import TestCase, TestSuite, TestSuiteSpec + from framework.test_suite import BaseConfig, TestCase, TestSuite, TestSuiteSpec @unique @@ -283,6 +289,27 @@ def fetch_all_test_suites() -> list[TestSuiteConfig]: ] +def make_test_suite_config_field(config_obj: type["BaseConfig"]): + """Make a field for a test suite's configuration. + + If the test suite's configuration has required fields, then make the field required. Otherwise + make it optional. + """ + if any(f.is_required() for f in config_obj.model_fields.values()): + return config_obj, Field() + else: + return config_obj, Field(default_factory=config_obj) + + +def create_test_suites_config_model(test_suites: Iterable[TestSuiteConfig]) -> type[BaseModel]: + """Create model for the test suites configuration.""" + test_suites_kwargs = { + t.test_suite_name: make_test_suite_config_field(t.test_suite_spec.config_obj) + for t in test_suites + } + return create_model("TestSuitesConfiguration", **test_suites_kwargs) + + class LinkPortIdentifier(NamedTuple): """A tuple linking test run node type to port name.""" @@ -455,8 +482,8 @@ class TestRunConfiguration(FrozenModel): ) def filter_tests( - self, - ) -> Iterable[tuple[type["TestSuite"], deque[type["TestCase"]]]]: + self, tests_config: dict[str, "BaseConfig"] + ) -> Iterable[tuple[type["TestSuite"], "BaseConfig", deque[type["TestCase"]]]]: """Filter test suites and cases selected for execution.""" from framework.test_suite import TestCaseType @@ -470,6 +497,7 @@ def filter_tests( return ( ( t.test_suite_spec.class_obj, + tests_config[t.test_suite_name], deque( tt for tt in t.test_cases diff --git a/dts/framework/runner.py b/dts/framework/runner.py index a0016d5d57..f822e8a8fc 100644 --- a/dts/framework/runner.py +++ b/dts/framework/runner.py @@ -60,7 +60,12 @@ def run(self) -> None: nodes.append(Node(node_config)) test_run_result = self._result.add_test_run(self._configuration.test_run) - test_run = TestRun(self._configuration.test_run, nodes, test_run_result) + test_run = TestRun( + self._configuration.test_run, + self._configuration.tests_config, + nodes, + test_run_result, + ) test_run.spin() except Exception as e: diff --git a/dts/framework/settings.py b/dts/framework/settings.py index 256afd5cfb..3f21615223 100644 --- a/dts/framework/settings.py +++ b/dts/framework/settings.py @@ -24,6 +24,11 @@ The path to the YAML configuration file of the nodes. +.. option:: --tests-config-file +.. envvar:: DTS_TESTS_CFG_FILE + + The path to the YAML configuration file of the test suites. + .. option:: --output-dir, --output .. envvar:: DTS_OUTPUT_DIR @@ -129,6 +134,8 @@ class Settings: #: nodes_config_path: Path = Path(__file__).parent.parent.joinpath("nodes.yaml") #: + tests_config_path: Path | None = None + #: output_dir: str = "output" #: timeout: float = 15 @@ -342,6 +349,16 @@ def _get_parser() -> _DTSArgumentParser: ) _add_env_var_to_action(action, "NODES_CFG_FILE") + action = parser.add_argument( + "--tests-config-file", + default=SETTINGS.tests_config_path, + type=Path, + help="Configuration file used to override variable values inside specific test suites.", + metavar="FILE_PATH", + dest="tests_config_path", + ) + _add_env_var_to_action(action, "TESTS_CFG_FILE") + action = parser.add_argument( "--output-dir", "--output", diff --git a/dts/framework/test_run.py b/dts/framework/test_run.py index 2808d013f6..3f2eea693f 100644 --- a/dts/framework/test_run.py +++ b/dts/framework/test_run.py @@ -118,7 +118,7 @@ from framework.remote_session.dpdk import DPDKBuildEnvironment, DPDKRuntimeEnvironment from framework.settings import SETTINGS from framework.test_result import BaseResult, Result, TestCaseResult, TestRunResult, TestSuiteResult -from framework.test_suite import TestCase, TestSuite +from framework.test_suite import BaseConfig, TestCase, TestSuite from framework.testbed_model.capability import ( Capability, get_supported_capabilities, @@ -128,7 +128,7 @@ from framework.testbed_model.topology import PortLink, Topology from framework.testbed_model.traffic_generator import create_traffic_generator -TestScenario = tuple[type[TestSuite], deque[type[TestCase]]] +TestScenario = tuple[type[TestSuite], BaseConfig, deque[type[TestCase]]] class TestRun: @@ -176,11 +176,18 @@ class TestRun: remaining_test_cases: deque[type[TestCase]] supported_capabilities: set[Capability] - def __init__(self, config: TestRunConfiguration, nodes: Iterable[Node], result: TestRunResult): + def __init__( + self, + config: TestRunConfiguration, + tests_config: dict[str, BaseConfig], + nodes: Iterable[Node], + result: TestRunResult, + ): """Test run constructor. Args: config: The test run's own configuration. + tests_config: The test run's test suites configurations. nodes: A reference to all the available nodes. result: A reference to the test run result object. """ @@ -201,7 +208,7 @@ def __init__(self, config: TestRunConfiguration, nodes: Iterable[Node], result: self.ctx = Context(sut_node, tg_node, topology, dpdk_runtime_env, traffic_generator) self.result = result - self.selected_tests = list(self.config.filter_tests()) + self.selected_tests = list(self.config.filter_tests(tests_config)) self.blocked = False self.remaining_tests = deque() self.remaining_test_cases = deque() @@ -214,7 +221,7 @@ def required_capabilities(self) -> set[Capability]: """The capabilities required to run this test run in its totality.""" caps = set() - for test_suite, test_cases in self.selected_tests: + for test_suite, _, test_cases in self.selected_tests: caps.update(test_suite.required_capabilities) for test_case in test_cases: caps.update(test_case.required_capabilities) @@ -371,8 +378,10 @@ def next(self) -> State | None: """Next state.""" test_run = self.test_run try: - test_suite_class, test_run.remaining_test_cases = test_run.remaining_tests.popleft() - test_suite = test_suite_class() + test_suite_class, test_config, test_run.remaining_test_cases = ( + test_run.remaining_tests.popleft() + ) + test_suite = test_suite_class(test_config) test_suite_result = test_run.result.add_test_suite(test_suite.name) if test_run.blocked: diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py index 58da26adf0..e07c327b77 100644 --- a/dts/framework/test_suite.py +++ b/dts/framework/test_suite.py @@ -31,6 +31,7 @@ from scapy.packet import Packet, Padding, raw from typing_extensions import Self +from framework.config.common import FrozenModel from framework.testbed_model.capability import TestProtocol from framework.testbed_model.topology import Topology from framework.testbed_model.traffic_generator.capturing_traffic_generator import ( @@ -46,6 +47,10 @@ from framework.context import Context +class BaseConfig(FrozenModel): + """Base for a custom test suite configuration.""" + + class TestSuite(TestProtocol): """The base class with building blocks needed by most test cases. @@ -70,8 +75,13 @@ class TestSuite(TestProtocol): The test suite is aware of the testbed (the SUT and TG) it's running on. From this, it can properly choose the IP addresses and other configuration that must be tailored to the testbed. + + Attributes: + config: The test suite configuration. """ + config: BaseConfig + #: Whether the test suite is blocking. A failure of a blocking test suite #: will block the execution of all subsequent test suites in the current test run. is_blocking: ClassVar[bool] = False @@ -82,19 +92,15 @@ class TestSuite(TestProtocol): _tg_ip_address_ingress: Union[IPv4Interface, IPv6Interface] _tg_ip_address_egress: Union[IPv4Interface, IPv6Interface] - def __init__(self): + def __init__(self, config: BaseConfig): """Initialize the test suite testbed information and basic configuration. - Find links between ports and set up default IP addresses to be used when - configuring them. - Args: - sut_node: The SUT node where the test suite will run. - tg_node: The TG node where the test suite will run. - topology: The topology where the test suite will run. + config: The test suite configuration. """ from framework.context import get_ctx + self.config = config self._ctx = get_ctx() self._logger = get_dts_logger(self.__class__.__name__) self._sut_ip_address_ingress = ip_interface("192.168.100.2/24") @@ -678,6 +684,11 @@ def is_test_suite(obj) -> bool: f"Expected class {self.class_name} not found in module {self.module_name}." ) + @cached_property + def config_obj(self) -> type[BaseConfig]: + """A reference to the test suite's configuration class.""" + return self.class_obj.__annotations__.get("config", BaseConfig) + @classmethod def discover_all( cls, package_name: str | None = None, module_prefix: str | None = None diff --git a/dts/tests/TestSuite_hello_world.py b/dts/tests/TestSuite_hello_world.py index 141f2bc4c9..6c9ecc1177 100644 --- a/dts/tests/TestSuite_hello_world.py +++ b/dts/tests/TestSuite_hello_world.py @@ -1,5 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2024 University of New Hampshire +# Copyright(c) 2025 Arm Limited """DPDK Hello World test suite. @@ -8,12 +9,21 @@ """ from framework.remote_session.testpmd_shell import TestPmdShell -from framework.test_suite import TestSuite, func_test +from framework.test_suite import BaseConfig, TestSuite, func_test + + +class Config(BaseConfig): + """Example custom configuration.""" + + #: The hello world message to print. + msg: str = "Hello World!" class TestHelloWorld(TestSuite): """Hello World test suite. One test case, which starts and stops a testpmd session.""" + config: Config + @func_test def test_hello_world(self) -> None: """EAL confidence test. @@ -25,4 +35,4 @@ def test_hello_world(self) -> None: """ with TestPmdShell() as testpmd: testpmd.start() - self.log("Hello World!") + self.log(self.config.msg) diff --git a/dts/tests_config.example.yaml b/dts/tests_config.example.yaml index e69de29bb2..72af9da260 100644 --- a/dts/tests_config.example.yaml +++ b/dts/tests_config.example.yaml @@ -0,0 +1,2 @@ +hello_world: + msg: A custom hello world to you! \ No newline at end of file From patchwork Mon Mar 3 14:57:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 152201 X-Patchwork-Delegate: paul.szczepanek@arm.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7B73C46331; Mon, 3 Mar 2025 15:58:08 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4A62540657; Mon, 3 Mar 2025 15:57:29 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 4D7CC40613 for ; Mon, 3 Mar 2025 15:57:26 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D730D175A; Mon, 3 Mar 2025 06:57:39 -0800 (PST) Received: from localhost.localdomain (unknown [10.57.64.81]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 13A423F66E; Mon, 3 Mar 2025 06:57:24 -0800 (PST) From: Luca Vizzarro To: dev@dpdk.org Cc: Luca Vizzarro , Paul Szczepanek , Patrick Robb Subject: [PATCH v4 7/7] dts: improve configuration errors Date: Mon, 3 Mar 2025 16:57:09 +0200 Message-ID: <20250303145709.126126-8-Luca.Vizzarro@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250303145709.126126-1-Luca.Vizzarro@arm.com> References: <20241108134532.130681-1-luca.vizzarro@arm.com> <20250303145709.126126-1-Luca.Vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Luca Vizzarro Improve the way that configuration errors are displayed to the user. Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Dean Marx --- dts/framework/config/__init__.py | 4 ++-- dts/framework/config/test_run.py | 2 +- dts/framework/runner.py | 14 +++++++++++++- 3 files changed, 16 insertions(+), 4 deletions(-) diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py index 129e6f3222..1ec744d1d4 100644 --- a/dts/framework/config/__init__.py +++ b/dts/framework/config/__init__.py @@ -141,7 +141,7 @@ def _load_and_parse_model(file_path: Path, model_type: type[T], ctx: ValidationC data = yaml.safe_load(f) return TypeAdapter(model_type).validate_python(data, context=cast(dict[str, Any], ctx)) except ValidationError as e: - msg = f"failed to load the configuration file {file_path}" + msg = f"Failed to load the configuration file {file_path}." raise ConfigurationError(msg) from e @@ -190,4 +190,4 @@ def load_config(ctx: ValidationContext) -> Configuration: {"test_run": test_run, "nodes": nodes, "tests_config": dict(tests_config)}, context=ctx ) except ValidationError as e: - raise ConfigurationError("the configurations supplied are invalid") from e + raise ConfigurationError("The configurations supplied are invalid.") from e diff --git a/dts/framework/config/test_run.py b/dts/framework/config/test_run.py index 688688e88e..06fe28143c 100644 --- a/dts/framework/config/test_run.py +++ b/dts/framework/config/test_run.py @@ -445,7 +445,7 @@ def use_first_core(self) -> bool: class DPDKConfiguration(DPDKRuntimeConfiguration): """The DPDK configuration needed to test.""" - #: The DPDKD build configuration used to test. + #: The DPDK build configuration used to test. build: DPDKBuildConfiguration diff --git a/dts/framework/runner.py b/dts/framework/runner.py index f822e8a8fc..f20aa3576a 100644 --- a/dts/framework/runner.py +++ b/dts/framework/runner.py @@ -11,8 +11,10 @@ import os import sys +import textwrap from framework.config.common import ValidationContext +from framework.exception import ConfigurationError from framework.test_run import TestRun from framework.testbed_model.node import Node @@ -37,7 +39,17 @@ class DTSRunner: def __init__(self): """Initialize the instance with configuration, logger, result and string constants.""" - self._configuration = load_config(ValidationContext(settings=SETTINGS)) + try: + self._configuration = load_config(ValidationContext(settings=SETTINGS)) + except ConfigurationError as e: + if e.__cause__: + print(f"{e} Reason:", file=sys.stderr) + print(file=sys.stderr) + print(textwrap.indent(str(e.__cause__), prefix=" " * 2), file=sys.stderr) + else: + print(e, file=sys.stderr) + sys.exit(e.severity) + self._logger = get_dts_logger() if not os.path.exists(SETTINGS.output_dir): os.makedirs(SETTINGS.output_dir)