From patchwork Wed Nov 15 13:09:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Juraj_Linke=C5=A1?= X-Patchwork-Id: 134398 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 55CD643339; Wed, 15 Nov 2023 14:14:35 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 880CE427E5; Wed, 15 Nov 2023 14:12:25 +0100 (CET) Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by mails.dpdk.org (Postfix) with ESMTP id 1669940ED0 for ; Wed, 15 Nov 2023 14:12:19 +0100 (CET) Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-9e5dd91b0acso827357666b.1 for ; Wed, 15 Nov 2023 05:12:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1700053939; x=1700658739; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tm6Opm89TVep6rTOjKIkP+0MrI/vnwJrCVP9TMX0aHA=; b=rC80orYDDbX7yCSq3c7mj5oE7kH3jSAsb/jVoWWFaNaD3cJD49B3Uc37tUhhnqZ1my zgqtneiT3VWiHmAhvDzjnTVrlVtxSn76vG+39Xr9HlZTbjl99yZ/YJ/c6hjXvGxdGRpU TblVXQltZjSGw0vBocqaa4xZjH3eJdus02e+hALhH6+ip0865Y+vqvOu86lviJklxRKi 69hMOV4kRWZIfuxz96N/Hu2E0tj95zJ7TiMZV3jcg5Qe14fuf8tu+UNTWIhykA1kHDgm e8M7GnbuF8sOhkMPBu4EKvKTkzv8vYfENO0TtiGYVIZW7HBD96fLNV/FKtOT0rZHAtI8 QLDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700053939; x=1700658739; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tm6Opm89TVep6rTOjKIkP+0MrI/vnwJrCVP9TMX0aHA=; b=BJNIeapPvDd49pw9oCkuIym+IYGAfx4mSbaUNT5eifCBKsAP8viANGFbz9czXi4NNX TkMkNDU/kQfWfP6AqDbf1twD9YEEvVl5Ju4ahhBnTOUNEZ5iFjpkH9SloLV2UBDKb2hA d/eVb32yF+xejfh2F1s3WxaYtrV4c4TgRVIYuI9d1NP2tf71UN1SLflzfdnbIqobU8oT XbPASNEs/NHFyTrk5s65dNqvoTvIFZVXVlwMN0i5MTYIW3vf5hqMtItrurqFZa0znqz+ UCbR2mCCOp1ce/WwhoW59ENunhWvA4Dg/Mr/qLcuuNAJedp3fKtVuivVoHrUAWs6HOBl hNOg== X-Gm-Message-State: AOJu0YwWzqDuunXWIINvSiEYb07/ib4HkCuyisj+7OM41oMcrRGnT4ep Vo5VZ9htBW7vLIsnuinMN7M5oA== X-Google-Smtp-Source: AGHT+IFszhz9Zg+FIkzpb7rELnYe780Tpe7pKLRhbAQYks5NyXPzT9ns3CtSY4+Arc6HX6x/hxx/RA== X-Received: by 2002:a17:906:708f:b0:9e5:2ab3:b74e with SMTP id b15-20020a170906708f00b009e52ab3b74emr9818130ejk.75.1700053938570; Wed, 15 Nov 2023 05:12:18 -0800 (PST) Received: from jlinkes-PT-Latitude-5530.pantheon.local (81.89.53.154.host.vnet.sk. [81.89.53.154]) by smtp.gmail.com with ESMTPSA id tb16-20020a1709078b9000b009f2b7282387sm1011914ejc.46.2023.11.15.05.12.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 05:12:18 -0800 (PST) From: =?utf-8?q?Juraj_Linke=C5=A1?= To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, jspewock@iol.unh.edu, probb@iol.unh.edu, paul.szczepanek@arm.com, yoan.picchi@foss.arm.com Cc: dev@dpdk.org, =?utf-8?q?Juraj_Linke=C5=A1?= Subject: [PATCH v7 21/21] dts: test suites docstring update Date: Wed, 15 Nov 2023 14:09:59 +0100 Message-Id: <20231115130959.39420-22-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231115130959.39420-1-juraj.linkes@pantheon.tech> References: <20231108125324.191005-23-juraj.linkes@pantheon.tech> <20231115130959.39420-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Format according to the Google format and PEP257, with slight deviations. Signed-off-by: Juraj Linkeš --- dts/tests/TestSuite_hello_world.py | 16 +++++---- dts/tests/TestSuite_os_udp.py | 19 +++++++---- dts/tests/TestSuite_smoke_tests.py | 53 +++++++++++++++++++++++++++--- 3 files changed, 70 insertions(+), 18 deletions(-) diff --git a/dts/tests/TestSuite_hello_world.py b/dts/tests/TestSuite_hello_world.py index 7e3d95c0cf..662a8f8726 100644 --- a/dts/tests/TestSuite_hello_world.py +++ b/dts/tests/TestSuite_hello_world.py @@ -1,7 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014 Intel Corporation -""" +"""The DPDK hello world app test suite. + Run the helloworld example app and verify it prints a message for each used core. No other EAL parameters apart from cores are used. """ @@ -15,22 +16,25 @@ class TestHelloWorld(TestSuite): + """DPDK hello world app test suite.""" + def set_up_suite(self) -> None: - """ + """Set up the test suite. + Setup: Build the app we're about to test - helloworld. """ self.app_helloworld_path = self.sut_node.build_dpdk_app("helloworld") def test_hello_world_single_core(self) -> None: - """ + """Single core test case. + Steps: Run the helloworld app on the first usable logical core. Verify: The app prints a message from the used core: "hello from core " """ - # get the first usable core lcore_amount = LogicalCoreCount(1, 1, 1) lcores = LogicalCoreCountFilter(self.sut_node.lcores, lcore_amount).filter() @@ -44,14 +48,14 @@ def test_hello_world_single_core(self) -> None: ) def test_hello_world_all_cores(self) -> None: - """ + """All cores test case. + Steps: Run the helloworld app on all usable logical cores. Verify: The app prints a message from all used cores: "hello from core " """ - # get the maximum logical core number eal_para = self.sut_node.create_eal_parameters( lcore_filter_specifier=LogicalCoreList(self.sut_node.lcores) diff --git a/dts/tests/TestSuite_os_udp.py b/dts/tests/TestSuite_os_udp.py index bf6b93deb5..e0c5239612 100644 --- a/dts/tests/TestSuite_os_udp.py +++ b/dts/tests/TestSuite_os_udp.py @@ -1,7 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 PANTHEON.tech s.r.o. -""" +"""Basic IPv4 OS routing test suite. + Configure SUT node to route traffic from if1 to if2. Send a packet to the SUT node, verify it comes back on the second port on the TG node. """ @@ -13,24 +14,27 @@ class TestOSUdp(TestSuite): + """IPv4 UDP OS routing test suite.""" + def set_up_suite(self) -> None: - """ + """Set up the test suite. + Setup: - Configure SUT ports and SUT to route traffic from if1 to if2. + Bind the SUT ports to the OS driver, configure the ports and configure the SUT + to route traffic from if1 to if2. """ - # This test uses kernel drivers self.sut_node.bind_ports_to_driver(for_dpdk=False) self.configure_testbed_ipv4() def test_os_udp(self) -> None: - """ + """Basic UDP IPv4 traffic test case. + Steps: Send a UDP packet. Verify: The packet with proper addresses arrives at the other TG port. """ - packet = Ether() / IP() / UDP() received_packets = self.send_packet_and_capture(packet) @@ -40,7 +44,8 @@ def test_os_udp(self) -> None: self.verify_packets(expected_packet, received_packets) def tear_down_suite(self) -> None: - """ + """Tear down the test suite. + Teardown: Remove the SUT port configuration configured in setup. """ diff --git a/dts/tests/TestSuite_smoke_tests.py b/dts/tests/TestSuite_smoke_tests.py index e8016d1b54..6fae099a0e 100644 --- a/dts/tests/TestSuite_smoke_tests.py +++ b/dts/tests/TestSuite_smoke_tests.py @@ -1,6 +1,17 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 University of New Hampshire +"""Smoke test suite. + +Smoke tests are a class of tests which are used for validating a minimal set of important features. +These are the most important features without which (or when they're faulty) the software wouldn't +work properly. Thus, if any failure occurs while testing these features, +there isn't that much of a reason to continue testing, as the software is fundamentally broken. + +These tests don't have to include only DPDK tests, as the reason for failures could be +in the infrastructure (a faulty link between NICs or a misconfiguration). +""" + import re from framework.config import PortConfig @@ -11,13 +22,25 @@ class SmokeTests(TestSuite): + """DPDK and infrastructure smoke test suite. + + The test cases validate the most basic DPDK functionality needed for all other test suites. + The infrastructure also needs to be tested, as that is also used by all other test suites. + + Attributes: + is_blocking: This test suite will block the execution of all other test suites + in the build target after it. + nics_in_node: The NICs present on the SUT node. + """ + is_blocking = True # dicts in this list are expected to have two keys: # "pci_address" and "current_driver" nics_in_node: list[PortConfig] = [] def set_up_suite(self) -> None: - """ + """Set up the test suite. + Setup: Set the build directory path and generate a list of NICs in the SUT node. """ @@ -25,7 +48,13 @@ def set_up_suite(self) -> None: self.nics_in_node = self.sut_node.config.ports def test_unit_tests(self) -> None: - """ + """DPDK meson fast-tests unit tests. + + The DPDK unit tests are basic tests that indicate regressions and other critical failures. + These need to be addressed before other testing. + + The fast-tests unit tests are a subset with only the most basic tests. + Test: Run the fast-test unit-test suite through meson. """ @@ -37,7 +66,14 @@ def test_unit_tests(self) -> None: ) def test_driver_tests(self) -> None: - """ + """DPDK meson driver-tests unit tests. + + The DPDK unit tests are basic tests that indicate regressions and other critical failures. + These need to be addressed before other testing. + + The driver-tests unit tests are a subset that test only drivers. These may be run + with virtual devices as well. + Test: Run the driver-test unit-test suite through meson. """ @@ -63,7 +99,10 @@ def test_driver_tests(self) -> None: ) def test_devices_listed_in_testpmd(self) -> None: - """ + """Testpmd device discovery. + + If the configured devices can't be found in testpmd, they can't be tested. + Test: Uses testpmd driver to verify that devices have been found by testpmd. """ @@ -79,7 +118,11 @@ def test_devices_listed_in_testpmd(self) -> None: ) def test_device_bound_to_driver(self) -> None: - """ + """Device driver in OS. + + The devices must be bound to the proper driver, otherwise they can't be used by DPDK + or the traffic generators. + Test: Ensure that all drivers listed in the config are bound to the correct driver.