From patchwork Fri Jun 2 15:13:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 127970 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3572642C11; Fri, 2 Jun 2023 17:14:19 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4C02442B8B; Fri, 2 Jun 2023 17:14:18 +0200 (CEST) Received: from mail-pj1-f98.google.com (mail-pj1-f98.google.com [209.85.216.98]) by mails.dpdk.org (Postfix) with ESMTP id 91CBF4161A for ; Fri, 2 Jun 2023 17:14:16 +0200 (CEST) Received: by mail-pj1-f98.google.com with SMTP id 98e67ed59e1d1-2563ca70f64so1143726a91.0 for ; Fri, 02 Jun 2023 08:14:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1685718856; x=1688310856; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yjvPZKeuHJ1oIh2tiZihxINItV2CKfQlN4pVXYJZbqM=; b=iJ9k8DQWchcx9O9IjjXiXKS+QeVYLJVLEpkAWDFcW8n89YQyQJHxjGJDPYrHzS63s8 I6hRcwLMWL0fJTTTcS4TdE+lW+JnttWJD0J/Fn/gyTjdsNo1m/5L55FWpocOLq7SQy0V 6RFGPFqR2JJQ+pXglUptUAXGWRCtvFPgQuzyw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685718856; x=1688310856; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yjvPZKeuHJ1oIh2tiZihxINItV2CKfQlN4pVXYJZbqM=; b=WgHtgR2ipowPuPkTTAlvi1WujlHrz4n1cNw6+w7qLmKKt8STRbzWJRGpsbdTo2Pbti 3tl8si91bMaUM/m16QkBqw/LRUX34imB7vErOZJObVKiQHQKtXKV9BL3kXrDEqgiGG+L zjOnJjlW1Eflx6VMlbUACf/pS3W8g1BV4wUNVMv3Nk24j5gy+hly/Mv7zG4l/mca3Sft aRBIFkEvM+dDhLsSAvUrGfdkhIXmwA88eeOPe93dRPCXOL6S1zzD105aDMowxocvLmOm AeyB+X4HbnUsY56olb8+b5QPPGa19QCfu7A/4Bt4TBmYdufkukoO9qDJnXvVY420cI3U dXPA== X-Gm-Message-State: AC+VfDyUSnQDxw4IoZ9GPWd9+DmZoycOcixBStMzwnlJoc/+hO05CafO SvECIdXmIeNbZ6VE0X+qg1N1a+TrYbLneBE7LyTB8tu0630QPZWDdEzs2RHDX6//Hg== X-Google-Smtp-Source: ACHHUZ5GYuf23zWxpKga0WVNQQaK2BpoRRWBvmLD3lz2wyzpfharhV13yZhAhuhES/MCDYCfntttTAVo2asn X-Received: by 2002:a17:90b:685:b0:255:83b6:2d0b with SMTP id m5-20020a17090b068500b0025583b62d0bmr249897pjz.17.1685718855709; Fri, 02 Jun 2023 08:14:15 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id pc10-20020a17090b3b8a00b002497d05e7b5sm844570pjb.7.2023.06.02.08.14.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Jun 2023 08:14:15 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1257::1083]) by postal.iol.unh.edu (Postfix) with ESMTP id 145DF605246B; Fri, 2 Jun 2023 11:14:13 -0400 (EDT) From: jspewock@iol.unh.edu To: Honnappa.Nagarahalli@arm.com, juraj.linkes@pantheon.tech, thomas@monjalon.net, lijuan.tu@intel.com, wathsala.vithanage@arm.com Cc: dev@dpdk.org, Jeremy Spewock Subject: [RFC v3 1/2] dts: add smoke tests Date: Fri, 2 Jun 2023 11:13:31 -0400 Message-Id: <20230602151332.6404-3-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230602151332.6404-2-jspewock@iol.unh.edu> References: <20230602151332.6404-2-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock Adds a new test suite for running smoke tests that verify general configuration aspects of the system under test. If any of these tests fail, the DTS execution terminates for that build target as part of a "fail-fast" model. Signed-off-by: Jeremy Spewock --- dts/conf.yaml | 8 ++ dts/framework/config/__init__.py | 88 ++++++++++++++ dts/framework/config/conf_yaml_schema.json | 32 ++++- dts/framework/dts.py | 26 +++- dts/framework/exception.py | 12 ++ dts/framework/remote_session/__init__.py | 10 +- dts/framework/remote_session/os_session.py | 34 +++++- dts/framework/remote_session/posix_session.py | 30 +++++ .../remote_session/remote/__init__.py | 12 ++ .../remote/interactive_remote_session.py | 113 ++++++++++++++++++ .../remote/interactive_shell.py | 98 +++++++++++++++ .../remote_session/remote/testpmd_shell.py | 58 +++++++++ dts/framework/test_result.py | 38 +++++- dts/framework/test_suite.py | 31 ++++- dts/framework/testbed_model/node.py | 2 + dts/framework/testbed_model/sut_node.py | 110 ++++++++++++++++- dts/tests/TestSuite_smoke_tests.py | 109 +++++++++++++++++ 17 files changed, 792 insertions(+), 19 deletions(-) create mode 100644 dts/framework/remote_session/remote/interactive_remote_session.py create mode 100644 dts/framework/remote_session/remote/interactive_shell.py create mode 100644 dts/framework/remote_session/remote/testpmd_shell.py create mode 100644 dts/tests/TestSuite_smoke_tests.py diff --git a/dts/conf.yaml b/dts/conf.yaml index a9bd8a3e..de537c06 100644 --- a/dts/conf.yaml +++ b/dts/conf.yaml @@ -10,7 +10,15 @@ executions: compiler_wrapper: ccache perf: false func: true + nics: #physical devices to be used for testing + - addresses: + - "0000:11:00.0" + - "0000:11:00.1" + driver: "i40e" + vdevs: #names of virtual devices to be used for testing + - "crypto_openssl" test_suites: + - smoke_tests - hello_world system_under_test: "SUT 1" nodes: diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py index ebb0823f..a5840429 100644 --- a/dts/framework/config/__init__.py +++ b/dts/framework/config/__init__.py @@ -107,6 +107,46 @@ def from_dict(d: dict) -> "NodeConfiguration": ) +@dataclass(slots=True) +class NodeVersionInfo: + """Class to hold important versions within the node. + + This class, unlike the NodeConfiguration class, cannot be generated at the start. + This is because we need to initialize a connection with the node before we can + collet the information needed in this class. Therefore, it cannot be a part of + the configuration class above. + """ + + os_name: str + os_version: str + kernel_version: str + + @staticmethod + def from_dict(d: dict): + return NodeVersionInfo( + os_name=d["os_name"], + os_version=d["os_version"], + kernel_version=d["kernel_version"], + ) + + +@dataclass(slots=True, frozen=True) +class NICConfiguration: + addresses: list[str] + driver: str + + @staticmethod + def from_dict(d: dict) -> "NICConfiguration": + return NICConfiguration( + addresses=[addr for addr in d.get("addresses", [])], + driver=d.get("driver", ""), + ) + + @staticmethod + def from_list(nics: list[dict]) -> list["NICConfiguration"]: + return [NICConfiguration.from_dict(x) for x in nics] + + @dataclass(slots=True, frozen=True) class BuildTargetConfiguration: arch: Architecture @@ -128,6 +168,24 @@ def from_dict(d: dict) -> "BuildTargetConfiguration": ) +@dataclass(slots=True) +class BuildTargetVersionInfo: + """Class to hold important versions within the build target. + + This is very similar to the NodeVersionInfo class, it just instead holds information + for the build target. + """ + + dpdk_version: str + compiler_version: str + + @staticmethod + def from_dict(d: dict): + return BuildTargetVersionInfo( + dpdk_version=d["dpdk_version"], compiler_version=d["compiler_version"] + ) + + class TestSuiteConfigDict(TypedDict): suite: str cases: list[str] @@ -157,6 +215,8 @@ class ExecutionConfiguration: func: bool test_suites: list[TestSuiteConfig] system_under_test: NodeConfiguration + nics: list[NICConfiguration] + vdevs: list[str] @staticmethod def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration": @@ -166,7 +226,9 @@ def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration": test_suites: list[TestSuiteConfig] = list( map(TestSuiteConfig.from_dict, d["test_suites"]) ) + nic_conf = NICConfiguration.from_list(d["nics"]) sut_name = d["system_under_test"] + list_of_vdevs = d["vdevs"] assert sut_name in node_map, f"Unknown SUT {sut_name} in execution {d}" return ExecutionConfiguration( @@ -174,7 +236,9 @@ def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration": perf=d["perf"], func=d["func"], test_suites=test_suites, + nics=nic_conf, system_under_test=node_map[sut_name], + vdevs=list_of_vdevs, ) @@ -221,3 +285,27 @@ def load_config() -> Configuration: CONFIGURATION = load_config() + + +@unique +class InteractiveApp(Enum): + """An enum that represents different supported interactive applications + + The values in this enum must all be set to objects that have a key called + "default_path" where "default_path" represents an array for the path to the + application. This default path will be passed into the handler class for the + application so that it can start the application. For every key other than + the default shell option, the path will be appended to the path to the DPDK + build directory for the current SUT node. + """ + + shell = {"default_path": [""]} + testpmd = {"default_path": ["app", "dpdk-testpmd"]} + + def get_path(self): + """A method for getting the default paths of an application + + Returns: + String array that represents an OS agnostic path to the application. + """ + return self.value["default_path"] diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json index ca2d4a1e..603859de 100644 --- a/dts/framework/config/conf_yaml_schema.json +++ b/dts/framework/config/conf_yaml_schema.json @@ -40,6 +40,18 @@ "mscv" ] }, + "single_nic" : { + "type":"object", + "description": "an object that holds nic information", + "properties": { + "addresses": { + "type":"array", + "items": { + "type":"string" + } + } + } + }, "build_target": { "type": "object", "description": "Targets supported by DTS", @@ -97,7 +109,8 @@ "test_suite": { "type": "string", "enum": [ - "hello_world" + "hello_world", + "smoke_tests" ] }, "test_target": { @@ -211,6 +224,23 @@ ] } }, + "nics": { + "type": "array", + "items": { + "oneOf": [ + { + "$ref": "#/definitions/single_nic" + } + ] + } + }, + "vdevs": { + "description": "Names of vdevs to be used in execution", + "type": "array", + "items": { + "type":"string" + } + }, "system_under_test": { "$ref": "#/definitions/node_name" } diff --git a/dts/framework/dts.py b/dts/framework/dts.py index 05022845..7f238963 100644 --- a/dts/framework/dts.py +++ b/dts/framework/dts.py @@ -6,6 +6,7 @@ import sys from .config import CONFIGURATION, BuildTargetConfiguration, ExecutionConfiguration +from .exception import BlockingTestSuiteError from .logger import DTSLOG, getLogger from .test_result import BuildTargetResult, DTSResult, ExecutionResult, Result from .test_suite import get_test_suites @@ -82,7 +83,9 @@ def _run_execution( running all build targets in the given execution. """ dts_logger.info(f"Running execution with SUT '{execution.system_under_test.name}'.") - execution_result = result.add_execution(sut_node.config) + execution_result = result.add_execution( + sut_node.config, sut_node.get_node_versions() + ) try: sut_node.set_up_execution(execution) @@ -118,7 +121,10 @@ def _run_build_target( try: sut_node.set_up_build_target(build_target) - result.dpdk_version = sut_node.dpdk_version + # result.dpdk_version = sut_node.dpdk_version + build_target_result.add_build_target_versions( + sut_node.get_build_target_versions() + ) build_target_result.update_setup(Result.PASS) except Exception as e: dts_logger.exception("Build target setup failed.") @@ -146,6 +152,7 @@ def _run_suites( with possibly only a subset of test cases. If no subset is specified, run all test cases. """ + end_build_target = False for test_suite_config in execution.test_suites: try: full_suite_path = f"tests.TestSuite_{test_suite_config.test_suite}" @@ -165,8 +172,21 @@ def _run_suites( test_suite_config.test_cases, execution.func, build_target_result, + sut_node._build_target_config, + result, ) - test_suite.run() + try: + test_suite.run() + except BlockingTestSuiteError as e: + dts_logger.exception( + "An error occurred within a blocking TestSuite, " + "execution will now end." + ) + result.add_error(e) + end_build_target = True + # if a blocking test failed and we need to bail out of suite executions + if end_build_target: + break def _exit_dts() -> None: diff --git a/dts/framework/exception.py b/dts/framework/exception.py index ca353d98..dfb12df4 100644 --- a/dts/framework/exception.py +++ b/dts/framework/exception.py @@ -25,6 +25,7 @@ class ErrorSeverity(IntEnum): SSH_ERR = 4 DPDK_BUILD_ERR = 10 TESTCASE_VERIFY_ERR = 20 + BLOCKING_TESTSUITE_ERR = 25 class DTSError(Exception): @@ -144,3 +145,14 @@ def __init__(self, value: str): def __str__(self) -> str: return repr(self.value) + + +class BlockingTestSuiteError(DTSError): + suite_name: str + severity: ClassVar[ErrorSeverity] = ErrorSeverity.BLOCKING_TESTSUITE_ERR + + def __init__(self, suite_name: str) -> None: + self.suite_name = suite_name + + def __str__(self) -> str: + return f"Blocking suite {self.suite_name} failed." diff --git a/dts/framework/remote_session/__init__.py b/dts/framework/remote_session/__init__.py index ee221503..04342b8d 100644 --- a/dts/framework/remote_session/__init__.py +++ b/dts/framework/remote_session/__init__.py @@ -1,5 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 PANTHEON.tech s.r.o. +# Copyright(c) 2022-2023 University of New Hampshire """ The package provides modules for managing remote connections to a remote host (node), @@ -17,7 +18,14 @@ from .linux_session import LinuxSession from .os_session import OSSession -from .remote import CommandResult, RemoteSession, SSHSession +from .remote import ( + CommandResult, + InteractiveRemoteSession, + InteractiveShell, + RemoteSession, + SSHSession, + TestPmdShell, +) def create_session( diff --git a/dts/framework/remote_session/os_session.py b/dts/framework/remote_session/os_session.py index 4c48ae25..f5f53923 100644 --- a/dts/framework/remote_session/os_session.py +++ b/dts/framework/remote_session/os_session.py @@ -12,7 +12,13 @@ from framework.testbed_model import LogicalCore from framework.utils import EnvVarsDict, MesonArgs -from .remote import CommandResult, RemoteSession, create_remote_session +from .remote import ( + CommandResult, + InteractiveRemoteSession, + RemoteSession, + create_interactive_session, + create_remote_session, +) class OSSession(ABC): @@ -26,6 +32,7 @@ class OSSession(ABC): name: str _logger: DTSLOG remote_session: RemoteSession + interactive_session: InteractiveRemoteSession def __init__( self, @@ -37,6 +44,7 @@ def __init__( self.name = name self._logger = logger self.remote_session = create_remote_session(node_config, name, logger) + self.interactive_session = create_interactive_session(node_config, name, logger) def close(self, force: bool = False) -> None: """ @@ -173,3 +181,27 @@ def setup_hugepages(self, hugepage_amount: int, force_first_numa: bool) -> None: if needed and mount the hugepages if needed. If force_first_numa is True, configure hugepages just on the first socket. """ + + @abstractmethod + def get_compiler_version(self, compiler_name: str) -> str: + """ + Get installed version of compiler used for DPDK + """ + + @abstractmethod + def get_os_name(self) -> str: + """ + Get name of OS for the node + """ + + @abstractmethod + def get_os_version(self) -> str: + """ + Get version of OS for the node + """ + + @abstractmethod + def get_kernel_version(self) -> str: + """ + Get kernel version for the node + """ diff --git a/dts/framework/remote_session/posix_session.py b/dts/framework/remote_session/posix_session.py index d38062e8..dc6689f5 100644 --- a/dts/framework/remote_session/posix_session.py +++ b/dts/framework/remote_session/posix_session.py @@ -219,3 +219,33 @@ def _remove_dpdk_runtime_dirs( def get_dpdk_file_prefix(self, dpdk_prefix) -> str: return "" + + def get_compiler_version(self, compiler_name: str) -> str: + match compiler_name: + case "gcc": + return self.send_command(f"{compiler_name} --version", 60).stdout.split( + "\n" + )[0] + case "clang": + return self.send_command(f"{compiler_name} --version", 60).stdout.split( + "\n" + )[0] + case "msvc": + return self.send_command("cl", 60).stdout + case "icc": + return self.send_command(f"{compiler_name} -V", 60).stdout + case _: + raise ValueError(f"Unknown compiler {compiler_name}") + + def get_os_name(self) -> str: + return self.send_command( + "awk -F= '$1==\"NAME\" {print $2}' /etc/os-release", 60 + ).stdout + + def get_os_version(self) -> str: + return self.send_command( + "awk -F= '$1==\"VERSION\" {print $2}' /etc/os-release", 60, True + ).stdout + + def get_kernel_version(self) -> str: + return self.send_command("uname -r", 60).stdout diff --git a/dts/framework/remote_session/remote/__init__.py b/dts/framework/remote_session/remote/__init__.py index 8a151221..ef7b4851 100644 --- a/dts/framework/remote_session/remote/__init__.py +++ b/dts/framework/remote_session/remote/__init__.py @@ -1,16 +1,28 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 PANTHEON.tech s.r.o. +# Copyright(c) 2022-2023 University of New Hampshire # pylama:ignore=W0611 +from paramiko import AutoAddPolicy, SSHClient + from framework.config import NodeConfiguration from framework.logger import DTSLOG +from .interactive_remote_session import InteractiveRemoteSession +from .interactive_shell import InteractiveShell from .remote_session import CommandResult, RemoteSession from .ssh_session import SSHSession +from .testpmd_shell import TestPmdShell def create_remote_session( node_config: NodeConfiguration, name: str, logger: DTSLOG ) -> RemoteSession: return SSHSession(node_config, name, logger) + + +def create_interactive_session( + node_config: NodeConfiguration, name: str, logger: DTSLOG +) -> InteractiveRemoteSession: + return InteractiveRemoteSession(node_config, logger) diff --git a/dts/framework/remote_session/remote/interactive_remote_session.py b/dts/framework/remote_session/remote/interactive_remote_session.py new file mode 100644 index 00000000..4e88308a --- /dev/null +++ b/dts/framework/remote_session/remote/interactive_remote_session.py @@ -0,0 +1,113 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2023 University of New Hampshire + +import socket +import traceback +from pathlib import PurePath + +from paramiko import AutoAddPolicy, SSHClient, Transport +from paramiko.ssh_exception import ( + AuthenticationException, + BadHostKeyException, + NoValidConnectionsError, + SSHException, +) + +from framework.config import InteractiveApp, NodeConfiguration +from framework.exception import SSHConnectionError +from framework.logger import DTSLOG +from framework.settings import SETTINGS + +from .interactive_shell import InteractiveShell +from .testpmd_shell import TestPmdShell + + +class InteractiveRemoteSession: + hostname: str + ip: str + port: int + username: str + password: str + _logger: DTSLOG + _node_config: NodeConfiguration + session: SSHClient + _transport: Transport | None + + def __init__(self, node_config: NodeConfiguration, _logger: DTSLOG) -> None: + self._node_config = node_config + self._logger = _logger + self.hostname = node_config.hostname + self.username = node_config.user + self.password = node_config.password if node_config.password else "" + port = 22 + self.ip = node_config.hostname + if ":" in node_config.hostname: + self.ip, port = node_config.hostname.split(":") + self.port = int(port) + self._logger.info( + f"Initializing interactive connection for {self.username}@{self.hostname}" + ) + self._connect() + self._logger.info( + f"Interactive connection successful for {self.username}@{self.hostname}" + ) + + def _connect(self) -> None: + client = SSHClient() + client.set_missing_host_key_policy(AutoAddPolicy) + self.session = client + retry_attempts = 10 + for retry_attempt in range(retry_attempts): + try: + client.connect( + self.ip, + username=self.username, + port=self.port, + password=self.password, + timeout=20 if self.port else 10, + ) + except (TypeError, BadHostKeyException, AuthenticationException) as e: + self._logger.exception(e) + raise SSHConnectionError(self.hostname) from e + except (NoValidConnectionsError, socket.error, SSHException) as e: + self._logger.debug(traceback.format_exc()) + self._logger.warning(e) + self._logger.info( + "Retrying interactive session connection: " + f"retry number {retry_attempt +1}" + ) + else: + break + else: + raise SSHConnectionError(self.hostname) + # Interactive sessions are used on an "as needed" basis so we have + # to set a keepalive + self._transport = self.session.get_transport() + if self._transport is not None: + self._transport.set_keepalive(30) + + def create_interactive_shell( + self, + shell_type: InteractiveApp, + path_to_app: PurePath, + timeout: float = SETTINGS.timeout, + eal_parameters: str = "", + app_parameters="", + ) -> InteractiveShell: + """ + See "create_interactive_shell" in SutNode + """ + match (shell_type.name): + case "shell": + return InteractiveShell( + self.session, self._logger, path_to_app, timeout + ) + case "testpmd": + return TestPmdShell( + self.session, + self._logger, + path_to_app, + timeout=timeout, + eal_flags=eal_parameters, + cmd_line_options=app_parameters, + ) diff --git a/dts/framework/remote_session/remote/interactive_shell.py b/dts/framework/remote_session/remote/interactive_shell.py new file mode 100644 index 00000000..ba68e1ae --- /dev/null +++ b/dts/framework/remote_session/remote/interactive_shell.py @@ -0,0 +1,98 @@ +from pathlib import PurePath + +from paramiko import Channel, SSHClient, channel + +from framework.logger import DTSLOG +from framework.settings import SETTINGS + + +class InteractiveShell: + + _interactive_session: SSHClient + _stdin: channel.ChannelStdinFile + _stdout: channel.ChannelFile + _ssh_channel: Channel + _logger: DTSLOG + _timeout: float + + def __init__( + self, + interactive_session: SSHClient, + logger: DTSLOG, + path_to_app: PurePath | None = None, + timeout: float = SETTINGS.timeout, + ) -> None: + self._interactive_session = interactive_session + self._ssh_channel = self._interactive_session.invoke_shell() + self._stdin = self._ssh_channel.makefile_stdin("w") + self._stdout = self._ssh_channel.makefile("r") + self._ssh_channel.settimeout(timeout) + self._ssh_channel.set_combine_stderr(True) # combines stdout and stderr streams + self._logger = logger + self._timeout = timeout + if path_to_app: + self.send_command_no_output(str(path_to_app)) # starts the application + + def empty_stdout_buffer(self) -> None: + """Removes all data from the stdout buffer. + + Because of the way paramiko handles read buffers, there is no way to effectively + remove data from, or "flush", read buffers. This method essentially moves our + offset on the buffer to the end and thus "removes" the data from the buffer. + Timeouts are thrown on read operations of paramiko pipes based on whether data + had been recieved before timeout so we assume that if we reach the timeout then + we are at the end of the buffer. + """ + self._ssh_channel.settimeout(1) + try: + for line in self._stdout: + pass + except TimeoutError: + pass + self._ssh_channel.settimeout(self._timeout) # reset timeout + + def send_command_no_output(self, command: str) -> None: + """Send command to channel without recording output. + + This method will not verify any input or output, it will simply assume the + command succeeded. This method will also consume all output in the buffer + after executing the command. + """ + self._logger.info( + f"Sending command {command.strip()} and not collecting output" + ) + self._stdin.write(f"{command}\n") + self._stdin.flush() + self.empty_stdout_buffer() + + def send_command_get_output(self, command: str, prompt: str) -> str: + """Send a command and get all output before the expected ending string. + + Lines that expect input are not included in the stdout buffer so they cannot be + used for expect. For example, if you were prompted to log into something + with a username and password, you cannot expect "username:" because it wont + yet be in the stdout buffer. A work around for this could be consuming an + extra newline character to force the current prompt into the stdout buffer. + + Returns: + All output in the buffer before expected string + """ + self._logger.info(f"Sending command {command.strip()}...") + self._stdin.write(f"{command}\n") + self._stdin.flush() + out: str = "" + for line in self._stdout: + out += line + if prompt in line and not line.rstrip().endswith( + command.rstrip() + ): # ignore line that sent command + break + self._logger.debug(f"Got output: {out}") + return out + + def close(self) -> None: + self._stdin.close() + self._ssh_channel.close() + + def __del__(self) -> None: + self.close() diff --git a/dts/framework/remote_session/remote/testpmd_shell.py b/dts/framework/remote_session/remote/testpmd_shell.py new file mode 100644 index 00000000..80936bee --- /dev/null +++ b/dts/framework/remote_session/remote/testpmd_shell.py @@ -0,0 +1,58 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2023 University of New Hampshire + + +from pathlib import PurePath + +from paramiko import SSHClient + +from framework.logger import DTSLOG +from framework.settings import SETTINGS + +from .interactive_shell import InteractiveShell + + +class TestPmdShell(InteractiveShell): + expected_prompt: str = "testpmd>" + + def __init__( + self, + interactive_session: SSHClient, + logger: DTSLOG, + path_to_testpmd: PurePath, + eal_flags: str = "", + cmd_line_options: str = "", + timeout: float = SETTINGS.timeout, + ) -> None: + """Initalizes an interactive testpmd session using specified parameters.""" + super(TestPmdShell, self).__init__( + interactive_session, logger=logger, timeout=timeout + ) + self.send_command_get_output( + f"{path_to_testpmd} {eal_flags} -- -i {cmd_line_options}\n", + self.expected_prompt, + ) + + def send_command(self, command: str, prompt: str = expected_prompt) -> str: + """Specific way of handling the command for testpmd + + An extra newline character is consumed in order to force the current line into + the stdout buffer. + """ + return self.send_command_get_output(f"{command}\n", prompt) + + def get_devices(self) -> list[str]: + """Get a list of device names that are known to testpmd + + Uses the device info listed in testpmd and then parses the output to + return only the names of the devices. + + Returns: + A list of strings representing device names (e.g. 0000:14:00.1) + """ + dev_info: str = self.send_command("show device info all") + dev_list: list[str] = [] + for line in dev_info.split("\n"): + if "device name:" in line.lower(): + dev_list.append(line.strip().split(": ")[1].strip()) + return dev_list diff --git a/dts/framework/test_result.py b/dts/framework/test_result.py index 74391982..029665fb 100644 --- a/dts/framework/test_result.py +++ b/dts/framework/test_result.py @@ -1,5 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 PANTHEON.tech s.r.o. +# Copyright(c) 2022-2023 University of New Hampshire """ Generic result container and reporters @@ -8,14 +9,17 @@ import os.path from collections.abc import MutableSequence from enum import Enum, auto +from typing import Dict from .config import ( OS, Architecture, BuildTargetConfiguration, + BuildTargetVersionInfo, Compiler, CPUType, NodeConfiguration, + NodeVersionInfo, ) from .exception import DTSError, ErrorSeverity from .logger import DTSLOG @@ -67,12 +71,14 @@ class Statistics(dict): Using a dict provides a convenient way to format the data. """ - def __init__(self, dpdk_version): + def __init__(self, output_info: Dict[str, str] | None): super(Statistics, self).__init__() for result in Result: self[result.name] = 0 self["PASS RATE"] = 0.0 - self["DPDK VERSION"] = dpdk_version + if output_info: + for info_key, info_val in output_info.items(): + self[info_key] = info_val def __iadd__(self, other: Result) -> "Statistics": """ @@ -206,6 +212,8 @@ class BuildTargetResult(BaseResult): os: OS cpu: CPUType compiler: Compiler + compiler_version: str | None + dpdk_version: str | None def __init__(self, build_target: BuildTargetConfiguration): super(BuildTargetResult, self).__init__() @@ -213,6 +221,12 @@ def __init__(self, build_target: BuildTargetConfiguration): self.os = build_target.os self.cpu = build_target.cpu self.compiler = build_target.compiler + self.compiler_version = None + self.dpdk_version = None + + def add_build_target_versions(self, versions: BuildTargetVersionInfo) -> None: + self.compiler_version = versions.compiler_version + self.dpdk_version = versions.dpdk_version def add_test_suite(self, test_suite_name: str) -> TestSuiteResult: test_suite_result = TestSuiteResult(test_suite_name) @@ -228,10 +242,17 @@ class ExecutionResult(BaseResult): """ sut_node: NodeConfiguration + sut_os_name: str + sut_os_version: str + sut_kernel_version: str - def __init__(self, sut_node: NodeConfiguration): + def __init__(self, sut_node: NodeConfiguration, sut_version_info: NodeVersionInfo): super(ExecutionResult, self).__init__() self.sut_node = sut_node + self.sut_version_info = sut_version_info + self.sut_os_name = sut_version_info.os_name + self.sut_os_version = sut_version_info.os_version + self.sut_kernel_version = sut_version_info.kernel_version def add_build_target( self, build_target: BuildTargetConfiguration @@ -258,6 +279,7 @@ class DTSResult(BaseResult): """ dpdk_version: str | None + output: dict | None _logger: DTSLOG _errors: list[Exception] _return_code: ErrorSeverity @@ -267,14 +289,17 @@ class DTSResult(BaseResult): def __init__(self, logger: DTSLOG): super(DTSResult, self).__init__() self.dpdk_version = None + self.output = None self._logger = logger self._errors = [] self._return_code = ErrorSeverity.NO_ERR self._stats_result = None self._stats_filename = os.path.join(SETTINGS.output_dir, "statistics.txt") - def add_execution(self, sut_node: NodeConfiguration) -> ExecutionResult: - execution_result = ExecutionResult(sut_node) + def add_execution( + self, sut_node: NodeConfiguration, sut_version_info: NodeVersionInfo + ) -> ExecutionResult: + execution_result = ExecutionResult(sut_node, sut_version_info) self._inner_results.append(execution_result) return execution_result @@ -296,7 +321,8 @@ def process(self) -> None: for error in self._errors: self._logger.debug(repr(error)) - self._stats_result = Statistics(self.dpdk_version) + self._stats_result = Statistics(self.output) + # add information gathered from the smoke tests to the statistics self.add_stats(self._stats_result) with open(self._stats_filename, "w+") as stats_file: stats_file.write(str(self._stats_result)) diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py index 0705f38f..ad66014c 100644 --- a/dts/framework/test_suite.py +++ b/dts/framework/test_suite.py @@ -10,11 +10,24 @@ import inspect import re from types import MethodType +from typing import Dict -from .exception import ConfigurationError, SSHTimeoutError, TestCaseVerifyError +from .config import BuildTargetConfiguration +from .exception import ( + BlockingTestSuiteError, + ConfigurationError, + SSHTimeoutError, + TestCaseVerifyError, +) from .logger import DTSLOG, getLogger from .settings import SETTINGS -from .test_result import BuildTargetResult, Result, TestCaseResult, TestSuiteResult +from .test_result import ( + BuildTargetResult, + DTSResult, + Result, + TestCaseResult, + TestSuiteResult, +) from .testbed_model import SutNode @@ -37,10 +50,12 @@ class TestSuite(object): """ sut_node: SutNode + is_blocking = False _logger: DTSLOG _test_cases_to_run: list[str] _func: bool _result: TestSuiteResult + _dts_result: DTSResult def __init__( self, @@ -48,6 +63,8 @@ def __init__( test_cases: list[str], func: bool, build_target_result: BuildTargetResult, + build_target_conf: BuildTargetConfiguration, + dts_result: DTSResult, ): self.sut_node = sut_node self._logger = getLogger(self.__class__.__name__) @@ -55,6 +72,8 @@ def __init__( self._test_cases_to_run.extend(SETTINGS.test_cases) self._func = func self._result = build_target_result.add_test_suite(self.__class__.__name__) + self.build_target_info = build_target_conf + self._dts_result = dts_result def set_up_suite(self) -> None: """ @@ -118,6 +137,8 @@ def run(self) -> None: f"the next test suite may be affected." ) self._result.update_setup(Result.ERROR, e) + if len(self._result.get_errors()) > 0 and self.is_blocking: + raise BlockingTestSuiteError(test_suite_name) def _execute_test_suite(self) -> None: """ @@ -232,6 +253,12 @@ def _execute_test_case( test_case_result.update(Result.SKIP) raise KeyboardInterrupt("Stop DTS") + def write_to_statistics_file(self, output: Dict[str, str]): + if self._dts_result.output is not None: + self._dts_result.output.update(output) + else: + self._dts_result.output = output + def get_test_suites(testsuite_module_path: str) -> list[type[TestSuite]]: def is_test_suite(object) -> bool: diff --git a/dts/framework/testbed_model/node.py b/dts/framework/testbed_model/node.py index d48fafe6..c5147e0e 100644 --- a/dts/framework/testbed_model/node.py +++ b/dts/framework/testbed_model/node.py @@ -40,6 +40,7 @@ class Node(object): lcores: list[LogicalCore] _logger: DTSLOG _other_sessions: list[OSSession] + _execution_config: ExecutionConfiguration def __init__(self, node_config: NodeConfiguration): self.config = node_config @@ -64,6 +65,7 @@ def set_up_execution(self, execution_config: ExecutionConfiguration) -> None: """ self._setup_hugepages() self._set_up_execution(execution_config) + self._execution_config = execution_config def _set_up_execution(self, execution_config: ExecutionConfiguration) -> None: """ diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py index 2b2b50d9..27849da7 100644 --- a/dts/framework/testbed_model/sut_node.py +++ b/dts/framework/testbed_model/sut_node.py @@ -1,14 +1,21 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014 Intel Corporation # Copyright(c) 2023 PANTHEON.tech s.r.o. +# Copyright(c) 2022-2023 University of New Hampshire import os import tarfile import time from pathlib import PurePath -from framework.config import BuildTargetConfiguration, NodeConfiguration -from framework.remote_session import CommandResult, OSSession +from framework.config import ( + BuildTargetConfiguration, + BuildTargetVersionInfo, + InteractiveApp, + NodeConfiguration, + NodeVersionInfo, +) +from framework.remote_session import CommandResult, InteractiveShell, OSSession from framework.settings import SETTINGS from framework.utils import EnvVarsDict, MesonArgs @@ -26,13 +33,17 @@ class SutNode(Node): _dpdk_prefix_list: list[str] _dpdk_timestamp: str - _build_target_config: BuildTargetConfiguration | None + _build_target_config: BuildTargetConfiguration _env_vars: EnvVarsDict _remote_tmp_dir: PurePath __remote_dpdk_dir: PurePath | None - _dpdk_version: str | None _app_compile_timeout: float _dpdk_kill_session: OSSession | None + _dpdk_version: str | None + _os_name: str | None + _os_version: str | None + _kernel_version: str | None + _compiler_version: str | None def __init__(self, node_config: NodeConfiguration): super(SutNode, self).__init__(node_config) @@ -41,12 +52,16 @@ def __init__(self, node_config: NodeConfiguration): self._env_vars = EnvVarsDict() self._remote_tmp_dir = self.main_session.get_remote_tmp_dir() self.__remote_dpdk_dir = None - self._dpdk_version = None self._app_compile_timeout = 90 self._dpdk_kill_session = None self._dpdk_timestamp = ( f"{str(os.getpid())}_{time.strftime('%Y%m%d%H%M%S', time.localtime())}" ) + self._dpdk_version = None + self._os_name = None + self._os_version = None + self._kernel_version = None + self._compiler_version = None @property def _remote_dpdk_dir(self) -> PurePath: @@ -75,6 +90,44 @@ def dpdk_version(self) -> str: ) return self._dpdk_version + @property + def os_name(self) -> str: + if self._os_name is None: + self._os_name = self.main_session.get_os_name() + return self._os_name + + @property + def os_version(self) -> str: + if self._os_version is None: + self._os_version = self.main_session.get_os_version() + return self._os_version + + @property + def kernel_version(self) -> str: + if self._kernel_version is None: + self._kernel_version = self.main_session.get_kernel_version() + return self._kernel_version + + @property + def compiler_version(self) -> str: + if self._compiler_version is None: + self._compiler_version = self.main_session.get_compiler_version( + self._build_target_config.compiler.name + ) + return self._compiler_version + + def get_node_versions(self) -> NodeVersionInfo: + return NodeVersionInfo( + os_name=self.os_name, + os_version=self.os_version, + kernel_version=self.kernel_version, + ) + + def get_build_target_versions(self) -> BuildTargetVersionInfo: + return BuildTargetVersionInfo( + dpdk_version=self.dpdk_version, compiler_version=self.compiler_version + ) + def _guess_dpdk_remote_dir(self) -> PurePath: return self.main_session.guess_dpdk_remote_dir(self._remote_tmp_dir) @@ -84,6 +137,10 @@ def _set_up_build_target( """ Setup DPDK on the SUT node. """ + # we want to ensure that dpdk_version and compiler_version is reset for new + # build targets + self._dpdk_version = None + self._compiler_version = None self._configure_build_target(build_target_config) self._copy_dpdk_tarball() self._build_dpdk() @@ -262,6 +319,49 @@ def run_dpdk_app( f"{app_path} {eal_args}", timeout, verify=True ) + def create_interactive_shell( + self, + shell_type: InteractiveApp, + path_to_app: PurePath | None = None, + timeout: float = SETTINGS.timeout, + eal_parameters: str = "", + app_parameters="", + ) -> InteractiveShell: + """Create a handler for an interactive session. + + This method is a factory that calls a method in OSSession to create shells for + different DPDK applications. + + Args: + shell_type: Enum value representing the desired application. + path_to_app: Represents a path to the application you are attempting to + launch. This path will be executed at the start of the app + initialization. If one isn't provided, the default specified in the + enumeration will be used. + timeout: Timeout for the ssh channel + eal_parameters: List of EAL parameters to use to launch the app. This is + ignored for base "shell" types. + app_parameters: Command-line flags to pass into the application on launch. + Returns: + Instance of the desired interactive application. + """ + default_path: PurePath | None + # if we just want a default shell, there is no need to append the DPDK build + # directory to the path + if shell_type.name == "shell": + default_path = None + else: + default_path = self.main_session.join_remote_path( + self.remote_dpdk_build_dir, *shell_type.get_path() + ) + return self.main_session.interactive_session.create_interactive_shell( + shell_type, + path_to_app or default_path, + timeout, + eal_parameters, + app_parameters, + ) + class EalParameters(object): def __init__( diff --git a/dts/tests/TestSuite_smoke_tests.py b/dts/tests/TestSuite_smoke_tests.py new file mode 100644 index 00000000..4ab99df2 --- /dev/null +++ b/dts/tests/TestSuite_smoke_tests.py @@ -0,0 +1,109 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2023 University of New Hampshire + +from framework.config import InteractiveApp +from framework.remote_session import TestPmdShell +from framework.test_suite import TestSuite + + +class SmokeTests(TestSuite): + is_blocking = True + # in this list, the first index is the address of the nic and the second is + # the driver for that nic. + list_of_nics: list[tuple[str, str]] = [] + + def set_up_suite(self) -> None: + """ + Setup: + build all DPDK + """ + self.dpdk_build_dir_path = self.sut_node.remote_dpdk_build_dir + for nic in self.sut_node._execution_config.nics: + for address in nic.addresses: + new_tuple = (address, nic.driver.strip()) + self.list_of_nics.append(new_tuple) + + def test_unit_tests(self) -> None: + """ + Test: + run the fast-test unit-test suite through meson + """ + self.sut_node.main_session.send_command( + f"meson test -C {self.dpdk_build_dir_path} --suite fast-tests", 300 + ) + + def test_driver_tests(self) -> None: + """ + Test: + run the driver-test unit-test suite through meson + """ + list_of_vdevs = "" + for dev in self.sut_node._execution_config.vdevs: + list_of_vdevs += f"{dev}," + if list_of_vdevs: + self._logger.info( + "Running driver tests with the following virtual " + f"devices: {list_of_vdevs}" + ) + self.sut_node.main_session.send_command( + "meson test -C {self.dpdk_build_dir_path} --suite driver-tests " + f'--test-args "--vdev {list_of_vdevs}"', + 300, + ) + else: + self.sut_node.main_session.send_command( + f"meson test -C {self.dpdk_build_dir_path} --suite driver-tests", 300 + ) + + def test_gather_info(self) -> None: + """ + Test: + gather information about the system and send output to statistics.txt + """ + out = {} + + out["OS"] = self.sut_node.os_name + out["OS VERSION"] = self.sut_node.os_version + out["COMPILER VERSION"] = self.sut_node.compiler_version + out["DPDK VERSION"] = self.sut_node.dpdk_version + out["KERNEL VERSION"] = self.sut_node.kernel_version + self.write_to_statistics_file(out) + + def test_devices_listed_in_testpmd(self) -> None: + """ + Uses testpmd driver to verify that devices have been found by testpmd + """ + testpmd_driver: TestPmdShell = self.sut_node.create_interactive_shell( + InteractiveApp.testpmd + ) + dev_list: list[str] = testpmd_driver.get_devices() + for nic in self.list_of_nics: + self.verify( + nic[0] in dev_list, + f"Device {nic[0]} was not listed in testpmd's availible devices, " + "please check your configuration", + ) + + def test_device_bound_to_driver(self) -> None: + """ + Test: + ensure that all drivers listed in the config are bound to the correct driver + """ + path_to_dev = self.sut_node.main_session.join_remote_path( + self.sut_node._remote_dpdk_dir, "usertools", "dpdk-devbind.py" + ) + for nic in self.list_of_nics: + out = self.sut_node.main_session.send_command( + f"{path_to_dev} --status | grep {nic[0]}", 60 + ) + self.verify( + len(out.stdout) != 0, + f"Failed to find configured device ({nic[0]}) using dpdk-devbind.py", + ) + for string in out.stdout.split(" "): + if "drv=" in string: + self.verify( + string.split("=")[1] == nic[1], + f"Driver for device {nic[0]} does not match driver listed in " + f'configuration (bound to {string.split("=")[1]})', + ) From patchwork Fri Jun 2 15:13:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 127971 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3CD0342C11; Fri, 2 Jun 2023 17:14:28 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BC2B742D0D; Fri, 2 Jun 2023 17:14:26 +0200 (CEST) Received: from mail-pf1-f227.google.com (mail-pf1-f227.google.com [209.85.210.227]) by mails.dpdk.org (Postfix) with ESMTP id 1136B42D0D for ; Fri, 2 Jun 2023 17:14:25 +0200 (CEST) Received: by mail-pf1-f227.google.com with SMTP id d2e1a72fcca58-650c8cb68aeso1293682b3a.3 for ; Fri, 02 Jun 2023 08:14:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1685718864; x=1688310864; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UO8b4KPtjSkOEoxFE2oHnX0wYw4daxeIOAkBFJX4KQA=; b=TXI2TEpsJUQK4nZMs61rYuXR7U38u0vj7XGeivUXG/RBzhfJtSUmWFhDuyqM+SUgVN ScQEUpN9CuAQxOL5hQTyLci3EyrwlB26+PhYIiNcDutPwfBzkwOQ5PtpztI5kVDvG5NA jDJG76bItqS0och9guDJvgmYYAA6meDD9OWDg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685718864; x=1688310864; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UO8b4KPtjSkOEoxFE2oHnX0wYw4daxeIOAkBFJX4KQA=; b=OqiGbmeSyCswOWV+LyMr4DkgqM862l96uq2sShv61xSlwl6Q+plQTlXS6gRbYQxjM8 rqZKBZXhpu6f96cxLgWLVnTPekq7NaMqySG/+tuOF5YYdAoldNx1Ddt+yc1xTyD297U/ Rdiob0FRZ2Hm+zIKsKu4yLY8fiMwoFaFiUtze+DX9Oy2yMNrKvKY/ilQTcDWXz3J6/Ce IDG/1mpqDMuXRCM5bIZSrxId2hNVRBMiHm5RNuWMxGw+jtLRXC7tHP5FcdKkvHKBguse b+INDBevKd8tsRwec9PbHkDUipOmEi5rxK5pXmhyuhKTzhzzLIsTjBjlTDzoYFjgkUoI mh7g== X-Gm-Message-State: AC+VfDxUmr5P1EabltkUqUsVL9n2wUuqyzc0gx0iRrJTQ6RjAes2Ess+ BvqxYtmnN0LCPFP4z7J3xalNl5MrUKJjRTEjMH3Io5XY+Sr2PQ== X-Google-Smtp-Source: ACHHUZ6x8Kp8tDVawrua8FnoeZ7wEDJTSNkn2K39A/p0ZakEX5AhNTECGUovIO4IBz3ej0yfLiuiHH9cwcTL X-Received: by 2002:a05:6a21:7889:b0:100:6863:8be7 with SMTP id bf9-20020a056a21788900b0010068638be7mr9139841pzc.62.1685718864197; Fri, 02 Jun 2023 08:14:24 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id e18-20020a635012000000b005346c97e315sm113576pgb.6.2023.06.02.08.14.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Jun 2023 08:14:24 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1257::1083]) by postal.iol.unh.edu (Postfix) with ESMTP id 63286605246B; Fri, 2 Jun 2023 11:14:23 -0400 (EDT) From: jspewock@iol.unh.edu To: Honnappa.Nagarahalli@arm.com, juraj.linkes@pantheon.tech, thomas@monjalon.net, lijuan.tu@intel.com, wathsala.vithanage@arm.com Cc: dev@dpdk.org, Jeremy Spewock Subject: [RFC v3 2/2] dts: added paramiko to dependencies Date: Fri, 2 Jun 2023 11:13:32 -0400 Message-Id: <20230602151332.6404-4-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230602151332.6404-2-jspewock@iol.unh.edu> References: <20230602151332.6404-2-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock added paramiko to the dependency files Signed-off-by: Jeremy Spewock --- dts/poetry.lock | 160 ++++++++++++++++++++++++++++++++++----------- dts/pyproject.toml | 1 + 2 files changed, 124 insertions(+), 37 deletions(-) diff --git a/dts/poetry.lock b/dts/poetry.lock index 0b2a007d..dfd9a240 100644 --- a/dts/poetry.lock +++ b/dts/poetry.lock @@ -1,20 +1,33 @@ [[package]] name = "attrs" -version = "22.1.0" +version = "23.1.0" description = "Classes Without Boilerplate" category = "main" optional = false -python-versions = ">=3.5" +python-versions = ">=3.7" [package.extras] -dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"] -docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"] -tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "cloudpickle"] -tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "cloudpickle"] +cov = ["attrs", "coverage[toml] (>=5.3)"] +dev = ["attrs", "pre-commit"] +docs = ["furo", "myst-parser", "sphinx", "sphinx-notfound-page", "sphinxcontrib-towncrier", "towncrier", "zope-interface"] +tests = ["attrs", "zope-interface"] +tests-no-zope = ["cloudpickle", "hypothesis", "mypy (>=1.1.1)", "pympler", "pytest-mypy-plugins", "pytest-xdist", "pytest (>=4.3.0)"] + +[[package]] +name = "bcrypt" +version = "4.0.1" +description = "Modern password hashing for your software and your servers" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +tests = ["pytest (>=3.2.1,!=3.3.0)"] +typecheck = ["mypy"] [[package]] name = "black" -version = "22.10.0" +version = "22.12.0" description = "The uncompromising code formatter." category = "dev" optional = false @@ -33,6 +46,17 @@ d = ["aiohttp (>=3.7.4)"] jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"] uvloop = ["uvloop (>=0.15.2)"] +[[package]] +name = "cffi" +version = "1.15.1" +description = "Foreign Function Interface for Python calling C code." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +pycparser = "*" + [[package]] name = "click" version = "8.1.3" @@ -52,18 +76,39 @@ category = "dev" optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7" +[[package]] +name = "cryptography" +version = "41.0.1" +description = "cryptography is a package which provides cryptographic recipes and primitives to Python developers." +category = "main" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +cffi = ">=1.12" + +[package.extras] +docs = ["sphinx (>=5.3.0)", "sphinx-rtd-theme (>=1.1.1)"] +docstest = ["pyenchant (>=1.6.11)", "twine (>=1.12.0)", "sphinxcontrib-spelling (>=4.0.1)"] +nox = ["nox"] +pep8test = ["black", "ruff", "mypy", "check-sdist"] +sdist = ["build"] +ssh = ["bcrypt (>=3.1.5)"] +test = ["pytest (>=6.2.0)", "pytest-benchmark", "pytest-cov", "pytest-xdist", "pretend"] +test-randomorder = ["pytest-randomly"] + [[package]] name = "isort" -version = "5.10.1" +version = "5.12.0" description = "A Python utility / library to sort Python imports." category = "dev" optional = false -python-versions = ">=3.6.1,<4.0" +python-versions = ">=3.8.0" [package.extras] -pipfile_deprecated_finder = ["pipreqs", "requirementslib"] -requirements_deprecated_finder = ["pipreqs", "pip-api"] -colors = ["colorama (>=0.4.3,<0.5.0)"] +colors = ["colorama (>=0.4.3)"] +requirements-deprecated-finder = ["pip-api", "pipreqs"] +pipfile-deprecated-finder = ["pip-shims (>=0.5.2)", "pipreqs", "requirementslib"] plugins = ["setuptools"] [[package]] @@ -87,7 +132,7 @@ python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" [[package]] name = "jsonschema" -version = "4.17.0" +version = "4.17.3" description = "An implementation of JSON Schema validation for Python" category = "main" optional = false @@ -129,15 +174,33 @@ reports = ["lxml"] [[package]] name = "mypy-extensions" -version = "0.4.3" -description = "Experimental type system extensions for programs checked with the mypy typechecker." +version = "1.0.0" +description = "Type system extensions for programs checked with the mypy type checker." category = "dev" optional = false -python-versions = "*" +python-versions = ">=3.5" + +[[package]] +name = "paramiko" +version = "3.2.0" +description = "SSH2 protocol library" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +bcrypt = ">=3.2" +cryptography = ">=3.3" +pynacl = ">=1.5" + +[package.extras] +all = ["pyasn1 (>=0.1.7)", "invoke (>=2.0)", "gssapi (>=1.4.1)", "pywin32 (>=2.1.8)"] +gssapi = ["pyasn1 (>=0.1.7)", "gssapi (>=1.4.1)", "pywin32 (>=2.1.8)"] +invoke = ["invoke (>=2.0)"] [[package]] name = "pathspec" -version = "0.10.1" +version = "0.11.1" description = "Utility library for gitignore style pattern matching of file paths." category = "dev" optional = false @@ -156,15 +219,15 @@ ptyprocess = ">=0.5" [[package]] name = "platformdirs" -version = "2.5.2" -description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." +version = "3.5.1" +description = "A small Python package for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." category = "dev" optional = false python-versions = ">=3.7" [package.extras] -docs = ["furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)", "sphinx (>=4)"] -test = ["appdirs (==1.4.4)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)", "pytest (>=6)"] +docs = ["furo (>=2023.3.27)", "proselint (>=0.13)", "sphinx-autodoc-typehints (>=1.23,!=1.23.4)", "sphinx (>=6.2.1)"] +test = ["appdirs (==1.4.4)", "covdefaults (>=2.3)", "pytest-cov (>=4)", "pytest-mock (>=3.10)", "pytest (>=7.3.1)"] [[package]] name = "ptyprocess" @@ -176,25 +239,33 @@ python-versions = "*" [[package]] name = "pycodestyle" -version = "2.9.1" +version = "2.10.0" description = "Python style guide checker" category = "dev" optional = false python-versions = ">=3.6" +[[package]] +name = "pycparser" +version = "2.21" +description = "C parser in Python" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + [[package]] name = "pydocstyle" -version = "6.1.1" +version = "6.3.0" description = "Python docstring style checker" category = "dev" optional = false python-versions = ">=3.6" [package.dependencies] -snowballstemmer = "*" +snowballstemmer = ">=2.2.0" [package.extras] -toml = ["toml"] +toml = ["tomli (>=1.2.3)"] [[package]] name = "pyflakes" @@ -228,9 +299,24 @@ tests = ["pytest (>=7.1.2)", "pytest-mypy", "eradicate (>=2.0.0)", "radon (>=5.1 toml = ["toml (>=0.10.2)"] vulture = ["vulture"] +[[package]] +name = "pynacl" +version = "1.5.0" +description = "Python binding to the Networking and Cryptography (NaCl) library" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cffi = ">=1.4.1" + +[package.extras] +docs = ["sphinx (>=1.6.5)", "sphinx-rtd-theme"] +tests = ["pytest (>=3.2.1,!=3.3.0)", "hypothesis (>=3.27.0)"] + [[package]] name = "pyrsistent" -version = "0.19.1" +version = "0.19.3" description = "Persistent/Functional/Immutable data structures" category = "main" optional = false @@ -270,7 +356,7 @@ python-versions = ">=3.7" [[package]] name = "types-pyyaml" -version = "6.0.12.1" +version = "6.0.12.10" description = "Typing stubs for PyYAML" category = "main" optional = false @@ -278,7 +364,7 @@ python-versions = "*" [[package]] name = "typing-extensions" -version = "4.4.0" +version = "4.6.2" description = "Backported and Experimental Type Hints for Python 3.7+" category = "dev" optional = false @@ -299,13 +385,16 @@ jsonschema = ">=4,<5" [metadata] lock-version = "1.1" python-versions = "^3.10" -content-hash = "a0f040b07fc6ce4deb0be078b9a88c2a465cb6bccb9e260a67e92c2403e2319f" +content-hash = "c119901b1c13b14adf9b53624430aff8720d9a3a180b028579b82c65d49474df" [metadata.files] attrs = [] +bcrypt = [] black = [] +cffi = [] click = [] colorama = [] +cryptography = [] isort = [] jsonpatch = [] jsonpointer = [] @@ -313,20 +402,17 @@ jsonschema = [] mccabe = [] mypy = [] mypy-extensions = [] +paramiko = [] pathspec = [] -pexpect = [ - {file = "pexpect-4.8.0-py2.py3-none-any.whl", hash = "sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937"}, - {file = "pexpect-4.8.0.tar.gz", hash = "sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c"}, -] -platformdirs = [ - {file = "platformdirs-2.5.2-py3-none-any.whl", hash = "sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788"}, - {file = "platformdirs-2.5.2.tar.gz", hash = "sha256:58c8abb07dcb441e6ee4b11d8df0ac856038f944ab98b7be6b27b2a3c7feef19"}, -] +pexpect = [] +platformdirs = [] ptyprocess = [] pycodestyle = [] +pycparser = [] pydocstyle = [] pyflakes = [] pylama = [] +pynacl = [] pyrsistent = [] pyyaml = [] snowballstemmer = [] diff --git a/dts/pyproject.toml b/dts/pyproject.toml index a136c91e..928837cb 100644 --- a/dts/pyproject.toml +++ b/dts/pyproject.toml @@ -13,6 +13,7 @@ pexpect = "^4.8.0" warlock = "^2.0.1" PyYAML = "^6.0" types-PyYAML = "^6.0.8" +paramiko = "^3.1.0" [tool.poetry.dev-dependencies] mypy = "^0.961"