From patchwork Thu May 9 11:26:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140007 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DE61C43F7C; Thu, 9 May 2024 13:26:53 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4682F406B6; Thu, 9 May 2024 13:26:50 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id C1C29402ED for ; Thu, 9 May 2024 13:26:48 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9886D12FC; Thu, 9 May 2024 04:27:13 -0700 (PDT) Received: from localhost.localdomain (unknown [10.1.194.74]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 72D683F6A8; Thu, 9 May 2024 04:26:47 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: =?utf-8?q?Juraj_Linke=C5=A1?= , Jeremy Spewock , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v2 1/5] dts: fix InteractiveShell command prompt filtering Date: Thu, 9 May 2024 12:26:31 +0100 Message-Id: <20240509112635.1170557-2-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240509112635.1170557-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240509112635.1170557-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When sending a command using an instance of InteractiveShell the output should filter out the trailing shell prompt when returning it. After every command two shell prompts are summoned. One is consumed as it is used as a delimiter for the command output. The second one is not consumed and left for the next command to be sent. Given that the consumed prompt is merely a delimiter, this should not be added to the returned output, as it may be mistakenly be interpreted as the command's own output. Bugzilla ID: 1411 Fixes: 88489c0501af ("dts: add smoke tests") Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek --- dts/framework/remote_session/interactive_shell.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dts/framework/remote_session/interactive_shell.py b/dts/framework/remote_session/interactive_shell.py index 074a541279..aa5d2d9be8 100644 --- a/dts/framework/remote_session/interactive_shell.py +++ b/dts/framework/remote_session/interactive_shell.py @@ -132,11 +132,11 @@ def send_command(self, command: str, prompt: str | None = None) -> str: self._stdin.flush() out: str = "" for line in self._stdout: - out += line if prompt in line and not line.rstrip().endswith( command.rstrip() ): # ignore line that sent command break + out += line self._logger.debug(f"Got output: {out}") return out From patchwork Thu May 9 11:26:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140008 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3DF4A43F7C; Thu, 9 May 2024 13:26:59 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5D5B440A6C; Thu, 9 May 2024 13:26:51 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 856674064C for ; Thu, 9 May 2024 13:26:49 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 808CD106F; Thu, 9 May 2024 04:27:14 -0700 (PDT) Received: from localhost.localdomain (unknown [10.1.194.74]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 558193F6A8; Thu, 9 May 2024 04:26:48 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: =?utf-8?q?Juraj_Linke=C5=A1?= , Jeremy Spewock , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v2 2/5] dts: skip first line of send command output Date: Thu, 9 May 2024 12:26:32 +0100 Message-Id: <20240509112635.1170557-3-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240509112635.1170557-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240509112635.1170557-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The first line of the InteractiveShell send_command method is generally the command input field. This sometimes is unwanted, therefore this commit enables the possibility of omitting the first line from the returned output. Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek --- dts/framework/remote_session/interactive_shell.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/dts/framework/remote_session/interactive_shell.py b/dts/framework/remote_session/interactive_shell.py index aa5d2d9be8..c025c52ba3 100644 --- a/dts/framework/remote_session/interactive_shell.py +++ b/dts/framework/remote_session/interactive_shell.py @@ -105,7 +105,9 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None start_command = get_privileged_command(start_command) self.send_command(start_command) - def send_command(self, command: str, prompt: str | None = None) -> str: + def send_command( + self, command: str, prompt: str | None = None, skip_first_line: bool = False + ) -> str: """Send `command` and get all output before the expected ending string. Lines that expect input are not included in the stdout buffer, so they cannot @@ -121,6 +123,7 @@ def send_command(self, command: str, prompt: str | None = None) -> str: command: The command to send. prompt: After sending the command, `send_command` will be expecting this string. If :data:`None`, will use the class's default prompt. + skip_first_line: Skip the first line when capturing the output. Returns: All output in the buffer before expected string. @@ -132,6 +135,9 @@ def send_command(self, command: str, prompt: str | None = None) -> str: self._stdin.flush() out: str = "" for line in self._stdout: + if skip_first_line: + skip_first_line = False + continue if prompt in line and not line.rstrip().endswith( command.rstrip() ): # ignore line that sent command From patchwork Thu May 9 11:26:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140009 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4EF3743F7C; Thu, 9 May 2024 13:27:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 97ABF40A70; Thu, 9 May 2024 13:26:52 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 939DE406BC for ; Thu, 9 May 2024 13:26:50 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 656CE12FC; Thu, 9 May 2024 04:27:15 -0700 (PDT) Received: from localhost.localdomain (unknown [10.1.194.74]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3E4B43F6A8; Thu, 9 May 2024 04:26:49 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: =?utf-8?q?Juraj_Linke=C5=A1?= , Jeremy Spewock , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v2 3/5] dts: add parsing utility module Date: Thu, 9 May 2024 12:26:33 +0100 Message-Id: <20240509112635.1170557-4-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240509112635.1170557-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240509112635.1170557-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds parsing text into a custom dataclass. It provides a new `TextParser` dataclass to be inherited. This implements the `parse` method, which combined with the parser functions, it can automatically parse the value for each field. This new utility will facilitate and simplify the parsing of complex command outputs, while ensuring that the codebase does not get bloated and stays flexible. Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek --- dts/framework/parser.py | 199 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 199 insertions(+) create mode 100644 dts/framework/parser.py diff --git a/dts/framework/parser.py b/dts/framework/parser.py new file mode 100644 index 0000000000..5b4acddead --- /dev/null +++ b/dts/framework/parser.py @@ -0,0 +1,199 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2024 Arm Limited + +r"""Parsing utility module. + +This module provides :class:`~TextParser` which can be used to model any dataclass to a block of +text. + +Usage example:: +..code:: python + + from dataclasses import dataclass, field + from enum import Enum + from framework.parser import TextParser + + class Colour(Enum): + BLACK = 1 + WHITE = 2 + + @classmethod + def from_str(cls, text: str): + match text: + case "black": + return cls.BLACK + case "white": + return cls.WHITE + case _: + return None # unsupported colour + + @classmethod + def make_parser(cls): + # make a parser function that finds a match and + # then makes it a Colour object through Colour.from_str + return TextParser.compose(cls.from_str, TextParser.find(r"is a (\w+)")) + + @dataclass + class Animal(TextParser): + kind: str = field(metadata=TextParser.find(r"is a \w+ (\w+)")) + name: str = field(metadata=TextParser.find(r"^(\w+)")) + colour: Colour = field(metadata=Colour.make_parser()) + age: int = field(metadata=TextParser.find_int(r"aged (\d+)")) + + steph = Animal.parse("Stephanie is a white cat aged 10") + print(steph) # Animal(kind='cat', name='Stephanie', colour=, age=10) +""" + +import re +from abc import ABC +from dataclasses import MISSING, dataclass, fields +from functools import partial +from typing import Any, Callable, TypedDict, cast + +from typing_extensions import Self + + +class ParserFn(TypedDict): + """Parser function in a dict compatible with the :func:`dataclasses.field` metadata param.""" + + #: + TextParser_fn: Callable[[str], Any] + + +@dataclass +class TextParser(ABC): + """Helper abstract dataclass that parses a text according to the fields' rules. + + This class provides a selection of parser functions and a function to compose generic functions + with parser functions. Parser functions are designed to be passed to the fields' metadata param + to enable parsing. + """ + + """============ BEGIN PARSER FUNCTIONS ============""" + + @staticmethod + def compose(f: Callable, parser_fn: ParserFn) -> ParserFn: + """Makes a composite parser function. + + The parser function is run and if a non-None value was returned, f is called with it. + Otherwise the function returns early with None. + + Metadata modifier for :func:`dataclasses.field`. + + Args: + f: the function to apply to the parser's result + parser_fn: the dictionary storing the parser function + """ + g = parser_fn["TextParser_fn"] + + def _composite_parser_fn(text: str) -> Any: + intermediate_value = g(text) + if intermediate_value is None: + return None + return f(intermediate_value) + + return ParserFn(TextParser_fn=_composite_parser_fn) + + @staticmethod + def find( + pattern: str | re.Pattern[str], + flags: re.RegexFlag = re.RegexFlag(0), + named: bool = False, + ) -> ParserFn: + """Makes a parser function that finds a regular expression match in the text. + + If the pattern has capturing groups, it returns None if no match was found. If the pattern + has only one capturing group and a match was found, its value is returned. If the pattern + has no capturing groups then either True or False is returned if the pattern had a match or + not. + + Metadata modifier for :func:`dataclasses.field`. + + Args: + pattern: the regular expression pattern + flags: the regular expression flags. Not used if the given pattern is already compiled + named: if set to True only the named capture groups will be returned as a dictionary + """ + if isinstance(pattern, str): + pattern = re.compile(pattern, flags) + + def _find(text: str) -> Any: + m = pattern.search(text) + if m is None: + return None if pattern.groups > 0 else False + + if pattern.groups == 0: + return True + + if named: + return m.groupdict() + + matches = m.groups() + if len(matches) == 1: + return matches[0] + + return matches + + return ParserFn(TextParser_fn=_find) + + @classmethod + def find_int( + cls, + pattern: str | re.Pattern[str], + flags: re.RegexFlag = re.RegexFlag(0), + int_base: int = 0, + ) -> ParserFn: + """Makes a parser function that converts the match of :meth:`~find` to int. + + This function is compatible only with a pattern containing one capturing group. + + Metadata modifier for :func:`dataclasses.field`. + + Args: + pattern: the regular expression pattern + flags: the regular expression flags + int_base: the base of the number to convert from + Raises: + RuntimeError: if the pattern does not have exactly one capturing group + """ + if isinstance(pattern, str): + pattern = re.compile(pattern, flags) + + if pattern.groups != 1: + raise RuntimeError("only one capturing group is allowed with this parser function") + + return cls.compose(partial(int, base=int_base), cls.find(pattern)) + + """============ END PARSER FUNCTIONS ============""" + + @classmethod + def parse(cls, text: str) -> Self: + """Creates a new instance of the class from the given text. + + A new class instance is created with all the fields that have a parser function in their + metadata. Fields without one are ignored and are expected to have a default value, otherwise + the class initialization will fail. + + A field is populated with the value returned by its corresponding parser function. + + Args: + text: the text to parse + Raises: + RuntimeError: if the parser did not find a match and the field does not have a default + value or default factory. + """ + fields_values = {} + for field in fields(cls): + parse = cast(ParserFn, field.metadata).get("TextParser_fn") + if parse is None: + continue + + value = parse(text) + if value is not None: + fields_values[field.name] = value + elif field.default is MISSING and field.default_factory is MISSING: + raise RuntimeError( + f"parser for field {field.name} returned None, but the field has no default" + ) + + return cls(**fields_values) From patchwork Thu May 9 11:26:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140010 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3B55643F7C; Thu, 9 May 2024 13:27:11 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C6F31402ED; Thu, 9 May 2024 13:26:53 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 7D1B840A6F for ; Thu, 9 May 2024 13:26:51 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4E2F7106F; Thu, 9 May 2024 04:27:16 -0700 (PDT) Received: from localhost.localdomain (unknown [10.1.194.74]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 239CE3F6A8; Thu, 9 May 2024 04:26:50 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: =?utf-8?q?Juraj_Linke=C5=A1?= , Jeremy Spewock , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v2 4/5] dts: add `show port info` command to TestPmdShell Date: Thu, 9 May 2024 12:26:34 +0100 Message-Id: <20240509112635.1170557-5-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240509112635.1170557-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240509112635.1170557-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add a new TestPmdPort data structure to represent the output returned by `show port info`, which is implemented as part of TestPmdShell. The TestPmdPort data structure and its derived classes are modelled based on the relevant testpmd source code. This implementation makes extensive use of regular expressions, which all parse individually. The rationale behind this is to lower the risk of the testpmd output changing as part of development. Therefore minimising breakage. Bugzilla ID: 1407 Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek --- dts/framework/remote_session/testpmd_shell.py | 490 +++++++++++++++++- 1 file changed, 489 insertions(+), 1 deletion(-) diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py index cb2ab6bd00..7910e17fed 100644 --- a/dts/framework/remote_session/testpmd_shell.py +++ b/dts/framework/remote_session/testpmd_shell.py @@ -1,6 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 University of New Hampshire # Copyright(c) 2023 PANTHEON.tech s.r.o. +# Copyright(c) 2024 Arm Limited """Testpmd interactive shell. @@ -15,12 +16,17 @@ testpmd_shell.close() """ +import re import time -from enum import auto +from dataclasses import dataclass, field +from enum import Flag, auto from pathlib import PurePath from typing import Callable, ClassVar +from typing_extensions import Self + from framework.exception import InteractiveCommandExecutionError +from framework.parser import ParserFn, TextParser from framework.settings import SETTINGS from framework.utils import StrEnum @@ -80,6 +86,451 @@ class TestPmdForwardingModes(StrEnum): recycle_mbufs = auto() +class VLANOffloadFlag(Flag): + """Flag representing the VLAN offload settings of a NIC port.""" + + #: + STRIP = auto() + #: + FILTER = auto() + #: + EXTEND = auto() + #: + QINQ_STRIP = auto() + + @classmethod + def from_str_dict(cls, d): + """Makes an instance from a dict containing the flag member names with an "on" value.""" + flag = cls(0) + for name in cls.__members__: + if d.get(name) == "on": + flag |= cls[name] + return flag + + @classmethod + def make_parser(cls) -> ParserFn: + """Makes a parser function.""" + return TextParser.compose( + cls.from_str_dict, + TextParser.find( + r"VLAN offload:\s+" + r"strip (?Pon|off), " + r"filter (?Pon|off), " + r"extend (?Pon|off), " + r"qinq strip (?Pon|off)$", + re.MULTILINE, + named=True, + ), + ) + + +class RSSOffloadTypesFlag(Flag): + """Flag representing the RSS offload flow types supported by the NIC port.""" + + #: + ipv4 = auto() + #: + ipv4_frag = auto() + #: + ipv4_tcp = auto() + #: + ipv4_udp = auto() + #: + ipv4_sctp = auto() + #: + ipv4_other = auto() + #: + ipv6 = auto() + #: + ipv6_frag = auto() + #: + ipv6_tcp = auto() + #: + ipv6_udp = auto() + #: + ipv6_sctp = auto() + #: + ipv6_other = auto() + #: + l2_payload = auto() + #: + ipv6_ex = auto() + #: + ipv6_tcp_ex = auto() + #: + ipv6_udp_ex = auto() + #: + port = auto() + #: + vxlan = auto() + #: + geneve = auto() + #: + nvgre = auto() + #: + user_defined_22 = auto() + #: + gtpu = auto() + #: + eth = auto() + #: + s_vlan = auto() + #: + c_vlan = auto() + #: + esp = auto() + #: + ah = auto() + #: + l2tpv3 = auto() + #: + pfcp = auto() + #: + pppoe = auto() + #: + ecpri = auto() + #: + mpls = auto() + #: + ipv4_chksum = auto() + #: + l4_chksum = auto() + #: + l2tpv2 = auto() + #: + ipv6_flow_label = auto() + #: + user_defined_38 = auto() + #: + user_defined_39 = auto() + #: + user_defined_40 = auto() + #: + user_defined_41 = auto() + #: + user_defined_42 = auto() + #: + user_defined_43 = auto() + #: + user_defined_44 = auto() + #: + user_defined_45 = auto() + #: + user_defined_46 = auto() + #: + user_defined_47 = auto() + #: + user_defined_48 = auto() + #: + user_defined_49 = auto() + #: + user_defined_50 = auto() + #: + user_defined_51 = auto() + #: + l3_pre96 = auto() + #: + l3_pre64 = auto() + #: + l3_pre56 = auto() + #: + l3_pre48 = auto() + #: + l3_pre40 = auto() + #: + l3_pre32 = auto() + #: + l2_dst_only = auto() + #: + l2_src_only = auto() + #: + l4_dst_only = auto() + #: + l4_src_only = auto() + #: + l3_dst_only = auto() + #: + l3_src_only = auto() + + #: + ip = ipv4 | ipv4_frag | ipv4_other | ipv6 | ipv6_frag | ipv6_other | ipv6_ex + #: + udp = ipv4_udp | ipv6_udp | ipv6_udp_ex + #: + tcp = ipv4_tcp | ipv6_tcp | ipv6_tcp_ex + #: + sctp = ipv4_sctp | ipv6_sctp + #: + tunnel = vxlan | geneve | nvgre + #: + vlan = s_vlan | c_vlan + #: + all = ( + eth + | vlan + | ip + | tcp + | udp + | sctp + | l2_payload + | l2tpv3 + | esp + | ah + | pfcp + | gtpu + | ecpri + | mpls + | l2tpv2 + ) + + @classmethod + def from_list_string(cls, names: str) -> Self: + """Makes a flag from a whitespace-separated list of names.""" + flag = cls(0) + for name in names.split(): + flag |= cls.from_str(name) + return flag + + @classmethod + def from_str(cls, name: str) -> Self: + """Returns the flag corresponding to the supplied name.""" + member_name = name.strip().replace("-", "_") + return cls[member_name] + + def __str__(self): + """String representation.""" + return self.name.replace("_", "-") + + @classmethod + def make_parser(cls) -> ParserFn: + """Makes a parser function.""" + return TextParser.compose( + RSSOffloadTypesFlag.from_list_string, + TextParser.find(r"Supported RSS offload flow types:((?:\r?\n? \S+)+)", re.MULTILINE), + ) + + +class DeviceCapabilitiesFlag(Flag): + """Flag representing the device capabilities.""" + + RUNTIME_RX_QUEUE_SETUP = auto() + """Device supports Rx queue setup after device started.""" + RUNTIME_TX_QUEUE_SETUP = auto() + """Device supports Tx queue setup after device started.""" + RXQ_SHARE = auto() + """Device supports shared Rx queue among ports within Rx domain and switch domain.""" + FLOW_RULE_KEEP = auto() + """Device supports keeping flow rules across restart.""" + FLOW_SHARED_OBJECT_KEEP = auto() + """Device supports keeping shared flow objects across restart.""" + + @classmethod + def make_parser(cls) -> ParserFn: + """Makes a parser function.""" + return TextParser.compose( + cls, + TextParser.find_int(r"Device capabilities: (0x[A-Fa-f\d]+)"), + ) + + +class DeviceErrorHandlingMode(StrEnum): + """Enum representing the device error handling mode.""" + + #: + none = auto() + #: + passive = auto() + #: + proactive = auto() + #: + unknown = auto() + + @classmethod + def make_parser(cls) -> ParserFn: + """Makes a parser function.""" + return TextParser.compose(cls, TextParser.find(r"Device error handling mode: (\w+)")) + + +def make_device_private_info_parser() -> ParserFn: + """Device private information parser. + + Ensure that we are not parsing invalid device private info output. + """ + + def _validate(info: str): + info = info.strip() + if info == "none" or info.startswith("Invalid file") or info.startswith("Failed to dump"): + return None + return info + + return TextParser.compose(_validate, TextParser.find(r"Device private info:\s+([\s\S]+)")) + + +@dataclass +class TestPmdPort(TextParser): + """Dataclass representing the result of testpmd's ``show port info`` command.""" + + #: + id: int = field(metadata=TextParser.find_int(r"Infos for port (\d+)\b")) + #: + device_name: str = field(metadata=TextParser.find(r"Device name: ([^\r\n]+)")) + #: + driver_name: str = field(metadata=TextParser.find(r"Driver name: ([^\r\n]+)")) + #: + socket_id: int = field(metadata=TextParser.find_int(r"Connect to socket: (\d+)")) + #: + is_link_up: bool = field(metadata=TextParser.find("Link status: up")) + #: + link_speed: str = field(metadata=TextParser.find(r"Link speed: ([^\r\n]+)")) + #: + is_link_full_duplex: bool = field(metadata=TextParser.find("Link duplex: full-duplex")) + #: + is_link_autonegotiated: bool = field(metadata=TextParser.find("Autoneg status: On")) + #: + is_promiscuous_mode_enabled: bool = field(metadata=TextParser.find("Promiscuous mode: enabled")) + #: + is_allmulticast_mode_enabled: bool = field( + metadata=TextParser.find("Allmulticast mode: enabled") + ) + #: Maximum number of MAC addresses + max_mac_addresses_num: int = field( + metadata=TextParser.find_int(r"Maximum number of MAC addresses: (\d+)") + ) + #: Maximum configurable length of RX packet + max_hash_mac_addresses_num: int = field( + metadata=TextParser.find_int(r"Maximum number of MAC addresses of hash filtering: (\d+)") + ) + #: Minimum size of RX buffer + min_rx_bufsize: int = field(metadata=TextParser.find_int(r"Minimum size of RX buffer: (\d+)")) + #: Maximum configurable length of RX packet + max_rx_packet_length: int = field( + metadata=TextParser.find_int(r"Maximum configurable length of RX packet: (\d+)") + ) + #: Maximum configurable size of LRO aggregated packet + max_lro_packet_size: int = field( + metadata=TextParser.find_int(r"Maximum configurable size of LRO aggregated packet: (\d+)") + ) + + #: Current number of RX queues + rx_queues_num: int = field(metadata=TextParser.find_int(r"Current number of RX queues: (\d+)")) + #: Max possible RX queues + max_rx_queues_num: int = field(metadata=TextParser.find_int(r"Max possible RX queues: (\d+)")) + #: Max possible number of RXDs per queue + max_queue_rxd_num: int = field( + metadata=TextParser.find_int(r"Max possible number of RXDs per queue: (\d+)") + ) + #: Min possible number of RXDs per queue + min_queue_rxd_num: int = field( + metadata=TextParser.find_int(r"Min possible number of RXDs per queue: (\d+)") + ) + #: RXDs number alignment + rxd_alignment_num: int = field(metadata=TextParser.find_int(r"RXDs number alignment: (\d+)")) + + #: Current number of TX queues + tx_queues_num: int = field(metadata=TextParser.find_int(r"Current number of TX queues: (\d+)")) + #: Max possible TX queues + max_tx_queues_num: int = field(metadata=TextParser.find_int(r"Max possible TX queues: (\d+)")) + #: Max possible number of TXDs per queue + max_queue_txd_num: int = field( + metadata=TextParser.find_int(r"Max possible number of TXDs per queue: (\d+)") + ) + #: Min possible number of TXDs per queue + min_queue_txd_num: int = field( + metadata=TextParser.find_int(r"Min possible number of TXDs per queue: (\d+)") + ) + #: TXDs number alignment + txd_alignment_num: int = field(metadata=TextParser.find_int(r"TXDs number alignment: (\d+)")) + #: Max segment number per packet + max_packet_segment_num: int = field( + metadata=TextParser.find_int(r"Max segment number per packet: (\d+)") + ) + #: Max segment number per MTU/TSO + max_mtu_segment_num: int = field( + metadata=TextParser.find_int(r"Max segment number per MTU\/TSO: (\d+)") + ) + + #: + device_capabilities: DeviceCapabilitiesFlag = field( + metadata=DeviceCapabilitiesFlag.make_parser(), + ) + #: + device_error_handling_mode: DeviceErrorHandlingMode = field( + metadata=DeviceErrorHandlingMode.make_parser() + ) + #: + device_private_info: str | None = field( + default=None, + metadata=make_device_private_info_parser(), + ) + + #: + hash_key_size: int | None = field( + default=None, metadata=TextParser.find_int(r"Hash key size in bytes: (\d+)") + ) + #: + redirection_table_size: int | None = field( + default=None, metadata=TextParser.find_int(r"Redirection table size: (\d+)") + ) + #: + supported_rss_offload_flow_types: RSSOffloadTypesFlag = field( + default=RSSOffloadTypesFlag(0), metadata=RSSOffloadTypesFlag.make_parser() + ) + + #: + mac_address: str | None = field( + default=None, metadata=TextParser.find(r"MAC address: ([A-Fa-f0-9:]+)") + ) + #: + fw_version: str | None = field( + default=None, metadata=TextParser.find(r"Firmware-version: ([^\r\n]+)") + ) + #: + dev_args: str | None = field(default=None, metadata=TextParser.find(r"Devargs: ([^\r\n]+)")) + #: Socket id of the memory allocation + mem_alloc_socket_id: int | None = field( + default=None, + metadata=TextParser.find_int(r"memory allocation on the socket: (\d+)"), + ) + #: + mtu: int | None = field(default=None, metadata=TextParser.find_int(r"MTU: (\d+)")) + + #: + vlan_offload: VLANOffloadFlag | None = field( + default=None, + metadata=VLANOffloadFlag.make_parser(), + ) + + #: Maximum size of RX buffer + max_rx_bufsize: int | None = field( + default=None, metadata=TextParser.find_int(r"Maximum size of RX buffer: (\d+)") + ) + #: Maximum number of VFs + max_vfs_num: int | None = field( + default=None, metadata=TextParser.find_int(r"Maximum number of VFs: (\d+)") + ) + #: Maximum number of VMDq pools + max_vmdq_pools_num: int | None = field( + default=None, metadata=TextParser.find_int(r"Maximum number of VMDq pools: (\d+)") + ) + + #: + switch_name: str | None = field( + default=None, metadata=TextParser.find(r"Switch name: ([\r\n]+)") + ) + #: + switch_domain_id: int | None = field( + default=None, metadata=TextParser.find_int(r"Switch domain Id: (\d+)") + ) + #: + switch_port_id: int | None = field( + default=None, metadata=TextParser.find_int(r"Switch Port Id: (\d+)") + ) + #: + switch_rx_domain: int | None = field( + default=None, metadata=TextParser.find_int(r"Switch Rx domain: (\d+)") + ) + + class TestPmdShell(InteractiveShell): """Testpmd interactive shell. @@ -225,6 +676,43 @@ def set_forward_mode(self, mode: TestPmdForwardingModes, verify: bool = True): f"Test pmd failed to set fwd mode to {mode.value}" ) + def show_port_info_all(self) -> list[TestPmdPort]: + """Returns the information of all the ports.""" + output = self.send_command("show port info all") + + # Sample output of the "all" command looks like: + # + # + # + # ********************* Infos for port 0 ********************* + # Key: value + # + # ********************* Infos for port 1 ********************* + # Key: value + # + # + # Take advantage of the double new line in between ports as end delimiter. + # But we need to artificially add a new line at the end to pick up the last port. + # Because commands are executed on a pseudo-terminal created by paramiko on the remote + # target lines end with CRLF. Therefore we also need to take the carriage return in account. + iter = re.finditer(r"\*{21}.*?[\r\n]{4}", output + "\r\n", re.S) + return [TestPmdPort.parse(block.group(0)) for block in iter] + + def show_port_info(self, port_id: int) -> TestPmdPort: + """Returns the given port information. + + Args: + port_id: The port ID to gather information for. + + Raises: + InteractiveCommandExecutionError: If `port_id` is invalid. + """ + output = self.send_command(f"show port info {port_id}", skip_first_line=True) + if output.startswith("Invalid port"): + raise InteractiveCommandExecutionError("invalid port given") + + return TestPmdPort.parse(output) + def close(self) -> None: """Overrides :meth:`~.interactive_shell.close`.""" self.send_command("quit", "") From patchwork Thu May 9 11:26:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140011 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8155A43F7C; Thu, 9 May 2024 13:27:19 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6D1A940DDA; Thu, 9 May 2024 13:26:55 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 2939F40A6F for ; Thu, 9 May 2024 13:26:52 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 318CB12FC; Thu, 9 May 2024 04:27:17 -0700 (PDT) Received: from localhost.localdomain (unknown [10.1.194.74]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0B6313F6A8; Thu, 9 May 2024 04:26:50 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: =?utf-8?q?Juraj_Linke=C5=A1?= , Jeremy Spewock , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v2 5/5] dts: add `show port stats` command to TestPmdShell Date: Thu, 9 May 2024 12:26:35 +0100 Message-Id: <20240509112635.1170557-6-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240509112635.1170557-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240509112635.1170557-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add a new TestPmdPortStats data structure to represent the output returned by `show port stats`, which is implemented as part of TestPmdShell. Bugzilla ID: 1407 Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek --- dts/framework/remote_session/testpmd_shell.py | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py index 7910e17fed..d0b6da50f0 100644 --- a/dts/framework/remote_session/testpmd_shell.py +++ b/dts/framework/remote_session/testpmd_shell.py @@ -531,6 +531,42 @@ class TestPmdPort(TextParser): ) +@dataclass +class TestPmdPortStats(TextParser): + """Port statistics.""" + + #: + port_id: int = field(metadata=TextParser.find_int(r"NIC statistics for port (\d+)")) + + #: + rx_packets: int = field(metadata=TextParser.find_int(r"RX-packets:\s+(\d+)")) + #: + rx_missed: int = field(metadata=TextParser.find_int(r"RX-missed:\s+(\d+)")) + #: + rx_bytes: int = field(metadata=TextParser.find_int(r"RX-bytes:\s+(\d+)")) + #: + rx_errors: int = field(metadata=TextParser.find_int(r"RX-errors:\s+(\d+)")) + #: + rx_nombuf: int = field(metadata=TextParser.find_int(r"RX-nombuf:\s+(\d+)")) + + #: + tx_packets: int = field(metadata=TextParser.find_int(r"TX-packets:\s+(\d+)")) + #: + tx_errors: int = field(metadata=TextParser.find_int(r"TX-errors:\s+(\d+)")) + #: + tx_bytes: int = field(metadata=TextParser.find_int(r"TX-bytes:\s+(\d+)")) + + #: + rx_pps: int = field(metadata=TextParser.find_int(r"Rx-pps:\s+(\d+)")) + #: + rx_bps: int = field(metadata=TextParser.find_int(r"Rx-bps:\s+(\d+)")) + + #: + tx_pps: int = field(metadata=TextParser.find_int(r"Tx-pps:\s+(\d+)")) + #: + tx_bps: int = field(metadata=TextParser.find_int(r"Tx-bps:\s+(\d+)")) + + class TestPmdShell(InteractiveShell): """Testpmd interactive shell. @@ -713,6 +749,38 @@ def show_port_info(self, port_id: int) -> TestPmdPort: return TestPmdPort.parse(output) + def show_port_stats_all(self) -> list[TestPmdPortStats]: + """Returns the statistics of all the ports.""" + output = self.send_command("show port stats all") + + # Sample output of the "all" command looks like: + # + # ########### NIC statistics for port 0 ########### + # values... + # ################################################# + # + # ########### NIC statistics for port 1 ########### + # values... + # ################################################# + # + iter = re.finditer(r"(^ #*.+#*$[^#]+)^ #*$", output, re.MULTILINE) + return [TestPmdPortStats.parse(block.group(1)) for block in iter] + + def show_port_stats(self, port_id: int) -> TestPmdPortStats: + """Returns the given port statistics. + + Args: + port_id: The port ID to gather information for. + + Raises: + InteractiveCommandExecutionError: If `port_id` is invalid. + """ + output = self.send_command(f"show port stats {port_id}", skip_first_line=True) + if output.startswith("Invalid port"): + raise InteractiveCommandExecutionError("invalid port given") + + return TestPmdPortStats.parse(output) + def close(self) -> None: """Overrides :meth:`~.interactive_shell.close`.""" self.send_command("quit", "")