Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/124471/?format=api
{ "id": 124471, "url": "http://patches.dpdk.org/api/patches/124471/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/patch/20230223152840.634183-11-juraj.linkes@pantheon.tech/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20230223152840.634183-11-juraj.linkes@pantheon.tech>", "list_archive_url": "https://inbox.dpdk.org/dev/20230223152840.634183-11-juraj.linkes@pantheon.tech", "date": "2023-02-23T15:28:40", "name": "[v5,10/10] doc: update DTS setup and test suite cookbook", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "27e03e0acbcd8d249bf1aacab8c53fee820fe069", "submitter": { "id": 1626, "url": "http://patches.dpdk.org/api/people/1626/?format=api", "name": "Juraj Linkeš", "email": "juraj.linkes@pantheon.tech" }, "delegate": { "id": 1, "url": "http://patches.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "http://patches.dpdk.org/project/dpdk/patch/20230223152840.634183-11-juraj.linkes@pantheon.tech/mbox/", "series": [ { "id": 27159, "url": "http://patches.dpdk.org/api/series/27159/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=27159", "date": "2023-02-23T15:28:30", "name": "dts: add hello world testcase", "version": 5, "mbox": "http://patches.dpdk.org/series/27159/mbox/" } ], "comments": "http://patches.dpdk.org/api/patches/124471/comments/", "check": "success", "checks": "http://patches.dpdk.org/api/patches/124471/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 8684A41D52;\n\tThu, 23 Feb 2023 16:30:11 +0100 (CET)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 0B560430A7;\n\tThu, 23 Feb 2023 16:29:05 +0100 (CET)", "from mail-ed1-f53.google.com (mail-ed1-f53.google.com\n [209.85.208.53]) by mails.dpdk.org (Postfix) with ESMTP id 8BF4540ED5\n for <dev@dpdk.org>; Thu, 23 Feb 2023 16:28:56 +0100 (CET)", "by mail-ed1-f53.google.com with SMTP id d30so2236499eda.4\n for <dev@dpdk.org>; Thu, 23 Feb 2023 07:28:56 -0800 (PST)", "from localhost.localdomain ([84.245.121.112])\n by smtp.gmail.com with ESMTPSA id\n r6-20020a50c006000000b004af6a8617ffsm1158892edb.46.2023.02.23.07.28.54\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Thu, 23 Feb 2023 07:28:55 -0800 (PST)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=pantheon-tech.20210112.gappssmtp.com; s=20210112;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:from:to:cc:subject:date\n :message-id:reply-to;\n bh=TtKGqXREbdV24BtUnuvxKsFeNtDkF+SAMUN6axw569Q=;\n b=jm4ldl9g+mmLJ15m7AkT9cPfkIKaniXQXVS2dPrvYqHJe/Zi+V2vxh/Mi+iGtFko6e\n B1sTFWNytfp+tC1FHzXQI5UyA6HitAj8biw5KJs6brcWYmvkxIXy3EBE0gVsXNKf2VLJ\n lwSIaSqwGiMyNRo5s4nBjX398tSPZBbWYTmslOYvqk7maRga22J2q/u1jmJRYZri+icL\n OVKka2EdgCHE4GmJl1PRxTf0c+gnHiq+xWD2nVXQJEHCymPS0TUSNWLoU73fZxT+8NAV\n 9yux1bY9LiTLAfI/uxsIOvEwDfxV4bf7DSpcGtw0jleSVBq0ieAzRbacUToDHSaFy5P5\n hHpQ==", "X-Google-DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20210112;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n :subject:date:message-id:reply-to;\n bh=TtKGqXREbdV24BtUnuvxKsFeNtDkF+SAMUN6axw569Q=;\n b=TTYG9CIoDlN6a3HxIEW42zHMmJ8gEnII499kviFq/tcAxkHtkkGDslkHYiicpPy8Xw\n Qh0kQ1Uk7vdOvbJRqd0Q77vtUa5jgFnf4ZP9BmTW1RN19VR4LWk/m5UaRbo7MiqSCtAF\n ylz1KbPj3DVa4ZHgx42czFu8Mn12gGM/NQ6k59KzZ+8fT0BN1pc0Th+YVnNTmQn/ceWJ\n YXG2ETQ+NGSK3bLGM001YkUm21fcrDAIHt1/SpjxDPgWLTr+BZ0YuP+N0orp76SPmpEN\n BDhHzJp5HBnIqPijgv4R8S8MUSISzjxOLr9xvN0JDB32f4Vr9re0WVAnw8XNwMNC6qvH\n CIgg==", "X-Gm-Message-State": "AO0yUKXoyTrde89ssFXcieqQoBhy0hwywDAvd68l67DcUcASEHR0C86T\n 8/0ruRtSXgOYtaasVk/+0rM+qKkGdN+/+DC6", "X-Google-Smtp-Source": "\n AK7set/rz9AxD+NfYTQcQmm8UifGw11Cg5N8OtMJEHGQOeYOdUuOc55UnKqllDfupj+4FTz7hI8NhA==", "X-Received": "by 2002:a17:906:d979:b0:8af:2107:6ce5 with SMTP id\n rp25-20020a170906d97900b008af21076ce5mr17851785ejb.35.1677166136201;\n Thu, 23 Feb 2023 07:28:56 -0800 (PST)", "From": "=?utf-8?q?Juraj_Linke=C5=A1?= <juraj.linkes@pantheon.tech>", "To": "thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, lijuan.tu@intel.com,\n bruce.richardson@intel.com, probb@iol.unh.edu", "Cc": "dev@dpdk.org, =?utf-8?q?Juraj_Linke=C5=A1?= <juraj.linkes@pantheon.tech>", "Subject": "[PATCH v5 10/10] doc: update DTS setup and test suite cookbook", "Date": "Thu, 23 Feb 2023 16:28:40 +0100", "Message-Id": "<20230223152840.634183-11-juraj.linkes@pantheon.tech>", "X-Mailer": "git-send-email 2.30.2", "In-Reply-To": "<20230223152840.634183-1-juraj.linkes@pantheon.tech>", "References": "<20230213152846.284191-1-juraj.linkes@pantheon.tech>\n <20230223152840.634183-1-juraj.linkes@pantheon.tech>", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=UTF-8", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Document how to configure and run DTS.\nAlso add documentation related to new features: SUT setup and a brief\ntest suite implementation cookbook.\n\nSigned-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>\n---\n doc/guides/tools/dts.rst | 165 ++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 163 insertions(+), 2 deletions(-)", "diff": "diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst\nindex daf54359ed..ebd6dceb6a 100644\n--- a/doc/guides/tools/dts.rst\n+++ b/doc/guides/tools/dts.rst\n@@ -1,5 +1,5 @@\n .. SPDX-License-Identifier: BSD-3-Clause\n- Copyright(c) 2022 PANTHEON.tech s.r.o.\n+ Copyright(c) 2022-2023 PANTHEON.tech s.r.o.\n \n DPDK Test Suite\n ===============\n@@ -56,7 +56,7 @@ DTS runtime environment or just plain DTS environment are used interchangeably.\n \n \n Setting up DTS environment\n---------------------------\n+~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n #. **Python Version**\n \n@@ -93,6 +93,167 @@ Setting up DTS environment\n poetry install\n poetry shell\n \n+#. **SSH Connection**\n+\n+ DTS uses Python pexpect for SSH connections between DTS environment and the other hosts.\n+ The pexpect implementation is a wrapper around the ssh command in the DTS environment.\n+ This means it'll use the SSH agent providing the ssh command and its keys.\n+\n+\n+Setting up System Under Test\n+----------------------------\n+\n+There are two areas that need to be set up on a System Under Test:\n+\n+#. **DPDK dependencies**\n+\n+ DPDK will be built and run on the SUT.\n+ Consult the Getting Started guides for the list of dependencies for each distribution.\n+\n+#. **Hardware dependencies**\n+\n+ Any hardware DPDK uses needs a proper driver\n+ and most OS distributions provide those, but the version may not be satisfactory.\n+ It's up to each user to install the driver they're interested in testing.\n+ The hardware also may also need firmware upgrades, which is also left at user discretion.\n+\n+#. **Hugepages**\n+\n+ There are two ways to configure hugepages:\n+\n+ * DTS configuration\n+\n+ You may specify the optional hugepage configuration in the DTS config file.\n+ If you do, DTS will take care of configuring hugepages,\n+ overwriting your current SUT hugepage configuration.\n+\n+ * System under test configuration\n+\n+ It's possible to use the hugepage configuration already present on the SUT.\n+ If you wish to do so, don't specify the hugepage configuration in the DTS config file.\n+\n+\n+Running DTS\n+-----------\n+\n+DTS needs to know which nodes to connect to and what hardware to use on those nodes.\n+Once that's configured, DTS needs a DPDK tarball and it's ready to run.\n+\n+Configuring DTS\n+~~~~~~~~~~~~~~~\n+\n+DTS configuration is split into nodes and executions and build targets within executions.\n+By default, DTS will try to use the ``dts/conf.yaml`` config file,\n+which is a template that illustrates what can be configured in DTS:\n+\n+ .. literalinclude:: ../../../dts/conf.yaml\n+ :language: yaml\n+ :start-at: executions:\n+\n+\n+The user must be root or any other user with prompt starting with ``#``.\n+The other fields are mostly self-explanatory\n+and documented in more detail in ``dts/framework/config/conf_yaml_schema.json``.\n+\n+DTS Execution\n+~~~~~~~~~~~~~\n+\n+DTS is run with ``main.py`` located in the ``dts`` directory after entering Poetry shell::\n+\n+ usage: main.py [-h] [--config-file CONFIG_FILE] [--output-dir OUTPUT_DIR] [-t TIMEOUT]\n+ [-v VERBOSE] [-s SKIP_SETUP] [--tarball TARBALL]\n+ [--compile-timeout COMPILE_TIMEOUT] [--test-cases TEST_CASES]\n+ [--re-run RE_RUN]\n+\n+ Run DPDK test suites. All options may be specified with the environment variables provided in\n+ brackets. Command line arguments have higher priority.\n+\n+ options:\n+ -h, --help show this help message and exit\n+ --config-file CONFIG_FILE\n+ [DTS_CFG_FILE] configuration file that describes the test cases, SUTs\n+ and targets. (default: conf.yaml)\n+ --output-dir OUTPUT_DIR, --output OUTPUT_DIR\n+ [DTS_OUTPUT_DIR] Output directory where dts logs and results are\n+ saved. (default: output)\n+ -t TIMEOUT, --timeout TIMEOUT\n+ [DTS_TIMEOUT] The default timeout for all DTS operations except for\n+ compiling DPDK. (default: 15)\n+ -v VERBOSE, --verbose VERBOSE\n+ [DTS_VERBOSE] Set to 'Y' to enable verbose output, logging all\n+ messages to the console. (default: N)\n+ -s SKIP_SETUP, --skip-setup SKIP_SETUP\n+ [DTS_SKIP_SETUP] Set to 'Y' to skip all setup steps on SUT and TG\n+ nodes. (default: N)\n+ --tarball TARBALL, --snapshot TARBALL\n+ [DTS_DPDK_TARBALL] Path to DPDK source code tarball which will be\n+ used in testing. (default: dpdk.tar.xz)\n+ --compile-timeout COMPILE_TIMEOUT\n+ [DTS_COMPILE_TIMEOUT] The timeout for compiling DPDK. (default: 1200)\n+ --test-cases TEST_CASES\n+ [DTS_TESTCASES] Comma-separated list of test cases to execute.\n+ Unknown test cases will be silently ignored. (default: )\n+ --re-run RE_RUN, --re_run RE_RUN\n+ [DTS_RERUN] Re-run each test case the specified amount of times if a\n+ test failure occurs (default: 0)\n+\n+\n+The brackets contain the names of environment variables that set the same thing.\n+The minimum DTS needs is a config file and a DPDK tarball.\n+You may pass those to DTS using the command line arguments or use the default paths.\n+\n+\n+DTS Results\n+~~~~~~~~~~~\n+\n+Results are stored in the output dir by default\n+which be changed with the ``--output-dir`` command line argument.\n+The results contain basic statistics of passed/failed test cases and DPDK version.\n+\n+\n+How To Write a Test Suite\n+-------------------------\n+\n+All test suites inherit from ``TestSuite`` defined in ``dts/framework/test_suite.py``.\n+There are four types of methods that comprise a test suite:\n+\n+#. **Test cases**\n+\n+ | Test cases are methods that start with a particular prefix.\n+ | Functional test cases start with ``test_``, e.g. ``test_hello_world_single_core``.\n+ | Performance test cases start with ``test_perf_``, e.g. ``test_perf_nic_single_core``.\n+ | A test suite may have any number of functional and/or performance test cases.\n+ However, these test cases must test the same feature,\n+ following the rule of one feature = one test suite.\n+ Test cases for one feature don't need to be grouped in just one test suite, though.\n+ If the feature requires many testing scenarios to cover,\n+ the test cases would be better off spread over multiple test suites\n+ so that each test suite doesn't take too long to execute.\n+\n+#. **Setup and Teardown methods**\n+\n+ | There are setup and teardown methods for the whole test suite and each individual test case.\n+ | Methods ``set_up_suite`` and ``tear_down_suite`` will be executed\n+ before any and after all test cases have been executed, respectively.\n+ | Methods ``set_up_test_case`` and ``tear_down_test_case`` will be executed\n+ before and after each test case, respectively.\n+ | These methods don't need to be implemented if there's no need for them in a test suite.\n+ In that case, nothing will happen when they're is executed.\n+\n+#. **Test case verification**\n+\n+ Test case verification should be done with the ``verify`` method, which records the result.\n+ The method should be called at the end of each test case.\n+\n+#. **Other methods**\n+\n+ Of course, all test suite code should adhere to coding standards.\n+ Only the above methods will be treated specially and any other methods may be defined\n+ (which should be mostly private methods needed by each particular test suite).\n+ Any specific features (such as NIC configuration) required by a test suite\n+ should be implemented in the ``SutNode`` class (and the underlying classes that ``SutNode`` uses)\n+ and used by the test suite via the ``sut_node`` field.\n+\n \n DTS Developer Tools\n -------------------\n", "prefixes": [ "v5", "10/10" ] }