From patchwork Mon Aug 9 11:24:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Owen Hilyard X-Patchwork-Id: 96726 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E5F0BA0C4C; Mon, 9 Aug 2021 13:24:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BF9874003D; Mon, 9 Aug 2021 13:24:08 +0200 (CEST) Received: from mail-oi1-f228.google.com (mail-oi1-f228.google.com [209.85.167.228]) by mails.dpdk.org (Postfix) with ESMTP id 1D91A4003C for ; Mon, 9 Aug 2021 13:24:06 +0200 (CEST) Received: by mail-oi1-f228.google.com with SMTP id h11so7761373oie.0 for ; Mon, 09 Aug 2021 04:24:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=RHPQueSVEZgJ+RhWvCNE8wh1qmtkboPxW6k895Gheg0=; b=BxB4R+rjxuK6fRLq2lq4Cez1dX+dP1WkrwQB5tdUYX4iD3eSGUlCXY+kBURz2sNx7l NInGFsn5yWz6VV+Jinr/sTrcfy3NO3kbDnfEyw4/gNaqR4WQLtglqnLvcsQhsicchLg3 3jTJRpZqETUMzSIx/ZBPWbctVFd+w6gaPaZLE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=RHPQueSVEZgJ+RhWvCNE8wh1qmtkboPxW6k895Gheg0=; b=hpx+WiCTU2lOO+hT4RvPAi334J09iUDAA6EI+zn+gtJUVPSRrMU7zOo6vU/xyPvZ/F LIfQQqtAapEaE8aBj4rODNzc4edUSiPULMU4FEE3NsCdgNsTrL4UiG5IcLlAy2q8aRAZ GbJ5/gzW15OwiMkREqJPYOCQ2wKLtMFCu9VdfMP/UCONxIGanXCB7OUofBgWilhb78h0 jNWE9jFJy9xhWJLchClbHe727ZPwp+17Y+xgIVWKC9Dn7Wp6Py5lvyA61L78hfRUSejO UsPF0YQZVtRryTjz4d+zx/sHrx19HuMAV2M0Ndzga+eJNT+Puig/IXD3LDeMmL4vSzkF Q4WA== X-Gm-Message-State: AOAM531A7CqxyoNDRBhssXkF8ojqaJXTvePC+DcFr1PZ74iiw32LbYFo +T+O82sgkWtttroHGeASw2kwlyoZPKITVOQ5VLfol6uBSToGhf+BhXg4NBvMfDDBxHhGzrTnryF UR99+jwo0eeuwTReo52KNvsCZ7O651wlR6vNn6wB6nUgGq7EJbS2tq7LQqK201vyK7DoKyV16Ed 4AgXMbqNaj2XuMyg== X-Google-Smtp-Source: ABdhPJxTXRpGT7xFhCoyrLJPiZmYbilZvyGAIjJSNtpVlRzZaxHZ0h9obOS7n+pA4BzCOkg4NHspTxttO4ml X-Received: by 2002:a05:6808:3c7:: with SMTP id o7mr23928871oie.115.1628508246262; Mon, 09 Aug 2021 04:24:06 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id l17sm3696536otu.9.2021.08.09.04.24.06 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Aug 2021 04:24:06 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1220:6f7f:5992:405a:bfd7]) by postal.iol.unh.edu (Postfix) with ESMTP id A64C5605248B; Mon, 9 Aug 2021 07:24:05 -0400 (EDT) From: ohilyard@iol.unh.edu To: dts@dpdk.org Cc: lijuan.tu@intel.com, Owen Hilyard Date: Mon, 9 Aug 2021 07:24:02 -0400 Message-Id: <20210809112404.7199-1-ohilyard@iol.unh.edu> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Subject: [dts] [PATCH 1/3] framework/test_result: Fix circular import X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" From: Owen Hilyard This circular import is normally fine because the debugger module is initialized before test_case is imported, but since that is not necessarily the case with the dependency script, the circular import needed to be removed. Signed-off-by: Owen Hilyard --- framework/test_case.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/framework/test_case.py b/framework/test_case.py index 98b716b9..3d7bc30e 100644 --- a/framework/test_case.py +++ b/framework/test_case.py @@ -33,7 +33,6 @@ A base class for creating DTF test cases. """ import re -import debugger import traceback import signal import time @@ -374,6 +373,10 @@ class TestCase(object): """ Execute all test cases in one suite. """ + + # local import to avoid circular import + import debugger + # prepare debugger rerun case environment if self._enable_debug or self._debug_case: debugger.AliveSuite = self From patchwork Mon Aug 9 11:24:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Owen Hilyard X-Patchwork-Id: 96728 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 918C8A0C53; Mon, 9 Aug 2021 13:24:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 32FFE410F6; Mon, 9 Aug 2021 13:24:09 +0200 (CEST) Received: from mail-pj1-f99.google.com (mail-pj1-f99.google.com [209.85.216.99]) by mails.dpdk.org (Postfix) with ESMTP id 119324003C for ; Mon, 9 Aug 2021 13:24:08 +0200 (CEST) Received: by mail-pj1-f99.google.com with SMTP id m24-20020a17090a7f98b0290178b1a81700so3524279pjl.4 for ; Mon, 09 Aug 2021 04:24:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=O6l8TXMmofBCMy24pXqjuBVQbtoBuBwbmH1JREBVG8A=; b=YI7hcHXIY1M8nxgOkHOK0+8kWINl4WJKXuoZXNsHr2KBnqWKKsnYds3F+/0AfUsK8v adKzjxXL90vD1A/Swe9sXCzdWffEn6fHS+ni9y6sBjbpLi7sTCYHdCO9M/rXLATe08X8 yMmESNU+fFFBlGhg0bhLlryQHcuvZ7jmuSloY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=O6l8TXMmofBCMy24pXqjuBVQbtoBuBwbmH1JREBVG8A=; b=eSkXpGvpBDnVIaGLOWo1UCFemw1lxw1faeLVhALjrAqqwWiVxgYGVnBYapmUyeZBvh da/0aYaNnBpUtZBdmHVMjn0tG/E3bJucbB4bHkfZiuvtdxGI1d2anboFewwjs2m0ZILh eC+EAQ58pcWSbcC+3SJqpj+oT252WqgInSqmymON8QeIVWOLpamKrAuBK3yI4nUTMAhb gZOnvcugwhV15j9zZf0vB456Wtx9Sc48jjfYmmENb07yz6chGuYKLrTnkXPJ4zjclytX yTlt6pSEGkWqDNEyobpOJ6YnG8z7LcJkLIfx0+XQ0v8g1TZi+gp6Zw5XaUU7Jrix6jQE bAeQ== X-Gm-Message-State: AOAM531OBXhwD5rTTJeavCbWcl8TTM2PTa0VZXFwIWf/VJDNjpcBQ9dR AlFF8BmTMzVb3vG0o3Hkn2wgmei5KEF8/Kw7TDXxClfVRlfOv5cgQ4gip0e45/8Is6N4j/QsF1F rhhipi9+NszGnQJ33kiFt8U+n2bhG7JmxZqvmP/1uxMYtfi/HX8/EkYNPs4SdLUlXRJaomX3xjp 1iMcqlk0hOMaAdiA== X-Google-Smtp-Source: ABdhPJzmfxPFvExMdbX9fAzL8v1PV5X6o+jOuzF58U+ueY0WbEWpCJkz0VVdOyDkNagMkX22IijKaU5GoEHj X-Received: by 2002:a62:7dcb:0:b029:3af:8a46:a7e4 with SMTP id y194-20020a627dcb0000b02903af8a46a7e4mr18245369pfc.20.1628508247148; Mon, 09 Aug 2021 04:24:07 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id o10sm1798126pjt.6.2021.08.09.04.24.06 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Aug 2021 04:24:07 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1220:6f7f:5992:405a:bfd7]) by postal.iol.unh.edu (Postfix) with ESMTP id 349F4605248E; Mon, 9 Aug 2021 07:24:06 -0400 (EDT) From: ohilyard@iol.unh.edu To: dts@dpdk.org Cc: lijuan.tu@intel.com, Owen Hilyard Date: Mon, 9 Aug 2021 07:24:03 -0400 Message-Id: <20210809112404.7199-2-ohilyard@iol.unh.edu> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210809112404.7199-1-ohilyard@iol.unh.edu> References: <20210809112404.7199-1-ohilyard@iol.unh.edu> MIME-Version: 1.0 Subject: [dts] [PATCH 2/3] requirements: Added required modules to requirements X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" From: Owen Hilyard Since the dependency script imports every module it looks at, it found a few requirements that are not listed. Signed-off-by: Owen Hilyard --- requirements.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/requirements.txt b/requirements.txt index fae0ace8..d88ccd26 100644 --- a/requirements.txt +++ b/requirements.txt @@ -38,3 +38,6 @@ pcapy xlrd scapy==2.4.4 threadpool +matplotlib +gitpython +pydes \ No newline at end of file From patchwork Mon Aug 9 11:24:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Owen Hilyard X-Patchwork-Id: 96727 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31843A0C52; Mon, 9 Aug 2021 13:24:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ED118410EA; Mon, 9 Aug 2021 13:24:08 +0200 (CEST) Received: from mail-oi1-f228.google.com (mail-oi1-f228.google.com [209.85.167.228]) by mails.dpdk.org (Postfix) with ESMTP id BCD6C4003C for ; Mon, 9 Aug 2021 13:24:07 +0200 (CEST) Received: by mail-oi1-f228.google.com with SMTP id bj40so3395107oib.6 for ; Mon, 09 Aug 2021 04:24:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=APljMrA4kldZJGTWw8PUBmsuzO8IYoUibQF/O2l9pYI=; b=MyVU70lpzKHa3+bdE3T1AGLqPkPlFHr93Pxr5w8jhU2/RJTqrQ57/aqRCo/6SFBFSK ugb1Bo9YWlzACUTFhFM5679QxMyZ2WA/mwOUvWs/rCO5ECyeUfKO+Wm1HlKfmyAkPxlk LYic6I0ItL4KfB+3nj1+IAcHENv+s7fdN+MPE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=APljMrA4kldZJGTWw8PUBmsuzO8IYoUibQF/O2l9pYI=; b=s29olxCkaWHMYneGds7yuoAfwBb4DWKjQKkV2YGKKPbT6okpIf8HFEY/LwN8mQ8kp9 rIBfNR3zsVwugV8HZ8z6nJd5CZTLdc9bQGKL/4Qex1A3CuXCPMhksugjC5n8kGYyzy4d 1Aad0lEAKOqltBqyIXI9jXSLgHyU5gqxs9bWEarwlm/1mDGzb3A2KSFW3vZfjCjj83nG 7skDPyMDClfs/HhOrCnqbkNUajxI9dUpWa182EqWMY7LxLlGSogzgf8U+yIPcg1hy7tQ ocezgFOSm5yLbvWqdqZMea406RqDmJDQB9a51YiQDZSsMGqYp8sy0+YDRhA65CoPuIK/ eKvg== X-Gm-Message-State: AOAM530VUt48gJSQj+RcvBlKzjk/sVXCqrvqHVnHPsbSz6aG2UKYqw6G cFzhnSdKg2XS0VORhB2IMgAAQ23hTMKONQ/f/aGecVYf9v+khnwXFiEIXXGBFXk+11RUpSE75RX VruWuxuxzVAK+P6U9N7kili8NgzEx+cCl/R4YSybZ4IicQlh363CY4Y6mTZ8Ru1jj/ODSJCIKW8 f5KwZvMBZYZEx3uw== X-Google-Smtp-Source: ABdhPJzEvPcY71TeJ2WgA/BNeSoNdNsarl5JYgjcKNJO5lg82Cy813NYDySP60pNZf/+91adGyvrMTHBdn1C X-Received: by 2002:aca:c204:: with SMTP id s4mr23224109oif.49.1628508247196; Mon, 09 Aug 2021 04:24:07 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id d8sm1950311ooo.1.2021.08.09.04.24.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Aug 2021 04:24:07 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1220:6f7f:5992:405a:bfd7]) by postal.iol.unh.edu (Postfix) with ESMTP id 9E360605248B; Mon, 9 Aug 2021 07:24:06 -0400 (EDT) From: ohilyard@iol.unh.edu To: dts@dpdk.org Cc: lijuan.tu@intel.com, Owen Hilyard Date: Mon, 9 Aug 2021 07:24:04 -0400 Message-Id: <20210809112404.7199-3-ohilyard@iol.unh.edu> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210809112404.7199-1-ohilyard@iol.unh.edu> References: <20210809112404.7199-1-ohilyard@iol.unh.edu> MIME-Version: 1.0 Subject: [dts] [PATCH 3/3] ci/initial: Added script to get the tests for a patchset X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" From: Owen Hilyard This script should be run after the patchset has been applied. It will check all files that have a diff to the the git ref in DTS_MAIN_BRANCH_REF (currently origin/master). It will also issue warnings to standard error if a "protected path" is changed. This is currently configured to only by the ci scripts folder, since under most circumstances a patch should not need to change anything in there. This warning will be in the format: "WARNING: {file_name} is protected" The script will also issue a warning if a config file is changed. This warning is also sent to standard error and takes the form of: "WARNING: {file_name} is a config file and was changed" The script will output a list of the test suites to run to standard out, with each entry having one line. Signed-off-by: Owen Hilyard --- ci/README.txt | 37 +++++++ ci/get_tests_for_patchset.py | 195 +++++++++++++++++++++++++++++++++++ ci/requirements.txt | 33 ++++++ 3 files changed, 265 insertions(+) create mode 100644 ci/README.txt create mode 100644 ci/get_tests_for_patchset.py create mode 100644 ci/requirements.txt diff --git a/ci/README.txt b/ci/README.txt new file mode 100644 index 00000000..281329f7 --- /dev/null +++ b/ci/README.txt @@ -0,0 +1,37 @@ +# BSD LICENSE +# +# Copyright(c) 2021 University of New Hampshire Interoperability Laboratory. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +This directory contains scripts used for ci in DTS. Nothing in this directory +should be modified during the course of submitting a patch to DPDK. + +Additional python requirements only needed for running DTS in ci are present in the +requirements.txt file in this directory. + diff --git a/ci/get_tests_for_patchset.py b/ci/get_tests_for_patchset.py new file mode 100644 index 00000000..fdcf2371 --- /dev/null +++ b/ci/get_tests_for_patchset.py @@ -0,0 +1,195 @@ +# BSD LICENSE +# +# Copyright(c) 2021 University of New Hampshire Interoperability Laboratory. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +import argparse +import os +import pkgutil +import re +import subprocess +import sys +from pkgutil import walk_packages +from types import ModuleType +from typing import List, Iterable + +DTS_MAIN_BRANCH_REF: str = "origin/master" + +DTS_TEST_MODULE_PATH: str = "tests" +# Will be unnecessary once DTS moves to a normal module structure +DTS_MODULE_PATHS: List[str] = ["framework", "nic", "dep", DTS_TEST_MODULE_PATH] + +# This is primarily intended for folders which contain CI scripts, as these +# should not be modified in the course of normal CI. A file in any of these +# folders being modified will cause a warning to be emitted to standard error +DTS_PROTECTED_PATHS: List[str] = [ + "ci", +] + +# This is intended to help detect when a config files have been changed. +# The intention behind this detection is to enable additional regression +# tests related to ensuring stable config file formats +DTS_CONFIG_PATHS: List[str] = [ + "conf", + "execution.cfg", +] + + +def get_args() -> str: + parser: argparse.ArgumentParser = argparse.ArgumentParser( + description="After a patchset is applied, run this script" + "It will then output a list " + "of applicable test suites to standard output, with 1 " + "test suite per line. All other output, such as warnings," + " errors or informational messages will be sent to " + "standard error. This script may produce no output at all," + "in which case it should be assumed that there are no " + "applicable test suites.\n\n " + "Exit Codes:\n" + "value | meaning\n" + "----- | -------\n" + " 0 | OK \n" + " 1 | Error, check standard error", + ) + + dts_directory: str = os.path.dirname(os.path.dirname(os.path.join(os.getcwd(), __file__))) + parser.add_argument("-d", "--dts-directory", type=str, default=dts_directory, required=False) + args = parser.parse_args() + if not os.path.isdir(args.dts_directory): + print(f"{args.dts_directory} is not a directory.", file=sys.stderr) + exit(1) + + return args.dts_directory + + +def get_modified_files() -> List[str]: + cmd = ['git', 'diff', '--name-only', DTS_MAIN_BRANCH_REF, 'HEAD'] + process: subprocess.CompletedProcess = subprocess.run(cmd, capture_output=True) + if process.returncode != 0: + print(f"{' '.join(cmd)} returned {process.returncode}") + exit(1) + + # This explicit conversion to utf8 will catch encoding errors + stdout: bytes = process.stdout + stdout_str: str = stdout.decode("utf-8") + + return stdout_str.splitlines() + + +def get_names_of_modified_python_files(files: List[str]) -> List[str]: + return list( + map( + lambda f: str(re.sub("\\.py", "", f)), + map( + lambda f: os.path.basename(f), + filter( + lambda f: f.endswith(".py"), files + ) + ) + ) + ) + + +def get_modules() -> List[ModuleType]: + return list(map(lambda m: pkgutil.resolve_name(m.name), walk_packages(DTS_MODULE_PATHS))) + + +def get_module_imports(mod: ModuleType) -> Iterable[ModuleType]: + imports = [] + for attribute_label in dir(mod): + try: + attribute = getattr(mod, attribute_label) + except ModuleNotFoundError as _: # some standard library modules don't like being directly imported + continue + if isinstance(attribute, type(mod)): + imports.append(attribute) + return imports + + +def get_modules_in_tree(mod: ModuleType) -> Iterable[ModuleType]: + yield from get_module_imports(mod) + + +def get_only_test_suites(modules: Iterable[ModuleType]) -> Iterable[ModuleType]: + test_package_path = os.path.join(os.getcwd(), DTS_TEST_MODULE_PATH) + mod: ModuleType + return filter(lambda mod: mod.__name__.startswith("TestSuite_"), + filter(lambda mod: mod.__file__.startswith(test_package_path), modules)) + + +def get_test_suite_names(modules: Iterable[ModuleType]) -> Iterable[str]: + # Moving this into a set is there to ensure that there are no duplicates + return set(list(map(lambda mod: re.sub("TestSuite_", "", mod.__name__), modules))) + + +def get_tests_to_run() -> List[str]: + dts_directory: str + dts_directory = get_args() + + # This all needs to be done at the top level, so I have to do it here. + + # chdir to the DTS directory, since we want that to be + # the context for any commands that are run. + os.chdir(dts_directory) + + for path in DTS_MODULE_PATHS: + sys.path.append(os.path.join(dts_directory, path)) + + files: List[str] = get_modified_files() + changed_module_name = get_names_of_modified_python_files(files) + + for protected_folder in DTS_PROTECTED_PATHS: + for file_name in files: + if file_name.startswith(protected_folder): + print(f"WARNING: {file_name} is protected", file=sys.stderr) + + for config_file_path in DTS_CONFIG_PATHS: + for file_name in files: + if file_name.startswith(config_file_path): + print(f"WARNING: {file_name} is a config file and was changed", file=sys.stderr) + + # Each index is 1 level of the tree + module_list: List[ModuleType] = [pkgutil.resolve_name(name) for name in changed_module_name] + current_index: int = 0 + while current_index < len(module_list) and len(module_list) > 0: + mod = module_list[current_index] + if module_list.count(mod) == 1: + module_list = module_list + list(get_modules_in_tree(mod)) + current_index += 1 + + test_suites_to_run: List[str] = list(get_test_suite_names(get_only_test_suites(module_list))) + test_suites_to_run.sort() + return test_suites_to_run + +def main(): + test_suites_to_run = get_tests_to_run() + print("\n".join(test_suites_to_run)) + + +if __name__ == "__main__": + main() diff --git a/ci/requirements.txt b/ci/requirements.txt new file mode 100644 index 00000000..92d57c30 --- /dev/null +++ b/ci/requirements.txt @@ -0,0 +1,33 @@ +# BSD LICENSE +# +# Copyright(c) 2021 University of New Hampshire Interoperability Laboratory. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +argparse==1.4.0 +python>=3.8.0 \ No newline at end of file