From patchwork Fri Sep 15 11:52:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 131504 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1D809425A4; Fri, 15 Sep 2023 13:52:32 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 608AB406B4; Fri, 15 Sep 2023 13:52:30 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 4C3C4402D7 for ; Fri, 15 Sep 2023 13:52:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1694778749; x=1726314749; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hOqMFsmY06YtN35eu+oMgcYWLx6apd8xyqN04h6MokM=; b=iAFYbcKqMXc2ckpWUQZVHM5uA7s1GWM+rSUp9cTQrVVYwe3+xiN8Lcce qUq/TRJ5BqI5E7XjPLPZ6e8ZNy9gDFmWUjlipOdRTY35eOxfraBjdm66P jiU7SDuC/XgeffoH9UZja81RpJeNuIpVdwsX4rT8Rr+SYyrnzvs1S0sjS fb6y7kcr/xLQBNkKrjeRRq1mjgXs8PZL7Yl+kLltupUNuEShDjSI9LLQa iNDTyIs4e9RUXkd7o3+/fYasxQwr3F1Vw9FNv0zJjwmTFmeqK+7PYD5jy Mgm/PAd19uhxFDoYyi1ziWJago/LMCe1ngAUNEXVQO6wQuP/xWOHEznE6 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="379149155" X-IronPort-AV: E=Sophos;i="6.02,149,1688454000"; d="scan'208";a="379149155" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2023 04:52:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="918631210" X-IronPort-AV: E=Sophos;i="6.02,149,1688454000"; d="scan'208";a="918631210" Received: from silpixa00401385.ir.intel.com ([10.237.214.14]) by orsmga005.jf.intel.com with ESMTP; 15 Sep 2023 04:52:14 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: david.marchand@redhat.com, Bruce Richardson Subject: [PATCH 1/2] app/test: emit warning for tests not in a test suite Date: Fri, 15 Sep 2023 12:52:05 +0100 Message-Id: <20230915115206.132198-2-bruce.richardson@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230915115206.132198-1-bruce.richardson@intel.com> References: <20230915115206.132198-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When doing a scan for tests to add to test suites, we can also detect the tests which have not been added to any suite. By doing so: a) we can emit a warning informing the developer that the test is not in a suite (if developer mode is enabled) b) we can still register a test for that command making these tests callable through "meson test" Signed-off-by: Bruce Richardson --- app/test/suites/meson.build | 13 ++++++++++++- buildtools/get-test-suites.py | 12 +++++++++--- 2 files changed, 21 insertions(+), 4 deletions(-) diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build index 19a6b902fa..478f245a54 100644 --- a/app/test/suites/meson.build +++ b/app/test/suites/meson.build @@ -21,7 +21,18 @@ foreach suite:test_suites suite = suite.split('=') suite_name = suite[0] suite_tests = suite[1].split(',') - if suite_name != 'fast-tests' + if suite_name == 'non_suite_tests' + # tests not in any suite + foreach t: suite_tests + if developer_mode + warning('Test "@0@" is not defined in any test suite'.format(t)) + endif + test(t, dpdk_test, + env: ['DPDK_TEST=' + t], + timeout: timeout_seconds, + is_parallel: false) + endforeach + elif suite_name != 'fast-tests' # simple cases - tests without parameters or special handling foreach t: suite_tests test(t, dpdk_test, diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py index 95a9cad4c8..574c233aa8 100644 --- a/buildtools/get-test-suites.py +++ b/buildtools/get-test-suites.py @@ -8,18 +8,23 @@ input_list = sys.argv[1:] test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)") test_suites = {} +# track tests not in any test suite. +non_suite_regex = re.compile("REGISTER_TEST_COMMAND\s*\(\s*([a-z0-9_]+)") +non_suite_tests = [] def get_fast_test_params(test_name, ln): "Extract the extra fast-test parameters from the line" - #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}") (_, rest_of_line) = ln.split(test_name, 1) (_, nohuge, asan, _func) = rest_of_line.split(',', 3) return f":{nohuge.strip().lower()}:{asan.strip().lower()}" for fname in input_list: with open(fname) as f: - contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())] - for ln in contents: + contents = [ln.strip() for ln in f.readlines()] + test_lines = [ln for ln in contents if test_def_regex.match(ln)] + non_suite_tests.extend([non_suite_regex.match(ln).group(1) + for ln in contents if non_suite_regex.match(ln)]) + for ln in test_lines: (test_suite, test_name) = test_def_regex.match(ln).group(1, 2) suite_name = f"{test_suite.lower()}-tests" if suite_name in test_suites: @@ -31,3 +36,4 @@ def get_fast_test_params(test_name, ln): for suite in test_suites.keys(): print(f"{suite}={','.join(test_suites[suite])}") +print(f"non_suite_tests={','.join(non_suite_tests)}") From patchwork Fri Sep 15 11:52:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 131505 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9C7E9425A4; Fri, 15 Sep 2023 13:52:42 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8D44D40A79; Fri, 15 Sep 2023 13:52:42 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 3CAC240A79 for ; Fri, 15 Sep 2023 13:52:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1694778761; x=1726314761; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JCdNnq7onBVdGuhlO85HdpsvppPo45mEiw/NXfWfU7Y=; b=PqUgNrR/NMVs5VbzCUASItRTIoi3WWVSlnfB/y3ZO+IOj902cLWXAqPR QF8ufKoTZzqTAixe4t5ctUTmftuSs6xKuVqyF6DKXkAit9v5qt8nmoFt1 AhjzGB3lOZuTbzJejg2WgHOsPv4BsdEgsGgJS44dGKKQpiys1y29f9nhN Z/hNE/eVSIzuu8txbA6XGBrR72IywvO1mrD/dprghvlmKlUNDFlKCsPUJ YC82pK98GAHNV+ISYfO7aUO69Amu3mamhcloYOAtvhUd+m2LQb6KaNjEb 2MPewedNy1mUXTRoAx0GaAFr1WV/27C4eFK5wq0Fz64HOqgAv1SMQePGC A==; X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="379149251" X-IronPort-AV: E=Sophos;i="6.02,149,1688454000"; d="scan'208";a="379149251" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2023 04:52:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="918631228" X-IronPort-AV: E=Sophos;i="6.02,149,1688454000"; d="scan'208";a="918631228" Received: from silpixa00401385.ir.intel.com ([10.237.214.14]) by orsmga005.jf.intel.com with ESMTP; 15 Sep 2023 04:52:24 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: david.marchand@redhat.com, Bruce Richardson Subject: [PATCH 2/2] devtools: check for tests added without a test suite Date: Fri, 15 Sep 2023 12:52:06 +0100 Message-Id: <20230915115206.132198-3-bruce.richardson@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230915115206.132198-1-bruce.richardson@intel.com> References: <20230915115206.132198-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add a check in checkpatches.sh for the use of the old REGISTER_TEST_COMMAND macro, which just adds a test without including it in a test suite. Suggest to the user to add the test using the newer macros which will include the test in a test suite. Signed-off-by: Bruce Richardson --- devtools/checkpatches.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/devtools/checkpatches.sh b/devtools/checkpatches.sh index 55fabc5458..9b201db0a2 100755 --- a/devtools/checkpatches.sh +++ b/devtools/checkpatches.sh @@ -159,6 +159,14 @@ check_forbidden_additions() { # -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1 + # prevent addition of tests not in one of our test suites + awk -v FOLDERS='app/test' \ + -v EXPRESSIONS='REGISTER_TEST_COMMAND' \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using REGISTER_TEST_COMMAND instead of REGISTER__TEST' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + # SVG must be included with wildcard extension to allow conversion awk -v FOLDERS='doc' \ -v EXPRESSIONS='::[[:space:]]*[^[:space:]]*\\.svg' \