From patchwork Thu Aug 31 10:06:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 130970 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 27DB041FDB; Thu, 31 Aug 2023 12:06:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 263AE4027B; Thu, 31 Aug 2023 12:06:57 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 82A7B40279 for ; Thu, 31 Aug 2023 12:06:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1693476415; x=1725012415; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5Fd3l/CEYyg1kkqnHyUrUbyUhmhudBXfiNgK+rcpgr4=; b=QCQzekeMQXz3+jQUP2m1E1A48WhlF9NXdozugbmOO1+/yp6uzH1Uj2Nc IWwdhOa58qlF+0TVzcKXe/Pw6i/Waoypj8kb8R/9vND1AVC2vSdVDa5JD 7lWpnPRnWjUDS/nDwP6ZjssfJAJ90aNnfaI0Okm6+CZ1kXd4q4ikMgsFP BoQSfIDe6840UzBLtzXYlnfhxzKKRLZ/mKMMFCJ/feSAOWMR0gykA2wWd MKkuZPnH6rjEtR6bhxkSUyygNJMFUNlpx2fUF/FdkdEUbKCCBbUmxY7Zo +eo1J3d9DsN4gRtHWcDb3QGe9Lgg70OFIbhNSnMzfF6T70wQuZW6Tbxll g==; X-IronPort-AV: E=McAfee;i="6600,9927,10818"; a="373290085" X-IronPort-AV: E=Sophos;i="6.02,216,1688454000"; d="scan'208";a="373290085" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2023 03:06:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10818"; a="774474526" X-IronPort-AV: E=Sophos;i="6.02,216,1688454000"; d="scan'208";a="774474526" Received: from silpixa00401385.ir.intel.com ([10.237.214.14]) by orsmga001.jf.intel.com with ESMTP; 31 Aug 2023 03:06:51 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Thomas Monjalon Subject: [PATCH v2] app/test: add support for skipping tests Date: Thu, 31 Aug 2023 11:06:42 +0100 Message-Id: <20230831100642.8729-1-bruce.richardson@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230817105851.501947-1-bruce.richardson@intel.com> References: <20230817105851.501947-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When called from automated tools, like meson test, it is often useful to skip tests in a test suite, without having to alter the test build. To do so, we add support for DPDK_TEST_SKIP environment variable, where one can provide a comma-separated list of tests. When the test binary is called to run one of the tests on the list via either cmdline parameter or environment variable (as done with meson test), the test will not actually be run, but will be reported skipped. Example run: $ DPDK_TEST_SKIP=dump_devargs,dump_ring meson test --suite=debug-tests ... 1/9 DPDK:debug-tests / dump_devargs SKIP 1.11s 2/9 DPDK:debug-tests / dump_log_types OK 1.06s 3/9 DPDK:debug-tests / dump_malloc_heaps OK 1.11s 4/9 DPDK:debug-tests / dump_malloc_stats OK 1.07s 5/9 DPDK:debug-tests / dump_mempool OK 1.11s 6/9 DPDK:debug-tests / dump_memzone OK 1.06s 7/9 DPDK:debug-tests / dump_physmem OK 1.13s 8/9 DPDK:debug-tests / dump_ring SKIP 1.04s 9/9 DPDK:debug-tests / dump_struct_sizes OK 1.10s Ok: 7 Expected Fail: 0 Fail: 0 Unexpected Pass: 0 Skipped: 2 Timeout: 0 Signed-off-by: Bruce Richardson Acked-by: Thomas Monjalon --- V2: Check return value from rte_strsplit, just in case. --- app/test/test.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/app/test/test.c b/app/test/test.c index fb073ff795..1143285af2 100644 --- a/app/test/test.c +++ b/app/test/test.c @@ -193,6 +193,25 @@ main(int argc, char **argv) if (test_count > 0) { char buf[1024]; + char *dpdk_test_skip = getenv("DPDK_TEST_SKIP"); + char *skip_tests[128] = {0}; + size_t n_skip_tests = 0; + + if (dpdk_test_skip != NULL && strlen(dpdk_test_skip) > 0){ + int split_ret; + char *dpdk_test_skip_cp = strdup(dpdk_test_skip); + if (dpdk_test_skip_cp == NULL) { + ret = -1; + goto out; + } + dpdk_test_skip = dpdk_test_skip_cp; + split_ret = rte_strsplit(dpdk_test_skip, strlen(dpdk_test_skip), + skip_tests, RTE_DIM(skip_tests), ','); + if (split_ret > 0) + n_skip_tests = split_ret; + else + free(dpdk_test_skip); + } cl = cmdline_new(main_ctx, "RTE>>", 0, 1); if (cl == NULL) { @@ -201,6 +220,15 @@ main(int argc, char **argv) } for (i = 0; i < test_count; i++) { + /* check if test is to be skipped */ + for (size_t j = 0; j < n_skip_tests; j++) { + if (strcmp(tests[i], skip_tests[j]) == 0) { + fprintf(stderr, "Skipping %s [DPDK_TEST_SKIP]\n", tests[i]); + ret = TEST_SKIPPED; + goto end_of_cmd; + } + } + snprintf(buf, sizeof(buf), "%s\n", tests[i]); if (cmdline_parse_check(cl, buf) < 0) { printf("Error: invalid test command: '%s'\n", tests[i]); @@ -211,9 +239,13 @@ main(int argc, char **argv) } else ret = last_test_result; +end_of_cmd: if (ret != 0) break; } + if (n_skip_tests > 0) + free(dpdk_test_skip); + cmdline_free(cl); goto out; } else {