diff mbox

[dpdk-dev,2/2] app/test: filter out unavailable tests

Message ID 1468445050-1081-2-git-send-email-thomas.monjalon@6wind.com (mailing list archive)
State Accepted, archived
Headers

Commit Message

Thomas Monjalon July 13, 2016, 9:24 p.m. UTC 
  Some tests can fail to run because they are not compiled.
It has been more visible recently when the PCI test became disabled
in the default configuration because of dependency on libarchive:
    PCI autotest:    Fail [Not found]

The autotest script catch them and do not count them as an error anymore:
    PCI autotest:    Skipped [Not Available]

The commands dump_ring and dump_mempool are removed from the autotest list
because they are some internal commands but not some registered tests.
Thus they cannot be parsed as available test commands.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 app/test/autotest_data.py   | 12 ------------
 app/test/autotest_runner.py | 12 ++++++++++--
 2 files changed, 10 insertions(+), 14 deletions(-)

Comments

David Marchand July 15, 2016, 2:23 p.m. UTC | #1 
On Wed, Jul 13, 2016 at 11:24 PM, Thomas Monjalon
<thomas.monjalon@6wind.com> wrote:
> Some tests can fail to run because they are not compiled.
> It has been more visible recently when the PCI test became disabled
> in the default configuration because of dependency on libarchive:
>     PCI autotest:    Fail [Not found]
>
> The autotest script catch them and do not count them as an error anymore:
>     PCI autotest:    Skipped [Not Available]
>
> The commands dump_ring and dump_mempool are removed from the autotest list
> because they are some internal commands but not some registered tests.
> Thus they cannot be parsed as available test commands.
>
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>

Suggested-by: David Marchand <david.marchand@6wind.com>

and I trust you for the python ;-)
Thomas Monjalon July 15, 2016, 2:30 p.m. UTC | #2 
2016-07-15 16:23, David Marchand:
> On Wed, Jul 13, 2016 at 11:24 PM, Thomas Monjalon
> <thomas.monjalon@6wind.com> wrote:
> > Some tests can fail to run because they are not compiled.
> > It has been more visible recently when the PCI test became disabled
> > in the default configuration because of dependency on libarchive:
> >     PCI autotest:    Fail [Not found]
> >
> > The autotest script catch them and do not count them as an error anymore:
> >     PCI autotest:    Skipped [Not Available]
> >
> > The commands dump_ring and dump_mempool are removed from the autotest list
> > because they are some internal commands but not some registered tests.
> > Thus they cannot be parsed as available test commands.
> >
> > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> 
> Suggested-by: David Marchand <david.marchand@6wind.com>

Thanks for giving me the idea of the nm trick.

> and I trust you for the python ;-)

You should not ;)
Thomas Monjalon July 15, 2016, 3:25 p.m. UTC | #3 
> > Some tests can fail to run because they are not compiled.
> > It has been more visible recently when the PCI test became disabled
> > in the default configuration because of dependency on libarchive:
> >     PCI autotest:    Fail [Not found]
> >
> > The autotest script catch them and do not count them as an error anymore:
> >     PCI autotest:    Skipped [Not Available]
> >
> > The commands dump_ring and dump_mempool are removed from the autotest list
> > because they are some internal commands but not some registered tests.
> > Thus they cannot be parsed as available test commands.
> >
> > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> 
> Suggested-by: David Marchand <david.marchand@6wind.com>

Series applied
diff mbox

Patch

diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py
index 1e6b422..c69705e 100644
--- a/app/test/autotest_data.py
+++ b/app/test/autotest_data.py
@@ -99,18 +99,6 @@  parallel_test_group_list = [
 		 "Func" :	default_autotest,
 		 "Report" :	None,
 		},
-		{
-		 "Name" :	"Dump rings",
-		 "Command" :	"dump_ring",
-		 "Func" :	dump_autotest,
-		 "Report" :	None,
-		},
-		{
-		 "Name" :	"Dump mempools",
-		 "Command" :	"dump_mempool",
-		 "Func" :	dump_autotest,
-		 "Report" :	None,
-		},
 	]
 },
 {
diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
index 291a821..bd99e19 100644
--- a/app/test/autotest_runner.py
+++ b/app/test/autotest_runner.py
@@ -33,7 +33,7 @@ 
 
 # The main logic behind running autotests in parallel
 
-import multiprocessing, sys, pexpect, time, os, StringIO, csv
+import multiprocessing, subprocess, sys, pexpect, re, time, os, StringIO, csv
 
 # wait for prompt
 def wait_prompt(child):
@@ -105,6 +105,11 @@  def run_test_group(cmdline, test_group):
 	results.append((0, "Success", "Start %s" % test_group["Prefix"],
 		time.time() - start_time, startuplog.getvalue(), None))
 
+	# parse the binary for available test commands
+	binary = cmdline.split()[0]
+	symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+	avail_cmds = re.findall('test_register_(\w+)', symbols)
+
 	# run all tests in test group
 	for test in test_group["Tests"]:
 
@@ -124,7 +129,10 @@  def run_test_group(cmdline, test_group):
 			print >>logfile, "\n%s %s\n" % ("-"*20, test["Name"])
 
 			# run test function associated with the test
-			result = test["Func"](child, test["Command"])
+			if test["Command"] in avail_cmds:
+				result = test["Func"](child, test["Command"])
+			else:
+				result = (0, "Skipped [Not Available]")
 
 			# make a note when the test was finished
 			end_time = time.time()