diff mbox series

[v2,08/10] autotest: add new test cases to autotest

Message ID 1531498808-21940-9-git-send-email-reshma.pattan@intel.com (mailing list archive)
State Superseded, archived
Headers
Series Make unit tests great again |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Pattan, Reshma July 13, 2018, 4:20 p.m. UTC 
  Autotest is enhanced with additional test cases
being added to autotest_data.py

Cc: stable@dpdk.org

Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
---
 test/test/autotest_data.py | 370 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 362 insertions(+), 8 deletions(-)

Comments

Burakov, Anatoly July 13, 2018, 4:41 p.m. UTC | #1 
On 13-Jul-18 5:20 PM, Reshma Pattan wrote:
> Autotest is enhanced with additional test cases
> being added to autotest_data.py

You're also removing PCI autotest - commit message needs to call this 
out as well.

> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
> Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
> ---
>   test/test/autotest_data.py | 370 ++++++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 362 insertions(+), 8 deletions(-)
> 
> diff --git a/test/test/autotest_data.py b/test/test/autotest_data.py

<snip>

> +    {
> +        "Name":    "Dump_ring",
> +        "Command": "dump_ring",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Quit",
> +        "Command": "quit",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },

Why is "quit" an autotest? If you want to test "quit" functionality, i 
would suggest putting it at the end of non-parallel tests, to avoid 
situation where test prematurely stops (for example, this will happen on 
FreeBSD, where there's only one thread executing these tests).

But really, having it in this list is IMO dubious.

> +    {
> +        "Name":    "Set rxtx mode",
> +        "Command": "set_rxtx_mode",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Set rxtx anchor",
> +        "Command": "set_rxtx_anchor",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Set rxtx sc",
> +        "Command": "set_rxtx_sc",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Event eth rx adapter autotest",
> +        "Command": "event_eth_rx_adapter_autotest",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Rawdev autotest",
> +        "Command": "rawdev_autotest",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Kvargs autotest",
> +        "Command": "kvargs_autotest",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Dump physmem",
> +        "Command": "dump_physmem",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Dump memzone",
> +        "Command": "dump_memzone",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },

Here and in other "dump" autotests - do these dump tests work as 
autotests? Default autotest function expects a "Test OK" at the end of 
test, and i don't think dump autotests return that, so this will 
(should?) cause a very long timeout and a test failure. These should 
have "dump_autotest" as "Func" value - that function correctly parses 
output of dump autotests (or, to be more accurate, it doesn't - as long 
as test doesn't crash, it's considered to be successful :) ).

Also, for readability, we should move these dump tests together and have 
all dump tests one after the other.
Pattan, Reshma July 16, 2018, 1:29 p.m. UTC | #2 
Hi,

> -----Original Message-----
> From: Burakov, Anatoly
> Sent: Friday, July 13, 2018 5:41 PM
> To: Pattan, Reshma <reshma.pattan@intel.com>; thomas@monjalon.net;
> dev@dpdk.org
> Cc: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>;
> stable@dpdk.org
> Subject: Re: [PATCH v2 08/10] autotest: add new test cases to autotest
> 
> On 13-Jul-18 5:20 PM, Reshma Pattan wrote:
> > Autotest is enhanced with additional test cases being added to
> > autotest_data.py
> 
> You're also removing PCI autotest - commit message needs to call this out as
> well.
> 

OK

> <snip>
> 
> > +    {
> > +        "Name":    "Dump_ring",
> > +        "Command": "dump_ring",
> > +        "Func":    default_autotest,
> > +        "Report":  None,
> > +    },
> > +    {
> > +        "Name":    "Quit",
> > +        "Command": "quit",
> > +        "Func":    default_autotest,
> > +        "Report":  None,
> > +    },
> 
> Why is "quit" an autotest? If you want to test "quit" functionality, i would
> suggest putting it at the end of non-parallel tests, to avoid situation where
> test prematurely stops (for example, this will happen on FreeBSD, where
> there's only one thread executing these tests).
> 
> But really, having it in this list is IMO dubious.
> 
Ok, I will remove this 

> Here and in other "dump" autotests - do these dump tests work as autotests?
> Default autotest function expects a "Test OK" at the end of test, and i don't
> think dump autotests return that, so this will
> (should?) cause a very long timeout and a test failure. These should have
> "dump_autotest" as "Func" value - that function correctly parses output of
> dump autotests (or, to be more accurate, it doesn't - as long as test doesn't
> crash, it's considered to be successful :) ).
> 
> Also, for readability, we should move these dump tests together and have all
> dump tests one after the other.
> 

Ok will add "Func" value  as dump_autotest and move all of the dump tests to end of  parallel list.
diff mbox series

Patch

diff --git a/test/test/autotest_data.py b/test/test/autotest_data.py
index c24e7bc25..6e26cc190 100644
--- a/test/test/autotest_data.py
+++ b/test/test/autotest_data.py
@@ -134,12 +134,6 @@ 
         "Func":    default_autotest,
         "Report":  None,
     },
-    {
-        "Name":    "PCI autotest",
-        "Command": "pci_autotest",
-        "Func":    default_autotest,
-        "Report":  None,
-    },
     {
         "Name":    "Malloc autotest",
         "Command": "malloc_autotest",
@@ -248,6 +242,311 @@ 
         "Func":    default_autotest,
         "Report":  None,
     },
+    {
+        "Name":    "Eventdev selftest octeontx",
+        "Command": "eventdev_selftest_octeontx",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Event ring autotest",
+        "Command": "event_ring_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Table autotest",
+        "Command": "table_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Flow classify autotest",
+        "Command": "flow_classify_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump struct sizes",
+        "Command": "dump_struct_sizes",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump mempool",
+        "Command": "dump_mempool",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump malloc stats",
+        "Command": "dump_malloc_stats",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump devargs",
+        "Command": "dump_devargs",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump log types",
+        "Command": "dump_log_types",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump_ring",
+        "Command": "dump_ring",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Quit",
+        "Command": "quit",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Set rxtx mode",
+        "Command": "set_rxtx_mode",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Set rxtx anchor",
+        "Command": "set_rxtx_anchor",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Set rxtx sc",
+        "Command": "set_rxtx_sc",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Event eth rx adapter autotest",
+        "Command": "event_eth_rx_adapter_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Rawdev autotest",
+        "Command": "rawdev_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Kvargs autotest",
+        "Command": "kvargs_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump physmem",
+        "Command": "dump_physmem",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump memzone",
+        "Command": "dump_memzone",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Devargs autotest",
+        "Command": "devargs_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Link bonding autotest",
+        "Command": "link_bonding_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Link bonding mode4 autotest",
+        "Command": "link_bonding_mode4_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Link bonding rssconf autotest",
+        "Command": "link_bonding_rssconf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Crc autotest",
+        "Command": "crc_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Distributor autotest",
+        "Command": "distributor_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Reorder autotest",
+        "Command": "reorder_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Barrier autotest",
+        "Command": "barrier_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "User delay",
+        "Command": "user_delay_us",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Bitmap test",
+        "Command": "bitmap_test",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Hash scaling autotest",
+        "Command": "hash_scaling_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Hash multiwriter autotest",
+        "Command": "hash_multiwriter_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Service autotest",
+        "Command": "service_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Timer racecond autotest",
+        "Command": "timer_racecond_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Member autotest",
+        "Command": "member_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":   "Efd_autotest",
+        "Command": "efd_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Thash autotest",
+        "Command": "thash_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Hash function autotest",
+        "Command": "hash_functions_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+        "Name":    "Cryptodev sw mrvl autotest",
+        "Command": "cryptodev_sw_mrvl_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev dpaa2 sec autotest",
+        "Command": "cryptodev_dpaa2_sec_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev dpaa sec autotest",
+        "Command": "cryptodev_dpaa_sec_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev qat autotest",
+        "Command": "cryptodev_qat_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev aesni mb autotest",
+        "Command": "cryptodev_aesni_mb_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev openssl autotest",
+        "Command": "cryptodev_openssl_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev scheduler autotest",
+        "Command": "cryptodev_scheduler_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev aesni gcm autotest",
+        "Command": "cryptodev_aesni_gcm_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev null autotest",
+        "Command": "cryptodev_null_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev sw snow3g autotest",
+        "Command": "cryptodev_sw_snow3g_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev sw kasumi autotest",
+        "Command": "cryptodev_sw_kasumi_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev_sw_zuc_autotest",
+        "Command": "cryptodev_sw_zuc_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Reciprocal division",
+        "Command": "reciprocal_division",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Red all",
+        "Command": "red_all",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
 ]
 
 # tests that should not be run when any other tests are running
@@ -259,8 +558,8 @@ 
         "Report":  None,
     },
     {
-        "Name":    "Eventdev sw autotest",
-        "Command": "eventdev_sw_autotest",
+        "Name":    "Eventdev selftest sw",
+        "Command": "eventdev_selftest_sw",
         "Func":    default_autotest,
         "Report":  None,
     },
@@ -312,6 +611,61 @@ 
         "Func":    default_autotest,
         "Report":  None,
     },
+    {
+
+        "Name":    "Pmd perf autotest",
+        "Command": "pmd_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Ring pmd perf autotest",
+        "Command": "ring_pmd_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Distributor perf autotest",
+        "Command": "distributor_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Red_perf",
+        "Command": "red_perf",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Lpm6 perf autotest",
+        "Command": "lpm6_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Lpm perf autotest",
+        "Command": "lpm_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+         "Name":    "Efd perf autotest",
+         "Command": "efd_perf_autotest",
+         "Func":    default_autotest,
+         "Report":  None,
+    },
+    {
+        "Name":    "Member perf autotest",
+        "Command": "member_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Reciprocal division perf",
+        "Command": "reciprocal_division_perf",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
     #
     # Please always make sure that ring_perf is the last test!
     #