diff mbox series

[v3,8/9] autotest: update autotest test case list

Message ID 1531750337-4708-9-git-send-email-reshma.pattan@intel.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series None |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Pattan, Reshma July 16, 2018, 2:12 p.m. UTC 
  Autotest is enhanced with additional test cases
being added to autotest_data.py

Removed non existing PCI autotest.

Cc: stable@dpdk.org

Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
---
 test/test/autotest_data.py | 368 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 360 insertions(+), 8 deletions(-)

Comments

Burakov, Anatoly July 16, 2018, 3:16 p.m. UTC | #1 
On 16-Jul-18 3:12 PM, Reshma Pattan wrote:
> Autotest is enhanced with additional test cases
> being added to autotest_data.py
> 
> Removed non existing PCI autotest.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
> Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
> ---

<snip>

> +    },
> +    {
> +        "Name":    "Flow classify autotest",
> +        "Command": "flow_classify_autotest",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +        {
> +        "Name":    "Set rxtx mode",
> +        "Command": "set_rxtx_mode",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Set rxtx anchor",
> +        "Command": "set_rxtx_anchor",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Set rxtx sc",
> +        "Command": "set_rxtx_sc",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },

The above three tests don't look like autotests to me. I have no idea 
what they are for, but either they need a special function, or they need 
to be taken out.

> +    {
> +        "Name":    "Event eth rx adapter autotest",
> +        "Command": "event_eth_rx_adapter_autotest",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Rawdev autotest",
> +        "Command": "rawdev_autotest",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },

<snip>

> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Barrier autotest",
> +        "Command": "barrier_autotest",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "User delay",
> +        "Command": "user_delay_us",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },

This doesn't look like autotests to me. I have no idea what it is for, 
but either it needs a special function, or it needs to be taken out.

> +    {
> +        "Name":    "Bitmap test",
> +        "Command": "bitmap_test",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Hash scaling autotest",
> +        "Command": "hash_scaling_autotest",
> +        "Func":    default_autotest,
> +        "Report":  None,
> +    },
> +    {
> +        "Name":    "Hash multiwriter autotest",
> +        "Command": "hash_multiwriter_autotest",
Pattan, Reshma July 17, 2018, 9:18 a.m. UTC | #2 
Hi,

> -----Original Message-----
> From: Burakov, Anatoly
> Sent: Monday, July 16, 2018 4:16 PM
> To: Pattan, Reshma <reshma.pattan@intel.com>; thomas@monjalon.net;
> dev@dpdk.org
> Cc: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>;
> stable@dpdk.org
> Subject: Re: [PATCH v3 8/9] autotest: update autotest test case list
> 
> 
> > +        {
> > +        "Name":    "Set rxtx mode",
> > +        "Command": "set_rxtx_mode",
> > +        "Func":    default_autotest,
> > +        "Report":  None,
> > +    },
> > +    {
> > +        "Name":    "Set rxtx anchor",
> > +        "Command": "set_rxtx_anchor",
> > +        "Func":    default_autotest,
> > +        "Report":  None,
> > +    },
> > +    {
> > +        "Name":    "Set rxtx sc",
> > +        "Command": "set_rxtx_sc",
> > +        "Func":    default_autotest,
> > +        "Report":  None,
> > +    },
> 
> The above three tests don't look like autotests to me. I have no idea what
> they are for, but either they need a special function, or they need to be taken
> out.
> 

These commands needs to be run manually from test cmd prompt to various set rxtx mode, rxtx rate and rxtx direction .
These can be used to verify pmd perf test  with vaiours set of above values.

So this can be removed from autotest.

> > +        "Name":    "User delay",
> > +        "Command": "user_delay_us",
> > +        "Func":    default_autotest,
> > +        "Report":  None,
> > +    },
> 
> This doesn't look like autotests to me. I have no idea what it is for, but either
> it needs a special function, or it needs to be taken out.
> 
This is autotest but the name does'nt have the autotest name in it. So I will retain this.

Thanks,
Reshma
Burakov, Anatoly July 17, 2018, 9:23 a.m. UTC | #3 
On 17-Jul-18 10:18 AM, Pattan, Reshma wrote:
> Hi,
> 
>> -----Original Message-----
>> From: Burakov, Anatoly
>> Sent: Monday, July 16, 2018 4:16 PM
>> To: Pattan, Reshma <reshma.pattan@intel.com>; thomas@monjalon.net;
>> dev@dpdk.org
>> Cc: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>;
>> stable@dpdk.org
>> Subject: Re: [PATCH v3 8/9] autotest: update autotest test case list
>>
>>
>>> +        {
>>> +        "Name":    "Set rxtx mode",
>>> +        "Command": "set_rxtx_mode",
>>> +        "Func":    default_autotest,
>>> +        "Report":  None,
>>> +    },
>>> +    {
>>> +        "Name":    "Set rxtx anchor",
>>> +        "Command": "set_rxtx_anchor",
>>> +        "Func":    default_autotest,
>>> +        "Report":  None,
>>> +    },
>>> +    {
>>> +        "Name":    "Set rxtx sc",
>>> +        "Command": "set_rxtx_sc",
>>> +        "Func":    default_autotest,
>>> +        "Report":  None,
>>> +    },
>>
>> The above three tests don't look like autotests to me. I have no idea what
>> they are for, but either they need a special function, or they need to be taken
>> out.
>>
> 
> These commands needs to be run manually from test cmd prompt to various set rxtx mode, rxtx rate and rxtx direction .
> These can be used to verify pmd perf test  with vaiours set of above values.
> 
> So this can be removed from autotest.

We do have PMD perf tests in the script - do they call these functions? 
If they are required for PMD autotests, maybe PMD autotests deserve a 
special test function calling these commands before running the tests?

(if they also work without these commands, then we can perhaps postpone 
this to 18.11)

> 
>>> +        "Name":    "User delay",
>>> +        "Command": "user_delay_us",
>>> +        "Func":    default_autotest,
>>> +        "Report":  None,
>>> +    },
>>
>> This doesn't look like autotests to me. I have no idea what it is for, but either
>> it needs a special function, or it needs to be taken out.
>>
> This is autotest but the name does'nt have the autotest name in it. So I will retain this.

OK.
Pattan, Reshma July 17, 2018, 9:45 a.m. UTC | #4 
Hi,

> -----Original Message-----
> From: Burakov, Anatoly
> Sent: Tuesday, July 17, 2018 10:23 AM
> To: Pattan, Reshma <reshma.pattan@intel.com>; thomas@monjalon.net;
> dev@dpdk.org
> Cc: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>;
> stable@dpdk.org
> Subject: Re: [PATCH v3 8/9] autotest: update autotest test case list
> 
> On 17-Jul-18 10:18 AM, Pattan, Reshma wrote:
> > Hi,
> >
> >> -----Original Message-----
> >> From: Burakov, Anatoly
> >> Sent: Monday, July 16, 2018 4:16 PM
> >> To: Pattan, Reshma <reshma.pattan@intel.com>; thomas@monjalon.net;
> >> dev@dpdk.org
> >> Cc: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>;
> >> stable@dpdk.org
> >> Subject: Re: [PATCH v3 8/9] autotest: update autotest test case list
> >>
> >>
> >>> +        {
> >>> +        "Name":    "Set rxtx mode",
> >>> +        "Command": "set_rxtx_mode",
> >>> +        "Func":    default_autotest,
> >>> +        "Report":  None,
> >>> +    },
> >>> +    {
> >>> +        "Name":    "Set rxtx anchor",
> >>> +        "Command": "set_rxtx_anchor",
> >>> +        "Func":    default_autotest,
> >>> +        "Report":  None,
> >>> +    },
> >>> +    {
> >>> +        "Name":    "Set rxtx sc",
> >>> +        "Command": "set_rxtx_sc",
> >>> +        "Func":    default_autotest,
> >>> +        "Report":  None,
> >>> +    },
> >>
> >> The above three tests don't look like autotests to me. I have no idea
> >> what they are for, but either they need a special function, or they
> >> need to be taken out.
> >>
> >
> > These commands needs to be run manually from test cmd prompt to
> various set rxtx mode, rxtx rate and rxtx direction .
> > These can be used to verify pmd perf test  with vaiours set of above values.
> >
> > So this can be removed from autotest.
> 
> We do have PMD perf tests in the script - do they call these functions?
> If they are required for PMD autotests, maybe PMD autotests deserve a
> special test function calling these commands before running the tests?
> 
> (if they also work without these commands, then we can perhaps postpone
> this to 18.11)
> 

I ran pmd perf test manually and it passes without having to use above set_rxtx commands. 

Thanks,
Reshma
Burakov, Anatoly July 17, 2018, 10:10 a.m. UTC | #5 
On 17-Jul-18 10:45 AM, Pattan, Reshma wrote:
> Hi,
> 
>> -----Original Message-----
>> From: Burakov, Anatoly
>> Sent: Tuesday, July 17, 2018 10:23 AM
>> To: Pattan, Reshma <reshma.pattan@intel.com>; thomas@monjalon.net;
>> dev@dpdk.org
>> Cc: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>;
>> stable@dpdk.org
>> Subject: Re: [PATCH v3 8/9] autotest: update autotest test case list
>>
>> On 17-Jul-18 10:18 AM, Pattan, Reshma wrote:
>>> Hi,
>>>
>>>> -----Original Message-----
>>>> From: Burakov, Anatoly
>>>> Sent: Monday, July 16, 2018 4:16 PM
>>>> To: Pattan, Reshma <reshma.pattan@intel.com>; thomas@monjalon.net;
>>>> dev@dpdk.org
>>>> Cc: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>;
>>>> stable@dpdk.org
>>>> Subject: Re: [PATCH v3 8/9] autotest: update autotest test case list
>>>>
>>>>
>>>>> +        {
>>>>> +        "Name":    "Set rxtx mode",
>>>>> +        "Command": "set_rxtx_mode",
>>>>> +        "Func":    default_autotest,
>>>>> +        "Report":  None,
>>>>> +    },
>>>>> +    {
>>>>> +        "Name":    "Set rxtx anchor",
>>>>> +        "Command": "set_rxtx_anchor",
>>>>> +        "Func":    default_autotest,
>>>>> +        "Report":  None,
>>>>> +    },
>>>>> +    {
>>>>> +        "Name":    "Set rxtx sc",
>>>>> +        "Command": "set_rxtx_sc",
>>>>> +        "Func":    default_autotest,
>>>>> +        "Report":  None,
>>>>> +    },
>>>>
>>>> The above three tests don't look like autotests to me. I have no idea
>>>> what they are for, but either they need a special function, or they
>>>> need to be taken out.
>>>>
>>>
>>> These commands needs to be run manually from test cmd prompt to
>> various set rxtx mode, rxtx rate and rxtx direction .
>>> These can be used to verify pmd perf test  with vaiours set of above values.
>>>
>>> So this can be removed from autotest.
>>
>> We do have PMD perf tests in the script - do they call these functions?
>> If they are required for PMD autotests, maybe PMD autotests deserve a
>> special test function calling these commands before running the tests?
>>
>> (if they also work without these commands, then we can perhaps postpone
>> this to 18.11)
>>
> 
> I ran pmd perf test manually and it passes without having to use above set_rxtx commands.
> 
> Thanks,
> Reshma
> 
Great.

Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
diff mbox series

Patch

diff --git a/test/test/autotest_data.py b/test/test/autotest_data.py
index c24e7bc25..37f617146 100644
--- a/test/test/autotest_data.py
+++ b/test/test/autotest_data.py
@@ -134,12 +134,6 @@ 
         "Func":    default_autotest,
         "Report":  None,
     },
-    {
-        "Name":    "PCI autotest",
-        "Command": "pci_autotest",
-        "Func":    default_autotest,
-        "Report":  None,
-    },
     {
         "Name":    "Malloc autotest",
         "Command": "malloc_autotest",
@@ -248,6 +242,309 @@ 
         "Func":    default_autotest,
         "Report":  None,
     },
+    {
+        "Name":    "Eventdev selftest octeontx",
+        "Command": "eventdev_selftest_octeontx",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Event ring autotest",
+        "Command": "event_ring_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Table autotest",
+        "Command": "table_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Flow classify autotest",
+        "Command": "flow_classify_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+        {
+        "Name":    "Set rxtx mode",
+        "Command": "set_rxtx_mode",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Set rxtx anchor",
+        "Command": "set_rxtx_anchor",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Set rxtx sc",
+        "Command": "set_rxtx_sc",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Event eth rx adapter autotest",
+        "Command": "event_eth_rx_adapter_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Rawdev autotest",
+        "Command": "rawdev_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Kvargs autotest",
+        "Command": "kvargs_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Devargs autotest",
+        "Command": "devargs_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Link bonding autotest",
+        "Command": "link_bonding_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Link bonding mode4 autotest",
+        "Command": "link_bonding_mode4_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Link bonding rssconf autotest",
+        "Command": "link_bonding_rssconf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Crc autotest",
+        "Command": "crc_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Distributor autotest",
+        "Command": "distributor_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Reorder autotest",
+        "Command": "reorder_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Barrier autotest",
+        "Command": "barrier_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "User delay",
+        "Command": "user_delay_us",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Bitmap test",
+        "Command": "bitmap_test",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Hash scaling autotest",
+        "Command": "hash_scaling_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Hash multiwriter autotest",
+        "Command": "hash_multiwriter_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Service autotest",
+        "Command": "service_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Timer racecond autotest",
+        "Command": "timer_racecond_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Member autotest",
+        "Command": "member_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":   "Efd_autotest",
+        "Command": "efd_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Thash autotest",
+        "Command": "thash_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Hash function autotest",
+        "Command": "hash_functions_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev sw mrvl autotest",
+        "Command": "cryptodev_sw_mrvl_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev dpaa2 sec autotest",
+        "Command": "cryptodev_dpaa2_sec_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev dpaa sec autotest",
+        "Command": "cryptodev_dpaa_sec_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev qat autotest",
+        "Command": "cryptodev_qat_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev aesni mb autotest",
+        "Command": "cryptodev_aesni_mb_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev openssl autotest",
+        "Command": "cryptodev_openssl_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev scheduler autotest",
+        "Command": "cryptodev_scheduler_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev aesni gcm autotest",
+        "Command": "cryptodev_aesni_gcm_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev null autotest",
+        "Command": "cryptodev_null_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev sw snow3g autotest",
+        "Command": "cryptodev_sw_snow3g_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev sw kasumi autotest",
+        "Command": "cryptodev_sw_kasumi_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Cryptodev_sw_zuc_autotest",
+        "Command": "cryptodev_sw_zuc_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Reciprocal division",
+        "Command": "reciprocal_division",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Red all",
+        "Command": "red_all",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    #
+    #Please always keep all dump tests at the end and together!
+    #
+    {
+        "Name":    "Dump physmem",
+        "Command": "dump_physmem",
+        "Func":    dump_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump memzone",
+        "Command": "dump_memzone",
+        "Func":    dump_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump struct sizes",
+        "Command": "dump_struct_sizes",
+        "Func":    dump_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump mempool",
+        "Command": "dump_mempool",
+        "Func":    dump_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump malloc stats",
+        "Command": "dump_malloc_stats",
+        "Func":    dump_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump devargs",
+        "Command": "dump_devargs",
+        "Func":    dump_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump log types",
+        "Command": "dump_log_types",
+        "Func":    dump_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Dump_ring",
+        "Command": "dump_ring",
+        "Func":    dump_autotest,
+        "Report":  None,
+    },
 ]
 
 # tests that should not be run when any other tests are running
@@ -259,8 +556,8 @@ 
         "Report":  None,
     },
     {
-        "Name":    "Eventdev sw autotest",
-        "Command": "eventdev_sw_autotest",
+        "Name":    "Eventdev selftest sw",
+        "Command": "eventdev_selftest_sw",
         "Func":    default_autotest,
         "Report":  None,
     },
@@ -312,6 +609,61 @@ 
         "Func":    default_autotest,
         "Report":  None,
     },
+    {
+
+        "Name":    "Pmd perf autotest",
+        "Command": "pmd_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Ring pmd perf autotest",
+        "Command": "ring_pmd_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Distributor perf autotest",
+        "Command": "distributor_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Red_perf",
+        "Command": "red_perf",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Lpm6 perf autotest",
+        "Command": "lpm6_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Lpm perf autotest",
+        "Command": "lpm_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+         "Name":    "Efd perf autotest",
+         "Command": "efd_perf_autotest",
+         "Func":    default_autotest,
+         "Report":  None,
+    },
+    {
+        "Name":    "Member perf autotest",
+        "Command": "member_perf_autotest",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
+    {
+        "Name":    "Reciprocal division perf",
+        "Command": "reciprocal_division_perf",
+        "Func":    default_autotest,
+        "Report":  None,
+    },
     #
     # Please always make sure that ring_perf is the last test!
     #