diff mbox

[dpdk-dev] app/test: enhance sanity script

Message ID 1524835241-12778-1-git-send-email-reshma.pattan@intel.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Pattan, Reshma April 27, 2018, 1:20 p.m. UTC 
  1. Sanity Script is enhanced with additional test cases
being added to autotest_data.py
2. Fixed in autotest_test_funcs.py to handle test cases
which returns "Skipped" as result.
  The issue was skipped test cases got timed out,
causing delay in sanity script execution.
3. Enhanced support for FreeBSD to add the respective memory limit,
as FreeBSD doesn't support socket-mem, file-prefix options.

Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
---
 test/test/autotest_data.py       | 471 ++++++++++++++++++++++++++++++++++++++-
 test/test/autotest_runner.py     |   9 +-
 test/test/autotest_test_funcs.py |   6 +-
 3 files changed, 474 insertions(+), 12 deletions(-)
 mode change 100644 => 100755 test/test/autotest_data.py
 mode change 100644 => 100755 test/test/autotest_test_funcs.py

Comments

Anatoly Burakov April 27, 2018, 1:41 p.m. UTC | #1 
On 27-Apr-18 2:20 PM, Reshma Pattan wrote:
> 1. Sanity Script is enhanced with additional test cases
> being added to autotest_data.py
> 2. Fixed in autotest_test_funcs.py to handle test cases
> which returns "Skipped" as result.
>    The issue was skipped test cases got timed out,
> causing delay in sanity script execution.
> 3. Enhanced support for FreeBSD to add the respective memory limit,
> as FreeBSD doesn't support socket-mem, file-prefix options.
> 
> Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
> ---

I don't think number 3 on that list is needed. FreeBSD doesn't support 
file-prefixes, so it's not possible to run multiple unit tests in 
parallel in the first place, so no need for grouping and no need to 
limit allocated memory.

On a more general note, IMO this whole thing needs to be redone and 
simplified. I wanted to fix all this as part of Memory Hotplug patches, 
but didn't get to it yet.

(not to mention that this infrastructure itself is a lot like IVSHMEM, 
in that it was built to solve a specific problem, consequences be damned...)

The reasoning behind splitting tests into groups was largely the fact 
that we had to preallocate memory, while also running tests in parallel. 
This is no longer needed for two reasons.

First, most (if not all) tests now clean up properly after themselves 
(when this was written, it wasn't possible to un-reserve memzones, rings 
etc. - now it is), so we can run them one by one without restarting the 
test application.

Second, memory is now allocated on the fly, so there is no more reason 
to split autotests into groups at all. Instead, we can just group all 
tests into "performance" (to be run one by one) and "non performance" 
(can be run concurrently via different prefixes), and run them with a 
pool of workers.

For FreeBSD, since there's no dynamic memory allocation and no support 
for file prefixes, there needs to be only one worker, running all tests 
one by one. For Linux, you can add all unit tests into a queue, and use 
a pool of workers dequeuing and doing tests in parallel under different 
prefixes. No '-m' or '--socket-mem' switches needed.
Anatoly Burakov April 27, 2018, 1:56 p.m. UTC | #2 
On 27-Apr-18 2:20 PM, Reshma Pattan wrote:
> 1. Sanity Script is enhanced with additional test cases
> being added to autotest_data.py
> 2. Fixed in autotest_test_funcs.py to handle test cases
> which returns "Skipped" as result.
>    The issue was skipped test cases got timed out,
> causing delay in sanity script execution.
> 3. Enhanced support for FreeBSD to add the respective memory limit,
> as FreeBSD doesn't support socket-mem, file-prefix options.
> 
> Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
> ---

<...>

> @@ -1,5 +1,5 @@
>   # SPDX-License-Identifier: BSD-3-Clause
> -# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2010-2018 Intel Corporation
>   
>   # The main logic behind running autotests in parallel
>   
> @@ -201,6 +201,13 @@ def __init__(self, cmdline, target, blacklist, whitelist):
>       # set up cmdline string
>       def __get_cmdline(self, test):
>           cmdline = self.cmdline
> +        os_name = platform.system()
> +
> +        # append the respective memory limitation on FREEBSD
> +        # as socket-mem, file-prefix is not supported
> +        if os_name == "FreeBSD" :
> +            cmdline += " -m " + str(sum(map(int, test["Memory"].split(","))))
> +            return cmdline

A better fix would've been to completely avoid calling __get_cmdline() 
for FreeBSD. Since file prefixes are not supported, tests cannot be run 
in parallel, so there is no need to limit allocated memory in the first 
place, so neither -m nor --socket-mem switches need to be added either. 
And without those, there's no need to modify the test cmdline at all.
Jananee Parthasarathy May 3, 2018, 10:03 a.m. UTC | #3 
Hi Anatoly,

As per the comment " Since file prefixes are not supported, tests cannot be run in parallel" 
Does this means we need not run parallel test group test cases for FreeBSD. So the total test count for FreeBSD will be based on Non-Parallel test groups only.

Please confirm / share your views on the same.

Thanks
M.P.Jananee

-----Original Message-----
From: Burakov, Anatoly 

Sent: Friday, April 27, 2018 7:26 PM
To: Pattan, Reshma <reshma.pattan@intel.com>; Mcnamara, John <john.mcnamara@intel.com>; dev@dpdk.org
Cc: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>
Subject: Re: [dpdk-dev] [PATCH] app/test: enhance sanity script

On 27-Apr-18 2:20 PM, Reshma Pattan wrote:
> 1. Sanity Script is enhanced with additional test cases being added to 

> autotest_data.py 2. Fixed in autotest_test_funcs.py to handle test 

> cases which returns "Skipped" as result.

>    The issue was skipped test cases got timed out, causing delay in 

> sanity script execution.

> 3. Enhanced support for FreeBSD to add the respective memory limit, as 

> FreeBSD doesn't support socket-mem, file-prefix options.

> 

> Signed-off-by: Jananee Parthasarathy 

> <jananeex.m.parthasarathy@intel.com>

> ---


<...>

> @@ -1,5 +1,5 @@

>   # SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2010-2014 

> Intel Corporation

> +# Copyright(c) 2010-2018 Intel Corporation

>   

>   # The main logic behind running autotests in parallel

>   

> @@ -201,6 +201,13 @@ def __init__(self, cmdline, target, blacklist, whitelist):

>       # set up cmdline string

>       def __get_cmdline(self, test):

>           cmdline = self.cmdline

> +        os_name = platform.system()

> +

> +        # append the respective memory limitation on FREEBSD

> +        # as socket-mem, file-prefix is not supported

> +        if os_name == "FreeBSD" :

> +            cmdline += " -m " + str(sum(map(int, test["Memory"].split(","))))

> +            return cmdline


A better fix would've been to completely avoid calling __get_cmdline() for FreeBSD. Since file prefixes are not supported, tests cannot be run in parallel, so there is no need to limit allocated memory in the first place, so neither -m nor --socket-mem switches need to be added either. 
And without those, there's no need to modify the test cmdline at all.

--
Thanks,
Anatoly
--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
Anatoly Burakov May 3, 2018, 10:54 a.m. UTC | #4 
On 03-May-18 11:03 AM, Parthasarathy, JananeeX M wrote:
> Hi Anatoly,
> 
> As per the comment " Since file prefixes are not supported, tests cannot be run in parallel"
> Does this means we need not run parallel test group test cases for FreeBSD. So the total test count for FreeBSD will be based on Non-Parallel test groups only.

No, the tests in the "parallel group" can be run in parallel, but on 
FreeBSD you cannot run them *in parallel* - that doesn't mean you don't 
have to run them :) Just ensure that the thread pool size is set to 1 on 
FreeBSD.

> 
> Please confirm / share your views on the same.
> 
> Thanks
> M.P.Jananee
> 
> -----Original Message-----
> From: Burakov, Anatoly
> Sent: Friday, April 27, 2018 7:26 PM
> To: Pattan, Reshma <reshma.pattan@intel.com>; Mcnamara, John <john.mcnamara@intel.com>; dev@dpdk.org
> Cc: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>
> Subject: Re: [dpdk-dev] [PATCH] app/test: enhance sanity script
> 
> On 27-Apr-18 2:20 PM, Reshma Pattan wrote:
>> 1. Sanity Script is enhanced with additional test cases being added to
>> autotest_data.py 2. Fixed in autotest_test_funcs.py to handle test
>> cases which returns "Skipped" as result.
>>     The issue was skipped test cases got timed out, causing delay in
>> sanity script execution.
>> 3. Enhanced support for FreeBSD to add the respective memory limit, as
>> FreeBSD doesn't support socket-mem, file-prefix options.
>>
>> Signed-off-by: Jananee Parthasarathy
>> <jananeex.m.parthasarathy@intel.com>
>> ---
> 
> <...>
> 
>> @@ -1,5 +1,5 @@
>>    # SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2010-2014
>> Intel Corporation
>> +# Copyright(c) 2010-2018 Intel Corporation
>>    
>>    # The main logic behind running autotests in parallel
>>    
>> @@ -201,6 +201,13 @@ def __init__(self, cmdline, target, blacklist, whitelist):
>>        # set up cmdline string
>>        def __get_cmdline(self, test):
>>            cmdline = self.cmdline
>> +        os_name = platform.system()
>> +
>> +        # append the respective memory limitation on FREEBSD
>> +        # as socket-mem, file-prefix is not supported
>> +        if os_name == "FreeBSD" :
>> +            cmdline += " -m " + str(sum(map(int, test["Memory"].split(","))))
>> +            return cmdline
> 
> A better fix would've been to completely avoid calling __get_cmdline() for FreeBSD. Since file prefixes are not supported, tests cannot be run in parallel, so there is no need to limit allocated memory in the first place, so neither -m nor --socket-mem switches need to be added either.
> And without those, there's no need to modify the test cmdline at all.

Also, regarding this - i think what i suggested earlier is slightly 
incorrect. A better fix would be to simply return cmdline without 
modifications on FreeBSD. Just because we don't modify cmdline now 
doesn't mean we wouldn't be doing so in the future, so not calling 
__get_cmdline is not the best solution.

> 
> --
> Thanks,
> Anatoly
>
diff mbox

Patch

diff --git a/test/test/autotest_data.py b/test/test/autotest_data.py
old mode 100644
new mode 100755
index aacfe0a..2c6828d
--- a/test/test/autotest_data.py
+++ b/test/test/autotest_data.py
@@ -1,5 +1,5 @@ 
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2010-2018 Intel Corporation
 
 # Test data for autotests
 
@@ -70,6 +70,31 @@  def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Eventdev selftest octeontx",
+                "Command": "eventdev_selftest_octeontx",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Event ring autotest",
+                "Command": "event_ring_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Table autotest",
+                "Command": "table_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Flow classify autotest",
+                "Command": "flow_classify_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
         ]
     },
     {
@@ -125,6 +150,50 @@  def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Dump struct sizes",
+                "Command": "dump_struct_sizes",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump mempool",
+                "Command": "dump_mempool",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump malloc stats",
+                "Command": "dump_malloc_stats",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump devargs",
+                "Command": "dump_devargs",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump log types",
+                "Command": "dump_log_types",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump_ring",
+                "Command": "dump_ring",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Quit",
+                "Command": "quit",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
+
         ],
     },
     {
@@ -168,6 +237,26 @@  def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Set rxtx mode",
+                "Command": "set_rxtx_mode",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Set rxtx anchor",
+                "Command": "set_rxtx_anchor",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Set rxtx sc",
+                "Command": "set_rxtx_sc",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
+
         ]
     },
     {
@@ -176,12 +265,6 @@  def per_sockets(num):
         "Tests":
         [
             {
-                "Name":    "PCI autotest",
-                "Command": "pci_autotest",
-                "Func":    default_autotest,
-                "Report":  None,
-            },
-            {
                 "Name":    "Malloc autotest",
                 "Command": "malloc_autotest",
                 "Func":    default_autotest,
@@ -211,6 +294,37 @@  def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Event eth rx adapter autotest",
+                "Command": "event_eth_rx_adapter_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Rawdev autotest",
+                "Command": "rawdev_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Kvargs autotest",
+                "Command": "kvargs_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump physmem",
+                "Command": "dump_physmem",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump memzone",
+                "Command": "dump_memzone",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
         ]
     },
     {
@@ -248,6 +362,31 @@  def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Devargs autotest",
+                "Command": "devargs_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Link bonding autotest",
+                "Command": "link_bonding_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Link bonding mode4 autotest",
+                "Command": "link_bonding_mode4_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Link bonding rssconf autotest",
+                "Command": "link_bonding_rssconf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
         ]
     },
     {
@@ -285,6 +424,25 @@  def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Crc autotest",
+                "Command": "crc_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Distributor autotest",
+                "Command": "distributor_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Reorder autotest",
+                "Command": "reorder_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
         ]
     },
     {
@@ -310,8 +468,184 @@  def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Barrier autotest",
+                "Command": "barrier_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "User delay",
+                "Command": "user_delay_us",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Bitmap test",
+                "Command": "bitmap_test",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Hash scaling autotest",
+                "Command": "hash_scaling_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Hash multiwriter autotest",
+                "Command": "hash_multiwriter_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Service autotest",
+                "Command": "service_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
+
+        ]
+    },
+    {
+        "Prefix":   "group_8",
+        "Memory":   "128",
+        "Tests":
+        [
+            {
+                "Name":    "Timer racecond autotest",
+                "Command": "timer_racecond_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Member autotest",
+                "Command": "member_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":   "Efd_autotest",
+                "Command": "efd_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Thash autotest",
+                "Command": "thash_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Hash function autotest",
+                "Command": "hash_functions_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":   "group_9",
+        "Memory":   "64",
+        "Tests":
+        [
+            {
+                "Name":    "Cryptodev sw mrvl autotest",
+                "Command": "cryptodev_sw_mrvl_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev dpaa2 sec autotest",
+                "Command": "cryptodev_dpaa2_sec_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev dpaa sec autotest",
+                "Command": "cryptodev_dpaa_sec_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev qat autotest",
+                "Command": "cryptodev_qat_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev aesni mb autotest",
+                "Command": "cryptodev_aesni_mb_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev openssl autotest",
+                "Command": "cryptodev_openssl_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev scheduler autotest",
+                "Command": "cryptodev_scheduler_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev aesni gcm autotest",
+                "Command": "cryptodev_aesni_gcm_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev null autotest",
+                "Command": "cryptodev_null_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev sw snow3g autotest",
+                "Command": "cryptodev_sw_snow3g_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev sw kasumi autotest",
+                "Command": "cryptodev_sw_kasumi_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev_sw_zuc_autotest",
+                "Command": "cryptodev_sw_zuc_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
+
         ]
     },
+    {
+        "Prefix":   "group_10",
+        "Memory":   "64",
+        "Tests":
+        [
+            {
+                "Name":    "Reciprocal division",
+                "Command": "reciprocal_division",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Red all",
+                "Command": "red_all",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+
 ]
 
 # tests that should not be run when any other tests are running
@@ -336,8 +670,8 @@  def per_sockets(num):
         "Tests":
         [
             {
-                "Name":    "Eventdev sw autotest",
-                "Command": "eventdev_sw_autotest",
+                "Name":    "Eventdev selftest sw",
+                "Command": "eventdev_selftest_sw",
                 "Func":    default_autotest,
                 "Report":  None,
             },
@@ -447,6 +781,125 @@  def per_sockets(num):
             },
         ]
     },
+    {
+        "Prefix":    "reciprocal_division_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Reciprocal division perf",
+                "Command": "reciprocal_division_perf",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "member_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Member perf autotest",
+                "Command": "member_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "efd_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Efd perf autotest",
+                "Command": "efd_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "lpm_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Lpm perf autotest",
+                "Command": "lpm_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "lpm6_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Lpm6 perf autotest",
+                "Command": "lpm6_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "red_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Red_perf",
+                "Command": "red_perf",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "distributor_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Distributor perf autotest",
+                "Command": "distributor_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "ring_pmd_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Ring pmd perf autotest",
+                "Command": "ring_pmd_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "pmd_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Pmd perf autotest",
+                "Command": "pmd_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+
+
 
     #
     # Please always make sure that ring_perf is the last test!
diff --git a/test/test/autotest_runner.py b/test/test/autotest_runner.py
index a692f06..77f0ac7 100644
--- a/test/test/autotest_runner.py
+++ b/test/test/autotest_runner.py
@@ -1,5 +1,5 @@ 
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2010-2018 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
@@ -201,6 +201,13 @@  def __init__(self, cmdline, target, blacklist, whitelist):
     # set up cmdline string
     def __get_cmdline(self, test):
         cmdline = self.cmdline
+        os_name = platform.system()
+
+        # append the respective memory limitation on FREEBSD
+        # as socket-mem, file-prefix is not supported
+        if os_name == "FreeBSD" :
+            cmdline += " -m " + str(sum(map(int, test["Memory"].split(","))))
+            return cmdline
 
         # append memory limitations for each test
         # otherwise tests won't run in parallel
diff --git a/test/test/autotest_test_funcs.py b/test/test/autotest_test_funcs.py
old mode 100644
new mode 100755
index 65fe335..219c208
--- a/test/test/autotest_test_funcs.py
+++ b/test/test/autotest_test_funcs.py
@@ -1,5 +1,5 @@ 
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2010-2018 Intel Corporation
 
 # Test functions
 
@@ -12,12 +12,14 @@ 
 def default_autotest(child, test_name):
     child.sendline(test_name)
     result = child.expect(["Test OK", "Test Failed",
-                           "Command not found", pexpect.TIMEOUT], timeout=900)
+                           "Command not found", "Skipped", pexpect.TIMEOUT], timeout=900)
     if result == 1:
         return -1, "Fail"
     elif result == 2:
         return -1, "Fail [Not found]"
     elif result == 3:
+        return -1, "Fail [Test returns Skipped]"
+    elif result == 4:
         return -1, "Fail [Timeout]"
     return 0, "Success"