[v5,05/10] app/test: define unit tests suites based on test macros

  Rather than having the test suites listed out in the meson.build files
and having to have them enabled/disabled selectively based on what libs
are being built, pull the tests to run from the source files which were
added to the build.

Most test suites require no additional info other than the list of test
names in the suite. However the fast-test are special that they have
additional parameters associated with them. This requires some
additional work in the test extraction script and in the meson.build
file for processing the output.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/meson.build                               |  7 +-
 app/test/suites/meson.build                   | 74 +++++++++++++++++++
 buildtools/get-test-suites.py                 | 33 +++++++++
 .../has-hugepages.py                          |  0
 buildtools/meson.build                        |  2 +
 5 files changed, 115 insertions(+), 1 deletion(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)
  

On Tue, Aug 15, 2023 at 04:10:49PM +0100, Bruce Richardson wrote:
> Rather than having the test suites listed out in the meson.build files
> and having to have them enabled/disabled selectively based on what libs
> are being built, pull the tests to run from the source files which were
> added to the build.
> 
> Most test suites require no additional info other than the list of test
> names in the suite. However the fast-test are special that they have
> additional parameters associated with them. This requires some
> additional work in the test extraction script and in the meson.build
> file for processing the output.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> ---
>  app/meson.build                               |  7 +-
>  app/test/suites/meson.build                   | 74 +++++++++++++++++++
>  buildtools/get-test-suites.py                 | 33 +++++++++
>  .../has-hugepages.py                          |  0
>  buildtools/meson.build                        |  2 +
>  5 files changed, 115 insertions(+), 1 deletion(-)
>  create mode 100644 app/test/suites/meson.build
>  create mode 100644 buildtools/get-test-suites.py
>  rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)
> 
> diff --git a/app/meson.build b/app/meson.build
> index 0d8b618e7f..c14dc80892 100644
> --- a/app/meson.build
> +++ b/app/meson.build
> @@ -101,7 +101,7 @@ foreach app:apps
>          link_libs = dpdk_static_libraries + dpdk_drivers
>      endif
>  
> -    executable('dpdk-' + name,
> +    exec = executable('dpdk-' + name,
>              sources,
>              c_args: cflags,
>              link_args: ldflags,
> @@ -110,4 +110,9 @@ foreach app:apps
>              include_directories: includes,
>              install_rpath: join_paths(get_option('prefix'), driver_install_path),
>              install: true)
> +    if name == 'test'
> +        dpdk_test = exec
> +        autotest_sources = sources
> +        subdir('test/suites')  # define the pre-canned test suites
> +    endif
>  endforeach
> diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
> new file mode 100644
> index 0000000000..ec74d8adf2
> --- /dev/null
> +++ b/app/test/suites/meson.build
> @@ -0,0 +1,74 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2023 Intel Corporation
> +
> +# some perf tests (eg: memcpy perf autotest)take very long
> +# to complete, so timeout to 10 minutes
> +timeout_seconds = 600
> +timeout_seconds_fast = 10
> +
> +test_no_huge_args = ['--no-huge', '-m', '2048']
> +has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
> +message('hugepage availability: @0@'.format(has_hugepage))
> +
> +# process source files to determine the different unit test suites
> +# - fast_tests
> +# - perf_tests
> +# - driver_tests
> +test_suites = run_command(get_test_suites_cmd, autotest_sources,
> +         check: true).stdout().strip().split()
> +foreach suite:test_suites
> +    # simple cases - tests without parameters or special handling
> +    suite = suite.split('=')
> +    suite_name = suite[0]
> +    suite_tests = suite[1].split(',')
> +    if suite_name != 'fast-tests'
> +        # simple cases - tests without parameters or special handling
> +        foreach t: suite_tests
> +            test(t, dpdk_test,
> +                    env: ['DPDK_TEST=' + t],
> +                    timeout: timeout_seconds,
> +                    is_parallel: false,
> +                    suite: suite_name)
> +        endforeach
> +    else
> +    # special fast-test handling here
> +        foreach t: suite_tests
> +            params = t.split(':')
> +            test_name = params[0]
> +            nohuge = params[1] == 'true'
> +            asan = params[2] == 'true'
> +
> +            test_args = []
> +            if nohuge
> +                test_args += test_no_huge_args
> +            elif not has_hugepage
> +                continue  #skip this tests
> +            endif
> +            if not asan and (get_option('b_sanitize') == 'address'
> +                    or get_option('b_sanitize') == 'address,undefined')
> +                continue  # skip this test
> +            endif
> +
> +            if get_option('default_library') == 'shared'
> +                test_args += ['-d', dpdk_drivers_build_dir]
> +            endif

These lines here seem to be exposing a bug in the mempool unit tests for
shared builds, which is why we have a CI failure.

The mempool unit tests use the mempool "create_empty" API, and then call
the populate APIs to finish setting up the mempool. However, the
create_empty API does not explicitly configure a particular set of mempool
ops for the new mempool, leaving the ops field at 0. This means that unless
the app explicitly sets other ops, the mempool will use the ops from
whatever driver registers itself first. This occurs even when the driver is
unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
leading to failures in setting up and using the mempool.

In v6 of this set, I intend to fix this, by changing the create empty API
to explicitly set the ring driver as default for new mempools. It's the
most likely to work for common cases.

/Bruce
  

On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
> These lines here seem to be exposing a bug in the mempool unit tests for
> shared builds, which is why we have a CI failure.
>
> The mempool unit tests use the mempool "create_empty" API, and then call
> the populate APIs to finish setting up the mempool. However, the
> create_empty API does not explicitly configure a particular set of mempool
> ops for the new mempool, leaving the ops field at 0. This means that unless
> the app explicitly sets other ops, the mempool will use the ops from
> whatever driver registers itself first. This occurs even when the driver is
> unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> leading to failures in setting up and using the mempool.

Hum, it sounds like a bug to me.
The dpaa2 driver should not be registered as the default (or here,
default platform) mempool.
Other mempool drivers ensure that required hw is available, I guess
dpaa2 is missing some check.


>
> In v6 of this set, I intend to fix this, by changing the create empty API
> to explicitly set the ring driver as default for new mempools. It's the
> most likely to work for common cases.
  

On Wed, Aug 16, 2023 at 1:15 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> > These lines here seem to be exposing a bug in the mempool unit tests for
> > shared builds, which is why we have a CI failure.
> >
> > The mempool unit tests use the mempool "create_empty" API, and then call
> > the populate APIs to finish setting up the mempool. However, the
> > create_empty API does not explicitly configure a particular set of mempool
> > ops for the new mempool, leaving the ops field at 0. This means that unless
> > the app explicitly sets other ops, the mempool will use the ops from
> > whatever driver registers itself first. This occurs even when the driver is
> > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > leading to failures in setting up and using the mempool.
>
> Hum, it sounds like a bug to me.
> The dpaa2 driver should not be registered as the default (or here,
> default platform) mempool.
> Other mempool drivers ensure that required hw is available, I guess
> dpaa2 is missing some check.

Ok, re-reading your last comment, I agree it looks like an issue in
the unit test itself.
Copying Olivier.
  

On Wed, Aug 16, 2023 at 01:40:41PM +0200, David Marchand wrote:
> On Wed, Aug 16, 2023 at 1:15 PM David Marchand
> <david.marchand@redhat.com> wrote:
> >
> > On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > > These lines here seem to be exposing a bug in the mempool unit tests for
> > > shared builds, which is why we have a CI failure.
> > >
> > > The mempool unit tests use the mempool "create_empty" API, and then call
> > > the populate APIs to finish setting up the mempool. However, the
> > > create_empty API does not explicitly configure a particular set of mempool
> > > ops for the new mempool, leaving the ops field at 0. This means that unless
> > > the app explicitly sets other ops, the mempool will use the ops from
> > > whatever driver registers itself first. This occurs even when the driver is
> > > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > > leading to failures in setting up and using the mempool.
> >
> > Hum, it sounds like a bug to me.
> > The dpaa2 driver should not be registered as the default (or here,
> > default platform) mempool.
> > Other mempool drivers ensure that required hw is available, I guess
> > dpaa2 is missing some check.
> 
> Ok, re-reading your last comment, I agree it looks like an issue in
> the unit test itself.
> Copying Olivier.
> 
No, I think it's not a bug in the test, but rather in the mempool.
Unfortunately, I think the concept of the "default" driver for mempools
seems to exist only when using the mbuf library to create mempools. Even
then, the default mempool is different from what the first entry in the
list is. That's the fundamental issue here, we are using the zero-eth entry
in the ops list, rather than a default one.

/Bruce
  

Hi,

On Wed, Aug 16, 2023 at 01:33:46PM +0100, Bruce Richardson wrote:
> On Wed, Aug 16, 2023 at 01:40:41PM +0200, David Marchand wrote:
> > On Wed, Aug 16, 2023 at 1:15 PM David Marchand
> > <david.marchand@redhat.com> wrote:
> > >
> > > On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> > > <bruce.richardson@intel.com> wrote:
> > > > These lines here seem to be exposing a bug in the mempool unit tests for
> > > > shared builds, which is why we have a CI failure.
> > > >
> > > > The mempool unit tests use the mempool "create_empty" API, and then call
> > > > the populate APIs to finish setting up the mempool. However, the
> > > > create_empty API does not explicitly configure a particular set of mempool
> > > > ops for the new mempool, leaving the ops field at 0. This means that unless
> > > > the app explicitly sets other ops, the mempool will use the ops from
> > > > whatever driver registers itself first. This occurs even when the driver is
> > > > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > > > leading to failures in setting up and using the mempool.
> > >
> > > Hum, it sounds like a bug to me.
> > > The dpaa2 driver should not be registered as the default (or here,
> > > default platform) mempool.
> > > Other mempool drivers ensure that required hw is available, I guess
> > > dpaa2 is missing some check.
> > 
> > Ok, re-reading your last comment, I agree it looks like an issue in
> > the unit test itself.
> > Copying Olivier.
> > 
> No, I think it's not a bug in the test, but rather in the mempool.
> Unfortunately, I think the concept of the "default" driver for mempools
> seems to exist only when using the mbuf library to create mempools. Even
> then, the default mempool is different from what the first entry in the
> list is. That's the fundamental issue here, we are using the zero-eth entry
> in the ops list, rather than a default one.

Yes, Bruce is right.

As discussed off-list with David, moving rte_mempool_set_ops_byname() from
rte_mempool_create() to rte_mempool_create_empty() would ensure that the ring
driver is the default (and taking flags into account).
  

> From: Olivier Matz [mailto:olivier.matz@6wind.com]
> Sent: Wednesday, 16 August 2023 15.17
> 
> Hi,
> 
> On Wed, Aug 16, 2023 at 01:33:46PM +0100, Bruce Richardson wrote:
> > On Wed, Aug 16, 2023 at 01:40:41PM +0200, David Marchand wrote:
> > > On Wed, Aug 16, 2023 at 1:15 PM David Marchand
> > > <david.marchand@redhat.com> wrote:
> > > >
> > > > On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> > > > <bruce.richardson@intel.com> wrote:
> > > > > These lines here seem to be exposing a bug in the mempool unit tests
> for
> > > > > shared builds, which is why we have a CI failure.
> > > > >
> > > > > The mempool unit tests use the mempool "create_empty" API, and then
> call
> > > > > the populate APIs to finish setting up the mempool. However, the
> > > > > create_empty API does not explicitly configure a particular set of
> mempool
> > > > > ops for the new mempool, leaving the ops field at 0. This means that
> unless
> > > > > the app explicitly sets other ops, the mempool will use the ops from
> > > > > whatever driver registers itself first. This occurs even when the
> driver is
> > > > > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > > > > leading to failures in setting up and using the mempool.
> > > >
> > > > Hum, it sounds like a bug to me.
> > > > The dpaa2 driver should not be registered as the default (or here,
> > > > default platform) mempool.
> > > > Other mempool drivers ensure that required hw is available, I guess
> > > > dpaa2 is missing some check.
> > >
> > > Ok, re-reading your last comment, I agree it looks like an issue in
> > > the unit test itself.
> > > Copying Olivier.
> > >
> > No, I think it's not a bug in the test, but rather in the mempool.
> > Unfortunately, I think the concept of the "default" driver for mempools
> > seems to exist only when using the mbuf library to create mempools. Even
> > then, the default mempool is different from what the first entry in the
> > list is. That's the fundamental issue here, we are using the zero-eth entry
> > in the ops list, rather than a default one.
> 
> Yes, Bruce is right.
> 
> As discussed off-list with David, moving rte_mempool_set_ops_byname() from
> rte_mempool_create() to rte_mempool_create_empty() would ensure that the ring
> driver is the default (and taking flags into account).

I took a look at the mempool library source code, and reached the same conclusion.

Your suggested fix is also supported by the documentation of the "flags" parameter to rte_mempool_create_empty(), which - by referring to the "flags" parameter to rte_mempool_create() - says that the mempool ops will be set depending on the RTE_MEMPOOL_F_[S|M][P_PUT|C_GET] flags.

It should probably be flagged as a bug and backported to stable releases.
  

On Wed, Aug 16, 2023 at 03:35:47PM +0200, Morten Brørup wrote:
> > From: Olivier Matz [mailto:olivier.matz@6wind.com]
> > Sent: Wednesday, 16 August 2023 15.17
> > 
> > Hi,
> > 
> > On Wed, Aug 16, 2023 at 01:33:46PM +0100, Bruce Richardson wrote:
> > > On Wed, Aug 16, 2023 at 01:40:41PM +0200, David Marchand wrote:
> > > > On Wed, Aug 16, 2023 at 1:15 PM David Marchand
> > > > <david.marchand@redhat.com> wrote:
> > > > >
> > > > > On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> > > > > <bruce.richardson@intel.com> wrote:
> > > > > > These lines here seem to be exposing a bug in the mempool unit tests
> > for
> > > > > > shared builds, which is why we have a CI failure.
> > > > > >
> > > > > > The mempool unit tests use the mempool "create_empty" API, and then
> > call
> > > > > > the populate APIs to finish setting up the mempool. However, the
> > > > > > create_empty API does not explicitly configure a particular set of
> > mempool
> > > > > > ops for the new mempool, leaving the ops field at 0. This means that
> > unless
> > > > > > the app explicitly sets other ops, the mempool will use the ops from
> > > > > > whatever driver registers itself first. This occurs even when the
> > driver is
> > > > > > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > > > > > leading to failures in setting up and using the mempool.
> > > > >
> > > > > Hum, it sounds like a bug to me.
> > > > > The dpaa2 driver should not be registered as the default (or here,
> > > > > default platform) mempool.
> > > > > Other mempool drivers ensure that required hw is available, I guess
> > > > > dpaa2 is missing some check.
> > > >
> > > > Ok, re-reading your last comment, I agree it looks like an issue in
> > > > the unit test itself.
> > > > Copying Olivier.
> > > >
> > > No, I think it's not a bug in the test, but rather in the mempool.
> > > Unfortunately, I think the concept of the "default" driver for mempools
> > > seems to exist only when using the mbuf library to create mempools. Even
> > > then, the default mempool is different from what the first entry in the
> > > list is. That's the fundamental issue here, we are using the zero-eth entry
> > > in the ops list, rather than a default one.
> > 
> > Yes, Bruce is right.
> > 
> > As discussed off-list with David, moving rte_mempool_set_ops_byname() from
> > rte_mempool_create() to rte_mempool_create_empty() would ensure that the ring
> > driver is the default (and taking flags into account).
> 
> I took a look at the mempool library source code, and reached the same conclusion.
> 
> Your suggested fix is also supported by the documentation of the "flags" parameter to rte_mempool_create_empty(), which - by referring to the "flags" parameter to rte_mempool_create() - says that the mempool ops will be set depending on the RTE_MEMPOOL_F_[S|M][P_PUT|C_GET] flags.
> 
> It should probably be flagged as a bug and backported to stable releases.
>
Yep. David has kindly done up a patch to fix this, and I'm rolling it into
the v6 of this set.

/Bruce
  

On Tue, Aug 15, 2023 at 5:24 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Rather than having the test suites listed out in the meson.build files
> and having to have them enabled/disabled selectively based on what libs
> are being built, pull the tests to run from the source files which were
> added to the build.
>
> Most test suites require no additional info other than the list of test
> names in the suite. However the fast-test are special that they have
> additional parameters associated with them. This requires some
> additional work in the test extraction script and in the meson.build
> file for processing the output.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> ---
>  app/meson.build                               |  7 +-
>  app/test/suites/meson.build                   | 74 +++++++++++++++++++
>  buildtools/get-test-suites.py                 | 33 +++++++++
>  .../has-hugepages.py                          |  0
>  buildtools/meson.build                        |  2 +
>  5 files changed, 115 insertions(+), 1 deletion(-)
>  create mode 100644 app/test/suites/meson.build
>  create mode 100644 buildtools/get-test-suites.py
>  rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)
>
> diff --git a/app/meson.build b/app/meson.build
> index 0d8b618e7f..c14dc80892 100644
> --- a/app/meson.build
> +++ b/app/meson.build
> @@ -101,7 +101,7 @@ foreach app:apps
>          link_libs = dpdk_static_libraries + dpdk_drivers
>      endif
>
> -    executable('dpdk-' + name,
> +    exec = executable('dpdk-' + name,
>              sources,
>              c_args: cflags,
>              link_args: ldflags,
> @@ -110,4 +110,9 @@ foreach app:apps
>              include_directories: includes,
>              install_rpath: join_paths(get_option('prefix'), driver_install_path),
>              install: true)
> +    if name == 'test'
> +        dpdk_test = exec
> +        autotest_sources = sources
> +        subdir('test/suites')  # define the pre-canned test suites
> +    endif
>  endforeach
> diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
> new file mode 100644
> index 0000000000..ec74d8adf2
> --- /dev/null
> +++ b/app/test/suites/meson.build
> @@ -0,0 +1,74 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2023 Intel Corporation
> +
> +# some perf tests (eg: memcpy perf autotest)take very long

s/)take/) take/

> +# to complete, so timeout to 10 minutes
> +timeout_seconds = 600
> +timeout_seconds_fast = 10
> +
> +test_no_huge_args = ['--no-huge', '-m', '2048']
> +has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
> +message('hugepage availability: @0@'.format(has_hugepage))
> +
> +# process source files to determine the different unit test suites
> +# - fast_tests
> +# - perf_tests
> +# - driver_tests
> +test_suites = run_command(get_test_suites_cmd, autotest_sources,
> +         check: true).stdout().strip().split()
> +foreach suite:test_suites
> +    # simple cases - tests without parameters or special handling
> +    suite = suite.split('=')
> +    suite_name = suite[0]
> +    suite_tests = suite[1].split(',')
> +    if suite_name != 'fast-tests'
> +        # simple cases - tests without parameters or special handling
> +        foreach t: suite_tests
> +            test(t, dpdk_test,
> +                    env: ['DPDK_TEST=' + t],
> +                    timeout: timeout_seconds,
> +                    is_parallel: false,
> +                    suite: suite_name)
> +        endforeach
> +    else
> +    # special fast-test handling here

Indentation of this comment is off.

> +        foreach t: suite_tests
> +            params = t.split(':')
> +            test_name = params[0]
> +            nohuge = params[1] == 'true'
> +            asan = params[2] == 'true'
> +
> +            test_args = []
> +            if nohuge
> +                test_args += test_no_huge_args
> +            elif not has_hugepage
> +                continue  #skip this tests
> +            endif
> +            if not asan and (get_option('b_sanitize') == 'address'
> +                    or get_option('b_sanitize') == 'address,undefined')
> +                continue  # skip this test
> +            endif
> +
> +            if get_option('default_library') == 'shared'
> +                test_args += ['-d', dpdk_drivers_build_dir]
> +            endif
> +
> +            test(test_name, dpdk_test,
> +                args : test_args,
> +                env: ['DPDK_TEST=' + test_name],
> +                timeout : timeout_seconds_fast,
> +                is_parallel : false,
> +                suite : 'fast-tests')
> +            if not is_windows and test_name == 'trace_autotest'
> +                test_args += ['--trace=.*']
> +                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
> +                test(test_name + '_with_traces', dpdk_test,
> +                    args : test_args,
> +                    env: ['DPDK_TEST=' + test_name],
> +                    timeout : timeout_seconds_fast,
> +                    is_parallel : false,
> +                    suite : 'fast-tests')
> +            endif
> +        endforeach
> +    endif
> +endforeach
> diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
> new file mode 100644
> index 0000000000..95a9cad4c8
> --- /dev/null
> +++ b/buildtools/get-test-suites.py
> @@ -0,0 +1,33 @@
> +#! /usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2023 Intel Corporation
> +
> +import sys
> +import re
> +
> +input_list = sys.argv[1:]
> +test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
> +test_suites = {}
> +
> +def get_fast_test_params(test_name, ln):
> +    "Extract the extra fast-test parameters from the line"
> +    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
> +    (_, rest_of_line) = ln.split(test_name, 1)
> +    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
> +    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
> +
> +for fname in input_list:
> +    with open(fname) as f:
> +        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
> +    for ln in contents:
> +        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
> +        suite_name = f"{test_suite.lower()}-tests"
> +        if suite_name in test_suites:
> +            test_suites[suite_name].append(test_name)
> +        else:
> +            test_suites[suite_name] = [test_name]
> +        if suite_name == "fast-tests":
> +            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
> +
> +for suite in test_suites.keys():
> +    print(f"{suite}={','.join(test_suites[suite])}")
> diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
> similarity index 100%
> rename from app/test/has_hugepage.py
> rename to buildtools/has-hugepages.py

app/test/has_hugepage.py is still referenced in MAINTAINERS.
And new files (in this patch, and later patches) are not affiliated to someone.

> diff --git a/buildtools/meson.build b/buildtools/meson.build
> index e1c600e40f..ac5e4dcf08 100644
> --- a/buildtools/meson.build
> +++ b/buildtools/meson.build
> @@ -18,6 +18,8 @@ map_to_win_cmd = py3 + files('map_to_win.py')
>  sphinx_wrapper = py3 + files('call-sphinx-build.py')
>  get_cpu_count_cmd = py3 + files('get-cpu-count.py')
>  get_numa_count_cmd = py3 + files('get-numa-count.py')
> +get_test_suites_cmd = py3 + files('get-test-suites.py')
> +has_hugepages_cmd = py3 + files('has-hugepages.py')
>  binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
>                          [objdump] + cc.cmd_array())
>
> --
> 2.39.2
>
  

On Wed, Aug 16, 2023 at 04:57:07PM +0200, David Marchand wrote:
> On Tue, Aug 15, 2023 at 5:24 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Rather than having the test suites listed out in the meson.build files
> > and having to have them enabled/disabled selectively based on what libs
> > are being built, pull the tests to run from the source files which were
> > added to the build.
> >
> > Most test suites require no additional info other than the list of test
> > names in the suite. However the fast-test are special that they have
> > additional parameters associated with them. This requires some
> > additional work in the test extraction script and in the meson.build
> > file for processing the output.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Morten Brørup <mb@smartsharesystems.com>
> > ---
> >  app/meson.build                               |  7 +-
> >  app/test/suites/meson.build                   | 74 +++++++++++++++++++
> >  buildtools/get-test-suites.py                 | 33 +++++++++
> >  .../has-hugepages.py                          |  0
> >  buildtools/meson.build                        |  2 +
> >  5 files changed, 115 insertions(+), 1 deletion(-)
> >  create mode 100644 app/test/suites/meson.build
> >  create mode 100644 buildtools/get-test-suites.py
> >  rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)
> >
> > diff --git a/app/meson.build b/app/meson.build
> > index 0d8b618e7f..c14dc80892 100644
> > --- a/app/meson.build
> > +++ b/app/meson.build
> > @@ -101,7 +101,7 @@ foreach app:apps
> >          link_libs = dpdk_static_libraries + dpdk_drivers
> >      endif
> >
> > -    executable('dpdk-' + name,
> > +    exec = executable('dpdk-' + name,
> >              sources,
> >              c_args: cflags,
> >              link_args: ldflags,
> > @@ -110,4 +110,9 @@ foreach app:apps
> >              include_directories: includes,
> >              install_rpath: join_paths(get_option('prefix'), driver_install_path),
> >              install: true)
> > +    if name == 'test'
> > +        dpdk_test = exec
> > +        autotest_sources = sources
> > +        subdir('test/suites')  # define the pre-canned test suites
> > +    endif
> >  endforeach
> > diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
> > new file mode 100644
> > index 0000000000..ec74d8adf2
> > --- /dev/null
> > +++ b/app/test/suites/meson.build
> > @@ -0,0 +1,74 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2023 Intel Corporation
> > +
> > +# some perf tests (eg: memcpy perf autotest)take very long
> 
> s/)take/) take/
> 
Ack

> > +# to complete, so timeout to 10 minutes
> > +timeout_seconds = 600
> > +timeout_seconds_fast = 10
> > +
> > +test_no_huge_args = ['--no-huge', '-m', '2048']
> > +has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
> > +message('hugepage availability: @0@'.format(has_hugepage))
> > +
> > +# process source files to determine the different unit test suites
> > +# - fast_tests
> > +# - perf_tests
> > +# - driver_tests
> > +test_suites = run_command(get_test_suites_cmd, autotest_sources,
> > +         check: true).stdout().strip().split()
> > +foreach suite:test_suites
> > +    # simple cases - tests without parameters or special handling
> > +    suite = suite.split('=')
> > +    suite_name = suite[0]
> > +    suite_tests = suite[1].split(',')
> > +    if suite_name != 'fast-tests'
> > +        # simple cases - tests without parameters or special handling
> > +        foreach t: suite_tests
> > +            test(t, dpdk_test,
> > +                    env: ['DPDK_TEST=' + t],
> > +                    timeout: timeout_seconds,
> > +                    is_parallel: false,
> > +                    suite: suite_name)
> > +        endforeach
> > +    else
> > +    # special fast-test handling here
> 
> Indentation of this comment is off.
>
Ack
 
> > +        foreach t: suite_tests
> > +            params = t.split(':')
> > +            test_name = params[0]
> > +            nohuge = params[1] == 'true'
> > +            asan = params[2] == 'true'
> > +
> > +            test_args = []
> > +            if nohuge
> > +                test_args += test_no_huge_args
> > +            elif not has_hugepage
> > +                continue  #skip this tests
> > +            endif
> > +            if not asan and (get_option('b_sanitize') == 'address'
> > +                    or get_option('b_sanitize') == 'address,undefined')
> > +                continue  # skip this test
> > +            endif
> > +
> > +            if get_option('default_library') == 'shared'
> > +                test_args += ['-d', dpdk_drivers_build_dir]
> > +            endif
> > +
> > +            test(test_name, dpdk_test,
> > +                args : test_args,
> > +                env: ['DPDK_TEST=' + test_name],
> > +                timeout : timeout_seconds_fast,
> > +                is_parallel : false,
> > +                suite : 'fast-tests')
> > +            if not is_windows and test_name == 'trace_autotest'
> > +                test_args += ['--trace=.*']
> > +                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
> > +                test(test_name + '_with_traces', dpdk_test,
> > +                    args : test_args,
> > +                    env: ['DPDK_TEST=' + test_name],
> > +                    timeout : timeout_seconds_fast,
> > +                    is_parallel : false,
> > +                    suite : 'fast-tests')
> > +            endif
> > +        endforeach
> > +    endif
> > +endforeach
> > diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
> > new file mode 100644
> > index 0000000000..95a9cad4c8
> > --- /dev/null
> > +++ b/buildtools/get-test-suites.py
> > @@ -0,0 +1,33 @@
> > +#! /usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2023 Intel Corporation
> > +
> > +import sys
> > +import re
> > +
> > +input_list = sys.argv[1:]
> > +test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
> > +test_suites = {}
> > +
> > +def get_fast_test_params(test_name, ln):
> > +    "Extract the extra fast-test parameters from the line"
> > +    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
> > +    (_, rest_of_line) = ln.split(test_name, 1)
> > +    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
> > +    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
> > +
> > +for fname in input_list:
> > +    with open(fname) as f:
> > +        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
> > +    for ln in contents:
> > +        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
> > +        suite_name = f"{test_suite.lower()}-tests"
> > +        if suite_name in test_suites:
> > +            test_suites[suite_name].append(test_name)
> > +        else:
> > +            test_suites[suite_name] = [test_name]
> > +        if suite_name == "fast-tests":
> > +            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
> > +
> > +for suite in test_suites.keys():
> > +    print(f"{suite}={','.join(test_suites[suite])}")
> > diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
> > similarity index 100%
> > rename from app/test/has_hugepage.py
> > rename to buildtools/has-hugepages.py
> 
> app/test/has_hugepage.py is still referenced in MAINTAINERS.
> And new files (in this patch, and later patches) are not affiliated to someone.
>
Will look to fix in V6.