diff mbox series

[v1] ci: add test suite run without hugepage

Message ID 20200225073236.135581-1-ruifeng.wang@arm.com (mailing list archive)
State Superseded, archived
Delegated to: David Marchand
Headers
Series [v1] ci: add test suite run without hugepage |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/iol-testing success Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS
ci/Intel-compilation success Compilation OK
ci/travis-robot success Travis build: passed

Commit Message

Ruifeng Wang Feb. 25, 2020, 7:32 a.m. UTC 
  This test suite is derived from fast-tests suite. Cases in this
suite are run with '--no-huge' flag.

The suite aims to cover as many as possible test cases out of the
fast-tests suites in the environments without huge pages support,
like containers.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .ci/linux-build.sh   |  4 +++
 .travis.yml          | 12 +++++++
 app/test/meson.build | 75 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 91 insertions(+)

Comments

David Marchand Feb. 25, 2020, 9:14 a.m. UTC | #1 
On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>
> This test suite is derived from fast-tests suite. Cases in this
> suite are run with '--no-huge' flag.
>
> The suite aims to cover as many as possible test cases out of the
> fast-tests suites in the environments without huge pages support,
> like containers.
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>

Compilation time is what makes the most of a "tests" job in Travis.
So I'd prefer we avoid adding more jobs for x86_64 (see below).


> ---
>  .ci/linux-build.sh   |  4 +++
>  .travis.yml          | 12 +++++++
>  app/test/meson.build | 75 ++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 91 insertions(+)
>
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> index d500c4c00..39515d915 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -92,3 +92,7 @@ fi
>  if [ "$RUN_TESTS" = "1" ]; then
>      sudo meson test -C build --suite fast-tests -t 3
>  fi
> +
> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
> +    sudo meson test -C build --suite nohuge-tests -t 3
> +fi

You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
that contains a list of testsuites.
Then this part becomes:

for testsuite in ${TESTSUITES:-}; do
   sudo meson test -C build --suite $testsuite -t 3
done

(I wonder if we reaaaally need to be root to run those tests w/ and
w/o hugepages, Aaron?)


> diff --git a/.travis.yml b/.travis.yml
> index b64a81bd0..0e07d52d0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -43,6 +43,9 @@ jobs:
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: gcc
> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> +    arch: amd64
> +    compiler: gcc

And then we only need to update the existing RUN_TESTS jobs for x86_64.


>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: gcc
> @@ -66,6 +69,9 @@ jobs:
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: clang
> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> +    arch: amd64
> +    compiler: clang
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: clang
> @@ -101,6 +107,9 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: arm64
>      compiler: gcc
> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> +    arch: arm64
> +    compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
>      compiler: gcc
> @@ -124,3 +133,6 @@ jobs:
>    - env: DEF_LIB="shared"
>      arch: arm64
>      compiler: clang
> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> +    arch: arm64
> +    compiler: clang


--
David Marchand
Aaron Conole Feb. 25, 2020, 2:36 p.m. UTC | #2 
David Marchand <david.marchand@redhat.com> writes:

> On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>>
>> This test suite is derived from fast-tests suite. Cases in this
>> suite are run with '--no-huge' flag.
>>
>> The suite aims to cover as many as possible test cases out of the
>> fast-tests suites in the environments without huge pages support,
>> like containers.
>>
>> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>
> Compilation time is what makes the most of a "tests" job in Travis.
> So I'd prefer we avoid adding more jobs for x86_64 (see below).
>
>
>> ---
>>  .ci/linux-build.sh   |  4 +++
>>  .travis.yml          | 12 +++++++
>>  app/test/meson.build | 75 ++++++++++++++++++++++++++++++++++++++++++++
>>  3 files changed, 91 insertions(+)
>>
>> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
>> index d500c4c00..39515d915 100755
>> --- a/.ci/linux-build.sh
>> +++ b/.ci/linux-build.sh
>> @@ -92,3 +92,7 @@ fi
>>  if [ "$RUN_TESTS" = "1" ]; then
>>      sudo meson test -C build --suite fast-tests -t 3
>>  fi
>> +
>> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
>> +    sudo meson test -C build --suite nohuge-tests -t 3
>> +fi
>
> You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
> that contains a list of testsuites.
> Then this part becomes:
>
> for testsuite in ${TESTSUITES:-}; do
>    sudo meson test -C build --suite $testsuite -t 3
> done
>
> (I wonder if we reaaaally need to be root to run those tests w/ and
> w/o hugepages, Aaron?)

Last I tested, we didn't need root access when not using hugepages (at
least for core library functionality).  It's possible that the test
suites for PMDs might need that access, but we don't run them in the
travis environment.

>
>> diff --git a/.travis.yml b/.travis.yml
>> index b64a81bd0..0e07d52d0 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -43,6 +43,9 @@ jobs:
>>    - env: DEF_LIB="shared" RUN_TESTS=1
>>      arch: amd64
>>      compiler: gcc
>> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> +    arch: amd64
>> +    compiler: gcc
>
> And then we only need to update the existing RUN_TESTS jobs for x86_64.

I like the suggestion.  But I guess we'd just make RUN_TESTS=1 set
TESTSUITES="..." and otherwise, we can have an ARM64 test to do all the
test suites.

Actually, the ideal would be for the tests to automatically SKIP when no
hugepages are enabled.

>
>>    - env: DEF_LIB="shared" BUILD_DOCS=1
>>      arch: amd64
>>      compiler: gcc
>> @@ -66,6 +69,9 @@ jobs:
>>    - env: DEF_LIB="shared" RUN_TESTS=1
>>      arch: amd64
>>      compiler: clang
>> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> +    arch: amd64
>> +    compiler: clang
>>    - env: DEF_LIB="shared" BUILD_DOCS=1
>>      arch: amd64
>>      compiler: clang
>> @@ -101,6 +107,9 @@ jobs:
>>    - env: DEF_LIB="static"
>>      arch: arm64
>>      compiler: gcc
>> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> +    arch: arm64
>> +    compiler: gcc
>>    - env: DEF_LIB="shared" BUILD_DOCS=1
>>      arch: arm64
>>      compiler: gcc
>> @@ -124,3 +133,6 @@ jobs:
>>    - env: DEF_LIB="shared"
>>      arch: arm64
>>      compiler: clang
>> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> +    arch: arm64
>> +    compiler: clang
>
>
> --
> David Marchand
Ruifeng Wang Feb. 25, 2020, 3:23 p.m. UTC | #3 
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, February 25, 2020 17:15
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: Aaron Conole <aconole@redhat.com>; Michael Santana
> <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> 
> On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> >
> > This test suite is derived from fast-tests suite. Cases in this suite
> > are run with '--no-huge' flag.
> >
> > The suite aims to cover as many as possible test cases out of the
> > fast-tests suites in the environments without huge pages support, like
> > containers.
> >
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> 
> Compilation time is what makes the most of a "tests" job in Travis.
> So I'd prefer we avoid adding more jobs for x86_64 (see below).
> 
> 
> > ---
> >  .ci/linux-build.sh   |  4 +++
> >  .travis.yml          | 12 +++++++
> >  app/test/meson.build | 75
> > ++++++++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 91 insertions(+)
> >
> > diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
> > d500c4c00..39515d915 100755
> > --- a/.ci/linux-build.sh
> > +++ b/.ci/linux-build.sh
> > @@ -92,3 +92,7 @@ fi
> >  if [ "$RUN_TESTS" = "1" ]; then
> >      sudo meson test -C build --suite fast-tests -t 3  fi
> > +
> > +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
> > +    sudo meson test -C build --suite nohuge-tests -t 3 fi
> 
> You can replace the "boolean" RUN_TESTS with a TESTSUITES variable that
> contains a list of testsuites.
> Then this part becomes:
> 
> for testsuite in ${TESTSUITES:-}; do
>    sudo meson test -C build --suite $testsuite -t 3 done
> 
Thanks for the suggestion. 
A good way to consolidate test jobs.
I will follow this way in new version.

> (I wonder if we reaaaally need to be root to run those tests w/ and w/o
> hugepages, Aaron?)
> 
> 
> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..0e07d52d0
> > 100644
> > --- a/.travis.yml
> > +++ b/.travis.yml
> > @@ -43,6 +43,9 @@ jobs:
> >    - env: DEF_LIB="shared" RUN_TESTS=1
> >      arch: amd64
> >      compiler: gcc
> > +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> > +    arch: amd64
> > +    compiler: gcc
> 
> And then we only need to update the existing RUN_TESTS jobs for x86_64.
> 
> 
> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >      arch: amd64
> >      compiler: gcc
> > @@ -66,6 +69,9 @@ jobs:
> >    - env: DEF_LIB="shared" RUN_TESTS=1
> >      arch: amd64
> >      compiler: clang
> > +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> > +    arch: amd64
> > +    compiler: clang
> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >      arch: amd64
> >      compiler: clang
> > @@ -101,6 +107,9 @@ jobs:
> >    - env: DEF_LIB="static"
> >      arch: arm64
> >      compiler: gcc
> > +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> > +    arch: arm64
> > +    compiler: gcc
> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >      arch: arm64
> >      compiler: gcc
> > @@ -124,3 +133,6 @@ jobs:
> >    - env: DEF_LIB="shared"
> >      arch: arm64
> >      compiler: clang
> > +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> > +    arch: arm64
> > +    compiler: clang
> 
> 
> --
> David Marchand
Ruifeng Wang Feb. 25, 2020, 3:39 p.m. UTC | #4 
> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Tuesday, February 25, 2020 22:36
> To: David Marchand <david.marchand@redhat.com>
> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana
> <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> 
> David Marchand <david.marchand@redhat.com> writes:
> 
> > On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> >>
> >> This test suite is derived from fast-tests suite. Cases in this suite
> >> are run with '--no-huge' flag.
> >>
> >> The suite aims to cover as many as possible test cases out of the
> >> fast-tests suites in the environments without huge pages support,
> >> like containers.
> >>
> >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >
> > Compilation time is what makes the most of a "tests" job in Travis.
> > So I'd prefer we avoid adding more jobs for x86_64 (see below).
> >
> >
> >> ---
> >>  .ci/linux-build.sh   |  4 +++
> >>  .travis.yml          | 12 +++++++
> >>  app/test/meson.build | 75
> >> ++++++++++++++++++++++++++++++++++++++++++++
> >>  3 files changed, 91 insertions(+)
> >>
> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
> >> d500c4c00..39515d915 100755
> >> --- a/.ci/linux-build.sh
> >> +++ b/.ci/linux-build.sh
> >> @@ -92,3 +92,7 @@ fi
> >>  if [ "$RUN_TESTS" = "1" ]; then
> >>      sudo meson test -C build --suite fast-tests -t 3  fi
> >> +
> >> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
> >> +    sudo meson test -C build --suite nohuge-tests -t 3 fi
> >
> > You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
> > that contains a list of testsuites.
> > Then this part becomes:
> >
> > for testsuite in ${TESTSUITES:-}; do
> >    sudo meson test -C build --suite $testsuite -t 3 done
> >
> > (I wonder if we reaaaally need to be root to run those tests w/ and
> > w/o hugepages, Aaron?)
> 
> Last I tested, we didn't need root access when not using hugepages (at least
> for core library functionality).  It's possible that the test suites for PMDs might
> need that access, but we don't run them in the travis environment.
> 
> >
> >> diff --git a/.travis.yml b/.travis.yml index b64a81bd0..0e07d52d0
> >> 100644
> >> --- a/.travis.yml
> >> +++ b/.travis.yml
> >> @@ -43,6 +43,9 @@ jobs:
> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >>      arch: amd64
> >>      compiler: gcc
> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> +    arch: amd64
> >> +    compiler: gcc
> >
> > And then we only need to update the existing RUN_TESTS jobs for x86_64.
> 
> I like the suggestion.  But I guess we'd just make RUN_TESTS=1 set
> TESTSUITES="..." and otherwise, we can have an ARM64 test to do all the test
> suites.
It should be OK to do all applicable test suites in a single job?

> 
> Actually, the ideal would be for the tests to automatically SKIP when no
> hugepages are enabled.
In this way, case will run either w/ hugepage or w/o hugepage but not both.
It will run faster, but with less coverage?

> 
> >
> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >>      arch: amd64
> >>      compiler: gcc
> >> @@ -66,6 +69,9 @@ jobs:
> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >>      arch: amd64
> >>      compiler: clang
> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> +    arch: amd64
> >> +    compiler: clang
> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >>      arch: amd64
> >>      compiler: clang
> >> @@ -101,6 +107,9 @@ jobs:
> >>    - env: DEF_LIB="static"
> >>      arch: arm64
> >>      compiler: gcc
> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> +    arch: arm64
> >> +    compiler: gcc
> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >>      arch: arm64
> >>      compiler: gcc
> >> @@ -124,3 +133,6 @@ jobs:
> >>    - env: DEF_LIB="shared"
> >>      arch: arm64
> >>      compiler: clang
> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> +    arch: arm64
> >> +    compiler: clang
> >
> >
> > --
> > David Marchand
Aaron Conole Feb. 25, 2020, 3:56 p.m. UTC | #5 
Ruifeng Wang <Ruifeng.Wang@arm.com> writes:

>> -----Original Message-----
>> From: Aaron Conole <aconole@redhat.com>
>> Sent: Tuesday, February 25, 2020 22:36
>> To: David Marchand <david.marchand@redhat.com>
>> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana
>> <maicolgabriel@hotmail.com>; Bruce Richardson
>> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
>> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
>> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
>> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
>> 
>> David Marchand <david.marchand@redhat.com> writes:
>> 
>> > On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com>
>> wrote:
>> >>
>> >> This test suite is derived from fast-tests suite. Cases in this suite
>> >> are run with '--no-huge' flag.
>> >>
>> >> The suite aims to cover as many as possible test cases out of the
>> >> fast-tests suites in the environments without huge pages support,
>> >> like containers.
>> >>
>> >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>> >
>> > Compilation time is what makes the most of a "tests" job in Travis.
>> > So I'd prefer we avoid adding more jobs for x86_64 (see below).
>> >
>> >
>> >> ---
>> >>  .ci/linux-build.sh   |  4 +++
>> >>  .travis.yml          | 12 +++++++
>> >>  app/test/meson.build | 75
>> >> ++++++++++++++++++++++++++++++++++++++++++++
>> >>  3 files changed, 91 insertions(+)
>> >>
>> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
>> >> d500c4c00..39515d915 100755
>> >> --- a/.ci/linux-build.sh
>> >> +++ b/.ci/linux-build.sh
>> >> @@ -92,3 +92,7 @@ fi
>> >>  if [ "$RUN_TESTS" = "1" ]; then
>> >>      sudo meson test -C build --suite fast-tests -t 3  fi
>> >> +
>> >> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
>> >> +    sudo meson test -C build --suite nohuge-tests -t 3 fi
>> >
>> > You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
>> > that contains a list of testsuites.
>> > Then this part becomes:
>> >
>> > for testsuite in ${TESTSUITES:-}; do
>> >    sudo meson test -C build --suite $testsuite -t 3 done
>> >
>> > (I wonder if we reaaaally need to be root to run those tests w/ and
>> > w/o hugepages, Aaron?)
>> 
>> Last I tested, we didn't need root access when not using hugepages (at least
>> for core library functionality).  It's possible that the test suites for PMDs might
>> need that access, but we don't run them in the travis environment.
>> 
>> >
>> >> diff --git a/.travis.yml b/.travis.yml index b64a81bd0..0e07d52d0
>> >> 100644
>> >> --- a/.travis.yml
>> >> +++ b/.travis.yml
>> >> @@ -43,6 +43,9 @@ jobs:
>> >>    - env: DEF_LIB="shared" RUN_TESTS=1
>> >>      arch: amd64
>> >>      compiler: gcc
>> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> >> +    arch: amd64
>> >> +    compiler: gcc
>> >
>> > And then we only need to update the existing RUN_TESTS jobs for x86_64.
>> 
>> I like the suggestion.  But I guess we'd just make RUN_TESTS=1 set
>> TESTSUITES="..." and otherwise, we can have an ARM64 test to do all the test
>> suites.
> It should be OK to do all applicable test suites in a single job?

Yes, I think so.

>> 
>> Actually, the ideal would be for the tests to automatically SKIP when no
>> hugepages are enabled.
> In this way, case will run either w/ hugepage or w/o hugepage but not
> both.

Hrrm?  I don't understand.  If there are hugepages configured, all the
tests will run.  If there are no hugepages, only the tests that can pass
without hugepages will run.

> It will run faster, but with less coverage?

Sortof - if there aren't hugepages configured for an environment
reason (like unsupported) why get failure reports?  Better to either
fail when we try to setup the hugepages, or skip when none are available
because they are two separate concerns.

>> 
>> >
>> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
>> >>      arch: amd64
>> >>      compiler: gcc
>> >> @@ -66,6 +69,9 @@ jobs:
>> >>    - env: DEF_LIB="shared" RUN_TESTS=1
>> >>      arch: amd64
>> >>      compiler: clang
>> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> >> +    arch: amd64
>> >> +    compiler: clang
>> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
>> >>      arch: amd64
>> >>      compiler: clang
>> >> @@ -101,6 +107,9 @@ jobs:
>> >>    - env: DEF_LIB="static"
>> >>      arch: arm64
>> >>      compiler: gcc
>> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> >> +    arch: arm64
>> >> +    compiler: gcc
>> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
>> >>      arch: arm64
>> >>      compiler: gcc
>> >> @@ -124,3 +133,6 @@ jobs:
>> >>    - env: DEF_LIB="shared"
>> >>      arch: arm64
>> >>      compiler: clang
>> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> >> +    arch: arm64
>> >> +    compiler: clang
>> >
>> >
>> > --
>> > David Marchand
Ruifeng Wang Feb. 26, 2020, 2:47 a.m. UTC | #6 
> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Tuesday, February 25, 2020 23:57
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: David Marchand <david.marchand@redhat.com>; Michael Santana
> <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> 
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> 
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Tuesday, February 25, 2020 22:36
> >> To: David Marchand <david.marchand@redhat.com>
> >> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana
> >> <maicolgabriel@hotmail.com>; Bruce Richardson
> >> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> >> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> >> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> >> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> >>
> >> David Marchand <david.marchand@redhat.com> writes:
> >>
> >> > On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang
> <ruifeng.wang@arm.com>
> >> wrote:
> >> >>
> >> >> This test suite is derived from fast-tests suite. Cases in this
> >> >> suite are run with '--no-huge' flag.
> >> >>
> >> >> The suite aims to cover as many as possible test cases out of the
> >> >> fast-tests suites in the environments without huge pages support,
> >> >> like containers.
> >> >>
> >> >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> >
> >> > Compilation time is what makes the most of a "tests" job in Travis.
> >> > So I'd prefer we avoid adding more jobs for x86_64 (see below).
> >> >
> >> >
> >> >> ---
> >> >>  .ci/linux-build.sh   |  4 +++
> >> >>  .travis.yml          | 12 +++++++
> >> >>  app/test/meson.build | 75
> >> >> ++++++++++++++++++++++++++++++++++++++++++++
> >> >>  3 files changed, 91 insertions(+)
> >> >>
> >> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
> >> >> d500c4c00..39515d915 100755
> >> >> --- a/.ci/linux-build.sh
> >> >> +++ b/.ci/linux-build.sh
> >> >> @@ -92,3 +92,7 @@ fi
> >> >>  if [ "$RUN_TESTS" = "1" ]; then
> >> >>      sudo meson test -C build --suite fast-tests -t 3  fi
> >> >> +
> >> >> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
> >> >> +    sudo meson test -C build --suite nohuge-tests -t 3 fi
> >> >
> >> > You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
> >> > that contains a list of testsuites.
> >> > Then this part becomes:
> >> >
> >> > for testsuite in ${TESTSUITES:-}; do
> >> >    sudo meson test -C build --suite $testsuite -t 3 done
> >> >
> >> > (I wonder if we reaaaally need to be root to run those tests w/ and
> >> > w/o hugepages, Aaron?)
> >>
> >> Last I tested, we didn't need root access when not using hugepages
> >> (at least for core library functionality).  It's possible that the
> >> test suites for PMDs might need that access, but we don't run them in the
> travis environment.
> >>
> >> >
> >> >> diff --git a/.travis.yml b/.travis.yml index b64a81bd0..0e07d52d0
> >> >> 100644
> >> >> --- a/.travis.yml
> >> >> +++ b/.travis.yml
> >> >> @@ -43,6 +43,9 @@ jobs:
> >> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >> >>      arch: amd64
> >> >>      compiler: gcc
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: amd64
> >> >> +    compiler: gcc
> >> >
> >> > And then we only need to update the existing RUN_TESTS jobs for
> x86_64.
> >>
> >> I like the suggestion.  But I guess we'd just make RUN_TESTS=1 set
> >> TESTSUITES="..." and otherwise, we can have an ARM64 test to do all
> >> the test suites.
> > It should be OK to do all applicable test suites in a single job?
> 
> Yes, I think so.
> 
> >>
> >> Actually, the ideal would be for the tests to automatically SKIP when
> >> no hugepages are enabled.
> > In this way, case will run either w/ hugepage or w/o hugepage but not
> > both.
> 
> Hrrm?  I don't understand.  If there are hugepages configured, all the tests
> will run.  If there are no hugepages, only the tests that can pass without
> hugepages will run.
> 
I was wondering whether it makes sense to do tests with "--no-huge" in environment where hugepages are enabled.

> > It will run faster, but with less coverage?
> 
> Sortof - if there aren't hugepages configured for an environment reason (like
> unsupported) why get failure reports?  Better to either fail when we try to
> setup the hugepages, or skip when none are available because they are two
> separate concerns.
> 
> >>
> >> >
> >> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >>      arch: amd64
> >> >>      compiler: gcc
> >> >> @@ -66,6 +69,9 @@ jobs:
> >> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >> >>      arch: amd64
> >> >>      compiler: clang
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: amd64
> >> >> +    compiler: clang
> >> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >>      arch: amd64
> >> >>      compiler: clang
> >> >> @@ -101,6 +107,9 @@ jobs:
> >> >>    - env: DEF_LIB="static"
> >> >>      arch: arm64
> >> >>      compiler: gcc
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: arm64
> >> >> +    compiler: gcc
> >> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >>      arch: arm64
> >> >>      compiler: gcc
> >> >> @@ -124,3 +133,6 @@ jobs:
> >> >>    - env: DEF_LIB="shared"
> >> >>      arch: arm64
> >> >>      compiler: clang
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: arm64
> >> >> +    compiler: clang
> >> >
> >> >
> >> > --
> >> > David Marchand
diff mbox series

Patch

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index d500c4c00..39515d915 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -92,3 +92,7 @@  fi
 if [ "$RUN_TESTS" = "1" ]; then
     sudo meson test -C build --suite fast-tests -t 3
 fi
+
+if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
+    sudo meson test -C build --suite nohuge-tests -t 3
+fi
diff --git a/.travis.yml b/.travis.yml
index b64a81bd0..0e07d52d0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -43,6 +43,9 @@  jobs:
   - env: DEF_LIB="shared" RUN_TESTS=1
     arch: amd64
     compiler: gcc
+  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
+    arch: amd64
+    compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: amd64
     compiler: gcc
@@ -66,6 +69,9 @@  jobs:
   - env: DEF_LIB="shared" RUN_TESTS=1
     arch: amd64
     compiler: clang
+  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
+    arch: amd64
+    compiler: clang
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: amd64
     compiler: clang
@@ -101,6 +107,9 @@  jobs:
   - env: DEF_LIB="static"
     arch: arm64
     compiler: gcc
+  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
+    arch: arm64
+    compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: arm64
     compiler: gcc
@@ -124,3 +133,6 @@  jobs:
   - env: DEF_LIB="shared"
     arch: arm64
     compiler: clang
+  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
+    arch: arm64
+    compiler: clang
diff --git a/app/test/meson.build b/app/test/meson.build
index 0a2ce710f..43a23d351 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -237,6 +237,64 @@  fast_test_names = [
         'thash_autotest',
 ]
 
+nohuge_test_names = [
+        'alarm_autotest',
+        'byteorder_autotest',
+        'cmdline_autotest',
+        'common_autotest',
+        'cpuflags_autotest',
+        'cycles_autotest',
+        'debug_autotest',
+        'eal_flags_n_opt_autotest',
+        'eal_flags_no_huge_autotest',
+        'eal_flags_vdev_opt_autotest',
+        'eal_fs_autotest',
+        'errno_autotest',
+        'event_ring_autotest',
+        'fib_autotest',
+        'fib6_autotest',
+        'interrupt_autotest',
+        'logs_autotest',
+        'lpm_autotest',
+        'lpm6_autotest',
+        'mbuf_autotest',
+        'memcpy_autotest',
+        'memory_autotest',
+        'meter_autotest',
+        'multiprocess_autotest',
+        'per_lcore_autotest',
+        'prefetch_autotest',
+        'rcu_qsbr_autotest',
+        'red_autotest',
+        'rib_autotest',
+        'rib6_autotest',
+        'ring_autotest',
+        'rwlock_rda_autotest',
+        'rwlock_rds_wrm_autotest',
+        'rwlock_rde_wro_autotest',
+        'sched_autotest',
+        'spinlock_autotest',
+        'string_autotest',
+        'tailq_autotest',
+        'user_delay_us',
+        'version_autotest',
+        'crc_autotest',
+        'delay_us_sleep_autotest',
+        'eventdev_common_autotest',
+        'fbarray_autotest',
+        'ipsec_autotest',
+        'kni_autotest',
+        'kvargs_autotest',
+        'member_autotest',
+        'metrics_autotest',
+        'power_cpufreq_autotest',
+        'power_autotest',
+        'power_kvm_vm_autotest',
+        'reorder_autotest',
+        'service_autotest',
+        'thash_autotest',
+]
+
 perf_test_names = [
         'ring_perf_autotest',
         'mempool_perf_autotest',
@@ -341,6 +399,10 @@  if dpdk_conf.has('RTE_LIBRTE_RING_PMD')
 	fast_test_names += 'latencystats_autotest'
 	driver_test_names += 'link_bonding_mode4_autotest'
 	fast_test_names += 'pdump_autotest'
+	nohuge_test_names += 'ring_pmd_autotest'
+	nohuge_test_names += 'bitratestats_autotest'
+	nohuge_test_names += 'latencystats_autotest'
+	nohuge_test_names += 'pdump_autotest'
 endif
 
 if dpdk_conf.has('RTE_LIBRTE_POWER')
@@ -430,6 +492,19 @@  foreach arg : fast_test_names
 	endif
 endforeach
 
+foreach arg : nohuge_test_names
+	if host_machine.system() == 'linux'
+		test(arg, dpdk_test,
+			  env : ['DPDK_TEST=' + arg],
+			  args : test_args +
+				 ['--no-huge'] + ['-m 1024'] +
+				 ['--file-prefix=@0@'.format(arg)],
+		timeout : timeout_seconds_fast,
+		is_parallel : false,
+		suite : 'nohuge-tests')
+	endif
+endforeach
+
 foreach arg : perf_test_names
 	test(arg, dpdk_test,
 	env : ['DPDK_TEST=' + arg],