[4/4] ci: reorganise Travis jobs

  Let's prune the jobs list to limit the amount of time spent by the robot
in Travis.

Since meson enables automatically the relevant components, there is not
much gain in testing with extra_packages vs required_packages only.

For a given arch/compiler/env combination, compilation is first tested
in all jobs that run tests or build the docs or run the ABI checks.
In the same context, for jobs that accumulates running tests, building
the docs etc..., those steps are independent and can be split to save
some cpu on Travis.

With this, we go down from 21 to 15 jobs.

Note: this patch requires a flush of the existing caches in Travis.

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 .travis.yml | 51 +++++++++------------------------------------------
 1 file changed, 9 insertions(+), 42 deletions(-)
  

David Marchand <david.marchand@redhat.com> writes:

> Let's prune the jobs list to limit the amount of time spent by the robot
> in Travis.
>
> Since meson enables automatically the relevant components, there is not
> much gain in testing with extra_packages vs required_packages only.
>
> For a given arch/compiler/env combination, compilation is first tested
> in all jobs that run tests or build the docs or run the ABI checks.
> In the same context, for jobs that accumulates running tests, building
> the docs etc..., those steps are independent and can be split to save
> some cpu on Travis.
>
> With this, we go down from 21 to 15 jobs.
>
> Note: this patch requires a flush of the existing caches in Travis.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

In general, I think the idea with required vs. extra was to have a build
that did the minimum required, and one that did all the packages (to
allow a minimum vs. full DPDK).

At least, that's from
http://mails.dpdk.org/archives/dev/2019-January/124007.html

Not sure if that's still something anyone cares about.

>  .travis.yml | 51 +++++++++------------------------------------------
>  1 file changed, 9 insertions(+), 42 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index c174d5f30..b64a81bd0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -15,15 +15,12 @@ addons:
>      update: true
>      packages: &required_packages
>        - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip, ninja-build]
> +      - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev, libfdt-dev, libjansson4]
>
>  _aarch64_packages: &aarch64_packages
>    - *required_packages
>    - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross, pkg-config-aarch64-linux-gnu]
>  
> -_extra_packages: &extra_packages
> -  - *required_packages
> -  - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev, libfdt-dev, libjansson4]
> -
>  _libabigail_build_packages: &libabigail_build_packages
>    - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
>  
> @@ -43,66 +40,39 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="shared"
> -    arch: amd64
> -    compiler: gcc
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="static" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: gcc
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *doc_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      arch: amd64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *libabigail_build_packages
>    # x86_64 clang jobs
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="shared"
> -    arch: amd64
> -    compiler: clang
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="static" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: clang
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: clang
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: clang
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *doc_packages
>    # x86_64 cross-compiling 32-bits jobs
>    - env: DEF_LIB="static" BUILD_32BIT=1
> @@ -131,9 +101,6 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: arm64
>      compiler: gcc
> -  - env: DEF_LIB="shared"
> -    arch: arm64
> -    compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
>      compiler: gcc
> @@ -142,13 +109,13 @@ jobs:
>          packages:
>            - *required_packages
>            - *doc_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      arch: arm64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *libabigail_build_packages

>    # aarch64 clang jobs
>    - env: DEF_LIB="static"
  

On 2020-02-19 11:41, David Marchand wrote:
> Let's prune the jobs list to limit the amount of time spent by the 
> robot
> in Travis.
> 
> Since meson enables automatically the relevant components, there is not
> much gain in testing with extra_packages vs required_packages only.
> 
> For a given arch/compiler/env combination, compilation is first tested
> in all jobs that run tests or build the docs or run the ABI checks.
> In the same context, for jobs that accumulates running tests, building
> the docs etc..., those steps are independent and can be split to save
> some cpu on Travis.
> 
> With this, we go down from 21 to 15 jobs.
> 
> Note: this patch requires a flush of the existing caches in Travis.
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
>  .travis.yml | 51 +++++++++------------------------------------------
>  1 file changed, 9 insertions(+), 42 deletions(-)
> 
> diff --git a/.travis.yml b/.travis.yml
> index c174d5f30..b64a81bd0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -15,15 +15,12 @@ addons:
>      update: true
>      packages: &required_packages
>        - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip,
> ninja-build]
> +      - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev,
> libfdt-dev, libjansson4]
> 
>  _aarch64_packages: &aarch64_packages
>    - *required_packages
>    - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross,
> pkg-config-aarch64-linux-gnu]
> 
> -_extra_packages: &extra_packages
> -  - *required_packages
> -  - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev,
> libfdt-dev, libjansson4]
> -
>  _libabigail_build_packages: &libabigail_build_packages
>    - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
> 
> @@ -43,66 +40,39 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="shared"
> -    arch: amd64
> -    compiler: gcc
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="static" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: gcc
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *doc_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      arch: amd64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *libabigail_build_packages
>    # x86_64 clang jobs
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="shared"
> -    arch: amd64
> -    compiler: clang
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="static" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: clang
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: clang
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: clang
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *doc_packages
>    # x86_64 cross-compiling 32-bits jobs
>    - env: DEF_LIB="static" BUILD_32BIT=1
> @@ -131,9 +101,6 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: arm64
>      compiler: gcc
> -  - env: DEF_LIB="shared"
> -    arch: arm64
> -    compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
>      compiler: gcc
> @@ -142,13 +109,13 @@ jobs:
>          packages:
>            - *required_packages
>            - *doc_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      arch: arm64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *libabigail_build_packages
>    # aarch64 clang jobs
>    - env: DEF_LIB="static"

Much easier to follow,  I had no issues adding ppc64le jobs.
All my travis jobs ran fine.

LGTM  (the series)
  

19/02/2020 22:39, Aaron Conole:
> David Marchand <david.marchand@redhat.com> writes:
> 
> > Let's prune the jobs list to limit the amount of time spent by the robot
> > in Travis.
> >
> > Since meson enables automatically the relevant components, there is not
> > much gain in testing with extra_packages vs required_packages only.
> >
> > For a given arch/compiler/env combination, compilation is first tested
> > in all jobs that run tests or build the docs or run the ABI checks.
> > In the same context, for jobs that accumulates running tests, building
> > the docs etc..., those steps are independent and can be split to save
> > some cpu on Travis.
> >
> > With this, we go down from 21 to 15 jobs.
> >
> > Note: this patch requires a flush of the existing caches in Travis.
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
> 
> In general, I think the idea with required vs. extra was to have a build
> that did the minimum required, and one that did all the packages (to
> allow a minimum vs. full DPDK).
> 
> At least, that's from
> http://mails.dpdk.org/archives/dev/2019-January/124007.html

I think the benefit of a minimum build is to have a quick report,
and easy to setup.

> Not sure if that's still something anyone cares about.

Given that Travis knows how to satisfy the dependencies,
and that we must wait for all jobs to finish,
I don't see any benefit of a minimal setup.
  

19/02/2020 20:41, David Marchand:
> Let's prune the jobs list to limit the amount of time spent by the robot
> in Travis.
> 
> Since meson enables automatically the relevant components, there is not
> much gain in testing with extra_packages vs required_packages only.
> 
> For a given arch/compiler/env combination, compilation is first tested
> in all jobs that run tests or build the docs or run the ABI checks.
> In the same context, for jobs that accumulates running tests, building
> the docs etc..., those steps are independent and can be split to save
> some cpu on Travis.
> 
> With this, we go down from 21 to 15 jobs.
> 
> Note: this patch requires a flush of the existing caches in Travis.
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>

Acked-by: Thomas Monjalon <thomas@monjalon.net>

A small step for DPDK but a big save for the earth ;)
  

On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 19/02/2020 22:39, Aaron Conole:
> > David Marchand <david.marchand@redhat.com> writes:
> >
> > > Let's prune the jobs list to limit the amount of time spent by the robot
> > > in Travis.
> > >
> > > Since meson enables automatically the relevant components, there is not
> > > much gain in testing with extra_packages vs required_packages only.
> > >
> > > For a given arch/compiler/env combination, compilation is first tested
> > > in all jobs that run tests or build the docs or run the ABI checks.
> > > In the same context, for jobs that accumulates running tests, building
> > > the docs etc..., those steps are independent and can be split to save
> > > some cpu on Travis.
> > >
> > > With this, we go down from 21 to 15 jobs.
> > >
> > > Note: this patch requires a flush of the existing caches in Travis.
> > >
> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > > ---
> >
> > In general, I think the idea with required vs. extra was to have a build
> > that did the minimum required, and one that did all the packages (to
> > allow a minimum vs. full DPDK).
> >
> > At least, that's from
> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
>
> I think the benefit of a minimum build is to have a quick report,
> and easy to setup.

Yes, Travis serves as a first gate when submitting patches.
But since Travis is best effort/free, we can't have a full coverage.


> > Not sure if that's still something anyone cares about.
>
> Given that Travis knows how to satisfy the dependencies,
> and that we must wait for all jobs to finish,
> I don't see any benefit of a minimal setup.

This minimal setup also tests that dpdk dependencies are correct.
If a change makes something rely on libX and libX is in the packages
always installed in Travis, the missing dependency would not get
caught.

But here, this adds too many jobs.

UNH, Intel and other CIs should step in and fill this kind of gap.


--
David Marchand
  

David Marchand <david.marchand@redhat.com> writes:

> On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>>
>> 19/02/2020 22:39, Aaron Conole:
>> > David Marchand <david.marchand@redhat.com> writes:
>> >
>> > > Let's prune the jobs list to limit the amount of time spent by the robot
>> > > in Travis.
>> > >
>> > > Since meson enables automatically the relevant components, there is not
>> > > much gain in testing with extra_packages vs required_packages only.
>> > >
>> > > For a given arch/compiler/env combination, compilation is first tested
>> > > in all jobs that run tests or build the docs or run the ABI checks.
>> > > In the same context, for jobs that accumulates running tests, building
>> > > the docs etc..., those steps are independent and can be split to save
>> > > some cpu on Travis.
>> > >
>> > > With this, we go down from 21 to 15 jobs.
>> > >
>> > > Note: this patch requires a flush of the existing caches in Travis.
>> > >
>> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
>> > > ---
>> >
>> > In general, I think the idea with required vs. extra was to have a build
>> > that did the minimum required, and one that did all the packages (to
>> > allow a minimum vs. full DPDK).
>> >
>> > At least, that's from
>> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
>>
>> I think the benefit of a minimum build is to have a quick report,
>> and easy to setup.
>
> Yes, Travis serves as a first gate when submitting patches.
> But since Travis is best effort/free, we can't have a full coverage.
>
>
>> > Not sure if that's still something anyone cares about.
>>
>> Given that Travis knows how to satisfy the dependencies,
>> and that we must wait for all jobs to finish,
>> I don't see any benefit of a minimal setup.
>
> This minimal setup also tests that dpdk dependencies are correct.
> If a change makes something rely on libX and libX is in the packages
> always installed in Travis, the missing dependency would not get
> caught.
>
> But here, this adds too many jobs.
>
> UNH, Intel and other CIs should step in and fill this kind of gap.

Okay, makes sense to me.  Are one of these CI providers offering to
cover this?

Also,

Acked-by: Aaron Conole <aconole@redhat.com>

>
> --
> David Marchand
  

On Thu, Feb 20, 2020 at 3:35 PM Aaron Conole <aconole@redhat.com> wrote:
>
> David Marchand <david.marchand@redhat.com> writes:
>
> > On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >>
> >> 19/02/2020 22:39, Aaron Conole:
> >> > David Marchand <david.marchand@redhat.com> writes:
> >> >
> >> > > Let's prune the jobs list to limit the amount of time spent by the robot
> >> > > in Travis.
> >> > >
> >> > > Since meson enables automatically the relevant components, there is not
> >> > > much gain in testing with extra_packages vs required_packages only.
> >> > >
> >> > > For a given arch/compiler/env combination, compilation is first tested
> >> > > in all jobs that run tests or build the docs or run the ABI checks.
> >> > > In the same context, for jobs that accumulates running tests, building
> >> > > the docs etc..., those steps are independent and can be split to save
> >> > > some cpu on Travis.
> >> > >
> >> > > With this, we go down from 21 to 15 jobs.
> >> > >
> >> > > Note: this patch requires a flush of the existing caches in Travis.
> >> > >
> >> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> >> > > ---
> >> >
> >> > In general, I think the idea with required vs. extra was to have a build
> >> > that did the minimum required, and one that did all the packages (to
> >> > allow a minimum vs. full DPDK).
> >> >
> >> > At least, that's from
> >> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
> >>
> >> I think the benefit of a minimum build is to have a quick report,
> >> and easy to setup.
> >
> > Yes, Travis serves as a first gate when submitting patches.
> > But since Travis is best effort/free, we can't have a full coverage.
> >
> >
> >> > Not sure if that's still something anyone cares about.
> >>
> >> Given that Travis knows how to satisfy the dependencies,
> >> and that we must wait for all jobs to finish,
> >> I don't see any benefit of a minimal setup.
> >
> > This minimal setup also tests that dpdk dependencies are correct.
> > If a change makes something rely on libX and libX is in the packages
> > always installed in Travis, the missing dependency would not get
> > caught.
> >
> > But here, this adds too many jobs.
> >
> > UNH, Intel and other CIs should step in and fill this kind of gap.
>
> Okay, makes sense to me.  Are one of these CI providers offering to
> cover this?

Maybe it is already covered, the best is to ask, so sending to ci@dpdk.org.

For the CI guys, which packages are installed on the systems/vms that
do compilation tests?
Is it possible to have a summary of the different setups?


Thanks.

--
David Marchand
  

On Thu, Feb 20, 2020 at 11:02 AM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Thu, Feb 20, 2020 at 3:35 PM Aaron Conole <aconole@redhat.com> wrote:
> >
> > David Marchand <david.marchand@redhat.com> writes:
> >
> > > On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > >>
> > >> 19/02/2020 22:39, Aaron Conole:
> > >> > David Marchand <david.marchand@redhat.com> writes:
> > >> >
> > >> > > Let's prune the jobs list to limit the amount of time spent by the robot
> > >> > > in Travis.
> > >> > >
> > >> > > Since meson enables automatically the relevant components, there is not
> > >> > > much gain in testing with extra_packages vs required_packages only.
> > >> > >
> > >> > > For a given arch/compiler/env combination, compilation is first tested
> > >> > > in all jobs that run tests or build the docs or run the ABI checks.
> > >> > > In the same context, for jobs that accumulates running tests, building
> > >> > > the docs etc..., those steps are independent and can be split to save
> > >> > > some cpu on Travis.
> > >> > >
> > >> > > With this, we go down from 21 to 15 jobs.
> > >> > >
> > >> > > Note: this patch requires a flush of the existing caches in Travis.
> > >> > >
> > >> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > >> > > ---
> > >> >
> > >> > In general, I think the idea with required vs. extra was to have a build
> > >> > that did the minimum required, and one that did all the packages (to
> > >> > allow a minimum vs. full DPDK).
> > >> >
> > >> > At least, that's from
> > >> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
> > >>
> > >> I think the benefit of a minimum build is to have a quick report,
> > >> and easy to setup.
> > >
> > > Yes, Travis serves as a first gate when submitting patches.
> > > But since Travis is best effort/free, we can't have a full coverage.
> > >
> > >
> > >> > Not sure if that's still something anyone cares about.
> > >>
> > >> Given that Travis knows how to satisfy the dependencies,
> > >> and that we must wait for all jobs to finish,
> > >> I don't see any benefit of a minimal setup.
> > >
> > > This minimal setup also tests that dpdk dependencies are correct.
> > > If a change makes something rely on libX and libX is in the packages
> > > always installed in Travis, the missing dependency would not get
> > > caught.
> > >
> > > But here, this adds too many jobs.
> > >
> > > UNH, Intel and other CIs should step in and fill this kind of gap.
> >
> > Okay, makes sense to me.  Are one of these CI providers offering to
> > cover this?
>
> Maybe it is already covered, the best is to ask, so sending to ci@dpdk.org.
>
> For the CI guys, which packages are installed on the systems/vms that
> do compilation tests?

For our compile tests, we compile with pretty much all dpdk
dependencies that can easily be installed from the distro's repository
(the dependencies we installed are based on the meson output). The
thought process was to test compiling as much of dpdk as possible
(with constraints mentioned before).

Here are the dependencies we install on some of the setups:

Ubuntu 18.04: make gcc ninja-build pkg-config libnuma-dev
librdmacm-dev librdmacm1 rdma-core libelf-dev libmnl-dev libpcap-dev
libcrypto++-dev libjansson-dev
(meson installed with pip)

CentOS 8 and Fedora 31: diffutils gcc make ninja-build meson
pkg-config numactl-devel librdmacm rdma-core-devel
elfutils-libelf-devel libmnl-devel libpcap-devel jansson-devel

openSuse Leap 15: gcc make ninja pkg-config libnuma-devel librdmacm1
rdma-core-devel libelf-devel libmnl-devel libpcap-devel
libjansson-devel
(meson installed with pip)

> Is it possible to have a summary of the different setups?
>
I've attached an overview/summary of what we do at the UNH-IOL DPDK
Community Lab.
>
> Thanks.

>
> --
> David Marchand
>