[v6,0/4] devtools: abi breakage checks

This patchset will help developers discover abi breakages more easily
before upstreaming their code. Currently checking that the DPDK ABI
has not changed before up-streaming code is not intuitive and the
process is time consuming. Currently contributors must use the
test-meson-builds.sh tool, alongside some environmental variables to
test their changes. Contributors in many cases are either unaware or
unable to do this themselves, leading to a potentially serious situation
where they are unknowingly up-streaming code that breaks the ABI. These
breakages are caught by Travis, but it would be more efficient if they
were caught locally before up-streaming. This patchset introduces changes
to test-meson-builds.sh, check-abi.sh and adds a new script
gen-abi-tarballs.sh. The changes to test-meson-builds.sh include UX
changes such as adding command line arguments and allowing the use of
relative paths. Reduced the number of abi checks to just two, one for both
x86_64 and ARM, the references for these tests can now be prebuilt and
downloaded by test-meson-builds.sh, these changes will allow the tests to
run much faster. check-abi.sh is updated to use the prebuilt references.
gen-abi-tarballs.sh is a new script to generate the prebuilt abi
references used by test-meson-builds.sh, these compressed archives can be
retrieved from either a local directory or a remote http location.

---
v6: Corrected a mistake in the doc patch

v5:
 - Patchset has been completely reworked following feedback
 - Patchset is now part of test-meson-builds.sh not the meson build system

v4:
 - Reworked both Python scripts to use more native Python functions
   and modules.
 - Python scripts are now in line with how other Python scripts in
   DPDK are structured.

v3:
 - Fix for bug which now allows meson < 0.48.0 to be used
 - Various coding style changes throughout
 - Minor bug fixes to the various meson.build files

v2: Spelling mistake, corrected spelling of environmental

Conor Walsh (4):
  devtools: add generation of compressed abi dump archives
  devtools: abi and UX changes for test-meson-builds.sh
  devtools: change dump file not found to warning in check-abi.sh
  doc: test-meson-builds.sh doc updates

 devtools/check-abi.sh               |   3 +-
 devtools/gen-abi-tarballs.sh        |  48 ++++++++
 devtools/test-meson-builds.sh       | 170 ++++++++++++++++++++++------
 doc/guides/contributing/patches.rst |  26 +++--
 4 files changed, 201 insertions(+), 46 deletions(-)
 create mode 100755 devtools/gen-abi-tarballs.sh
  

On 12/10/2020 14:03, Conor Walsh wrote:
> This patchset will help developers discover abi breakages more easily
> before upstreaming their code. Currently checking that the DPDK ABI
> has not changed before up-streaming code is not intuitive and the
> process is time consuming. Currently contributors must use the
> test-meson-builds.sh tool, alongside some environmental variables to
> test their changes. Contributors in many cases are either unaware or
> unable to do this themselves, leading to a potentially serious situation
> where they are unknowingly up-streaming code that breaks the ABI. These
> breakages are caught by Travis, but it would be more efficient if they
> were caught locally before up-streaming. 

I would remove everything in the git log text before this line... 

> This patchset introduces changes
> to test-meson-builds.sh, check-abi.sh and adds a new script
> gen-abi-tarballs.sh. The changes to test-meson-builds.sh include UX

UX changes = improvements

> changes such as adding command line arguments and allowing the use of
> relative paths. Reduced the number of abi checks to just two, one for both
> x86_64 and ARM, the references for these tests can now be prebuilt and
> downloaded by test-meson-builds.sh, these changes will allow the tests to
> run much faster. check-abi.sh is updated to use the prebuilt references.
> gen-abi-tarballs.sh is a new script to generate the prebuilt abi
> references used by test-meson-builds.sh, these compressed archives can be
> retrieved from either a local directory or a remote http location.
> 
> ---
> v6: Corrected a mistake in the doc patch
> 
> v5:
>  - Patchset has been completely reworked following feedback
>  - Patchset is now part of test-meson-builds.sh not the meson build system
> 
> v4:
>  - Reworked both Python scripts to use more native Python functions
>    and modules.
>  - Python scripts are now in line with how other Python scripts in
>    DPDK are structured.
> 
> v3:
>  - Fix for bug which now allows meson < 0.48.0 to be used
>  - Various coding style changes throughout
>  - Minor bug fixes to the various meson.build files
> 
> v2: Spelling mistake, corrected spelling of environmental
> 
> Conor Walsh (4):
>   devtools: add generation of compressed abi dump archives
>   devtools: abi and UX changes for test-meson-builds.sh
>   devtools: change dump file not found to warning in check-abi.sh
>   doc: test-meson-builds.sh doc updates
> 
>  devtools/check-abi.sh               |   3 +-
>  devtools/gen-abi-tarballs.sh        |  48 ++++++++
>  devtools/test-meson-builds.sh       | 170 ++++++++++++++++++++++------
>  doc/guides/contributing/patches.rst |  26 +++--
>  4 files changed, 201 insertions(+), 46 deletions(-)
>  create mode 100755 devtools/gen-abi-tarballs.sh
>
  

Thanks for your feedback Ray,

V7 with your suggested changes for the patchset is on its way.

/Conor

> -----Original Message-----
> From: Kinsella, Ray <mdr@ashroe.eu>
> Sent: Wednesday 14 October 2020 10:37
> To: Walsh, Conor <conor.walsh@intel.com>; nhorman@tuxdriver.com;
> Richardson, Bruce <bruce.richardson@intel.com>; thomas@monjalon.net;
> david.marchand@redhat.com
> Cc: dev@dpdk.org
> Subject: Re: [PATCH v6 0/4] devtools: abi breakage checks
> 
> 
> 
> On 12/10/2020 14:03, Conor Walsh wrote:
> > This patchset will help developers discover abi breakages more easily
> > before upstreaming their code. Currently checking that the DPDK ABI
> > has not changed before up-streaming code is not intuitive and the
> > process is time consuming. Currently contributors must use the
> > test-meson-builds.sh tool, alongside some environmental variables to
> > test their changes. Contributors in many cases are either unaware or
> > unable to do this themselves, leading to a potentially serious situation
> > where they are unknowingly up-streaming code that breaks the ABI. These
> > breakages are caught by Travis, but it would be more efficient if they
> > were caught locally before up-streaming.
> 
> I would remove everything in the git log text before this line...
> 
> > This patchset introduces changes
> > to test-meson-builds.sh, check-abi.sh and adds a new script
> > gen-abi-tarballs.sh. The changes to test-meson-builds.sh include UX
> 
> UX changes = improvements
> 
> > changes such as adding command line arguments and allowing the use of
> > relative paths. Reduced the number of abi checks to just two, one for both
> > x86_64 and ARM, the references for these tests can now be prebuilt and
> > downloaded by test-meson-builds.sh, these changes will allow the tests to
> > run much faster. check-abi.sh is updated to use the prebuilt references.
> > gen-abi-tarballs.sh is a new script to generate the prebuilt abi
> > references used by test-meson-builds.sh, these compressed archives can
> be
> > retrieved from either a local directory or a remote http location.
> >
> > ---
> > v6: Corrected a mistake in the doc patch
> >
> > v5:
> >  - Patchset has been completely reworked following feedback
> >  - Patchset is now part of test-meson-builds.sh not the meson build system
> >
> > v4:
> >  - Reworked both Python scripts to use more native Python functions
> >    and modules.
> >  - Python scripts are now in line with how other Python scripts in
> >    DPDK are structured.
> >
> > v3:
> >  - Fix for bug which now allows meson < 0.48.0 to be used
> >  - Various coding style changes throughout
> >  - Minor bug fixes to the various meson.build files
> >
> > v2: Spelling mistake, corrected spelling of environmental
> >
> > Conor Walsh (4):
> >   devtools: add generation of compressed abi dump archives
> >   devtools: abi and UX changes for test-meson-builds.sh
> >   devtools: change dump file not found to warning in check-abi.sh
> >   doc: test-meson-builds.sh doc updates
> >
> >  devtools/check-abi.sh               |   3 +-
> >  devtools/gen-abi-tarballs.sh        |  48 ++++++++
> >  devtools/test-meson-builds.sh       | 170 ++++++++++++++++++++++------
> >  doc/guides/contributing/patches.rst |  26 +++--
> >  4 files changed, 201 insertions(+), 46 deletions(-)
> >  create mode 100755 devtools/gen-abi-tarballs.sh
> >