[v7,0/3] pmdinfogen: rewrite in Python

  This patchset implements existing pmdinfogen logic in Python, replaces
and removes the old code. The goals of rewriting are:

* easier maintenance by using a more high-level language,
* simpler build process without host application and libelf,
* foundation for adding Windows support.

Travis CI script is adjusted to install python3-pyelftools, but other CI
systems may need similar tweaking. Particularly, testing on FreeBSD and
big-endian targets is desired.

Acked-by: Neil Horman <nhorman@tuxdriver.com>

---
Changes in v7:

    Add configure-time check for elftools module (Maxime Coquelin).

Dmitry Kozlyuk (3):
  pmdinfogen: add Python implementation
  build: use Python pmdinfogen
  pmdinfogen: remove C implementation

 .travis.yml                        |   2 +-
 MAINTAINERS                        |   3 +-
 buildtools/gen-pmdinfo-cfile.sh    |   6 +-
 buildtools/meson.build             |  15 +
 buildtools/pmdinfogen.py           | 189 ++++++++++++
 buildtools/pmdinfogen/meson.build  |  14 -
 buildtools/pmdinfogen/pmdinfogen.c | 456 -----------------------------
 buildtools/pmdinfogen/pmdinfogen.h | 119 --------
 doc/guides/linux_gsg/sys_reqs.rst  |   6 +
 drivers/meson.build                |   2 +-
 meson.build                        |   1 -
 11 files changed, 217 insertions(+), 596 deletions(-)
 create mode 100755 buildtools/pmdinfogen.py
 delete mode 100644 buildtools/pmdinfogen/meson.build
 delete mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 delete mode 100644 buildtools/pmdinfogen/pmdinfogen.h
  

Hello Dmitry,

On Wed, Oct 14, 2020 at 8:31 PM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
>
> This patchset implements existing pmdinfogen logic in Python, replaces
> and removes the old code. The goals of rewriting are:
>
> * easier maintenance by using a more high-level language,
> * simpler build process without host application and libelf,
> * foundation for adding Windows support.
>
> Travis CI script is adjusted to install python3-pyelftools, but other CI
> systems may need similar tweaking. Particularly, testing on FreeBSD and
> big-endian targets is desired.
>

- Travis reported an error in meson:
https://travis-ci.com/github/ovsrobot/dpdk/jobs/399762460#L478

Build machine cpu: x86_64
buildtools/meson.build:27:10: ERROR:  Expecting endforeach got eol_cont.
script = \
          ^
For a block that started at 26,0
foreach module : python3_required_modules
^


- In a thread on ci@dpdk.org
http://inbox.dpdk.org/ci/CAOeXdvbosqRGouSX06_phVTCam57XUt0s=sf_kpe-VmiwFTgMA@mail.gmail.com/T/#m9bb193c74ca979c151f2dbfa35664f95093a9e71,
UNH people confirmed that python3-elftools should be available, but
the last test had an error too, which might be linked to incorrect
meson syntax:
http://mails.dpdk.org/archives/test-report/2020-October/159038.html

ninja: error: loading 'build.ninja': No such file or directory


- As far as Intel CI systems are concerned, there might be an issue
with the Windows system:
http://mails.dpdk.org/archives/test-report/2020-October/159123.html

OS: WIN10-64
Target: x86_64-windows-clang
Program binutils-avx512-check.sh found: YES
Program python3 found: YES (c:\program files\python37\python.exe)

buildtools\meson.build:32:2: ERROR: Problem encountered: missing
python module: elftools

A full log can be found at C:\windows-dpdk\dpdk\build\meson-logs\meson-log.txt
ninja: Entering directory `build'
DPDK STV team
  

Hello David,

> - Travis reported an error in meson:
> https://travis-ci.com/github/ovsrobot/dpdk/jobs/399762460#L478
> 
> Build machine cpu: x86_64
> buildtools/meson.build:27:10: ERROR:  Expecting endforeach got eol_cont.
> script = \
>           ^
> For a block that started at 26,0
> foreach module : python3_required_modules
> ^

My bad, meson 0.47.1 doesn't understand this indeed, fixed in v8.

> - In a thread on ci@dpdk.org
> http://inbox.dpdk.org/ci/CAOeXdvbosqRGouSX06_phVTCam57XUt0s=sf_kpe-VmiwFTgMA@mail.gmail.com/T/#m9bb193c74ca979c151f2dbfa35664f95093a9e71,
> UNH people confirmed that python3-elftools should be available, but
> the last test had an error too, which might be linked to incorrect
> meson syntax:
> http://mails.dpdk.org/archives/test-report/2020-October/159038.html
> 
> ninja: error: loading 'build.ninja': No such file or directory

Makes sense.

> - As far as Intel CI systems are concerned, there might be an issue
> with the Windows system:
> http://mails.dpdk.org/archives/test-report/2020-October/159123.html
>
> OS: WIN10-64
> Target: x86_64-windows-clang
> Program binutils-avx512-check.sh found: YES
> Program python3 found: YES (c:\program files\python37\python.exe)
> 
> buildtools\meson.build:32:2: ERROR: Problem encountered: missing
> python module: elftools

This is the check added in the last revision per Maxime's suggestion.
Disabled the check on Windows in v8 until pmdinfogen supports COFF (and this
will probably introduce an alternative dependency).

I'll watch new CI results, haven't been doing so closely because failures were
expected for these patches. Tested v8 build on Linux->Linux, Linux->Windows,
Windows->Windows (GCC & clang, without pyelftools installed).
  

Hi IOL team,

The following error persists in v8:

> - In a thread on ci@dpdk.org
> http://inbox.dpdk.org/ci/CAOeXdvbosqRGouSX06_phVTCam57XUt0s=sf_kpe-VmiwFTgMA@mail.gmail.com/T/#m9bb193c74ca979c151f2dbfa35664f95093a9e71,
> UNH people confirmed that python3-elftools should be available, but
> the last test had an error too, which might be linked to incorrect
> meson syntax:
> http://mails.dpdk.org/archives/test-report/2020-October/159038.html
> 
> ninja: error: loading 'build.ninja': No such file or directory

https://inbox.dpdk.org/test-report/20201020181426.1BC8788FC9@noxus.dpdklab.iol.unh.edu/T/#u

The message isn't very informative. I suspect meson failed to configure,
but there's no way to tell what went wrong. How can I get more information
about this failure or specification of your environment to reproduce it?