[v2,0/7] improve DPDK static builds with meson
mbox series

Message ID 20200501135401.28124-1-bruce.richardson@intel.com
Headers show
Series
  • improve DPDK static builds with meson
Related show

Message

Bruce Richardson May 1, 2020, 1:53 p.m. UTC
This set fixes a number of minor issues with static builds when using
meson, both for linking apps/examples as part of a meson build itself or
when using pkg-config subsequently.

Following this patchset, all DPDK static builds should be linking with
--whole-archive to ensure all lib and driver constructors are included,
and the use of pkg-config for doing static builds is simplified. The
downside is that for correctness we need two .pc files for DPDK rather
than just one.

v2: improved log messages for a number of patches, and clearly marked
    internal-only pkg-config file as such

Bruce Richardson (7):
  build: always link-whole DPDK static libraries
  build: remove unnecessary variable
  devtools/test-meson-builds.sh: add pkg-config static builds
  build: move pkg-config creation to separate file
  build/pkg-config: output driver libs first for static build
  build/pkg-config: improve static linking flags
  build/pkg-config: prevent overlinking

 app/meson.build                               |  2 +-
 app/test/meson.build                          |  2 +-
 buildtools/pkg-config/meson.build             | 43 +++++++++++++++++++
 .../pkg-config/set-static-linker-flags.py     | 38 ++++++++++++++++
 devtools/test-meson-builds.sh                 |  2 +-
 doc/guides/prog_guide/build-sdk-meson.rst     |  2 +-
 drivers/meson.build                           |  2 +-
 examples/bbdev_app/Makefile                   |  2 +-
 examples/bond/Makefile                        |  2 +-
 examples/cmdline/Makefile                     |  2 +-
 examples/distributor/Makefile                 |  2 +-
 examples/eventdev_pipeline/Makefile           |  2 +-
 examples/fips_validation/Makefile             |  2 +-
 examples/flow_classify/Makefile               |  2 +-
 examples/flow_filtering/Makefile              |  2 +-
 examples/helloworld/Makefile                  |  2 +-
 examples/ioat/Makefile                        |  2 +-
 examples/ip_fragmentation/Makefile            |  2 +-
 examples/ip_pipeline/Makefile                 |  2 +-
 examples/ip_reassembly/Makefile               |  2 +-
 examples/ipsec-secgw/Makefile                 |  2 +-
 examples/ipv4_multicast/Makefile              |  2 +-
 examples/kni/Makefile                         |  2 +-
 examples/l2fwd-cat/Makefile                   |  2 +-
 examples/l2fwd-crypto/Makefile                |  2 +-
 examples/l2fwd-event/Makefile                 |  2 +-
 examples/l2fwd-jobstats/Makefile              |  2 +-
 examples/l2fwd-keepalive/Makefile             |  2 +-
 examples/l2fwd/Makefile                       |  2 +-
 examples/l3fwd-acl/Makefile                   |  2 +-
 examples/l3fwd-power/Makefile                 |  2 +-
 examples/l3fwd/Makefile                       |  2 +-
 examples/link_status_interrupt/Makefile       |  2 +-
 examples/meson.build                          |  6 +--
 examples/ntb/Makefile                         |  2 +-
 examples/packet_ordering/Makefile             |  2 +-
 examples/ptpclient/Makefile                   |  2 +-
 examples/qos_meter/Makefile                   |  2 +-
 examples/qos_sched/Makefile                   |  2 +-
 examples/rxtx_callbacks/Makefile              |  2 +-
 examples/service_cores/Makefile               |  2 +-
 examples/skeleton/Makefile                    |  2 +-
 examples/tep_termination/Makefile             |  2 +-
 examples/timer/Makefile                       |  2 +-
 examples/vdpa/Makefile                        |  2 +-
 examples/vhost/Makefile                       |  2 +-
 examples/vhost_blk/Makefile                   |  2 +-
 examples/vhost_crypto/Makefile                |  2 +-
 examples/vmdq/Makefile                        |  2 +-
 examples/vmdq_dcb/Makefile                    |  2 +-
 lib/librte_telemetry/meson.build              |  1 -
 lib/meson.build                               |  2 +-
 meson.build                                   | 26 +----------
 53 files changed, 134 insertions(+), 76 deletions(-)
 create mode 100644 buildtools/pkg-config/meson.build
 create mode 100644 buildtools/pkg-config/set-static-linker-flags.py

Comments

Luca Boccassi May 1, 2020, 2:02 p.m. UTC | #1
On Fri, 2020-05-01 at 14:53 +0100, Bruce Richardson wrote:
> This set fixes a number of minor issues with static builds when using
> meson, both for linking apps/examples as part of a meson build itself or
> when using pkg-config subsequently.
> 
> Following this patchset, all DPDK static builds should be linking with
> --whole-archive to ensure all lib and driver constructors are included,
> and the use of pkg-config for doing static builds is simplified. The
> downside is that for correctness we need two .pc files for DPDK rather
> than just one.
> 
> v2: improved log messages for a number of patches, and clearly marked
>     internal-only pkg-config file as such
> 
> Bruce Richardson (7):
>   build: always link-whole DPDK static libraries
>   build: remove unnecessary variable
>   devtools/test-meson-builds.sh: add pkg-config static builds
>   build: move pkg-config creation to separate file
>   build/pkg-config: output driver libs first for static build
>   build/pkg-config: improve static linking flags
>   build/pkg-config: prevent overlinking
> 
>  app/meson.build                               |  2 +-
>  app/test/meson.build                          |  2 +-
>  buildtools/pkg-config/meson.build             | 43 +++++++++++++++++++
>  .../pkg-config/set-static-linker-flags.py     | 38 ++++++++++++++++
>  devtools/test-meson-builds.sh                 |  2 +-
>  doc/guides/prog_guide/build-sdk-meson.rst     |  2 +-
>  drivers/meson.build                           |  2 +-
>  examples/bbdev_app/Makefile                   |  2 +-
>  examples/bond/Makefile                        |  2 +-
>  examples/cmdline/Makefile                     |  2 +-
>  examples/distributor/Makefile                 |  2 +-
>  examples/eventdev_pipeline/Makefile           |  2 +-
>  examples/fips_validation/Makefile             |  2 +-
>  examples/flow_classify/Makefile               |  2 +-
>  examples/flow_filtering/Makefile              |  2 +-
>  examples/helloworld/Makefile                  |  2 +-
>  examples/ioat/Makefile                        |  2 +-
>  examples/ip_fragmentation/Makefile            |  2 +-
>  examples/ip_pipeline/Makefile                 |  2 +-
>  examples/ip_reassembly/Makefile               |  2 +-
>  examples/ipsec-secgw/Makefile                 |  2 +-
>  examples/ipv4_multicast/Makefile              |  2 +-
>  examples/kni/Makefile                         |  2 +-
>  examples/l2fwd-cat/Makefile                   |  2 +-
>  examples/l2fwd-crypto/Makefile                |  2 +-
>  examples/l2fwd-event/Makefile                 |  2 +-
>  examples/l2fwd-jobstats/Makefile              |  2 +-
>  examples/l2fwd-keepalive/Makefile             |  2 +-
>  examples/l2fwd/Makefile                       |  2 +-
>  examples/l3fwd-acl/Makefile                   |  2 +-
>  examples/l3fwd-power/Makefile                 |  2 +-
>  examples/l3fwd/Makefile                       |  2 +-
>  examples/link_status_interrupt/Makefile       |  2 +-
>  examples/meson.build                          |  6 +--
>  examples/ntb/Makefile                         |  2 +-
>  examples/packet_ordering/Makefile             |  2 +-
>  examples/ptpclient/Makefile                   |  2 +-
>  examples/qos_meter/Makefile                   |  2 +-
>  examples/qos_sched/Makefile                   |  2 +-
>  examples/rxtx_callbacks/Makefile              |  2 +-
>  examples/service_cores/Makefile               |  2 +-
>  examples/skeleton/Makefile                    |  2 +-
>  examples/tep_termination/Makefile             |  2 +-
>  examples/timer/Makefile                       |  2 +-
>  examples/vdpa/Makefile                        |  2 +-
>  examples/vhost/Makefile                       |  2 +-
>  examples/vhost_blk/Makefile                   |  2 +-
>  examples/vhost_crypto/Makefile                |  2 +-
>  examples/vmdq/Makefile                        |  2 +-
>  examples/vmdq_dcb/Makefile                    |  2 +-
>  lib/librte_telemetry/meson.build              |  1 -
>  lib/meson.build                               |  2 +-
>  meson.build                                   | 26 +----------
>  53 files changed, 134 insertions(+), 76 deletions(-)
>  create mode 100644 buildtools/pkg-config/meson.build
>  create mode 100644 buildtools/pkg-config/set-static-linker-flags.py

Series-acked-by: Luca Boccassi <bluca@debian.org>
Pai G, Sunil June 30, 2020, 1:25 p.m. UTC | #2
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Bruce Richardson
> Sent: Friday, May 1, 2020 7:24 PM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; bluca@debian.org;
> david.marchand@redhat.com; ktraynor@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: [dpdk-dev] [PATCH v2 0/7] improve DPDK static builds with meson
> 
> This set fixes a number of minor issues with static builds when using meson,
> both for linking apps/examples as part of a meson build itself or when using
> pkg-config subsequently.
> 
> Following this patchset, all DPDK static builds should be linking with --whole-
> archive to ensure all lib and driver constructors are included, and the use of
> pkg-config for doing static builds is simplified. The downside is that for
> correctness we need two .pc files for DPDK rather than just one.
> 
> v2: improved log messages for a number of patches, and clearly marked
>     internal-only pkg-config file as such
> 
> Bruce Richardson (7):
>   build: always link-whole DPDK static libraries
>   build: remove unnecessary variable
>   devtools/test-meson-builds.sh: add pkg-config static builds
>   build: move pkg-config creation to separate file
>   build/pkg-config: output driver libs first for static build
>   build/pkg-config: improve static linking flags
>   build/pkg-config: prevent overlinking
> 
>  app/meson.build                               |  2 +-
>  app/test/meson.build                          |  2 +-
>  buildtools/pkg-config/meson.build             | 43 +++++++++++++++++++
>  .../pkg-config/set-static-linker-flags.py     | 38 ++++++++++++++++
>  devtools/test-meson-builds.sh                 |  2 +-
>  doc/guides/prog_guide/build-sdk-meson.rst     |  2 +-
>  drivers/meson.build                           |  2 +-
>  examples/bbdev_app/Makefile                   |  2 +-
>  examples/bond/Makefile                        |  2 +-
>  examples/cmdline/Makefile                     |  2 +-
>  examples/distributor/Makefile                 |  2 +-
>  examples/eventdev_pipeline/Makefile           |  2 +-
>  examples/fips_validation/Makefile             |  2 +-
>  examples/flow_classify/Makefile               |  2 +-
>  examples/flow_filtering/Makefile              |  2 +-
>  examples/helloworld/Makefile                  |  2 +-
>  examples/ioat/Makefile                        |  2 +-
>  examples/ip_fragmentation/Makefile            |  2 +-
>  examples/ip_pipeline/Makefile                 |  2 +-
>  examples/ip_reassembly/Makefile               |  2 +-
>  examples/ipsec-secgw/Makefile                 |  2 +-
>  examples/ipv4_multicast/Makefile              |  2 +-
>  examples/kni/Makefile                         |  2 +-
>  examples/l2fwd-cat/Makefile                   |  2 +-
>  examples/l2fwd-crypto/Makefile                |  2 +-
>  examples/l2fwd-event/Makefile                 |  2 +-
>  examples/l2fwd-jobstats/Makefile              |  2 +-
>  examples/l2fwd-keepalive/Makefile             |  2 +-
>  examples/l2fwd/Makefile                       |  2 +-
>  examples/l3fwd-acl/Makefile                   |  2 +-
>  examples/l3fwd-power/Makefile                 |  2 +-
>  examples/l3fwd/Makefile                       |  2 +-
>  examples/link_status_interrupt/Makefile       |  2 +-
>  examples/meson.build                          |  6 +--
>  examples/ntb/Makefile                         |  2 +-
>  examples/packet_ordering/Makefile             |  2 +-
>  examples/ptpclient/Makefile                   |  2 +-
>  examples/qos_meter/Makefile                   |  2 +-
>  examples/qos_sched/Makefile                   |  2 +-
>  examples/rxtx_callbacks/Makefile              |  2 +-
>  examples/service_cores/Makefile               |  2 +-
>  examples/skeleton/Makefile                    |  2 +-
>  examples/tep_termination/Makefile             |  2 +-
>  examples/timer/Makefile                       |  2 +-
>  examples/vdpa/Makefile                        |  2 +-
>  examples/vhost/Makefile                       |  2 +-
>  examples/vhost_blk/Makefile                   |  2 +-
>  examples/vhost_crypto/Makefile                |  2 +-
>  examples/vmdq/Makefile                        |  2 +-
>  examples/vmdq_dcb/Makefile                    |  2 +-
>  lib/librte_telemetry/meson.build              |  1 -
>  lib/meson.build                               |  2 +-
>  meson.build                                   | 26 +----------
>  53 files changed, 134 insertions(+), 76 deletions(-)  create mode 100644
> buildtools/pkg-config/meson.build  create mode 100644 buildtools/pkg-
> config/set-static-linker-flags.py
> 
> --
> 2.20.1

This series allows OVS to consume the DPDK libraries appropriately (static and shared) and simplifies the build. 
When building OVS with DPDK, without this patch, the linker would always pick the shared libraries over static even after specifying the -Bstatic flag. 
With this patch, no such behavior is observed. So, it would be very helpful for the OVS community if this patch is merged in DPDK.

I would also ask this series to be back ported to 19.11 as OVS can immediately consume it on the latest master instead of waiting on the next DPDK release i.e 20.11 

Series-acked-by: Sunil Pai G <sunil.pai.g@intel.com>
Stokes, Ian June 30, 2020, 4:37 p.m. UTC | #3
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Bruce Richardson
> > Sent: Friday, May 1, 2020 7:24 PM
> > To: dev@dpdk.org
> > Cc: thomas@monjalon.net; bluca@debian.org; david.marchand@redhat.com;
> > ktraynor@redhat.com; Richardson, Bruce <bruce.richardson@intel.com>
> > Subject: [dpdk-dev] [PATCH v2 0/7] improve DPDK static builds with
> > meson
> >
> > This set fixes a number of minor issues with static builds when using
> > meson, both for linking apps/examples as part of a meson build itself
> > or when using pkg-config subsequently.
> >
> > Following this patchset, all DPDK static builds should be linking with
> > --whole- archive to ensure all lib and driver constructors are
> > included, and the use of pkg-config for doing static builds is
> > simplified. The downside is that for correctness we need two .pc files for DPDK
> rather than just one.
> >
> > v2: improved log messages for a number of patches, and clearly marked
> >     internal-only pkg-config file as such
> >
> > Bruce Richardson (7):
> >   build: always link-whole DPDK static libraries
> >   build: remove unnecessary variable
> >   devtools/test-meson-builds.sh: add pkg-config static builds
> >   build: move pkg-config creation to separate file
> >   build/pkg-config: output driver libs first for static build
> >   build/pkg-config: improve static linking flags
> >   build/pkg-config: prevent overlinking
> >
> >  app/meson.build                               |  2 +-
> >  app/test/meson.build                          |  2 +-
> >  buildtools/pkg-config/meson.build             | 43 +++++++++++++++++++
> >  .../pkg-config/set-static-linker-flags.py     | 38 ++++++++++++++++
> >  devtools/test-meson-builds.sh                 |  2 +-
> >  doc/guides/prog_guide/build-sdk-meson.rst     |  2 +-
> >  drivers/meson.build                           |  2 +-
> >  examples/bbdev_app/Makefile                   |  2 +-
> >  examples/bond/Makefile                        |  2 +-
> >  examples/cmdline/Makefile                     |  2 +-
> >  examples/distributor/Makefile                 |  2 +-
> >  examples/eventdev_pipeline/Makefile           |  2 +-
> >  examples/fips_validation/Makefile             |  2 +-
> >  examples/flow_classify/Makefile               |  2 +-
> >  examples/flow_filtering/Makefile              |  2 +-
> >  examples/helloworld/Makefile                  |  2 +-
> >  examples/ioat/Makefile                        |  2 +-
> >  examples/ip_fragmentation/Makefile            |  2 +-
> >  examples/ip_pipeline/Makefile                 |  2 +-
> >  examples/ip_reassembly/Makefile               |  2 +-
> >  examples/ipsec-secgw/Makefile                 |  2 +-
> >  examples/ipv4_multicast/Makefile              |  2 +-
> >  examples/kni/Makefile                         |  2 +-
> >  examples/l2fwd-cat/Makefile                   |  2 +-
> >  examples/l2fwd-crypto/Makefile                |  2 +-
> >  examples/l2fwd-event/Makefile                 |  2 +-
> >  examples/l2fwd-jobstats/Makefile              |  2 +-
> >  examples/l2fwd-keepalive/Makefile             |  2 +-
> >  examples/l2fwd/Makefile                       |  2 +-
> >  examples/l3fwd-acl/Makefile                   |  2 +-
> >  examples/l3fwd-power/Makefile                 |  2 +-
> >  examples/l3fwd/Makefile                       |  2 +-
> >  examples/link_status_interrupt/Makefile       |  2 +-
> >  examples/meson.build                          |  6 +--
> >  examples/ntb/Makefile                         |  2 +-
> >  examples/packet_ordering/Makefile             |  2 +-
> >  examples/ptpclient/Makefile                   |  2 +-
> >  examples/qos_meter/Makefile                   |  2 +-
> >  examples/qos_sched/Makefile                   |  2 +-
> >  examples/rxtx_callbacks/Makefile              |  2 +-
> >  examples/service_cores/Makefile               |  2 +-
> >  examples/skeleton/Makefile                    |  2 +-
> >  examples/tep_termination/Makefile             |  2 +-
> >  examples/timer/Makefile                       |  2 +-
> >  examples/vdpa/Makefile                        |  2 +-
> >  examples/vhost/Makefile                       |  2 +-
> >  examples/vhost_blk/Makefile                   |  2 +-
> >  examples/vhost_crypto/Makefile                |  2 +-
> >  examples/vmdq/Makefile                        |  2 +-
> >  examples/vmdq_dcb/Makefile                    |  2 +-
> >  lib/librte_telemetry/meson.build              |  1 -
> >  lib/meson.build                               |  2 +-
> >  meson.build                                   | 26 +----------
> >  53 files changed, 134 insertions(+), 76 deletions(-)  create mode
> > 100644 buildtools/pkg-config/meson.build  create mode 100644
> > buildtools/pkg- config/set-static-linker-flags.py
> >
> > --
> > 2.20.1
> 
> This series allows OVS to consume the DPDK libraries appropriately (static and
> shared) and simplifies the build.
> When building OVS with DPDK, without this patch, the linker would always pick
> the shared libraries over static even after specifying the -Bstatic flag.
> With this patch, no such behavior is observed. So, it would be very helpful for
> the OVS community if this patch is merged in DPDK.
> 
> I would also ask this series to be back ported to 19.11 as OVS can immediately
> consume it on the latest master instead of waiting on the next DPDK release i.e
> 20.11

+1, if this would be possible to apply to 19.11 as a backport it would enable OVS to move quicker and update the OVS Travis as well as docs and build scripts rather than waiting for DPDK 20.11. Sunil has already been working on these changes I believe.

Alternatively if it cant be backported to 19.11, then possibly we could look at merging Sunil's patches to the dpdk-latest branch in OVS once makefiles are deprecated in DPDK master?

Regards
Ian
> 
> Series-acked-by: Sunil Pai G <sunil.pai.g@intel.com>
>
Thomas Monjalon July 1, 2020, 5:29 p.m. UTC | #4
30/06/2020 18:37, Stokes, Ian:
> > This series allows OVS to consume the DPDK libraries appropriately (static and
> > shared) and simplifies the build.
> > When building OVS with DPDK, without this patch, the linker would always pick
> > the shared libraries over static even after specifying the -Bstatic flag.
> > With this patch, no such behavior is observed. So, it would be very helpful for
> > the OVS community if this patch is merged in DPDK.
> > 
> > I would also ask this series to be back ported to 19.11 as OVS can immediately
> > consume it on the latest master instead of waiting on the next DPDK release i.e
> > 20.11
> 
> +1, if this would be possible to apply to 19.11 as a backport it would enable OVS to move quicker and update the OVS Travis as well as docs and build scripts rather than waiting for DPDK 20.11. Sunil has already been working on these changes I believe.
> 
> Alternatively if it cant be backported to 19.11, then possibly we could look at merging Sunil's patches to the dpdk-latest branch in OVS once makefiles are deprecated in DPDK master?

Why waiting for makefiles to be deprecated?
These patches are merged in latest DPDK, so you can proceed in OVS.
Stokes, Ian July 2, 2020, 10:46 a.m. UTC | #5
> 30/06/2020 18:37, Stokes, Ian:
> > > This series allows OVS to consume the DPDK libraries appropriately
> > > (static and
> > > shared) and simplifies the build.
> > > When building OVS with DPDK, without this patch, the linker would
> > > always pick the shared libraries over static even after specifying the -
> Bstatic flag.
> > > With this patch, no such behavior is observed. So, it would be very
> > > helpful for the OVS community if this patch is merged in DPDK.
> > >
> > > I would also ask this series to be back ported to 19.11 as OVS can
> > > immediately consume it on the latest master instead of waiting on
> > > the next DPDK release i.e
> > > 20.11
> >
> > +1, if this would be possible to apply to 19.11 as a backport it would enable
> OVS to move quicker and update the OVS Travis as well as docs and build
> scripts rather than waiting for DPDK 20.11. Sunil has already been working on
> these changes I believe.
> >
> > Alternatively if it cant be backported to 19.11, then possibly we could look
> at merging Sunil's patches to the dpdk-latest branch in OVS once makefiles
> are deprecated in DPDK master?
> 
> Why waiting for makefiles to be deprecated?
> These patches are merged in latest DPDK, so you can proceed in OVS.
> 
> 

Good point, Sunil has a patchset ready to go, will have him post them to the OVS ML for dpdk-latest branch today.

Regards
Ian