[v2,0/3] Allow overriding of build-time constants

  A number of the more advanced DPDK build settings which are not expected to
be user modified are stored in config/rte_config.h. In some cases, for a
custom build a user may want to override those settings via CFLAGS, so we
need to ensure that the definitions do not override the user-provided
values.

Bruce Richardson (3):
  config: remove explicit undefinition of unset values
  config: allow overriding some build defaults
  doc: add notes on overriding extra config values

 config/rte_config.h                       | 122 +++++++++++++++++++---
 doc/guides/linux_gsg/build_dpdk.rst       |   8 ++
 doc/guides/prog_guide/build-sdk-meson.rst |   8 ++
 3 files changed, 121 insertions(+), 17 deletions(-)
  

On Thu, Sep 03, 2020 at 03:49:39PM +0100, Bruce Richardson wrote:
> A number of the more advanced DPDK build settings which are not expected to
> be user modified are stored in config/rte_config.h. In some cases, for a
> custom build a user may want to override those settings via CFLAGS, so we
> need to ensure that the definitions do not override the user-provided
> values.
> 
> Bruce Richardson (3):
>   config: remove explicit undefinition of unset values
>   config: allow overriding some build defaults
>   doc: add notes on overriding extra config values
> 
Ping for any further reviews or comments on this set?
  

Hi, Bruce

Can this patch be merged before RC1? It is very helpful for our verification.
Thanks.

Regards,
Chen Bo

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Bruce Richardson
> Sent: October 14, 2020 22:20
> To: dev@dpdk.org
> Cc: Ma, LihongX <lihongx.ma@intel.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>
> Subject: Re: [dpdk-dev] [PATCH v2 0/3] Allow overriding of build-time
> constants
> 
> On Thu, Sep 03, 2020 at 03:49:39PM +0100, Bruce Richardson wrote:
> > A number of the more advanced DPDK build settings which are not
> > expected to be user modified are stored in config/rte_config.h. In
> > some cases, for a custom build a user may want to override those
> > settings via CFLAGS, so we need to ensure that the definitions do not
> > override the user-provided values.
> >
> > Bruce Richardson (3):
> >   config: remove explicit undefinition of unset values
> >   config: allow overriding some build defaults
> >   doc: add notes on overriding extra config values
> >
> Ping for any further reviews or comments on this set?
  

On Thu, Oct 15, 2020 at 09:55:18AM +0100, Chen, BoX C wrote:
> Hi, Bruce
> 
> Can this patch be merged before RC1? It is very helpful for our verification.
> Thanks.
> 
> Regards,
> Chen Bo
> 

Understood, however, it is not fully reviewed and acked, so if you have
tested this whole set, please provide a tested-by tag in an email.
Similarly if you, or others, have time to review the set please do and
provide feedback and/or acks on it.

/Bruce

> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Bruce Richardson
> > Sent: October 14, 2020 22:20
> > To: dev@dpdk.org
> > Cc: Ma, LihongX <lihongx.ma@intel.com>; Hemant Agrawal
> > <hemant.agrawal@nxp.com>
> > Subject: Re: [dpdk-dev] [PATCH v2 0/3] Allow overriding of build-time
> > constants
> >
> > On Thu, Sep 03, 2020 at 03:49:39PM +0100, Bruce Richardson wrote:
> > > A number of the more advanced DPDK build settings which are not
> > > expected to be user modified are stored in config/rte_config.h. In
> > > some cases, for a custom build a user may want to override those
> > > settings via CFLAGS, so we need to ensure that the definitions do not
> > > override the user-provided values.
> > >
> > > Bruce Richardson (3):
> > >   config: remove explicit undefinition of unset values
> > >   config: allow overriding some build defaults
> > >   doc: add notes on overriding extra config values
> > >
> > Ping for any further reviews or comments on this set?
  

Hello Bruce,

On Thu, Sep 3, 2020 at 4:50 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> A number of the more advanced DPDK build settings which are not expected to
> be user modified are stored in config/rte_config.h. In some cases, for a
> custom build a user may want to override those settings via CFLAGS, so we
> need to ensure that the definitions do not override the user-provided
> values.
>
> Bruce Richardson (3):
>   config: remove explicit undefinition of unset values
>   config: allow overriding some build defaults
>   doc: add notes on overriding extra config values

$ CFLAGS="-DRTE_MAX_MEMSEG_LISTS=64" meson setup
--default-library=shared --buildtype=debugoptimized
-Dprefix=/home/dmarchan/git/pub/dpdk.org/build/install build
$ ninja-build -C build -j4 install


librte_eal.so is indeed built with the 64 value:
$ pahole -C rte_mem_config build/install/lib64/librte_eal.so |grep memsegs
die__process_function: tag not supported (INVALID)!
    struct rte_memseg_list     memsegs[64];          /*   136  8704 */


But no trace of the custom value for external applications:
$ grep -r RTE_MAX_MEMSEG_LISTS build/install
build/install/include/rte_config.h:#ifndef RTE_MAX_MEMSEG_LISTS
build/install/include/rte_config.h:#define RTE_MAX_MEMSEG_LISTS 128
Binary file build/install/lib64/librte_eal.a matches
Binary file build/install/lib64/librte_eal.so.21.0 matches

I can see the same using the meson option -Dc_args.
  

On Fri, Oct 16, 2020 at 05:47:45PM +0200, David Marchand wrote:
> Hello Bruce,
> 
> On Thu, Sep 3, 2020 at 4:50 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > A number of the more advanced DPDK build settings which are not expected to
> > be user modified are stored in config/rte_config.h. In some cases, for a
> > custom build a user may want to override those settings via CFLAGS, so we
> > need to ensure that the definitions do not override the user-provided
> > values.
> >
> > Bruce Richardson (3):
> >   config: remove explicit undefinition of unset values
> >   config: allow overriding some build defaults
> >   doc: add notes on overriding extra config values
> 
> $ CFLAGS="-DRTE_MAX_MEMSEG_LISTS=64" meson setup
> --default-library=shared --buildtype=debugoptimized
> -Dprefix=/home/dmarchan/git/pub/dpdk.org/build/install build
> $ ninja-build -C build -j4 install
> 
> 
> librte_eal.so is indeed built with the 64 value:
> $ pahole -C rte_mem_config build/install/lib64/librte_eal.so |grep memsegs
> die__process_function: tag not supported (INVALID)!
>     struct rte_memseg_list     memsegs[64];          /*   136  8704 */
> 
> 
> But no trace of the custom value for external applications:
> $ grep -r RTE_MAX_MEMSEG_LISTS build/install
> build/install/include/rte_config.h:#ifndef RTE_MAX_MEMSEG_LISTS
> build/install/include/rte_config.h:#define RTE_MAX_MEMSEG_LISTS 128
> Binary file build/install/lib64/librte_eal.a matches
> Binary file build/install/lib64/librte_eal.so.21.0 matches
> 
> I can see the same using the meson option -Dc_args.
> 

Good point, I had not thought of external apps using these values.

They are mostly for internal use, so maybe its worthwhile looking to not
have them in a public header file. What do you think? Is it likely that
apps would be using some of these values, or needs to know the specifics?

/Bruce
  

On Fri, Oct 16, 2020 at 5:56 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
> > librte_eal.so is indeed built with the 64 value:
> > $ pahole -C rte_mem_config build/install/lib64/librte_eal.so |grep memsegs
> > die__process_function: tag not supported (INVALID)!
> >     struct rte_memseg_list     memsegs[64];          /*   136  8704 */
> >
> >
> > But no trace of the custom value for external applications:
> > $ grep -r RTE_MAX_MEMSEG_LISTS build/install
> > build/install/include/rte_config.h:#ifndef RTE_MAX_MEMSEG_LISTS
> > build/install/include/rte_config.h:#define RTE_MAX_MEMSEG_LISTS 128
> > Binary file build/install/lib64/librte_eal.a matches
> > Binary file build/install/lib64/librte_eal.so.21.0 matches
> >
> > I can see the same using the meson option -Dc_args.
> >
>
> Good point, I had not thought of external apps using these values.
>
> They are mostly for internal use, so maybe its worthwhile looking to not
> have them in a public header file. What do you think? Is it likely that
> apps would be using some of these values, or needs to know the specifics?

Some are publicly exposed, like RTE_MEMPOOL_CACHE_MAX_SIZE,
RTE_PKTMBUF_HEADROOM, RTE_ETHDEV_RXTX_CALLBACKS,
For those, either we propagate the overriden value to the installed
rte_config.h or we refuse customisation.
  

On Fri, Oct 16, 2020 at 06:46:12PM +0200, David Marchand wrote:
> On Fri, Oct 16, 2020 at 5:56 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> > > librte_eal.so is indeed built with the 64 value:
> > > $ pahole -C rte_mem_config build/install/lib64/librte_eal.so |grep memsegs
> > > die__process_function: tag not supported (INVALID)!
> > >     struct rte_memseg_list     memsegs[64];          /*   136  8704 */
> > >
> > >
> > > But no trace of the custom value for external applications:
> > > $ grep -r RTE_MAX_MEMSEG_LISTS build/install
> > > build/install/include/rte_config.h:#ifndef RTE_MAX_MEMSEG_LISTS
> > > build/install/include/rte_config.h:#define RTE_MAX_MEMSEG_LISTS 128
> > > Binary file build/install/lib64/librte_eal.a matches
> > > Binary file build/install/lib64/librte_eal.so.21.0 matches
> > >
> > > I can see the same using the meson option -Dc_args.
> > >
> >
> > Good point, I had not thought of external apps using these values.
> >
> > They are mostly for internal use, so maybe its worthwhile looking to not
> > have them in a public header file. What do you think? Is it likely that
> > apps would be using some of these values, or needs to know the specifics?
> 
> Some are publicly exposed, like RTE_MEMPOOL_CACHE_MAX_SIZE,
> RTE_PKTMBUF_HEADROOM, RTE_ETHDEV_RXTX_CALLBACKS,
> For those, either we propagate the overriden value to the installed
> rte_config.h or we refuse customisation.
> 
I'd suggest the first 2 of those should possibly be global meson options.
Third should probably not be exposed at all.

/Bruce
  

19/10/2020 12:21, Bruce Richardson:
> On Fri, Oct 16, 2020 at 06:46:12PM +0200, David Marchand wrote:
> > On Fri, Oct 16, 2020 at 5:56 PM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > > > librte_eal.so is indeed built with the 64 value:
> > > > $ pahole -C rte_mem_config build/install/lib64/librte_eal.so |grep memsegs
> > > > die__process_function: tag not supported (INVALID)!
> > > >     struct rte_memseg_list     memsegs[64];          /*   136  8704 */
> > > >
> > > >
> > > > But no trace of the custom value for external applications:
> > > > $ grep -r RTE_MAX_MEMSEG_LISTS build/install
> > > > build/install/include/rte_config.h:#ifndef RTE_MAX_MEMSEG_LISTS
> > > > build/install/include/rte_config.h:#define RTE_MAX_MEMSEG_LISTS 128
> > > > Binary file build/install/lib64/librte_eal.a matches
> > > > Binary file build/install/lib64/librte_eal.so.21.0 matches
> > > >
> > > > I can see the same using the meson option -Dc_args.
> > >
> > > Good point, I had not thought of external apps using these values.
> > >
> > > They are mostly for internal use, so maybe its worthwhile looking to not
> > > have them in a public header file. What do you think? Is it likely that
> > > apps would be using some of these values, or needs to know the specifics?
> > 
> > Some are publicly exposed, like RTE_MEMPOOL_CACHE_MAX_SIZE,
> > RTE_PKTMBUF_HEADROOM, RTE_ETHDEV_RXTX_CALLBACKS,
> > For those, either we propagate the overriden value to the installed
> > rte_config.h or we refuse customisation.
> > 
> I'd suggest the first 2 of those should possibly be global meson options.

How the application is reading the meson options?

> Third should probably not be exposed at all.

I think everything should be exposed.
The application may need to know whether a feature is enabled or not,
and what is a specific tuning value.

I suspect it is the last blocker for meson adoption.
Now that we removed the makefiles, we need to fill this gap urgently.
  

On Mon, Oct 19, 2020 at 11:04:54PM +0200, Thomas Monjalon wrote:
> 19/10/2020 12:21, Bruce Richardson:
> > On Fri, Oct 16, 2020 at 06:46:12PM +0200, David Marchand wrote:
> > > On Fri, Oct 16, 2020 at 5:56 PM Bruce Richardson
> > > <bruce.richardson@intel.com> wrote:
> > > > > librte_eal.so is indeed built with the 64 value:
> > > > > $ pahole -C rte_mem_config build/install/lib64/librte_eal.so |grep memsegs
> > > > > die__process_function: tag not supported (INVALID)!
> > > > >     struct rte_memseg_list     memsegs[64];          /*   136  8704 */
> > > > >
> > > > >
> > > > > But no trace of the custom value for external applications:
> > > > > $ grep -r RTE_MAX_MEMSEG_LISTS build/install
> > > > > build/install/include/rte_config.h:#ifndef RTE_MAX_MEMSEG_LISTS
> > > > > build/install/include/rte_config.h:#define RTE_MAX_MEMSEG_LISTS 128
> > > > > Binary file build/install/lib64/librte_eal.a matches
> > > > > Binary file build/install/lib64/librte_eal.so.21.0 matches
> > > > >
> > > > > I can see the same using the meson option -Dc_args.
> > > >
> > > > Good point, I had not thought of external apps using these values.
> > > >
> > > > They are mostly for internal use, so maybe its worthwhile looking to not
> > > > have them in a public header file. What do you think? Is it likely that
> > > > apps would be using some of these values, or needs to know the specifics?
> > > 
> > > Some are publicly exposed, like RTE_MEMPOOL_CACHE_MAX_SIZE,
> > > RTE_PKTMBUF_HEADROOM, RTE_ETHDEV_RXTX_CALLBACKS,
> > > For those, either we propagate the overriden value to the installed
> > > rte_config.h or we refuse customisation.
> > > 
> > I'd suggest the first 2 of those should possibly be global meson options.
> 
> How the application is reading the meson options?
> 
The meson options are reflected in the rte_build_config.h file. It's not
automatic, but they should be set there by the config/meson.build file. If
some are missed, they can be added, but I disagree that all meson options
should always be passed through to apps. It makes them part of the API,
perhaps unnecessarily, and therefore harder to change or adjust.
Furthermore why should an app ever need to care if a DPDK build included
the docs, or the kmods?

> > Third should probably not be exposed at all.
> 
> I think everything should be exposed.
> The application may need to know whether a feature is enabled or not,
> and what is a specific tuning value.
> 
> I suspect it is the last blocker for meson adoption.  Now that we removed
> the makefiles, we need to fill this gap urgently.
> 
I really don't see this as a gap. With "make" we struggled with massive
amounts of build config, and we tried to remove as much as we can. While
this is reporting what's there rather than tweaking it, surely many of
these settings are just better as #defines in the individual header files -
if they need to be exposed at all.

/Bruce
  

20/10/2020 10:34, Bruce Richardson:
> On Mon, Oct 19, 2020 at 11:04:54PM +0200, Thomas Monjalon wrote:
> > 19/10/2020 12:21, Bruce Richardson:
> > > On Fri, Oct 16, 2020 at 06:46:12PM +0200, David Marchand wrote:
> > > > On Fri, Oct 16, 2020 at 5:56 PM Bruce Richardson
> > > > <bruce.richardson@intel.com> wrote:
> > > > > > librte_eal.so is indeed built with the 64 value:
> > > > > > $ pahole -C rte_mem_config build/install/lib64/librte_eal.so |grep memsegs
> > > > > > die__process_function: tag not supported (INVALID)!
> > > > > >     struct rte_memseg_list     memsegs[64];          /*   136  8704 */
> > > > > >
> > > > > >
> > > > > > But no trace of the custom value for external applications:
> > > > > > $ grep -r RTE_MAX_MEMSEG_LISTS build/install
> > > > > > build/install/include/rte_config.h:#ifndef RTE_MAX_MEMSEG_LISTS
> > > > > > build/install/include/rte_config.h:#define RTE_MAX_MEMSEG_LISTS 128
> > > > > > Binary file build/install/lib64/librte_eal.a matches
> > > > > > Binary file build/install/lib64/librte_eal.so.21.0 matches
> > > > > >
> > > > > > I can see the same using the meson option -Dc_args.
> > > > >
> > > > > Good point, I had not thought of external apps using these values.
> > > > >
> > > > > They are mostly for internal use, so maybe its worthwhile looking to not
> > > > > have them in a public header file. What do you think? Is it likely that
> > > > > apps would be using some of these values, or needs to know the specifics?
> > > > 
> > > > Some are publicly exposed, like RTE_MEMPOOL_CACHE_MAX_SIZE,
> > > > RTE_PKTMBUF_HEADROOM, RTE_ETHDEV_RXTX_CALLBACKS,
> > > > For those, either we propagate the overriden value to the installed
> > > > rte_config.h or we refuse customisation.
> > > > 
> > > I'd suggest the first 2 of those should possibly be global meson options.
> > 
> > How the application is reading the meson options?
> > 
> The meson options are reflected in the rte_build_config.h file. It's not
> automatic, but they should be set there by the config/meson.build file. If
> some are missed, they can be added, but I disagree that all meson options
> should always be passed through to apps. It makes them part of the API,
> perhaps unnecessarily, and therefore harder to change or adjust.
> Furthermore why should an app ever need to care if a DPDK build included
> the docs, or the kmods?
> 
> > > Third should probably not be exposed at all.
> > 
> > I think everything should be exposed.
> > The application may need to know whether a feature is enabled or not,
> > and what is a specific tuning value.
> > 
> > I suspect it is the last blocker for meson adoption.  Now that we removed
> > the makefiles, we need to fill this gap urgently.
> > 
> I really don't see this as a gap. With "make" we struggled with massive
> amounts of build config, and we tried to remove as much as we can. While
> this is reporting what's there rather than tweaking it, surely many of
> these settings are just better as #defines in the individual header files -
> if they need to be exposed at all.

I agree with the goal of moving these #defines internally.
I just feel having wrong values in rte_config.h looks to be a bug.
  

On Tue, Oct 20, 2020 at 12:04:56PM +0200, Thomas Monjalon wrote:
> 20/10/2020 10:34, Bruce Richardson:
> > On Mon, Oct 19, 2020 at 11:04:54PM +0200, Thomas Monjalon wrote:
> > > 19/10/2020 12:21, Bruce Richardson:
> > > > On Fri, Oct 16, 2020 at 06:46:12PM +0200, David Marchand wrote:
> > > > > On Fri, Oct 16, 2020 at 5:56 PM Bruce Richardson
> > > > > <bruce.richardson@intel.com> wrote:
> > > > > > > librte_eal.so is indeed built with the 64 value: $ pahole -C
> > > > > > > rte_mem_config build/install/lib64/librte_eal.so |grep
> > > > > > > memsegs die__process_function: tag not supported (INVALID)!
> > > > > > > struct rte_memseg_list     memsegs[64];          /*   136
> > > > > > > 8704 */
> > > > > > >
> > > > > > >
> > > > > > > But no trace of the custom value for external applications: $
> > > > > > > grep -r RTE_MAX_MEMSEG_LISTS build/install
> > > > > > > build/install/include/rte_config.h:#ifndef
> > > > > > > RTE_MAX_MEMSEG_LISTS
> > > > > > > build/install/include/rte_config.h:#define
> > > > > > > RTE_MAX_MEMSEG_LISTS 128 Binary file
> > > > > > > build/install/lib64/librte_eal.a matches Binary file
> > > > > > > build/install/lib64/librte_eal.so.21.0 matches
> > > > > > >
> > > > > > > I can see the same using the meson option -Dc_args.
> > > > > >
> > > > > > Good point, I had not thought of external apps using these
> > > > > > values.
> > > > > >
> > > > > > They are mostly for internal use, so maybe its worthwhile
> > > > > > looking to not have them in a public header file. What do you
> > > > > > think? Is it likely that apps would be using some of these
> > > > > > values, or needs to know the specifics?
> > > > > 
> > > > > Some are publicly exposed, like RTE_MEMPOOL_CACHE_MAX_SIZE,
> > > > > RTE_PKTMBUF_HEADROOM, RTE_ETHDEV_RXTX_CALLBACKS, For those,
> > > > > either we propagate the overriden value to the installed
> > > > > rte_config.h or we refuse customisation.
> > > > > 
> > > > I'd suggest the first 2 of those should possibly be global meson
> > > > options.
> > > 
> > > How the application is reading the meson options?
> > > 
> > The meson options are reflected in the rte_build_config.h file. It's
> > not automatic, but they should be set there by the config/meson.build
> > file. If some are missed, they can be added, but I disagree that all
> > meson options should always be passed through to apps. It makes them
> > part of the API, perhaps unnecessarily, and therefore harder to change
> > or adjust.  Furthermore why should an app ever need to care if a DPDK
> > build included the docs, or the kmods?
> > 
> > > > Third should probably not be exposed at all.
> > > 
> > > I think everything should be exposed.  The application may need to
> > > know whether a feature is enabled or not, and what is a specific
> > > tuning value.
> > > 
> > > I suspect it is the last blocker for meson adoption.  Now that we
> > > removed the makefiles, we need to fill this gap urgently.
> > > 
> > I really don't see this as a gap. With "make" we struggled with massive
> > amounts of build config, and we tried to remove as much as we can.
> > While this is reporting what's there rather than tweaking it, surely
> > many of these settings are just better as #defines in the individual
> > header files - if they need to be exposed at all.
> 
> I agree with the goal of moving these #defines internally.  I just feel
> having wrong values in rte_config.h looks to be a bug.
> 
Oh agree, absolutely!  The other attitude to take (and document) here is
that if someone is going to the trouble to override the default DPDK value
when building DPDK they also need to provide that CFLAG when building their
app too. If they don't want that, they need to adjust the value in the code
directly.
  

On Thu, Sep 03, 2020 at 03:49:39PM +0100, Bruce Richardson wrote:
> A number of the more advanced DPDK build settings which are not expected to
> be user modified are stored in config/rte_config.h. In some cases, for a
> custom build a user may want to override those settings via CFLAGS, so we
> need to ensure that the definitions do not override the user-provided
> values.
> 
> Bruce Richardson (3):
>   config: remove explicit undefinition of unset values
>   config: allow overriding some build defaults
>   doc: add notes on overriding extra config values

Since there are issues flagged on this set, and I don't have a good
solution to fix them right now, I think the best approach is to drop this
set for consideration for merging in 20.11 release. As such, I'm going to
mark this set as rejected in patchwork.

If we still have a problem here in future, we can look at this again, but I
don't see the current patchset going ahead in its current form.

Regards,
/Bruce