[dpdk-dev,PATCHv2,3/4] Makefiles: Add experimental tag check and warnings to trigger on use

  Add checks during build to ensure that all symbols in the EXPERIMENTAL
version map section have __experimental tags on their definitions, and
enable the warnings needed to announce their use.  Also add a
ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
libraries to override those checks.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Thomas Monjalon <thomas@monjalon.net>
CC: "Mcnamara, John" <john.mcnamara@intel.com>
---
 app/test-eventdev/Makefile                 |  2 ++
 app/test-pmd/Makefile                      |  2 ++
 drivers/event/sw/Makefile                  |  2 ++
 drivers/net/failsafe/Makefile              |  2 ++
 drivers/net/ixgbe/Makefile                 |  2 ++
 examples/eventdev_pipeline_sw_pmd/Makefile |  2 ++
 examples/flow_classify/Makefile            |  2 ++
 examples/ipsec-secgw/Makefile              |  2 ++
 examples/service_cores/Makefile            |  2 ++
 lib/librte_eal/bsdapp/eal/Makefile         |  2 ++
 lib/librte_eal/linuxapp/Makefile           |  2 ++
 lib/librte_eal/linuxapp/eal/Makefile       |  4 ++++
 lib/librte_eventdev/Makefile               |  3 +++
 lib/librte_security/Makefile               |  3 +++
 mk/internal/rte.compile-pre.mk             | 14 ++++++++++++--
 mk/toolchain/clang/rte.vars.mk             |  2 +-
 mk/toolchain/gcc/rte.vars.mk               |  2 +-
 mk/toolchain/icc/rte.vars.mk               |  2 +-
 18 files changed, 47 insertions(+), 5 deletions(-)
  

On Fri, Dec 08, 2017 at 12:14:34PM -0500, Neil Horman wrote:
> Add checks during build to ensure that all symbols in the EXPERIMENTAL
> version map section have __experimental tags on their definitions, and
> enable the warnings needed to announce their use.  Also add a
> ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
> libraries to override those checks.
> 
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Thomas Monjalon <thomas@monjalon.net>
> CC: "Mcnamara, John" <john.mcnamara@intel.com>
> ---

Hi Neil,

if I read the patch functionality correctly, the
ALLOW_EXPERIMENTAL_FUNCTIONS variable simply suppresses all errors for
deprecated functions. However, what we really want is just to suppress
the errors for the experimental functions, and not any that really are
deprecated. While we may not have any now in DPDK, that doesn't mean it
might not be useful to have some in future.

Therefore, would an alternative scheme work where the experimental tag
is set to empty if the ALLOW_EXPERIMENTAL_FUNCTIONS define is set when
compiling?

/Bruce
  

On Mon, Dec 11, 2017 at 11:35:57AM +0000, Bruce Richardson wrote:
> On Fri, Dec 08, 2017 at 12:14:34PM -0500, Neil Horman wrote:
> > Add checks during build to ensure that all symbols in the EXPERIMENTAL
> > version map section have __experimental tags on their definitions, and
> > enable the warnings needed to announce their use.  Also add a
> > ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
> > libraries to override those checks.
> > 
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > CC: Thomas Monjalon <thomas@monjalon.net>
> > CC: "Mcnamara, John" <john.mcnamara@intel.com>
> > ---
> 
> Hi Neil,
> 
> if I read the patch functionality correctly, the
> ALLOW_EXPERIMENTAL_FUNCTIONS variable simply suppresses all errors for
> deprecated functions. However, what we really want is just to suppress
> the errors for the experimental functions, and not any that really are
> deprecated. While we may not have any now in DPDK, that doesn't mean it
> might not be useful to have some in future.
> 
Well, they are all deprecated by the stict definition of the term, which is to
say that their usage is discouraged/disapproved.  What I think you are saying is
that you would like to differentiate reasons for deprecation (i.e. deprecation
because an api is experimental, vs deprecation because an api is broken or going
to be removed).  I'm not sure I see that we will get there anytime soon, but
yes, that seems like it would be a good ability to add in here.

> Therefore, would an alternative scheme work where the experimental tag
> is set to empty if the ALLOW_EXPERIMENTAL_FUNCTIONS define is set when
> compiling?
> 
Yes, I think that could work.  Perhaps what we can do is modify
ALLOW_EXPERIMENTAL_FUNCTIONS to set a CFLAGS like:
-DSUPPRESS_EXPERIMENTAL

And use that to toggle the definition on rte_compat.h between something that
deprecates the API call and an empty definition

Neil

> /Bruce
>
  

On Mon, Dec 11, 2017 at 07:40:43AM -0500, Neil Horman wrote:
> On Mon, Dec 11, 2017 at 11:35:57AM +0000, Bruce Richardson wrote:
> > On Fri, Dec 08, 2017 at 12:14:34PM -0500, Neil Horman wrote:
> > > Add checks during build to ensure that all symbols in the EXPERIMENTAL
> > > version map section have __experimental tags on their definitions, and
> > > enable the warnings needed to announce their use.  Also add a
> > > ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
> > > libraries to override those checks.
> > > 
> > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > CC: Thomas Monjalon <thomas@monjalon.net>
> > > CC: "Mcnamara, John" <john.mcnamara@intel.com>
> > > ---
> > 
> > Hi Neil,
> > 
> > if I read the patch functionality correctly, the
> > ALLOW_EXPERIMENTAL_FUNCTIONS variable simply suppresses all errors for
> > deprecated functions. However, what we really want is just to suppress
> > the errors for the experimental functions, and not any that really are
> > deprecated. While we may not have any now in DPDK, that doesn't mean it
> > might not be useful to have some in future.
> > 
> Well, they are all deprecated by the stict definition of the term, which is to
> say that their usage is discouraged/disapproved.  What I think you are saying is
> that you would like to differentiate reasons for deprecation (i.e. deprecation
> because an api is experimental, vs deprecation because an api is broken or going
> to be removed).  I'm not sure I see that we will get there anytime soon, but
> yes, that seems like it would be a good ability to add in here.
> 
> > Therefore, would an alternative scheme work where the experimental tag
> > is set to empty if the ALLOW_EXPERIMENTAL_FUNCTIONS define is set when
> > compiling?
> > 
> Yes, I think that could work.  Perhaps what we can do is modify
> ALLOW_EXPERIMENTAL_FUNCTIONS to set a CFLAGS like:
> -DSUPPRESS_EXPERIMENTAL
> 
> And use that to toggle the definition on rte_compat.h between something that
> deprecates the API call and an empty definition
> 

Yes, exactly what I was thinking, but failed to express so clearly.

Also, would the flag be best set globally when building DPDK itself,
rather than changing individual makefiles per library?
  

On Mon, Dec 11, 2017 at 12:45:50PM +0000, Bruce Richardson wrote:
> On Mon, Dec 11, 2017 at 07:40:43AM -0500, Neil Horman wrote:
> > On Mon, Dec 11, 2017 at 11:35:57AM +0000, Bruce Richardson wrote:
> > > On Fri, Dec 08, 2017 at 12:14:34PM -0500, Neil Horman wrote:
> > > > Add checks during build to ensure that all symbols in the EXPERIMENTAL
> > > > version map section have __experimental tags on their definitions, and
> > > > enable the warnings needed to announce their use.  Also add a
> > > > ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
> > > > libraries to override those checks.
> > > > 
> > > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > > CC: Thomas Monjalon <thomas@monjalon.net>
> > > > CC: "Mcnamara, John" <john.mcnamara@intel.com>
> > > > ---
> > > 
> > > Hi Neil,
> > > 
> > > if I read the patch functionality correctly, the
> > > ALLOW_EXPERIMENTAL_FUNCTIONS variable simply suppresses all errors for
> > > deprecated functions. However, what we really want is just to suppress
> > > the errors for the experimental functions, and not any that really are
> > > deprecated. While we may not have any now in DPDK, that doesn't mean it
> > > might not be useful to have some in future.
> > > 
> > Well, they are all deprecated by the stict definition of the term, which is to
> > say that their usage is discouraged/disapproved.  What I think you are saying is
> > that you would like to differentiate reasons for deprecation (i.e. deprecation
> > because an api is experimental, vs deprecation because an api is broken or going
> > to be removed).  I'm not sure I see that we will get there anytime soon, but
> > yes, that seems like it would be a good ability to add in here.
> > 
> > > Therefore, would an alternative scheme work where the experimental tag
> > > is set to empty if the ALLOW_EXPERIMENTAL_FUNCTIONS define is set when
> > > compiling?
> > > 
> > Yes, I think that could work.  Perhaps what we can do is modify
> > ALLOW_EXPERIMENTAL_FUNCTIONS to set a CFLAGS like:
> > -DSUPPRESS_EXPERIMENTAL
> > 
> > And use that to toggle the definition on rte_compat.h between something that
> > deprecates the API call and an empty definition
> > 
> 
> Yes, exactly what I was thinking, but failed to express so clearly.
> 
> Also, would the flag be best set globally when building DPDK itself,
> rather than changing individual makefiles per library?
> 
My personal opinion would be that the ability to set allowances on a fine
grained scale would be preferable to a global exception.  I say that because I
could envision a new library getting written in which the author would not want
to use experimental APIS and would like the option to catch that during
development.  I would like to think that adding -DALLOW_EXPERIMENTAL_APIS to any
new makefile in the DPDK build would be both little enough trouble to do without
irritation, and significant enough of a bump during development as to force the
developer to make a conscious choice about enabling experimental APIS.  The idea
here is not to prevent developers from using experimental APIS (especially not
in the core dpdk library, as I noted previously), but to make them consciously
choose to use them, rather than just do so out of hand.

That said, I could be swayed if there was a sufficiently motivating reason to
grant a global exception.

Neil