[dpdk-dev,PATCHv2,3/4] Makefiles: Add experimental tag check and warnings to trigger on use
Checks
Commit Message
Add checks during build to ensure that all symbols in the EXPERIMENTAL
version map section have __experimental tags on their definitions, and
enable the warnings needed to announce their use. Also add a
ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
libraries to override those checks.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Thomas Monjalon <thomas@monjalon.net>
CC: "Mcnamara, John" <john.mcnamara@intel.com>
---
app/test-eventdev/Makefile | 2 ++
app/test-pmd/Makefile | 2 ++
drivers/event/sw/Makefile | 2 ++
drivers/net/failsafe/Makefile | 2 ++
drivers/net/ixgbe/Makefile | 2 ++
examples/eventdev_pipeline_sw_pmd/Makefile | 2 ++
examples/flow_classify/Makefile | 2 ++
examples/ipsec-secgw/Makefile | 2 ++
examples/service_cores/Makefile | 2 ++
lib/librte_eal/bsdapp/eal/Makefile | 2 ++
lib/librte_eal/linuxapp/Makefile | 2 ++
lib/librte_eal/linuxapp/eal/Makefile | 4 ++++
lib/librte_eventdev/Makefile | 3 +++
lib/librte_security/Makefile | 3 +++
mk/internal/rte.compile-pre.mk | 14 ++++++++++++--
mk/toolchain/clang/rte.vars.mk | 2 +-
mk/toolchain/gcc/rte.vars.mk | 2 +-
mk/toolchain/icc/rte.vars.mk | 2 +-
18 files changed, 47 insertions(+), 5 deletions(-)
Comments
On Fri, Dec 08, 2017 at 12:14:34PM -0500, Neil Horman wrote:
> Add checks during build to ensure that all symbols in the EXPERIMENTAL
> version map section have __experimental tags on their definitions, and
> enable the warnings needed to announce their use. Also add a
> ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
> libraries to override those checks.
>
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Thomas Monjalon <thomas@monjalon.net>
> CC: "Mcnamara, John" <john.mcnamara@intel.com>
> ---
Hi Neil,
if I read the patch functionality correctly, the
ALLOW_EXPERIMENTAL_FUNCTIONS variable simply suppresses all errors for
deprecated functions. However, what we really want is just to suppress
the errors for the experimental functions, and not any that really are
deprecated. While we may not have any now in DPDK, that doesn't mean it
might not be useful to have some in future.
Therefore, would an alternative scheme work where the experimental tag
is set to empty if the ALLOW_EXPERIMENTAL_FUNCTIONS define is set when
compiling?
/Bruce
On Mon, Dec 11, 2017 at 11:35:57AM +0000, Bruce Richardson wrote:
> On Fri, Dec 08, 2017 at 12:14:34PM -0500, Neil Horman wrote:
> > Add checks during build to ensure that all symbols in the EXPERIMENTAL
> > version map section have __experimental tags on their definitions, and
> > enable the warnings needed to announce their use. Also add a
> > ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
> > libraries to override those checks.
> >
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > CC: Thomas Monjalon <thomas@monjalon.net>
> > CC: "Mcnamara, John" <john.mcnamara@intel.com>
> > ---
>
> Hi Neil,
>
> if I read the patch functionality correctly, the
> ALLOW_EXPERIMENTAL_FUNCTIONS variable simply suppresses all errors for
> deprecated functions. However, what we really want is just to suppress
> the errors for the experimental functions, and not any that really are
> deprecated. While we may not have any now in DPDK, that doesn't mean it
> might not be useful to have some in future.
>
Well, they are all deprecated by the stict definition of the term, which is to
say that their usage is discouraged/disapproved. What I think you are saying is
that you would like to differentiate reasons for deprecation (i.e. deprecation
because an api is experimental, vs deprecation because an api is broken or going
to be removed). I'm not sure I see that we will get there anytime soon, but
yes, that seems like it would be a good ability to add in here.
> Therefore, would an alternative scheme work where the experimental tag
> is set to empty if the ALLOW_EXPERIMENTAL_FUNCTIONS define is set when
> compiling?
>
Yes, I think that could work. Perhaps what we can do is modify
ALLOW_EXPERIMENTAL_FUNCTIONS to set a CFLAGS like:
-DSUPPRESS_EXPERIMENTAL
And use that to toggle the definition on rte_compat.h between something that
deprecates the API call and an empty definition
Neil
> /Bruce
>
On Mon, Dec 11, 2017 at 07:40:43AM -0500, Neil Horman wrote:
> On Mon, Dec 11, 2017 at 11:35:57AM +0000, Bruce Richardson wrote:
> > On Fri, Dec 08, 2017 at 12:14:34PM -0500, Neil Horman wrote:
> > > Add checks during build to ensure that all symbols in the EXPERIMENTAL
> > > version map section have __experimental tags on their definitions, and
> > > enable the warnings needed to announce their use. Also add a
> > > ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
> > > libraries to override those checks.
> > >
> > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > CC: Thomas Monjalon <thomas@monjalon.net>
> > > CC: "Mcnamara, John" <john.mcnamara@intel.com>
> > > ---
> >
> > Hi Neil,
> >
> > if I read the patch functionality correctly, the
> > ALLOW_EXPERIMENTAL_FUNCTIONS variable simply suppresses all errors for
> > deprecated functions. However, what we really want is just to suppress
> > the errors for the experimental functions, and not any that really are
> > deprecated. While we may not have any now in DPDK, that doesn't mean it
> > might not be useful to have some in future.
> >
> Well, they are all deprecated by the stict definition of the term, which is to
> say that their usage is discouraged/disapproved. What I think you are saying is
> that you would like to differentiate reasons for deprecation (i.e. deprecation
> because an api is experimental, vs deprecation because an api is broken or going
> to be removed). I'm not sure I see that we will get there anytime soon, but
> yes, that seems like it would be a good ability to add in here.
>
> > Therefore, would an alternative scheme work where the experimental tag
> > is set to empty if the ALLOW_EXPERIMENTAL_FUNCTIONS define is set when
> > compiling?
> >
> Yes, I think that could work. Perhaps what we can do is modify
> ALLOW_EXPERIMENTAL_FUNCTIONS to set a CFLAGS like:
> -DSUPPRESS_EXPERIMENTAL
>
> And use that to toggle the definition on rte_compat.h between something that
> deprecates the API call and an empty definition
>
Yes, exactly what I was thinking, but failed to express so clearly.
Also, would the flag be best set globally when building DPDK itself,
rather than changing individual makefiles per library?
On Mon, Dec 11, 2017 at 12:45:50PM +0000, Bruce Richardson wrote:
> On Mon, Dec 11, 2017 at 07:40:43AM -0500, Neil Horman wrote:
> > On Mon, Dec 11, 2017 at 11:35:57AM +0000, Bruce Richardson wrote:
> > > On Fri, Dec 08, 2017 at 12:14:34PM -0500, Neil Horman wrote:
> > > > Add checks during build to ensure that all symbols in the EXPERIMENTAL
> > > > version map section have __experimental tags on their definitions, and
> > > > enable the warnings needed to announce their use. Also add a
> > > > ALLOW_EXPERIMENTAL_FUNCTIONS variable check to allow for in-tree dpdk
> > > > libraries to override those checks.
> > > >
> > > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > > CC: Thomas Monjalon <thomas@monjalon.net>
> > > > CC: "Mcnamara, John" <john.mcnamara@intel.com>
> > > > ---
> > >
> > > Hi Neil,
> > >
> > > if I read the patch functionality correctly, the
> > > ALLOW_EXPERIMENTAL_FUNCTIONS variable simply suppresses all errors for
> > > deprecated functions. However, what we really want is just to suppress
> > > the errors for the experimental functions, and not any that really are
> > > deprecated. While we may not have any now in DPDK, that doesn't mean it
> > > might not be useful to have some in future.
> > >
> > Well, they are all deprecated by the stict definition of the term, which is to
> > say that their usage is discouraged/disapproved. What I think you are saying is
> > that you would like to differentiate reasons for deprecation (i.e. deprecation
> > because an api is experimental, vs deprecation because an api is broken or going
> > to be removed). I'm not sure I see that we will get there anytime soon, but
> > yes, that seems like it would be a good ability to add in here.
> >
> > > Therefore, would an alternative scheme work where the experimental tag
> > > is set to empty if the ALLOW_EXPERIMENTAL_FUNCTIONS define is set when
> > > compiling?
> > >
> > Yes, I think that could work. Perhaps what we can do is modify
> > ALLOW_EXPERIMENTAL_FUNCTIONS to set a CFLAGS like:
> > -DSUPPRESS_EXPERIMENTAL
> >
> > And use that to toggle the definition on rte_compat.h between something that
> > deprecates the API call and an empty definition
> >
>
> Yes, exactly what I was thinking, but failed to express so clearly.
>
> Also, would the flag be best set globally when building DPDK itself,
> rather than changing individual makefiles per library?
>
My personal opinion would be that the ability to set allowances on a fine
grained scale would be preferable to a global exception. I say that because I
could envision a new library getting written in which the author would not want
to use experimental APIS and would like the option to catch that during
development. I would like to think that adding -DALLOW_EXPERIMENTAL_APIS to any
new makefile in the DPDK build would be both little enough trouble to do without
irritation, and significant enough of a bump during development as to force the
developer to make a conscious choice about enabling experimental APIS. The idea
here is not to prevent developers from using experimental APIS (especially not
in the core dpdk library, as I noted previously), but to make them consciously
choose to use them, rather than just do so out of hand.
That said, I could be swayed if there was a sufficiently motivating reason to
grant a global exception.
Neil
@@ -32,6 +32,8 @@ include $(RTE_SDK)/mk/rte.vars.mk
APP = dpdk-test-eventdev
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
@@ -33,6 +33,8 @@ include $(RTE_SDK)/mk/rte.vars.mk
ifeq ($(CONFIG_RTE_TEST_PMD),y)
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
#
# library name
#
@@ -33,6 +33,8 @@ include $(RTE_SDK)/mk/rte.vars.mk
# library name
LIB = librte_pmd_sw_event.a
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
# build flags
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
@@ -36,6 +36,8 @@ LIB = librte_pmd_failsafe.a
EXPORT_MAP := rte_pmd_failsafe_version.map
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
LIBABIVER := 1
# Sources are stored in SRCS-y
@@ -36,6 +36,8 @@ include $(RTE_SDK)/mk/rte.vars.mk
#
LIB = librte_pmd_ixgbe.a
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
@@ -32,6 +32,8 @@ ifeq ($(RTE_SDK),)
$(error "Please define RTE_SDK environment variable")
endif
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
# Default target, can be overridden by command line or environment
RTE_TARGET ?= x86_64-native-linuxapp-gcc
@@ -33,6 +33,8 @@ ifeq ($(RTE_SDK),)
$(error "Please define RTE_SDK environment variable")
endif
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
# Default target, can be overridden by command line or environment
RTE_TARGET ?= x86_64-native-linuxapp-gcc
@@ -33,6 +33,8 @@ ifeq ($(RTE_SDK),)
$(error "Please define RTE_SDK environment variable")
endif
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
# Default target, can be overridden by command line or environment
RTE_TARGET ?= x86_64-native-linuxapp-gcc
@@ -32,6 +32,8 @@ ifeq ($(RTE_SDK),)
$(error "Please define RTE_SDK environment variable")
endif
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
# Default target, can be overridden by command line or environment
RTE_TARGET ?= x86_64-native-linuxapp-gcc
@@ -33,6 +33,8 @@ include $(RTE_SDK)/mk/rte.vars.mk
LIB = librte_eal.a
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
ARCH_DIR ?= $(RTE_ARCH)
VPATH += $(RTE_SDK)/lib/librte_eal/common
VPATH += $(RTE_SDK)/lib/librte_eal/common/arch/$(ARCH_DIR)
@@ -31,6 +31,8 @@
include $(RTE_SDK)/mk/rte.vars.mk
+ALLOW_EXPERIMENTAL_FUNCTIONS := 2
+
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal
DIRS-$(CONFIG_RTE_EAL_IGB_UIO) += igb_uio
DIRS-$(CONFIG_RTE_KNI_KMOD) += kni
@@ -34,6 +34,10 @@ include $(RTE_SDK)/mk/rte.vars.mk
LIB = librte_eal.a
ARCH_DIR ?= $(RTE_ARCH)
+
+#Allow the use of experimental functions without warning
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
EXPORT_MAP := ../../rte_eal_version.map
VPATH += $(RTE_SDK)/lib/librte_eal/common/arch/$(ARCH_DIR)
@@ -36,6 +36,9 @@ LIB = librte_eventdev.a
# library version
LIBABIVER := 3
+# Allow the use of experimental functions
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
# build flags
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
@@ -36,6 +36,9 @@ LIB = librte_security.a
# library version
LIBABIVER := 1
+# Allow the use of experimental functions without warnings
+ALLOW_EXPERIMENTAL_FUNCTIONS := 1
+
# build flags
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
@@ -72,18 +72,27 @@ CMDS-all := $(CMDS-y) $(CMDS-n) $(CMDS-)
-include $(DEPS-y) $(CMDS-y)
+ifdef ALLOW_EXPERIMENTAL_FUNCTIONS
+EXPERIMENTAL_CFLAGS := $(subst 1, -Wno-deprecated-declarations -Wno-deprecated -Wno-error=deprecated-declarations -Wno-error=deprecated, $(ALLOW_EXPERIMENTAL_FUNCTIONS))
+else
+EXPERIMENTAL_CFLAGS :=
+endif
+
# command to compile a .c file to generate an object
ifeq ($(USE_HOST),1)
C_TO_O = $(HOSTCC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(HOST_CPPFLAGS) $(HOST_CFLAGS) \
- $(CFLAGS_$(@)) $(HOST_EXTRA_CPPFLAGS) $(HOST_EXTRA_CFLAGS) -o $@ -c $<
+ $(CFLAGS_$(@)) $(HOST_EXTRA_CPPFLAGS) $(HOST_EXTRA_CFLAGS) $(EXPERIMENTAL_CFLAGS) -o $@ -c $<
C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," HOSTCC $(@)")
else
C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CPPFLAGS) $(CFLAGS) \
- $(CFLAGS_$(@)) $(EXTRA_CPPFLAGS) $(EXTRA_CFLAGS) -o $@ -c $<
+ $(CFLAGS_$(@)) $(EXTRA_CPPFLAGS) $(EXTRA_CFLAGS) $(EXPERIMENTAL_CFLAGS) -o $@ -c $<
C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)")
endif
+EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/experimentalsyms.sh
+CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
+
PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ $@.pmd.c
PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) -c -o $@.pmd.o $@.pmd.c
PMDINFO_LD = $(CROSS)ld $(LDFLAGS) -r -o $@.o $@.pmd.o $@
@@ -100,6 +109,7 @@ C_TO_O_DO = @set -e; \
echo $(C_TO_O_DISP); \
$(C_TO_O) && \
$(PMDINFO_TO_O) && \
+ $(CHECK_EXPERIMENTAL) && \
echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
rm -f $(call obj2dep,$(@)).tmp
@@ -67,7 +67,7 @@ WERROR_FLAGS := -W -Wall -Wstrict-prototypes -Wmissing-prototypes
WERROR_FLAGS += -Wmissing-declarations -Wold-style-definition -Wpointer-arith
WERROR_FLAGS += -Wnested-externs -Wcast-qual
WERROR_FLAGS += -Wformat-nonliteral -Wformat-security
-WERROR_FLAGS += -Wundef -Wwrite-strings
+WERROR_FLAGS += -Wundef -Wwrite-strings -Wdeprecated
ifeq ($(RTE_DEVEL_BUILD),y)
WERROR_FLAGS += -Werror
@@ -75,7 +75,7 @@ WERROR_FLAGS := -W -Wall -Wstrict-prototypes -Wmissing-prototypes
WERROR_FLAGS += -Wmissing-declarations -Wold-style-definition -Wpointer-arith
WERROR_FLAGS += -Wcast-align -Wnested-externs -Wcast-qual
WERROR_FLAGS += -Wformat-nonliteral -Wformat-security
-WERROR_FLAGS += -Wundef -Wwrite-strings
+WERROR_FLAGS += -Wundef -Wwrite-strings -Wdeprecated
ifeq ($(RTE_DEVEL_BUILD),y)
WERROR_FLAGS += -Werror
@@ -73,7 +73,7 @@ TOOLCHAIN_ASFLAGS =
WERROR_FLAGS := -Wall -w2 -diag-disable 271 -diag-warning 1478
WERROR_FLAGS += -diag-disable 13368 -diag-disable 15527
WERROR_FLAGS += -diag-disable 188
-WERROR_FLAGS += -diag-disable 11074 -diag-disable 11076
+WERROR_FLAGS += -diag-disable 11074 -diag-disable 11076 -Wdeprecated
ifeq ($(RTE_DEVEL_BUILD),y)
WERROR_FLAGS += -Werror-all