[dpdk-dev,v2] mk: fix compile error and ABI versioning for combined shared library

  Fixes following error (observed when versioning macros used):
  LD libdpdk.so
  /usr/bin/ld: /root/dpdk/build/lib/libdpdk.so: version node not found
  for symbol <function>@DPDK_x.y

Also resulting combined library contains symbol version information:
$ readelf -a build/lib/libdpdk.so | grep rte_eal_ | grep @ | head
   <...>    GLOBAL DEFAULT   12 rte_eal_alarm_set@@DPDK_2.0
   <...>    GLOBAL DEFAULT   12 rte_eal_pci_write_config@@DPDK_2.1
   <...>    GLOBAL DEFAULT   12 rte_eal_remote_launch@@DPDK_2.0
...

Versioning fixed by merging all version scripts into one automatically and
feeding it to final library.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 drivers/net/Makefile  |  3 +++
 lib/Makefile          |  3 +++
 mk/rte.sdkbuild.mk    |  2 +-
 mk/rte.sharelib.mk    |  3 +++
 scripts/merge_maps.sh | 29 +++++++++++++++++++++++++++++
 5 files changed, 39 insertions(+), 1 deletion(-)
 create mode 100755 scripts/merge_maps.sh
  

Hi Ferruh,

Thanks for working on it.

2015-12-03 01:22, Ferruh Yigit:
> +ifeq ($(COMBINED_BUILD),1)
>  include $(RTE_SDK)/mk/rte.sharelib.mk
> +endif
[...]
>  	@if [ $@ = drivers -a $(CONFIG_RTE_BUILD_COMBINE_LIBS) = y ]; then \
> -		$(MAKE) -f $(RTE_SDK)/lib/Makefile sharelib; \
> +		COMBINED_BUILD=1 $(MAKE) -f $(RTE_SDK)/lib/Makefile sharelib; \

What is it fixing?
The badly named sharelib is for combined build only.

[...]
> +FILES=$(find $RTE_SDK -name "*.map" | grep -v build)

The build dir is not always "build/"
  

Hi Ferruh,
some minor bash improvements that could be made in the next revision:

On Thu, Dec 3, 2015 at 2:22 AM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> diff --git a/scripts/merge_maps.sh b/scripts/merge_maps.sh
> new file mode 100755
> index 0000000..bc40dc8
> --- /dev/null
> +++ b/scripts/merge_maps.sh
> @@ -0,0 +1,29 @@
> +#!/bin/sh
> +
> +FILES=$(find $RTE_SDK -name "*.map" | grep -v build)
> +SYMBOLS=$(grep -h "{" $FILES | sort -u | sed 's/{//')

Guarding $RTE_SDK and $FILES with "" will help avoid some potential
issues due to words splitting.

[...]
> +               sed -n "/$s {/,/}/p" $f | sed '/^$/d' | grep -v global | grep -v local | sed '1d' | sed '$d'

As above with $f

[...]
> +       if [ "$prev_sym" == "none" ]; then

Should be only one = as == is non standard and could fail in some environments.


Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd
  

On 12/03/2015 03:22 AM, Ferruh Yigit wrote:
> Fixes following error (observed when versioning macros used):
>    LD libdpdk.so
>    /usr/bin/ld: /root/dpdk/build/lib/libdpdk.so: version node not found
>    for symbol <function>@DPDK_x.y
>
> Also resulting combined library contains symbol version information:
> $ readelf -a build/lib/libdpdk.so | grep rte_eal_ | grep @ | head
>     <...>    GLOBAL DEFAULT   12 rte_eal_alarm_set@@DPDK_2.0
>     <...>    GLOBAL DEFAULT   12 rte_eal_pci_write_config@@DPDK_2.1
>     <...>    GLOBAL DEFAULT   12 rte_eal_remote_launch@@DPDK_2.0
> ...
>
> Versioning fixed by merging all version scripts into one automatically and
> feeding it to final library.
>
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> ---
>   drivers/net/Makefile  |  3 +++
>   lib/Makefile          |  3 +++
>   mk/rte.sdkbuild.mk    |  2 +-
>   mk/rte.sharelib.mk    |  3 +++
>   scripts/merge_maps.sh | 29 +++++++++++++++++++++++++++++
>   5 files changed, 39 insertions(+), 1 deletion(-)
>   create mode 100755 scripts/merge_maps.sh
>
> diff --git a/drivers/net/Makefile b/drivers/net/Makefile
> index cddcd57..d3c865b 100644
> --- a/drivers/net/Makefile
> +++ b/drivers/net/Makefile
> @@ -51,5 +51,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio
>   DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3
>   DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt
>
> +ifeq ($(COMBINED_BUILD),1)
>   include $(RTE_SDK)/mk/rte.sharelib.mk
> +endif
> +
>   include $(RTE_SDK)/mk/rte.subdir.mk
> diff --git a/lib/Makefile b/lib/Makefile
> index ef172ea..d0f7fb8 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -64,5 +64,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
>   DIRS-$(CONFIG_RTE_LIBRTE_IVSHMEM) += librte_ivshmem
>   endif
>
> +ifeq ($(COMBINED_BUILD),1)
>   include $(RTE_SDK)/mk/rte.sharelib.mk
> +endif
> +
>   include $(RTE_SDK)/mk/rte.subdir.mk
> diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
> index 38ec7bd..d4e3abf 100644
> --- a/mk/rte.sdkbuild.mk
> +++ b/mk/rte.sdkbuild.mk
> @@ -94,7 +94,7 @@ $(ROOTDIRS-y):
>   	@echo "== Build $@"
>   	$(Q)$(MAKE) S=$@ -f $(RTE_SRCDIR)/$@/Makefile -C $(BUILDDIR)/$@ all
>   	@if [ $@ = drivers -a $(CONFIG_RTE_BUILD_COMBINE_LIBS) = y ]; then \
> -		$(MAKE) -f $(RTE_SDK)/lib/Makefile sharelib; \
> +		COMBINED_BUILD=1 $(MAKE) -f $(RTE_SDK)/lib/Makefile sharelib; \
>   	fi
>
>   %_clean:
> diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
> index 7bb7219..76ead09 100644
> --- a/mk/rte.sharelib.mk
> +++ b/mk/rte.sharelib.mk
> @@ -40,6 +40,8 @@ LIB_ONE := lib$(RTE_LIBNAME).so
>   else
>   LIB_ONE := lib$(RTE_LIBNAME).a
>   endif
> +COMBINED_MAP=$(BUILDDIR)/lib/libdpdk.map
> +CPU_LDFLAGS += --version-script=$(COMBINED_MAP)
>   endif
>
>   .PHONY:sharelib
> @@ -79,6 +81,7 @@ ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
>   ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
>   $(LIB_ONE): FORCE
>   	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
> +	@$(SRCDIR)/scripts/merge_maps.sh > $(COMBINED_MAP)
>   	$(O_TO_S_DO)
>   else
>   $(LIB_ONE): FORCE
> diff --git a/scripts/merge_maps.sh b/scripts/merge_maps.sh
> new file mode 100755
> index 0000000..bc40dc8
> --- /dev/null
> +++ b/scripts/merge_maps.sh
> @@ -0,0 +1,29 @@
> +#!/bin/sh
> +
> +FILES=$(find $RTE_SDK -name "*.map" | grep -v build)
> +SYMBOLS=$(grep -h "{" $FILES | sort -u | sed 's/{//')
> +
> +first=0
> +prev_sym="none"
> +
> +for s in $SYMBOLS; do
> +	echo "$s {"
> +	echo "    global:"
> +	echo ""
> +	for f in $FILES; do
> +		sed -n "/$s {/,/}/p" $f | sed '/^$/d' | grep -v global | grep -v local | sed '1d' | sed '$d'
> +	done | sort -u
> +	echo ""
> +	if [ $first -eq 0 ]; then
> +		first=1;
> +		echo "    local: *;";
> +	fi
> +	if [ "$prev_sym" == "none" ]; then
> +		echo "};";
> +		prev_sym=$s;
> +	else
> +		echo "} $prev_sym;";
> +		prev_sym=$s;
> +	fi
> +	echo ""
> +done
>

I'd still rather see the combined library replaced with a linker script, 
but as long as it is there then +1 for this: with symbol versioning in 
place, applications linked to it more likely refuse to start than 
randomly crash when ABI changes, internal symbols are hidden etc. And 
doesn't require manual updating of two maps since its all scripted.

	- Panu -