[dpdk-dev,v3] ABI: Add abi checking utility

  There was a request for an abi validation utilty for the ongoing ABI stability
work.  As it turns out there is a abi compliance checker in development that
seems to be under active development and provides fairly detailed ABI compliance
reports.  Its not yet intellegent enough to understand symbol versioning, but it
does provide the ability to identify symbols which have changed between
releases, along with details of the change, and offers developers the
opportunity to identify which symbols then need versioning and validation for a
given update via manual testing.

This script automates the use of the compliance checker between two arbitrarily
specified tags within the dpdk tree.  To execute enter the $RTE_SDK directory
and run:

./scripts/validate_abi.sh $GIT_TAG1 $GIT_TAG2 $CONFIG

where $GIT_TAG1 and 2 are git tags and $CONFIG is a config specification
suitable for passing as the T= variable in the make config command.

Note the upstream source for the abi compliance checker is here:
http://ispras.linuxbase.org/index.php/ABI_compliance_checker

It generates a report for each DSO built from the requested tags that developers
can review to find ABI compliance issues.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>

---

Change Notes:

v2) Fixed some typos as requested by Thomas

v3) Fixed some additional typos Thomas requested
    Improved script to work from detached state
    Added some documentation to the changelog
    Added some comments to the scripts
---
 scripts/validate_abi.sh | 248 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 248 insertions(+)
 create mode 100755 scripts/validate_abi.sh
  

2015-03-04 11:26, Neil Horman:
> +#trap on ctrl-c to clean up
> +trap cleanup_and_exit SIGINT

I think INT is preffered over SIGINT.
You may also add QUIT and TERM.
With QUIT, you can replace cleanup_and_exit calls by a simple exit.

> +	CURRENT_BRANCH=`git log --pretty=format:%H HEAD~1..HEAD`

May be simpler "git log -1 --format=%H"

> +log "INFO" "We're going to check and make sure that applications built"
> +log "INFO" "against DPDK DSOs from tag $TAG1 will still run when executed"
> +log "INFO" "against DPDK DSOs built from tag $TAG2."

I think it may be removed as no app is run.

> +# Make sure we configure SHARED libraries
> +# Also turn off IGB and KNI as those require kernel headers to build
> +sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
> +sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
> +sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET

So you prefer modifying defconfig instead of .config, right?
(you sent it while I was answering on v2)

> +# Checking abi compliance relies on using the dwarf information in
> +# The shared objects.  Thats only included in the DSO's if we build
> +# with -g
> +export EXTRA_CFLAGS=-g
> +export EXTRA_LDFLAGS=-g
[...]
> +export EXTRA_CFLAGS=-g
> +export EXTRA_LDFLAGS=-g

Already exported.

> +	OLDNAME=`basename $i | sed -e"s/1.dump/0.dump/"`

Could be OLDNAME=$(basename $i 1.dump)0.dump

> +	LIBNAME=`basename $i | sed -e"s/-ABI-1.dump//"`

Could be LIBNAME=$(basename $i -ABI-1.dump)

Thanks
  

On Wed, Mar 04, 2015 at 05:49:50PM +0100, Thomas Monjalon wrote:
> 2015-03-04 11:26, Neil Horman:
> > +#trap on ctrl-c to clean up
> > +trap cleanup_and_exit SIGINT
> 
> I think INT is preffered over SIGINT.
> You may also add QUIT and TERM.
> With QUIT, you can replace cleanup_and_exit calls by a simple exit.
> 
> > +	CURRENT_BRANCH=`git log --pretty=format:%H HEAD~1..HEAD`
> 
> May be simpler "git log -1 --format=%H"
> 
It might be, but the above is equivalent, and --format is a more recent git-log
feature.  Older versions still require --pretty=format

> > +log "INFO" "We're going to check and make sure that applications built"
> > +log "INFO" "against DPDK DSOs from tag $TAG1 will still run when executed"
> > +log "INFO" "against DPDK DSOs built from tag $TAG2."
> 
> I think it may be removed as no app is run.
> 
The above doesn't indicate that an application will be run, only that the
purpose of this script is to ensure that older applications will still run,
which I think is appropriate.

> > +# Make sure we configure SHARED libraries
> > +# Also turn off IGB and KNI as those require kernel headers to build
> > +sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
> > +sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
> > +sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET
> 
> So you prefer modifying defconfig instead of .config, right?
> (you sent it while I was answering on v2)
> 
Yes, correct.

> > +# Checking abi compliance relies on using the dwarf information in
> > +# The shared objects.  Thats only included in the DSO's if we build
> > +# with -g
> > +export EXTRA_CFLAGS=-g
> > +export EXTRA_LDFLAGS=-g
> [...]
> > +export EXTRA_CFLAGS=-g
> > +export EXTRA_LDFLAGS=-g
> 
> Already exported.
> 
Yeah, I'll clean that up later.

> > +	OLDNAME=`basename $i | sed -e"s/1.dump/0.dump/"`
> 
> Could be OLDNAME=$(basename $i 1.dump)0.dump
> 
> > +	LIBNAME=`basename $i | sed -e"s/-ABI-1.dump//"`
> 
> Could be LIBNAME=$(basename $i -ABI-1.dump)
> 
It could be, but I prefer the clarity of the sed replacement.

Neil

> Thanks
> 
>
  

On Thu, Mar 05, 2015 at 11:57:27AM -0500, Neil Horman wrote:
> On Wed, Mar 04, 2015 at 05:49:50PM +0100, Thomas Monjalon wrote:
> > 2015-03-04 11:26, Neil Horman:
> > > +#trap on ctrl-c to clean up
> > > +trap cleanup_and_exit SIGINT
> > 
> > I think INT is preffered over SIGINT.
> > You may also add QUIT and TERM.
> > With QUIT, you can replace cleanup_and_exit calls by a simple exit.
> > 
> > > +	CURRENT_BRANCH=`git log --pretty=format:%H HEAD~1..HEAD`
> > 
> > May be simpler "git log -1 --format=%H"
> > 
> It might be, but the above is equivalent, and --format is a more recent git-log
> feature.  Older versions still require --pretty=format
> 
> > > +log "INFO" "We're going to check and make sure that applications built"
> > > +log "INFO" "against DPDK DSOs from tag $TAG1 will still run when executed"
> > > +log "INFO" "against DPDK DSOs built from tag $TAG2."
> > 
> > I think it may be removed as no app is run.
> > 
> The above doesn't indicate that an application will be run, only that the
> purpose of this script is to ensure that older applications will still run,
> which I think is appropriate.
> 
> > > +# Make sure we configure SHARED libraries
> > > +# Also turn off IGB and KNI as those require kernel headers to build
> > > +sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
> > > +sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
> > > +sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET
> > 
> > So you prefer modifying defconfig instead of .config, right?
> > (you sent it while I was answering on v2)
> > 
> Yes, correct.
> 
> > > +# Checking abi compliance relies on using the dwarf information in
> > > +# The shared objects.  Thats only included in the DSO's if we build
> > > +# with -g
> > > +export EXTRA_CFLAGS=-g
> > > +export EXTRA_LDFLAGS=-g
> > [...]
> > > +export EXTRA_CFLAGS=-g
> > > +export EXTRA_LDFLAGS=-g
> > 
> > Already exported.
> > 
> Yeah, I'll clean that up later.
> 
> > > +	OLDNAME=`basename $i | sed -e"s/1.dump/0.dump/"`
> > 
> > Could be OLDNAME=$(basename $i 1.dump)0.dump
> > 
> > > +	LIBNAME=`basename $i | sed -e"s/-ABI-1.dump//"`
> > 
> > Could be LIBNAME=$(basename $i -ABI-1.dump)
> > 
> It could be, but I prefer the clarity of the sed replacement.
> 
> Neil
> 
> > Thanks
> > 
> > 
> 

Ping Thomas, is this going to make 2.0?

Thanks
Neil
  

Hi Neil,

2015-03-11 15:36, Neil Horman:
> On Thu, Mar 05, 2015 at 11:57:27AM -0500, Neil Horman wrote:
> > On Wed, Mar 04, 2015 at 05:49:50PM +0100, Thomas Monjalon wrote:
> > > 2015-03-04 11:26, Neil Horman:
> > > > +#trap on ctrl-c to clean up
> > > > +trap cleanup_and_exit SIGINT
> > > 
> > > I think INT is preffered over SIGINT.
> > > You may also add QUIT and TERM.
> > > With QUIT, you can replace cleanup_and_exit calls by a simple exit.
> > > 
> > > > +	CURRENT_BRANCH=`git log --pretty=format:%H HEAD~1..HEAD`
> > > 
> > > May be simpler "git log -1 --format=%H"
> > > 
> > It might be, but the above is equivalent, and --format is a more recent git-log
> > feature.  Older versions still require --pretty=format
> > 
> > > > +log "INFO" "We're going to check and make sure that applications built"
> > > > +log "INFO" "against DPDK DSOs from tag $TAG1 will still run when executed"
> > > > +log "INFO" "against DPDK DSOs built from tag $TAG2."
> > > 
> > > I think it may be removed as no app is run.
> > > 
> > The above doesn't indicate that an application will be run, only that the
> > purpose of this script is to ensure that older applications will still run,
> > which I think is appropriate.
> > 
> > > > +# Make sure we configure SHARED libraries
> > > > +# Also turn off IGB and KNI as those require kernel headers to build
> > > > +sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
> > > > +sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
> > > > +sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET
> > > 
> > > So you prefer modifying defconfig instead of .config, right?
> > > (you sent it while I was answering on v2)
> > > 
> > Yes, correct.
> > 
> > > > +# Checking abi compliance relies on using the dwarf information in
> > > > +# The shared objects.  Thats only included in the DSO's if we build
> > > > +# with -g
> > > > +export EXTRA_CFLAGS=-g
> > > > +export EXTRA_LDFLAGS=-g
> > > [...]
> > > > +export EXTRA_CFLAGS=-g
> > > > +export EXTRA_LDFLAGS=-g
> > > 
> > > Already exported.
> > > 
> > Yeah, I'll clean that up later.

OK, could you send a v4 please?

> > > > +	OLDNAME=`basename $i | sed -e"s/1.dump/0.dump/"`
> > > 
> > > Could be OLDNAME=$(basename $i 1.dump)0.dump
> > > 
> > > > +	LIBNAME=`basename $i | sed -e"s/-ABI-1.dump//"`
> > > 
> > > Could be LIBNAME=$(basename $i -ABI-1.dump)
> > > 
> > It could be, but I prefer the clarity of the sed replacement.
> > 
> > Neil
> > 
> > > Thanks
> > > 
> > > 
> > 
> 
> Ping Thomas, is this going to make 2.0?

Yes sure, waiting a v4.
  

>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
>Sent: Wednesday, March 4, 2015 4:27 PM
>To: dev@dpdk.org
>Subject: [dpdk-dev] [PATCH v3] ABI: Add abi checking utility
>
>There was a request for an abi validation utilty for the ongoing ABI stability
>work.  As it turns out there is a abi compliance checker in development that
>seems to be under active development and provides fairly detailed ABI compliance
>reports.  Its not yet intellegent enough to understand symbol versioning, but it
>does provide the ability to identify symbols which have changed between
>releases, along with details of the change, and offers developers the
>opportunity to identify which symbols then need versioning and validation for a
>given update via manual testing.
>
>This script automates the use of the compliance checker between two arbitrarily
>specified tags within the dpdk tree.  To execute enter the $RTE_SDK directory
>and run:
>
>./scripts/validate_abi.sh $GIT_TAG1 $GIT_TAG2 $CONFIG
>
>where $GIT_TAG1 and 2 are git tags and $CONFIG is a config specification
>suitable for passing as the T= variable in the make config command.
>
>Note the upstream source for the abi compliance checker is here:
>http://ispras.linuxbase.org/index.php/ABI_compliance_checker
>
>It generates a report for each DSO built from the requested tags that developers
>can review to find ABI compliance issues.
>
>Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
>
>---
>
>Change Notes:
>
>v2) Fixed some typos as requested by Thomas
>
>v3) Fixed some additional typos Thomas requested
>    Improved script to work from detached state
>    Added some documentation to the changelog
>    Added some comments to the scripts
>---
> scripts/validate_abi.sh | 248 ++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 248 insertions(+)
> create mode 100755 scripts/validate_abi.sh
>
>diff --git a/scripts/validate_abi.sh b/scripts/validate_abi.sh
>new file mode 100755
>index 0000000..899cf5f
>--- /dev/null
>+++ b/scripts/validate_abi.sh
>@@ -0,0 +1,248 @@
>+#!/bin/sh
>+#   BSD LICENSE
>+#
>+#   Copyright(c) 2015 Neil Horman. All rights reserved.
>+#   All rights reserved.
>+#
>+#   Redistribution and use in source and binary forms, with or without
>+#   modification, are permitted provided that the following conditions
>+#   are met:
>+#
>+#     * Redistributions of source code must retain the above copyright
>+#       notice, this list of conditions and the following disclaimer.
>+#     * Redistributions in binary form must reproduce the above copyright
>+#       notice, this list of conditions and the following disclaimer in
>+#       the documentation and/or other materials provided with the
>+#       distribution.
>+#
>+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
>+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
>+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
>+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
>+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
>+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
>+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>+
>+TAG1=$1
>+TAG2=$2
>+TARGET=$3
>+ABI_DIR=`mktemp -d -p /tmp ABI.XXXXXX`
>+
>+usage() {
>+	echo "$0 <TAG1> <TAG2> <TARGET>"
>+}
>+
>+log() {
>+	local level=$1
>+	shift
>+	echo "$*"
>+}
>+
>+validate_tags() {
>+	git tag -l | grep -q "$TAG1"
>+	if [ $? -ne 0 ]
>+	then
>+		echo "$TAG1 is invalid"
>+		return
>+	fi
>+	git tag -l | grep -q "$TAG2"
>+	if [ $? -ne 0 ]
>+	then
>+		echo "$TAG2 is invalid"
>+		return
>+	fi
>+}
>+
>+validate_args() {
>+	if [ -z "$TAG1" ]
>+	then
>+		echo "Must Specify TAG1"
>+		return
>+	fi
>+	if [ -z "$TAG2" ]
>+	then
>+		echo "Must Specify TAG2"
>+		return
>+	fi
>+	if [ -z "$TARGET" ]
>+	then
>+		echo "Must Specify a build target"
>+	fi
>+}
>+
>+
>+cleanup_and_exit() {
>+	rm -rf $ABI_DIR
>+	exit $1
>+}
>+
>+###########################################
>+#START
>+############################################
>+
>+#trap on ctrl-c to clean up
>+trap cleanup_and_exit SIGINT
>+
>+#Save the current branch
>+CURRENT_BRANCH=`git branch | grep \* | cut -d' ' -f2`
>+
>+if [ -z "$CURRENT_BRANCH" ]
>+then
>+	CURRENT_BRANCH=`git log --pretty=format:%H HEAD~1..HEAD`
>+fi
>+
>+if [ -n "$VERBOSE" ]
>+then
>+	export VERBOSE=/dev/stdout
>+else
>+	export VERBOSE=/dev/null
>+fi
>+
>+# Validate that we have all the arguments we need
>+res=$(validate_args)
>+if [ -n "$res" ]
>+then
>+	echo $res
>+	usage
>+	cleanup_and_exit 1
>+fi
>+
>+# Make sure our tags exist
>+res=$(validate_tags)
>+if [ -n "$res" ]
>+then
>+	echo $res
>+	cleanup_and_exit 1
>+fi
>+
>+ABICHECK=`which abi-compliance-checker 2>/dev/null`
>+if [ $? -ne 0 ]
>+then
>+	log "INFO" "Cant find abi-compliance-checker utility"
>+	cleanup_and_exit 1
>+fi
>+
>+ABIDUMP=`which abi-dumper 2>/dev/null`
>+if [ $? -ne 0 ]
>+then
>+	log "INFO" "Cant find abi-dumper utility"
>+	cleanup_and_exit 1
>+fi
>+
>+log "INFO" "We're going to check and make sure that applications built"
>+log "INFO" "against DPDK DSOs from tag $TAG1 will still run when executed"
>+log "INFO" "against DPDK DSOs built from tag $TAG2."
>+log "INFO" ""
>+
>+# Check to make sure we have a clean tree
>+git status | grep -q clean
>+if [ $? -ne 0 ]
>+then
>+	log "WARN" "Working directory not clean, aborting"
>+	cleanup_and_exit 1
>+fi
>+
>+# Move to the root of the git tree
>+cd $(dirname $0)/..
>+
>+log "INFO" "Checking out version $TAG1 of the dpdk"
>+# Move to the old version of the tree
>+git checkout $TAG1
>+
>+# Make sure we configure SHARED libraries
>+# Also turn off IGB and KNI as those require kernel headers to build
>+sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
>+sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
>+sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET
>+
>+# Checking abi compliance relies on using the dwarf information in
>+# The shared objects.  Thats only included in the DSO's if we build
>+# with -g
>+export EXTRA_CFLAGS=-g
>+export EXTRA_LDFLAGS=-g
>+
>+# Now configure the build
>+log "INFO" "Configuring DPDK $TAG1"
>+make config T=$TARGET O=$TARGET > $VERBOSE 2>&1
>+
>+log "INFO" "Building DPDK $TAG1. This might take a moment"
>+make O=$TARGET > $VERBOSE 2>&1
>+
>+if [ $? -ne 0 ]
>+then
>+	log "INFO" "THE BUILD FAILED.  ABORTING"

If the build fails while TAG1 is checked out, the user must check out their original local branch manually. I'd prefer it if the script checked out $CURRENT_BRANCH in the 'cleanup_and_exit' function. 

Same applies to TAG2, if the user CTRL-C's out of the script, and to any other command that might fail when a particular branch/tag is checked out (for example, the 'sed' commands fail when I run the script; however, they work when I run them on the command line - I'm investigating this currently).

>+	cleanup_and_exit 1
>+fi
>+
>+# Move to the lib directory
>+cd $TARGET/lib
>+log "INFO" "COLLECTING ABI INFORMATION FOR $TAG1"
>+for i in `ls *.so`
>+do
>+	$ABIDUMP $i -o $ABI_DIR/$i-ABI-0.dump -lver $TAG1
>+done
>+cd ../..
>+
>+# Now clean the tree, checkout the second tag, and rebuild
>+git clean -f -d
>+git reset --hard
>+# Move to the new version of the tree
>+log "INFO" "Checking out version $TAG2 of the dpdk"
>+git checkout $TAG2
>+
>+export EXTRA_CFLAGS=-g
>+export EXTRA_LDFLAGS=-g
>+
>+# Make sure we configure SHARED libraries
>+# Also turn off IGB and KNI as those require kernel headers to build
>+sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
>+sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
>+sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET
>+
>+# Now configure the build
>+log "INFO" "Configuring DPDK $TAG2"
>+make config T=$TARGET O=$TARGET > $VERBOSE 2>&1
>+
>+log "INFO" "Building DPDK $TAG2. This might take a moment"
>+make O=$TARGET > $VERBOSE 2>&1
>+
>+if [ $? -ne 0 ]
>+then
>+	log "INFO" "THE BUILD FAILED.  ABORTING"
>+	cleanup_and_exit 1
>+fi
>+
>+cd $TARGET/lib
>+log "INFO" "COLLECTING ABI INFORMATION FOR $TAG2"
>+for i in `ls *.so`
>+do
>+	$ABIDUMP $i -o $ABI_DIR/$i-ABI-1.dump -lver $TAG2
>+done
>+cd ../..
>+
>+# Start comparison of ABI dumps
>+for i in `ls $ABI_DIR/*-1.dump`
>+do
>+	NEWNAME=`basename $i`
>+	OLDNAME=`basename $i | sed -e"s/1.dump/0.dump/"`
>+	LIBNAME=`basename $i | sed -e"s/-ABI-1.dump//"`
>+
>+	if [ ! -f $ABI_DIR/$OLDNAME ]
>+	then
>+		log "INFO" "$OLDNAME DOES NOT EXIST IN $TAG1. SKIPPING..."
>+	fi
>+
>+	#compare the abi dumps
>+	$ABICHECK -l $LIBNAME -old $ABI_DIR/$OLDNAME -new $ABI_DIR/$NEWNAME
>+done
>+
>+git reset --hard
>+git checkout $CURRENT_BRANCH
>+log "INFO" "ABI CHECK COMPLETE.  REPORTS ARE IN compat_report directory"
>+cleanup_and_exit 0
>+
>+
>--
>2.1.0
  

On Fri, Mar 13, 2015 at 11:56:59AM +0000, Kavanagh, Mark B wrote:
> 
> 
> >-----Original Message-----
> >From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> >Sent: Wednesday, March 4, 2015 4:27 PM
> >To: dev@dpdk.org
> >Subject: [dpdk-dev] [PATCH v3] ABI: Add abi checking utility
> >
> >There was a request for an abi validation utilty for the ongoing ABI stability
> >work.  As it turns out there is a abi compliance checker in development that
> >seems to be under active development and provides fairly detailed ABI compliance
> >reports.  Its not yet intellegent enough to understand symbol versioning, but it
> >does provide the ability to identify symbols which have changed between
> >releases, along with details of the change, and offers developers the
> >opportunity to identify which symbols then need versioning and validation for a
> >given update via manual testing.
> >
> >This script automates the use of the compliance checker between two arbitrarily
> >specified tags within the dpdk tree.  To execute enter the $RTE_SDK directory
> >and run:
> >
> >./scripts/validate_abi.sh $GIT_TAG1 $GIT_TAG2 $CONFIG
> >
> >where $GIT_TAG1 and 2 are git tags and $CONFIG is a config specification
> >suitable for passing as the T= variable in the make config command.
> >
> >Note the upstream source for the abi compliance checker is here:
> >http://ispras.linuxbase.org/index.php/ABI_compliance_checker
> >
> >It generates a report for each DSO built from the requested tags that developers
> >can review to find ABI compliance issues.
> >
> >Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> >
> >---
> >
> >Change Notes:
> >
> >v2) Fixed some typos as requested by Thomas
> >
> >v3) Fixed some additional typos Thomas requested
> >    Improved script to work from detached state
> >    Added some documentation to the changelog
> >    Added some comments to the scripts
> >---
> > scripts/validate_abi.sh | 248 ++++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 248 insertions(+)
> > create mode 100755 scripts/validate_abi.sh
> >
> >diff --git a/scripts/validate_abi.sh b/scripts/validate_abi.sh
> >new file mode 100755
> >index 0000000..899cf5f
> >--- /dev/null
> >+++ b/scripts/validate_abi.sh
> >@@ -0,0 +1,248 @@
> >+#!/bin/sh
> >+#   BSD LICENSE
> >+#
> >+#   Copyright(c) 2015 Neil Horman. All rights reserved.
> >+#   All rights reserved.
> >+#
> >+#   Redistribution and use in source and binary forms, with or without
> >+#   modification, are permitted provided that the following conditions
> >+#   are met:
> >+#
> >+#     * Redistributions of source code must retain the above copyright
> >+#       notice, this list of conditions and the following disclaimer.
> >+#     * Redistributions in binary form must reproduce the above copyright
> >+#       notice, this list of conditions and the following disclaimer in
> >+#       the documentation and/or other materials provided with the
> >+#       distribution.
> >+#
> >+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> >+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> >+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> >+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> >+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> >+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> >+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> >+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> >+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> >+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> >+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> >+
> >+TAG1=$1
> >+TAG2=$2
> >+TARGET=$3
> >+ABI_DIR=`mktemp -d -p /tmp ABI.XXXXXX`
> >+
> >+usage() {
> >+	echo "$0 <TAG1> <TAG2> <TARGET>"
> >+}
> >+
> >+log() {
> >+	local level=$1
> >+	shift
> >+	echo "$*"
> >+}
> >+
> >+validate_tags() {
> >+	git tag -l | grep -q "$TAG1"
> >+	if [ $? -ne 0 ]
> >+	then
> >+		echo "$TAG1 is invalid"
> >+		return
> >+	fi
> >+	git tag -l | grep -q "$TAG2"
> >+	if [ $? -ne 0 ]
> >+	then
> >+		echo "$TAG2 is invalid"
> >+		return
> >+	fi
> >+}
> >+
> >+validate_args() {
> >+	if [ -z "$TAG1" ]
> >+	then
> >+		echo "Must Specify TAG1"
> >+		return
> >+	fi
> >+	if [ -z "$TAG2" ]
> >+	then
> >+		echo "Must Specify TAG2"
> >+		return
> >+	fi
> >+	if [ -z "$TARGET" ]
> >+	then
> >+		echo "Must Specify a build target"
> >+	fi
> >+}
> >+
> >+
> >+cleanup_and_exit() {
> >+	rm -rf $ABI_DIR
> >+	exit $1
> >+}
> >+
> >+###########################################
> >+#START
> >+############################################
> >+
> >+#trap on ctrl-c to clean up
> >+trap cleanup_and_exit SIGINT
> >+
> >+#Save the current branch
> >+CURRENT_BRANCH=`git branch | grep \* | cut -d' ' -f2`
> >+
> >+if [ -z "$CURRENT_BRANCH" ]
> >+then
> >+	CURRENT_BRANCH=`git log --pretty=format:%H HEAD~1..HEAD`
> >+fi
> >+
> >+if [ -n "$VERBOSE" ]
> >+then
> >+	export VERBOSE=/dev/stdout
> >+else
> >+	export VERBOSE=/dev/null
> >+fi
> >+
> >+# Validate that we have all the arguments we need
> >+res=$(validate_args)
> >+if [ -n "$res" ]
> >+then
> >+	echo $res
> >+	usage
> >+	cleanup_and_exit 1
> >+fi
> >+
> >+# Make sure our tags exist
> >+res=$(validate_tags)
> >+if [ -n "$res" ]
> >+then
> >+	echo $res
> >+	cleanup_and_exit 1
> >+fi
> >+
> >+ABICHECK=`which abi-compliance-checker 2>/dev/null`
> >+if [ $? -ne 0 ]
> >+then
> >+	log "INFO" "Cant find abi-compliance-checker utility"
> >+	cleanup_and_exit 1
> >+fi
> >+
> >+ABIDUMP=`which abi-dumper 2>/dev/null`
> >+if [ $? -ne 0 ]
> >+then
> >+	log "INFO" "Cant find abi-dumper utility"
> >+	cleanup_and_exit 1
> >+fi
> >+
> >+log "INFO" "We're going to check and make sure that applications built"
> >+log "INFO" "against DPDK DSOs from tag $TAG1 will still run when executed"
> >+log "INFO" "against DPDK DSOs built from tag $TAG2."
> >+log "INFO" ""
> >+
> >+# Check to make sure we have a clean tree
> >+git status | grep -q clean
> >+if [ $? -ne 0 ]
> >+then
> >+	log "WARN" "Working directory not clean, aborting"
> >+	cleanup_and_exit 1
> >+fi
> >+
> >+# Move to the root of the git tree
> >+cd $(dirname $0)/..
> >+
> >+log "INFO" "Checking out version $TAG1 of the dpdk"
> >+# Move to the old version of the tree
> >+git checkout $TAG1
> >+
> >+# Make sure we configure SHARED libraries
> >+# Also turn off IGB and KNI as those require kernel headers to build
> >+sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
> >+sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
> >+sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET
> >+
> >+# Checking abi compliance relies on using the dwarf information in
> >+# The shared objects.  Thats only included in the DSO's if we build
> >+# with -g
> >+export EXTRA_CFLAGS=-g
> >+export EXTRA_LDFLAGS=-g
> >+
> >+# Now configure the build
> >+log "INFO" "Configuring DPDK $TAG1"
> >+make config T=$TARGET O=$TARGET > $VERBOSE 2>&1
> >+
> >+log "INFO" "Building DPDK $TAG1. This might take a moment"
> >+make O=$TARGET > $VERBOSE 2>&1
> >+
> >+if [ $? -ne 0 ]
> >+then
> >+	log "INFO" "THE BUILD FAILED.  ABORTING"
> 
> If the build fails while TAG1 is checked out, the user must check out their original local branch manually. I'd prefer it if the script checked out $CURRENT_BRANCH in the 'cleanup_and_exit' function. 
> 
Sure, its in V4.

> Same applies to TAG2, if the user CTRL-C's out of the script, and to any other command that might fail when a particular branch/tag is checked out (for example, the 'sed' commands fail when I run the script; however, they work when I run them on the command line - I'm investigating this currently).
> 
What does the log say?  Please post it here.  If it helps add a set -x to the
top of the script for additional verbosity.

Neil
  

>On Fri, Mar 13, 2015 at 11:56:59AM +0000, Kavanagh, Mark B wrote:
>>
>>
>> >-----Original Message-----
>> >From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
>> >Sent: Wednesday, March 4, 2015 4:27 PM
>> >To: dev@dpdk.org
>> >Subject: [dpdk-dev] [PATCH v3] ABI: Add abi checking utility
>> >


(snip)

>> >+log "INFO" "Building DPDK $TAG1. This might take a moment"
>> >+make O=$TARGET > $VERBOSE 2>&1
>> >+
>> >+if [ $? -ne 0 ]
>> >+then
>> >+	log "INFO" "THE BUILD FAILED.  ABORTING"
>>
>> If the build fails while TAG1 is checked out, the user must check out their original
>local branch manually. I'd prefer it if the script checked out $CURRENT_BRANCH in the
>'cleanup_and_exit' function.
>>
>Sure, its in V4.

Cool.

>
>> Same applies to TAG2, if the user CTRL-C's out of the script, and to any other command
>that might fail when a particular branch/tag is checked out (for example, the 'sed'
>commands fail when I run the script; however, they work when I run them on the command
>line - I'm investigating this currently).
>>
>What does the log say?  Please post it here.  If it helps add a set -x to the
>top of the script for additional verbosity.
>

Hey Neil - this is the error, but it's not a problem with the script; presumably I'd cleaned my DPDK installation directory, so 'sed' couldn't find the defconfig file:
"sed: can't read config/defconfig_x86_64-ivshmem-linuxapp-gcc/: Not a directory"

Thanks,
Mark

>Neil
  

On Fri, Mar 13, 2015 at 02:25:17PM +0000, Kavanagh, Mark B wrote:
> >On Fri, Mar 13, 2015 at 11:56:59AM +0000, Kavanagh, Mark B wrote:
> >>
> >>
> >> >-----Original Message-----
> >> >From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> >> >Sent: Wednesday, March 4, 2015 4:27 PM
> >> >To: dev@dpdk.org
> >> >Subject: [dpdk-dev] [PATCH v3] ABI: Add abi checking utility
> >> >
> 
> 
> (snip)
> 
> >> >+log "INFO" "Building DPDK $TAG1. This might take a moment"
> >> >+make O=$TARGET > $VERBOSE 2>&1
> >> >+
> >> >+if [ $? -ne 0 ]
> >> >+then
> >> >+	log "INFO" "THE BUILD FAILED.  ABORTING"
> >>
> >> If the build fails while TAG1 is checked out, the user must check out their original
> >local branch manually. I'd prefer it if the script checked out $CURRENT_BRANCH in the
> >'cleanup_and_exit' function.
> >>
> >Sure, its in V4.
> 
> Cool.
> 
> >
> >> Same applies to TAG2, if the user CTRL-C's out of the script, and to any other command
> >that might fail when a particular branch/tag is checked out (for example, the 'sed'
> >commands fail when I run the script; however, they work when I run them on the command
> >line - I'm investigating this currently).
> >>
> >What does the log say?  Please post it here.  If it helps add a set -x to the
> >top of the script for additional verbosity.
> >
> 
> Hey Neil - this is the error, but it's not a problem with the script; presumably I'd cleaned my DPDK installation directory, so 'sed' couldn't find the defconfig file:
> "sed: can't read config/defconfig_x86_64-ivshmem-linuxapp-gcc/: Not a directory"
> 
Actually, it looks to me like you added a trailing "/" to the end of the third
argument on the script command line, so sed bombs when it tries to modify a
directory instead of a file.  Try specifying:
x86_64-ivshmem-linuxapp-gcc
instead of
x86_64-ivshmem-linuxapp-gcc/

Neil

> Thanks,
> Mark
> 
> >Neil
> 
>
  

>-----Original Message-----
>From: Neil Horman [mailto:nhorman@tuxdriver.com]
>Sent: Friday, March 13, 2015 2:59 PM
>To: Kavanagh, Mark B
>Cc: dev@dpdk.org
>Subject: Re: [dpdk-dev] [PATCH v3] ABI: Add abi checking utility
>
>On Fri, Mar 13, 2015 at 02:25:17PM +0000, Kavanagh, Mark B wrote:
>> >On Fri, Mar 13, 2015 at 11:56:59AM +0000, Kavanagh, Mark B wrote:
>> >>
>> >>
>> >> >-----Original Message-----
>> >> >From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
>> >> >Sent: Wednesday, March 4, 2015 4:27 PM
>> >> >To: dev@dpdk.org
>> >> >Subject: [dpdk-dev] [PATCH v3] ABI: Add abi checking utility
>> >> >
>>
>>
>> (snip)
>>
>> >> >+log "INFO" "Building DPDK $TAG1. This might take a moment"
>> >> >+make O=$TARGET > $VERBOSE 2>&1
>> >> >+
>> >> >+if [ $? -ne 0 ]
>> >> >+then
>> >> >+	log "INFO" "THE BUILD FAILED.  ABORTING"
>> >>
>> >> If the build fails while TAG1 is checked out, the user must check out their original
>> >local branch manually. I'd prefer it if the script checked out $CURRENT_BRANCH in the
>> >'cleanup_and_exit' function.
>> >>
>> >Sure, its in V4.
>>
>> Cool.
>>
>> >
>> >> Same applies to TAG2, if the user CTRL-C's out of the script, and to any other
>command
>> >that might fail when a particular branch/tag is checked out (for example, the 'sed'
>> >commands fail when I run the script; however, they work when I run them on the command
>> >line - I'm investigating this currently).
>> >>
>> >What does the log say?  Please post it here.  If it helps add a set -x to the
>> >top of the script for additional verbosity.
>> >
>>
>> Hey Neil - this is the error, but it's not a problem with the script; presumably I'd
>cleaned my DPDK installation directory, so 'sed' couldn't find the defconfig file:
>> "sed: can't read config/defconfig_x86_64-ivshmem-linuxapp-gcc/: Not a directory"
>>
>Actually, it looks to me like you added a trailing "/" to the end of the third
>argument on the script command line, so sed bombs when it tries to modify a
>directory instead of a file.  Try specifying:
>x86_64-ivshmem-linuxapp-gcc
>instead of
>x86_64-ivshmem-linuxapp-gcc/
>
>Neil

Nice catch - thanks!

>
>> Thanks,
>> Mark
>>
>> >Neil
>>
>>