From patchwork Wed Jan 29 17:26:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 65347 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E803A052F; Wed, 29 Jan 2020 18:26:45 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 903FC1C030; Wed, 29 Jan 2020 18:26:43 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 62EAD1BFFC for ; Wed, 29 Jan 2020 18:26:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580318801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+QX0YRH7qp6dd1qyUl32BtThQClgNHDxYPfn7+t1rHA=; b=JbONTY409I41mQOd415OWRgAVWrwywIq3/xpZ0etRCk+h7jcwxgYRp8nZjeDY3I5JIoHHa 6U3NXFH1ugKl6N2RltaVPFK62jxs+3TxHjLSYcf92QFSjI/p51Ru2SD6VVPjVqwGYVVhRT U2eNn5PRDm604VfzmqwgfqOejEqF9i0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-161-4Aqy3pEIO-S4XXTNsFn9aA-1; Wed, 29 Jan 2020 12:26:37 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 15D6297314; Wed, 29 Jan 2020 17:26:35 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-128.brq.redhat.com [10.40.204.128]) by smtp.corp.redhat.com (Postfix) with ESMTP id A9BF488821; Wed, 29 Jan 2020 17:26:31 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, kevin.laatz@intel.com, aconole@redhat.com, nhorman@tuxdriver.com, akhil.goyal@nxp.com, anoobj@marvell.com, stable@dpdk.org, Yipeng Wang , Sameh Gobriel , Harry van Haaren , Luca Boccassi , Keith Wiles Date: Wed, 29 Jan 2020 18:26:18 +0100 Message-Id: <20200129172621.28565-2-david.marchand@redhat.com> In-Reply-To: <20200129172621.28565-1-david.marchand@redhat.com> References: <20191220152058.10739-1-david.marchand@redhat.com> <20200129172621.28565-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: 4Aqy3pEIO-S4XXTNsFn9aA-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 1/4] hash: fix meson headers packaging X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Those headers are internal and should not be distributed. Fixes: 5b9656b157d3 ("lib: build with meson") Cc: stable@dpdk.org Signed-off-by: David Marchand --- lib/librte_hash/meson.build | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/lib/librte_hash/meson.build b/lib/librte_hash/meson.build index 5d02b3084..bce11ad9e 100644 --- a/lib/librte_hash/meson.build +++ b/lib/librte_hash/meson.build @@ -1,10 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -headers = files('rte_cmp_arm64.h', - 'rte_cmp_x86.h', - 'rte_crc_arm64.h', - 'rte_cuckoo_hash.h', +headers = files('rte_crc_arm64.h', 'rte_fbk_hash.h', 'rte_hash_crc.h', 'rte_hash.h', From patchwork Wed Jan 29 17:26:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 65348 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4052EA052F; Wed, 29 Jan 2020 18:26:53 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 17CDE1C037; Wed, 29 Jan 2020 18:26:45 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id 7ECEA1C029 for ; Wed, 29 Jan 2020 18:26:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580318803; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WyEJQhUZdn4DLYjGGZtqw0jZGmkh2u6q2Ib0mDa8Ft4=; b=Ga2iGqb1EM8Wh310+dwNc8SItx3NatgPOiPZs8j2r6nAm8ABkqDM2G6PG70TZCBtcNe9Q0 rIYhkHecnSgev9KvObTGt3xgWuPAhLGGNZvwTYZm+lHLsrb/Sug2wwgnWQaJTbiu/eAurZ D183rcdT3GLRsKFQ9L6G3DpTyuLtR3M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-160-YxvRr1SCPdqdcjbv6JZGOw-1; Wed, 29 Jan 2020 12:26:39 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BF7F98C0CA1; Wed, 29 Jan 2020 17:26:37 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-128.brq.redhat.com [10.40.204.128]) by smtp.corp.redhat.com (Postfix) with ESMTP id A906384BC5; Wed, 29 Jan 2020 17:26:35 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, kevin.laatz@intel.com, aconole@redhat.com, nhorman@tuxdriver.com, akhil.goyal@nxp.com, anoobj@marvell.com Date: Wed, 29 Jan 2020 18:26:19 +0100 Message-Id: <20200129172621.28565-3-david.marchand@redhat.com> In-Reply-To: <20200129172621.28565-1-david.marchand@redhat.com> References: <20191220152058.10739-1-david.marchand@redhat.com> <20200129172621.28565-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: YxvRr1SCPdqdcjbv6JZGOw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 2/4] build: split build helper X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" No functional change intended, prepare for reusing this code. The config and compilation parts are separated in helpers. Unsetting CC is moved to the caller of the helper. Signed-off-by: David Marchand --- devtools/test-meson-builds.sh | 46 ++++++++++++++++++++++++----------- 1 file changed, 32 insertions(+), 14 deletions(-) diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index 688567714..aed175889 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson-builds.sh @@ -57,25 +57,27 @@ load_env () # . $srcdir/devtools/load-devel-config } -build () # +config () # { - builddir=$builds_dir/$1 + dir=$1 shift - targetcc=$1 + builddir=$1 shift - # skip build if compiler not available - command -v ${CC##* } >/dev/null 2>&1 || return 0 - load_env $targetcc || return 0 + options="--werror -Dexamples=all" + for option in $DPDK_MESON_OPTIONS ; do + options="$options -D$option" + done + options="$options $*" if [ ! -f "$builddir/build.ninja" ] ; then - options="--werror -Dexamples=all" - for option in $DPDK_MESON_OPTIONS ; do - options="$options -D$option" - done - options="$options $*" - echo "$MESON $options $srcdir $builddir" - $MESON $options $srcdir $builddir - unset CC + echo "$MESON $options $dir $builddir" + $MESON $options $dir $builddir fi +} + +compile () # +{ + builddir=$1 + shift if [ -n "$TEST_MESON_BUILD_VERY_VERBOSE" ] ; then # for full output from ninja use "-v" echo "$ninja_cmd -v -C $builddir" @@ -90,6 +92,19 @@ build () # fi } +build () # +{ + targetdir=$1 + shift + targetcc=$1 + shift + # skip build if compiler not available + command -v ${CC##* } >/dev/null 2>&1 || return 0 + load_env $targetcc || return 0 + config $srcdir $builds_dir/$targetdir $* + compile $builds_dir/$targetdir +} + if [ "$1" = "-vv" ] ; then TEST_MESON_BUILD_VERY_VERBOSE=1 elif [ "$1" = "-v" ] ; then @@ -107,6 +122,7 @@ for c in gcc clang ; do for s in static shared ; do export CC="$CCACHE $c" build build-$c-$s $c --default-library=$s + unset CC done done @@ -125,11 +141,13 @@ c=aarch64-linux-gnu-gcc export CC="clang" build build-arm64-host-clang $c $use_shared \ --cross-file $srcdir/config/arm/arm64_armv8_linux_gcc +unset CC # all gcc/arm configurations for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do export CC="$CCACHE gcc" build build-$(basename $f | tr '_' '-' | cut -d'-' -f-2) $c \ $use_shared --cross-file $f + unset CC done # Test installation of the x86-default target, to be used for checking From patchwork Wed Jan 29 17:26:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 65350 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 17BDAA052F; Wed, 29 Jan 2020 18:27:15 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 723CD1C06D; Wed, 29 Jan 2020 18:27:03 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id 6A1091C024 for ; Wed, 29 Jan 2020 18:27:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580318821; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fFO/5NxOR7o0aOc+KSThB10kzMAWUpXhuSuyzfp0fVo=; b=eHr3ZkFLWdjvJTYgbrGx59CIGm4D0qLD24wpBy/ZtijwhJ5nZIEf9h8u28qJvsJg2WeX0i +Ot7yDLgXw7YKc1j6K3PvBAf23vdEYHjY7dZw+DFYE2HQDHrSLf/cSx+nHaNOzXL9l/KaE WqmoiFeXTMlN5XHQwuaVwskqlxS8hko= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-201-mXP5cI_CM8CsurDrVi2KMg-1; Wed, 29 Jan 2020 12:26:42 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2CDF4804AAE; Wed, 29 Jan 2020 17:26:41 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-128.brq.redhat.com [10.40.204.128]) by smtp.corp.redhat.com (Postfix) with ESMTP id 46F9F84BC9; Wed, 29 Jan 2020 17:26:38 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, kevin.laatz@intel.com, aconole@redhat.com, nhorman@tuxdriver.com, akhil.goyal@nxp.com, anoobj@marvell.com, Michael Santana , Ferruh Yigit , Andrew Rybchenko Date: Wed, 29 Jan 2020 18:26:20 +0100 Message-Id: <20200129172621.28565-4-david.marchand@redhat.com> In-Reply-To: <20200129172621.28565-1-david.marchand@redhat.com> References: <20191220152058.10739-1-david.marchand@redhat.com> <20200129172621.28565-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: mXP5cI_CM8CsurDrVi2KMg-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 3/4] build: test meson installation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Let's test installing with meson as part of Travis and the devtools/test-meson-builds.sh script. The resulting headers and binaries make more sense to run checks against, since they should be the same as what the final users get. In this patch, test-null.sh is now called on an installed testpmd. For this to work LD_LIBRARY_PATH must be set appropriately, but it can be hard to debug, so a call to ldd has been added. The (future) ABI compatibility checks will also need to filter its report against installed public headers. Signed-off-by: David Marchand --- .ci/linux-build.sh | 6 +++++- devtools/test-meson-builds.sh | 27 ++++++++++++++++----------- devtools/test-null.sh | 1 + 3 files changed, 22 insertions(+), 12 deletions(-) diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index ccc3a7ccd..e61aa2b0a 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -29,11 +29,15 @@ if [ "$BUILD_32BIT" = "1" ]; then fi OPTS="$OPTS --default-library=$DEF_LIB" +OPTS="$OPTS --prefix=/usr -Dlibdir=lib" meson build --werror -Dexamples=all $OPTS ninja -C build +DESTDIR=$(pwd)/install ninja -C build install if [ "$AARCH64" != "1" ]; then - devtools/test-null.sh + export LD_LIBRARY_PATH=$(pwd)/install/usr/lib + devtools/test-null.sh install/usr/bin/dpdk-testpmd + unset LD_LIBRARY_PATH fi if [ "$RUN_TESTS" = "1" ]; then diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index aed175889..254588ae6 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson-builds.sh @@ -63,7 +63,7 @@ config () # shift builddir=$1 shift - options="--werror -Dexamples=all" + options="--werror -Dexamples=all --prefix=/usr -Dlibdir=lib" for option in $DPDK_MESON_OPTIONS ; do options="$options -D$option" done @@ -74,22 +74,29 @@ config () # fi } -compile () # +compile () # { builddir=$1 shift + export DESTDIR=$1 + shift + rm -rf $DESTDIR if [ -n "$TEST_MESON_BUILD_VERY_VERBOSE" ] ; then # for full output from ninja use "-v" echo "$ninja_cmd -v -C $builddir" $ninja_cmd -v -C $builddir + $ninja_cmd -v -C $builddir install elif [ -n "$TEST_MESON_BUILD_VERBOSE" ] ; then # for keeping the history of short cmds, pipe through cat echo "$ninja_cmd -C $builddir | cat" $ninja_cmd -C $builddir | cat + $ninja_cmd -C $builddir install | cat else echo "$ninja_cmd -C $builddir" $ninja_cmd -C $builddir + $ninja_cmd -C $builddir install fi + unset DESTDIR } build () # @@ -102,7 +109,8 @@ build () # command -v ${CC##* } >/dev/null 2>&1 || return 0 load_env $targetcc || return 0 config $srcdir $builds_dir/$targetdir $* - compile $builds_dir/$targetdir + compile $builds_dir/$targetdir \ + $(readlink -f $builds_dir/$targetdir/install) } if [ "$1" = "-vv" ] ; then @@ -134,7 +142,7 @@ ok=$(cc -march=$default_machine -E - < /dev/null > /dev/null 2>&1 || echo false) if [ "$ok" = "false" ] ; then default_machine='corei7' fi -build build-x86-default cc -Dlibdir=lib -Dmachine=$default_machine $use_shared +build build-x86-default cc -Dmachine=$default_machine $use_shared c=aarch64-linux-gnu-gcc # generic armv8a with clang as host compiler @@ -150,12 +158,9 @@ for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do unset CC done -# Test installation of the x86-default target, to be used for checking -# the sample apps build using the pkg-config file for cflags and libs -build_path=$(readlink -f $builds_dir/build-x86-default) -export DESTDIR=$build_path/install-root -$ninja_cmd -C $build_path install - +# Use the x86-default target, to check the sample apps build using the +# pkg-config file for cflags and libs +export DESTDIR=$(readlink -f $builds_dir/build-x86-default/install) load_env cc pc_file=$(find $DESTDIR -name libdpdk.pc) export PKG_CONFIG_PATH=$(dirname $pc_file):$PKG_CONFIG_PATH @@ -165,6 +170,6 @@ if pkg-config --define-prefix libdpdk >/dev/null 2>&1; then export PKGCONF="pkg-config --define-prefix" for example in cmdline helloworld l2fwd l3fwd skeleton timer; do echo "## Building $example" - $MAKE -C $DESTDIR/usr/local/share/dpdk/examples/$example clean all + $MAKE -C $DESTDIR/usr/share/dpdk/examples/$example clean all done fi diff --git a/devtools/test-null.sh b/devtools/test-null.sh index 548de8113..8121ec9e3 100755 --- a/devtools/test-null.sh +++ b/devtools/test-null.sh @@ -21,6 +21,7 @@ fi if ldd $testpmd | grep -q librte_ ; then export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH + ldd $testpmd libs='-d librte_mempool_ring.so -d librte_pmd_null.so' else libs= From patchwork Wed Jan 29 17:26:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 65351 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B2C36A052F; Wed, 29 Jan 2020 18:27:24 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F03191C0AC; Wed, 29 Jan 2020 18:27:06 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id D1BE71C0AB for ; Wed, 29 Jan 2020 18:27:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580318824; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AwqJsHphzE8HoQSA4h5l2U/VtrYfjQM87iq/riluFTk=; b=Lu0QW/oWtOPPGbTVuxcFE9moY50DZf9xFigO/l+4DgcIW16lZMMDy2yRNNdIrn4vdbqh5B iE4W+hrA6ftL69D/YpbTLk1yQSU7XGreMagdh5vTl68D0gc64KtOzvm4Tatit8DihkpYfk 4bOv3Vi9wk1vdMgoQVScy3QWjjeTGP4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-265-Itm83PelPHSwZZNckyU-ag-1; Wed, 29 Jan 2020 12:26:46 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 70B5A8018A8; Wed, 29 Jan 2020 17:26:44 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-128.brq.redhat.com [10.40.204.128]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE71684BC9; Wed, 29 Jan 2020 17:26:41 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, kevin.laatz@intel.com, aconole@redhat.com, nhorman@tuxdriver.com, akhil.goyal@nxp.com, anoobj@marvell.com, Michael Santana , John McNamara , Marko Kovacevic Date: Wed, 29 Jan 2020 18:26:21 +0100 Message-Id: <20200129172621.28565-5-david.marchand@redhat.com> In-Reply-To: <20200129172621.28565-1-david.marchand@redhat.com> References: <20191220152058.10739-1-david.marchand@redhat.com> <20200129172621.28565-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: Itm83PelPHSwZZNckyU-ag-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 4/4] add ABI checks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" For normal developers, those checks are disabled. Enabling them requires a configuration that will trigger the ABI dumps generation as part of the existing devtools/test-build.sh and devtools/test-meson-builds.sh scripts. Those checks are enabled in the CI for the default meson options on x86 and aarch64 so that proposed patches are validated via our CI robot. A cache of the ABI is stored in travis jobs to avoid rebuilding too often. Checks can be only informational by setting ABI_CHECKS_WARN_ONLY when breaking the ABI in a future release. Explicit suppression rules have been added on internal structures exposed to crypto and security drivers as the current ABI policy does not apply to them. This could be improved in the future by carefully splitting the headers content with application and driver "users" in mind. We currently have issues reported for librte_crypto recent changes for which suppression rules have been added too. Mellanox glue libraries are explicitly skipped as they are not part of the application ABI. Signed-off-by: David Marchand --- Changelog since v1: - reworked the scripts so that the build test scripts clone and build the reference automatically. A developer only needs to set one variable to enable the checks, - meson builds are done with debug so that abidiff can inspect the structures, - abidiff checks only public types by looking at installed headers, - abidiff has some issue when comparing a dump with a .so built with clang so all diff are now done with dump files only, - suppression rules have been added to waive warnings on exposed internal types, - an abi breakage has been reported on changes in cryptodev. For now, suppression rules have been put in place to let the CI run, --- .ci/linux-build.sh | 23 +++++++++++ .travis.yml | 20 +++++++++- MAINTAINERS | 2 + devtools/check-abi.sh | 59 +++++++++++++++++++++++++++++ devtools/dpdk.abignore | 20 ++++++++++ devtools/gen-abi.sh | 26 +++++++++++++ devtools/test-build.sh | 45 ++++++++++++++++++++-- devtools/test-meson-builds.sh | 35 ++++++++++++++++- doc/guides/contributing/patches.rst | 13 +++++++ 9 files changed, 236 insertions(+), 7 deletions(-) create mode 100755 devtools/check-abi.sh create mode 100644 devtools/dpdk.abignore create mode 100755 devtools/gen-abi.sh diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index e61aa2b0a..95bd869c3 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -30,6 +30,7 @@ fi OPTS="$OPTS --default-library=$DEF_LIB" OPTS="$OPTS --prefix=/usr -Dlibdir=lib" +OPTS="$OPTS --buildtype=debugoptimized" meson build --werror -Dexamples=all $OPTS ninja -C build DESTDIR=$(pwd)/install ninja -C build install @@ -40,6 +41,28 @@ if [ "$AARCH64" != "1" ]; then unset LD_LIBRARY_PATH fi +if [ "$ABI_CHECKS" = "1" ]; then + REF_GIT_REPO=${REF_GIT_REPO:-https://dpdk.org/git/dpdk} + REF_GIT_TAG=${REF_GIT_TAG:-v19.11} + + if [ "$(cat reference/VERSION 2>/dev/null)" != "$REF_GIT_TAG" ]; then + rm -rf reference + fi + + if [ ! -d reference ]; then + refsrcdir=$(readlink -f $(pwd)/../dpdk-$REF_GIT_TAG) + git clone --single-branch -b $REF_GIT_TAG $REF_GIT_REPO $refsrcdir + meson --werror $OPTS $refsrcdir $refsrcdir/build + ninja -C $refsrcdir/build + DESTDIR=$(pwd)/reference ninja -C $refsrcdir/build install + devtools/gen-abi.sh reference + echo $REF_GIT_TAG > reference/VERSION + fi + + devtools/gen-abi.sh install + devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-} +fi + if [ "$RUN_TESTS" = "1" ]; then sudo meson test -C build --suite fast-tests -t 3 fi diff --git a/.travis.yml b/.travis.yml index 8162f1c05..22539d823 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,8 @@ language: c -cache: ccache +cache: + ccache: true + directories: + - reference compiler: - gcc - clang @@ -21,7 +24,7 @@ aarch64_packages: &aarch64_packages extra_packages: &extra_packages - *required_packages - - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4] + - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools] build_32b_packages: &build_32b_packages - *required_packages @@ -151,5 +154,18 @@ matrix: packages: - *required_packages - *doc_packages + - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1 + compiler: gcc + addons: + apt: + packages: + - *extra_packages + - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1 + arch: arm64 + compiler: gcc + addons: + apt: + packages: + - *extra_packages script: ./.ci/${TRAVIS_OS_NAME}-build.sh diff --git a/MAINTAINERS b/MAINTAINERS index 94bccae6d..6dae4ee63 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -144,8 +144,10 @@ M: Neil Horman F: lib/librte_eal/common/include/rte_compat.h F: lib/librte_eal/common/include/rte_function_versioning.h F: doc/guides/rel_notes/deprecation.rst +F: devtools/check-abi.sh F: devtools/check-abi-version.sh F: devtools/check-symbol-change.sh +F: devtools/gen-abi.sh F: devtools/update-abi.sh F: devtools/update_version_map_abi.py F: devtools/validate-abi.sh diff --git a/devtools/check-abi.sh b/devtools/check-abi.sh new file mode 100755 index 000000000..5872499ec --- /dev/null +++ b/devtools/check-abi.sh @@ -0,0 +1,59 @@ +#!/bin/sh -e +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Red Hat, Inc. + +if [ $# != 2 ] && [ $# != 3 ]; then + echo "Usage: $0 refdir newdir [warnonly]" + exit 1 +fi + +refdir=$1 +newdir=$2 +warnonly=${3:-} +ABIDIFF_OPTIONS="--suppr $(dirname $0)/dpdk.abignore --no-added-syms" + +if [ ! -d $refdir ]; then + echo "Error: reference directory '$refdir' does not exist." + exit 1 +fi +incdir=$(find $refdir -type d -a -name include) +if [ -z "$incdir" ] || [ ! -e "$incdir" ]; then + echo "WARNING: could not identify a include directory for $refdir, expect false positives..." +else + ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir1 $incdir" +fi + +if [ ! -d $newdir ]; then + echo "Error: directory to check '$newdir' does not exist." + exit 1 +fi +incdir2=$(find $newdir -type d -a -name include) +if [ -z "$incdir2" ] || [ ! -e "$incdir2" ]; then + echo "WARNING: could not identify a include directory for $newdir, expect false positives..." +else + ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir2 $incdir2" +fi + +error= +for dump in $(find $refdir -name "*.dump"); do + name=$(basename $dump) + # skip glue drivers, example librte_pmd_mlx5_glue.dump + # We can't rely on a suppression rule for now: + # https://sourceware.org/bugzilla/show_bug.cgi?id=25480 + if [ "$name" != "${name%%_glue.dump}" ]; then + echo "Skipping ${dump}..." + continue + fi + dump2=$(find $newdir -name $name) + if [ -z "$dump2" ] || [ ! -e "$dump2" ]; then + echo "Error: can't find $name in $newdir" + error=1 + continue + fi + if ! abidiff $ABIDIFF_OPTIONS $dump $dump2; then + echo "Error: ABI issue reported for 'abidiff $ABIDIFF_OPTIONS $dump $dump2'" + error=1 + fi +done + +[ -z "$error" ] || [ -n "$warnonly" ] diff --git a/devtools/dpdk.abignore b/devtools/dpdk.abignore new file mode 100644 index 000000000..0c01eebea --- /dev/null +++ b/devtools/dpdk.abignore @@ -0,0 +1,20 @@ +[suppress_function] + symbol_version = EXPERIMENTAL +[suppress_variable] + symbol_version = EXPERIMENTAL + +; Explicit ignore for driver-only ABI +[suppress_type] + name = rte_cryptodev_ops + +; FIXME +[suppress_type] + type_kind = enum + name = rte_crypto_aead_algorithm + changed_enumerators = RTE_CRYPTO_AEAD_LIST_END +[suppress_type] + type_kind = enum + name = rte_crypto_asym_xform_type + changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END +[suppress_variable] + name = rte_crypto_aead_algorithm_strings diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh new file mode 100755 index 000000000..c44b0e228 --- /dev/null +++ b/devtools/gen-abi.sh @@ -0,0 +1,26 @@ +#!/bin/sh -e +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Red Hat, Inc. + +if [ $# != 1 ]; then + echo "Usage: $0 installdir" + exit 1 +fi + +installdir=$1 +if [ ! -d $installdir ]; then + echo "Error: install directory '$installdir' does not exist." + exit 1 +fi + +dumpdir=$installdir/dump +rm -rf $dumpdir +mkdir -p $dumpdir +for f in $(find $installdir -name "*.so.*"); do + if test -L $f; then + continue + fi + + libname=$(basename $f) + abidw --out-file $dumpdir/${libname%.so*}.dump $f +done diff --git a/devtools/test-build.sh b/devtools/test-build.sh index 52305fbb8..fba30bac9 100755 --- a/devtools/test-build.sh +++ b/devtools/test-build.sh @@ -30,7 +30,8 @@ default_path=$PATH # - LIBSSO_SNOW3G_PATH # - LIBSSO_KASUMI_PATH # - LIBSSO_ZUC_PATH -. $(dirname $(readlink -f $0))/load-devel-config +devtools_dir=$(dirname $(readlink -f $0)) +. $devtools_dir/load-devel-config print_usage () { echo "usage: $(basename $0) [-h] [-jX] [-s] [config1 [config2] ...]]" @@ -64,6 +65,7 @@ print_help () { [ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1 J=$DPDK_MAKE_JOBS +refsrcdir=$(mktemp -d -t dpdk-${DPDK_ABI_REF_VERSION:-}.XXX) builds_dir=${DPDK_BUILD_TEST_DIR:-.} short=false unset verbose @@ -91,13 +93,14 @@ on_exit () [ "$DPDK_NOTIFY" != notify-send ] || \ notify-send -u low --icon=dialog-error 'DPDK build' 'failed' fi + rm -rf $refsrcdir } # catch manual interrupt to ignore notification trap "signal=INT ; trap - INT ; kill -INT $$" INT # notify result on exit trap on_exit EXIT -cd $(dirname $(readlink -f $0))/.. +cd $devtools_dir/.. reset_env () { @@ -233,7 +236,7 @@ for conf in $configs ; do # reload config with DPDK_TARGET set DPDK_TARGET=$target reset_env - . $(dirname $(readlink -f $0))/load-devel-config + . $devtools_dir/load-devel-config options=$(echo $conf | sed 's,[^~+]*,,') dir=$builds_dir/$conf @@ -253,6 +256,42 @@ for conf in $configs ; do EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \ O=$(readlink -f $dir)/examples unset RTE_TARGET + if [ -n "$DPDK_ABI_REF_VERSION" ]; then + DPDK_ABI_REF_DIR=${DPDK_ABI_REF_DIR:-reference} + abirefdir=$DPDK_ABI_REF_DIR/$DPDK_ABI_REF_VERSION/$conf + if [ ! -d $abirefdir ]; then + # clone current sources + if [ ! -d $refsrcdir/.git ]; then + git clone --local --no-hardlinks \ + --single-branch \ + -b $DPDK_ABI_REF_VERSION \ + $(pwd) $refsrcdir + fi + + cd $refsrcdir + + rm -rf build + config build $target $options + + echo -n "================== Build $conf " + echo "($DPDK_ABI_REF_VERSION)" + ${MAKE} -j$J EXTRA_CFLAGS="$maxerr $DPDK_DEP_CFLAGS" \ + EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \ + O=build + ! $short || break + export RTE_TARGET=$target + ${MAKE} install O=build DESTDIR=$abirefdir \ + prefix= + $devtools_dir/gen-abi.sh $abirefdir + + # back to current workdir + cd $devtools_dir/.. + fi + + echo "================== Check ABI $conf" + $devtools_dir/gen-abi.sh $dir/install + $devtools_dir/check-abi.sh $abirefdir $dir/install + fi echo "################## $conf done." unset dir done diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index 254588ae6..1b410c784 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson-builds.sh @@ -16,8 +16,15 @@ srcdir=$(dirname $(readlink -f $0))/.. MESON=${MESON:-meson} use_shared="--default-library=shared" +refsrcdir=$(mktemp -d -t dpdk-${DPDK_ABI_REF_VERSION:-}.XXX) builds_dir=${DPDK_BUILD_TEST_DIR:-.} +on_exit () +{ + rm -rf $refsrcdir +} +trap on_exit EXIT + if command -v gmake >/dev/null 2>&1 ; then MAKE=gmake else @@ -63,7 +70,9 @@ config () # shift builddir=$1 shift - options="--werror -Dexamples=all --prefix=/usr -Dlibdir=lib" + options= + options="$options --werror --buildtype=debugoptimized -Dexamples=all" + options="$options --prefix=/usr -Dlibdir=lib" for option in $DPDK_MESON_OPTIONS ; do options="$options -D$option" done @@ -96,7 +105,6 @@ compile () # $ninja_cmd -C $builddir $ninja_cmd -C $builddir install fi - unset DESTDIR } build () # @@ -111,6 +119,29 @@ build () # config $srcdir $builds_dir/$targetdir $* compile $builds_dir/$targetdir \ $(readlink -f $builds_dir/$targetdir/install) + if [ -n "$DPDK_ABI_REF_VERSION" ]; then + DPDK_ABI_REF_DIR=${DPDK_ABI_REF_DIR:-reference} + abirefdir=$DPDK_ABI_REF_DIR/$DPDK_ABI_REF_VERSION/$targetdir + if [ ! -d $abirefdir ]; then + # clone current sources + if [ ! -d $refsrcdir/.git ]; then + git clone --local --no-hardlinks \ + --single-branch \ + -b $DPDK_ABI_REF_VERSION \ + $srcdir $refsrcdir + fi + + rm -rf $refsrcdir/build + config $refsrcdir $refsrcdir/build $* + compile $refsrcdir/build $abirefdir + $srcdir/devtools/gen-abi.sh $abirefdir + fi + + $srcdir/devtools/gen-abi.sh \ + $(readlink -f $builds_dir/$targetdir/install) + $srcdir/devtools/check-abi.sh $abirefdir \ + $(readlink -f $builds_dir/$targetdir/install) + fi } if [ "$1" = "-vv" ] ; then diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst index 0686450e4..2e16741ca 100644 --- a/doc/guides/contributing/patches.rst +++ b/doc/guides/contributing/patches.rst @@ -513,6 +513,19 @@ in a single subfolder called "__builds" created in the current directory. Setting ``DPDK_BUILD_TEST_DIR`` to an absolute directory path e.g. ``/tmp`` is also supported. +Checking ABI compatibility +-------------------------- + +By default, ABI compatibility checks are disabled. + +To enable them, a reference version must be selected via the environment +variable ``DPDK_ABI_REF_VERSION``. + +The ``devtools/test-build.sh`` and ``devtools/test-meson-builds.sh`` scripts +then build this reference version in a temporary directory and store the +results in the ``DPDK_ABI_REF_DIR`` directory. + + Sending Patches ---------------