From patchwork Sun Feb 2 21:08:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 65476 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9704BA04FA; Sun, 2 Feb 2020 22:09:05 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C87681BF7A; Sun, 2 Feb 2020 22:09:04 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id BDC7DDE3 for ; Sun, 2 Feb 2020 22:09:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580677741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tI53hqCxl+qze2IVg0G1+cgPFMOlGoHbqBanEqlrDxc=; b=E2yNNN01EvKUSFQxOm8OxzcuUkJDqi6rIM22Wt23Z9JWMuqgp7GbKsDwhc3plYPBiiFnyD nJWGfExfjsD0qdCxzkgFm/uyy+V1gPUuIbm2c5gU7Aj3pcGWm5jzFTFVCHeQ1g3xLGsk/X +vRF5FUgfABsvohAyqOPpaEpYP2V3Gk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-393-qV0pXyVaOhmjbcPm1cHl5g-1; Sun, 02 Feb 2020 16:08:57 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B9717800D4E; Sun, 2 Feb 2020 21:08:54 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-51.brq.redhat.com [10.40.204.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id EE38787ECB; Sun, 2 Feb 2020 21:08:50 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, kevin.laatz@intel.com, aconole@redhat.com, nhorman@tuxdriver.com, akhil.goyal@nxp.com, anoobj@marvell.com, bluca@debian.org, fiona.trahe@intel.com, ferruh.yigit@intel.com, stable@dpdk.org, Yipeng Wang , Sameh Gobriel , Harry van Haaren , Luca Boccassi , Keith Wiles Date: Sun, 2 Feb 2020 22:08:32 +0100 Message-Id: <20200202210835.29791-2-david.marchand@redhat.com> In-Reply-To: <20200202210835.29791-1-david.marchand@redhat.com> References: <20191220152058.10739-1-david.marchand@redhat.com> <20200202210835.29791-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: qV0pXyVaOhmjbcPm1cHl5g-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 1/3] hash: fix meson headers packaging X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Those headers are internal and should not be distributed. Fixes: 5b9656b157d3 ("lib: build with meson") Cc: stable@dpdk.org Signed-off-by: David Marchand Acked-by: Luca Boccassi Acked-by: Yipeng Wang --- lib/librte_hash/meson.build | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/lib/librte_hash/meson.build b/lib/librte_hash/meson.build index 5d02b3084..bce11ad9e 100644 --- a/lib/librte_hash/meson.build +++ b/lib/librte_hash/meson.build @@ -1,10 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -headers = files('rte_cmp_arm64.h', - 'rte_cmp_x86.h', - 'rte_crc_arm64.h', - 'rte_cuckoo_hash.h', +headers = files('rte_crc_arm64.h', 'rte_fbk_hash.h', 'rte_hash_crc.h', 'rte_hash.h', From patchwork Sun Feb 2 21:08:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 65478 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B1C2BA04FA; Sun, 2 Feb 2020 22:09:24 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DAEF11BFF8; Sun, 2 Feb 2020 22:09:07 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id A8288DE3 for ; Sun, 2 Feb 2020 22:09:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580677743; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V9P1sb0i9UOEGssiU6PRP8bXQF4VckTs3RYvJAe0hF8=; b=eqjr4mJFozFugHyo6pe32gEYgFF2PrrcVbBOy7WHzqj+fEyyeuZagNjwDXB3PpvQg5ZYYa S9YH5ZBc9rs4jxPrK6Oci8fRJy7v2jLiMmabmf/T04waP0PFiMqn4gAQxrCut1ND7pch8s YnpNaSRqcsCREz/b6l7Y1zYQSCqVeCg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-395-tNLpNra-MlWPl8siEZhrag-1; Sun, 02 Feb 2020 16:08:59 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ACCD8107ACC7; Sun, 2 Feb 2020 21:08:57 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-51.brq.redhat.com [10.40.204.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4180887ECB; Sun, 2 Feb 2020 21:08:55 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, kevin.laatz@intel.com, aconole@redhat.com, nhorman@tuxdriver.com, akhil.goyal@nxp.com, anoobj@marvell.com, bluca@debian.org, fiona.trahe@intel.com, ferruh.yigit@intel.com Date: Sun, 2 Feb 2020 22:08:33 +0100 Message-Id: <20200202210835.29791-3-david.marchand@redhat.com> In-Reply-To: <20200202210835.29791-1-david.marchand@redhat.com> References: <20191220152058.10739-1-david.marchand@redhat.com> <20200202210835.29791-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: tNLpNra-MlWPl8siEZhrag-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 2/3] build: split build helper X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" No functional change intended, prepare for reusing this code. The config and compilation parts are separated in helpers. Unsetting CC is moved to the caller of the helper. Signed-off-by: David Marchand Acked-by: Luca Boccassi --- Changelog since v2: - changed indent in config(), - removed useless shift in compile(), --- devtools/test-meson-builds.sh | 48 ++++++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 15 deletions(-) diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index 688567714..fb6c404e5 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson-builds.sh @@ -57,25 +57,27 @@ load_env () # . $srcdir/devtools/load-devel-config } -build () # +config () # { - builddir=$builds_dir/$1 + dir=$1 shift - targetcc=$1 + builddir=$1 shift - # skip build if compiler not available - command -v ${CC##* } >/dev/null 2>&1 || return 0 - load_env $targetcc || return 0 - if [ ! -f "$builddir/build.ninja" ] ; then - options="--werror -Dexamples=all" - for option in $DPDK_MESON_OPTIONS ; do - options="$options -D$option" - done - options="$options $*" - echo "$MESON $options $srcdir $builddir" - $MESON $options $srcdir $builddir - unset CC + if [ -f "$builddir/build.ninja" ] ; then + return fi + options="--werror -Dexamples=all" + for option in $DPDK_MESON_OPTIONS ; do + options="$options -D$option" + done + options="$options $*" + echo "$MESON $options $dir $builddir" + $MESON $options $dir $builddir +} + +compile () # +{ + builddir=$1 if [ -n "$TEST_MESON_BUILD_VERY_VERBOSE" ] ; then # for full output from ninja use "-v" echo "$ninja_cmd -v -C $builddir" @@ -90,6 +92,19 @@ build () # fi } +build () # +{ + targetdir=$1 + shift + targetcc=$1 + shift + # skip build if compiler not available + command -v ${CC##* } >/dev/null 2>&1 || return 0 + load_env $targetcc || return 0 + config $srcdir $builds_dir/$targetdir $* + compile $builds_dir/$targetdir +} + if [ "$1" = "-vv" ] ; then TEST_MESON_BUILD_VERY_VERBOSE=1 elif [ "$1" = "-v" ] ; then @@ -107,6 +122,7 @@ for c in gcc clang ; do for s in static shared ; do export CC="$CCACHE $c" build build-$c-$s $c --default-library=$s + unset CC done done @@ -125,11 +141,13 @@ c=aarch64-linux-gnu-gcc export CC="clang" build build-arm64-host-clang $c $use_shared \ --cross-file $srcdir/config/arm/arm64_armv8_linux_gcc +unset CC # all gcc/arm configurations for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do export CC="$CCACHE gcc" build build-$(basename $f | tr '_' '-' | cut -d'-' -f-2) $c \ $use_shared --cross-file $f + unset CC done # Test installation of the x86-default target, to be used for checking From patchwork Sun Feb 2 21:08:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 65479 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 99690A04FA; Sun, 2 Feb 2020 22:09:32 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 449DE1BFE3; Sun, 2 Feb 2020 22:09:14 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id 28EFA1BF7B for ; Sun, 2 Feb 2020 22:09:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580677752; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=01SVZcXmINevzvzyV4TWF/UasMQPopBC6K5terdK1ho=; b=E7X/+TmMu4BfL5ef5oABowsyJ2H6f6cDUW+hoQKO1DewDi8ic81LF59tF/k3IoPtNt79hz h8cwr/kP+n2JBBisnL7S93qjcjUtyqFvUj4etU8jupHKDuQcZVCBBnFv9M+feEpmlcyr6T WZwufC+C1P8POM0HUwSjOyS2NI7Wx+8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-321-dL7CuTYKMP2UwQ78m-gVUQ-1; Sun, 02 Feb 2020 16:09:03 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 461528017DF; Sun, 2 Feb 2020 21:09:01 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-51.brq.redhat.com [10.40.204.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3FEC98E9E5; Sun, 2 Feb 2020 21:08:58 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, kevin.laatz@intel.com, aconole@redhat.com, nhorman@tuxdriver.com, akhil.goyal@nxp.com, anoobj@marvell.com, bluca@debian.org, fiona.trahe@intel.com, ferruh.yigit@intel.com, Michael Santana , John McNamara , Marko Kovacevic Date: Sun, 2 Feb 2020 22:08:34 +0100 Message-Id: <20200202210835.29791-4-david.marchand@redhat.com> In-Reply-To: <20200202210835.29791-1-david.marchand@redhat.com> References: <20191220152058.10739-1-david.marchand@redhat.com> <20200202210835.29791-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: dL7CuTYKMP2UwQ78m-gVUQ-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 3/3] add ABI checks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" For normal developers, those checks are disabled. Enabling them requires a configuration that will trigger the ABI dumps generation as part of the existing devtools/test-build.sh and devtools/test-meson-builds.sh scripts. Those checks are enabled in the CI for the default meson options on x86 and aarch64 so that proposed patches are validated via our CI robot. A cache of the ABI is stored in travis jobs to avoid rebuilding too often. Checks can be informational only, by setting ABI_CHECKS_WARN_ONLY when breaking the ABI in a future release. Explicit suppression rules have been added on internal structures exposed to crypto drivers as the current ABI policy does not apply to them. This could be improved in the future by carefully splitting the headers content with application and driver "users" in mind. We currently have issues reported for librte_crypto recent changes for which suppression rules have been added too. Mellanox glue libraries are explicitly skipped as they are not part of the application ABI. Signed-off-by: David Marchand Acked-by: Luca Boccassi --- Changelog since v3: - rebased following patch 3 removal, - added dpdk.abignore to MAINTAINERS, - avoided meson reconfiguration when unneeded, - switched to fixed sources location for building ABI references, Changelog since v2: - forced -g / buildtype=debugoptimised in the test scripts so that we can check ABI in existing environments, - little update on the documentation, Changelog since v1: - reworked the scripts so that the build test scripts clone and build the reference automatically. A developer only needs to set one variable to enable the checks, - meson builds are done with debug so that abidiff can inspect the structures, - abidiff checks only public types by looking at installed headers, - abidiff has some issue when comparing a dump with a .so built with clang so all diff are now done with dump files only, - suppression rules have been added to waive warnings on exposed internal types, - an abi breakage has been reported on changes in cryptodev. For now, suppression rules have been put in place to let the CI run, --- .ci/linux-build.sh | 24 ++++++++++++ .travis.yml | 20 +++++++++- MAINTAINERS | 3 ++ devtools/check-abi.sh | 59 +++++++++++++++++++++++++++++ devtools/dpdk.abignore | 21 ++++++++++ devtools/gen-abi.sh | 26 +++++++++++++ devtools/test-build.sh | 52 ++++++++++++++++++++++--- devtools/test-meson-builds.sh | 50 ++++++++++++++++++++++-- doc/guides/contributing/patches.rst | 15 ++++++++ 9 files changed, 260 insertions(+), 10 deletions(-) create mode 100755 devtools/check-abi.sh create mode 100644 devtools/dpdk.abignore create mode 100755 devtools/gen-abi.sh diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index ccc3a7ccd..c7c3840fc 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -29,6 +29,7 @@ if [ "$BUILD_32BIT" = "1" ]; then fi OPTS="$OPTS --default-library=$DEF_LIB" +OPTS="$OPTS --buildtype=debugoptimized" meson build --werror -Dexamples=all $OPTS ninja -C build @@ -36,6 +37,29 @@ if [ "$AARCH64" != "1" ]; then devtools/test-null.sh fi +if [ "$ABI_CHECKS" = "1" ]; then + REF_GIT_REPO=${REF_GIT_REPO:-https://dpdk.org/git/dpdk} + REF_GIT_TAG=${REF_GIT_TAG:-v19.11} + + if [ "$(cat reference/VERSION 2>/dev/null)" != "$REF_GIT_TAG" ]; then + rm -rf reference + fi + + if [ ! -d reference ]; then + refsrcdir=$(readlink -f $(pwd)/../dpdk-$REF_GIT_TAG) + git clone --single-branch -b $REF_GIT_TAG $REF_GIT_REPO $refsrcdir + meson --werror $OPTS $refsrcdir $refsrcdir/build + ninja -C $refsrcdir/build + DESTDIR=$(pwd)/reference ninja -C $refsrcdir/build install + devtools/gen-abi.sh reference + echo $REF_GIT_TAG > reference/VERSION + fi + + DESTDIR=$(pwd)/install ninja -C build install + devtools/gen-abi.sh install + devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-} +fi + if [ "$RUN_TESTS" = "1" ]; then sudo meson test -C build --suite fast-tests -t 3 fi diff --git a/.travis.yml b/.travis.yml index 8162f1c05..22539d823 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,8 @@ language: c -cache: ccache +cache: + ccache: true + directories: + - reference compiler: - gcc - clang @@ -21,7 +24,7 @@ aarch64_packages: &aarch64_packages extra_packages: &extra_packages - *required_packages - - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4] + - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools] build_32b_packages: &build_32b_packages - *required_packages @@ -151,5 +154,18 @@ matrix: packages: - *required_packages - *doc_packages + - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1 + compiler: gcc + addons: + apt: + packages: + - *extra_packages + - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1 + arch: arm64 + compiler: gcc + addons: + apt: + packages: + - *extra_packages script: ./.ci/${TRAVIS_OS_NAME}-build.sh diff --git a/MAINTAINERS b/MAINTAINERS index 94bccae6d..db4b8c5b6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -144,8 +144,11 @@ M: Neil Horman F: lib/librte_eal/common/include/rte_compat.h F: lib/librte_eal/common/include/rte_function_versioning.h F: doc/guides/rel_notes/deprecation.rst +F: devtools/check-abi.sh F: devtools/check-abi-version.sh F: devtools/check-symbol-change.sh +F: devtools/dpdk.abignore +F: devtools/gen-abi.sh F: devtools/update-abi.sh F: devtools/update_version_map_abi.py F: devtools/validate-abi.sh diff --git a/devtools/check-abi.sh b/devtools/check-abi.sh new file mode 100755 index 000000000..5872499ec --- /dev/null +++ b/devtools/check-abi.sh @@ -0,0 +1,59 @@ +#!/bin/sh -e +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Red Hat, Inc. + +if [ $# != 2 ] && [ $# != 3 ]; then + echo "Usage: $0 refdir newdir [warnonly]" + exit 1 +fi + +refdir=$1 +newdir=$2 +warnonly=${3:-} +ABIDIFF_OPTIONS="--suppr $(dirname $0)/dpdk.abignore --no-added-syms" + +if [ ! -d $refdir ]; then + echo "Error: reference directory '$refdir' does not exist." + exit 1 +fi +incdir=$(find $refdir -type d -a -name include) +if [ -z "$incdir" ] || [ ! -e "$incdir" ]; then + echo "WARNING: could not identify a include directory for $refdir, expect false positives..." +else + ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir1 $incdir" +fi + +if [ ! -d $newdir ]; then + echo "Error: directory to check '$newdir' does not exist." + exit 1 +fi +incdir2=$(find $newdir -type d -a -name include) +if [ -z "$incdir2" ] || [ ! -e "$incdir2" ]; then + echo "WARNING: could not identify a include directory for $newdir, expect false positives..." +else + ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir2 $incdir2" +fi + +error= +for dump in $(find $refdir -name "*.dump"); do + name=$(basename $dump) + # skip glue drivers, example librte_pmd_mlx5_glue.dump + # We can't rely on a suppression rule for now: + # https://sourceware.org/bugzilla/show_bug.cgi?id=25480 + if [ "$name" != "${name%%_glue.dump}" ]; then + echo "Skipping ${dump}..." + continue + fi + dump2=$(find $newdir -name $name) + if [ -z "$dump2" ] || [ ! -e "$dump2" ]; then + echo "Error: can't find $name in $newdir" + error=1 + continue + fi + if ! abidiff $ABIDIFF_OPTIONS $dump $dump2; then + echo "Error: ABI issue reported for 'abidiff $ABIDIFF_OPTIONS $dump $dump2'" + error=1 + fi +done + +[ -z "$error" ] || [ -n "$warnonly" ] diff --git a/devtools/dpdk.abignore b/devtools/dpdk.abignore new file mode 100644 index 000000000..f2903612c --- /dev/null +++ b/devtools/dpdk.abignore @@ -0,0 +1,21 @@ +[suppress_function] + symbol_version = EXPERIMENTAL +[suppress_variable] + symbol_version = EXPERIMENTAL + +; Explicit ignore for driver-only ABI +[suppress_type] + name = rte_cryptodev_ops +; Ignore this enum update as it is part of an experimental API +[suppress_type] + type_kind = enum + name = rte_crypto_asym_xform_type + changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END + +; FIXME +[suppress_type] + type_kind = enum + name = rte_crypto_aead_algorithm + changed_enumerators = RTE_CRYPTO_AEAD_LIST_END +[suppress_variable] + name = rte_crypto_aead_algorithm_strings diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh new file mode 100755 index 000000000..c44b0e228 --- /dev/null +++ b/devtools/gen-abi.sh @@ -0,0 +1,26 @@ +#!/bin/sh -e +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Red Hat, Inc. + +if [ $# != 1 ]; then + echo "Usage: $0 installdir" + exit 1 +fi + +installdir=$1 +if [ ! -d $installdir ]; then + echo "Error: install directory '$installdir' does not exist." + exit 1 +fi + +dumpdir=$installdir/dump +rm -rf $dumpdir +mkdir -p $dumpdir +for f in $(find $installdir -name "*.so.*"); do + if test -L $f; then + continue + fi + + libname=$(basename $f) + abidw --out-file $dumpdir/${libname%.so*}.dump $f +done diff --git a/devtools/test-build.sh b/devtools/test-build.sh index 52305fbb8..2f55e9147 100755 --- a/devtools/test-build.sh +++ b/devtools/test-build.sh @@ -6,6 +6,8 @@ default_path=$PATH # Load config options: # - ARMV8_CRYPTO_LIB_PATH +# - DPDK_ABI_REF_DIR +# - DPDK_ABI_REF_VERSION # - DPDK_BUILD_TEST_CONFIGS (defconfig1+option1+option2 defconfig2) # - DPDK_BUILD_TEST_DIR # - DPDK_DEP_ARCHIVE @@ -30,7 +32,8 @@ default_path=$PATH # - LIBSSO_SNOW3G_PATH # - LIBSSO_KASUMI_PATH # - LIBSSO_ZUC_PATH -. $(dirname $(readlink -f $0))/load-devel-config +devtools_dir=$(dirname $(readlink -f $0)) +. $devtools_dir/load-devel-config print_usage () { echo "usage: $(basename $0) [-h] [-jX] [-s] [config1 [config2] ...]]" @@ -67,7 +70,8 @@ J=$DPDK_MAKE_JOBS builds_dir=${DPDK_BUILD_TEST_DIR:-.} short=false unset verbose -maxerr=-Wfatal-errors +# for ABI checks, we need debuginfo +test_cflags="-Wfatal-errors -g" while getopts hj:sv ARG ; do case $ARG in j ) J=$OPTARG ;; @@ -97,7 +101,7 @@ trap "signal=INT ; trap - INT ; kill -INT $$" INT # notify result on exit trap on_exit EXIT -cd $(dirname $(readlink -f $0))/.. +cd $devtools_dir/.. reset_env () { @@ -233,14 +237,14 @@ for conf in $configs ; do # reload config with DPDK_TARGET set DPDK_TARGET=$target reset_env - . $(dirname $(readlink -f $0))/load-devel-config + . $devtools_dir/load-devel-config options=$(echo $conf | sed 's,[^~+]*,,') dir=$builds_dir/$conf config $dir $target $options echo "================== Build $conf" - ${MAKE} -j$J EXTRA_CFLAGS="$maxerr $DPDK_DEP_CFLAGS" \ + ${MAKE} -j$J EXTRA_CFLAGS="$test_cflags $DPDK_DEP_CFLAGS" \ EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose O=$dir ! $short || break export RTE_TARGET=$target @@ -253,6 +257,44 @@ for conf in $configs ; do EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \ O=$(readlink -f $dir)/examples unset RTE_TARGET + if [ -n "$DPDK_ABI_REF_VERSION" ]; then + abirefdir=${DPDK_ABI_REF_DIR:-reference}/$DPDK_ABI_REF_VERSION + if [ ! -d $abirefdir/$conf ]; then + # clone current sources + if [ ! -d $abirefdir/src ]; then + git clone --local --no-hardlinks \ + --single-branch \ + -b $DPDK_ABI_REF_VERSION \ + $(pwd) $abirefdir/src + fi + + cd $abirefdir/src + + rm -rf $abirefdir/build + config $abirefdir/build $target $options + + echo -n "================== Build $conf " + echo "($DPDK_ABI_REF_VERSION)" + ${MAKE} -j$J \ + EXTRA_CFLAGS="$test_cflags $DPDK_DEP_CFLAGS" \ + EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \ + O=$abirefdir/build + ! $short || break + export RTE_TARGET=$target + ${MAKE} install O=$abirefdir/build \ + DESTDIR=$abirefdir/$conf \ + prefix= + unset RTE_TARGET + $devtools_dir/gen-abi.sh $abirefdir/$conf + + # back to current workdir + cd $devtools_dir/.. + fi + + echo "================== Check ABI $conf" + $devtools_dir/gen-abi.sh $dir/install + $devtools_dir/check-abi.sh $abirefdir/$conf $dir/install + fi echo "################## $conf done." unset dir done diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index fb6c404e5..c2d33b3b9 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson-builds.sh @@ -64,9 +64,18 @@ config () # builddir=$1 shift if [ -f "$builddir/build.ninja" ] ; then + # for existing environments, switch to debugoptimized if unset + # so that ABI checks can run + if ! $MESON configure $builddir | + awk '$1=="buildtype" {print $2}' | + grep -qw debugoptimized; then + $MESON configure --buildtype=debugoptimized $builddir + fi return fi - options="--werror -Dexamples=all" + options= + options="$options --werror -Dexamples=all" + options="$options --buildtype=debugoptimized" for option in $DPDK_MESON_OPTIONS ; do options="$options -D$option" done @@ -92,6 +101,13 @@ compile () # fi } +install_target () # +{ + rm -rf $2 + echo "DESTDIR=$2 $ninja_cmd -C $1 install" + DESTDIR=$2 $ninja_cmd -C $1 install +} + build () # { targetdir=$1 @@ -103,6 +119,31 @@ build () # load_env $targetcc || return 0 config $srcdir $builds_dir/$targetdir $* compile $builds_dir/$targetdir + if [ -n "$DPDK_ABI_REF_VERSION" ]; then + abirefdir=${DPDK_ABI_REF_DIR:-reference}/$DPDK_ABI_REF_VERSION + if [ ! -d $abirefdir/$targetdir ]; then + # clone current sources + if [ ! -d $abirefdir/src ]; then + git clone --local --no-hardlinks \ + --single-branch \ + -b $DPDK_ABI_REF_VERSION \ + $srcdir $abirefdir/src + fi + + rm -rf $abirefdir/build + config $abirefdir/src $abirefdir/build $* + compile $abirefdir/build + install_target $abirefdir/build $abirefdir/$targetdir + $srcdir/devtools/gen-abi.sh $abirefdir/$targetdir + fi + + install_target $builds_dir/$targetdir \ + $(readlink -f $builds_dir/$targetdir/install) + $srcdir/devtools/gen-abi.sh \ + $(readlink -f $builds_dir/$targetdir/install) + $srcdir/devtools/check-abi.sh $abirefdir/$targetdir \ + $(readlink -f $builds_dir/$targetdir/install) + fi } if [ "$1" = "-vv" ] ; then @@ -153,8 +194,11 @@ done # Test installation of the x86-default target, to be used for checking # the sample apps build using the pkg-config file for cflags and libs build_path=$(readlink -f $builds_dir/build-x86-default) -export DESTDIR=$build_path/install-root -$ninja_cmd -C $build_path install +export DESTDIR=$build_path/install +# No need to reinstall if ABI checks are enabled +if [ -z "$DPDK_ABI_REF_VERSION" ]; then + install_target $build_path $DESTDIR +fi load_env cc pc_file=$(find $DESTDIR -name libdpdk.pc) diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst index 0686450e4..59442824a 100644 --- a/doc/guides/contributing/patches.rst +++ b/doc/guides/contributing/patches.rst @@ -513,6 +513,21 @@ in a single subfolder called "__builds" created in the current directory. Setting ``DPDK_BUILD_TEST_DIR`` to an absolute directory path e.g. ``/tmp`` is also supported. +Checking ABI compatibility +-------------------------- + +By default, ABI compatibility checks are disabled. + +To enable them, a reference version must be selected via the environment +variable ``DPDK_ABI_REF_VERSION``. + +The ``devtools/test-build.sh`` and ``devtools/test-meson-builds.sh`` scripts +then build this reference version in a temporary directory and store the +results in a subfolder of the current working directory. +The environment variable ``DPDK_ABI_REF_DIR`` can be set so that the results go +to a different location. + + Sending Patches ---------------