From patchwork Fri Sep 18 12:11:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 78100 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6DC5A04C8; Fri, 18 Sep 2020 14:11:59 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E110E1DA14; Fri, 18 Sep 2020 14:11:55 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 0DD9B1DA06 for ; Fri, 18 Sep 2020 14:11:50 +0200 (CEST) IronPort-SDR: Sp9fc+xAqJX91/xHqYZW6u2RpLSB9/iD5XhIkXQLpeKya+vB2SzRiwkc+/kLsQASMzQqxl2mr/ +pZZSYb5sm9Q== X-IronPort-AV: E=McAfee;i="6000,8403,9747"; a="160842563" X-IronPort-AV: E=Sophos;i="5.77,274,1596524400"; d="scan'208";a="160842563" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2020 05:11:50 -0700 IronPort-SDR: h2ytMrKUwJzM72Vqdz4fqZoRGGtOom/x65rF60XZrdNQbWOpheP9agsBR+tpSVj8xn1+5xvsdc zXQgsprC1x6w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,274,1596524400"; d="scan'208";a="410271198" Received: from silpixa00400466.ir.intel.com ([10.237.213.195]) by fmsmga001.fm.intel.com with ESMTP; 18 Sep 2020 05:11:48 -0700 From: Conor Walsh To: dev@dpdk.org Cc: david.marchand@redhat.com, ray.kinsella@intel.com, nhorman@tuxdriver.com, aconole@redhat.com, maicolgabriel@hotmail.com, thomas@monjalon.net, bruce.richardson@intel.com, anatoly.burakov@intel.com, Conor Walsh Date: Fri, 18 Sep 2020 12:11:34 +0000 Message-Id: <20200918121137.1370883-2-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918121137.1370883-1-conor.walsh@intel.com> References: <20200911160332.256343-1-conor.walsh@intel.com> <20200918121137.1370883-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 1/4] devtools: bug fix for gen-abi.sh X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch fixes a bug with the gen-abi.sh script in devtools. When ran on an install directory the script would try to generate .dump files from directories as well as the .so files which is not correct. Example error: abidw: gcc/lib/librte_net.so.21.0.p is not a regular file To rectify this the regex that finds the appropriate .so files has been changed and the file test has been removed. This change was tested with the ABI_CHECK Travis checks in DPDK 20.08. Travis build: https://travis-ci.com/github/conorwalsh-intel/dpdk/jobs/382812849 Signed-off-by: Conor Walsh --- devtools/gen-abi.sh | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh index c44b0e228..da6fe0556 100755 --- a/devtools/gen-abi.sh +++ b/devtools/gen-abi.sh @@ -16,11 +16,7 @@ fi dumpdir=$installdir/dump rm -rf $dumpdir mkdir -p $dumpdir -for f in $(find $installdir -name "*.so.*"); do - if test -L $f; then - continue - fi - +for f in $(find $installdir -name "*.so"); do libname=$(basename $f) abidw --out-file $dumpdir/${libname%.so*}.dump $f done From patchwork Fri Sep 18 12:11:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 78101 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D399BA04C8; Fri, 18 Sep 2020 14:12:07 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E4C311DA1C; Fri, 18 Sep 2020 14:11:56 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 9E7CA1DA0E for ; Fri, 18 Sep 2020 14:11:53 +0200 (CEST) IronPort-SDR: GLMs37r0raqMpmh63YBqt+UIbsVwOBkdVYUhTdAuOXb3nzXOdnH03+V3AqBhruoB9vWIZ0Kpgj FZ3hm2GyvjFw== X-IronPort-AV: E=McAfee;i="6000,8403,9747"; a="160842566" X-IronPort-AV: E=Sophos;i="5.77,274,1596524400"; d="scan'208";a="160842566" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2020 05:11:53 -0700 IronPort-SDR: 6tkZIPA5/RqiUIJRIvoZIad/lvTWpUva0kVQHASdPKpFKXLck7jIL4x80Dav+R855KByk91c0W fdNNMamFOSeQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,274,1596524400"; d="scan'208";a="410271212" Received: from silpixa00400466.ir.intel.com ([10.237.213.195]) by fmsmga001.fm.intel.com with ESMTP; 18 Sep 2020 05:11:50 -0700 From: Conor Walsh To: dev@dpdk.org Cc: david.marchand@redhat.com, ray.kinsella@intel.com, nhorman@tuxdriver.com, aconole@redhat.com, maicolgabriel@hotmail.com, thomas@monjalon.net, bruce.richardson@intel.com, anatoly.burakov@intel.com, Conor Walsh Date: Fri, 18 Sep 2020 12:11:35 +0000 Message-Id: <20200918121137.1370883-3-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918121137.1370883-1-conor.walsh@intel.com> References: <20200911160332.256343-1-conor.walsh@intel.com> <20200918121137.1370883-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 2/4] devtools: add generation of compressed abi dump archives X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds a script that generates a compressed archive containing .dump files which can be used to perform ABI breakage checking for the build specified in the parameters. Invoke using "./gen-abi-tarball.py [-t ] [-a ] [-cf ]" - : dpdk tag e.g. "v20.11", default: latest - : required architecture e.g. "arm" or "x86_64", default: current system's architecture - : configuration file for cross compiling for another system, this flag is not required. e.g. "config/arm/arm64_armv8_linux_gcc", default: None E.g. "./gen-abi-tarball.py -t latest -a x86_64" If a compiler is not specified using the CC environmental variable then the script will default to using gcc. Using these parameters the script will produce a .tar.gz archive containing .dump files required to do ABI breakage checking Signed-off-by: Conor Walsh --- devtools/gen-abi-tarball.py | 142 ++++++++++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100755 devtools/gen-abi-tarball.py diff --git a/devtools/gen-abi-tarball.py b/devtools/gen-abi-tarball.py new file mode 100755 index 000000000..6c337104e --- /dev/null +++ b/devtools/gen-abi-tarball.py @@ -0,0 +1,142 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +""" +This Python script generates a compressed archive containing .dump +files which can be used to perform ABI breakage checking for the +build specified in the parameters. +""" + +import os +from os.path import abspath, realpath, dirname, basename, join, getsize +import sys +import argparse +import platform +import tarfile +import subprocess +import shutil +import tempfile + +# Get command line options +def args_parse(): + parser = argparse.ArgumentParser( + description='This script is intended to generate ABI dump tarballs\n\n'+ + 'Supported environmental variables:\n'+ + '\t- CC: The required compiler will be determined using this environmental variable.\n', + formatter_class=argparse.RawTextHelpFormatter) + parser.add_argument( + '-t', '--tag', type=str, dest='tag', + help='DPDK tag e.g. latest or v20.11', default='latest') + parser.add_argument( + '-cf', '--cross-file', type=str, dest='crosscompile', + help='Set the location of a cross compile config') + parser.add_argument( + '-a', '--arch', type=str, dest='arch', + help='Architecture arm or x86_64', default=platform.machine()) + args = parser.parse_args() + return args + +# Function to execute git commands +def call_git(args): + args = list(filter(None, args)) + git_call = subprocess.run(['git'] + args, capture_output=True) + if git_call.returncode != 0: + print('ERROR Git returned an error', file=sys.stderr) + exit(1) + return git_call.stdout.decode('utf-8').strip() + +# Function to execute commands +def call_exec(args): + args = list(filter(None, args)) + exec_call = subprocess.run(args, stdout=subprocess.DEVNULL) + if exec_call.returncode != 0: + print('ERROR Script returned an error', file=sys.stderr) + exit(1) + +# Get the required git tag +def get_tag(tag): + tags = call_git(['ls-remote', '--tags', 'http://dpdk.org/git/dpdk']).split('\n') + tags = [t.split('/')[-1].strip() for t in tags if 'rc' not in t and not t.endswith('{}')] + if tag == 'latest': + tag = tags[-1] + if tag not in tags: + print('ERROR supplied tag does not exist in DPDK repo', file=sys.stderr) + exit(1) + return tag + +def main(): + args = args_parse() + + # Get the cross-compile option + cross_comp_meson = [None, None] + if args.crosscompile: + cross_comp_meson = ['--cross-file', abspath(args.crosscompile)] + + tag = get_tag(args.tag) + + # Get the specified compiler from system + if 'CC' in os.environ: + comp = os.environ['CC'] + else: + print('No compiler specified in environmental varibles, setting CC=gcc') + comp = 'gcc' + os.environ['CC'] = 'gcc' + + # Print the configuration to the user + print('\nSelected Build: {}, Compiler: {}, Architecture: {}, Cross Compile: {}'.format(tag,comp,args.arch,cross_comp_meson[1])) + + # Store the users working directory + baseDir = os.getcwd() + # Store devtools dir + devtoolsDir = abspath(dirname(realpath(sys.argv[0]))) + + # Create directory for DPDK git repo and build + tmpDir = tempfile.TemporaryDirectory(dir = "/tmp") + + os.chdir(tmpDir.name) + # Clone DPDK and switch to specified tag + print('Cloning {} from DPDK git'.format(tag)) + call_git(['clone', '--quiet', 'http://dpdk.org/git/dpdk']) + os.chdir('dpdk') + call_git(['checkout', '--quiet', tag]) + + # Create build folder with meson and set debug build and cross compile (if needed) + print('Configuring Meson') + call_exec(['meson', '-Dbuildtype=debug', 'dumpbuild'] + cross_comp_meson) + #os.system('meson -Dbuildtype=debug dumpbuild {} >/dev/null'.format(cross_comp_meson)) + print('Building DPDK . . .') + #Build DPDK with ninja + call_exec(['ninja', '-C', 'dumpbuild']) + + # Create dump files and output to dump directory + dumpDir = join(baseDir,'{}-{}-{}-abi_dump'.format(tag,comp,args.arch)) + print('Generating ABI dump files') + call_exec([join(devtoolsDir,'gen-abi.sh'), 'dumpbuild']) + try: + shutil.copytree('dumpbuild/dump', dumpDir) + except FileExistsError as error: + print('ERROR The {} directory already exists, ensure it is not present before running script'.format(dumpDir), file=sys.stderr) + tmpDir.cleanup() + exit(1) + + # Compress the dump directory + print('Creating Tarball of dump files') + os.chdir(baseDir) + origSize = 0 + for f in os.scandir(dumpDir): + origSize += getsize(f) + with tarfile.open('{}.tar.gz'.format(dumpDir), "w:gz") as tar: + tar.add(dumpDir, arcname=basename(dumpDir)) + newSize = getsize('{}.tar.gz'.format(dumpDir)) + + # Remove all temporary directories + print('Cleaning up temporary directories') + shutil.rmtree(dumpDir) + tmpDir.cleanup() + + #Print output of the script to the user + print('\nDump of DPDK ABI {} is available in {}.tar.gz (Original Size: {:.1f}MB, Compressed Size: {:.1f}MB)\n'.format(tag,dumpDir.split('/')[-1],float(origSize)*1e-6,float(newSize)*1e-6)) + +if __name__ == "__main__": + main() From patchwork Fri Sep 18 12:11:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 78102 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8480DA04C8; Fri, 18 Sep 2020 14:12:16 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 50DEE1DA22; Fri, 18 Sep 2020 14:12:03 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id EB6C61DA15 for ; Fri, 18 Sep 2020 14:11:55 +0200 (CEST) IronPort-SDR: ZQt0pmg1+u775nrUAqSrWNuloefXfKzbl29gHFsamUXl5Ie0lHRMv8YQyAOPJa5NJoN0TEWZbj atmAdK0W8qtw== X-IronPort-AV: E=McAfee;i="6000,8403,9747"; a="160842573" X-IronPort-AV: E=Sophos;i="5.77,274,1596524400"; d="scan'208";a="160842573" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2020 05:11:55 -0700 IronPort-SDR: f0Wa+/dURiKhwigqAXnyhGRFf7aI+DFCOsvVvknMhHueojZt+vaj2noMh8A+ZR+Gry8+N6NbAK 19Z5jknpYs9A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,274,1596524400"; d="scan'208";a="410271223" Received: from silpixa00400466.ir.intel.com ([10.237.213.195]) by fmsmga001.fm.intel.com with ESMTP; 18 Sep 2020 05:11:53 -0700 From: Conor Walsh To: dev@dpdk.org Cc: david.marchand@redhat.com, ray.kinsella@intel.com, nhorman@tuxdriver.com, aconole@redhat.com, maicolgabriel@hotmail.com, thomas@monjalon.net, bruce.richardson@intel.com, anatoly.burakov@intel.com, Conor Walsh Date: Fri, 18 Sep 2020 12:11:36 +0000 Message-Id: <20200918121137.1370883-4-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918121137.1370883-1-conor.walsh@intel.com> References: <20200911160332.256343-1-conor.walsh@intel.com> <20200918121137.1370883-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 3/4] buildtools: add script to setup abi checks for meson X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds a script that is intended to be invoked by meson to do the required setup for performing ABI breakage checks at build time. The required ABI dump archives can come from several sources including being generated at build time or prebuilt archives can be pulled from a remote http location or local directory. Invoke using "./abi-setup.py [-t ] -d " - : dpdk tag e.g. "v20.11", default: latest - : path to dpdk source directory, required E.g. "./abi-setup.py -t v20.08 -d /root/dpdk" As this script is intended to be run by meson during a build some options can be specified by environmental variables: - DPDK_ABI_DUMPS_PATH: Can be used to specify a custom directory for the systems dump directories. - CC: The required compiler will be determined using this environmental variable - DPDK_ABI_TAR_URI: Can be used to specify a location that the script can pull prebuilt or cached dump archives from. This can be a remote http location or a local directory After the script has setup an appropriate ABI dump directory using one of the multiple methods available to it, it will print the location of this directory to the command line. Signed-off-by: Conor Walsh --- buildtools/abi-setup.py | 141 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100755 buildtools/abi-setup.py diff --git a/buildtools/abi-setup.py b/buildtools/abi-setup.py new file mode 100755 index 000000000..39616d21f --- /dev/null +++ b/buildtools/abi-setup.py @@ -0,0 +1,141 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +""" +This script is intended to be invoked by meson to do the required setup +for performing ABI breakage checks at build time. +The required ABI dump archives can come from several sources including +being generated at build time or prebuilt archives can be pulled from a +remote http location or local directory. +""" + +import sys +import os +from os.path import abspath, join, exists, isfile +import argparse +import platform +import subprocess +import requests +import tarfile +import shutil + +# Get command line options +def args_parse(): + # Get command line arguments + parser = argparse.ArgumentParser( + description='This script is intended to setup ABI dumps for meson to perform ABI checks\n'+ + 'Supported environmental variables\n'+ + '\t- DPDK_ABI_DUMPS_PATH: Can be used to specify a custom directory for the systems dump directories.\n'+ + '\t- CC: The required compiler will be determined using this environmental variable.\n'+ + '\t- DPDK_ABI_TAR_URI: Can be used to specify a location that the script can pull prebuilt or cached dump archives from. This can be a remote http location or a local directory.\n', + formatter_class=argparse.RawTextHelpFormatter) + parser.add_argument( + '-t', '--tag', dest='tag', type=str, + help='DPDK tag e.g. latest or v20.11', default='latest') + parser.add_argument( + '-d', '--dpdk', dest='dpdk', type=str, + help='Path to DPDK source directory', required=True) + args = parser.parse_args() + return args + +# Function to execute git commands +def call_git(args): + args = list(filter(None, args)) + git_call = subprocess.run(['git'] + args, capture_output=True) + if git_call.returncode != 0: + print('ERROR Git returned an error', file=sys.stderr) + exit(1) + return git_call.stdout.decode('utf-8') + +# Function to execute commands +def call_exec(args): + args = list(filter(None, args)) + exec_call = subprocess.run(args, stdout=subprocess.DEVNULL) + if exec_call.returncode != 0: + print('ERROR Script returned an error', file=sys.stderr) + exit(1) + +# Get required git tag +def get_tag(tag): + tags = call_git(['ls-remote', '--tags', 'http://dpdk.org/git/dpdk']).split('\n') + tags = [t.split('/')[-1].strip() for t in tags if 'rc' not in t and not t.endswith('{}') and t != ''] + if tag == 'latest': + tag = tags[-1] + if tag not in tags: + print('ERROR supplied tag does not exist in DPDK repo', file=sys.stderr) + exit(1) + return tag + +def main(): + args = args_parse() + + tag = get_tag(args.tag) + + # Get the specified compiler from system + if 'CC' in os.environ: + comp = os.environ['CC'] + else: + comp = 'gcc' + + # Get the systems architecture + arch = platform.machine() + + # Get devtools path + devtools_path = abspath(join(args.dpdk,'devtools')) + + # Get the abi dumps folder from args or env fail if none supplied + abi_folder = '' + abi_env = 'DPDK_ABI_DUMPS_PATH' + if abi_env in os.environ: + abi_folder = abspath(os.environ[abi_env]) + else: + abi_folder = abspath(join(args.dpdk,'abi_dumps')) + + # If the directory doesn't exist create it and add a README to explain what it does + if not exists(abi_folder): + os.makedirs(abi_folder) + f=open(abi_folder+'/README','w+') + f.write('This directory has been setup to contain the ABI dump folders needed to perform ABI checks\n') + f.write('Directories here must be in the format {DPDK Tag}-{Compiler ID}-{Architecture}-abi_dump\n') + f.write('e.g. v20.11-gcc-x86_64-abi_dump\n') + f.write('Directories that do not use this format will not be picked up by the meson ABI checks\n') + f.write('This directory is managed automatically unless desired by the user\n') + f.close() + + # Move to abi folder + os.chdir(abi_folder) + abi_dump=tag+'-'+comp+'-'+arch+'-abi_dump' + # Download and untar abi dump if not present + if not exists(abi_dump): + # Check DPDK_ABI_TAR_URI for the location of the tarballs local or web + tar_uri_env = 'DPDK_ABI_TAR_URI' + if tar_uri_env in os.environ: + abi_tar_uri = os.environ[tar_uri_env] + if abi_tar_uri.startswith('http'): + # Download the required tarball + tar_loc = '{}.tar.gz'.format(join(abi_tar_uri,abi_dump)) + r = requests.get(tar_loc) + if r.status_code == 200: + with open('{}.tar.gz'.format(abi_dump), 'wb') as f: + f.write(r.content) + else: + abi_tar_uri = abspath(abi_tar_uri) + try: + shutil.copy('{}.tar.gz'.format(join(abi_tar_uri,abi_dump)), '.') + except FileNotFoundError as error: + pass + if not isfile(abi_dump+'.tar.gz'): + call_exec([join(devtools_path,'gen-abi-tarball.py'), '-t', tag, '-a', arch]) + if not isfile(abi_dump+'.tar.gz'): + print('ERROR ABI check generation failed', file=sys.stderr) + exit(1) + f = tarfile.open('{}.tar.gz'.format(abi_dump)) + f.extractall() + os.remove('{}.tar.gz'.format(abi_dump)) + + # Tell user where specified directory is + print(abspath(abi_dump)) + +if __name__ == "__main__": + main() From patchwork Fri Sep 18 12:11:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 78103 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7137CA04C8; Fri, 18 Sep 2020 14:12:25 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 80CDF1DA06; Fri, 18 Sep 2020 14:12:05 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 59FBE1DA26 for ; Fri, 18 Sep 2020 14:11:58 +0200 (CEST) IronPort-SDR: i9RgfHcgUtYQQGCntipUIDBjMV4+TpICN+90P2zFXC0+bSQ9D2D/qdCqZjtaBhfY/vsT84R5cL cb1qkDkJuSZQ== X-IronPort-AV: E=McAfee;i="6000,8403,9747"; a="160842584" X-IronPort-AV: E=Sophos;i="5.77,274,1596524400"; d="scan'208";a="160842584" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2020 05:11:57 -0700 IronPort-SDR: fAJ33JgRviig61gtv+c8ZFXJ8SzYccIhdmlfa85sCK+IScMjT05D1487893yGHFAxmekSywXLU AT7ofb4A6dfQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,274,1596524400"; d="scan'208";a="410271235" Received: from silpixa00400466.ir.intel.com ([10.237.213.195]) by fmsmga001.fm.intel.com with ESMTP; 18 Sep 2020 05:11:55 -0700 From: Conor Walsh To: dev@dpdk.org Cc: david.marchand@redhat.com, ray.kinsella@intel.com, nhorman@tuxdriver.com, aconole@redhat.com, maicolgabriel@hotmail.com, thomas@monjalon.net, bruce.richardson@intel.com, anatoly.burakov@intel.com, Conor Walsh Date: Fri, 18 Sep 2020 12:11:37 +0000 Message-Id: <20200918121137.1370883-5-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918121137.1370883-1-conor.walsh@intel.com> References: <20200911160332.256343-1-conor.walsh@intel.com> <20200918121137.1370883-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 4/4] build: add abi breakage checks to meson X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds the ability to run ABI breakage checks to meson. To do this the developer needs to set the meson build type to debug and set the version of DPDK that they want to check the ABI against. The option "abi_checks" has been added to meson for this, the option accepts DPDK tags e.g. "latest" or "v20.11". Example meson command: "meson -Dbuildtype=debug -Dabi_checks=v20.08 build" When the build is done using ninja the ABI checks will be performed if any breakages are present the build will fail. Signed-off-by: Conor Walsh --- buildtools/meson.build | 18 ++++++++++++++++++ config/meson.build | 15 +++++++++++++++ drivers/meson.build | 14 ++++++++++++++ lib/meson.build | 14 ++++++++++++++ meson_options.txt | 2 ++ 5 files changed, 63 insertions(+) diff --git a/buildtools/meson.build b/buildtools/meson.build index 04808dabc..c3ee69a44 100644 --- a/buildtools/meson.build +++ b/buildtools/meson.build @@ -17,3 +17,21 @@ else endif map_to_win_cmd = py3 + files('map_to_win.py') sphinx_wrapper = py3 + files('call-sphinx-build.py') + +abi_check_version = get_option('abi_checks') +check_abi = (abi_check_version != '') + +abi_dir = '' +# If abi checks enabled setup abi dump directory +if check_abi + message('ABI checks are being setup. This could take several minutes') + setup_run = run_command('abi-setup.py', '-t', abi_check_version, '-d', meson.source_root()) + # Check if error returned from script + if setup_run.returncode() != 0 + setup_err = setup_run.stderr().strip() + error('ABI checks setup script returned an error: ' + setup_err) + endif + abi_dir = setup_run.stdout().strip() +endif +abidiff = find_program('abidiff', required: check_abi) +abignore = files('../devtools/libabigail.abignore') diff --git a/config/meson.build b/config/meson.build index 6996e5cbe..79b90f2cc 100644 --- a/config/meson.build +++ b/config/meson.build @@ -47,6 +47,21 @@ else dpdk_conf.set('RTE_VER_RELEASE', 99) endif +# abi checks cannot be run on windows +if is_windows and check_abi + error('ABI checks cannot be run on windows') +endif +# abi checks can only be run on a debug build +# meson <0.48 does not support get_option('debug') +if meson.version().version_compare('>=0.48') + debug_enabled = get_option('debug') +else + debug_enabled = get_option('buildtype').startswith('debug') +endif +if check_abi and not debug_enabled + error('Build type must have debug symbols when abi_checks are enabled') +endif + pmd_subdir_opt = get_option('drivers_install_subdir') if pmd_subdir_opt.contains('') pmd_subdir_opt = abi_version.join(pmd_subdir_opt.split('')) diff --git a/drivers/meson.build b/drivers/meson.build index 5f9526557..b24a875da 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -208,6 +208,20 @@ foreach subpath:subdirs include_directories: includes, dependencies: static_deps) + # If abidiff found, abi checks are enabled and the abi dump files for the library are available run abi check + dump_name = abi_dir+'/lib' + lib_name + '.dump' + if check_abi and run_command('[', '-f', dump_name, ']').returncode() == 0 + custom_target('lib' + lib_name + '.abi_chk', + command: [abidiff, '--no-added-syms', + '--suppr', abignore, + '@INPUT@'], + input: [dump_name, shared_lib], + output: 'lib' + lib_name + '.abi_chk', + capture: true, + install: false, + build_by_default: true) + endif + dpdk_drivers += static_lib set_variable('shared_@0@'.format(lib_name), shared_dep) diff --git a/lib/meson.build b/lib/meson.build index 3852c0156..0d9325b90 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -190,6 +190,20 @@ foreach l:libraries include_directories: includes, dependencies: shared_deps) + # If abidiff found, abi checks are enabled and the abi dump files for the library are available run abi check + dump_name = abi_dir+'/' + dir_name + '.dump' + if check_abi and run_command('[', '-f', dump_name, ']').returncode() == 0 + custom_target(dir_name + '.abi_chk', + command: [abidiff, '--no-added-syms', + '--suppr', abignore, + '@INPUT@'], + input: [dump_name, shared_lib], + output: dir_name + '.abi_chk', + capture: true, + install: false, + build_by_default: true) + endif + dpdk_libraries = [shared_lib] + dpdk_libraries dpdk_static_libraries = [static_lib] + dpdk_static_libraries endif # sources.length() > 0 diff --git a/meson_options.txt b/meson_options.txt index 9bf18ab6b..26ac48f45 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -1,5 +1,7 @@ # Please keep these options sorted alphabetically. +option('abi_checks', type: 'string', value: '', + description: 'Enable abi compatibility checks to run during the build. This requires debug build to be enabled. Input is latest or git tag e.g. v20.11') option('armv8_crypto_dir', type: 'string', value: '', description: 'path to the armv8_crypto library installation directory') option('disable_drivers', type: 'string', value: '',