From patchwork Fri Dec 15 06:43:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Dwivedi X-Patchwork-Id: 135220 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C9C5436F8; Fri, 15 Dec 2023 07:44:20 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EE4C9432C6; Fri, 15 Dec 2023 07:44:18 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 04833432A6 for ; Fri, 15 Dec 2023 07:44:17 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BF5EV8A020537; Thu, 14 Dec 2023 22:44:17 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=9OHPL+/HLP84ouTIr8hlV8TWuzRfyEQkcA8aWMZuvnI=; b=Dtc bo3jGD8ULB1caip7I/GQhBv91IoyVqAS0Kn0jsGdGusHopLkcUFY5Em1nWdrvp8h 0xX9EKdVAswzdeElZ+6uUxMCnHeCloczC8CeASQKRuMdDG1TdY272VQWMpCvTTHn T3MdNQJpJai1APMOktnFq1cV3LMo6mJfG9JWzDFawu8W4FMT+sFokIiaBvGA3Tdy O5czxyNZqtzjx4LQROXbqYtevNOf5QLWyjle428FAhCCnQG6xnIIngN4V2RtpREN Wsc0OVSSEoe0/DmNW0JvynhT9xRbTwUi+c9NeQBgHj4oQdvJ4vx2te1+B7NrtCJ8 ECIIj24gsvAEAqalDlA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3v06m0t92x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 14 Dec 2023 22:44:17 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 14 Dec 2023 22:44:15 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 14 Dec 2023 22:44:15 -0800 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id B0F2F3F706F; Thu, 14 Dec 2023 22:44:13 -0800 (PST) From: Ankur Dwivedi To: CC: , , Ankur Dwivedi Subject: [PATCH v6 1/2] devtools: move build map changes function Date: Fri, 15 Dec 2023 12:13:54 +0530 Message-ID: <20231215064355.1429709-2-adwivedi@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231215064355.1429709-1-adwivedi@marvell.com> References: <20230307120514.2774917-1-adwivedi@marvell.com> <20231215064355.1429709-1-adwivedi@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 4kzn9WokSquWuKasTLMxWL8kSaMfUD4W X-Proofpoint-ORIG-GUID: 4kzn9WokSquWuKasTLMxWL8kSaMfUD4W X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch moves the build_map_changes function from check-symbol-change.sh to a new symbol-map-util.sh file. This function can be used in other scripts by including symbol-map-util.sh file. The license in check-symbol-change.sh is copied to symbol-map-util.sh. Signed-off-by: Ankur Dwivedi --- devtools/check-symbol-change.sh | 76 +------------------------------- devtools/symbol-map-util.sh | 78 +++++++++++++++++++++++++++++++++ 2 files changed, 80 insertions(+), 74 deletions(-) create mode 100644 devtools/symbol-map-util.sh diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh index 8992214ac8..68613f63cc 100755 --- a/devtools/check-symbol-change.sh +++ b/devtools/check-symbol-change.sh @@ -2,80 +2,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Neil Horman -build_map_changes() -{ - local fname="$1" - local mapdb="$2" - - cat "$fname" | awk ' - # Initialize our variables - BEGIN {map="";sym="";ar="";sec=""; in_sec=0; in_map=0} - - # Anything that starts with + or -, followed by an a - # and ends in the string .map is the name of our map file - # This may appear multiple times in a patch if multiple - # map files are altered, and all section/symbol names - # appearing between a triggering of this rule and the - # next trigger of this rule are associated with this file - /[-+] [ab]\/.*\.map/ {map=$2; in_map=1; next} - - # The previous rule catches all .map files, anything else - # indicates we left the map chunk. - /[-+] [ab]\// {in_map=0} - - # Triggering this rule, which starts a line and ends it - # with a { identifies a versioned section. The section name is - # the rest of the line with the + and { symbols removed. - # Triggering this rule sets in_sec to 1, which actives the - # symbol rule below - /^.*{/ { - gsub("+", ""); - if (in_map == 1) { - sec=$(NF-1); in_sec=1; - } - } - - # This rule identifies the end of a section, and disables the - # symbol rule - /.*}/ {in_sec=0} - - # This rule matches on a + followed by any characters except a : - # (which denotes a global vs local segment), and ends with a ;. - # The semicolon is removed and the symbol is printed with its - # association file name and version section, along with an - # indicator that the symbol is a new addition. Note this rule - # only works if we have found a version section in the rule - # above (hence the in_sec check) And found a map file (the - # in_map check). If we are not in a map chunk, do nothing. If - # we are in a map chunk but not a section chunk, record it as - # unknown. - /^+[^}].*[^:*];/ {gsub(";","");sym=$2; - if (in_map == 1) { - if (in_sec == 1) { - print map " " sym " " sec " add" - } else { - print map " " sym " unknown add" - } - } - } - - # This is the same rule as above, but the rule matches on a - # leading - rather than a +, denoting that the symbol is being - # removed. - /^-[^}].*[^:*];/ {gsub(";","");sym=$2; - if (in_map == 1) { - if (in_sec == 1) { - print map " " sym " " sec " del" - } else { - print map " " sym " unknown del" - } - } - }' > "$mapdb" - - sort -u "$mapdb" > "$mapdb.2" - mv -f "$mapdb.2" "$mapdb" - -} +selfdir=$(dirname $(readlink -f $0)) +. $selfdir/symbol-map-util.sh is_stable_section() { [ "$1" != 'EXPERIMENTAL' ] && [ "$1" != 'INTERNAL' ] diff --git a/devtools/symbol-map-util.sh b/devtools/symbol-map-util.sh new file mode 100644 index 0000000000..86e4279cdc --- /dev/null +++ b/devtools/symbol-map-util.sh @@ -0,0 +1,78 @@ +#!/bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Neil Horman + +build_map_changes() +{ + local fname="$1" + local mapdb="$2" + + cat "$fname" | awk ' + # Initialize our variables + BEGIN {map="";sym="";ar="";sec=""; in_sec=0; in_map=0} + + # Anything that starts with + or -, followed by an a + # and ends in the string .map is the name of our map file + # This may appear multiple times in a patch if multiple + # map files are altered, and all section/symbol names + # appearing between a triggering of this rule and the + # next trigger of this rule are associated with this file + /[-+] [ab]\/.*\.map/ {map=$2; in_map=1; next} + + # The previous rule catches all .map files, anything else + # indicates we left the map chunk. + /[-+] [ab]\// {in_map=0} + + # Triggering this rule, which starts a line and ends it + # with a { identifies a versioned section. The section name is + # the rest of the line with the + and { symbols removed. + # Triggering this rule sets in_sec to 1, which actives the + # symbol rule below + /^.*{/ { + gsub("+", ""); + if (in_map == 1) { + sec=$(NF-1); in_sec=1; + } + } + + # This rule identifies the end of a section, and disables the + # symbol rule + /.*}/ {in_sec=0} + + # This rule matches on a + followed by any characters except a : + # (which denotes a global vs local segment), and ends with a ;. + # The semicolon is removed and the symbol is printed with its + # association file name and version section, along with an + # indicator that the symbol is a new addition. Note this rule + # only works if we have found a version section in the rule + # above (hence the in_sec check) And found a map file (the + # in_map check). If we are not in a map chunk, do nothing. If + # we are in a map chunk but not a section chunk, record it as + # unknown. + /^+[^}].*[^:*];/ {gsub(";","");sym=$2; + if (in_map == 1) { + if (in_sec == 1) { + print map " " sym " " sec " add" + } else { + print map " " sym " unknown add" + } + } + } + + # This is the same rule as above, but the rule matches on a + # leading - rather than a +, denoting that the symbol is being + # removed. + /^-[^}].*[^:*];/ {gsub(";","");sym=$2; + if (in_map == 1) { + if (in_sec == 1) { + print map " " sym " " sec " del" + } else { + print map " " sym " unknown del" + } + } + }' > "$mapdb" + + sort -u "$mapdb" > "$mapdb.2" + mv -f "$mapdb.2" "$mapdb" + +} From patchwork Fri Dec 15 06:43:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Dwivedi X-Patchwork-Id: 135221 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 399AF436F8; Fri, 15 Dec 2023 07:44:27 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2822A43314; Fri, 15 Dec 2023 07:44:27 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 53B6C4330C for ; Fri, 15 Dec 2023 07:44:25 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BF4rELK019910; Thu, 14 Dec 2023 22:44:24 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=0TUplLQjiK4JbgZ6lt2SxfmgpCHY1gz2hVnkr5PdDzc=; b=Ym1 8LDWfQTDdRY7FBap79YiZ5vaIT7bQfqThAhBR67OHJSqpHm30sFIgbaJVx4eFazd btrVJSEwNNwgz98/G+3bgVHpsL1O24w6MAU8KE3GqkGpQEfGPYIlrtc8aYtq1Z56 Sedrq2W6iiokxgpDFBJqhvgjkPKer6S1InxuaGlNPau/WHcuacnbwQx2Log7RVz2 LGLWLJ7kgUeZ+7ZfX8Q6sg8NyBE3p7840vDpCihPS5xBgTh8q/04C0STtIfSgG/q 1BSNBHQELN+YYV+1p26XGfUsndzpCPv/gfKxXEQgdsrjgoQHhXmrF+GpsdGcJYTM rHz3sYM6fYOyJ8ejLKA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3v06m0t93c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 14 Dec 2023 22:44:24 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 14 Dec 2023 22:44:22 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 14 Dec 2023 22:44:22 -0800 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id 090E43F706C; Thu, 14 Dec 2023 22:44:20 -0800 (PST) From: Ankur Dwivedi To: CC: , , Ankur Dwivedi Subject: [PATCH v6 2/2] devtools: add tracepoint check in checkpatch Date: Fri, 15 Dec 2023 12:13:55 +0530 Message-ID: <20231215064355.1429709-3-adwivedi@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231215064355.1429709-1-adwivedi@marvell.com> References: <20230307120514.2774917-1-adwivedi@marvell.com> <20231215064355.1429709-1-adwivedi@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 3tZY4J03uKl7e9l8ueaUo6VcMnvRmoPU X-Proofpoint-ORIG-GUID: 3tZY4J03uKl7e9l8ueaUo6VcMnvRmoPU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds a validation in checkpatch tool, to check if a tracepoint is present in any new function added in cryptodev, ethdev, eventdev and mempool library. It uses the existing build_map_changes function to create a map of functions. In the map, the newly added functions, added in the experimental section are identified and their definition are checked for the presence of tracepoint. The checkpatch return error if the tracepoint is not present. For functions for which trace is not needed, they can be added to devtools/trace-skiplist.txt file. The above tracepoint check will be skipped for them. Signed-off-by: Ankur Dwivedi --- devtools/check-tracepoint.sh | 148 +++++++++++++++++++++++++++++++++++ devtools/checkpatches.sh | 9 +++ devtools/trace-skiplist.txt | 0 3 files changed, 157 insertions(+) create mode 100755 devtools/check-tracepoint.sh create mode 100644 devtools/trace-skiplist.txt diff --git a/devtools/check-tracepoint.sh b/devtools/check-tracepoint.sh new file mode 100755 index 0000000000..f957ff780c --- /dev/null +++ b/devtools/check-tracepoint.sh @@ -0,0 +1,148 @@ +#!/bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2023 Marvell. + +selfdir=$(dirname $(readlink -f $0)) +. $selfdir/symbol-map-util.sh + +# Library for trace check +libdir="cryptodev ethdev eventdev mempool" + +# Functions for which the trace check can be skipped +skiplist="$selfdir/trace-skiplist.txt" + +find_trace_fn() +{ + local fname="$1" + + cat "$fname" | awk -v symname="$2 *\\\(" ' + # Initialize the variables. + # The variable symname provides a pattern to match + # "function_name(", zero or more spaces can be present + # between function_name and (. + BEGIN {state=0; ln_pth=0} + + # Matches the function name, set state=1. + $0 ~ symname {state=1} + + # If closing parentheses with semicolon ");" is found, then it + # is not the function definition. + /) *;/ { + if (state == 1) { + state=0 + } + } + + /)/ { + if (state == 1) { + state=2 + ln_pth=NR + next + } + } + + # If closing parentheses and then opening braces is found in + # adjacent line, then this is the start of function + # definition. Check for tracepoint in the function definition. + # The tracepoint has _trace_ in its name. + /{/ { + if (state == 2) { + if (ln_pth != NR - 1) { + state=0 + next + } + while (index($0, "}") != 2) { + if (index($0, "_trace_") != 0) { + exit 0 + } + if (getline <= 0) { + break + } + } + exit 1 + } + } + ' +} + +check_for_tracepoint() +{ + local patch="$1" + local mapdb="$2" + local skip_sym + local libname + local secname + local mname + local ret=0 + local pdir + local libp + local libn + local sym + local ar + + while read -r mname symname secname ar; do + pdir=$(echo $mname | awk 'BEGIN {FS="/"};{print $2}') + libname=$(echo $mname | awk 'BEGIN {FS="/"};{print $3}') + skip_sym=0 + libp=0 + + if [ "$pdir" != "lib" ]; then + continue + fi + + for libn in $libdir; do + if [ $libn = $libname ]; then + libp=1 + break + fi + done + + if [ $libp -eq 0 ]; then + continue + fi + + for sym in $(cat $skiplist); do + if [ $sym = $symname ]; then + skip_sym=1 + break + fi + done + + if [ $skip_sym -eq 1 ]; then + continue + fi + + if [ "$ar" = "add" ] && [ "$secname" = "EXPERIMENTAL" ]; then + # Check if new API is added with tracepoint + find_trace_fn $patch $symname + if [ $? -eq 1 ]; then + ret=1 + echo -n "ERROR: New function $symname is added " + echo -n "without a tracepoint. Please add a tracepoint " + echo -n "or add the function $symname in " + echo "devtools/trace-skiplist.txt to skip this error." + fi + fi + done < "$mapdb" + + return $ret +} + +trap clean_and_exit_on_sig EXIT + +mapfile=`mktemp -t dpdk.mapdb.XXXXXX` +patch=$1 +exit_code=1 + +clean_and_exit_on_sig() +{ + rm -f "$mapfile" + exit $exit_code +} + +build_map_changes "$patch" "$mapfile" +check_for_tracepoint "$patch" "$mapfile" +exit_code=$? +rm -f "$mapfile" + +exit $exit_code diff --git a/devtools/checkpatches.sh b/devtools/checkpatches.sh index 10b79ca2bc..d74e541c9f 100755 --- a/devtools/checkpatches.sh +++ b/devtools/checkpatches.sh @@ -10,6 +10,7 @@ . $(dirname $(readlink -f $0))/load-devel-config VALIDATE_NEW_API=$(dirname $(readlink -f $0))/check-symbol-change.sh +VALIDATE_TRACEPOINT=$(dirname $(readlink -f $0))/check-tracepoint.sh # Enable codespell by default. This can be overwritten from a config file. # Codespell can also be enabled by setting DPDK_CHECKPATCH_CODESPELL to a valid path @@ -413,6 +414,14 @@ check () { # ret=1 fi + ! $verbose || printf '\nChecking API additions with tracepoint:\n' + report=$($VALIDATE_TRACEPOINT "$tmpinput") + if [ $? -ne 0 ] ; then + $headline_printed || print_headline "$subject" + printf '%s\n' "$report" + ret=1 + fi + if [ "$tmpinput" != "$1" ]; then rm -f "$tmpinput" trap - INT diff --git a/devtools/trace-skiplist.txt b/devtools/trace-skiplist.txt new file mode 100644 index 0000000000..e69de29bb2