[dpdk-dev] checkpatches.sh: Add checks for ABI symbol addition
Checks
Commit Message
Recently, some additional patches were added to allow for programmatic
marking of C symbols as experimental. The addition of these markers is
dependent on the manual addition of exported symbols to the EXPERIMENTAL
section of the corresponding libraries version map file. The consensus
on review is that, in addition to mandating the addition of symbols to
the EXPERIMENTAL version in the map, we need a mechanism to enforce our
documented process of mandating that addition when they are introduced.
To that end, I am proposing this change. It is an addition to the
checkpatches script, which scan incoming patches for additions and
removals of symbols to the map file, and warns the user appropriately
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: thomas@monjalon.net
CC: john.mcnamara@intel.com
CC: bruce.richardson@intel.com
CC: Ferruh Yigit <ferruh.yigit@intel.com>
---
devtools/checkpatches.sh | 89 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 88 insertions(+), 1 deletion(-)
Comments
15/01/2018 20:05, Neil Horman:
> Recently, some additional patches were added to allow for programmatic
> marking of C symbols as experimental. The addition of these markers is
> dependent on the manual addition of exported symbols to the EXPERIMENTAL
> section of the corresponding libraries version map file. The consensus
> on review is that, in addition to mandating the addition of symbols to
> the EXPERIMENTAL version in the map, we need a mechanism to enforce our
> documented process of mandating that addition when they are introduced.
> To that end, I am proposing this change. It is an addition to the
> checkpatches script, which scan incoming patches for additions and
> removals of symbols to the map file, and warns the user appropriately
Thanks for working on this.
I won't pretend that I understand anything in this awk script :)
I think it would be better to put this code in a new script,
let's say check-symbol-change.sh, and call it in checkpatches.sh.
It would be just moving functions, add your copyright, and list the new
script in your MAINTAINERS section "ABI versioning".
On Mon, 15 Jan 2018 14:05:45 -0500
Neil Horman <nhorman@tuxdriver.com> wrote:
>
> +build_map_changes()
> +{
> + local fname=$1
> + local mapdb=$2
> +
> + cat $fname | filterdiff -i *.map | awk '
You don't need cat, use shell redirection same for later while loop.
> + BEGIN {map="";sym="";ar="";sec=""; in_sec=0}
> + /[-+] a\/.*\.map/ {map=$2}
> + /+.*{/ {gsub("+","");sec=$1; in_sec=1}
Add some whitespace and indentation to awk?
> + /.*}/ {in_sec=0}
> + /^+.*[^:*];/ {gsub(";","");sym=$2;
> + if (in_sec == 1) {
> + print map " " sym " " sec " add"
> + }
> + }
> + /^-.*[^:*];/ {gsub(";","");sym=$2;
> + if (in_sec == 1) {
> + print map " " sym " " sec " del"
> + }
> + }' > ./$mapdb
> +
> + sort $mapdb | uniq > ./$mapdb.2
sort -u
> + mv -f $mapdb.2 $mapdb
> +
> +}
> +
On Mon, Jan 15, 2018 at 02:20:38PM -0800, Stephen Hemminger wrote:
> On Mon, 15 Jan 2018 14:05:45 -0500
> Neil Horman <nhorman@tuxdriver.com> wrote:
>
> >
> > +build_map_changes()
> > +{
> > + local fname=$1
> > + local mapdb=$2
> > +
> > + cat $fname | filterdiff -i *.map | awk '
>
> You don't need cat, use shell redirection same for later while loop.
>
This is likely moot given Thomas' request to put this in a separate script, but
point taken
> > + BEGIN {map="";sym="";ar="";sec=""; in_sec=0}
> > + /[-+] a\/.*\.map/ {map=$2}
> > + /+.*{/ {gsub("+","");sec=$1; in_sec=1}
> Add some whitespace and indentation to awk?
Yeah, I can document this block as a whole as well
>
> > + /.*}/ {in_sec=0}
> > + /^+.*[^:*];/ {gsub(";","");sym=$2;
> > + if (in_sec == 1) {
> > + print map " " sym " " sec " add"
> > + }
> > + }
> > + /^-.*[^:*];/ {gsub(";","");sym=$2;
> > + if (in_sec == 1) {
> > + print map " " sym " " sec " del"
> > + }
> > + }' > ./$mapdb
> > +
> > + sort $mapdb | uniq > ./$mapdb.2
>
> sort -u
Copy that.
>
> > + mv -f $mapdb.2 $mapdb
> > +
> > +}
> > +
>
On Mon, Jan 15, 2018 at 10:52:25PM +0100, Thomas Monjalon wrote:
> 15/01/2018 20:05, Neil Horman:
> > Recently, some additional patches were added to allow for programmatic
> > marking of C symbols as experimental. The addition of these markers is
> > dependent on the manual addition of exported symbols to the EXPERIMENTAL
> > section of the corresponding libraries version map file. The consensus
> > on review is that, in addition to mandating the addition of symbols to
> > the EXPERIMENTAL version in the map, we need a mechanism to enforce our
> > documented process of mandating that addition when they are introduced.
> > To that end, I am proposing this change. It is an addition to the
> > checkpatches script, which scan incoming patches for additions and
> > removals of symbols to the map file, and warns the user appropriately
>
> Thanks for working on this.
Sure.
> I won't pretend that I understand anything in this awk script :)
>
Stephen suggested that I clean it up and document it a bit, which is probably a good idea.
> I think it would be better to put this code in a new script,
> let's say check-symbol-change.sh, and call it in checkpatches.sh.
> It would be just moving functions, add your copyright, and list the new
> script in your MAINTAINERS section "ABI versioning".
Yeah, I can do that. New version in the AM
Neil
>
@@ -61,6 +61,77 @@ print_usage () {
END_OF_HELP
}
+build_map_changes()
+{
+ local fname=$1
+ local mapdb=$2
+
+ cat $fname | filterdiff -i *.map | awk '
+ BEGIN {map="";sym="";ar="";sec=""; in_sec=0}
+ /[-+] a\/.*\.map/ {map=$2}
+ /+.*{/ {gsub("+","");sec=$1; in_sec=1}
+ /.*}/ {in_sec=0}
+ /^+.*[^:*];/ {gsub(";","");sym=$2;
+ if (in_sec == 1) {
+ print map " " sym " " sec " add"
+ }
+ }
+ /^-.*[^:*];/ {gsub(";","");sym=$2;
+ if (in_sec == 1) {
+ print map " " sym " " sec " del"
+ }
+ }' > ./$mapdb
+
+ sort $mapdb | uniq > ./$mapdb.2
+ mv -f $mapdb.2 $mapdb
+
+}
+
+check_for_rule_violations()
+{
+ local mapdb=$1
+ local mname
+ local symname
+ local secname
+ local ar
+
+ cat $mapdb | while read mname symname secname ar
+ do
+ if [ "$ar" == "add" -a "$secname" != "EXPERIMENTAL" ]
+ then
+ # Symbols that are getting added in a section other
+ # Than the experimental section need to be moving
+ # from an already supported section or its a violation
+ grep -q "$mname $symname [^EXPERIMENTAL] del" $mapdb
+ if [ $? -ne 0 ]
+ then
+ echo "ERROR: symbol $symname is added in a section other than the EXPERIMENTAL section of the version map"
+ fi
+ fi
+
+ if [ "$ar" == "del" -a "$secname" != "EXPERIMENTAL" ]
+ then
+ # Just inform users that non-experimenal symbols need
+ # to go through a deprecation process
+ echo "INFO: symbol $symname is being removed, ensure that it has gone through the deprecation process"
+ fi
+
+ done
+}
+
+validate_api_changes()
+{
+ mapfile=`mktemp mapdb.XXXXXX`
+ patch=$1
+
+ build_map_changes $patch $mapfile
+ result=$(check_for_rule_violations $mapfile)
+
+ rm -f $mapfile
+
+ echo $result
+}
+
number=0
quiet=false
verbose=false
@@ -96,9 +167,25 @@ check () { # <patch> <commit> <title>
else
report=$($DPDK_CHECKPATCH_PATH $options - 2>/dev/null)
fi
- [ $? -ne 0 ] || return 0
+ reta=$?
+
$verbose || printf '\n### %s\n\n' "$3"
printf '%s\n' "$report" | sed -n '1,/^total:.*lines checked$/p'
+
+ echo
+ echo "Checking API additions/removals:"
+ if [ -n "$1" ] ; then
+ report=$(validate_api_changes $1)
+ elif [ -n "$2" ] ; then
+ report=$(git format-patch \
+ --find-renames --no-stat --stdout -1 $commit |
+ validate_api_changes /dev/stdin)
+ else
+ report=$(validate_api_changes /dev/stdin)
+ fi
+ [ $? -ne 0 -o $reta -ne 0 ] || return 0
+ printf '%s\n' "$report" | sed -n '1,/^total:.*lines checked$/p'
+
status=$(($status + 1))
}