[v7,02/10] buildtools: add script for updating symbols abi version

Message ID 1c066505836ad4cde9e4935881c28a03d8889126.1573230233.git.anatoly.burakov@intel.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series Implement the new ABI policy and add helper scripts |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Burakov, Anatoly Nov. 8, 2019, 4:25 p.m. UTC
  From: Pawel Modrak <pawelx.modrak@intel.com>

Add a script that automatically merges all stable ABI's under one
ABI section with the new version, while leaving experimental
section exactly as it is.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---

Notes:
    v7:
    - Do not remove stable ABI if it was empty
    
    v6:
    - Split map file generation function in two
    - Do not print stable ABI if it wasn't present
    
    v3:
    - Add comments to regex patterns
    
    v2:
    - Reworked script to be pep8-compliant and more reliable

 buildtools/update_version_map_abi.py | 173 +++++++++++++++++++++++++++
 1 file changed, 173 insertions(+)
 create mode 100755 buildtools/update_version_map_abi.py
  

Comments

Thomas Monjalon Nov. 19, 2019, 2:01 p.m. UTC | #1
08/11/2019 17:25, Anatoly Burakov:
> From: Pawel Modrak <pawelx.modrak@intel.com>
> 
> Add a script that automatically merges all stable ABI's under one
> ABI section with the new version, while leaving experimental
> section exactly as it is.
> 
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> --- /dev/null
> +++ b/buildtools/update_version_map_abi.py
> +"""
> +A Python program to update the ABI version and function names in a DPDK
> +lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
> +"""

Clearly this script is doing more than updating a version number and names.
I already see more in the commit log.
Please would you like to add something in this description?
  
Burakov, Anatoly Nov. 19, 2019, 3:38 p.m. UTC | #2
On 19-Nov-19 2:01 PM, Thomas Monjalon wrote:
> 08/11/2019 17:25, Anatoly Burakov:
>> From: Pawel Modrak <pawelx.modrak@intel.com>
>>
>> Add a script that automatically merges all stable ABI's under one
>> ABI section with the new version, while leaving experimental
>> section exactly as it is.
>>
>> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
>> ---
>> --- /dev/null
>> +++ b/buildtools/update_version_map_abi.py
>> +"""
>> +A Python program to update the ABI version and function names in a DPDK
>> +lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
>> +"""
> 
> Clearly this script is doing more than updating a version number and names.
> I already see more in the commit log.
> Please would you like to add something in this description?
> 
> 

Not sure what you're referring to. That's exactly what it's doing. What 
else is there that i'm missing?
  
Thomas Monjalon Nov. 19, 2019, 4:05 p.m. UTC | #3
19/11/2019 16:38, Burakov, Anatoly:
> On 19-Nov-19 2:01 PM, Thomas Monjalon wrote:
> > 08/11/2019 17:25, Anatoly Burakov:
> >> From: Pawel Modrak <pawelx.modrak@intel.com>
> >>
> >> Add a script that automatically merges all stable ABI's under one
> >> ABI section with the new version, while leaving experimental
> >> section exactly as it is.
> >>
> >> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> >> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> >> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> >> ---
> >> --- /dev/null
> >> +++ b/buildtools/update_version_map_abi.py
> >> +"""
> >> +A Python program to update the ABI version and function names in a DPDK
> >> +lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
> >> +"""
> > 
> > Clearly this script is doing more than updating a version number and names.
> > I already see more in the commit log.
> > Please would you like to add something in this description?
> 
> Not sure what you're referring to. That's exactly what it's doing. What 
> else is there that i'm missing?

"merges all stable ABI's under one ABI section"
"while leaving experimental section exactly as it is"

I think this needs to be in the comment at the top of the file,
so we have an introduction for the code in the rest of the file.
  

Patch

diff --git a/buildtools/update_version_map_abi.py b/buildtools/update_version_map_abi.py
new file mode 100755
index 0000000000..0f6196140e
--- /dev/null
+++ b/buildtools/update_version_map_abi.py
@@ -0,0 +1,173 @@ 
+#!/usr/bin/env python
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+"""
+A Python program to update the ABI version and function names in a DPDK
+lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
+"""
+
+from __future__ import print_function
+import argparse
+import sys
+import re
+
+
+def __parse_map_file(f_in):
+    # match function name, followed by semicolon, followed by EOL, optionally
+    # with whitespace inbetween each item
+    func_line_regex = re.compile(r"\s*"
+                                 r"(?P<func>[a-zA-Z_0-9]+)"
+                                 r"\s*"
+                                 r";"
+                                 r"\s*"
+                                 r"$")
+    # match section name, followed by opening bracked, followed by EOL,
+    # optionally with whitespace inbetween each item
+    section_begin_regex = re.compile(r"\s*"
+                                     r"(?P<version>[a-zA-Z0-9_\.]+)"
+                                     r"\s*"
+                                     r"{"
+                                     r"\s*"
+                                     r"$")
+    # match closing bracket, optionally followed by section name (for when we
+    # inherit from another ABI version), followed by semicolon, followed by
+    # EOL, optionally with whitespace inbetween each item
+    section_end_regex = re.compile(r"\s*"
+                                   r"}"
+                                   r"\s*"
+                                   r"(?P<parent>[a-zA-Z0-9_\.]+)?"
+                                   r"\s*"
+                                   r";"
+                                   r"\s*"
+                                   r"$")
+
+    # for stable ABI, we don't care about which version introduced which
+    # function, we just flatten the list. there are dupes in certain files, so
+    # use a set instead of a list
+    stable_lines = set()
+    # copy experimental section as is
+    experimental_lines = []
+    in_experimental = False
+    has_stable = False
+
+    # gather all functions
+    for line in f_in:
+        # clean up the line
+        line = line.strip('\n').strip()
+
+        # is this an end of section?
+        match = section_end_regex.match(line)
+        if match:
+            # whatever section this was, it's not active any more
+            in_experimental = False
+            continue
+
+        # if we're in the middle of experimental section, we need to copy
+        # the section verbatim, so just add the line
+        if in_experimental:
+            experimental_lines += [line]
+            continue
+
+        # skip empty lines
+        if not line:
+            continue
+
+        # is this a beginning of a new section?
+        match = section_begin_regex.match(line)
+        if match:
+            cur_section = match.group("version")
+            # is it experimental?
+            in_experimental = cur_section == "EXPERIMENTAL"
+            if not in_experimental:
+                has_stable = True
+            continue
+
+        # is this a function?
+        match = func_line_regex.match(line)
+        if match:
+            stable_lines.add(match.group("func"))
+
+    return has_stable, stable_lines, experimental_lines
+
+
+def __generate_stable_abi(f_out, abi_version, lines):
+    # print ABI version header
+    print("DPDK_{} {{".format(abi_version), file=f_out)
+
+    # print global section if it exists
+    if lines:
+        print("\tglobal:", file=f_out)
+        # blank line
+        print(file=f_out)
+
+        # print all stable lines, alphabetically sorted
+        for line in sorted(lines):
+            print("\t{};".format(line), file=f_out)
+
+        # another blank line
+        print(file=f_out)
+
+    # print local section
+    print("\tlocal: *;", file=f_out)
+
+    # end stable version
+    print("};", file=f_out)
+
+
+def __generate_experimental_abi(f_out, lines):
+    # start experimental section
+    print("EXPERIMENTAL {", file=f_out)
+
+    # print all experimental lines as they were
+    for line in lines:
+        # don't print empty whitespace
+        if not line:
+            print("", file=f_out)
+        else:
+            print("\t{}".format(line), file=f_out)
+
+    # end section
+    print("};", file=f_out)
+
+
+def __main():
+    arg_parser = argparse.ArgumentParser(
+        description='Merge versions in linker version script.')
+
+    arg_parser.add_argument("map_file", type=str,
+                            help='path to linker version script file '
+                                 '(pattern: *version.map)')
+    arg_parser.add_argument("abi_version", type=str,
+                            help='target ABI version (pattern: MAJOR.MINOR)')
+
+    parsed = arg_parser.parse_args()
+
+    if not parsed.map_file.endswith('version.map'):
+        print("Invalid input file: {}".format(parsed.map_file),
+              file=sys.stderr)
+        arg_parser.print_help()
+        sys.exit(1)
+
+    if not re.match(r"\d{1,2}\.\d{1,2}", parsed.abi_version):
+        print("Invalid ABI version: {}".format(parsed.abi_version),
+              file=sys.stderr)
+        arg_parser.print_help()
+        sys.exit(1)
+
+    with open(parsed.map_file) as f_in:
+        has_stable, stable_lines, experimental_lines = __parse_map_file(f_in)
+
+    with open(parsed.map_file, 'w') as f_out:
+        need_newline = has_stable and experimental_lines
+        if has_stable:
+            __generate_stable_abi(f_out, parsed.abi_version, stable_lines)
+        if need_newline:
+            # separate sections with a newline
+            print(file=f_out)
+        if experimental_lines:
+            __generate_experimental_abi(f_out, experimental_lines)
+
+
+if __name__ == "__main__":
+    __main()