From patchwork Fri Jan 8 02:47:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 86142 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 258EBA0524; Fri, 8 Jan 2021 03:47:43 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DCC58140DD0; Fri, 8 Jan 2021 03:47:36 +0100 (CET) Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by mails.dpdk.org (Postfix) with ESMTP id B8310140DB4 for ; Fri, 8 Jan 2021 03:47:35 +0100 (CET) Received: by mail-lf1-f51.google.com with SMTP id 23so19487784lfg.10 for ; Thu, 07 Jan 2021 18:47:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=th0aCz+hOrNzE/+zodKztsegy9THJ+QqOEZKcblfXmc=; b=FywfHagyuyqpLr4GLJAWHz3BWRihjYRtKuJ99k392IbDPiCHRHPwV+r+KoCJHPxAKE HvL6PjNfM49Nw6mvB6w94Nw29DnA/6brop3rT4lW+VXybFSWiqtGSjO9Wp8sbwi2mmSH fRnmexMXHv15cIIiko2SNjDXZ8b0xBC8i54ObmM5j4Fjqu4tA/lsO+Y8btKO+bHBECXP m6xb2724wk0Odvz9xN1x9a3npPIFfLs0nJM3P4hOll0Exy1yAZ7DHapCWWNYpx976a/0 lMfpMzN3vIAyr7JROGJZiKkyW6PVAlLEvs89aAKkeFOrbDDtsS6dkgMDoS3TpjmZUIqV MrlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=th0aCz+hOrNzE/+zodKztsegy9THJ+QqOEZKcblfXmc=; b=DfJD6YsdW3k/5w28UlNO9TTYyKyhHI4J/1gMfaqj40eJyBKD5Z01IbeXxMbJijj5cQ dg7uOAJ7iu96jasUpkFe5GDVJpDb+YtsV/pa5HE8SPO0GCWAM3Nejvvc3zrQ9LrdP8wz 93ZPrg4GaxWz5j2nGCAWB2h4Vuq0Wz43ZJuV2bGAe08mCj5HPEJpMHK8eyPLa+FQb6qi cDUyww46X47oHWLeagZBoVqc5oEUwR6Iem3F/Ubc4sgIP1yD3ypRSYhXBC4agVN0DTXO bxPjfUUkHhVN6stG/FrP2MZuTTkqegld7nK7ZowgU66hpWHR9xVtFk4G72MmJOmSteTM JD3w== X-Gm-Message-State: AOAM532XqXDInoeBUQV2yw56vM9hbVZeglPKVSix9z8L1OpfhF+3Yvmv WXQIQuoKG/jhJ9TV62+rq4dSvO3wzF+Xs/xi X-Google-Smtp-Source: ABdhPJx4JbYHWaXlOdE3JQHBnM5qcfMulkcxvs7U1tu6YkdZrT4Pt+W+jhARlii6ZSmWIkhTLt7PLA== X-Received: by 2002:a2e:b4d3:: with SMTP id r19mr543720ljm.419.1610074054961; Thu, 07 Jan 2021 18:47:34 -0800 (PST) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id i19sm1733599ljj.26.2021.01.07.18.47.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Jan 2021 18:47:34 -0800 (PST) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Narcisa Ana Maria Vasile , Pallavi Kadam , Dmitry Malloy , Bruce Richardson , Neil Horman , Dmitry Kozlyuk Date: Fri, 8 Jan 2021 05:47:20 +0300 Message-Id: <20210108024723.26210-2-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210108024723.26210-1-dmitry.kozliuk@gmail.com> References: <20201212233447.23154-1-dmitry.kozliuk@gmail.com> <20210108024723.26210-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 1/4] pmdinfogen: support COFF X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Common Object File Format (COFF) is used on Windows in place of ELF. Add COFF parser to pmdinfogen. Also add an argument to specify input file format, which is selected at configure time based on the target. Signed-off-by: Dmitry Kozlyuk --- Depends-on: series-13153 ("pmdinfogen: rewrite in Python") buildtools/coff.py | 154 +++++++++++++++++++++++++++++++++++++++ buildtools/meson.build | 7 ++ buildtools/pmdinfogen.py | 117 +++++++++++++++++++++-------- 3 files changed, 248 insertions(+), 30 deletions(-) create mode 100644 buildtools/coff.py diff --git a/buildtools/coff.py b/buildtools/coff.py new file mode 100644 index 000000000..86fb0602b --- /dev/null +++ b/buildtools/coff.py @@ -0,0 +1,154 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2020 Dmitry Kozlyuk + +import ctypes + +# x86_64 little-endian +COFF_MAGIC = 0x8664 + +# Names up to this length are stored immediately in symbol table entries. +COFF_NAMELEN = 8 + +# Special "section numbers" changing the meaning of symbol table entry. +COFF_SN_UNDEFINED = 0 +COFF_SN_ABSOLUTE = -1 +COFF_SN_DEBUG = -2 + + +class CoffFileHeader(ctypes.LittleEndianStructure): + _pack_ = True + _fields_ = [ + ("magic", ctypes.c_uint16), + ("section_count", ctypes.c_uint16), + ("timestamp", ctypes.c_uint32), + ("symbol_table_offset", ctypes.c_uint32), + ("symbol_count", ctypes.c_uint32), + ("optional_header_size", ctypes.c_uint16), + ("flags", ctypes.c_uint16), + ] + + +class CoffName(ctypes.Union): + class Reference(ctypes.LittleEndianStructure): + _pack_ = True + _fields_ = [ + ("zeroes", ctypes.c_uint32), + ("offset", ctypes.c_uint32), + ] + + Immediate = ctypes.c_char * 8 + + _pack_ = True + _fields_ = [ + ("immediate", Immediate), + ("reference", Reference), + ] + + +class CoffSection(ctypes.LittleEndianStructure): + _pack_ = True + _fields_ = [ + ("name", CoffName), + ("physical_address", ctypes.c_uint32), + ("physical_address", ctypes.c_uint32), + ("size", ctypes.c_uint32), + ("data_offset", ctypes.c_uint32), + ("relocations_offset", ctypes.c_uint32), + ("line_numbers_offset", ctypes.c_uint32), + ("relocation_count", ctypes.c_uint16), + ("line_number_count", ctypes.c_uint16), + ("flags", ctypes.c_uint32), + ] + + +class CoffSymbol(ctypes.LittleEndianStructure): + _pack_ = True + _fields_ = [ + ("name", CoffName), + ("value", ctypes.c_uint32), + ("section_number", ctypes.c_int16), + ("type", ctypes.c_uint16), + ("storage_class", ctypes.c_uint8), + ("auxiliary_count", ctypes.c_uint8), + ] + + +class Symbol: + def __init__(self, image, symbol: CoffSymbol): + self._image = image + self._coff = symbol + + @property + def name(self): + if self._coff.name.reference.zeroes: + return decode_asciiz(bytes(self._coff.name.immediate)) + + offset = self._coff.name.reference.offset + offset -= ctypes.sizeof(ctypes.c_uint32) + return self._image.get_string(offset) + + def get_value(self, offset): + section_number = self._coff.section_number + + if section_number == COFF_SN_UNDEFINED: + return None + + if section_number == COFF_SN_DEBUG: + return None + + if section_number == COFF_SN_ABSOLUTE: + return bytes(ctypes.c_uint32(self._coff.value)) + + section_data = self._image.get_section_data(section_number) + section_offset = self._coff.value + offset + return section_data[section_offset:] + + +class Image: + def __init__(self, data): + header = CoffFileHeader.from_buffer_copy(data) + header_size = ctypes.sizeof(header) + header.optional_header_size + + sections_desc = CoffSection * header.section_count + sections = sections_desc.from_buffer_copy(data, header_size) + + symbols_desc = CoffSymbol * header.symbol_count + symbols = symbols_desc.from_buffer_copy(data, header.symbol_table_offset) + + strings_offset = header.symbol_table_offset + ctypes.sizeof(symbols) + strings = Image._parse_strings(data[strings_offset:]) + + self._data = data + self._header = header + self._sections = sections + self._symbols = symbols + self._strings = strings + + @staticmethod + def _parse_strings(data): + full_size = ctypes.c_uint32.from_buffer_copy(data) + header_size = ctypes.sizeof(full_size) + return data[header_size : full_size.value] + + @property + def symbols(self): + i = 0 + while i < self._header.symbol_count: + symbol = self._symbols[i] + yield Symbol(self, symbol) + i += symbol.auxiliary_count + 1 + + def get_section_data(self, number): + # section numbers are 1-based + section = self._sections[number - 1] + base = section.data_offset + return self._data[base : base + section.size] + + def get_string(self, offset): + return decode_asciiz(self._strings[offset:]) + + +def decode_asciiz(data): + index = data.find(b'\x00') + end = index if index >= 0 else len(data) + return data[:end].decode() diff --git a/buildtools/meson.build b/buildtools/meson.build index dd4c0f640..23cefd4be 100644 --- a/buildtools/meson.build +++ b/buildtools/meson.build @@ -17,7 +17,14 @@ else endif map_to_win_cmd = py3 + files('map_to_win.py') sphinx_wrapper = py3 + files('call-sphinx-build.py') + +# select object file format pmdinfogen = py3 + files('pmdinfogen.py') +if host_machine.system() == 'windows' + pmdinfogen += 'coff' +else + pmdinfogen += 'elf' +endif # TODO: starting from Meson 0.51.0 use # python3 = import('python').find_installation('python', diff --git a/buildtools/pmdinfogen.py b/buildtools/pmdinfogen.py index 0cca47ff1..474168f21 100755 --- a/buildtools/pmdinfogen.py +++ b/buildtools/pmdinfogen.py @@ -9,8 +9,13 @@ import sys import tempfile -from elftools.elf.elffile import ELFFile -from elftools.elf.sections import SymbolTableSection +try: + from elftools.elf.elffile import ELFFile + from elftools.elf.sections import SymbolTableSection +except ImportError: + pass + +import coff class ELFSymbol: @@ -18,21 +23,18 @@ def __init__(self, image, symbol): self._image = image self._symbol = symbol - @property - def size(self): - return self._symbol["st_size"] - - @property - def value(self): - data = self._image.get_section_data(self._symbol["st_shndx"]) - base = self._symbol["st_value"] - return data[base:base + self.size] - @property def string_value(self): - value = self.value + size = self._symbol["st_size"] + value = self.get_value(0, size) return value[:-1].decode() if value else "" + def get_value(self, offset, size): + section = self._symbol["st_shndx"] + data = self._image.get_section(section).data() + base = self._symbol["st_value"] + offset + return data[base : base + size] + class ELFImage: def __init__(self, data): @@ -45,18 +47,50 @@ def __init__(self, data): def is_big_endian(self): return not self._image.little_endian - def get_section_data(self, name): - return self._image.get_section(name).data() - def find_by_name(self, name): symbol = self._symtab.get_symbol_by_name(name) - return ELFSymbol(self, symbol[0]) if symbol else None + return ELFSymbol(self._image, symbol[0]) if symbol else None def find_by_prefix(self, prefix): for i in range(self._symtab.num_symbols()): symbol = self._symtab.get_symbol(i) if symbol.name.startswith(prefix): - yield ELFSymbol(self, symbol) + yield ELFSymbol(self._image, symbol) + + +class COFFSymbol: + def __init__(self, image, symbol): + self._image = image + self._symbol = symbol + + def get_value(self, offset, size): + value = self._symbol.get_value(offset) + return value[:size] if value else value + + @property + def string_value(self): + value = self._symbol.get_value(0) + return coff.decode_asciiz(value) if value else '' + + +class COFFImage: + def __init__(self, data): + self._image = coff.Image(data) + + @property + def is_big_endian(self): + return False + + def find_by_prefix(self, prefix): + for symbol in self._image.symbols: + if symbol.name.startswith(prefix): + yield COFFSymbol(self._image, symbol) + + def find_by_name(self, name): + for symbol in self._image.symbols: + if symbol.name == name: + return COFFSymbol(self._image, symbol) + return None def define_rte_pci_id(is_big_endian): @@ -117,19 +151,24 @@ def _load_pci_ids(image, table_name_symbol): rte_pci_id = define_rte_pci_id(image.is_big_endian) - pci_id_size = ctypes.sizeof(rte_pci_id) - pci_ids_desc = rte_pci_id * (table_symbol.size // pci_id_size) - pci_ids = pci_ids_desc.from_buffer_copy(table_symbol.value) result = [] - for pci_id in pci_ids: + while True: + size = ctypes.sizeof(rte_pci_id) + offset = size * len(result) + data = table_symbol.get_value(offset, size) + if not data: + break + pci_id = rte_pci_id.from_buffer_copy(data) if not pci_id.device_id: break - result.append([ - pci_id.vendor_id, - pci_id.device_id, - pci_id.subsystem_vendor_id, - pci_id.subsystem_device_id, - ]) + result.append( + [ + pci_id.vendor_id, + pci_id.device_id, + pci_id.subsystem_vendor_id, + pci_id.subsystem_device_id, + ] + ) return result def dump(self, file): @@ -157,6 +196,7 @@ def dump_drivers(drivers, file): def parse_args(): parser = argparse.ArgumentParser() + parser.add_argument("format", help="object file format, 'elf' or 'coff'") parser.add_argument("input", help="input object file path or '-' for stdin") parser.add_argument("output", help="output C file path or '-' for stdout") return parser.parse_args() @@ -170,6 +210,21 @@ def open_input(path): return open(path, "rb") +def read_input(path): + if path == "-": + return sys.stdin.buffer.read() + with open(path, "rb") as file: + return file.read() + + +def load_image(fmt, path): + if fmt == "elf": + return ELFImage(open_input(path)) + if fmt == "coff": + return COFFImage(read_input(path)) + raise Exception("unsupported object file format") + + def open_output(path): if path == "-": return sys.stdout @@ -178,8 +233,10 @@ def open_output(path): def main(): args = parse_args() - infile = open_input(args.input) - image = ELFImage(infile) + if args.format == "elf" and "ELFFile" not in globals(): + raise Exception("elftools module not found") + + image = load_image(args.format, args.input) drivers = load_drivers(image) output = open_output(args.output) dump_drivers(drivers, output) From patchwork Fri Jan 8 02:47:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 86143 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D9799A0524; Fri, 8 Jan 2021 03:47:50 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0F11F140DDB; Fri, 8 Jan 2021 03:47:38 +0100 (CET) Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) by mails.dpdk.org (Postfix) with ESMTP id 8ECB6140DCE for ; Fri, 8 Jan 2021 03:47:36 +0100 (CET) Received: by mail-lf1-f41.google.com with SMTP id a12so19557461lfl.6 for ; Thu, 07 Jan 2021 18:47:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XW19IglQaLhCMyVF34hwkQGQYKaq2Wb5TZQ6lyw3ttI=; b=LY6nlGrIqBlRSV4l/PnTzZsxDhE0Kmb9NpxbSxRX1awznNbzDAa+W/sXmJ9BsI4hAD m0Ay39Y/GR6nNPMW/h9emadktzw9XZu9+HFPSr+BsJkQsdE4Z+Y1mobSHV2b56fG649M 737Wvs/d3Wi83T0s10wDpQCoGZQ1qBqlK37cdjv6L7JkMKuS1s+Nep508iB39VOD3oRD MQ36EzYCHiJrHZpLDpHfOHsEL1iE3eavTZnCWGUSZDOmpv7h40duv6Uv+FcXXGb+TAXk We1suX0TUpLPN2M2hvhSoeqY9uLROYYDlVKZYerUvYJsHXxwYOG9b/ti22+N5IS5gtCV 0QUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XW19IglQaLhCMyVF34hwkQGQYKaq2Wb5TZQ6lyw3ttI=; b=muBRMPW4I5EGA2jZ9ZGTekzkuHhr6cS0BZNXZS/NuH8Tc6ao6r1vJz4XCU8mXCtqLV OnXRaZLMy+Cxqj3wMzD7QuLToB2s8viaWXgkWPpPsiCwkmTzyvAZVgEKuDxr3XAAkXib klLoSWgQ5oAIhaPjFtnWXJbevmKeNr6fEgyV/nIn7oQ6bo5EBFud5BA/EBUdNg4w27Ro m2arZE4pgU5pKQHcVAz4Y2hVehZRSH7CBT9i2WbgIo20d+9KplM5wT7B4L3i7i6FMHPV Gnsisp3PpL8yFqUXdNx6FDRFVvxx5uUpoL4VFC1u2LsqBepEIrnm6o5fAt2N+Un9KUWf puIQ== X-Gm-Message-State: AOAM531al4+ODKEYC0nKUXOWeYAdxIxargQpsVlm6+zMWeFMif2LMd0M tIGFlbHld7nDFePFrHn9a2+0I/HTWVZWxv8Q X-Google-Smtp-Source: ABdhPJyoCMvUbbuURZaguq6s5JzKzPQ4X3M+2mGB+hCLgUqriAE1Qu64qy27/8BkEM/w40+Xrt5WBA== X-Received: by 2002:a05:651c:1255:: with SMTP id h21mr633313ljh.8.1610074055993; Thu, 07 Jan 2021 18:47:35 -0800 (PST) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id i19sm1733599ljj.26.2021.01.07.18.47.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Jan 2021 18:47:35 -0800 (PST) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Narcisa Ana Maria Vasile , Pallavi Kadam , Dmitry Malloy , Bruce Richardson , Neil Horman , Dmitry Kozlyuk Date: Fri, 8 Jan 2021 05:47:21 +0300 Message-Id: <20210108024723.26210-3-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210108024723.26210-1-dmitry.kozliuk@gmail.com> References: <20201212233447.23154-1-dmitry.kozliuk@gmail.com> <20210108024723.26210-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 2/4] pmdinfogen: allow multiple input files X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Process any number of input object files and write a unified output. Signed-off-by: Dmitry Kozlyuk --- buildtools/pmdinfogen.py | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/buildtools/pmdinfogen.py b/buildtools/pmdinfogen.py index 474168f21..965c08945 100755 --- a/buildtools/pmdinfogen.py +++ b/buildtools/pmdinfogen.py @@ -197,7 +197,9 @@ def dump_drivers(drivers, file): def parse_args(): parser = argparse.ArgumentParser() parser.add_argument("format", help="object file format, 'elf' or 'coff'") - parser.add_argument("input", help="input object file path or '-' for stdin") + parser.add_argument( + "input", nargs='+', help="input object file path or '-' for stdin" + ) parser.add_argument("output", help="output C file path or '-' for stdout") return parser.parse_args() @@ -233,13 +235,16 @@ def open_output(path): def main(): args = parse_args() + if args.input.count('-') > 1: + raise Exception("'-' input cannot be used multiple times") if args.format == "elf" and "ELFFile" not in globals(): raise Exception("elftools module not found") - image = load_image(args.format, args.input) - drivers = load_drivers(image) output = open_output(args.output) - dump_drivers(drivers, output) + for path in args.input: + image = load_image(args.format, path) + drivers = load_drivers(image) + dump_drivers(drivers, output) if __name__ == "__main__": From patchwork Fri Jan 8 02:47:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 86144 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3921BA0524; Fri, 8 Jan 2021 03:47:57 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5B893140DE5; Fri, 8 Jan 2021 03:47:39 +0100 (CET) Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) by mails.dpdk.org (Postfix) with ESMTP id AE000140DD8 for ; Fri, 8 Jan 2021 03:47:37 +0100 (CET) Received: by mail-lf1-f44.google.com with SMTP id o13so19600609lfr.3 for ; Thu, 07 Jan 2021 18:47:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yu3Ev+EXmzylcsL2AC1AfozvTyvbRWbK8+Qown2cmgs=; b=uopUUA0JbAv0MnOnJrOI2ji0p/y0yIweZEf0lZVuxxGbUKiIr2uI5yv5HJaOqVKHcC UiYBDnsQe1qpSAv2aikO0R2vi8AKtVeDH2fYY8ggyeSiG0ExWpDwPegSoQnzvXASHCOI am0Ja33Qa6E7HCfFUGcbSq1LZfliakiUm/CjI73t9On5FH3+kKLy+mN8ac4CGDlumMCz Wjmb34yq3awYQsGh4eO4BnnT1OsGhJ9WoplgEj4cSemuvwGWiqkNepvNz2P3+i8tFGIU dIXfNtCaYkljT9q6rRO3DJP7xVtApVTcrt7d9PniXLbGumdQJTpRPq+3WDIeo9kCnoMW Cb2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yu3Ev+EXmzylcsL2AC1AfozvTyvbRWbK8+Qown2cmgs=; b=Oxzh6+4cT5FANcEUI7Zdw7kooviubfevdHgFwKmLuvFPHVWx8+An4qhqasRA1dw5Q8 DroZOueF53++uWvE8nOxp+gKbjPAysjx41xncJ5QvRRZJ7sBVh+sRTdm/a/09B9+r+w7 qtktC9PhCxCrlG01WiRfuEXxy6GcK2SQG60WFQdEYW8U58fwx7mlxbn3vQeICALSnauH DZlLlOJCFdZ1SHHrYS0Vjmny/gfjVf5gIKa23WMWGDOBLAV+fO3XFcMZKtwhqmxkH7pT bQg6ObWVOvrY2mZe8/GnRn+FQmHnuZyU+XIQ8Gcvc7ULO5qjyFZgQU1+XQ2swDa3EwrA pqeA== X-Gm-Message-State: AOAM530TJCKunpr26CRiQJgCis4FgPyXHZfejW3XMl0X6KGRGuIRP+y0 bYQ4IEd1MLGYqTNslubHyKZreMcGZNfee6mT X-Google-Smtp-Source: ABdhPJxxg/TxztkWI5w688vOVhASExZfYhFODeJbW+Vh1hNqbZtPAADF+pwMAXeuKYAnxKcRDk1jWg== X-Received: by 2002:a19:44:: with SMTP id 65mr659817lfa.151.1610074057075; Thu, 07 Jan 2021 18:47:37 -0800 (PST) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id i19sm1733599ljj.26.2021.01.07.18.47.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Jan 2021 18:47:36 -0800 (PST) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Narcisa Ana Maria Vasile , Pallavi Kadam , Dmitry Malloy , Bruce Richardson , Neil Horman , Dmitry Kozlyuk Date: Fri, 8 Jan 2021 05:47:22 +0300 Message-Id: <20210108024723.26210-4-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210108024723.26210-1-dmitry.kozliuk@gmail.com> References: <20201212233447.23154-1-dmitry.kozliuk@gmail.com> <20210108024723.26210-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 3/4] buildtools: support object file extraction for Windows X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" clang archiver tool is llvm-ar on Windows and ar on other platforms. MinGW always uses ar. Replace shell script (Unix-only) that calls ar with a Python script (OS-independent) that calls an appropriate archiver tool selected at configuration time. Move the logic not to generate empty sources into pmdinfogen. Signed-off-by: Dmitry Kozlyuk --- buildtools/gen-pmdinfo-cfile.py | 19 +++++++++++++++++++ buildtools/gen-pmdinfo-cfile.sh | 14 -------------- buildtools/meson.build | 10 ++++++++-- buildtools/pmdinfogen.py | 7 +++++++ 4 files changed, 34 insertions(+), 16 deletions(-) create mode 100644 buildtools/gen-pmdinfo-cfile.py delete mode 100755 buildtools/gen-pmdinfo-cfile.sh diff --git a/buildtools/gen-pmdinfo-cfile.py b/buildtools/gen-pmdinfo-cfile.py new file mode 100644 index 000000000..f1f289ffe --- /dev/null +++ b/buildtools/gen-pmdinfo-cfile.py @@ -0,0 +1,19 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2020 Dmitry Kozlyuk + +import os +import subprocess +import sys +import tempfile + +_, ar, archive, output, *pmdinfogen = sys.argv +with tempfile.TemporaryDirectory() as temp: + proc = subprocess.run( + # Don't use "ar p", because its output is corrupted on Windows. + [ar, "xv", os.path.abspath(archive)], capture_output=True, check=True, cwd=temp + ) + lines = proc.stdout.decode().splitlines() + names = [line[len("x - ") :] for line in lines] + paths = [os.path.join(temp, name) for name in names] + subprocess.run(pmdinfogen + paths + [output], check=True) diff --git a/buildtools/gen-pmdinfo-cfile.sh b/buildtools/gen-pmdinfo-cfile.sh deleted file mode 100755 index 109ee461e..000000000 --- a/buildtools/gen-pmdinfo-cfile.sh +++ /dev/null @@ -1,14 +0,0 @@ -#! /bin/sh -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2017 Intel Corporation - -arfile=$1 -output=$2 -shift 2 -pmdinfogen=$* - -# The generated file must not be empty if compiled in pedantic mode -echo 'static __attribute__((unused)) const char *generator = "'$0'";' > $output -for ofile in `ar t $arfile` ; do - ar p $arfile $ofile | $pmdinfogen - - >> $output -done diff --git a/buildtools/meson.build b/buildtools/meson.build index 23cefd4be..0a2e91a7b 100644 --- a/buildtools/meson.build +++ b/buildtools/meson.build @@ -2,7 +2,6 @@ # Copyright(c) 2017-2019 Intel Corporation pkgconf = find_program('pkg-config', 'pkgconf', required: false) -pmdinfo = find_program('gen-pmdinfo-cfile.sh') list_dir_globs = find_program('list-dir-globs.py') check_symbols = find_program('check-symbols.sh') ldflags_ibverbs_static = find_program('options-ibverbs-static.sh') @@ -18,11 +17,18 @@ endif map_to_win_cmd = py3 + files('map_to_win.py') sphinx_wrapper = py3 + files('call-sphinx-build.py') -# select object file format +# select library and object file format +pmdinfo = py3 + files('gen-pmdinfo-cfile.py') pmdinfogen = py3 + files('pmdinfogen.py') if host_machine.system() == 'windows' + if cc.get_id() == 'gcc' + pmdinfo += 'ar' + else + pmdinfo += 'llvm-ar' + endif pmdinfogen += 'coff' else + pmdinfo += 'ar' pmdinfogen += 'elf' endif diff --git a/buildtools/pmdinfogen.py b/buildtools/pmdinfogen.py index 965c08945..893a6c571 100755 --- a/buildtools/pmdinfogen.py +++ b/buildtools/pmdinfogen.py @@ -233,6 +233,12 @@ def open_output(path): return open(path, "w") +def write_header(output): + output.write( + "static __attribute__((unused)) const char *generator = \"%s\";\n" % sys.argv[0] + ) + + def main(): args = parse_args() if args.input.count('-') > 1: @@ -241,6 +247,7 @@ def main(): raise Exception("elftools module not found") output = open_output(args.output) + write_header(output) for path in args.input: image = load_image(args.format, path) drivers = load_drivers(image) From patchwork Fri Jan 8 02:47:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 86145 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E09AAA0524; Fri, 8 Jan 2021 03:48:05 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DB396140DF2; Fri, 8 Jan 2021 03:47:40 +0100 (CET) Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by mails.dpdk.org (Postfix) with ESMTP id B4964140DE3 for ; Fri, 8 Jan 2021 03:47:38 +0100 (CET) Received: by mail-lf1-f42.google.com with SMTP id a12so19557620lfl.6 for ; Thu, 07 Jan 2021 18:47:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mKfGueTukYS9ZvV46ndDlu6bSNYtp889J1FjP+toIeY=; b=Pe7mPLoNg++zsE/c8Zn+swV38YEdDIpJiA4dZOnxz50L+tRaQE6kklp7GjJGSyiCr5 zVCg4QL8sRdsB8KwIXixdLzXAm5JoKs/ygWNphKeacRQ3EI+KMfxXBeUwpQEsYH/vGds UHm/nKvlM/bPA8BSjRg4mCGuhX0uAg7R/CITJSpiu2Abfh+qWqpT0b4eOhL2UPTfjV2Q ykfgwMBBZdHOPtL6OsEJc5QtCs2XDGFhT6AwGKHdZe1/n+J7WToifO/kqI9T4hbczLZ7 RKW5gwS8Xa0u12S7dA5sVf+oqIe7BZKnxDkKGud7xdiyac5Utef2TZSpxaPVweCsOOkE L56Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mKfGueTukYS9ZvV46ndDlu6bSNYtp889J1FjP+toIeY=; b=cDNJgC8SXn3Gl/7b9w+rIy9yQHW1V5z47ISLuqzGel+LGLdAX53Eahw9yxEGaFpa2Z gLn6B5amUMIYP7AF1/vMkUBHVHElRLEO6oMNIn2zyJy5dlBaEG+McnCruCfbrV9EUlOC rO4NtlJe2/CQnL1thxibawMJcUj6J3KoNbAWdZMbBS781w4oCWPG1jZwe7Gm/aKxg9YV fBliRQl+q0gNBtRGsEnci0TiZBTcpL7N5q17e3ATwB8QqAYtJsCxBPYt8+bBMBftierH MzYI5v5HiB2zWX/zyQ1sE5ZV1R2VVmZ47RO25o/lrh6IL3Z7aYQ3pabDCVLWl6TuiHo6 s0DQ== X-Gm-Message-State: AOAM5339hUlpyOVslnEmMr/eVfpCmdCjg5UJGs7uWiWLJvuxtUKCAdCX erIpxbnDWgc1PhyEV9fiMVq7IEGM1Gbt8KsP X-Google-Smtp-Source: ABdhPJwq/ZG1FC02O9Due6Hwjj8kZhkNYQJPYWswxgWCiH6VwE3VIwDNr2ZzrVFMralNxNrK5I31Xg== X-Received: by 2002:a2e:9cc3:: with SMTP id g3mr644789ljj.0.1610074058144; Thu, 07 Jan 2021 18:47:38 -0800 (PST) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id i19sm1733599ljj.26.2021.01.07.18.47.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Jan 2021 18:47:37 -0800 (PST) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Narcisa Ana Maria Vasile , Pallavi Kadam , Dmitry Malloy , Bruce Richardson , Neil Horman , Dmitry Kozlyuk Date: Fri, 8 Jan 2021 05:47:23 +0300 Message-Id: <20210108024723.26210-5-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210108024723.26210-1-dmitry.kozliuk@gmail.com> References: <20201212233447.23154-1-dmitry.kozliuk@gmail.com> <20210108024723.26210-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 4/4] build: enable pmdinfogen for Windows X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Remove platform restriction when building drivers. Signed-off-by: Dmitry Kozlyuk Tested-by: Jie Zhou --- drivers/meson.build | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/drivers/meson.build b/drivers/meson.build index f49d4f79b..1dfa8738f 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -143,20 +143,18 @@ foreach subpath:subdirs # lib and then running pmdinfogen on the contents of # that lib. The final lib reuses the object files and # adds in the new source file. - if not is_windows - out_filename = lib_name + '.pmd.c' - tmp_lib = static_library('tmp_' + lib_name, - sources, - include_directories: includes, - dependencies: static_deps, - c_args: cflags) - objs += tmp_lib.extract_all_objects() - sources = custom_target(out_filename, - command: [pmdinfo, tmp_lib.full_path(), - '@OUTPUT@', pmdinfogen], - output: out_filename, - depends: [tmp_lib]) - endif + out_filename = lib_name + '.pmd.c' + tmp_lib = static_library('tmp_' + lib_name, + sources, + include_directories: includes, + dependencies: static_deps, + c_args: cflags) + objs += tmp_lib.extract_all_objects() + sources = custom_target(out_filename, + command: [pmdinfo, tmp_lib.full_path(), + '@OUTPUT@', pmdinfogen], + output: out_filename, + depends: [tmp_lib]) # now build the static driver static_lib = static_library(lib_name,