From patchwork Sun Jan 24 20:51:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 87160 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3595DA052A; Sun, 24 Jan 2021 21:52:14 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D4ED5140D49; Sun, 24 Jan 2021 21:52:08 +0100 (CET) Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) by mails.dpdk.org (Postfix) with ESMTP id 717FB140D43 for ; Sun, 24 Jan 2021 21:52:07 +0100 (CET) Received: by mail-lj1-f171.google.com with SMTP id i17so12902132ljn.1 for ; Sun, 24 Jan 2021 12:52:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+wwEn0uvC6ZwE2YT5goyQUlpvw6CXN6BT2E3Ec/kmAM=; b=spVUUFUXpX2pWBohQXBsTXaBEs89gU7RdtWmZFTwks55Gn1o0vP1zbWR4ao7OrONIB WrFXz4sJvPprK9TkQuRzLvFQnSOE9t5zdDW26RUBXzXXN9pmwTh3a2jzZ/S7YGrCELMZ N6g8i6EcER/f2DxG1nnO3tZS93uALIzS1FF0rP/o4z9WmRBaXZCWkohnCmvbQZghsUBB dYfD9j6fNZ47dXQjndW7AN+/6dJEvD9zAAxFAIa0Aj/xwUm8NncMfrFZi6beUt/RufqC 0i+y9Tt3kT2O9QkidpUk8rTbaOcFnFCDNjS/K7CCkq5H6u6GnY3PX8N4Sw5FLH8we2BU hbfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+wwEn0uvC6ZwE2YT5goyQUlpvw6CXN6BT2E3Ec/kmAM=; b=dizJT2an3U9xgxj9B+gRhvOlTMM5msJ05wCxlUEIRPdpKaemfymjz8Uo+7PS1yMYy/ QQhA6hBvkdFH3yOBBZlmsy6esMDznHgqRmVDvfUQ5hGiyCO4wXRIK8qqaGnOyp9uXhJz 6C6SExUMOXoTY0SpEzpUX2lHuShmvGS/cVYody0fKehDZnPqKWJ4C6WaUapQsPxedkgS 4L11dt2E6UdGBY6e1aMKiuIo72GPSwJoAs8re9Ft3R12S839VWwyH/FBZUEN5MF+eIka +KCUlAurNDmBA4q5lNWhwY6DAi7HgESXcgraC2Je9G+J7cSjda6tjdpuClreyPsEqkvP etpA== X-Gm-Message-State: AOAM533BkIQBl06zQqLAloVRwY28IUKSCK4+xhy5L+RZgc92qdvxcSty eDyrHQ2QLCGAPAV+TorPkHV7EcZrehHWRQ== X-Google-Smtp-Source: ABdhPJyoFXczD2IGe1zHHE4O7F37ZCOl6E26Lfumi7gweOwJF3EdbmZXVURHqHVl0wtylTwEmu9KBA== X-Received: by 2002:a05:651c:8d:: with SMTP id 13mr272768ljq.33.1611521526578; Sun, 24 Jan 2021 12:52:06 -0800 (PST) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id r72sm1168551lff.214.2021.01.24.12.52.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Jan 2021 12:52:05 -0800 (PST) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Maxime Coquelin , Bruce Richardson , Thomas Monjalon , Dmitry Kozlyuk , Neil Horman Date: Sun, 24 Jan 2021 23:51:55 +0300 Message-Id: <20210124205157.14386-2-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210124205157.14386-1-dmitry.kozliuk@gmail.com> References: <20210122224327.5621-1-dmitry.kozliuk@gmail.com> <20210124205157.14386-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v10 1/3] pmdinfogen: add Python implementation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using a high-level, interpreted language simplifies maintenance and build process. Furthermore, ELF handling is delegated to pyelftools package. Original logic is kept, the copyright recognizes that. Signed-off-by: Dmitry Kozlyuk --- buildtools/pmdinfogen.py | 189 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 189 insertions(+) create mode 100755 buildtools/pmdinfogen.py diff --git a/buildtools/pmdinfogen.py b/buildtools/pmdinfogen.py new file mode 100755 index 000000000..0cca47ff1 --- /dev/null +++ b/buildtools/pmdinfogen.py @@ -0,0 +1,189 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2016 Neil Horman +# Copyright (c) 2020 Dmitry Kozlyuk + +import argparse +import ctypes +import json +import sys +import tempfile + +from elftools.elf.elffile import ELFFile +from elftools.elf.sections import SymbolTableSection + + +class ELFSymbol: + def __init__(self, image, symbol): + self._image = image + self._symbol = symbol + + @property + def size(self): + return self._symbol["st_size"] + + @property + def value(self): + data = self._image.get_section_data(self._symbol["st_shndx"]) + base = self._symbol["st_value"] + return data[base:base + self.size] + + @property + def string_value(self): + value = self.value + return value[:-1].decode() if value else "" + + +class ELFImage: + def __init__(self, data): + self._image = ELFFile(data) + self._symtab = self._image.get_section_by_name(".symtab") + if not isinstance(self._symtab, SymbolTableSection): + raise Exception(".symtab section is not a symbol table") + + @property + def is_big_endian(self): + return not self._image.little_endian + + def get_section_data(self, name): + return self._image.get_section(name).data() + + def find_by_name(self, name): + symbol = self._symtab.get_symbol_by_name(name) + return ELFSymbol(self, symbol[0]) if symbol else None + + def find_by_prefix(self, prefix): + for i in range(self._symtab.num_symbols()): + symbol = self._symtab.get_symbol(i) + if symbol.name.startswith(prefix): + yield ELFSymbol(self, symbol) + + +def define_rte_pci_id(is_big_endian): + base_type = ctypes.LittleEndianStructure + if is_big_endian: + base_type = ctypes.BigEndianStructure + + class rte_pci_id(base_type): + _pack_ = True + _fields_ = [ + ("class_id", ctypes.c_uint32), + ("vendor_id", ctypes.c_uint16), + ("device_id", ctypes.c_uint16), + ("subsystem_vendor_id", ctypes.c_uint16), + ("subsystem_device_id", ctypes.c_uint16), + ] + + return rte_pci_id + + +class Driver: + OPTIONS = [ + ("params", "_param_string_export"), + ("kmod", "_kmod_dep_export"), + ] + + def __init__(self, name, options): + self.name = name + for key, value in options.items(): + setattr(self, key, value) + self.pci_ids = [] + + @classmethod + def load(cls, image, symbol): + name = symbol.string_value + + options = {} + for key, suffix in cls.OPTIONS: + option_symbol = image.find_by_name("__%s%s" % (name, suffix)) + if option_symbol: + value = option_symbol.string_value + options[key] = value + + driver = cls(name, options) + + pci_table_name_symbol = image.find_by_name("__%s_pci_tbl_export" % name) + if pci_table_name_symbol: + driver.pci_ids = cls._load_pci_ids(image, pci_table_name_symbol) + + return driver + + @staticmethod + def _load_pci_ids(image, table_name_symbol): + table_name = table_name_symbol.string_value + table_symbol = image.find_by_name(table_name) + if not table_symbol: + raise Exception("PCI table declared but not defined: %d" % table_name) + + rte_pci_id = define_rte_pci_id(image.is_big_endian) + + pci_id_size = ctypes.sizeof(rte_pci_id) + pci_ids_desc = rte_pci_id * (table_symbol.size // pci_id_size) + pci_ids = pci_ids_desc.from_buffer_copy(table_symbol.value) + result = [] + for pci_id in pci_ids: + if not pci_id.device_id: + break + result.append([ + pci_id.vendor_id, + pci_id.device_id, + pci_id.subsystem_vendor_id, + pci_id.subsystem_device_id, + ]) + return result + + def dump(self, file): + dumped = json.dumps(self.__dict__) + escaped = dumped.replace('"', '\\"') + print( + 'const char %s_pmd_info[] __attribute__((used)) = "PMD_INFO_STRING= %s";' + % (self.name, escaped), + file=file, + ) + + +def load_drivers(image): + drivers = [] + for symbol in image.find_by_prefix("this_pmd_name"): + drivers.append(Driver.load(image, symbol)) + return drivers + + +def dump_drivers(drivers, file): + # Keep legacy order of definitions. + for driver in reversed(drivers): + driver.dump(file) + + +def parse_args(): + parser = argparse.ArgumentParser() + parser.add_argument("input", help="input object file path or '-' for stdin") + parser.add_argument("output", help="output C file path or '-' for stdout") + return parser.parse_args() + + +def open_input(path): + if path == "-": + temp = tempfile.TemporaryFile() + temp.write(sys.stdin.buffer.read()) + return temp + return open(path, "rb") + + +def open_output(path): + if path == "-": + return sys.stdout + return open(path, "w") + + +def main(): + args = parse_args() + infile = open_input(args.input) + image = ELFImage(infile) + drivers = load_drivers(image) + output = open_output(args.output) + dump_drivers(drivers, output) + + +if __name__ == "__main__": + main() From patchwork Sun Jan 24 20:51:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 87161 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7ECE6A052A; Sun, 24 Jan 2021 21:52:21 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 13698140D6A; Sun, 24 Jan 2021 21:52:10 +0100 (CET) Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by mails.dpdk.org (Postfix) with ESMTP id 69CD0140D48 for ; Sun, 24 Jan 2021 21:52:08 +0100 (CET) Received: by mail-lf1-f46.google.com with SMTP id q8so14853317lfm.10 for ; Sun, 24 Jan 2021 12:52:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vqdym5e+UBO2PzJQ1URI92yL6kWjETYN2y0cidYrZoU=; b=he1tncVIzTu/TD9N4dXBYZoYd6G8X30HiiJiNBES4y/8UoldnXcOMCmP+3ErKCZDll JNTxURLCysW18svXZTV+5432F9sSpsvmi0fEL2s8GPaNm7pn4aPnq0/eRxsSYtD6p/lS jkqm7bKqDbXv+JxIzhtx/50qvTmaK2ndymcYAbx7nwx02zj5qGlCTChtZY66vaMBkMOl zPJbEE0sN/h/L9Nd5a9waP6qquR/0sfWqOUCvrnZOwb7xIXIy02r2UfF9xFAWGpiJDRM g2WNl/88O9s7pJm1/5idVm237/PFxDK/UYSkXAWqx7XaLK1QMA6cuzZt3PhNN7jSFAe9 /xIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vqdym5e+UBO2PzJQ1URI92yL6kWjETYN2y0cidYrZoU=; b=WuUvwEuBXXV0s1nrQX5Q2fuTqDGIO2m7X784qkR2Hz96dZaXzy6eZ7+9gR3zXXfzIv Nq98Dat1w7Y0DLXj6ylX3O0N55iPQ3Lgkk0XOcX5WzQGyAVBJlMxkSN9pNAC5xRAaAxj oYEV4l+zhppqkFOeJgPwVm5CVjxRw1NPpUwEvmnTfvUGmmgUDyJ774fVho5b6t7+9wMk esmI/ubSGIH5SyFvJAzR/huN126mjNzVaIHfuLRLqwFM9D+avP6MPPraCv9KN76OLEad EoAy1cxF1XtHdhl7cASafeKpXlTn6j643uYWIxLVp3e6ib72oLYq2NAhTULOFEgyNbH5 UYGg== X-Gm-Message-State: AOAM532aQc8mRK3u9DBRxIZXzA/3IyNjtpqZ3pxuTrQnHrFsGOWfCChH da8aCBhUJ4A/Pw1aGEdrXls0rToe/CZWFw== X-Google-Smtp-Source: ABdhPJxMztxg+yOhJtbh70PQ2wHCqKYEvTtKqbVEV4GqeWPMnKULUBKg2CDtoZYB59/IyPXZOX3unQ== X-Received: by 2002:a19:3f94:: with SMTP id m142mr17438lfa.196.1611521527738; Sun, 24 Jan 2021 12:52:07 -0800 (PST) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id r72sm1168551lff.214.2021.01.24.12.52.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Jan 2021 12:52:07 -0800 (PST) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Maxime Coquelin , Bruce Richardson , Thomas Monjalon , Dmitry Kozlyuk , Aaron Conole , Michael Santana , Ray Kinsella , Neil Horman Date: Sun, 24 Jan 2021 23:51:56 +0300 Message-Id: <20210124205157.14386-3-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210124205157.14386-1-dmitry.kozliuk@gmail.com> References: <20210122224327.5621-1-dmitry.kozliuk@gmail.com> <20210124205157.14386-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v10 2/3] build: use Python pmdinfogen X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Use the same interpreter to run pmdinfogen as for other build scripts. Adjust wrapper script accordingly and also don't suppress stderr from ar and pmdinfogen. Add configure-time check for elftools Python module for Unix hosts. Add pyelftools to CI configuration and build requirements for Linux and FreeBSD. Windows targets are not currently using pmdinfogen. Suppress ABI warnings about generated PMD information strings. Signed-off-by: Dmitry Kozlyuk --- .github/workflows/build.yml | 4 ++-- .travis.yml | 2 +- buildtools/gen-pmdinfo-cfile.sh | 6 +++--- buildtools/meson.build | 15 +++++++++++++++ devtools/libabigail.abignore | 4 ++++ doc/guides/freebsd_gsg/build_dpdk.rst | 3 ++- doc/guides/linux_gsg/sys_reqs.rst | 6 ++++++ drivers/meson.build | 2 +- meson.build | 1 - 9 files changed, 34 insertions(+), 9 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 0b72df0eb..a5b579add 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -91,8 +91,8 @@ jobs: run: sudo apt update - name: Install packages run: sudo apt install -y ccache libnuma-dev python3-setuptools - python3-wheel python3-pip ninja-build libbsd-dev libpcap-dev - libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev + python3-wheel python3-pip python3-pyelftools ninja-build libbsd-dev + libpcap-dev libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev - name: Install libabigail build dependencies if no cache is available if: env.ABI_CHECKS == 'true' && steps.libabigail-cache.outputs.cache-hit != 'true' run: sudo apt install -y autoconf automake libtool pkg-config libxml2-dev diff --git a/.travis.yml b/.travis.yml index 5aa7ad49f..4391af1d5 100644 --- a/.travis.yml +++ b/.travis.yml @@ -14,7 +14,7 @@ addons: apt: update: true packages: &required_packages - - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip, ninja-build] + - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip, python3-pyelftools, ninja-build] - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev, libfdt-dev, libjansson-dev] _aarch64_packages: &aarch64_packages diff --git a/buildtools/gen-pmdinfo-cfile.sh b/buildtools/gen-pmdinfo-cfile.sh index 43059cf36..109ee461e 100755 --- a/buildtools/gen-pmdinfo-cfile.sh +++ b/buildtools/gen-pmdinfo-cfile.sh @@ -4,11 +4,11 @@ arfile=$1 output=$2 -pmdinfogen=$3 +shift 2 +pmdinfogen=$* # The generated file must not be empty if compiled in pedantic mode echo 'static __attribute__((unused)) const char *generator = "'$0'";' > $output for ofile in `ar t $arfile` ; do - ar p $arfile $ofile | $pmdinfogen - - >> $output 2> /dev/null + ar p $arfile $ofile | $pmdinfogen - - >> $output done -exit 0 diff --git a/buildtools/meson.build b/buildtools/meson.build index 04808dabc..dd4c0f640 100644 --- a/buildtools/meson.build +++ b/buildtools/meson.build @@ -17,3 +17,18 @@ else endif map_to_win_cmd = py3 + files('map_to_win.py') sphinx_wrapper = py3 + files('call-sphinx-build.py') +pmdinfogen = py3 + files('pmdinfogen.py') + +# TODO: starting from Meson 0.51.0 use +# python3 = import('python').find_installation('python', +# modules : python3_required_modules) +python3_required_modules = [] +if host_machine.system() != 'windows' + python3_required_modules = ['elftools'] +endif +foreach module : python3_required_modules + script = 'import importlib.util; import sys; exit(importlib.util.find_spec("@0@") is None)' + if run_command(py3, '-c', script.format(module)).returncode() != 0 + error('missing python module: @0@'.format(module)) + endif +endforeach diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore index 1dc84fa74..05afccc1a 100644 --- a/devtools/libabigail.abignore +++ b/devtools/libabigail.abignore @@ -16,3 +16,7 @@ [suppress_type] name = rte_cryptodev has_data_member_inserted_between = {0, 1023} + +; Ignore all changes in generated PMD information strings. +[suppress_variable] + name_regex = _pmd_info$ diff --git a/doc/guides/freebsd_gsg/build_dpdk.rst b/doc/guides/freebsd_gsg/build_dpdk.rst index e3005a7f3..bed353473 100644 --- a/doc/guides/freebsd_gsg/build_dpdk.rst +++ b/doc/guides/freebsd_gsg/build_dpdk.rst @@ -14,10 +14,11 @@ The following FreeBSD packages are required to build DPDK: * meson * ninja * pkgconf +* py37-pyelftools These can be installed using (as root):: - pkg install meson pkgconf + pkg install meson pkgconf py37-pyelftools To compile the required kernel modules for memory management and working with physical NIC devices, the kernel sources for FreeBSD also diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index be714adf2..a05b5bd81 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -52,6 +52,12 @@ Compilation of the DPDK * If the packaged version is below the minimum version, the latest versions can be installed from Python's "pip" repository: ``pip3 install meson ninja`` +* ``pyelftools`` (version 0.22+) + + * For RHEL/Fedora systems it can be installed using ``dnf install python-pyelftools`` + + * For Ubuntu/Debian it can be installed using ``apt install python3-pyelftools`` + * Library for handling NUMA (Non Uniform Memory Access). * ``numactl-devel`` in RHEL/Fedora; diff --git a/drivers/meson.build b/drivers/meson.build index 77f65fa90..ff5cdb952 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -132,7 +132,7 @@ foreach subpath:subdirs command: [pmdinfo, tmp_lib.full_path(), '@OUTPUT@', pmdinfogen], output: out_filename, - depends: [pmdinfogen, tmp_lib]) + depends: [tmp_lib]) endif # now build the static driver diff --git a/meson.build b/meson.build index 45d974cd2..2b9c37eb4 100644 --- a/meson.build +++ b/meson.build @@ -45,7 +45,6 @@ subdir('buildtools') subdir('config') # build libs and drivers -subdir('buildtools/pmdinfogen') subdir('lib') subdir('drivers') From patchwork Sun Jan 24 20:51:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 87162 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8E921A052A; Sun, 24 Jan 2021 21:52:27 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4C8F7140D96; Sun, 24 Jan 2021 21:52:13 +0100 (CET) Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by mails.dpdk.org (Postfix) with ESMTP id 02D90140D66 for ; Sun, 24 Jan 2021 21:52:09 +0100 (CET) Received: by mail-lf1-f49.google.com with SMTP id m22so14880433lfg.5 for ; Sun, 24 Jan 2021 12:52:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XmfNPsQTaVAYq4aarWz80+fc1PDj5qbtif/8Qsxculc=; b=H91ePoDZ8C9Y8fbyLkJ0mjHkXARswQQMEFYphXy3n+eYuBgU61xylCE/p5Chw/C37R 3SEN8wp0Ob8WOI6X2Ko1Hao2IvojPH1xgOq5LxUmT2vW0lHKETYHOUdb50H+e037o0ob 1L/hEpQ4ouEG1t1/BoV1lHFft5FfrxgYNm3XpfnrOBXknxwDf5FUaY2dB1/WU47cgLuX TiUZ0Q3RMiXbEjWqgj7A2kUhz0lVDroK7SWETHPH6wrfoGw1qqElOOJ4JNLjEvA0a5YE O1CHz6QrS3fXx8C5U7e+h6QGFJLYQF5Ai66CBwHbx3ygKSz3//jofca2lIXak6Ft+ukK +KoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XmfNPsQTaVAYq4aarWz80+fc1PDj5qbtif/8Qsxculc=; b=eskI3hd7+SQkIsBCsoSI6wrT7vbCdqtGl44m+fA4LFYEZKiSzDQYNpfgMfbxD6nLSn ecrZgHNesyIW+k0NlVwpiJJHuAG1EIP6dEESh1Q/OuIZEFK2jnBc/GX3ougF5CQi+Gq7 92PwBDFt7juIbYQMe++WoIWk9/MoeS3t0ZufizhO2ccsxk5KvGR1v059AE/xYymYZIZR 1f/cYwoihZjVmMDWXnzmJ9EYF25MjfMl4k6t7n5w6aqMHlodLFOKXQbHYIMiyHvwC/XQ kuW4aWl+v+MSYrkQyYRM6RIkdnKt4K1plEdSaTKQNgwrs5lQjB0nzgm/ukUImGfHGMp4 O1KQ== X-Gm-Message-State: AOAM531Lj3rhsdXdOhAnA94QcV3Y//W/B0KWIe0nP0KrL2W/2k9TdgLt 6DzFdqDRoc0ssE3HKS7SKARRYyJows/o/A== X-Google-Smtp-Source: ABdhPJxzrNhRzL0kjwXaKxcZQBFLbk3XWUkNBHYjf8PBb5GYV7PegmIEB51Mwuw6CcOxswLSeIyD8g== X-Received: by 2002:a19:6d07:: with SMTP id i7mr295987lfc.75.1611521529049; Sun, 24 Jan 2021 12:52:09 -0800 (PST) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id r72sm1168551lff.214.2021.01.24.12.52.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Jan 2021 12:52:08 -0800 (PST) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Maxime Coquelin , Bruce Richardson , Thomas Monjalon , Dmitry Kozlyuk Date: Sun, 24 Jan 2021 23:51:57 +0300 Message-Id: <20210124205157.14386-4-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210124205157.14386-1-dmitry.kozliuk@gmail.com> References: <20210122224327.5621-1-dmitry.kozliuk@gmail.com> <20210124205157.14386-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v10 3/3] pmdinfogen: remove C implementation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Delete the files no longer used in build process. Add myself as maintainer of new implementation. Signed-off-by: Dmitry Kozlyuk --- MAINTAINERS | 3 +- buildtools/pmdinfogen/meson.build | 14 - buildtools/pmdinfogen/pmdinfogen.c | 456 ----------------------------- buildtools/pmdinfogen/pmdinfogen.h | 119 -------- 4 files changed, 2 insertions(+), 590 deletions(-) delete mode 100644 buildtools/pmdinfogen/meson.build delete mode 100644 buildtools/pmdinfogen/pmdinfogen.c delete mode 100644 buildtools/pmdinfogen/pmdinfogen.h diff --git a/MAINTAINERS b/MAINTAINERS index aa973a396..65f6fffd1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -133,7 +133,8 @@ F: lib/*/*.map Driver information M: Neil Horman -F: buildtools/pmdinfogen/ +M: Dmitry Kozlyuk +F: buildtools/pmdinfogen.py F: usertools/dpdk-pmdinfo.py F: doc/guides/tools/pmdinfo.rst diff --git a/buildtools/pmdinfogen/meson.build b/buildtools/pmdinfogen/meson.build deleted file mode 100644 index 670528fac..000000000 --- a/buildtools/pmdinfogen/meson.build +++ /dev/null @@ -1,14 +0,0 @@ -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2017 Intel Corporation - -if is_windows - subdir_done() -endif - -pmdinfogen_inc = [global_inc] -pmdinfogen_inc += include_directories('../../lib/librte_eal/include') -pmdinfogen_inc += include_directories('../../lib/librte_pci') -pmdinfogen = executable('pmdinfogen', - 'pmdinfogen.c', - include_directories: pmdinfogen_inc, - native: true) diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c deleted file mode 100644 index a68d1ea99..000000000 --- a/buildtools/pmdinfogen/pmdinfogen.c +++ /dev/null @@ -1,456 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 - * Postprocess pmd object files to export hw support - * - * Copyright 2016 Neil Horman - * Based in part on modpost.c from the linux kernel - */ - -#include -#include -#include -#include -#include -#include -#include - -#include -#include "pmdinfogen.h" - -#ifdef RTE_ARCH_64 -#define ADDR_SIZE 64 -#else -#define ADDR_SIZE 32 -#endif - -static int use_stdin, use_stdout; - -static const char *sym_name(struct elf_info *elf, Elf_Sym *sym) -{ - if (sym) - return elf->strtab + sym->st_name; - else - return "(unknown)"; -} - -static void *grab_file(const char *filename, unsigned long *size) -{ - struct stat st; - void *map = MAP_FAILED; - int fd = -1; - - if (!use_stdin) { - fd = open(filename, O_RDONLY); - if (fd < 0) - return NULL; - } else { - /* from stdin, use a temporary file to mmap */ - FILE *infile; - char buffer[1024]; - int n; - - infile = tmpfile(); - if (infile == NULL) { - perror("tmpfile"); - return NULL; - } - fd = dup(fileno(infile)); - fclose(infile); - if (fd < 0) - return NULL; - - n = read(STDIN_FILENO, buffer, sizeof(buffer)); - while (n > 0) { - if (write(fd, buffer, n) != n) - goto failed; - n = read(STDIN_FILENO, buffer, sizeof(buffer)); - } - } - - if (fstat(fd, &st)) - goto failed; - - *size = st.st_size; - map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0); - -failed: - close(fd); - if (map == MAP_FAILED) - return NULL; - return map; -} - -/** - * Return a copy of the next line in a mmap'ed file. - * spaces in the beginning of the line is trimmed away. - * Return a pointer to a static buffer. - **/ -static void release_file(void *file, unsigned long size) -{ - munmap(file, size); -} - - -static void *get_sym_value(struct elf_info *info, const Elf_Sym *sym) -{ - return RTE_PTR_ADD(info->hdr, - info->sechdrs[sym->st_shndx].sh_offset + sym->st_value); -} - -static Elf_Sym *find_sym_in_symtab(struct elf_info *info, - const char *name, Elf_Sym *last) -{ - Elf_Sym *idx; - if (last) - idx = last+1; - else - idx = info->symtab_start; - - for (; idx < info->symtab_stop; idx++) { - const char *n = sym_name(info, idx); - if (!strncmp(n, name, strlen(name))) - return idx; - } - return NULL; -} - -static int parse_elf(struct elf_info *info, const char *filename) -{ - unsigned int i; - Elf_Ehdr *hdr; - Elf_Shdr *sechdrs; - Elf_Sym *sym; - int endian; - unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U; - - hdr = grab_file(filename, &info->size); - if (!hdr) { - perror(filename); - exit(1); - } - info->hdr = hdr; - if (info->size < sizeof(*hdr)) { - /* file too small, assume this is an empty .o file */ - return 0; - } - /* Is this a valid ELF file? */ - if ((hdr->e_ident[EI_MAG0] != ELFMAG0) || - (hdr->e_ident[EI_MAG1] != ELFMAG1) || - (hdr->e_ident[EI_MAG2] != ELFMAG2) || - (hdr->e_ident[EI_MAG3] != ELFMAG3)) { - /* Not an ELF file - silently ignore it */ - return 0; - } - - if (!hdr->e_ident[EI_DATA]) { - /* Unknown endian */ - return 0; - } - - endian = hdr->e_ident[EI_DATA]; - - /* Fix endianness in ELF header */ - hdr->e_type = TO_NATIVE(endian, 16, hdr->e_type); - hdr->e_machine = TO_NATIVE(endian, 16, hdr->e_machine); - hdr->e_version = TO_NATIVE(endian, 32, hdr->e_version); - hdr->e_entry = TO_NATIVE(endian, ADDR_SIZE, hdr->e_entry); - hdr->e_phoff = TO_NATIVE(endian, ADDR_SIZE, hdr->e_phoff); - hdr->e_shoff = TO_NATIVE(endian, ADDR_SIZE, hdr->e_shoff); - hdr->e_flags = TO_NATIVE(endian, 32, hdr->e_flags); - hdr->e_ehsize = TO_NATIVE(endian, 16, hdr->e_ehsize); - hdr->e_phentsize = TO_NATIVE(endian, 16, hdr->e_phentsize); - hdr->e_phnum = TO_NATIVE(endian, 16, hdr->e_phnum); - hdr->e_shentsize = TO_NATIVE(endian, 16, hdr->e_shentsize); - hdr->e_shnum = TO_NATIVE(endian, 16, hdr->e_shnum); - hdr->e_shstrndx = TO_NATIVE(endian, 16, hdr->e_shstrndx); - - sechdrs = RTE_PTR_ADD(hdr, hdr->e_shoff); - info->sechdrs = sechdrs; - - /* Check if file offset is correct */ - if (hdr->e_shoff > info->size) { - fprintf(stderr, "section header offset=%lu in file '%s' " - "is bigger than filesize=%lu\n", - (unsigned long)hdr->e_shoff, - filename, info->size); - return 0; - } - - if (hdr->e_shnum == SHN_UNDEF) { - /* - * There are more than 64k sections, - * read count from .sh_size. - */ - info->num_sections = - TO_NATIVE(endian, ADDR_SIZE, sechdrs[0].sh_size); - } else { - info->num_sections = hdr->e_shnum; - } - if (hdr->e_shstrndx == SHN_XINDEX) - info->secindex_strings = - TO_NATIVE(endian, 32, sechdrs[0].sh_link); - else - info->secindex_strings = hdr->e_shstrndx; - - /* Fix endianness in section headers */ - for (i = 0; i < info->num_sections; i++) { - sechdrs[i].sh_name = - TO_NATIVE(endian, 32, sechdrs[i].sh_name); - sechdrs[i].sh_type = - TO_NATIVE(endian, 32, sechdrs[i].sh_type); - sechdrs[i].sh_flags = - TO_NATIVE(endian, 32, sechdrs[i].sh_flags); - sechdrs[i].sh_addr = - TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_addr); - sechdrs[i].sh_offset = - TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_offset); - sechdrs[i].sh_size = - TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_size); - sechdrs[i].sh_link = - TO_NATIVE(endian, 32, sechdrs[i].sh_link); - sechdrs[i].sh_info = - TO_NATIVE(endian, 32, sechdrs[i].sh_info); - sechdrs[i].sh_addralign = - TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_addralign); - sechdrs[i].sh_entsize = - TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_entsize); - } - /* Find symbol table. */ - for (i = 1; i < info->num_sections; i++) { - int nobits = sechdrs[i].sh_type == SHT_NOBITS; - - if (!nobits && sechdrs[i].sh_offset > info->size) { - fprintf(stderr, "%s is truncated. " - "sechdrs[i].sh_offset=%lu > sizeof(*hrd)=%zu\n", - filename, (unsigned long)sechdrs[i].sh_offset, - sizeof(*hdr)); - return 0; - } - - if (sechdrs[i].sh_type == SHT_SYMTAB) { - unsigned int sh_link_idx; - symtab_idx = i; - info->symtab_start = RTE_PTR_ADD(hdr, - sechdrs[i].sh_offset); - info->symtab_stop = RTE_PTR_ADD(hdr, - sechdrs[i].sh_offset + sechdrs[i].sh_size); - sh_link_idx = sechdrs[i].sh_link; - info->strtab = RTE_PTR_ADD(hdr, - sechdrs[sh_link_idx].sh_offset); - } - - /* 32bit section no. table? ("more than 64k sections") */ - if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) { - symtab_shndx_idx = i; - info->symtab_shndx_start = RTE_PTR_ADD(hdr, - sechdrs[i].sh_offset); - info->symtab_shndx_stop = RTE_PTR_ADD(hdr, - sechdrs[i].sh_offset + sechdrs[i].sh_size); - } - } - if (!info->symtab_start) - fprintf(stderr, "%s has no symtab?\n", filename); - else { - /* Fix endianness in symbols */ - for (sym = info->symtab_start; sym < info->symtab_stop; sym++) { - sym->st_shndx = TO_NATIVE(endian, 16, sym->st_shndx); - sym->st_name = TO_NATIVE(endian, 32, sym->st_name); - sym->st_value = TO_NATIVE(endian, ADDR_SIZE, sym->st_value); - sym->st_size = TO_NATIVE(endian, ADDR_SIZE, sym->st_size); - } - } - - if (symtab_shndx_idx != ~0U) { - Elf32_Word *p; - if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link) - fprintf(stderr, - "%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n", - filename, sechdrs[symtab_shndx_idx].sh_link, - symtab_idx); - /* Fix endianness */ - for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop; - p++) - *p = TO_NATIVE(endian, 32, *p); - } - - return 1; -} - -static void parse_elf_finish(struct elf_info *info) -{ - struct pmd_driver *tmp, *idx = info->drivers; - release_file(info->hdr, info->size); - while (idx) { - tmp = idx->next; - free(idx); - idx = tmp; - } -} - -struct opt_tag { - const char *suffix; - const char *json_id; -}; - -static const struct opt_tag opt_tags[] = { - {"_param_string_export", "params"}, - {"_kmod_dep_export", "kmod"}, -}; - -static int complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv) -{ - const char *tname; - int i; - char tmpsymname[128]; - Elf_Sym *tmpsym; - - drv->name = get_sym_value(info, drv->name_sym); - - for (i = 0; i < PMD_OPT_MAX; i++) { - memset(tmpsymname, 0, 128); - sprintf(tmpsymname, "__%s%s", drv->name, opt_tags[i].suffix); - tmpsym = find_sym_in_symtab(info, tmpsymname, NULL); - if (!tmpsym) - continue; - drv->opt_vals[i] = get_sym_value(info, tmpsym); - } - - memset(tmpsymname, 0, 128); - sprintf(tmpsymname, "__%s_pci_tbl_export", drv->name); - - tmpsym = find_sym_in_symtab(info, tmpsymname, NULL); - - - /* - * If this returns NULL, then this is a PMD_VDEV, because - * it has no pci table reference - */ - if (!tmpsym) { - drv->pci_tbl = NULL; - return 0; - } - - tname = get_sym_value(info, tmpsym); - tmpsym = find_sym_in_symtab(info, tname, NULL); - if (!tmpsym) - return -ENOENT; - - drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, tmpsym); - if (!drv->pci_tbl) - return -ENOENT; - - return 0; -} - -static int locate_pmd_entries(struct elf_info *info) -{ - Elf_Sym *last = NULL; - struct pmd_driver *new; - - info->drivers = NULL; - - do { - new = calloc(sizeof(struct pmd_driver), 1); - if (new == NULL) { - fprintf(stderr, "Failed to calloc memory\n"); - return -1; - } - new->name_sym = find_sym_in_symtab(info, "this_pmd_name", last); - last = new->name_sym; - if (!new->name_sym) - free(new); - else { - if (complete_pmd_entry(info, new)) { - fprintf(stderr, - "Failed to complete pmd entry\n"); - free(new); - } else { - new->next = info->drivers; - info->drivers = new; - } - } - } while (last); - - return 0; -} - -static void output_pmd_info_string(struct elf_info *info, char *outfile) -{ - FILE *ofd; - struct pmd_driver *drv; - struct rte_pci_id *pci_ids; - int idx = 0; - - if (use_stdout) - ofd = stdout; - else { - ofd = fopen(outfile, "w+"); - if (!ofd) { - fprintf(stderr, "Unable to open output file\n"); - return; - } - } - - drv = info->drivers; - - while (drv) { - fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = " - "\"PMD_INFO_STRING= {", - drv->name); - fprintf(ofd, "\\\"name\\\" : \\\"%s\\\", ", drv->name); - - for (idx = 0; idx < PMD_OPT_MAX; idx++) { - if (drv->opt_vals[idx]) - fprintf(ofd, "\\\"%s\\\" : \\\"%s\\\", ", - opt_tags[idx].json_id, - drv->opt_vals[idx]); - } - - pci_ids = drv->pci_tbl; - fprintf(ofd, "\\\"pci_ids\\\" : ["); - - while (pci_ids && pci_ids->device_id) { - fprintf(ofd, "[%d, %d, %d, %d]", - pci_ids->vendor_id, pci_ids->device_id, - pci_ids->subsystem_vendor_id, - pci_ids->subsystem_device_id); - pci_ids++; - if (pci_ids->device_id) - fprintf(ofd, ","); - else - fprintf(ofd, " "); - } - fprintf(ofd, "]}\";\n"); - drv = drv->next; - } - - fclose(ofd); -} - -int main(int argc, char **argv) -{ - struct elf_info info = {0}; - int rc = 1; - - if (argc < 3) { - fprintf(stderr, - "usage: %s \n", - basename(argv[0])); - exit(127); - } - use_stdin = !strcmp(argv[1], "-"); - use_stdout = !strcmp(argv[2], "-"); - parse_elf(&info, argv[1]); - - if (locate_pmd_entries(&info) < 0) - exit(1); - - if (info.drivers) { - output_pmd_info_string(&info, argv[2]); - rc = 0; - } else { - fprintf(stderr, "No drivers registered\n"); - } - - parse_elf_finish(&info); - exit(rc); -} diff --git a/buildtools/pmdinfogen/pmdinfogen.h b/buildtools/pmdinfogen/pmdinfogen.h deleted file mode 100644 index 38fd3ddcc..000000000 --- a/buildtools/pmdinfogen/pmdinfogen.h +++ /dev/null @@ -1,119 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 - * Postprocess pmd object files to export hw support - * - * Copyright 2016 Neil Horman - * Based in part on modpost.c from the linux kernel - */ - -#include -#include -#include -#include -#include -#include -#include -#ifdef __linux__ -#include -#else -#include -#endif -#include -#include -#include -#include - -/* On BSD-alike OSes elf.h defines these according to host's word size */ -#undef ELF_ST_BIND -#undef ELF_ST_TYPE -#undef ELF_R_SYM -#undef ELF_R_TYPE - -/* - * Define ELF64_* to ELF_*, the latter being defined in both 32 and 64 bit - * flavors in elf.h. This makes our code a bit more generic between arches - * and allows us to support 32 bit code in the future should we ever want to - */ -#ifdef RTE_ARCH_64 -#define Elf_Ehdr Elf64_Ehdr -#define Elf_Shdr Elf64_Shdr -#define Elf_Sym Elf64_Sym -#define Elf_Addr Elf64_Addr -#define Elf_Sword Elf64_Sxword -#define Elf_Section Elf64_Half -#define ELF_ST_BIND ELF64_ST_BIND -#define ELF_ST_TYPE ELF64_ST_TYPE - -#define Elf_Rel Elf64_Rel -#define Elf_Rela Elf64_Rela -#define ELF_R_SYM ELF64_R_SYM -#define ELF_R_TYPE ELF64_R_TYPE -#else -#define Elf_Ehdr Elf32_Ehdr -#define Elf_Shdr Elf32_Shdr -#define Elf_Sym Elf32_Sym -#define Elf_Addr Elf32_Addr -#define Elf_Sword Elf32_Sxword -#define Elf_Section Elf32_Half -#define ELF_ST_BIND ELF32_ST_BIND -#define ELF_ST_TYPE ELF32_ST_TYPE - -#define Elf_Rel Elf32_Rel -#define Elf_Rela Elf32_Rela -#define ELF_R_SYM ELF32_R_SYM -#define ELF_R_TYPE ELF32_R_TYPE -#endif - - -/* - * Note, it seems odd that we have both a CONVERT_NATIVE and a TO_NATIVE macro - * below. We do this because the values passed to TO_NATIVE may themselves be - * macros and need both macros here to get expanded. Specifically its the width - * variable we are concerned with, because it needs to get expanded prior to - * string concatenation - */ -#define CONVERT_NATIVE(fend, width, x) ({ \ -typeof(x) ___x; \ -if ((fend) == ELFDATA2LSB) \ - ___x = le##width##toh(x); \ -else \ - ___x = be##width##toh(x); \ -___x; \ -}) - -#define TO_NATIVE(fend, width, x) CONVERT_NATIVE(fend, width, x) - -enum opt_params { - PMD_PARAM_STRING = 0, - PMD_KMOD_DEP, - PMD_OPT_MAX -}; - -struct pmd_driver { - Elf_Sym *name_sym; - const char *name; - struct rte_pci_id *pci_tbl; - struct pmd_driver *next; - - const char *opt_vals[PMD_OPT_MAX]; -}; - -struct elf_info { - unsigned long size; - Elf_Ehdr *hdr; - Elf_Shdr *sechdrs; - Elf_Sym *symtab_start; - Elf_Sym *symtab_stop; - char *strtab; - - /* support for 32bit section numbers */ - - unsigned int num_sections; /* max_secindex + 1 */ - unsigned int secindex_strings; - /* if Nth symbol table entry has .st_shndx = SHN_XINDEX, - * take shndx from symtab_shndx_start[N] instead - */ - Elf32_Word *symtab_shndx_start; - Elf32_Word *symtab_shndx_stop; - - struct pmd_driver *drivers; -};