Message ID | 20210124205157.14386-1-dmitry.kozliuk@gmail.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 612A9A052A; Sun, 24 Jan 2021 21:52:07 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D6A90140D36; Sun, 24 Jan 2021 21:52:06 +0100 (CET) Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) by mails.dpdk.org (Postfix) with ESMTP id 21CDA140D34 for <dev@dpdk.org>; Sun, 24 Jan 2021 21:52:06 +0100 (CET) Received: by mail-lf1-f50.google.com with SMTP id u25so14902153lfc.2 for <dev@dpdk.org>; Sun, 24 Jan 2021 12:52:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iagWgo1TC1yduCD2bU6B4hyrovn2d6dYHkzmkyyEDTg=; b=fUrsrr1cSMaSC7WnGbSU/oQNuCpj7a5zmjn+7zDR094/FylxEJz+tPUXFeZQWdvhAI TTVP64cQwEB2Ny822xqoq3JOjFYXesXmbMoqNVaQJ3aDf+zGa94U+2U37voI+KcKR4ff TucFh3Ac0WDCb+Ytl1WdsQKsIRFNcXMbSmUpRzcMd14OvqyKW07POZthEwnmO5iK78fK cKWTWEjkJkbtTNXaw1AUNiBgMlVvFW9fbtFvGMk86rLffIyC6fKcxUypgY3XnUZL/gtQ tGzedsqCbSWnW78xZ4bHeWi0VJ94jkP7B+j3lfYxGLNyFpdZw957W0UbIktjT3MLOPGJ r3/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iagWgo1TC1yduCD2bU6B4hyrovn2d6dYHkzmkyyEDTg=; b=AMyBCoWKTtFqftwDiMlJ919av2y5HkK0k95b+3FzvzKLqmTHzGVE9shR7bpecJs0Qa GIt/83z9HBES3lEStAn+iVr1mFK46UhqnMBJaDiQ3xlCXo2IYwBHot5JxTERDLK/+UtS ld8khknCLHR2VTOyiJvgq14bR6bl2o1ksuWab4zdx7Ghfpxj8n4NFLGj4L9S1CUILHr5 UpPQ2Id/SQdfVC3p7cTYMERxqQto+cgV3iw5j7N7+lO0KghsGpDy0uPrJSTsGIgCE0xF y33UovuYe1FzvP7bmeHyH7xGiFvbc6VBNIDgYhtgmaWJez7jQSm5gkxj3lLAlh4w0Qok c6sw== X-Gm-Message-State: AOAM5331DHA4dgnKucZzEmavCutMkmzrcehE04DQydogHZgoBoSfpKPJ lewxq6hguJKlofCNBVo+wR3elT8GLL8hOQ== X-Google-Smtp-Source: ABdhPJzOSXipg+Si0ALBzuZS1YTpkB1ZEC6UCr+/z4Xc3cdqrI9umIcxsrUbRIgtvhg5rA3MFoXgeA== X-Received: by 2002:a19:c8e:: with SMTP id 136mr601402lfm.636.1611521525344; Sun, 24 Jan 2021 12:52:05 -0800 (PST) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id r72sm1168551lff.214.2021.01.24.12.52.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Jan 2021 12:52:04 -0800 (PST) From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> To: dev@dpdk.org Cc: Maxime Coquelin <maxime.coquelin@redhat.com>, Bruce Richardson <bruce.richardson@intel.com>, Thomas Monjalon <thomas@monjalon.net>, Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>, Neil Horman <nhorman@tuxdriver.com>, Jie Zhou <jizh@linux.microsoft.com> Date: Sun, 24 Jan 2021 23:51:54 +0300 Message-Id: <20210124205157.14386-1-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122224327.5621-1-dmitry.kozliuk@gmail.com> References: <20210122224327.5621-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v10 0/3] pmdinfogen: rewrite in Python X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
pmdinfogen: rewrite in Python
|
|
Message
Dmitry Kozlyuk
Jan. 24, 2021, 8:51 p.m. UTC
This patchset implements existing pmdinfogen logic in Python, replaces and removes the old code. The goals of rewriting are: * easier maintenance by using a more high-level language, * simpler build process without host application and libelf, * foundation for adding Windows support. Identity of generated PMD information is checked by comparing output of pmdinfo before and after the patch: find build/drivers -name '*.so' -exec usertools/dpdk-pmdinfo.py Acked-by: Neil Horman <nhorman@tuxdriver.com> Tested-by: Jie Zhou <jizh@linux.microsoft.com> --- Changes in v10: * Suppress ABI warnings for generated strings (Thomas). Dmitry Kozlyuk (3): pmdinfogen: add Python implementation build: use Python pmdinfogen pmdinfogen: remove C implementation .github/workflows/build.yml | 4 +- .travis.yml | 2 +- MAINTAINERS | 3 +- buildtools/gen-pmdinfo-cfile.sh | 6 +- buildtools/meson.build | 15 + buildtools/pmdinfogen.py | 189 +++++++++++ buildtools/pmdinfogen/meson.build | 14 - buildtools/pmdinfogen/pmdinfogen.c | 456 -------------------------- buildtools/pmdinfogen/pmdinfogen.h | 119 ------- devtools/libabigail.abignore | 4 + doc/guides/freebsd_gsg/build_dpdk.rst | 3 +- doc/guides/linux_gsg/sys_reqs.rst | 6 + drivers/meson.build | 2 +- meson.build | 1 - 14 files changed, 225 insertions(+), 599 deletions(-) create mode 100755 buildtools/pmdinfogen.py delete mode 100644 buildtools/pmdinfogen/meson.build delete mode 100644 buildtools/pmdinfogen/pmdinfogen.c delete mode 100644 buildtools/pmdinfogen/pmdinfogen.h
Comments
> Dmitry Kozlyuk (3): > pmdinfogen: add Python implementation > build: use Python pmdinfogen > pmdinfogen: remove C implementation Applied, thanks. Reminder: it is adding a NEW DEPENDENCY on pyelftools, as highlighted already in CI and maintainers meetings: http://mails.dpdk.org/archives/ci/2021-January/000939.html http://mails.dpdk.org/archives/dev/2021-January/197814.html If not already done, please UPDATE your environment! Thanks
Hi Thomas, We have updated all UNH-IOL environments to include pyelftools. Thanks, Brandon On Mon, Jan 25, 2021 at 8:13 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > > Dmitry Kozlyuk (3): > > pmdinfogen: add Python implementation > > build: use Python pmdinfogen > > pmdinfogen: remove C implementation > > Applied, thanks. > > Reminder: it is adding a NEW DEPENDENCY on pyelftools, > as highlighted already in CI and maintainers meetings: > http://mails.dpdk.org/archives/ci/2021-January/000939.html > http://mails.dpdk.org/archives/dev/2021-January/197814.html > > If not already done, please UPDATE your environment! Thanks > >
Hi, > -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Dmitry Kozlyuk > Sent: Sunday, January 24, 2021 10:52 PM > To: dev@dpdk.org > Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; Bruce Richardson > <bruce.richardson@intel.com>; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>; Neil > Horman <nhorman@tuxdriver.com>; Jie Zhou <jizh@linux.microsoft.com> > Subject: [dpdk-dev] [PATCH v10 0/3] pmdinfogen: rewrite in Python > > This patchset implements existing pmdinfogen logic in Python, replaces and > removes the old code. The goals of rewriting are: > > * easier maintenance by using a more high-level language, > * simpler build process without host application and libelf, > * foundation for adding Windows support. > > Identity of generated PMD information is checked by comparing output of > pmdinfo before and after the patch: > > find build/drivers -name '*.so' -exec usertools/dpdk-pmdinfo.py > > Acked-by: Neil Horman <nhorman@tuxdriver.com> > Tested-by: Jie Zhou <jizh@linux.microsoft.com> > > --- Please see the following build errors: https://github.com/DPDK/dpdk/actions/runs/509771118 """ FAILED: drivers/rte_common_cpt.pmd.c /usr/bin/python3 ../buildtools/gen-pmdinfo-cfile.py ar /home/runner/work/dpdk/dpdk/build/drivers/libtmp_rte_common_cpt.a drivers/rte_common_cpt.pmd.c /usr/bin/python3 ../buildtools/pmdinfogen.py elf Traceback (most recent call last): File "../buildtools/gen-pmdinfo-cfile.py", line 14, in <module> [ar, "xv", os.path.abspath(archive)], capture_output=True, check=True, cwd=temp File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: TypeError: __init__() got an unexpected keyword argument 'capture_output' """ I also see the exception in Ubuntu 16.04 (Python 3.5.2, pyelftools 0.27), CentOS 7 (Python 3.6.8, pyelftools 0.27), CentOS 8 (Python 3.6.8, pyelftools 0.27) and OpenSUSE Leap 15.2 (Python 3.6.12, pyelftools 0.26). Thanks, Ali
Hi Ali, > Please see the following build errors: > https://github.com/DPDK/dpdk/actions/runs/509771118 > """ > FAILED: drivers/rte_common_cpt.pmd.c > /usr/bin/python3 ../buildtools/gen-pmdinfo-cfile.py ar /home/runner/work/dpdk/dpdk/build/drivers/libtmp_rte_common_cpt.a drivers/rte_common_cpt.pmd.c /usr/bin/python3 ../buildtools/pmdinfogen.py elf > Traceback (most recent call last): > File "../buildtools/gen-pmdinfo-cfile.py", line 14, in <module> > [ar, "xv", os.path.abspath(archive)], capture_output=True, check=True, cwd=temp > File "/usr/lib/python3.6/subprocess.py", line 423, in run > with Popen(*popenargs, **kwargs) as process: > TypeError: __init__() got an unexpected keyword argument 'capture_output' > """ > > I also see the exception in Ubuntu 16.04 (Python 3.5.2, pyelftools 0.27), CentOS 7 (Python 3.6.8, pyelftools 0.27), CentOS 8 (Python 3.6.8, pyelftools 0.27) and OpenSUSE Leap 15.2 (Python 3.6.12, pyelftools 0.26). If fixed the error you encountered and also discovered a bunch of other issues, all of them hopefully fixed in the following series: http://patchwork.dpdk.org/project/dpdk/list/?series=14938
> Subject: Re: [dpdk-dev] [PATCH v10 0/3] pmdinfogen: rewrite in Python > > External email: Use caution opening links or attachments > > > Hi Thomas, > > We have updated all UNH-IOL environments to include pyelftools. FreeBSD compilation test is still failing on the missing dependency: buildtools/meson.build:45:2: ERROR: Problem encountered: missing python module: elftools > > Thanks, > Brandon > > On Mon, Jan 25, 2021 at 8:13 AM Thomas Monjalon > <thomas@monjalon.net> wrote: > > > > > Dmitry Kozlyuk (3): > > > pmdinfogen: add Python implementation > > > build: use Python pmdinfogen > > > pmdinfogen: remove C implementation > > > > Applied, thanks. > > > > Reminder: it is adding a NEW DEPENDENCY on pyelftools, as highlighted > > already in CI and maintainers meetings: > > > https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails. > dpdk.org%2Farchives%2Fci%2F2021- > January%2F000939.html&data=04%7C01%7Ctalshn%40nvidia.com%7C70 > 5c7ce391d1429bc94508d8c14b9880%7C43083d15727340c1b7db39efd9ccc17a > %7C0%7C0%7C637471877725119466%7CUnknown%7CTWFpbGZsb3d8eyJWIj > oiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1 > 000&sdata=Lei7o0Mtgsb3WZJ14lGolduVJK3fA7EbYuw8ojVCb3g%3D&a > mp;reserved=0 > > > > > https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails > > .dpdk.org%2Farchives%2Fdev%2F2021- > January%2F197814.html&data=04%7C > > > 01%7Ctalshn%40nvidia.com%7C705c7ce391d1429bc94508d8c14b9880%7C430 > 83d15 > > > 727340c1b7db39efd9ccc17a%7C0%7C0%7C637471877725119466%7CUnknow > n%7CTWFp > > > bGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVC > I6Mn > > > 0%3D%7C1000&sdata=74Zp6%2B6YB8OCVfo%2F0Gb7C9C43QWAo1YD4 > AUANneBGdA% > > 3D&reserved=0 > > > > If not already done, please UPDATE your environment! Thanks > > > > > > > -- > > Brandon Lo > > UNH InterOperability Laboratory > > 21 Madbury Rd, Suite 100, Durham, NH 03824 > > blo@iol.unh.edu > > https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww > .iol.unh.edu%2F&data=04%7C01%7Ctalshn%40nvidia.com%7C705c7ce3 > 91d1429bc94508d8c14b9880%7C43083d15727340c1b7db39efd9ccc17a%7C0% > 7C0%7C637471877725119466%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4 > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&a > mp;sdata=2Fc4epSpG%2FhLFdxj9D1kVoafaxlfUhaZvpEm4xwHFd4%3D& > reserved=0