diff mbox series

[v3] usertools: fix pmdinfo parsing

Message ID 20201104155721.21627-1-david.marchand@redhat.com (mailing list archive)
State Accepted, archived
Delegated to: David Marchand
Headers show
Series [v3] usertools: fix pmdinfo parsing | expand

Checks

Context Check Description
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-intel-Functional fail Functional Testing issues
ci/iol-testing success Testing PASS
ci/iol-broadcom-Functional fail Functional Testing issues
ci/Intel-compilation success Compilation OK
ci/checkpatch success coding style OK

Commit Message

David Marchand Nov. 4, 2020, 3:57 p.m. UTC
This script inspects an ELF file (binary or shared library) and its
linked dependencies by following DT_NEEDED tags.
So far a simple librte_pmd prefix was used as a filter to only parse
DPDK drivers dependencies.
While the reason is not clear from the commitlog of the patch that
introduced this filter, it was probably added for performance reasons,
since going through all dependencies can be quite long.
Testing with a DPDK built before the driver name changes:
- running the script takes ~0.3s with the filter,
- running the script takes ~9s without the filter,

Now that we changed the driver library names, it becomes more difficult
to identify only DPDK drivers, but we can just filter on the librte_
prefix to identify DPDK libraries: the script later checks for the
PMD_INFO_STRING string in .rodata and it is enough to differentiate the
DPDK drivers from the other DPDK libraries.
A debug message was logged for each inspected file, it gives no useful
information and is removed.

Fixes: a20b2c01a7a1 ("build: standardize component names and defines")

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Robin Jarry <robin.jarry@6wind.com>
---
Changelog since v2:
- revisited the issue and simplified to only filter on librte_ prefix,

Changelog since v1:
- moved driver classes list as a class variable and did some cosmetic
  change for readibility,
- used dpdk_driver_classes variable name in the hope that someone changing
  meson will catch this script too,
- added bus, common, mempool and raw driver classes as some of them do
  carry some pmdinfo stuff and were skipped so far but I found no
  indication this skipping was intended,

---
 usertools/dpdk-pmdinfo.py | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

Comments

Bruce Richardson Nov. 4, 2020, 4:35 p.m. UTC | #1
On Wed, Nov 04, 2020 at 04:57:21PM +0100, David Marchand wrote:
> This script inspects an ELF file (binary or shared library) and its
> linked dependencies by following DT_NEEDED tags.
> So far a simple librte_pmd prefix was used as a filter to only parse
> DPDK drivers dependencies.
> While the reason is not clear from the commitlog of the patch that
> introduced this filter, it was probably added for performance reasons,
> since going through all dependencies can be quite long.
> Testing with a DPDK built before the driver name changes:
> - running the script takes ~0.3s with the filter,
> - running the script takes ~9s without the filter,
> 
> Now that we changed the driver library names, it becomes more difficult
> to identify only DPDK drivers, but we can just filter on the librte_
> prefix to identify DPDK libraries: the script later checks for the
> PMD_INFO_STRING string in .rodata and it is enough to differentiate the
> DPDK drivers from the other DPDK libraries.
> A debug message was logged for each inspected file, it gives no useful
> information and is removed.
> 
> Fixes: a20b2c01a7a1 ("build: standardize component names and defines")
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> Acked-by: Robin Jarry <robin.jarry@6wind.com>
> ---
> Changelog since v2:
> - revisited the issue and simplified to only filter on librte_ prefix,
> 

Given you provided some original perf numbers of 0.3 vs 9 seconds above,
rather than leave us all in suspense :-), can you perhaps provide the
numbers for this version too.

/Bruce
David Marchand Nov. 4, 2020, 4:48 p.m. UTC | #2
On Wed, Nov 4, 2020 at 5:36 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Wed, Nov 04, 2020 at 04:57:21PM +0100, David Marchand wrote:
> > This script inspects an ELF file (binary or shared library) and its
> > linked dependencies by following DT_NEEDED tags.
> > So far a simple librte_pmd prefix was used as a filter to only parse
> > DPDK drivers dependencies.
> > While the reason is not clear from the commitlog of the patch that
> > introduced this filter, it was probably added for performance reasons,
> > since going through all dependencies can be quite long.
> > Testing with a DPDK built before the driver name changes:
> > - running the script takes ~0.3s with the filter,
> > - running the script takes ~9s without the filter,
> >
> > Now that we changed the driver library names, it becomes more difficult
> > to identify only DPDK drivers, but we can just filter on the librte_
> > prefix to identify DPDK libraries: the script later checks for the
> > PMD_INFO_STRING string in .rodata and it is enough to differentiate the
> > DPDK drivers from the other DPDK libraries.
> > A debug message was logged for each inspected file, it gives no useful
> > information and is removed.
> >
> > Fixes: a20b2c01a7a1 ("build: standardize component names and defines")
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > Acked-by: Robin Jarry <robin.jarry@6wind.com>
> > ---
> > Changelog since v2:
> > - revisited the issue and simplified to only filter on librte_ prefix,
> >
>
> Given you provided some original perf numbers of 0.3 vs 9 seconds above,
> rather than leave us all in suspense :-), can you perhaps provide the
> numbers for this version too.

Ah ah, yes, thought I had put it.
I have values in the 0.40/0.50s range.
Bruce Richardson Nov. 5, 2020, 11:49 a.m. UTC | #3
On Wed, Nov 04, 2020 at 05:48:27PM +0100, David Marchand wrote:
> On Wed, Nov 4, 2020 at 5:36 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > On Wed, Nov 04, 2020 at 04:57:21PM +0100, David Marchand wrote:
> > > This script inspects an ELF file (binary or shared library) and its
> > > linked dependencies by following DT_NEEDED tags.
> > > So far a simple librte_pmd prefix was used as a filter to only parse
> > > DPDK drivers dependencies.
> > > While the reason is not clear from the commitlog of the patch that
> > > introduced this filter, it was probably added for performance reasons,
> > > since going through all dependencies can be quite long.
> > > Testing with a DPDK built before the driver name changes:
> > > - running the script takes ~0.3s with the filter,
> > > - running the script takes ~9s without the filter,
> > >
> > > Now that we changed the driver library names, it becomes more difficult
> > > to identify only DPDK drivers, but we can just filter on the librte_
> > > prefix to identify DPDK libraries: the script later checks for the
> > > PMD_INFO_STRING string in .rodata and it is enough to differentiate the
> > > DPDK drivers from the other DPDK libraries.
> > > A debug message was logged for each inspected file, it gives no useful
> > > information and is removed.
> > >
> > > Fixes: a20b2c01a7a1 ("build: standardize component names and defines")
> > >
> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > > Acked-by: Robin Jarry <robin.jarry@6wind.com>
> > > ---
> > > Changelog since v2:
> > > - revisited the issue and simplified to only filter on librte_ prefix,
> > >
> >
> > Given you provided some original perf numbers of 0.3 vs 9 seconds above,
> > rather than leave us all in suspense :-), can you perhaps provide the
> > numbers for this version too.
> 
> Ah ah, yes, thought I had put it.
> I have values in the 0.40/0.50s range.
> 
Sounds good.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>
diff mbox series

Patch

diff --git a/usertools/dpdk-pmdinfo.py b/usertools/dpdk-pmdinfo.py
index 1661982791..95fb0111d8 100755
--- a/usertools/dpdk-pmdinfo.py
+++ b/usertools/dpdk-pmdinfo.py
@@ -434,7 +434,6 @@  def process_dt_needed_entries(self):
         """ Look to see if there are any DT_NEEDED entries in the binary
             And process those if there are
         """
-        global raw_output
         runpath = ""
         ldlibpath = os.environ.get('LD_LIBRARY_PATH')
         if ldlibpath is None:
@@ -450,13 +449,11 @@  def process_dt_needed_entries(self):
         for tag in dynsec.iter_tags():
             # pyelftools may return byte-strings, force decode them
             if force_unicode(tag.entry.d_tag) == 'DT_NEEDED':
-                if 'librte_pmd' in force_unicode(tag.needed):
+                if 'librte_' in force_unicode(tag.needed):
                     library = search_file(force_unicode(tag.needed),
                                           runpath + ":" + ldlibpath +
                                           ":/usr/lib64:/lib64:/usr/lib:/lib")
                     if library is not None:
-                        if raw_output is False:
-                            print("Scanning %s for pmd information" % library)
                         with io.open(library, 'rb') as file:
                             try:
                                 libelf = ReadElf(file, sys.stdout)