[v3] usertools: fix pmdinfo parsing
Checks
Commit Message
This script inspects an ELF file (binary or shared library) and its
linked dependencies by following DT_NEEDED tags.
So far a simple librte_pmd prefix was used as a filter to only parse
DPDK drivers dependencies.
While the reason is not clear from the commitlog of the patch that
introduced this filter, it was probably added for performance reasons,
since going through all dependencies can be quite long.
Testing with a DPDK built before the driver name changes:
- running the script takes ~0.3s with the filter,
- running the script takes ~9s without the filter,
Now that we changed the driver library names, it becomes more difficult
to identify only DPDK drivers, but we can just filter on the librte_
prefix to identify DPDK libraries: the script later checks for the
PMD_INFO_STRING string in .rodata and it is enough to differentiate the
DPDK drivers from the other DPDK libraries.
A debug message was logged for each inspected file, it gives no useful
information and is removed.
Fixes: a20b2c01a7a1 ("build: standardize component names and defines")
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Robin Jarry <robin.jarry@6wind.com>
---
Changelog since v2:
- revisited the issue and simplified to only filter on librte_ prefix,
Changelog since v1:
- moved driver classes list as a class variable and did some cosmetic
change for readibility,
- used dpdk_driver_classes variable name in the hope that someone changing
meson will catch this script too,
- added bus, common, mempool and raw driver classes as some of them do
carry some pmdinfo stuff and were skipped so far but I found no
indication this skipping was intended,
---
usertools/dpdk-pmdinfo.py | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
Comments
On Wed, Nov 04, 2020 at 04:57:21PM +0100, David Marchand wrote:
> This script inspects an ELF file (binary or shared library) and its
> linked dependencies by following DT_NEEDED tags.
> So far a simple librte_pmd prefix was used as a filter to only parse
> DPDK drivers dependencies.
> While the reason is not clear from the commitlog of the patch that
> introduced this filter, it was probably added for performance reasons,
> since going through all dependencies can be quite long.
> Testing with a DPDK built before the driver name changes:
> - running the script takes ~0.3s with the filter,
> - running the script takes ~9s without the filter,
>
> Now that we changed the driver library names, it becomes more difficult
> to identify only DPDK drivers, but we can just filter on the librte_
> prefix to identify DPDK libraries: the script later checks for the
> PMD_INFO_STRING string in .rodata and it is enough to differentiate the
> DPDK drivers from the other DPDK libraries.
> A debug message was logged for each inspected file, it gives no useful
> information and is removed.
>
> Fixes: a20b2c01a7a1 ("build: standardize component names and defines")
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> Acked-by: Robin Jarry <robin.jarry@6wind.com>
> ---
> Changelog since v2:
> - revisited the issue and simplified to only filter on librte_ prefix,
>
Given you provided some original perf numbers of 0.3 vs 9 seconds above,
rather than leave us all in suspense :-), can you perhaps provide the
numbers for this version too.
/Bruce
On Wed, Nov 4, 2020 at 5:36 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Wed, Nov 04, 2020 at 04:57:21PM +0100, David Marchand wrote:
> > This script inspects an ELF file (binary or shared library) and its
> > linked dependencies by following DT_NEEDED tags.
> > So far a simple librte_pmd prefix was used as a filter to only parse
> > DPDK drivers dependencies.
> > While the reason is not clear from the commitlog of the patch that
> > introduced this filter, it was probably added for performance reasons,
> > since going through all dependencies can be quite long.
> > Testing with a DPDK built before the driver name changes:
> > - running the script takes ~0.3s with the filter,
> > - running the script takes ~9s without the filter,
> >
> > Now that we changed the driver library names, it becomes more difficult
> > to identify only DPDK drivers, but we can just filter on the librte_
> > prefix to identify DPDK libraries: the script later checks for the
> > PMD_INFO_STRING string in .rodata and it is enough to differentiate the
> > DPDK drivers from the other DPDK libraries.
> > A debug message was logged for each inspected file, it gives no useful
> > information and is removed.
> >
> > Fixes: a20b2c01a7a1 ("build: standardize component names and defines")
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > Acked-by: Robin Jarry <robin.jarry@6wind.com>
> > ---
> > Changelog since v2:
> > - revisited the issue and simplified to only filter on librte_ prefix,
> >
>
> Given you provided some original perf numbers of 0.3 vs 9 seconds above,
> rather than leave us all in suspense :-), can you perhaps provide the
> numbers for this version too.
Ah ah, yes, thought I had put it.
I have values in the 0.40/0.50s range.
On Wed, Nov 04, 2020 at 05:48:27PM +0100, David Marchand wrote:
> On Wed, Nov 4, 2020 at 5:36 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > On Wed, Nov 04, 2020 at 04:57:21PM +0100, David Marchand wrote:
> > > This script inspects an ELF file (binary or shared library) and its
> > > linked dependencies by following DT_NEEDED tags.
> > > So far a simple librte_pmd prefix was used as a filter to only parse
> > > DPDK drivers dependencies.
> > > While the reason is not clear from the commitlog of the patch that
> > > introduced this filter, it was probably added for performance reasons,
> > > since going through all dependencies can be quite long.
> > > Testing with a DPDK built before the driver name changes:
> > > - running the script takes ~0.3s with the filter,
> > > - running the script takes ~9s without the filter,
> > >
> > > Now that we changed the driver library names, it becomes more difficult
> > > to identify only DPDK drivers, but we can just filter on the librte_
> > > prefix to identify DPDK libraries: the script later checks for the
> > > PMD_INFO_STRING string in .rodata and it is enough to differentiate the
> > > DPDK drivers from the other DPDK libraries.
> > > A debug message was logged for each inspected file, it gives no useful
> > > information and is removed.
> > >
> > > Fixes: a20b2c01a7a1 ("build: standardize component names and defines")
> > >
> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > > Acked-by: Robin Jarry <robin.jarry@6wind.com>
> > > ---
> > > Changelog since v2:
> > > - revisited the issue and simplified to only filter on librte_ prefix,
> > >
> >
> > Given you provided some original perf numbers of 0.3 vs 9 seconds above,
> > rather than leave us all in suspense :-), can you perhaps provide the
> > numbers for this version too.
>
> Ah ah, yes, thought I had put it.
> I have values in the 0.40/0.50s range.
>
Sounds good.
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
@@ -434,7 +434,6 @@ def process_dt_needed_entries(self):
""" Look to see if there are any DT_NEEDED entries in the binary
And process those if there are
"""
- global raw_output
runpath = ""
ldlibpath = os.environ.get('LD_LIBRARY_PATH')
if ldlibpath is None:
@@ -450,13 +449,11 @@ def process_dt_needed_entries(self):
for tag in dynsec.iter_tags():
# pyelftools may return byte-strings, force decode them
if force_unicode(tag.entry.d_tag) == 'DT_NEEDED':
- if 'librte_pmd' in force_unicode(tag.needed):
+ if 'librte_' in force_unicode(tag.needed):
library = search_file(force_unicode(tag.needed),
runpath + ":" + ldlibpath +
":/usr/lib64:/lib64:/usr/lib:/lib")
if library is not None:
- if raw_output is False:
- print("Scanning %s for pmd information" % library)
with io.open(library, 'rb') as file:
try:
libelf = ReadElf(file, sys.stdout)