[dpdk-dev,PATCHv2,4/4] pmdinfo.py: Add tool to query binaries for hw and other support information
Commit Message
This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format
Note that, in the case of dynamically linked applications, pmdinfo.py will scan
for implicitly linked PMDs by searching the specified binaries .dynamic section
for DT_NEEDED entries that contain the substring librte_pmd. The DT_RUNPATH,
LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
order
If a file is specified with no path, it is assumed to be a PMD DSO, and the
LD_LIBRARY_PATH, /usr/lib/ and /lib is searched for it
Currently the tool can output data in 3 formats:
a) raw, suitable for scripting, where the raw JSON strings are dumped out
b) table format (default) where hex pci ids are dumped in a table format
c) pretty, where a user supplied pci.ids file is used to print out vendor and
device strings
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
tools/pmdinfo.py | 457 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 457 insertions(+)
create mode 100755 tools/pmdinfo.py
Comments
On 05/19/2016 12:08 AM, Neil Horman wrote:
> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> and, if found parses the remainder of the string as a json encoded string,
> outputting the results in either a human readable or raw, script parseable
> format
>
> Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> for implicitly linked PMDs by searching the specified binaries .dynamic section
> for DT_NEEDED entries that contain the substring librte_pmd. The DT_RUNPATH,
> LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> order
Scanning /usr/lib and /lib does little good on systems where /usr/lib64
and /lib64 are the standard path, such as x86_64 Fedora / RHEL and
derivates.
With the path changed (or LD_LIBRARY_PATH set manually), I can confirm
it works for a shared binary which is --whole-archive linked to all of
DPDK such as ovs-vswitchd currently is (because it needs to for static
DPDK linkage and is not aware of plugin autoloading).
It doesn't help testpmd though because its not linked with
--whole-archive in the shared case, so its not working for the main DPDK
executable...
In any case, using --whole-archive is a sledgehammer solution at best,
and against the spirit of shared libs and plugins in particular.
I think the shared linkage case can be solved by exporting the PMD path
from librte_eal (either through an elf section or c-level symbol) and
teach the script to detect the case of an executable dynamically linked
to librte_eal, fish the path from there and then process everything in
that path.
>
> If a file is specified with no path, it is assumed to be a PMD DSO, and the
> LD_LIBRARY_PATH, /usr/lib/ and /lib is searched for it
Same as above, /usr/lib/ and /lib is incorrect for a large number of
systems.
>
> Currently the tool can output data in 3 formats:
>
> a) raw, suitable for scripting, where the raw JSON strings are dumped out
> b) table format (default) where hex pci ids are dumped in a table format
> c) pretty, where a user supplied pci.ids file is used to print out vendor and
> device strings
c) is a nice addition. Would be even nicer if it knew the most common
pci.ids locations so it doesn't need extra arguments in those cases, ie
see if /usr/share/hwdata/pci.ids or /usr/share/misc/pci.ids exists and
use that unless overridden on the cli.
- Panu -
On Thu, May 19, 2016 at 12:02:27PM +0300, Panu Matilainen wrote:
> On 05/19/2016 12:08 AM, Neil Horman wrote:
> > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > and, if found parses the remainder of the string as a json encoded string,
> > outputting the results in either a human readable or raw, script parseable
> > format
> >
> > Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> > for implicitly linked PMDs by searching the specified binaries .dynamic section
> > for DT_NEEDED entries that contain the substring librte_pmd. The DT_RUNPATH,
> > LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> > order
>
> Scanning /usr/lib and /lib does little good on systems where /usr/lib64 and
> /lib64 are the standard path, such as x86_64 Fedora / RHEL and derivates.
>
Ah, sorry, forgot the 64 bit variants, I can add those in.
> With the path changed (or LD_LIBRARY_PATH set manually), I can confirm it
> works for a shared binary which is --whole-archive linked to all of DPDK
> such as ovs-vswitchd currently is (because it needs to for static DPDK
> linkage and is not aware of plugin autoloading).
>
Right, thats why it works, because DPDK always requires --whole-archive for
static linking, and likely always will (see commit
20afd76a504155e947c770783ef5023e87136ad8)
> It doesn't help testpmd though because its not linked with --whole-archive
> in the shared case, so its not working for the main DPDK executable...
>
This sentence doesn't make sense --whole-archive is only applicable in the
static binary case, and only when linking archive files.
> In any case, using --whole-archive is a sledgehammer solution at best, and
> against the spirit of shared libs and plugins in particular.
>
It may be a sledgehammer solution, but its the one dpdk uses, and will likely
use in perpituity.
> I think the shared linkage case can be solved by exporting the PMD path from
> librte_eal (either through an elf section or c-level symbol) and teach the
> script to detect the case of an executable dynamically linked to librte_eal,
> fish the path from there and then process everything in that path.
>
I really disagree with this, because its a half-measure at best. Yes, if its
set, you will definately get all the shared objects in that directory loaded,
but that is in no way a guarantee that those are the only libraries that get
loaded (the application may load its own independently). So you're left in this
situation in which you get maybe some of the hardware support an application
offers. Its also transient. That is to say, if you configure a plugin
directory and search it when you scan an application, its contents may change
post scan, leading to erroneous results.
The way I see it, we have 3 cases that we need to handle:
1) Statically linked application - in this case, all pmds that are statically
linked in to the application will be reported, so we're good here
2) Dynamically loaded via DT_NEEDED entries - This is effectively the same as a
static linking case, in that we have a list of libraries that must be resolved
at run time, so we are safe to search for and scan the DSO's that the
application ennumerates
3) Dynamically loaded via dlopen - In this case, we don't actually know until
runtime what DSO's are going to get loaded, even if RTE_EAL_PMD_PATH is set,
because the contents of that path can change at arbitrary times. In this case,
its correct to indicate that the application itself _doesn't_ actually support
the hardware of the PMD's in that path, because until the application is
executed, it has none of the support embodied in any DSO that it loads via
dlopen. The hardware support travels with the DSO itself, and so its correct to
only display hardware support when the PMD shared library itself is scanned.
Handling case 3 the way I'm proposing is exactly the way the OS does it (that is
to say, it only details hardware support for the module being queried, and you
have to specify the module name to get that). I don't see there being any
problem with that.
> >
> > If a file is specified with no path, it is assumed to be a PMD DSO, and the
> > LD_LIBRARY_PATH, /usr/lib/ and /lib is searched for it
>
> Same as above, /usr/lib/ and /lib is incorrect for a large number of
> systems.
>
Yeah, I'll add 64 bit detection and correct that path accordingly
> >
> > Currently the tool can output data in 3 formats:
> >
> > a) raw, suitable for scripting, where the raw JSON strings are dumped out
> > b) table format (default) where hex pci ids are dumped in a table format
> > c) pretty, where a user supplied pci.ids file is used to print out vendor and
> > device strings
>
> c) is a nice addition. Would be even nicer if it knew the most common
> pci.ids locations so it doesn't need extra arguments in those cases, ie see
> if /usr/share/hwdata/pci.ids or /usr/share/misc/pci.ids exists and use that
> unless overridden on the cli.
>
Yeah, I can do that.
> - Panu -
>
>
>
On 05/19/2016 03:00 PM, Neil Horman wrote:
> On Thu, May 19, 2016 at 12:02:27PM +0300, Panu Matilainen wrote:
>> On 05/19/2016 12:08 AM, Neil Horman wrote:
>>> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
>>> and, if found parses the remainder of the string as a json encoded string,
>>> outputting the results in either a human readable or raw, script parseable
>>> format
>>>
>>> Note that, in the case of dynamically linked applications, pmdinfo.py will scan
>>> for implicitly linked PMDs by searching the specified binaries .dynamic section
>>> for DT_NEEDED entries that contain the substring librte_pmd. The DT_RUNPATH,
>>> LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
>>> order
>>
>> Scanning /usr/lib and /lib does little good on systems where /usr/lib64 and
>> /lib64 are the standard path, such as x86_64 Fedora / RHEL and derivates.
>>
> Ah, sorry, forgot the 64 bit variants, I can add those in.
>
>> With the path changed (or LD_LIBRARY_PATH set manually), I can confirm it
>> works for a shared binary which is --whole-archive linked to all of DPDK
>> such as ovs-vswitchd currently is (because it needs to for static DPDK
>> linkage and is not aware of plugin autoloading).
>>
> Right, thats why it works, because DPDK always requires --whole-archive for
> static linking, and likely always will (see commit
> 20afd76a504155e947c770783ef5023e87136ad8)
>
>> It doesn't help testpmd though because its not linked with --whole-archive
>> in the shared case, so its not working for the main DPDK executable...
>>
> This sentence doesn't make sense --whole-archive is only applicable in the
> static binary case, and only when linking archive files.
Okay sorry I was indeed mixing up things here.
1) Testpmd doesn't get linked to those pmds at all, because of this in
mk/rte.app.mk:
ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
# plugins (link only if static libraries)
...
endif
2) I'm could swear I've seen --whole-archive it make a difference on
newer toolchains with the linker script, but I can't reproduce that now
no matter what. Must've been hallucinating ;)
>
>> In any case, using --whole-archive is a sledgehammer solution at best, and
>> against the spirit of shared libs and plugins in particular.
>>
> It may be a sledgehammer solution, but its the one dpdk uses, and will likely
> use in perpituity.
>
>> I think the shared linkage case can be solved by exporting the PMD path from
>> librte_eal (either through an elf section or c-level symbol) and teach the
>> script to detect the case of an executable dynamically linked to librte_eal,
>> fish the path from there and then process everything in that path.
>>
> I really disagree with this, because its a half-measure at best. Yes, if its
> set, you will definately get all the shared objects in that directory loaded,
> but that is in no way a guarantee that those are the only libraries that get
> loaded (the application may load its own independently).
That is in no way different from statically linked apps loading
additional drivers with the EAL -d switch.
> So you're left in this
> situation in which you get maybe some of the hardware support an application
> offers. Its also transient. That is to say, if you configure a plugin
> directory and search it when you scan an application, its contents may change
> post scan, leading to erroneous results.
Yes, changing system state such as installing or removing packages
between scanning and running can "do stuff" such as change results.
People are quite aware of this because that's how a huge number of
things in the system works, you install plugins for multimedia formats
you need, maybe remove others you dont to clean up system etc. I fail to
see how that is a problem when it's the expected behavior with plugins.
> The way I see it, we have 3 cases that we need to handle:
>
> 1) Statically linked application - in this case, all pmds that are statically
> linked in to the application will be reported, so we're good here
>
> 2) Dynamically loaded via DT_NEEDED entries - This is effectively the same as a
> static linking case, in that we have a list of libraries that must be resolved
> at run time, so we are safe to search for and scan the DSO's that the
> application ennumerates
>
> 3) Dynamically loaded via dlopen - In this case, we don't actually know until
> runtime what DSO's are going to get loaded, even if RTE_EAL_PMD_PATH is set,
> because the contents of that path can change at arbitrary times. In this case,
> its correct to indicate that the application itself _doesn't_ actually support
> the hardware of the PMD's in that path, because until the application is
> executed, it has none of the support embodied in any DSO that it loads via
> dlopen. The hardware support travels with the DSO itself, and so its correct to
> only display hardware support when the PMD shared library itself is scanned.
>
> Handling case 3 the way I'm proposing is exactly the way the OS does it (that is
> to say, it only details hardware support for the module being queried, and you
> have to specify the module name to get that). I don't see there being any
> problem with that.
Ability to query individual DSOs is a building block for other things
like automation (you dont expect normal users to go manually loading hw
support modules for the OS either), but its not an end-user solution.
Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:
"This tool should not behave differently depending of how DPDK was
compiled (static or shared)."
Its entirely possible to handle all the three above cases virtually
identically (point the tool to the app executable), so I have a hard
time understanding the level of resistance to handling the plugin case.
- Panu -
>
>>>
>>> If a file is specified with no path, it is assumed to be a PMD DSO, and the
>>> LD_LIBRARY_PATH, /usr/lib/ and /lib is searched for it
>>
>> Same as above, /usr/lib/ and /lib is incorrect for a large number of
>> systems.
>>
> Yeah, I'll add 64 bit detection and correct that path accordingly
>
>>>
>>> Currently the tool can output data in 3 formats:
>>>
>>> a) raw, suitable for scripting, where the raw JSON strings are dumped out
>>> b) table format (default) where hex pci ids are dumped in a table format
>>> c) pretty, where a user supplied pci.ids file is used to print out vendor and
>>> device strings
>>
>> c) is a nice addition. Would be even nicer if it knew the most common
>> pci.ids locations so it doesn't need extra arguments in those cases, ie see
>> if /usr/share/hwdata/pci.ids or /usr/share/misc/pci.ids exists and use that
>> unless overridden on the cli.
>>
> Yeah, I can do that.
>
>> - Panu -
>>
>>
>>
2016-05-20 08:22, Panu Matilainen:
> Ability to query individual DSOs is a building block for other things
> like automation (you dont expect normal users to go manually loading hw
> support modules for the OS either), but its not an end-user solution.
>
> Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:
>
> "This tool should not behave differently depending of how DPDK was
> compiled (static or shared)."
I meant the basic tool must be usable on static binary and shared library.
> Its entirely possible to handle all the three above cases virtually
> identically (point the tool to the app executable), so I have a hard
> time understanding the level of resistance to handling the plugin case.
We need first a basic tool. Then we'll check how to build on it for
end user needs. I think you have some good ideas.
We could also try (later) to inspect a running DPDK app.
On 05/20/2016 11:55 AM, Thomas Monjalon wrote:
> 2016-05-20 08:22, Panu Matilainen:
>> Ability to query individual DSOs is a building block for other things
>> like automation (you dont expect normal users to go manually loading hw
>> support modules for the OS either), but its not an end-user solution.
>>
>> Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:
>>
>> "This tool should not behave differently depending of how DPDK was
>> compiled (static or shared)."
>
> I meant the basic tool must be usable on static binary and shared library.
Well, I'm not sure I would interpret that any differently in terms of
what to expect. But since its all up to interpretation and different
points of view...
>> Its entirely possible to handle all the three above cases virtually
>> identically (point the tool to the app executable), so I have a hard
>> time understanding the level of resistance to handling the plugin case.
>
> We need first a basic tool. Then we'll check how to build on it for
> end user needs. I think you have some good ideas.
> We could also try (later) to inspect a running DPDK app.
>
...I'll just say fair enough, I've made my points and I'll go shut up
for now.
- Panu -
On Fri, May 20, 2016 at 08:22:34AM +0300, Panu Matilainen wrote:
> On 05/19/2016 03:00 PM, Neil Horman wrote:
> > On Thu, May 19, 2016 at 12:02:27PM +0300, Panu Matilainen wrote:
> > > On 05/19/2016 12:08 AM, Neil Horman wrote:
> > > > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > > > and, if found parses the remainder of the string as a json encoded string,
> > > > outputting the results in either a human readable or raw, script parseable
> > > > format
> > > >
> > > > Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> > > > for implicitly linked PMDs by searching the specified binaries .dynamic section
> > > > for DT_NEEDED entries that contain the substring librte_pmd. The DT_RUNPATH,
> > > > LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> > > > order
> > >
> > > Scanning /usr/lib and /lib does little good on systems where /usr/lib64 and
> > > /lib64 are the standard path, such as x86_64 Fedora / RHEL and derivates.
> > >
> > Ah, sorry, forgot the 64 bit variants, I can add those in.
> >
> > > With the path changed (or LD_LIBRARY_PATH set manually), I can confirm it
> > > works for a shared binary which is --whole-archive linked to all of DPDK
> > > such as ovs-vswitchd currently is (because it needs to for static DPDK
> > > linkage and is not aware of plugin autoloading).
> > >
> > Right, thats why it works, because DPDK always requires --whole-archive for
> > static linking, and likely always will (see commit
> > 20afd76a504155e947c770783ef5023e87136ad8)
> >
> > > It doesn't help testpmd though because its not linked with --whole-archive
> > > in the shared case, so its not working for the main DPDK executable...
> > >
> > This sentence doesn't make sense --whole-archive is only applicable in the
> > static binary case, and only when linking archive files.
>
> Okay sorry I was indeed mixing up things here.
>
> 1) Testpmd doesn't get linked to those pmds at all, because of this in
> mk/rte.app.mk:
>
> ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
> # plugins (link only if static libraries)
> ...
> endif
>
> 2) I'm could swear I've seen --whole-archive it make a difference on newer
> toolchains with the linker script, but I can't reproduce that now no matter
> what. Must've been hallucinating ;)
>
I think so. DSO's don't really mesh with the concept of --whole-archive because
they automatically keep their entire contents (since you won't know what symbols
a given application will use at run time)
> >
> > > In any case, using --whole-archive is a sledgehammer solution at best, and
> > > against the spirit of shared libs and plugins in particular.
> > >
> > It may be a sledgehammer solution, but its the one dpdk uses, and will likely
> > use in perpituity.
> >
> > > I think the shared linkage case can be solved by exporting the PMD path from
> > > librte_eal (either through an elf section or c-level symbol) and teach the
> > > script to detect the case of an executable dynamically linked to librte_eal,
> > > fish the path from there and then process everything in that path.
> > >
> > I really disagree with this, because its a half-measure at best. Yes, if its
> > set, you will definately get all the shared objects in that directory loaded,
> > but that is in no way a guarantee that those are the only libraries that get
> > loaded (the application may load its own independently).
>
> That is in no way different from statically linked apps loading additional
> drivers with the EAL -d switch.
>
Sooo....we agree?
> > So you're left in this
> > situation in which you get maybe some of the hardware support an application
> > offers. Its also transient. That is to say, if you configure a plugin
> > directory and search it when you scan an application, its contents may change
> > post scan, leading to erroneous results.
>
> Yes, changing system state such as installing or removing packages between
> scanning and running can "do stuff" such as change results. People are quite
> aware of this because that's how a huge number of things in the system
> works, you install plugins for multimedia formats you need, maybe remove
> others you dont to clean up system etc. I fail to see how that is a problem
> when it's the expected behavior with plugins.
>
> > The way I see it, we have 3 cases that we need to handle:
> >
> > 1) Statically linked application - in this case, all pmds that are statically
> > linked in to the application will be reported, so we're good here
> >
> > 2) Dynamically loaded via DT_NEEDED entries - This is effectively the same as a
> > static linking case, in that we have a list of libraries that must be resolved
> > at run time, so we are safe to search for and scan the DSO's that the
> > application ennumerates
> >
> > 3) Dynamically loaded via dlopen - In this case, we don't actually know until
> > runtime what DSO's are going to get loaded, even if RTE_EAL_PMD_PATH is set,
> > because the contents of that path can change at arbitrary times. In this case,
> > its correct to indicate that the application itself _doesn't_ actually support
> > the hardware of the PMD's in that path, because until the application is
> > executed, it has none of the support embodied in any DSO that it loads via
> > dlopen. The hardware support travels with the DSO itself, and so its correct to
> > only display hardware support when the PMD shared library itself is scanned.
> >
> > Handling case 3 the way I'm proposing is exactly the way the OS does it (that is
> > to say, it only details hardware support for the module being queried, and you
> > have to specify the module name to get that). I don't see there being any
> > problem with that.
>
> Ability to query individual DSOs is a building block for other things like
> automation (you dont expect normal users to go manually loading hw support
> modules for the OS either), but its not an end-user solution.
>
> Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:
>
> "This tool should not behave differently depending of how DPDK was compiled
> (static or shared)."
>
> Its entirely possible to handle all the three above cases virtually
> identically (point the tool to the app executable), so I have a hard time
> understanding the level of resistance to handling the plugin case.
>
Because you're applying some meaning to the idea that Thomas listed, and
it simply doesn't work like that. Look, try it by using this statement:
"pmdinfo.py lists hardware support that an application has at run time"
Using that statement:
1) Any pmd that is statically linked into the application is guaranteed to be
present at run time, and so is reported
2) Any pmd that is dynamically linked at link time is guarateed to be available
at run time (because the DT_NEEDED entry implies that said pmd has to be loaded
for the application to run), and so the hw support is reported
3) Any pmd that is dynamically loaded via the the RTE_EAL_PMD_PATH or the -d
command line option is _not_ guaranteed to be there at run time, and so is not
reported.
The only reason that your argument about automatically scanning stuff in the
plugin directory makes sense is because you're enforcing the assumption that the
contents of that directory remain unchanged between scan time and run time. If
thats true great, if not, we get incorrect resulsts. And we don't address the
-d option at all.
If you want to make the consistency argument, I assert that what we have now is
in fact the most consistent solution.
Please see my elarier email for a compromise solution I'm willing to implement
Neil
On Fri, May 20, 2016 at 10:55:26AM +0200, Thomas Monjalon wrote:
> 2016-05-20 08:22, Panu Matilainen:
> > Ability to query individual DSOs is a building block for other things
> > like automation (you dont expect normal users to go manually loading hw
> > support modules for the OS either), but its not an end-user solution.
> >
> > Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:
> >
> > "This tool should not behave differently depending of how DPDK was
> > compiled (static or shared)."
>
> I meant the basic tool must be usable on static binary and shared library.
>
> > Its entirely possible to handle all the three above cases virtually
> > identically (point the tool to the app executable), so I have a hard
> > time understanding the level of resistance to handling the plugin case.
>
> We need first a basic tool. Then we'll check how to build on it for
> end user needs. I think you have some good ideas.
> We could also try (later) to inspect a running DPDK app.
>
I've thought about that, and its actually pretty easily doable. If we move the
driver registration list to a shared memory segment, we could just attach to it
and read it.
Neil
new file mode 100755
@@ -0,0 +1,457 @@
+#!/usr/bin/python
+#-------------------------------------------------------------------------------
+# scripts/pmd_hw_support.py
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+#-------------------------------------------------------------------------------
+import os, sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+ ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+ GNUVerSymSection, GNUVerDefSection,
+ GNUVerNeedSection,
+ )
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+ describe_ei_class, describe_ei_data, describe_ei_version,
+ describe_ei_osabi, describe_e_type, describe_e_machine,
+ describe_e_version_numeric, describe_p_type, describe_p_flags,
+ describe_sh_type, describe_sh_flags,
+ describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+ describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+ describe_ver_flags,
+ )
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+ describe_reg_name, describe_attr_value, set_global_machine_arch,
+ describe_CFI_instructions, describe_CFI_register_rule,
+ describe_CFI_CFA_rule,
+ )
+from elftools.dwarf.constants import (
+ DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False
+pcidb = None
+
+#===========================================
+
+class Vendor:
+ """
+ Class for vendors. This is the top level class
+ for the devices belong to a specific vendor.
+ self.devices is the device dictionary
+ subdevices are in each device.
+ """
+ def __init__(self, vendorStr):
+ """
+ Class initializes with the raw line from pci.ids
+ Parsing takes place inside __init__
+ """
+ self.ID = vendorStr.split()[0]
+ self.name = vendorStr.replace("%s " % self.ID,"").rstrip()
+ self.devices = {}
+
+ def addDevice(self, deviceStr):
+ """
+ Adds a device to self.devices
+ takes the raw line from pci.ids
+ """
+ s = deviceStr.strip()
+ devID = s.split()[0]
+ if devID in self.devices:
+ pass
+ else:
+ self.devices[devID] = Device(deviceStr)
+
+ def report(self):
+ print self.ID, self.name
+ for id, dev in self.devices.items():
+ dev.report()
+
+ def find_device(self, devid):
+ # convert to a hex string and remove 0x
+ devid = hex(devid)[2:]
+ try:
+ return self.devices[devid]
+ except:
+ return Device("%s (Unknown Device %s)" % (devid, devid))
+
+class Device:
+ def __init__(self, deviceStr):
+ """
+ Class for each device.
+ Each vendor has its own devices dictionary.
+ """
+ s = deviceStr.strip()
+ self.ID = s.split()[0]
+ self.name = s.replace("%s " % self.ID,"")
+ self.subdevices = {}
+
+ def report(self):
+ print "\t%s\t%s" % (self.ID, self.name)
+ for subID, subdev in self.subdevices.items():
+ subdev.report()
+
+ def addSubDevice(self, subDeviceStr):
+ """
+ Adds a subvendor, subdevice to device.
+ Uses raw line from pci.ids
+ """
+ s = subDeviceStr.strip()
+ spl = s.split()
+ subVendorID = spl[0]
+ subDeviceID = spl[1]
+ subDeviceName = s.split(" ")[-1]
+ devID = "%s:%s" % (subVendorID,subDeviceID)
+ self.subdevices[devID] = SubDevice(subVendorID,subDeviceID,subDeviceName)
+
+ def find_subid(self, subven, subdev):
+ subven = hex(subven)[2:]
+ subdev = hex(subdev)[2:]
+ devid ="%s:%s" % (subven, subdev)
+
+ try:
+ return self.subdevices[devid]
+ except:
+ if (subven == "ffff" and subdev == "ffff"):
+ return SubDevice("ffff", "ffff", "(All Subdevices)");
+ else:
+ return SubDevice(subven, subdev, "(Unknown Subdevice)")
+
+
+class SubDevice:
+ """
+ Class for subdevices.
+ """
+ def __init__(self, vendor, device, name):
+ """
+ Class initializes with vendorid, deviceid and name
+ """
+ self.vendorID = vendor
+ self.deviceID = device
+ self.name = name
+
+ def report(self):
+ print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID,self.name)
+
+class PCIIds:
+ """
+ Top class for all pci.ids entries.
+ All queries will be asked to this class.
+ PCIIds.vendors["0e11"].devices["0046"].subdevices["0e11:4091"].name = "Smart Array 6i"
+ """
+ def __init__(self, filename):
+ """
+ Prepares the directories.
+ Checks local data file.
+ Tries to load from local, if not found, downloads from web
+ """
+ self.version = ""
+ self.date = ""
+ self.vendors = {}
+ self.contents = None
+ self.readLocal(filename)
+ self.parse()
+
+ def reportVendors(self):
+ """Reports the vendors
+ """
+ for vid, v in self.vendors.items():
+ print v.ID, v.name
+
+ def report(self, vendor = None):
+ """
+ Reports everything for all vendors or a specific vendor
+ PCIIds.report() reports everything
+ PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
+ """
+ if vendor != None:
+ self.vendors[vendor].report()
+ else:
+ for vID, v in self.vendors.items():
+ v.report()
+
+ def find_vendor(self, vid):
+ # convert vid to a hex string and remove the 0x
+ vid = hex(vid)[2:]
+
+ try:
+ return self.vendors[vid]
+ except:
+ return Vendor("%s (Unknown Vendor %s)" % (vid, vid))
+
+ def findDate(self, content):
+ for l in content:
+ if l.find("Date:") > -1:
+ return l.split()[-2].replace("-", "")
+ return None
+
+ def parse(self):
+ if len(self.contents) < 1:
+ print "data/%s-pci.ids not found" % self.date
+ else:
+ vendorID = ""
+ deviceID = ""
+ for l in self.contents:
+ if l[0] == "#":
+ continue
+ elif len(l.strip()) == 0:
+ continue
+ else:
+ if l.find("\t\t") == 0:
+ self.vendors[vendorID].devices[deviceID].addSubDevice(l)
+ elif l.find("\t") == 0:
+ deviceID = l.strip().split()[0]
+ self.vendors[vendorID].addDevice(l)
+ else:
+ vendorID = l.split()[0]
+ self.vendors[vendorID] = Vendor(l)
+
+ def readLocal(self, filename):
+ """
+ Reads the local file
+ """
+ self.contents = open(filename).readlines()
+ self.date = self.findDate(self.contents)
+
+ def loadLocal(self):
+ """
+ Loads database from local. If there is no file,
+ it creates a new one from web
+ """
+ self.date = idsfile[0].split("/")[1].split("-")[0]
+ self.readLocal()
+
+
+
+#=======================================
+
+def search_file(filename, search_path):
+ """ Given a search path, find file with requested name """
+ for path in string.split(search_path, ":"):
+ candidate = os.path.join(path, filename)
+ if os.path.exists(candidate): return os.path.abspath(candidate)
+ return None
+
+class ReadElf(object):
+ """ display_* methods are used to emit output into the output stream
+ """
+ def __init__(self, file, output):
+ """ file:
+ stream object with the ELF file to read
+
+ output:
+ output stream to write to
+ """
+ self.elffile = ELFFile(file)
+ self.output = output
+
+ # Lazily initialized if a debug dump is requested
+ self._dwarfinfo = None
+
+ self._versioninfo = None
+
+ def _section_from_spec(self, spec):
+ """ Retrieve a section given a "spec" (either number or name).
+ Return None if no such section exists in the file.
+ """
+ try:
+ num = int(spec)
+ if num < self.elffile.num_sections():
+ return self.elffile.get_section(num)
+ else:
+ return None
+ except ValueError:
+ # Not a number. Must be a name then
+ return self.elffile.get_section_by_name(str2bytes(spec))
+
+ def pretty_print_pmdinfo(self, pmdinfo):
+ global pcidb
+
+ for i in pmdinfo["pci_ids"]:
+ vendor = pcidb.find_vendor(i[0])
+ device = vendor.find_device(i[1])
+ subdev = device.find_subid(i[2], i[3])
+ print("%s (%s) : %s (%s) %s" % (vendor.name, vendor.ID, device.name, device.ID, subdev.name))
+
+ def parse_pmd_info_string(self, mystring):
+ global raw_output
+ global pcidb
+
+ optional_pmd_info = [{'id': 'params', 'tag' : 'PMD PARAMETERS'}]
+
+ i = mystring.index("=");
+ mystring = mystring[i+2:]
+ pmdinfo = json.loads(mystring)
+
+ if raw_output:
+ print(pmdinfo)
+ return
+
+ print("PMD NAME: " + pmdinfo["name"])
+ print("PMD TYPE: " + pmdinfo["type"])
+ for i in optional_pmd_info:
+ try:
+ print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
+ except KeyError as e:
+ continue
+
+ if (pmdinfo["type"] == "PMD_PDEV"):
+ print("PMD HW SUPPORT:")
+ if pcidb != None:
+ self.pretty_print_pmdinfo(pmdinfo)
+ else:
+ print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+ for i in pmdinfo["pci_ids"]:
+ print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" % (i[0], i[1], i[2], i[3]))
+
+ print("")
+
+
+ def display_pmd_info_strings(self, section_spec):
+ """ Display a strings dump of a section. section_spec is either a
+ section number or a name.
+ """
+ section = self._section_from_spec(section_spec)
+ if section is None:
+ return
+
+
+ data = section.data()
+ dataptr = 0
+
+ while dataptr < len(data):
+ while ( dataptr < len(data) and
+ not (32 <= byte2int(data[dataptr]) <= 127)):
+ dataptr += 1
+
+ if dataptr >= len(data):
+ break
+
+ endptr = dataptr
+ while endptr < len(data) and byte2int(data[endptr]) != 0:
+ endptr += 1
+
+ mystring = bytes2str(data[dataptr:endptr])
+ rc = mystring.find("PMD_INFO_STRING")
+ if (rc != -1):
+ self.parse_pmd_info_string(mystring)
+
+ dataptr = endptr
+
+ def get_dt_runpath(self, dynsec):
+ for tag in dynsec.iter_tags():
+ if tag.entry.d_tag == 'DT_RUNPATH':
+ return tag.runpath
+ return ""
+
+
+ def process_dt_needed_entries(self):
+ """ Look to see if there are any DT_NEEDED entries in the binary
+ And process those if there are
+ """
+ global raw_output
+ runpath = ""
+ ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+ if (ldlibpath == None):
+ ldlibpath = ""
+
+ dynsec = self._section_from_spec(".dynamic")
+ try:
+ runpath = self.get_dt_runpath(dynsec)
+ except AttributeError:
+ # dynsec is None, just return
+ return
+
+ for tag in dynsec.iter_tags():
+ if tag.entry.d_tag == 'DT_NEEDED':
+ rc = tag.needed.find("librte_pmd")
+ if (rc != -1):
+ library = search_file(tag.needed,
+ runpath + ":" + ldlibpath +
+ ":/usr/lib:/lib")
+ if (library != None):
+ if (raw_output == False):
+ print("Scanning %s for pmd information" % library)
+ with open(library, 'rb') as file:
+ libelf = ReadElf(file, sys.stdout)
+ libelf.process_dt_needed_entries()
+ libelf.display_pmd_info_strings(".rodata")
+ file.close()
+
+
+def main(stream=None):
+ global raw_output
+ global pcidb
+
+ optparser = OptionParser(
+ usage='usage: %prog [-h|-r] <elf-file>',
+ description="Dump pmd hardware support info",
+ add_help_option=True,
+ prog='pmdinfo.py')
+ optparser.add_option('-r', '--raw',
+ action='store_true', dest='raw_output',
+ help='Dump raw json strings')
+ optparser.add_option("-d", "--pcidb", dest="pcifile",
+ help="", metavar="FILE")
+
+ options, args = optparser.parse_args()
+
+ if options.raw_output:
+ raw_output = True
+
+ if options.pcifile:
+ pcidb = PCIIds(options.pcifile)
+ if pcidb == None:
+ print("Pci DB file not found")
+ exit(1)
+
+ ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+ if (ldlibpath == None):
+ ldlibpath = ""
+
+ if (os.path.exists(args[0]) == True):
+ myelffile = args[0]
+ else:
+ myelffile = search_file(args[0], ldlibpath + ":/usr/lib:/lib")
+
+ if (myelffile == None):
+ print("File not found")
+ sys.exit(1)
+
+ with open(myelffile, 'rb') as file:
+ try:
+ readelf = ReadElf(file, sys.stdout)
+ readelf.process_dt_needed_entries()
+ readelf.display_pmd_info_strings(".rodata")
+ sys.exit(0)
+
+ except ELFError as ex:
+ sys.stderr.write('ELF error: %s\n' % ex)
+ sys.exit(1)
+
+
+#-------------------------------------------------------------------------------
+if __name__ == '__main__':
+ main()
+
+