[v8,00/21] Device querying
mbox series

Message ID cover.1530031921.git.gaetan.rivet@6wind.com
Headers show
Series
  • Device querying
Related show

Message

Gaëtan Rivet June 26, 2018, 4:56 p.m. UTC
This patchset introduces a new EAL API for querying devices,
filtered by arbitrary properties.

The following elements are introduced to this end:

 * A new object, "rte_class", is used to describe
   the device class abstraction layer (eth, crypto, ...).

 * Both rte_bus and rte_class now offer a way to
   list their devices and filter the result
   using locally defined properties.

 * The rte_dev API now has an rte_dev_iterator, which
   is the way for the user to define the device filter
   and iterate upon the resulting set.

As an example, the "eth" device class is implemented.

Additionally, the device filters for

  + rte_bus_pci
  + rte_bus_vdev
  + rte_class_eth

are implemented and can be used with some
properties each, to show how to extend those.

Some example of filters:

  "bus=pci/class=eth"
  "bus=pci"
  "class=eth"
  "class=eth,name=net_ring0"
  "bus=pci,id=00:00.0"
  "bus=vdev,driver=net_ring"

---

v2:

  * Reworked the dev_iterate callback to simplify
    its implementation.

    Now dev_iterate implementation do not need to learn
    about the intricacies of the rte_dev_iterator.
    The rte_dev_iterator is managed purely by the
    rte_dev_iterator_next function. Buses and classes then
    do not have to care about settings things right.

    Additionally, dev_iterate implementations do not
    have to sanitize their dev string anymore, they
    are prepared by the rte_dev layer prior, which also
    reduces the number of dynamic allocations.

v3:

  * Introduced central constructor priority list.
  * Removed lightweight kvarg parsing utility,
    using librte_kvargs instead.
  * Reversed dependencies of librte_kvargs and
    librte_eal.
  * Fixed a few bugs.
  * @Bruce: I have noted the request for meson support.
    I will install it and attempt it once the bulk of the work is done.

v4:

  * Fixed a few bugs, added relevant acks,
    fixed some typos.
  * Made each matching functions actually check for a proper
    list of accepted properties.
  * rte_kvargs now includes rte_eal directly and keeps rte_log.
  * added generic string comparison function to rte_kvargs,
    as some kind of comparison should probably be shared by many layers.

v5:

  * Rebased on master
  * Use strcspn instead of custom function.
  * Introduce private generic rte_eth_dev iterator.
    This could be generalized to other iterators
    (port_next, owner_id-aware, etc).
  * Attempted to support meson.build.
    Got lost in the implicit variables declared
    when inversing dependencies between kvargs and EAL.
    Removed anything related to meson.
  * Postponed genericization of work from
    device query to device declaration.
    Much bigger than anticipated, will let this
    part get in first and iterate over it.

v6:

  * Rebased on master
  * Introduce RTE_PRIORITY_LAST, to explicitly set
    the lowest constructor priority.
  * Fix copyright notice for eth_privage.* files.

v7:

  * Rebased on master
  * Fix rte_kvargs_strcmp return value.
  * Fix layer parsing error
    devstr "bus=pci/onemorelayer" now tells
    that the additional layer is not recognized.

v8:

  * Rebased on master
  * Cleaned kvargs use: introduced
    a new parser function, that simplifies
    using the library for DPDK devargs.
  * Refactored devargs parsing in a single
    function within rte_devargs.
    This function is useful both for rte_dev
    parsing its iterator, and for rte_devargs
    parsing --dev parameters (not yet implemented).
  * A few small bugfixes.

Gaetan Rivet (21):
  devargs: add non-variadic parsing function
  kvargs: build before EAL
  kvargs: introduce a more flexible parsing function
  eal: introduce dtor macros
  eal: introduce device class abstraction
  eal/class: register destructor
  devargs: add function to parse device layers
  eal/dev: add device iterator interface
  eal/class: add device iteration
  eal/bus: add device iteration
  eal/dev: implement device iteration initialization
  eal/dev: implement device iteration
  kvargs: add generic string matching callback
  bus/pci: implement device iteration and comparison
  bus/pci: add device matching field id
  bus/vdev: implement device iteration
  bus/vdev: add device matching field driver
  ethdev: add private generic device iterator
  ethdev: register ether layer as a class
  ethdev: add device matching field name
  app/testpmd: add show device command

 app/test-pmd/cmdline.c                      |  51 +++++++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |  24 +++
 drivers/bus/pci/Makefile                    |   3 +-
 drivers/bus/pci/pci_common.c                |   3 +-
 drivers/bus/pci/pci_params.c                |  78 ++++++++++
 drivers/bus/pci/private.h                   |  25 +++
 drivers/bus/vdev/Makefile                   |   3 +-
 drivers/bus/vdev/vdev.c                     |  10 +-
 drivers/bus/vdev/vdev_params.c              |  62 ++++++++
 drivers/bus/vdev/vdev_private.h             |  26 ++++
 drivers/net/failsafe/failsafe_args.c        |   2 +-
 drivers/net/failsafe/failsafe_eal.c         |   2 +-
 lib/Makefile                                |   5 +-
 lib/librte_eal/bsdapp/eal/Makefile          |   2 +
 lib/librte_eal/common/Makefile              |   2 +-
 lib/librte_eal/common/eal_common_class.c    |  62 ++++++++
 lib/librte_eal/common/eal_common_dev.c      | 227 +++++++++++++++++++++++++++-
 lib/librte_eal/common/eal_common_devargs.c  | 186 +++++++++++++++++++++--
 lib/librte_eal/common/eal_private.h         |  27 ++++
 lib/librte_eal/common/include/rte_bus.h     |   1 +
 lib/librte_eal/common/include/rte_class.h   | 127 ++++++++++++++++
 lib/librte_eal/common/include/rte_common.h  |  24 +++
 lib/librte_eal/common/include/rte_dev.h     |  97 ++++++++++++
 lib/librte_eal/common/include/rte_devargs.h |  53 ++++++-
 lib/librte_eal/linuxapp/eal/Makefile        |   2 +
 lib/librte_eal/rte_eal_version.map          |   5 +
 lib/librte_ethdev/Makefile                  |   4 +-
 lib/librte_ethdev/eth_private.c             |  31 ++++
 lib/librte_ethdev/eth_private.h             |  26 ++++
 lib/librte_ethdev/rte_class_eth.c           |  86 +++++++++++
 lib/librte_ethdev/rte_ethdev.c              |   2 +-
 lib/librte_kvargs/Makefile                  |   2 +-
 lib/librte_kvargs/rte_kvargs.c              |  35 +++++
 lib/librte_kvargs/rte_kvargs.h              |  58 +++++++
 lib/librte_kvargs/rte_kvargs_version.map    |   8 +
 35 files changed, 1328 insertions(+), 33 deletions(-)
 create mode 100644 drivers/bus/pci/pci_params.c
 create mode 100644 drivers/bus/vdev/vdev_params.c
 create mode 100644 drivers/bus/vdev/vdev_private.h
 create mode 100644 lib/librte_eal/common/eal_common_class.c
 create mode 100644 lib/librte_eal/common/include/rte_class.h
 create mode 100644 lib/librte_ethdev/eth_private.c
 create mode 100644 lib/librte_ethdev/eth_private.h
 create mode 100644 lib/librte_ethdev/rte_class_eth.c

Comments

Bruce Richardson June 27, 2018, 10:55 a.m. UTC | #1
On Tue, Jun 26, 2018 at 06:56:03PM +0200, Gaetan Rivet wrote:
> This patchset introduces a new EAL API for querying devices,
> filtered by arbitrary properties.
> 
> The following elements are introduced to this end:
> 
>  * A new object, "rte_class", is used to describe
>    the device class abstraction layer (eth, crypto, ...).
> 
>  * Both rte_bus and rte_class now offer a way to
>    list their devices and filter the result
>    using locally defined properties.
> 
>  * The rte_dev API now has an rte_dev_iterator, which
>    is the way for the user to define the device filter
>    and iterate upon the resulting set.
> 
> As an example, the "eth" device class is implemented.
> 
> Additionally, the device filters for
> 
>   + rte_bus_pci
>   + rte_bus_vdev
>   + rte_class_eth
> 
> are implemented and can be used with some
> properties each, to show how to extend those.
> 
> Some example of filters:
> 
>   "bus=pci/class=eth"
>   "bus=pci"
>   "class=eth"
>   "class=eth,name=net_ring0"
>   "bus=pci,id=00:00.0"
>   "bus=vdev,driver=net_ring"
> 
> ---
> 
> v2:
> 
>   * Reworked the dev_iterate callback to simplify
>     its implementation.
> 
>     Now dev_iterate implementation do not need to learn
>     about the intricacies of the rte_dev_iterator.
>     The rte_dev_iterator is managed purely by the
>     rte_dev_iterator_next function. Buses and classes then
>     do not have to care about settings things right.
> 
>     Additionally, dev_iterate implementations do not
>     have to sanitize their dev string anymore, they
>     are prepared by the rte_dev layer prior, which also
>     reduces the number of dynamic allocations.
> 
> v3:
> 
>   * Introduced central constructor priority list.
>   * Removed lightweight kvarg parsing utility,
>     using librte_kvargs instead.
>   * Reversed dependencies of librte_kvargs and
>     librte_eal.
>   * Fixed a few bugs.
>   * @Bruce: I have noted the request for meson support.
>     I will install it and attempt it once the bulk of the work is done.
> 
> v4:
> 
>   * Fixed a few bugs, added relevant acks,
>     fixed some typos.
>   * Made each matching functions actually check for a proper
>     list of accepted properties.
>   * rte_kvargs now includes rte_eal directly and keeps rte_log.
>   * added generic string comparison function to rte_kvargs,
>     as some kind of comparison should probably be shared by many layers.
> 
> v5:
> 
>   * Rebased on master
>   * Use strcspn instead of custom function.
>   * Introduce private generic rte_eth_dev iterator.
>     This could be generalized to other iterators
>     (port_next, owner_id-aware, etc).
>   * Attempted to support meson.build.
>     Got lost in the implicit variables declared
>     when inversing dependencies between kvargs and EAL.
>     Removed anything related to meson.
>   * Postponed genericization of work from
>     device query to device declaration.
>     Much bigger than anticipated, will let this
>     part get in first and iterate over it.
> 
> v6:
> 
>   * Rebased on master
>   * Introduce RTE_PRIORITY_LAST, to explicitly set
>     the lowest constructor priority.
>   * Fix copyright notice for eth_privage.* files.
> 
> v7:
> 
>   * Rebased on master
>   * Fix rte_kvargs_strcmp return value.
>   * Fix layer parsing error
>     devstr "bus=pci/onemorelayer" now tells
>     that the additional layer is not recognized.
> 
> v8:
> 
>   * Rebased on master
>   * Cleaned kvargs use: introduced
>     a new parser function, that simplifies
>     using the library for DPDK devargs.
>   * Refactored devargs parsing in a single
>     function within rte_devargs.
>     This function is useful both for rte_dev
>     parsing its iterator, and for rte_devargs
>     parsing --dev parameters (not yet implemented).
>   * A few small bugfixes.
> 

Hi Gaetan,

did you test building with shared library builds? I get build failures for
shared libs after applying this set. It appears you may still have a
circular dependency between EAL and kvargs libs.

/Bruce
Gaëtan Rivet June 27, 2018, 11:29 a.m. UTC | #2
On Wed, Jun 27, 2018 at 11:55:02AM +0100, Bruce Richardson wrote:
> On Tue, Jun 26, 2018 at 06:56:03PM +0200, Gaetan Rivet wrote:
> > This patchset introduces a new EAL API for querying devices,
> > filtered by arbitrary properties.
> > 
> > The following elements are introduced to this end:
> > 
> >  * A new object, "rte_class", is used to describe
> >    the device class abstraction layer (eth, crypto, ...).
> > 
> >  * Both rte_bus and rte_class now offer a way to
> >    list their devices and filter the result
> >    using locally defined properties.
> > 
> >  * The rte_dev API now has an rte_dev_iterator, which
> >    is the way for the user to define the device filter
> >    and iterate upon the resulting set.
> > 
> > As an example, the "eth" device class is implemented.
> > 
> > Additionally, the device filters for
> > 
> >   + rte_bus_pci
> >   + rte_bus_vdev
> >   + rte_class_eth
> > 
> > are implemented and can be used with some
> > properties each, to show how to extend those.
> > 
> > Some example of filters:
> > 
> >   "bus=pci/class=eth"
> >   "bus=pci"
> >   "class=eth"
> >   "class=eth,name=net_ring0"
> >   "bus=pci,id=00:00.0"
> >   "bus=vdev,driver=net_ring"
> > 
> > ---
> > 
> > v2:
> > 
> >   * Reworked the dev_iterate callback to simplify
> >     its implementation.
> > 
> >     Now dev_iterate implementation do not need to learn
> >     about the intricacies of the rte_dev_iterator.
> >     The rte_dev_iterator is managed purely by the
> >     rte_dev_iterator_next function. Buses and classes then
> >     do not have to care about settings things right.
> > 
> >     Additionally, dev_iterate implementations do not
> >     have to sanitize their dev string anymore, they
> >     are prepared by the rte_dev layer prior, which also
> >     reduces the number of dynamic allocations.
> > 
> > v3:
> > 
> >   * Introduced central constructor priority list.
> >   * Removed lightweight kvarg parsing utility,
> >     using librte_kvargs instead.
> >   * Reversed dependencies of librte_kvargs and
> >     librte_eal.
> >   * Fixed a few bugs.
> >   * @Bruce: I have noted the request for meson support.
> >     I will install it and attempt it once the bulk of the work is done.
> > 
> > v4:
> > 
> >   * Fixed a few bugs, added relevant acks,
> >     fixed some typos.
> >   * Made each matching functions actually check for a proper
> >     list of accepted properties.
> >   * rte_kvargs now includes rte_eal directly and keeps rte_log.
> >   * added generic string comparison function to rte_kvargs,
> >     as some kind of comparison should probably be shared by many layers.
> > 
> > v5:
> > 
> >   * Rebased on master
> >   * Use strcspn instead of custom function.
> >   * Introduce private generic rte_eth_dev iterator.
> >     This could be generalized to other iterators
> >     (port_next, owner_id-aware, etc).
> >   * Attempted to support meson.build.
> >     Got lost in the implicit variables declared
> >     when inversing dependencies between kvargs and EAL.
> >     Removed anything related to meson.
> >   * Postponed genericization of work from
> >     device query to device declaration.
> >     Much bigger than anticipated, will let this
> >     part get in first and iterate over it.
> > 
> > v6:
> > 
> >   * Rebased on master
> >   * Introduce RTE_PRIORITY_LAST, to explicitly set
> >     the lowest constructor priority.
> >   * Fix copyright notice for eth_privage.* files.
> > 
> > v7:
> > 
> >   * Rebased on master
> >   * Fix rte_kvargs_strcmp return value.
> >   * Fix layer parsing error
> >     devstr "bus=pci/onemorelayer" now tells
> >     that the additional layer is not recognized.
> > 
> > v8:
> > 
> >   * Rebased on master
> >   * Cleaned kvargs use: introduced
> >     a new parser function, that simplifies
> >     using the library for DPDK devargs.
> >   * Refactored devargs parsing in a single
> >     function within rte_devargs.
> >     This function is useful both for rte_dev
> >     parsing its iterator, and for rte_devargs
> >     parsing --dev parameters (not yet implemented).
> >   * A few small bugfixes.
> > 
> 
> Hi Gaetan,
> 
> did you test building with shared library builds? I get build failures for
> shared libs after applying this set. It appears you may still have a
> circular dependency between EAL and kvargs libs.
> 
> /Bruce

Hi Bruce,

Ha, yes, I forgot about that, there is a simple fix, I will update it.
I have other fixes as well, a v9 is on its way.