[v1,59/72] net/mlx5/windows: support VF PCI address
Checks
Commit Message
From: Tal Shnaiderman <talshn@nvidia.com>
Support VF BDF scanning by checking both the BDF and raw BDF provided by
DevX. In Linux a PCI address is formatted as: domain, bus, device,
function (DBDF). This is right for both a PF and a VF. In Windows a PF
also has a DBDF format, but the domain is always 0, while a VF has a
special "domain" called "Virtual PCI Bus, Serial" (for example: "Virtual
PCI Bus Slot 2 Serial 2") or segment. The full VF format under Windows
is called raw DBF. Windows special domain must be considered and DevX
must be called to support it.
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
drivers/net/mlx5/windows/mlx5_os.c | 67 ++++++++++++++++++++++++++++++++++++--
1 file changed, 64 insertions(+), 3 deletions(-)
Comments
On Tue, Oct 27, 2020 at 11:23:22PM +0000, Ophir Munk wrote:
> From: Tal Shnaiderman <talshn@nvidia.com>
>
> Support VF BDF scanning by checking both the BDF and raw BDF provided by
> DevX. In Linux a PCI address is formatted as: domain, bus, device,
> function (DBDF). This is right for both a PF and a VF. In Windows a PF
> also has a DBDF format, but the domain is always 0, while a VF has a
> special "domain" called "Virtual PCI Bus, Serial" (for example: "Virtual
> PCI Bus Slot 2 Serial 2") or segment. The full VF format under Windows
> is called raw DBF. Windows special domain must be considered and DevX
> must be called to support it.
>
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> ---
> drivers/net/mlx5/windows/mlx5_os.c | 67 ++++++++++++++++++++++++++++++++++++--
> 1 file changed, 64 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
> index f9b469f..4374b05 100644
> --- a/drivers/net/mlx5/windows/mlx5_os.c
> +++ b/drivers/net/mlx5/windows/mlx5_os.c
> @@ -901,6 +901,68 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
> }
> + * @return
> + * 1 on Device match, 0 on mismatch, rte_errno code on failure.
> + */
> +static int
> +mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf,
> + struct rte_pci_addr *addr)
> +{
> + err = mlx5_glue->query_device(devx_bdf, &mlx5_dev);
> + if (err) {
> + DRV_LOG(ERR, "query_device failed");
> + rte_errno = err;
> + return 0;
Return rte_errno.
> + }
> + if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr))
> + return 1;
> + return 0;
> +}
> +
> --
> 2.8.4
> Subject: Re: [dpdk-dev] [PATCH v1 59/72] net/mlx5/windows: support VF PCI
> address
>
> On Tue, Oct 27, 2020 at 11:23:22PM +0000, Ophir Munk wrote:
> > From: Tal Shnaiderman <talshn@nvidia.com>
> >
> > Support VF BDF scanning by checking both the BDF and raw BDF provided
> > by DevX. In Linux a PCI address is formatted as: domain, bus, device,
> > function (DBDF). This is right for both a PF and a VF. In Windows a
> > PF also has a DBDF format, but the domain is always 0, while a VF has
> > a special "domain" called "Virtual PCI Bus, Serial" (for example:
> > "Virtual PCI Bus Slot 2 Serial 2") or segment. The full VF format
> > under Windows is called raw DBF. Windows special domain must be
> > considered and DevX must be called to support it.
> >
> > Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> > ---
> > drivers/net/mlx5/windows/mlx5_os.c | 67
> > ++++++++++++++++++++++++++++++++++++--
> > 1 file changed, 64 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/windows/mlx5_os.c
> > b/drivers/net/mlx5/windows/mlx5_os.c
> > index f9b469f..4374b05 100644
> > --- a/drivers/net/mlx5/windows/mlx5_os.c
> > +++ b/drivers/net/mlx5/windows/mlx5_os.c
> > @@ -901,6 +901,68 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int
> > enable) }
> > + * @return
> > + * 1 on Device match, 0 on mismatch, rte_errno code on failure.
> > + */
> > +static int
> > +mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf,
> > + struct rte_pci_addr *addr)
> > +{
> > + err = mlx5_glue->query_device(devx_bdf, &mlx5_dev);
> > + if (err) {
> > + DRV_LOG(ERR, "query_device failed");
> > + rte_errno = err;
> > + return 0;
>
> Return rte_errno.
Actually updating rte_errno is enough in the function, but the caller need to check it, I'll fix in v2.
>
> > + }
> > + if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr))
> > + return 1;
> > + return 0;
> > +}
> > +
> > --
> > 2.8.4
@@ -901,6 +901,68 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
}
/**
+ * Detect if a devx_device_bdf object has identical DBDF values to the
+ * rte_pci_addr found in bus/pci probing
+ *
+ * @param[in] devx_bdf
+ * Pointer to the devx_device_bdf structure.
+ * @param[in] addr
+ * Pointer to the rte_pci_addr structure.
+ *
+ * @return
+ * 1 on Device match, 0 on mismatch.
+ */
+static int
+mlx5_match_devx_bdf_to_addr(struct devx_device_bdf *devx_bdf,
+ struct rte_pci_addr *addr)
+{
+ if (addr->domain != (devx_bdf->bus_id >> 8) ||
+ addr->bus != (devx_bdf->bus_id & 0xff) ||
+ addr->devid != devx_bdf->dev_id ||
+ addr->function != devx_bdf->fnc_id) {
+ return 0;
+ }
+ return 1;
+}
+
+/**
+ * Detect if a devx_device_bdf object matches the rte_pci_addr
+ * found in bus/pci probing
+ * Compare both the Native/PF BDF and the raw_bdf representing a VF BDF.
+ *
+ * @param[in] devx_bdf
+ * Pointer to the devx_device_bdf structure.
+ * @param[in] addr
+ * Pointer to the rte_pci_addr structure.
+ *
+ * @return
+ * 1 on Device match, 0 on mismatch, rte_errno code on failure.
+ */
+static int
+mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf,
+ struct rte_pci_addr *addr)
+{
+ int err;
+ struct devx_device mlx5_dev;
+ if (mlx5_match_devx_bdf_to_addr(devx_bdf, addr))
+ return 1;
+
+ /**
+ * Didn't match on Native/PF BDF, could still
+ * Match a VF BDF, check it next
+ */
+ err = mlx5_glue->query_device(devx_bdf, &mlx5_dev);
+ if (err) {
+ DRV_LOG(ERR, "query_device failed");
+ rte_errno = err;
+ return 0;
+ }
+ if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr))
+ return 1;
+ return 0;
+}
+
+/**
* DPDK callback to register a PCI device.
*
* This function spawns Ethernet devices out of a given PCI device.
@@ -971,9 +1033,8 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
struct devx_device_bdf *devx_bdf_match[ret + 1];
while (ret-- > 0) {
- if (pci_dev->addr.bus != devx_bdf_devs->bus_id ||
- pci_dev->addr.devid != devx_bdf_devs->dev_id ||
- pci_dev->addr.function != devx_bdf_devs->fnc_id) {
+ if (!mlx5_match_devx_devices_to_addr(devx_bdf_devs,
+ &pci_dev->addr)) {
devx_bdf_devs++;
continue;
}