[v1,59/72] net/mlx5/windows: support VF PCI address

Message ID 20201027232335.31427-60-ophirmu@nvidia.com (mailing list archive)
State Superseded, archived
Delegated to: Raslan Darawsheh
Headers
Series mlx5 Windows support - part #5 |

Checks

Context Check Description
ci/checkpatch success coding style OK

Commit Message

Ophir Munk Oct. 27, 2020, 11:23 p.m. UTC
  From: Tal Shnaiderman <talshn@nvidia.com>

Support VF BDF scanning by checking both the BDF and raw BDF provided by
DevX. In Linux a PCI address is formatted as: domain, bus, device,
function (DBDF).  This is right for both a PF and a VF. In Windows a PF
also has a DBDF format, but the domain is always 0, while a VF has a
special "domain" called "Virtual PCI Bus, Serial" (for example: "Virtual
PCI Bus Slot 2 Serial 2") or segment.  The full VF format under Windows
is called raw DBF.  Windows special domain must be considered and DevX
must be called to support it.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 67 ++++++++++++++++++++++++++++++++++++--
 1 file changed, 64 insertions(+), 3 deletions(-)
  

Comments

Narcisa Ana Maria Vasile Nov. 11, 2020, 12:04 a.m. UTC | #1
On Tue, Oct 27, 2020 at 11:23:22PM +0000, Ophir Munk wrote:
> From: Tal Shnaiderman <talshn@nvidia.com>
> 
> Support VF BDF scanning by checking both the BDF and raw BDF provided by
> DevX. In Linux a PCI address is formatted as: domain, bus, device,
> function (DBDF).  This is right for both a PF and a VF. In Windows a PF
> also has a DBDF format, but the domain is always 0, while a VF has a
> special "domain" called "Virtual PCI Bus, Serial" (for example: "Virtual
> PCI Bus Slot 2 Serial 2") or segment.  The full VF format under Windows
> is called raw DBF.  Windows special domain must be considered and DevX
> must be called to support it.
> 
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> ---
>  drivers/net/mlx5/windows/mlx5_os.c | 67 ++++++++++++++++++++++++++++++++++++--
>  1 file changed, 64 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
> index f9b469f..4374b05 100644
> --- a/drivers/net/mlx5/windows/mlx5_os.c
> +++ b/drivers/net/mlx5/windows/mlx5_os.c
> @@ -901,6 +901,68 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
>  }
> + * @return
> + *   1 on Device match, 0 on mismatch, rte_errno code on failure.
> + */
> +static int
> +mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf,
> +				struct rte_pci_addr *addr)
> +{
> +	err = mlx5_glue->query_device(devx_bdf, &mlx5_dev);
> +	if (err) {
> +		DRV_LOG(ERR, "query_device failed");
> +		rte_errno = err;
> +		return 0;

Return rte_errno.

> +	}
> +	if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr))
> +		return 1;
> +	return 0;
> +}
> +
> -- 
> 2.8.4
  
Tal Shnaiderman Nov. 14, 2020, 9:23 p.m. UTC | #2
> Subject: Re: [dpdk-dev] [PATCH v1 59/72] net/mlx5/windows: support VF PCI
> address
> 
> On Tue, Oct 27, 2020 at 11:23:22PM +0000, Ophir Munk wrote:
> > From: Tal Shnaiderman <talshn@nvidia.com>
> >
> > Support VF BDF scanning by checking both the BDF and raw BDF provided
> > by DevX. In Linux a PCI address is formatted as: domain, bus, device,
> > function (DBDF).  This is right for both a PF and a VF. In Windows a
> > PF also has a DBDF format, but the domain is always 0, while a VF has
> > a special "domain" called "Virtual PCI Bus, Serial" (for example:
> > "Virtual PCI Bus Slot 2 Serial 2") or segment.  The full VF format
> > under Windows is called raw DBF.  Windows special domain must be
> > considered and DevX must be called to support it.
> >
> > Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> > ---
> >  drivers/net/mlx5/windows/mlx5_os.c | 67
> > ++++++++++++++++++++++++++++++++++++--
> >  1 file changed, 64 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/windows/mlx5_os.c
> > b/drivers/net/mlx5/windows/mlx5_os.c
> > index f9b469f..4374b05 100644
> > --- a/drivers/net/mlx5/windows/mlx5_os.c
> > +++ b/drivers/net/mlx5/windows/mlx5_os.c
> > @@ -901,6 +901,68 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int
> > enable)  }
> > + * @return
> > + *   1 on Device match, 0 on mismatch, rte_errno code on failure.
> > + */
> > +static int
> > +mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf,
> > +				struct rte_pci_addr *addr)
> > +{
> > +	err = mlx5_glue->query_device(devx_bdf, &mlx5_dev);
> > +	if (err) {
> > +		DRV_LOG(ERR, "query_device failed");
> > +		rte_errno = err;
> > +		return 0;
> 
> Return rte_errno.

Actually updating rte_errno is enough in the function, but the caller need to check it, I'll fix in v2.

> 
> > +	}
> > +	if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr))
> > +		return 1;
> > +	return 0;
> > +}
> > +
> > --
> > 2.8.4
  

Patch

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index f9b469f..4374b05 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -901,6 +901,68 @@  mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
 }
 
 /**
+ * Detect if a devx_device_bdf object has identical DBDF values to the
+ * rte_pci_addr found in bus/pci probing
+ *
+ * @param[in] devx_bdf
+ *   Pointer to the devx_device_bdf structure.
+ * @param[in] addr
+ *   Pointer to the rte_pci_addr structure.
+ *
+ * @return
+ *   1 on Device match, 0 on mismatch.
+ */
+static int
+mlx5_match_devx_bdf_to_addr(struct devx_device_bdf *devx_bdf,
+			    struct rte_pci_addr *addr)
+{
+	if (addr->domain != (devx_bdf->bus_id >> 8) ||
+	    addr->bus != (devx_bdf->bus_id & 0xff) ||
+	    addr->devid != devx_bdf->dev_id ||
+	    addr->function != devx_bdf->fnc_id) {
+		return 0;
+	}
+	return 1;
+}
+
+/**
+ * Detect if a devx_device_bdf object matches the rte_pci_addr
+ * found in bus/pci probing
+ * Compare both the Native/PF BDF and the raw_bdf representing a VF BDF.
+ *
+ * @param[in] devx_bdf
+ *   Pointer to the devx_device_bdf structure.
+ * @param[in] addr
+ *   Pointer to the rte_pci_addr structure.
+ *
+ * @return
+ *   1 on Device match, 0 on mismatch, rte_errno code on failure.
+ */
+static int
+mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf,
+				struct rte_pci_addr *addr)
+{
+	int err;
+	struct devx_device mlx5_dev;
+	if (mlx5_match_devx_bdf_to_addr(devx_bdf, addr))
+		return 1;
+
+	/**
+	 * Didn't match on Native/PF BDF, could still
+	 * Match a VF BDF, check it next
+	 */
+	err = mlx5_glue->query_device(devx_bdf, &mlx5_dev);
+	if (err) {
+		DRV_LOG(ERR, "query_device failed");
+		rte_errno = err;
+		return 0;
+	}
+	if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr))
+		return 1;
+	return 0;
+}
+
+/**
  * DPDK callback to register a PCI device.
  *
  * This function spawns Ethernet devices out of a given PCI device.
@@ -971,9 +1033,8 @@  mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	struct devx_device_bdf *devx_bdf_match[ret + 1];
 
 	while (ret-- > 0) {
-		if (pci_dev->addr.bus != devx_bdf_devs->bus_id ||
-		    pci_dev->addr.devid != devx_bdf_devs->dev_id ||
-		    pci_dev->addr.function != devx_bdf_devs->fnc_id) {
+		if (!mlx5_match_devx_devices_to_addr(devx_bdf_devs,
+		    &pci_dev->addr)) {
 			devx_bdf_devs++;
 			continue;
 		}