diff mbox series

[v5,1/5] ethdev: add hairpin bind and unbind APIs

Message ID 1602740124-397688-2-git-send-email-bingz@nvidia.com (mailing list archive)
State Superseded, archived
Delegated to: Ferruh Yigit
Headers show
Series introduce support for hairpin between two ports | expand

Checks

Context Check Description
ci/checkpatch success coding style OK

Commit Message

Bing Zhao Oct. 15, 2020, 5:35 a.m. UTC
In single port hairpin mode, all the hairpin TX and RX queues belong
to the same device. After the queues are set up properly, there is
no other dependency between the TX queue and its RX peer queue. The
binding process that connected the TX and RX queues together from
hardware level will be done automatically during the device start
procedure. Everything required is configured and initialized already
for the binding process.

But in two ports hairpin mode, there will be some cross-dependences
between two different ports. Usually, the ports will be initialized
serially by the main thread but not in parallel. The earlier port
will not be able to enable the bind if the following peer port is
not yet configured with HW resources. What's more, if one port is
detached / attached dynamically, it would introduce more trouble
for the hairpin binding.

To overcome these, new APIs for binding and unbinding are added.
During startup, only the hairpin TX and RX peer queues will be set
up. Nothing will be done when starting the device if the queues are
without auto-bind attribute. Only after the required ports pair
started, the `rte_eth_hairpin_bind()` API can be called to bind the
all TX queues of the egress port to the RX queues of the peer port.
Then the connection between the egress and ingress ports pair will
be established.

The `rte_eth_hairpin_unbind()` API could be used to disconnect the
egress and the peer ingress ports. This should only be called before
the device is closed if needed. When doing the clean up, all the
egress and ingress pairs related to a single port should be taken
into consideration, especially in the hot unplug case.
mode is described.

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
v5:
  * Change EINVAL to ENODEV
  * add newline character in the end of log line
  * descriptions update
v4: squash release notes update
v2: remove the all peer ports logic from rte API
---
 doc/guides/rel_notes/release_20_11.rst   |  4 +++
 lib/librte_ethdev/rte_ethdev.c           | 46 ++++++++++++++++++++++++++++
 lib/librte_ethdev/rte_ethdev.h           | 52 ++++++++++++++++++++++++++++++++
 lib/librte_ethdev/rte_ethdev_driver.h    | 52 ++++++++++++++++++++++++++++++++
 lib/librte_ethdev/rte_ethdev_version.map |  2 ++
 5 files changed, 156 insertions(+)

Comments

Thomas Monjalon Oct. 15, 2020, 10:34 a.m. UTC | #1
15/10/2020 07:35, Bing Zhao:
> v5:
>   * Change EINVAL to ENODEV
>   * add newline character in the end of log line
>   * descriptions update

It looks good.
More minor coding style comments below. With those,
Acked-by: Thomas Monjalon <thomas@monjalon.net>

> +	if (ret)

Coding style recommends explicit comparison with == or !=

> +		RTE_ETHDEV_LOG(ERR, "Failed to bind hairpin TX %d "
> +			       "to RX %d (%d - all ports)\n", tx_port,
> +			       rx_port, RTE_MAX_ETHPORTS);

It is preferred not splitting the log lines,
or maybe only after a format specifier, so it can be grepped.
Here the space after %d would be better on the next line.

In general Rx/Tx is preferred over the full capital RX/TX version.

Thanks
Bing Zhao Oct. 15, 2020, 11:39 a.m. UTC | #2
Hi Thomas,

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Thursday, October 15, 2020 6:34 PM
> To: Bing Zhao <bingz@nvidia.com>
> Cc: Ori Kam <orika@nvidia.com>; ferruh.yigit@intel.com;
> arybchenko@solarflare.com; mdr@ashroe.eu; nhorman@tuxdriver.com;
> bernard.iremonger@intel.com; beilei.xing@intel.com;
> wenzhuo.lu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v5 1/5] ethdev: add hairpin bind and unbind APIs
> 
> External email: Use caution opening links or attachments
> 
> 
> 15/10/2020 07:35, Bing Zhao:
> > v5:
> >   * Change EINVAL to ENODEV
> >   * add newline character in the end of log line
> >   * descriptions update
> 
> It looks good.
> More minor coding style comments below. With those,
> Acked-by: Thomas Monjalon <thomas@monjalon.net>

Thanks for the review and comments

> 
> > +     if (ret)
> 
> Coding style recommends explicit comparison with == or !=

Done

> 
> > +             RTE_ETHDEV_LOG(ERR, "Failed to bind hairpin TX %d "
> > +                            "to RX %d (%d - all ports)\n",
> tx_port,
> > +                            rx_port, RTE_MAX_ETHPORTS);
> 
> It is preferred not splitting the log lines, or maybe only after a
> format specifier, so it can be grepped.
> Here the space after %d would be better on the next line.
> 
> In general Rx/Tx is preferred over the full capital RX/TX version.

Done

> 
> Thanks
> 

BR. Bing
diff mbox series

Patch

diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 30db8f2..0a9ae54 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -67,6 +67,10 @@  New Features
   Added the FEC API which provides functions for query FEC capabilities and
   current FEC mode from device. Also, API for configuring FEC mode is also provided.
 
+* **Updated the ethdev library to support hairpin between two ports.**
+
+  New APIs are introduced to support binding / unbinding 2 ports hairpin.
+
 * **Updated Broadcom bnxt driver.**
 
   Updated the Broadcom bnxt driver with new features and improvements, including:
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 5b7979a..150c555 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -2212,6 +2212,52 @@  struct rte_eth_dev *
 	return eth_err(port_id, ret);
 }
 
+int
+rte_eth_hairpin_bind(uint16_t tx_port, uint16_t rx_port)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(tx_port, -ENODEV);
+	dev = &rte_eth_devices[tx_port];
+	if (!dev->data->dev_started) {
+		RTE_ETHDEV_LOG(ERR, "TX port %d is not started\n", tx_port);
+		return -EBUSY;
+	}
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->hairpin_bind, -ENOTSUP);
+	ret = (*dev->dev_ops->hairpin_bind)(dev, rx_port);
+	if (ret)
+		RTE_ETHDEV_LOG(ERR, "Failed to bind hairpin TX %d "
+			       "to RX %d (%d - all ports)\n", tx_port,
+			       rx_port, RTE_MAX_ETHPORTS);
+
+	return ret;
+}
+
+int
+rte_eth_hairpin_unbind(uint16_t tx_port, uint16_t rx_port)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(tx_port, -ENODEV);
+	dev = &rte_eth_devices[tx_port];
+	if (!dev->data->dev_started) {
+		RTE_ETHDEV_LOG(ERR, "TX port %d is already stopped\n", tx_port);
+		return -EBUSY;
+	}
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->hairpin_unbind, -ENOTSUP);
+	ret = (*dev->dev_ops->hairpin_unbind)(dev, rx_port);
+	if (ret)
+		RTE_ETHDEV_LOG(ERR, "Failed to unbind hairpin "
+			       "TX %d from RX %d (%d - all ports)\n", tx_port,
+			       rx_port, RTE_MAX_ETHPORTS);
+
+	return ret;
+}
+
 void
 rte_eth_tx_buffer_drop_callback(struct rte_mbuf **pkts, uint16_t unsent,
 		void *userdata __rte_unused)
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index f4cc591..3bdb189 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -2158,6 +2158,58 @@  int rte_eth_tx_queue_setup(uint16_t port_id, uint16_t tx_queue_id,
 	 const struct rte_eth_hairpin_conf *conf);
 
 /**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Bind all hairpin TX queues of one port to the RX queues of the peer port.
+ * It is only allowed to call this function after all hairpin queues are
+ * configured properly and the devices are in started state.
+ *
+ * @param tx_port
+ *   The identifier of the TX port.
+ * @param rx_port
+ *   The identifier of peer RX port.
+ *   RTE_MAX_ETHPORTS is allowed for the traversal of all devices.
+ *   RX port ID could have the same value as TX port ID.
+ *
+ * @return
+ *   - (0) if successful.
+ *   - (-ENODEV) if TX port ID is invalid.
+ *   - (-EBUSY) if device is not in started state.
+ *   - (-ENOTSUP) if hardware doesn't support.
+ *   - Others detailed errors from PMD drivers.
+ */
+__rte_experimental
+int rte_eth_hairpin_bind(uint16_t tx_port, uint16_t rx_port);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Unbind all hairpin TX queues of one port from the RX queues of the peer port.
+ * This should be called before closing the TX or RX devices, if the bind
+ * function is called before.
+ * After unbinding the hairpin ports pair, it is allowed to bind them again.
+ * Changing queues configuration should be after stopping the device(s).
+ *
+ * @param tx_port
+ *   The identifier of the TX port.
+ * @param rx_port
+ *   The identifier of peer RX port.
+ *   RTE_MAX_ETHPORTS is allowed for traversal of all devices.
+ *   RX port ID could have the same value as TX port ID.
+ *
+ * @return
+ *   - (0) if successful.
+ *   - (-ENODEV) if TX port ID is invalid.
+ *   - (-EBUSY) if device is in stopped state.
+ *   - (-ENOTSUP) if hardware doesn't support.
+ *   - Others detailed errors from PMD drivers.
+ */
+__rte_experimental
+int rte_eth_hairpin_unbind(uint16_t tx_port, uint16_t rx_port);
+
+/**
  * Return the NUMA socket to which an Ethernet device is connected
  *
  * @param port_id
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 35cc4fb..26ded15 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -656,6 +656,54 @@  typedef int (*eth_fec_get_t)(struct rte_eth_dev *dev,
 typedef int (*eth_fec_set_t)(struct rte_eth_dev *dev, uint32_t fec_capa);
 
 /**
+ * @internal
+ * Bind all hairpin TX queues of one port to the RX queues of the peer port.
+ *
+ * @param dev
+ *   ethdev handle of port.
+ * @param rx_port
+ *   the peer RX port.
+ *
+ * @return
+ *   Negative errno value on error, 0 on success.
+ *
+ * @retval 0
+ *   Success, bind successfully.
+ * @retval -ENOTSUP
+ *   Bind API is not supported.
+ * @retval -EINVAL
+ *   One of the parameters is invalid.
+ * @retval -EBUSY
+ *   Device is not started.
+ */
+typedef int (*eth_hairpin_bind_t)(struct rte_eth_dev *dev,
+				uint16_t rx_port);
+
+/**
+ * @internal
+ * Unbind all hairpin TX queues of one port from the RX queues of the peer port.
+ *
+ * @param dev
+ *   ethdev handle of port.
+ * @param rx_port
+ *   the peer RX port.
+ *
+ * @return
+ *   Negative errno value on error, 0 on success.
+ *
+ * @retval 0
+ *   Success, unbind successfully.
+ * @retval -ENOTSUP
+ *   Bind API is not supported.
+ * @retval -EINVAL
+ *   One of the parameters is invalid.
+ * @retval -EBUSY
+ *   Device is already stopped.
+ */
+typedef int (*eth_hairpin_unbind_t)(struct rte_eth_dev *dev,
+				  uint16_t rx_port);
+
+/**
  * @internal A structure containing the functions exported by an Ethernet driver.
  */
 struct eth_dev_ops {
@@ -801,6 +849,10 @@  struct eth_dev_ops {
 	/**< Get Forward Error Correction(FEC) mode. */
 	eth_fec_set_t fec_set;
 	/**< Set Forward Error Correction(FEC) mode. */
+	eth_hairpin_bind_t hairpin_bind;
+	/**< Bind all hairpin TX queues of device to the peer port RX queues. */
+	eth_hairpin_unbind_t hairpin_unbind;
+	/**< Unbind all hairpin TX queues from the peer port RX queues. */
 };
 
 /**
diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map
index f8a0945..1aee03a 100644
--- a/lib/librte_ethdev/rte_ethdev_version.map
+++ b/lib/librte_ethdev/rte_ethdev_version.map
@@ -227,6 +227,8 @@  EXPERIMENTAL {
 	rte_tm_wred_profile_delete;
 
 	# added in 20.11
+	rte_eth_hairpin_bind;
+	rte_eth_hairpin_unbind;
 	rte_eth_link_speed_to_str;
 	rte_eth_link_to_str;
 	rte_eth_fec_get_capability;