[1/5] ethdev: add API to negotiate support for Rx meta information
Checks
Commit Message
Per-packet meta information (flag, mark and the likes) might
be expensive to deliver in terms of small packet performance.
If the features are not enabled by default, enabling them at
short notice (for example, when a flow rule with action MARK
gets created) without traffic disruption may not be possible.
Letting applications request delivery of Rx meta information
during initialisation can solve the said problem efficiently.
Technically, that could be accomplished by defining new bits
in DEV_RX_OFFLOAD namespace, but the ability to extract meta
data cannot be considered an offload on its own. For example,
Rx checksumming is an offload, while mark delivery is not as
it needs an external party, a flow rule with action MARK, to
hold the value and trigger mark insertion in the first place.
With this in mind, add a means to let applications negotiate
adapter support for the very delivery of Rx meta information.
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
---
lib/ethdev/ethdev_driver.h | 19 +++++++++++
lib/ethdev/rte_ethdev.c | 25 +++++++++++++++
lib/ethdev/rte_ethdev.h | 66 ++++++++++++++++++++++++++++++++++++++
lib/ethdev/version.map | 3 ++
4 files changed, 113 insertions(+)
Comments
On Thu, Sep 2, 2021 at 7:54 PM Ivan Malov <ivan.malov@oktetlabs.ru> wrote:
>
> Per-packet meta information (flag, mark and the likes) might
> be expensive to deliver in terms of small packet performance.
> If the features are not enabled by default, enabling them at
> short notice (for example, when a flow rule with action MARK
> gets created) without traffic disruption may not be possible.
>
> Letting applications request delivery of Rx meta information
> during initialisation can solve the said problem efficiently.
>
> Technically, that could be accomplished by defining new bits
> in DEV_RX_OFFLOAD namespace, but the ability to extract meta
> data cannot be considered an offload on its own. For example,
> Rx checksumming is an offload, while mark delivery is not as
> it needs an external party, a flow rule with action MARK, to
> hold the value and trigger mark insertion in the first place.
>
> With this in mind, add a means to let applications negotiate
> adapter support for the very delivery of Rx meta information.
Good stuff.
>
> Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
> Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> ---
> lib/ethdev/ethdev_driver.h | 19 +++++++++++
> lib/ethdev/rte_ethdev.c | 25 +++++++++++++++
> lib/ethdev/rte_ethdev.h | 66 ++++++++++++++++++++++++++++++++++++++
> lib/ethdev/version.map | 3 ++
> 4 files changed, 113 insertions(+)
>
> diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
> index 40e474aa7e..3e29555fc7 100644
> --- a/lib/ethdev/ethdev_driver.h
> +++ b/lib/ethdev/ethdev_driver.h
> @@ -789,6 +789,22 @@ typedef int (*eth_get_monitor_addr_t)(void *rxq,
> typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
> struct rte_eth_representor_info *info);
>
> +/**
> + * @internal
> + * Negotiate support for specific fractions of Rx meta information.
> + *
> + * @param[in] dev
> + * Port (ethdev) handle
> + *
> + * @param[inout] features
All the params are in by default, I think, only [out] only needed.
> + * Feature selection buffer
> + *
> + * @return
> + * Negative errno value on error, zero otherwise
> + */
> +typedef int (*eth_negotiate_rx_meta_t)(struct rte_eth_dev *dev,
> + uint64_t *features);
> +
> /**
> * @internal A structure containing the functions exported by an Ethernet driver.
> */
> @@ -949,6 +965,9 @@ struct eth_dev_ops {
>
> eth_representor_info_get_t representor_info_get;
> /**< Get representor info. */
> +
> + eth_negotiate_rx_meta_t negotiate_rx_meta;
> + /**< Negotiate support for specific fractions of Rx meta information. */
> };
>
> /**
> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
> index 9d95cd11e1..821450cbf9 100644
> --- a/lib/ethdev/rte_ethdev.c
> +++ b/lib/ethdev/rte_ethdev.c
> @@ -6311,6 +6311,31 @@ rte_eth_representor_info_get(uint16_t port_id,
> return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev, info));
> }
>
> +int
> +rte_eth_negotiate_rx_meta(uint16_t port_id, uint64_t *features)
> +{
> + struct rte_eth_dev *dev;
> +
> + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> + dev = &rte_eth_devices[port_id];
> +
> + if (dev->data->dev_configured != 0) {
> + RTE_ETHDEV_LOG(ERR,
> + "The port (id=%"PRIu16") is already configured\n",
> + port_id);
> + return -EBUSY;
> + }
> +
> + if (features == NULL) {
> + RTE_ETHDEV_LOG(ERR, "Invalid features (NULL)\n");
> + return -EINVAL;
> + }
> +
> + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->negotiate_rx_meta, -ENOTSUP);
> + return eth_err(port_id,
> + (*dev->dev_ops->negotiate_rx_meta)(dev, features));
> +}
> +
> RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO);
>
> RTE_INIT(ethdev_init_telemetry)
> diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
> index d2b27c351f..ac4d164aa8 100644
> --- a/lib/ethdev/rte_ethdev.h
> +++ b/lib/ethdev/rte_ethdev.h
> @@ -4888,6 +4888,72 @@ __rte_experimental
> int rte_eth_representor_info_get(uint16_t port_id,
> struct rte_eth_representor_info *info);
>
> +/**
> + * The ethdev will be able to detect flagged packets provided that
> + * there are active flow rules comprising the corresponding action.
> + */
> +#define RTE_ETH_RX_META_USER_FLAG (UINT64_C(1) << 0)
I think, we need to add @see note in rte_flow API where this is requested.
> +
> +/**
> + * The ethdev will manage to see mark IDs in packets provided that
> + * there are active flow rules comprising the corresponding action.
> + */
> +#define RTE_ETH_RX_META_USER_MARK (UINT64_C(1) << 1)
Same as above
> +
> +/**
> + * The ethdev will be able to identify partially offloaded packets
> + * and process rte_flow_get_restore_info() invocations accordingly
> + * provided that there're so-called "tunnel_set" flow rules in use.
> + */
> +#define RTE_ETH_RX_META_TUNNEL_ID (UINT64_C(1) << 2)
Same as above
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Negotiate support for specific fractions of Rx meta information.
> + *
> + * This function has to be invoked as early as possible, precisely,
> + * before the first rte_eth_dev_configure() invocation, to let the
> + * PMD make preparations which might be hard to do on later stages.
> + *
> + * The negotiation process is assumed to be carried out as follows:
> + *
> + * - the application composes a mask of preferred Rx meta features
> + * intending to enable at least some of them and invokes the API;
> + *
> + * - the ethdev driver reports back the optimal (from its point of
> + * view) subset of the initial feature set thus agreeing to make
> + * those comprising the subset simultaneously available to users;
> + *
> + * - should the application find the result unsatisfactory, it may
> + * come up with another pick of preferred features and try again;
> + *
> + * - the application can pass zero to clear the negotiation result;
> + *
> + * - the last negotiated result takes effect upon the ethdev start.
> + *
> + * If the method itself is unsupported by the PMD, the application
> + * may just ignore that and proceed with the rest of configuration
> + * procedure intending to simply try using the features it prefers.
> + *
> + * @param[in] port_id
> + * Port (ethdev) identifier
See above comment.
> + *
> + * @param[inout] features
> + * Feature selection buffer
> + *
> + * @return
> + * - (-EBUSY) if the port can't handle this in its current state;
> + * - (-ENOTSUP) if the method itself is not supported by the PMD;
> + * - (-ENODEV) if *port_id* is invalid;
> + * - (-EINVAL) if *features* is NULL;
> + * - (-EIO) if the device is removed;
> + * - (0) on success
> + */
> +__rte_experimental
> +int rte_eth_negotiate_rx_meta(uint16_t port_id, uint64_t *features);
# I would prefer to call it as rte_eth_rx_meta_negotiate as
action/verb can come the last API to
to have a proper namespace.
# IMO, We need to update the release notes for this behavior change.
# Is any of the in-tree examples or test application needs to update
to adaptor this behavior change
> +
> #include <rte_ethdev_core.h>
>
> /**
> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
> index 3eece75b72..e390e5718c 100644
> --- a/lib/ethdev/version.map
> +++ b/lib/ethdev/version.map
> @@ -249,6 +249,9 @@ EXPERIMENTAL {
> rte_mtr_meter_policy_delete;
> rte_mtr_meter_policy_update;
> rte_mtr_meter_policy_validate;
> +
> + # added in 21.11
> + rte_eth_negotiate_rx_meta;
> };
>
> INTERNAL {
> --
> 2.20.1
>
On 02/09/2021 15:23, Ivan Malov wrote:
> Per-packet meta information (flag, mark and the likes) might
> be expensive to deliver in terms of small packet performance.
> If the features are not enabled by default, enabling them at
> short notice (for example, when a flow rule with action MARK
> gets created) without traffic disruption may not be possible.
>
> Letting applications request delivery of Rx meta information
> during initialisation can solve the said problem efficiently.
>
> Technically, that could be accomplished by defining new bits
> in DEV_RX_OFFLOAD namespace, but the ability to extract meta
> data cannot be considered an offload on its own. For example,
> Rx checksumming is an offload, while mark delivery is not as
> it needs an external party, a flow rule with action MARK, to
> hold the value and trigger mark insertion in the first place.
>
> With this in mind, add a means to let applications negotiate
> adapter support for the very delivery of Rx meta information.
>
> Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
> Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> ---
> lib/ethdev/ethdev_driver.h | 19 +++++++++++
> lib/ethdev/rte_ethdev.c | 25 +++++++++++++++
> lib/ethdev/rte_ethdev.h | 66 ++++++++++++++++++++++++++++++++++++++
> lib/ethdev/version.map | 3 ++
> 4 files changed, 113 insertions(+)
>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
On Thu, Sep 2, 2021 at 7:54 PM Ivan Malov <ivan.malov@oktetlabs.ru> wrote:
>
> Per-packet meta information (flag, mark and the likes) might
> be expensive to deliver in terms of small packet performance.
> If the features are not enabled by default, enabling them at
> short notice (for example, when a flow rule with action MARK
> gets created) without traffic disruption may not be possible.
>
> Letting applications request delivery of Rx meta information
> during initialisation can solve the said problem efficiently.
>
> Technically, that could be accomplished by defining new bits
> in DEV_RX_OFFLOAD namespace, but the ability to extract meta
> data cannot be considered an offload on its own. For example,
> Rx checksumming is an offload, while mark delivery is not as
> it needs an external party, a flow rule with action MARK, to
> hold the value and trigger mark insertion in the first place.
>
> With this in mind, add a means to let applications negotiate
> adapter support for the very delivery of Rx meta information.
>
> Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
> Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Jerin Jacob <jerinj@marvell.com>
> ---
> lib/ethdev/ethdev_driver.h | 19 +++++++++++
> lib/ethdev/rte_ethdev.c | 25 +++++++++++++++
> lib/ethdev/rte_ethdev.h | 66 ++++++++++++++++++++++++++++++++++++++
> lib/ethdev/version.map | 3 ++
> 4 files changed, 113 insertions(+)
>
> diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
> index 40e474aa7e..3e29555fc7 100644
> --- a/lib/ethdev/ethdev_driver.h
> +++ b/lib/ethdev/ethdev_driver.h
> @@ -789,6 +789,22 @@ typedef int (*eth_get_monitor_addr_t)(void *rxq,
> typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
> struct rte_eth_representor_info *info);
>
> +/**
> + * @internal
> + * Negotiate support for specific fractions of Rx meta information.
> + *
> + * @param[in] dev
> + * Port (ethdev) handle
> + *
> + * @param[inout] features
> + * Feature selection buffer
> + *
> + * @return
> + * Negative errno value on error, zero otherwise
> + */
> +typedef int (*eth_negotiate_rx_meta_t)(struct rte_eth_dev *dev,
> + uint64_t *features);
> +
> /**
> * @internal A structure containing the functions exported by an Ethernet driver.
> */
> @@ -949,6 +965,9 @@ struct eth_dev_ops {
>
> eth_representor_info_get_t representor_info_get;
> /**< Get representor info. */
> +
> + eth_negotiate_rx_meta_t negotiate_rx_meta;
> + /**< Negotiate support for specific fractions of Rx meta information. */
> };
>
> /**
> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
> index 9d95cd11e1..821450cbf9 100644
> --- a/lib/ethdev/rte_ethdev.c
> +++ b/lib/ethdev/rte_ethdev.c
> @@ -6311,6 +6311,31 @@ rte_eth_representor_info_get(uint16_t port_id,
> return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev, info));
> }
>
> +int
> +rte_eth_negotiate_rx_meta(uint16_t port_id, uint64_t *features)
> +{
> + struct rte_eth_dev *dev;
> +
> + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> + dev = &rte_eth_devices[port_id];
> +
> + if (dev->data->dev_configured != 0) {
> + RTE_ETHDEV_LOG(ERR,
> + "The port (id=%"PRIu16") is already configured\n",
> + port_id);
> + return -EBUSY;
> + }
> +
> + if (features == NULL) {
> + RTE_ETHDEV_LOG(ERR, "Invalid features (NULL)\n");
> + return -EINVAL;
> + }
> +
> + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->negotiate_rx_meta, -ENOTSUP);
> + return eth_err(port_id,
> + (*dev->dev_ops->negotiate_rx_meta)(dev, features));
> +}
> +
> RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO);
>
> RTE_INIT(ethdev_init_telemetry)
> diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
> index d2b27c351f..ac4d164aa8 100644
> --- a/lib/ethdev/rte_ethdev.h
> +++ b/lib/ethdev/rte_ethdev.h
> @@ -4888,6 +4888,72 @@ __rte_experimental
> int rte_eth_representor_info_get(uint16_t port_id,
> struct rte_eth_representor_info *info);
>
> +/**
> + * The ethdev will be able to detect flagged packets provided that
> + * there are active flow rules comprising the corresponding action.
> + */
> +#define RTE_ETH_RX_META_USER_FLAG (UINT64_C(1) << 0)
> +
> +/**
> + * The ethdev will manage to see mark IDs in packets provided that
> + * there are active flow rules comprising the corresponding action.
> + */
> +#define RTE_ETH_RX_META_USER_MARK (UINT64_C(1) << 1)
> +
> +/**
> + * The ethdev will be able to identify partially offloaded packets
> + * and process rte_flow_get_restore_info() invocations accordingly
> + * provided that there're so-called "tunnel_set" flow rules in use.
> + */
> +#define RTE_ETH_RX_META_TUNNEL_ID (UINT64_C(1) << 2)
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Negotiate support for specific fractions of Rx meta information.
> + *
> + * This function has to be invoked as early as possible, precisely,
> + * before the first rte_eth_dev_configure() invocation, to let the
> + * PMD make preparations which might be hard to do on later stages.
> + *
> + * The negotiation process is assumed to be carried out as follows:
> + *
> + * - the application composes a mask of preferred Rx meta features
> + * intending to enable at least some of them and invokes the API;
> + *
> + * - the ethdev driver reports back the optimal (from its point of
> + * view) subset of the initial feature set thus agreeing to make
> + * those comprising the subset simultaneously available to users;
> + *
> + * - should the application find the result unsatisfactory, it may
> + * come up with another pick of preferred features and try again;
> + *
> + * - the application can pass zero to clear the negotiation result;
> + *
> + * - the last negotiated result takes effect upon the ethdev start.
> + *
> + * If the method itself is unsupported by the PMD, the application
> + * may just ignore that and proceed with the rest of configuration
> + * procedure intending to simply try using the features it prefers.
> + *
> + * @param[in] port_id
> + * Port (ethdev) identifier
> + *
> + * @param[inout] features
> + * Feature selection buffer
> + *
> + * @return
> + * - (-EBUSY) if the port can't handle this in its current state;
> + * - (-ENOTSUP) if the method itself is not supported by the PMD;
> + * - (-ENODEV) if *port_id* is invalid;
> + * - (-EINVAL) if *features* is NULL;
> + * - (-EIO) if the device is removed;
> + * - (0) on success
> + */
> +__rte_experimental
> +int rte_eth_negotiate_rx_meta(uint16_t port_id, uint64_t *features);
> +
> #include <rte_ethdev_core.h>
>
> /**
> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
> index 3eece75b72..e390e5718c 100644
> --- a/lib/ethdev/version.map
> +++ b/lib/ethdev/version.map
> @@ -249,6 +249,9 @@ EXPERIMENTAL {
> rte_mtr_meter_policy_delete;
> rte_mtr_meter_policy_update;
> rte_mtr_meter_policy_validate;
> +
> + # added in 21.11
> + rte_eth_negotiate_rx_meta;
> };
>
> INTERNAL {
> --
> 2.20.1
>
@@ -789,6 +789,22 @@ typedef int (*eth_get_monitor_addr_t)(void *rxq,
typedef int (*eth_representor_info_get_t)(struct rte_eth_dev *dev,
struct rte_eth_representor_info *info);
+/**
+ * @internal
+ * Negotiate support for specific fractions of Rx meta information.
+ *
+ * @param[in] dev
+ * Port (ethdev) handle
+ *
+ * @param[inout] features
+ * Feature selection buffer
+ *
+ * @return
+ * Negative errno value on error, zero otherwise
+ */
+typedef int (*eth_negotiate_rx_meta_t)(struct rte_eth_dev *dev,
+ uint64_t *features);
+
/**
* @internal A structure containing the functions exported by an Ethernet driver.
*/
@@ -949,6 +965,9 @@ struct eth_dev_ops {
eth_representor_info_get_t representor_info_get;
/**< Get representor info. */
+
+ eth_negotiate_rx_meta_t negotiate_rx_meta;
+ /**< Negotiate support for specific fractions of Rx meta information. */
};
/**
@@ -6311,6 +6311,31 @@ rte_eth_representor_info_get(uint16_t port_id,
return eth_err(port_id, (*dev->dev_ops->representor_info_get)(dev, info));
}
+int
+rte_eth_negotiate_rx_meta(uint16_t port_id, uint64_t *features)
+{
+ struct rte_eth_dev *dev;
+
+ RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+ dev = &rte_eth_devices[port_id];
+
+ if (dev->data->dev_configured != 0) {
+ RTE_ETHDEV_LOG(ERR,
+ "The port (id=%"PRIu16") is already configured\n",
+ port_id);
+ return -EBUSY;
+ }
+
+ if (features == NULL) {
+ RTE_ETHDEV_LOG(ERR, "Invalid features (NULL)\n");
+ return -EINVAL;
+ }
+
+ RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->negotiate_rx_meta, -ENOTSUP);
+ return eth_err(port_id,
+ (*dev->dev_ops->negotiate_rx_meta)(dev, features));
+}
+
RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO);
RTE_INIT(ethdev_init_telemetry)
@@ -4888,6 +4888,72 @@ __rte_experimental
int rte_eth_representor_info_get(uint16_t port_id,
struct rte_eth_representor_info *info);
+/**
+ * The ethdev will be able to detect flagged packets provided that
+ * there are active flow rules comprising the corresponding action.
+ */
+#define RTE_ETH_RX_META_USER_FLAG (UINT64_C(1) << 0)
+
+/**
+ * The ethdev will manage to see mark IDs in packets provided that
+ * there are active flow rules comprising the corresponding action.
+ */
+#define RTE_ETH_RX_META_USER_MARK (UINT64_C(1) << 1)
+
+/**
+ * The ethdev will be able to identify partially offloaded packets
+ * and process rte_flow_get_restore_info() invocations accordingly
+ * provided that there're so-called "tunnel_set" flow rules in use.
+ */
+#define RTE_ETH_RX_META_TUNNEL_ID (UINT64_C(1) << 2)
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Negotiate support for specific fractions of Rx meta information.
+ *
+ * This function has to be invoked as early as possible, precisely,
+ * before the first rte_eth_dev_configure() invocation, to let the
+ * PMD make preparations which might be hard to do on later stages.
+ *
+ * The negotiation process is assumed to be carried out as follows:
+ *
+ * - the application composes a mask of preferred Rx meta features
+ * intending to enable at least some of them and invokes the API;
+ *
+ * - the ethdev driver reports back the optimal (from its point of
+ * view) subset of the initial feature set thus agreeing to make
+ * those comprising the subset simultaneously available to users;
+ *
+ * - should the application find the result unsatisfactory, it may
+ * come up with another pick of preferred features and try again;
+ *
+ * - the application can pass zero to clear the negotiation result;
+ *
+ * - the last negotiated result takes effect upon the ethdev start.
+ *
+ * If the method itself is unsupported by the PMD, the application
+ * may just ignore that and proceed with the rest of configuration
+ * procedure intending to simply try using the features it prefers.
+ *
+ * @param[in] port_id
+ * Port (ethdev) identifier
+ *
+ * @param[inout] features
+ * Feature selection buffer
+ *
+ * @return
+ * - (-EBUSY) if the port can't handle this in its current state;
+ * - (-ENOTSUP) if the method itself is not supported by the PMD;
+ * - (-ENODEV) if *port_id* is invalid;
+ * - (-EINVAL) if *features* is NULL;
+ * - (-EIO) if the device is removed;
+ * - (0) on success
+ */
+__rte_experimental
+int rte_eth_negotiate_rx_meta(uint16_t port_id, uint64_t *features);
+
#include <rte_ethdev_core.h>
/**
@@ -249,6 +249,9 @@ EXPERIMENTAL {
rte_mtr_meter_policy_delete;
rte_mtr_meter_policy_update;
rte_mtr_meter_policy_validate;
+
+ # added in 21.11
+ rte_eth_negotiate_rx_meta;
};
INTERNAL {