From patchwork Mon Apr 5 09:59:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 90552 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9F653A0A0A; Mon, 5 Apr 2021 11:59:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DD0BD140E15; Mon, 5 Apr 2021 11:59:07 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id B6F7A140DFC for ; Mon, 5 Apr 2021 11:59:05 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@nvidia.com) with SMTP; 5 Apr 2021 12:59:03 +0300 Received: from nvidia.com (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1359x30e005983; Mon, 5 Apr 2021 12:59:03 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: rasland@nvidia.com, matan@nvidia.com, orika@nvidia.com, stable@dpdk.org Date: Mon, 5 Apr 2021 09:59:01 +0000 Message-Id: <20210405095901.29202-1-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH] net/mlx5: fix the drop action for the DR/DV X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" There are multiple branches in rdma-core library backing: the rte flows: - Verbs - Direct Verbs (DV) - Direct Rules (DR) The Verbs API always requires the specifying the queue even if there is the drop action in the flow, though the kernel optimizes out the actual queue usage for the flows containing the drop action. The PMD handles the dedicated Rx queue to provide Verbs API compatibility. The DV/DR API does not require explicit specifying the queue at the flow creation, but PMD still specified the dedicated drop queue as action. It performed the packet forwarding to the dummy queue (that was not polled at all) causing the steering pipeline resources usage and degrading the overall packet processing rate. For example, with inserted flow to drop all the ingress packets the statistics reported only 15Mpps of 64B packets were received over 100Gbps line. Since the Direct Rule API for E-Switch was introduced the rdma-core supports the dedicated drop action, that is recognized both for DV and DR and can be used for the entire device in unified fashion, regardless of steering domain. The similar drop action was introduced for E-Switch, the usage of this one can be extended for other steering domains, not for E-Switch's one only. This patch: - renames esw_drop_action to dr_drop_action to emphasize the global nature of the variable (not only E-Switch domain) - specifies this global drop action instead of dedicated drop queue for the DR/DV flows Cc: stable@dpdk.org Fixes: 34fa7c0268e7 ("net/mlx5: add drop action to Direct Verbs E-Switch") Fixes: 65b3cd0dc39b ("net/mlx5: create global drop action") Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/linux/mlx5_os.c | 24 +++++++++++++++++------- drivers/net/mlx5/mlx5.h | 2 +- drivers/net/mlx5/mlx5_flow_dv.c | 13 +++++++++++-- 3 files changed, 29 insertions(+), 10 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 5e3ae9f10e..c9f997963c 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -325,7 +325,17 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv) goto error; } sh->fdb_domain = domain; - sh->esw_drop_action = mlx5_glue->dr_create_flow_action_drop(); + } + /* + * The drop action is just some dummy placeholder in rdma-core. It + * does not belong to domains and has no any attributes, and, can be + * shared by the entire device. + */ + sh->dr_drop_action = mlx5_glue->dr_create_flow_action_drop(); + if (!sh->dr_drop_action) { + DRV_LOG(ERR, "FDB mlx5dv_dr_create_flow_action_drop"); + err = errno; + goto error; } #endif if (!sh->tunnel_hub) @@ -361,9 +371,9 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv) mlx5_glue->dr_destroy_domain(sh->fdb_domain); sh->fdb_domain = NULL; } - if (sh->esw_drop_action) { - mlx5_glue->destroy_flow_action(sh->esw_drop_action); - sh->esw_drop_action = NULL; + if (sh->dr_drop_action) { + mlx5_glue->destroy_flow_action(sh->dr_drop_action); + sh->dr_drop_action = NULL; } if (sh->pop_vlan_action) { mlx5_glue->destroy_flow_action(sh->pop_vlan_action); @@ -418,9 +428,9 @@ mlx5_os_free_shared_dr(struct mlx5_priv *priv) mlx5_glue->dr_destroy_domain(sh->fdb_domain); sh->fdb_domain = NULL; } - if (sh->esw_drop_action) { - mlx5_glue->destroy_flow_action(sh->esw_drop_action); - sh->esw_drop_action = NULL; + if (sh->dr_drop_action) { + mlx5_glue->destroy_flow_action(sh->dr_drop_action); + sh->dr_drop_action = NULL; } #endif if (sh->pop_vlan_action) { diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index e4963bd107..1c70e28c77 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -713,7 +713,7 @@ struct mlx5_dev_ctx_shared { struct mlx5_hlist *flow_tbls; struct mlx5_flow_tunnel_hub *tunnel_hub; /* Direct Rules tables for FDB, NIC TX+RX */ - void *esw_drop_action; /* Pointer to DR E-Switch drop action. */ + void *dr_drop_action; /* Pointer to DR drop action, any domain. */ void *pop_vlan_action; /* Pointer to DR pop VLAN action. */ struct mlx5_hlist *encaps_decaps; /* Encap/decap action hash list. */ struct mlx5_hlist *modify_cmds; diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 23e5849783..eb70bae900 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -12002,11 +12002,19 @@ flow_dv_apply(struct rte_eth_dev *dev, struct rte_flow *flow, n = dv->actions_n; if (dh->fate_action == MLX5_FLOW_FATE_DROP) { if (dv->transfer) { - dv->actions[n++] = priv->sh->esw_drop_action; + MLX5_ASSERT(priv->sh->dr_drop_action); + dv->actions[n++] = priv->sh->dr_drop_action; } else { +#ifdef HAVE_MLX5DV_DR + /* DR supports drop action placeholder. */ + MLX5_ASSERT(priv->sh->dr_drop_action); + dv->actions[n++] = priv->sh->dr_drop_action; +#else + /* For DV we use the explicit drop queue. */ MLX5_ASSERT(priv->drop_queue.hrxq); dv->actions[n++] = priv->drop_queue.hrxq->action; +#endif } } else if ((dh->fate_action == MLX5_FLOW_FATE_QUEUE && !dv_h->rix_sample && !dv_h->rix_dest_array)) { @@ -13714,7 +13722,8 @@ mlx5_flow_dv_discover_counter_offset_support(struct rte_eth_dev *dev) &actions[0]); if (ret) goto err; - actions[1] = priv->drop_queue.hrxq->action; + actions[1] = sh->dr_drop_action ? sh->dr_drop_action : + priv->drop_queue.hrxq->action; dv_attr.match_criteria_enable = flow_dv_matcher_enable(mask.buf); ret = mlx5_flow_os_create_flow_matcher(sh->ctx, &dv_attr, tbl->obj, &matcher);