From patchwork Wed Jan 6 08:19:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86025 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2835DA09FF; Wed, 6 Jan 2021 09:20:28 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 11D42160863; Wed, 6 Jan 2021 09:20:28 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 671061607A2 for ; Wed, 6 Jan 2021 09:20:26 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:20:22 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgU009291; Wed, 6 Jan 2021 10:20:22 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , stable@dpdk.org Date: Wed, 6 Jan 2021 08:19:23 +0000 Message-Id: <1609921181-5019-2-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 01/19] common/mlx5: fix completion queue entry size configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" According to the current data-path implementation in the PMD the CQE size must follow the cache-line size. So, the configuration of the CQE size should be depended in RTE_CACHE_LINE_SIZE. Wrongly, part of the CQE creations didn't follow it exactly what caused an incompatibility between HW and SW in the data-path when working in 128B cache-line size systems. Adjust the rule for any CQE creation. Remove the cqe_size attribute from the DevX CQ creation command and set it inside the command translation according to the cache-line size. Fixes: 79a7e409a2f6 ("common/mlx5: prepare support of packet pacing") Fixes: 5cd0a83f413e ("common/mlx5: support more fields in DevX CQ create") Cc: stable@dpdk.org Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_devx_cmds.c | 4 ++-- drivers/common/mlx5/mlx5_devx_cmds.h | 1 - drivers/net/mlx5/mlx5_devx.c | 4 ---- drivers/net/mlx5/mlx5_txpp.c | 4 ---- 4 files changed, 2 insertions(+), 11 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 12f51a9..59f0bcc 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1569,7 +1569,8 @@ struct mlx5_devx_obj * } else { MLX5_SET64(cqc, cqctx, dbr_addr, attr->db_addr); } - MLX5_SET(cqc, cqctx, cqe_sz, attr->cqe_size); + MLX5_SET(cqc, cqctx, cqe_sz, (RTE_CACHE_LINE_SIZE == 128) ? + MLX5_CQE_SIZE_128B : MLX5_CQE_SIZE_64B); MLX5_SET(cqc, cqctx, cc, attr->use_first_only); MLX5_SET(cqc, cqctx, oi, attr->overrun_ignore); MLX5_SET(cqc, cqctx, log_cq_size, attr->log_cq_size); @@ -1582,7 +1583,6 @@ struct mlx5_devx_obj * attr->mini_cqe_res_format); MLX5_SET(cqc, cqctx, mini_cqe_res_format_ext, attr->mini_cqe_res_format_ext); - MLX5_SET(cqc, cqctx, cqe_sz, attr->cqe_size); if (attr->q_umem_valid) { MLX5_SET(create_cq_in, in, cq_umem_valid, attr->q_umem_valid); MLX5_SET(create_cq_in, in, cq_umem_id, attr->q_umem_id); diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index b335b7c..a14f3bf 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -277,7 +277,6 @@ struct mlx5_devx_cq_attr { uint32_t cqe_comp_en:1; uint32_t mini_cqe_res_format:2; uint32_t mini_cqe_res_format_ext:2; - uint32_t cqe_size:3; uint32_t log_cq_size:5; uint32_t log_page_size:5; uint32_t uar_page_id; diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index da3bb78..5c5bea6 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -486,8 +486,6 @@ "Port %u Rx CQE compression is disabled for LRO.", dev->data->port_id); } - if (priv->config.cqe_pad) - cq_attr.cqe_size = MLX5_CQE_SIZE_128B; log_cqe_n = log2above(cqe_n); cq_size = sizeof(struct mlx5_cqe) * (1 << log_cqe_n); buf = rte_calloc_socket(__func__, 1, cq_size, page_size, @@ -1262,8 +1260,6 @@ DRV_LOG(ERR, "Failed to allocate CQ door-bell."); goto error; } - cq_attr.cqe_size = (sizeof(struct mlx5_cqe) == 128) ? - MLX5_CQE_SIZE_128B : MLX5_CQE_SIZE_64B; cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(priv->sh->tx_uar); cq_attr.eqn = priv->sh->eqn; cq_attr.q_umem_valid = 1; diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c index 726bdc6a..e998de3 100644 --- a/drivers/net/mlx5/mlx5_txpp.c +++ b/drivers/net/mlx5/mlx5_txpp.c @@ -278,8 +278,6 @@ goto error; } /* Create completion queue object for Rearm Queue. */ - cq_attr.cqe_size = (sizeof(struct mlx5_cqe) == 128) ? - MLX5_CQE_SIZE_128B : MLX5_CQE_SIZE_64B; cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar); cq_attr.eqn = sh->eqn; cq_attr.q_umem_valid = 1; @@ -516,8 +514,6 @@ goto error; } /* Create completion queue object for Clock Queue. */ - cq_attr.cqe_size = (sizeof(struct mlx5_cqe) == 128) ? - MLX5_CQE_SIZE_128B : MLX5_CQE_SIZE_64B; cq_attr.use_first_only = 1; cq_attr.overrun_ignore = 1; cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar); From patchwork Wed Jan 6 08:19:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86026 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A4CE0A09FF; Wed, 6 Jan 2021 09:20:37 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9183216089F; Wed, 6 Jan 2021 09:20:37 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 659F9160864 for ; Wed, 6 Jan 2021 09:20:36 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:20:31 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgV009291; Wed, 6 Jan 2021 10:20:31 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , stable@dpdk.org Date: Wed, 6 Jan 2021 08:19:24 +0000 Message-Id: <1609921181-5019-3-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 02/19] net/mlx5: remove CQE padding device argument X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The data-path code doesn't take care on 'rxq_cqe_pad_en' and use padded CQE for any case when the system cache-line size is 128B. This makes the argument redundant. Remove it. Fixes: bc91e8db12cd ("net/mlx5: add 128B padding of Rx completion entry") Cc: stable@dpdk.org Signed-off-by: Michael Baum Acked-by: Matan Azrad --- doc/guides/nics/mlx5.rst | 18 ------------------ drivers/net/mlx5/linux/mlx5_os.c | 12 ------------ drivers/net/mlx5/linux/mlx5_verbs.c | 2 +- drivers/net/mlx5/mlx5.c | 6 ------ drivers/net/mlx5/mlx5.h | 1 - drivers/net/mlx5/windows/mlx5_os.c | 7 ------- 6 files changed, 1 insertion(+), 45 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 3bda0f8..6950cc1 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -448,24 +448,6 @@ Driver options - POWER9 and ARMv8 with ConnectX-4 Lx, ConnectX-5, ConnectX-6, ConnectX-6 Dx, ConnectX-6 Lx, BlueField and BlueField-2. -- ``rxq_cqe_pad_en`` parameter [int] - - A nonzero value enables 128B padding of CQE on RX side. The size of CQE - is aligned with the size of a cacheline of the core. If cacheline size is - 128B, the CQE size is configured to be 128B even though the device writes - only 64B data on the cacheline. This is to avoid unnecessary cache - invalidation by device's two consecutive writes on to one cacheline. - However in some architecture, it is more beneficial to update entire - cacheline with padding the rest 64B rather than striding because - read-modify-write could drop performance a lot. On the other hand, - writing extra data will consume more PCIe bandwidth and could also drop - the maximum throughput. It is recommended to empirically set this - parameter. Disabled by default. - - Supported on: - - - CPU having 128B cacheline with ConnectX-5 and BlueField. - - ``rxq_pkt_pad_en`` parameter [int] A nonzero value enables padding Rx packet to the size of cacheline on PCI diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 6812a1f..9ac1d46 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -677,7 +677,6 @@ unsigned int hw_padding = 0; unsigned int mps; unsigned int cqe_comp; - unsigned int cqe_pad = 0; unsigned int tunnel_en = 0; unsigned int mpls_en = 0; unsigned int swp = 0; @@ -875,11 +874,6 @@ else cqe_comp = 1; config->cqe_comp = cqe_comp; -#ifdef HAVE_IBV_MLX5_MOD_CQE_128B_PAD - /* Whether device supports 128B Rx CQE padding. */ - cqe_pad = RTE_CACHE_LINE_SIZE == 128 && - (dv_attr.flags & MLX5DV_CONTEXT_FLAGS_CQE_128B_PAD); -#endif #ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT if (dv_attr.comp_mask & MLX5DV_CONTEXT_MASK_TUNNEL_OFFLOADS) { tunnel_en = ((dv_attr.tunnel_offloads_caps & @@ -1116,12 +1110,6 @@ DRV_LOG(WARNING, "Rx CQE compression isn't supported"); config->cqe_comp = 0; } - if (config->cqe_pad && !cqe_pad) { - DRV_LOG(WARNING, "Rx CQE padding isn't supported"); - config->cqe_pad = 0; - } else if (config->cqe_pad) { - DRV_LOG(INFO, "Rx CQE padding is enabled"); - } if (config->devx) { err = mlx5_devx_cmd_query_hca_attr(sh->ctx, &config->hca_attr); if (err) { diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c index b52ae2e..318e39b 100644 --- a/drivers/net/mlx5/linux/mlx5_verbs.c +++ b/drivers/net/mlx5/linux/mlx5_verbs.c @@ -234,7 +234,7 @@ dev->data->port_id); } #ifdef HAVE_IBV_MLX5_MOD_CQE_128B_PAD - if (priv->config.cqe_pad) { + if (RTE_CACHE_LINE_SIZE == 128) { cq_attr.mlx5.comp_mask |= MLX5DV_CQ_INIT_ATTR_MASK_FLAGS; cq_attr.mlx5.flags |= MLX5DV_CQ_INIT_ATTR_FLAGS_CQE_PAD; } diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 023ef50..91492c5 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -44,9 +44,6 @@ /* Device parameter to enable RX completion queue compression. */ #define MLX5_RXQ_CQE_COMP_EN "rxq_cqe_comp_en" -/* Device parameter to enable RX completion entry padding to 128B. */ -#define MLX5_RXQ_CQE_PAD_EN "rxq_cqe_pad_en" - /* Device parameter to enable padding Rx packet to cacheline size. */ #define MLX5_RXQ_PKT_PAD_EN "rxq_pkt_pad_en" @@ -1625,8 +1622,6 @@ struct mlx5_dev_ctx_shared * } config->cqe_comp = !!tmp; config->cqe_comp_fmt = tmp; - } else if (strcmp(MLX5_RXQ_CQE_PAD_EN, key) == 0) { - config->cqe_pad = !!tmp; } else if (strcmp(MLX5_RXQ_PKT_PAD_EN, key) == 0) { config->hw_padding = !!tmp; } else if (strcmp(MLX5_RX_MPRQ_EN, key) == 0) { @@ -1755,7 +1750,6 @@ struct mlx5_dev_ctx_shared * { const char **params = (const char *[]){ MLX5_RXQ_CQE_COMP_EN, - MLX5_RXQ_CQE_PAD_EN, MLX5_RXQ_PKT_PAD_EN, MLX5_RX_MPRQ_EN, MLX5_RX_MPRQ_LOG_STRIDE_NUM, diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 41034f5..92a5d04 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -212,7 +212,6 @@ struct mlx5_dev_config { unsigned int mpls_en:1; /* MPLS over GRE/UDP is enabled. */ unsigned int cqe_comp:1; /* CQE compression is enabled. */ unsigned int cqe_comp_fmt:3; /* CQE compression format. */ - unsigned int cqe_pad:1; /* CQE padding is enabled. */ unsigned int tso:1; /* Whether TSO is supported. */ unsigned int rx_vec_en:1; /* Rx vector is enabled. */ unsigned int mr_ext_memseg_en:1; diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index fdd69fd..b036432 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -313,7 +313,6 @@ struct mlx5_priv *priv = NULL; int err = 0; unsigned int cqe_comp; - unsigned int cqe_pad = 0; struct rte_ether_addr mac; char name[RTE_ETH_NAME_MAX_LEN]; int own_domain_id = 0; @@ -461,12 +460,6 @@ DRV_LOG(WARNING, "Rx CQE compression isn't supported."); config->cqe_comp = 0; } - if (config->cqe_pad && !cqe_pad) { - DRV_LOG(WARNING, "Rx CQE padding isn't supported."); - config->cqe_pad = 0; - } else if (config->cqe_pad) { - DRV_LOG(INFO, "Rx CQE padding is enabled."); - } if (config->devx) { err = mlx5_devx_cmd_query_hca_attr(sh->ctx, &config->hca_attr); if (err) { From patchwork Wed Jan 6 08:19:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86027 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 64EE4A09FF; Wed, 6 Jan 2021 09:20:44 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D75D71606D8; Wed, 6 Jan 2021 09:20:42 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 7320A1606D8 for ; Wed, 6 Jan 2021 09:20:41 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:20:37 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgW009291; Wed, 6 Jan 2021 10:20:37 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , stable@dpdk.org Date: Wed, 6 Jan 2021 08:19:25 +0000 Message-Id: <1609921181-5019-4-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 03/19] net/mlx5: fix ASO SQ creation error flow X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In ASO SQ creation, the PMD allocates umem buffer for SQ. When umem buffer allocation is fails, the MR and CQ memory are not freed what caused a memory leak. Free it. Fixes: f935ed4b645a ("net/mlx5: support flow hit action for aging") Cc: stable@dpdk.org Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow_age.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/mlx5/mlx5_flow_age.c b/drivers/net/mlx5/mlx5_flow_age.c index 1f15f19..e867607 100644 --- a/drivers/net/mlx5/mlx5_flow_age.c +++ b/drivers/net/mlx5/mlx5_flow_age.c @@ -279,7 +279,8 @@ sizeof(*sq->db_rec) * 2, 4096, socket); if (!sq->umem_buf) { DRV_LOG(ERR, "Can't allocate wqe buffer."); - return -ENOMEM; + rte_errno = ENOMEM; + goto error; } sq->wqe_umem = mlx5_os_umem_reg(ctx, (void *)(uintptr_t)sq->umem_buf, From patchwork Wed Jan 6 08:19:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86028 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E3AA0A09FF; Wed, 6 Jan 2021 09:20:51 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2812A1606CF; Wed, 6 Jan 2021 09:20:48 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 7CCF5160697 for ; Wed, 6 Jan 2021 09:20:46 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:20:44 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgX009291; Wed, 6 Jan 2021 10:20:44 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:26 +0000 Message-Id: <1609921181-5019-5-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 04/19] common/mlx5: share DevX CQ creation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The CQ object in DevX is created in several places and in several different drivers. In all places almost all the details are the same, and in particular the allocations of the required resources. Add a structure that contains all the resources, and provide creation and release functions for it. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/meson.build | 1 + drivers/common/mlx5/mlx5_common_devx.c | 155 ++++++++++++++++++++++++ drivers/common/mlx5/mlx5_common_devx.h | 31 +++++ drivers/common/mlx5/rte_common_mlx5_exports.def | 3 + drivers/common/mlx5/version.map | 3 + drivers/common/mlx5/windows/mlx5_win_ext.h | 1 + 6 files changed, 194 insertions(+) create mode 100644 drivers/common/mlx5/mlx5_common_devx.c create mode 100644 drivers/common/mlx5/mlx5_common_devx.h diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build index 3dacc6f..26cee06 100644 --- a/drivers/common/mlx5/meson.build +++ b/drivers/common/mlx5/meson.build @@ -16,6 +16,7 @@ sources += files( 'mlx5_common_mr.c', 'mlx5_malloc.c', 'mlx5_common_pci.c', + 'mlx5_common_devx.c', ) cflags_options = [ diff --git a/drivers/common/mlx5/mlx5_common_devx.c b/drivers/common/mlx5/mlx5_common_devx.c new file mode 100644 index 0000000..3ec0dd5 --- /dev/null +++ b/drivers/common/mlx5/mlx5_common_devx.c @@ -0,0 +1,155 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ +#include + +#include +#include +#include + +#include +#include + +#include "mlx5_prm.h" +#include "mlx5_devx_cmds.h" +#include "mlx5_common_utils.h" +#include "mlx5_malloc.h" +#include "mlx5_common.h" +#include "mlx5_common_devx.h" + +/** + * Destroy DevX Completion Queue. + * + * @param[in] cq + * DevX CQ to destroy. + */ +void +mlx5_devx_cq_destroy(struct mlx5_devx_cq *cq) +{ + if (cq->cq) + claim_zero(mlx5_devx_cmd_destroy(cq->cq)); + if (cq->umem_obj) + claim_zero(mlx5_os_umem_dereg(cq->umem_obj)); + if (cq->umem_buf) + mlx5_free((void *)(uintptr_t)cq->umem_buf); +} + +/* Mark all CQEs initially as invalid. */ +static void +mlx5_cq_init(struct mlx5_devx_cq *cq_obj, uint16_t cq_size) +{ + volatile struct mlx5_cqe *cqe = cq_obj->cqes; + uint16_t i; + + for (i = 0; i < cq_size; i++, cqe++) + cqe->op_own = (MLX5_CQE_INVALID << 4) | MLX5_CQE_OWNER_MASK; +} + +/** + * Create Completion Queue using DevX API. + * + * Get a pointer to partially initialized attributes structure, and updates the + * following fields: + * q_umem_valid + * q_umem_id + * q_umem_offset + * db_umem_valid + * db_umem_id + * db_umem_offset + * eqn + * log_cq_size + * log_page_size + * All other fields are updated by caller. + * + * @param[in] ctx + * Context returned from mlx5 open_device() glue function. + * @param[in/out] cq_obj + * Pointer to CQ to create. + * @param[in] log_desc_n + * Log of number of descriptors in queue. + * @param[in] attr + * Pointer to CQ attributes structure. + * @param[in] socket + * Socket to use for allocation. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_cq_create(void *ctx, struct mlx5_devx_cq *cq_obj, uint16_t log_desc_n, + struct mlx5_devx_cq_attr *attr, int socket) +{ + struct mlx5_devx_obj *cq = NULL; + struct mlx5dv_devx_umem *umem_obj = NULL; + void *umem_buf = NULL; + size_t page_size = rte_mem_page_size(); + size_t alignment = MLX5_CQE_BUF_ALIGNMENT; + uint32_t umem_size, umem_dbrec; + uint32_t eqn; + uint16_t cq_size = 1 << log_desc_n; + int ret; + + if (page_size == (size_t)-1 || alignment == (size_t)-1) { + DRV_LOG(ERR, "Failed to get page_size."); + rte_errno = ENOMEM; + return -rte_errno; + } + /* Query first EQN. */ + ret = mlx5_glue->devx_query_eqn(ctx, 0, &eqn); + if (ret) { + rte_errno = errno; + DRV_LOG(ERR, "Failed to query event queue number."); + return -rte_errno; + } + /* Allocate memory buffer for CQEs and doorbell record. */ + umem_size = sizeof(struct mlx5_cqe) * cq_size; + umem_dbrec = RTE_ALIGN(umem_size, MLX5_DBR_SIZE); + umem_size += MLX5_DBR_SIZE; + umem_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size, + alignment, socket); + if (!umem_buf) { + DRV_LOG(ERR, "Failed to allocate memory for CQ."); + rte_errno = ENOMEM; + return -rte_errno; + } + /* Register allocated buffer in user space with DevX. */ + umem_obj = mlx5_os_umem_reg(ctx, (void *)(uintptr_t)umem_buf, umem_size, + IBV_ACCESS_LOCAL_WRITE); + if (!umem_obj) { + DRV_LOG(ERR, "Failed to register umem for CQ."); + rte_errno = errno; + goto error; + } + /* Fill attributes for CQ object creation. */ + attr->q_umem_valid = 1; + attr->q_umem_id = mlx5_os_get_umem_id(umem_obj); + attr->q_umem_offset = 0; + attr->db_umem_valid = 1; + attr->db_umem_id = attr->q_umem_id; + attr->db_umem_offset = umem_dbrec; + attr->eqn = eqn; + attr->log_cq_size = log_desc_n; + attr->log_page_size = rte_log2_u32(page_size); + /* Create completion queue object with DevX. */ + cq = mlx5_devx_cmd_create_cq(ctx, attr); + if (!cq) { + DRV_LOG(ERR, "Can't create DevX CQ object."); + rte_errno = ENOMEM; + goto error; + } + cq_obj->umem_buf = umem_buf; + cq_obj->umem_obj = umem_obj; + cq_obj->cq = cq; + cq_obj->db_rec = RTE_PTR_ADD(cq_obj->umem_buf, umem_dbrec); + /* Mark all CQEs initially as invalid. */ + mlx5_cq_init(cq_obj, cq_size); + return 0; +error: + ret = rte_errno; + if (umem_obj) + claim_zero(mlx5_os_umem_dereg(umem_obj)); + if (umem_buf) + mlx5_free((void *)(uintptr_t)umem_buf); + rte_errno = ret; + return -rte_errno; +} diff --git a/drivers/common/mlx5/mlx5_common_devx.h b/drivers/common/mlx5/mlx5_common_devx.h new file mode 100644 index 0000000..20d5da0 --- /dev/null +++ b/drivers/common/mlx5/mlx5_common_devx.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#ifndef RTE_PMD_MLX5_COMMON_DEVX_H_ +#define RTE_PMD_MLX5_COMMON_DEVX_H_ + +#include "mlx5_devx_cmds.h" + +/* DevX Completion Queue structure. */ +struct mlx5_devx_cq { + struct mlx5_devx_obj *cq; /* The CQ DevX object. */ + void *umem_obj; /* The CQ umem object. */ + union { + volatile void *umem_buf; + volatile struct mlx5_cqe *cqes; /* The CQ ring buffer. */ + }; + volatile uint32_t *db_rec; /* The CQ doorbell record. */ +}; + +/* mlx5_common_devx.c */ + +__rte_internal +void mlx5_devx_cq_destroy(struct mlx5_devx_cq *cq); + +__rte_internal +int mlx5_devx_cq_create(void *ctx, struct mlx5_devx_cq *cq_obj, + uint16_t log_desc_n, struct mlx5_devx_cq_attr *attr, + int socket); + +#endif /* RTE_PMD_MLX5_COMMON_DEVX_H_ */ diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def index 65ae47a..d10db40 100644 --- a/drivers/common/mlx5/rte_common_mlx5_exports.def +++ b/drivers/common/mlx5/rte_common_mlx5_exports.def @@ -35,6 +35,9 @@ EXPORTS mlx5_devx_get_out_command_status mlx5_devx_cmd_create_flow_hit_aso_obj + mlx5_devx_cq_create + mlx5_devx_cq_destroy + mlx5_get_dbr mlx5_glue diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index fb3952b..850202c 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -43,6 +43,9 @@ INTERNAL { mlx5_devx_get_out_command_status; mlx5_devx_alloc_uar; + mlx5_devx_cq_create; + mlx5_devx_cq_destroy; + mlx5_get_ifname_sysfs; mlx5_get_dbr; diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h index 111af2e..60610df 100644 --- a/drivers/common/mlx5/windows/mlx5_win_ext.h +++ b/drivers/common/mlx5/windows/mlx5_win_ext.h @@ -9,6 +9,7 @@ extern "C" { #endif +#include "mlx5_prm.h" #include "mlx5devx.h" typedef struct mlx5_context { From patchwork Wed Jan 6 08:19:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86029 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9EAEBA09FF; Wed, 6 Jan 2021 09:20:59 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 533781608CC; Wed, 6 Jan 2021 09:20:53 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 8DA5516069F for ; Wed, 6 Jan 2021 09:20:51 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:20:50 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgY009291; Wed, 6 Jan 2021 10:20:50 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:27 +0000 Message-Id: <1609921181-5019-6-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 05/19] regex/mlx5: move DevX CQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for DevX CQ creation. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/regex/mlx5/mlx5_regex.c | 6 --- drivers/regex/mlx5/mlx5_regex.h | 9 +--- drivers/regex/mlx5/mlx5_regex_control.c | 91 ++++++-------------------------- drivers/regex/mlx5/mlx5_regex_fastpath.c | 4 +- 4 files changed, 20 insertions(+), 90 deletions(-) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index c91c444..c0d6331 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -170,12 +170,6 @@ rte_errno = rte_errno ? rte_errno : EINVAL; goto error; } - ret = mlx5_glue->devx_query_eqn(ctx, 0, &priv->eqn); - if (ret) { - DRV_LOG(ERR, "can't query event queue number."); - rte_errno = ENOMEM; - goto error; - } /* * This PMD always claims the write memory barrier on UAR * registers writings, it is safe to allocate UAR with any diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 2c4877c..9f7a388 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -12,6 +12,7 @@ #include #include +#include #include "mlx5_rxp.h" @@ -30,13 +31,8 @@ struct mlx5_regex_sq { struct mlx5_regex_cq { uint32_t log_nb_desc; /* Log 2 number of desc for this object. */ - struct mlx5_devx_obj *obj; /* The CQ DevX object. */ - int64_t dbr_offset; /* Door bell record offset. */ - uint32_t dbr_umem; /* Door bell record umem id. */ - volatile struct mlx5_cqe *cqe; /* The CQ ring buffer. */ - struct mlx5dv_devx_umem *cqe_umem; /* CQ buffer umem. */ + struct mlx5_devx_cq cq_obj; /* The CQ DevX object. */ size_t ci; - uint32_t *dbr; }; struct mlx5_regex_qp { @@ -75,7 +71,6 @@ struct mlx5_regex_priv { struct mlx5_regex_db db[MLX5_RXP_MAX_ENGINES + MLX5_RXP_EM_COUNT]; uint32_t nb_engines; /* Number of RegEx engines. */ - uint32_t eqn; /* EQ number. */ struct mlx5dv_devx_uar *uar; /* UAR object. */ struct ibv_pd *pd; struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index d6f452b..ca6c0f5 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -6,6 +6,7 @@ #include #include +#include #include #include #include @@ -17,6 +18,7 @@ #include #include #include +#include #include "mlx5_regex.h" #include "mlx5_regex_utils.h" @@ -44,8 +46,6 @@ /** * destroy CQ. * - * @param priv - * Pointer to the priv object. * @param cp * Pointer to the CQ to be destroyed. * @@ -53,24 +53,10 @@ * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -regex_ctrl_destroy_cq(struct mlx5_regex_priv *priv, struct mlx5_regex_cq *cq) +regex_ctrl_destroy_cq(struct mlx5_regex_cq *cq) { - if (cq->cqe_umem) { - mlx5_glue->devx_umem_dereg(cq->cqe_umem); - cq->cqe_umem = NULL; - } - if (cq->cqe) { - rte_free((void *)(uintptr_t)cq->cqe); - cq->cqe = NULL; - } - if (cq->dbr_offset) { - mlx5_release_dbr(&priv->dbrpgs, cq->dbr_umem, cq->dbr_offset); - cq->dbr_offset = -1; - } - if (cq->obj) { - mlx5_devx_cmd_destroy(cq->obj); - cq->obj = NULL; - } + mlx5_devx_cq_destroy(&cq->cq_obj); + memset(cq, 0, sizeof(*cq)); return 0; } @@ -89,65 +75,20 @@ regex_ctrl_create_cq(struct mlx5_regex_priv *priv, struct mlx5_regex_cq *cq) { struct mlx5_devx_cq_attr attr = { - .q_umem_valid = 1, - .db_umem_valid = 1, - .eqn = priv->eqn, + .uar_page_id = priv->uar->page_id, }; - struct mlx5_devx_dbr_page *dbr_page = NULL; - void *buf = NULL; - size_t pgsize = sysconf(_SC_PAGESIZE); - uint32_t cq_size = 1 << cq->log_nb_desc; - uint32_t i; - - cq->dbr_offset = mlx5_get_dbr(priv->ctx, &priv->dbrpgs, &dbr_page); - if (cq->dbr_offset < 0) { - DRV_LOG(ERR, "Can't allocate cq door bell record."); - rte_errno = ENOMEM; - goto error; - } - cq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); - cq->dbr = (uint32_t *)((uintptr_t)dbr_page->dbrs + - (uintptr_t)cq->dbr_offset); + int ret; - buf = rte_calloc(NULL, 1, sizeof(struct mlx5_cqe) * cq_size, 4096); - if (!buf) { - DRV_LOG(ERR, "Can't allocate cqe buffer."); - rte_errno = ENOMEM; - goto error; - } - cq->cqe = buf; - for (i = 0; i < cq_size; i++) - cq->cqe[i].op_own = 0xff; - cq->cqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, - sizeof(struct mlx5_cqe) * - cq_size, 7); cq->ci = 0; - if (!cq->cqe_umem) { - DRV_LOG(ERR, "Can't register cqe mem."); - rte_errno = ENOMEM; - goto error; - } - attr.db_umem_offset = cq->dbr_offset; - attr.db_umem_id = cq->dbr_umem; - attr.q_umem_id = mlx5_os_get_umem_id(cq->cqe_umem); - attr.log_cq_size = cq->log_nb_desc; - attr.uar_page_id = priv->uar->page_id; - attr.log_page_size = rte_log2_u32(pgsize); - cq->obj = mlx5_devx_cmd_create_cq(priv->ctx, &attr); - if (!cq->obj) { - DRV_LOG(ERR, "Can't create cq object."); - rte_errno = ENOMEM; - goto error; + ret = mlx5_devx_cq_create(priv->ctx, &cq->cq_obj, cq->log_nb_desc, + &attr, SOCKET_ID_ANY); + if (ret) { + DRV_LOG(ERR, "Can't create CQ object."); + memset(cq, 0, sizeof(*cq)); + rte_errno = ENOMEM; + return -rte_errno; } return 0; -error: - if (cq->cqe_umem) - mlx5_glue->devx_umem_dereg(cq->cqe_umem); - if (buf) - rte_free(buf); - if (cq->dbr_offset) - mlx5_release_dbr(&priv->dbrpgs, cq->dbr_umem, cq->dbr_offset); - return -rte_errno; } #ifdef HAVE_IBV_FLOW_DV_SUPPORT @@ -232,7 +173,7 @@ attr.tis_lst_sz = 0; attr.tis_num = 0; attr.user_index = q_ind; - attr.cqn = qp->cq.obj->id; + attr.cqn = qp->cq.cq_obj.cq->id; wq_attr->uar_page = priv->uar->page_id; regex_get_pdn(priv->pd, &pd_num); wq_attr->pd = pd_num; @@ -389,7 +330,7 @@ err_btree: for (i = 0; i < nb_sq_config; i++) regex_ctrl_destroy_sq(priv, qp, i); - regex_ctrl_destroy_cq(priv, &qp->cq); + regex_ctrl_destroy_cq(&qp->cq); err_cq: rte_free(qp->sqs); return ret; diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c index 5857617..255fd40 100644 --- a/drivers/regex/mlx5/mlx5_regex_fastpath.c +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -224,7 +224,7 @@ struct mlx5_regex_job { size_t next_cqe_offset; next_cqe_offset = (cq->ci & (cq_size_get(cq) - 1)); - cqe = (volatile struct mlx5_cqe *)(cq->cqe + next_cqe_offset); + cqe = (volatile struct mlx5_cqe *)(cq->cq_obj.cqes + next_cqe_offset); rte_io_wmb(); int ret = check_cqe(cqe, cq_size_get(cq), cq->ci); @@ -285,7 +285,7 @@ struct mlx5_regex_job { } cq->ci = (cq->ci + 1) & 0xffffff; rte_wmb(); - cq->dbr[0] = rte_cpu_to_be_32(cq->ci); + cq->cq_obj.db_rec[0] = rte_cpu_to_be_32(cq->ci); queue->free_sqs |= (1 << sqid); } From patchwork Wed Jan 6 08:19:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86030 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 46531A09FF; Wed, 6 Jan 2021 09:21:09 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D87DF1608D6; Wed, 6 Jan 2021 09:20:57 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id AC30F1608DD for ; Wed, 6 Jan 2021 09:20:56 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:20:56 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgZ009291; Wed, 6 Jan 2021 10:20:56 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:28 +0000 Message-Id: <1609921181-5019-7-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 06/19] vdpa/mlx5: move DevX CQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for DevX CQ creation. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/vdpa/mlx5/mlx5_vdpa.h | 10 +---- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 86 +++++++++++-------------------------- drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 2 +- 3 files changed, 28 insertions(+), 70 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h index d039ada..ddee9dc 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/mlx5_vdpa.h @@ -22,6 +22,7 @@ #include #include +#include #include @@ -46,13 +47,7 @@ struct mlx5_vdpa_cq { uint32_t armed:1; int callfd; rte_spinlock_t sl; - struct mlx5_devx_obj *cq; - struct mlx5dv_devx_umem *umem_obj; - union { - volatile void *umem_buf; - volatile struct mlx5_cqe *cqes; - }; - volatile uint32_t *db_rec; + struct mlx5_devx_cq cq_obj; uint64_t errors; }; @@ -144,7 +139,6 @@ struct mlx5_vdpa_priv { uint32_t gpa_mkey_index; struct ibv_mr *null_mr; struct rte_vhost_memory *vmem; - uint32_t eqn; struct mlx5dv_devx_event_channel *eventc; struct mlx5dv_devx_event_channel *err_chnl; struct mlx5dv_devx_uar *uar; diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c index fd61509..eba783a 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c @@ -7,6 +7,7 @@ #include #include +#include #include #include #include @@ -16,6 +17,7 @@ #include #include +#include #include #include "mlx5_vdpa_utils.h" @@ -48,7 +50,6 @@ priv->eventc = NULL; } #endif - priv->eqn = 0; } /* Prepare all the global resources for all the event objects.*/ @@ -59,11 +60,6 @@ if (priv->eventc) return 0; - if (mlx5_glue->devx_query_eqn(priv->ctx, 0, &priv->eqn)) { - rte_errno = errno; - DRV_LOG(ERR, "Failed to query EQ number %d.", rte_errno); - return -1; - } priv->eventc = mlx5_os_devx_create_event_channel(priv->ctx, MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA); if (!priv->eventc) { @@ -98,12 +94,7 @@ static void mlx5_vdpa_cq_destroy(struct mlx5_vdpa_cq *cq) { - if (cq->cq) - claim_zero(mlx5_devx_cmd_destroy(cq->cq)); - if (cq->umem_obj) - claim_zero(mlx5_glue->devx_umem_dereg(cq->umem_obj)); - if (cq->umem_buf) - rte_free((void *)(uintptr_t)cq->umem_buf); + mlx5_devx_cq_destroy(&cq->cq_obj); memset(cq, 0, sizeof(*cq)); } @@ -113,12 +104,12 @@ uint32_t arm_sn = cq->arm_sn << MLX5_CQ_SQN_OFFSET; uint32_t cq_ci = cq->cq_ci & MLX5_CI_MASK; uint32_t doorbell_hi = arm_sn | MLX5_CQ_DBR_CMD_ALL | cq_ci; - uint64_t doorbell = ((uint64_t)doorbell_hi << 32) | cq->cq->id; + uint64_t doorbell = ((uint64_t)doorbell_hi << 32) | cq->cq_obj.cq->id; uint64_t db_be = rte_cpu_to_be_64(doorbell); uint32_t *addr = RTE_PTR_ADD(priv->uar->base_addr, MLX5_CQ_DOORBELL); rte_io_wmb(); - cq->db_rec[MLX5_CQ_ARM_DB] = rte_cpu_to_be_32(doorbell_hi); + cq->cq_obj.db_rec[MLX5_CQ_ARM_DB] = rte_cpu_to_be_32(doorbell_hi); rte_wmb(); #ifdef RTE_ARCH_64 *(uint64_t *)addr = db_be; @@ -135,52 +126,25 @@ mlx5_vdpa_cq_create(struct mlx5_vdpa_priv *priv, uint16_t log_desc_n, int callfd, struct mlx5_vdpa_cq *cq) { - struct mlx5_devx_cq_attr attr = {0}; - size_t pgsize = sysconf(_SC_PAGESIZE); - uint32_t umem_size; + struct mlx5_devx_cq_attr attr = { + .use_first_only = 1, + .uar_page_id = priv->uar->page_id, + }; uint16_t event_nums[1] = {0}; - uint16_t cq_size = 1 << log_desc_n; int ret; - cq->log_desc_n = log_desc_n; - umem_size = sizeof(struct mlx5_cqe) * cq_size + sizeof(*cq->db_rec) * 2; - cq->umem_buf = rte_zmalloc(__func__, umem_size, 4096); - if (!cq->umem_buf) { - DRV_LOG(ERR, "Failed to allocate memory for CQ."); - rte_errno = ENOMEM; - return -ENOMEM; - } - cq->umem_obj = mlx5_glue->devx_umem_reg(priv->ctx, - (void *)(uintptr_t)cq->umem_buf, - umem_size, - IBV_ACCESS_LOCAL_WRITE); - if (!cq->umem_obj) { - DRV_LOG(ERR, "Failed to register umem for CQ."); - goto error; - } - attr.q_umem_valid = 1; - attr.db_umem_valid = 1; - attr.use_first_only = 1; - attr.overrun_ignore = 0; - attr.uar_page_id = priv->uar->page_id; - attr.q_umem_id = cq->umem_obj->umem_id; - attr.q_umem_offset = 0; - attr.db_umem_id = cq->umem_obj->umem_id; - attr.db_umem_offset = sizeof(struct mlx5_cqe) * cq_size; - attr.eqn = priv->eqn; - attr.log_cq_size = log_desc_n; - attr.log_page_size = rte_log2_u32(pgsize); - cq->cq = mlx5_devx_cmd_create_cq(priv->ctx, &attr); - if (!cq->cq) + ret = mlx5_devx_cq_create(priv->ctx, &cq->cq_obj, log_desc_n, &attr, + SOCKET_ID_ANY); + if (ret) goto error; - cq->db_rec = RTE_PTR_ADD(cq->umem_buf, (uintptr_t)attr.db_umem_offset); cq->cq_ci = 0; + cq->log_desc_n = log_desc_n; rte_spinlock_init(&cq->sl); /* Subscribe CQ event to the event channel controlled by the driver. */ - ret = mlx5_os_devx_subscribe_devx_event(priv->eventc, cq->cq->obj, - sizeof(event_nums), - event_nums, - (uint64_t)(uintptr_t)cq); + ret = mlx5_os_devx_subscribe_devx_event(priv->eventc, + cq->cq_obj.cq->obj, + sizeof(event_nums), event_nums, + (uint64_t)(uintptr_t)cq); if (ret) { DRV_LOG(ERR, "Failed to subscribe CQE event."); rte_errno = errno; @@ -188,8 +152,8 @@ } cq->callfd = callfd; /* Init CQ to ones to be in HW owner in the start. */ - cq->cqes[0].op_own = MLX5_CQE_OWNER_MASK; - cq->cqes[0].wqe_counter = rte_cpu_to_be_16(UINT16_MAX); + cq->cq_obj.cqes[0].op_own = MLX5_CQE_OWNER_MASK; + cq->cq_obj.cqes[0].wqe_counter = rte_cpu_to_be_16(UINT16_MAX); /* First arming. */ mlx5_vdpa_cq_arm(priv, cq); return 0; @@ -216,7 +180,7 @@ uint16_t cur_wqe_counter; uint16_t comp; - last_word.word = rte_read32(&cq->cqes[0].wqe_counter); + last_word.word = rte_read32(&cq->cq_obj.cqes[0].wqe_counter); cur_wqe_counter = rte_be_to_cpu_16(last_word.wqe_counter); comp = cur_wqe_counter + (uint16_t)1 - next_wqe_counter; if (comp) { @@ -230,7 +194,7 @@ cq->errors++; rte_io_wmb(); /* Ring CQ doorbell record. */ - cq->db_rec[0] = rte_cpu_to_be_32(cq->cq_ci); + cq->cq_obj.db_rec[0] = rte_cpu_to_be_32(cq->cq_ci); rte_io_wmb(); /* Ring SW QP doorbell record. */ eqp->db_rec[0] = rte_cpu_to_be_32(cq->cq_ci + cq_size); @@ -246,7 +210,7 @@ for (i = 0; i < priv->nr_virtqs; i++) { cq = &priv->virtqs[i].eqp.cq; - if (cq->cq && !cq->armed) + if (cq->cq_obj.cq && !cq->armed) mlx5_vdpa_cq_arm(priv, cq); } } @@ -291,7 +255,7 @@ pthread_mutex_lock(&priv->vq_config_lock); for (i = 0; i < priv->nr_virtqs; i++) { cq = &priv->virtqs[i].eqp.cq; - if (cq->cq && !cq->armed) { + if (cq->cq_obj.cq && !cq->armed) { uint32_t comp = mlx5_vdpa_cq_poll(cq); if (comp) { @@ -370,7 +334,7 @@ DRV_LOG(DEBUG, "Device %s virtq %d cq %d event was captured." " Timer is %s, cq ci is %u.\n", priv->vdev->device->name, - (int)virtq->index, cq->cq->id, + (int)virtq->index, cq->cq_obj.cq->id, priv->timer_on ? "on" : "off", cq->cq_ci); cq->armed = 0; } @@ -680,7 +644,7 @@ goto error; } attr.uar_index = priv->uar->page_id; - attr.cqn = eqp->cq.cq->id; + attr.cqn = eqp->cq.cq_obj.cq->id; attr.log_page_size = rte_log2_u32(sysconf(_SC_PAGESIZE)); attr.rq_size = 1 << log_desc_n; attr.log_rq_stride = rte_log2_u32(MLX5_WSEG_SIZE); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c index 3e882e4..cc77314 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c @@ -497,7 +497,7 @@ return -1; if (vq.size != virtq->vq_size || vq.kickfd != virtq->intr_handle.fd) return 1; - if (virtq->eqp.cq.cq) { + if (virtq->eqp.cq.cq_obj.cq) { if (vq.callfd != virtq->eqp.cq.callfd) return 1; } else if (vq.callfd != -1) { From patchwork Wed Jan 6 08:19:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86031 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 238E7A09FF; Wed, 6 Jan 2021 09:21:17 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 11EA816089B; Wed, 6 Jan 2021 09:21:08 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 13CC916069F for ; Wed, 6 Jan 2021 09:21:07 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:03 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAga009291; Wed, 6 Jan 2021 10:21:03 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:29 +0000 Message-Id: <1609921181-5019-8-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 07/19] net/mlx5: move rearm and clock queue CQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for CQ creation at rearm queue and clock queue. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.h | 9 +-- drivers/net/mlx5/mlx5_rxtx.c | 2 +- drivers/net/mlx5/mlx5_txpp.c | 139 ++++++++++--------------------------------- 3 files changed, 36 insertions(+), 114 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 92a5d04..ba2a8c4 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -26,6 +26,7 @@ #include #include #include +#include #include "mlx5_defs.h" #include "mlx5_utils.h" @@ -616,13 +617,7 @@ struct mlx5_flow_id_pool { /* Tx pacing queue structure - for Clock and Rearm queues. */ struct mlx5_txpp_wq { /* Completion Queue related data.*/ - struct mlx5_devx_obj *cq; - void *cq_umem; - union { - volatile void *cq_buf; - volatile struct mlx5_cqe *cqes; - }; - volatile uint32_t *cq_dbrec; + struct mlx5_devx_cq cq_obj; uint32_t cq_ci:24; uint32_t arm_sn:2; /* Send Queue related data.*/ diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 65a1f99..3497765 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -2325,7 +2325,7 @@ enum mlx5_txcmp_code { qs = RTE_PTR_ADD(wqe, MLX5_WSEG_SIZE); qs->max_index = rte_cpu_to_be_32(wci); - qs->qpn_cqn = rte_cpu_to_be_32(txq->sh->txpp.clock_queue.cq->id); + qs->qpn_cqn = rte_cpu_to_be_32(txq->sh->txpp.clock_queue.cq_obj.cq->id); qs->reserved0 = RTE_BE32(0); qs->reserved1 = RTE_BE32(0); } diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c index e998de3..bd679c2 100644 --- a/drivers/net/mlx5/mlx5_txpp.c +++ b/drivers/net/mlx5/mlx5_txpp.c @@ -13,6 +13,7 @@ #include #include +#include #include "mlx5.h" #include "mlx5_rxtx.h" @@ -134,12 +135,7 @@ claim_zero(mlx5_os_umem_dereg(wq->sq_umem)); if (wq->sq_buf) mlx5_free((void *)(uintptr_t)wq->sq_buf); - if (wq->cq) - claim_zero(mlx5_devx_cmd_destroy(wq->cq)); - if (wq->cq_umem) - claim_zero(mlx5_os_umem_dereg(wq->cq_umem)); - if (wq->cq_buf) - mlx5_free((void *)(uintptr_t)wq->cq_buf); + mlx5_devx_cq_destroy(&wq->cq_obj); memset(wq, 0, sizeof(*wq)); } @@ -189,19 +185,6 @@ } static void -mlx5_txpp_fill_cqe_rearm_queue(struct mlx5_dev_ctx_shared *sh) -{ - struct mlx5_txpp_wq *wq = &sh->txpp.rearm_queue; - struct mlx5_cqe *cqe = (struct mlx5_cqe *)(uintptr_t)wq->cqes; - uint32_t i; - - for (i = 0; i < MLX5_TXPP_REARM_CQ_SIZE; i++) { - cqe->op_own = (MLX5_CQE_INVALID << 4) | MLX5_CQE_OWNER_MASK; - ++cqe; - } -} - -static void mlx5_txpp_fill_wqe_rearm_queue(struct mlx5_dev_ctx_shared *sh) { struct mlx5_txpp_wq *wq = &sh->txpp.rearm_queue; @@ -236,7 +219,8 @@ index = (i * MLX5_TXPP_REARM / 2 + MLX5_TXPP_REARM / 2) & ((1 << MLX5_CQ_INDEX_WIDTH) - 1); qs->max_index = rte_cpu_to_be_32(index); - qs->qpn_cqn = rte_cpu_to_be_32(sh->txpp.clock_queue.cq->id); + qs->qpn_cqn = + rte_cpu_to_be_32(sh->txpp.clock_queue.cq_obj.cq->id); } } @@ -246,7 +230,9 @@ { struct mlx5_devx_create_sq_attr sq_attr = { 0 }; struct mlx5_devx_modify_sq_attr msq_attr = { 0 }; - struct mlx5_devx_cq_attr cq_attr = { 0 }; + struct mlx5_devx_cq_attr cq_attr = { + .uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar), + }; struct mlx5_txpp_wq *wq = &sh->txpp.rearm_queue; size_t page_size; uint32_t umem_size, umem_dbrec; @@ -257,48 +243,16 @@ DRV_LOG(ERR, "Failed to get mem page size"); return -ENOMEM; } - /* Allocate memory buffer for CQEs and doorbell record. */ - umem_size = sizeof(struct mlx5_cqe) * MLX5_TXPP_REARM_CQ_SIZE; - umem_dbrec = RTE_ALIGN(umem_size, MLX5_DBR_SIZE); - umem_size += MLX5_DBR_SIZE; - wq->cq_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size, - page_size, sh->numa_node); - if (!wq->cq_buf) { - DRV_LOG(ERR, "Failed to allocate memory for Rearm Queue."); - return -ENOMEM; - } - /* Register allocated buffer in user space with DevX. */ - wq->cq_umem = mlx5_os_umem_reg(sh->ctx, - (void *)(uintptr_t)wq->cq_buf, - umem_size, - IBV_ACCESS_LOCAL_WRITE); - if (!wq->cq_umem) { - rte_errno = errno; - DRV_LOG(ERR, "Failed to register umem for Rearm Queue."); - goto error; - } /* Create completion queue object for Rearm Queue. */ - cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar); - cq_attr.eqn = sh->eqn; - cq_attr.q_umem_valid = 1; - cq_attr.q_umem_offset = 0; - cq_attr.q_umem_id = mlx5_os_get_umem_id(wq->cq_umem); - cq_attr.db_umem_valid = 1; - cq_attr.db_umem_offset = umem_dbrec; - cq_attr.db_umem_id = mlx5_os_get_umem_id(wq->cq_umem); - cq_attr.log_cq_size = rte_log2_u32(MLX5_TXPP_REARM_CQ_SIZE); - cq_attr.log_page_size = rte_log2_u32(page_size); - wq->cq = mlx5_devx_cmd_create_cq(sh->ctx, &cq_attr); - if (!wq->cq) { - rte_errno = errno; + ret = mlx5_devx_cq_create(sh->ctx, &wq->cq_obj, + log2above(MLX5_TXPP_REARM_CQ_SIZE), &cq_attr, + sh->numa_node); + if (ret) { DRV_LOG(ERR, "Failed to create CQ for Rearm Queue."); - goto error; + return ret; } - wq->cq_dbrec = RTE_PTR_ADD(wq->cq_buf, umem_dbrec); wq->cq_ci = 0; wq->arm_sn = 0; - /* Mark all CQEs initially as invalid. */ - mlx5_txpp_fill_cqe_rearm_queue(sh); /* * Allocate memory buffer for Send Queue WQEs. * There should be no WQE leftovers in the cyclic queue. @@ -329,7 +283,7 @@ sq_attr.state = MLX5_SQC_STATE_RST; sq_attr.tis_lst_sz = 1; sq_attr.tis_num = sh->tis->id; - sq_attr.cqn = wq->cq->id; + sq_attr.cqn = wq->cq_obj.cq->id; sq_attr.cd_master = 1; sq_attr.wq_attr.uar_page = mlx5_os_get_devx_uar_page_id(sh->tx_uar); sq_attr.wq_attr.wq_type = MLX5_WQ_TYPE_CYCLIC; @@ -472,7 +426,11 @@ { struct mlx5_devx_create_sq_attr sq_attr = { 0 }; struct mlx5_devx_modify_sq_attr msq_attr = { 0 }; - struct mlx5_devx_cq_attr cq_attr = { 0 }; + struct mlx5_devx_cq_attr cq_attr = { + .use_first_only = 1, + .overrun_ignore = 1, + .uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar), + }; struct mlx5_txpp_wq *wq = &sh->txpp.clock_queue; size_t page_size; uint32_t umem_size, umem_dbrec; @@ -493,46 +451,14 @@ } sh->txpp.ts_p = 0; sh->txpp.ts_n = 0; - /* Allocate memory buffer for CQEs and doorbell record. */ - umem_size = sizeof(struct mlx5_cqe) * MLX5_TXPP_CLKQ_SIZE; - umem_dbrec = RTE_ALIGN(umem_size, MLX5_DBR_SIZE); - umem_size += MLX5_DBR_SIZE; - wq->cq_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size, - page_size, sh->numa_node); - if (!wq->cq_buf) { - DRV_LOG(ERR, "Failed to allocate memory for Clock Queue."); - return -ENOMEM; - } - /* Register allocated buffer in user space with DevX. */ - wq->cq_umem = mlx5_os_umem_reg(sh->ctx, - (void *)(uintptr_t)wq->cq_buf, - umem_size, - IBV_ACCESS_LOCAL_WRITE); - if (!wq->cq_umem) { - rte_errno = errno; - DRV_LOG(ERR, "Failed to register umem for Clock Queue."); - goto error; - } /* Create completion queue object for Clock Queue. */ - cq_attr.use_first_only = 1; - cq_attr.overrun_ignore = 1; - cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar); - cq_attr.eqn = sh->eqn; - cq_attr.q_umem_valid = 1; - cq_attr.q_umem_offset = 0; - cq_attr.q_umem_id = mlx5_os_get_umem_id(wq->cq_umem); - cq_attr.db_umem_valid = 1; - cq_attr.db_umem_offset = umem_dbrec; - cq_attr.db_umem_id = mlx5_os_get_umem_id(wq->cq_umem); - cq_attr.log_cq_size = rte_log2_u32(MLX5_TXPP_CLKQ_SIZE); - cq_attr.log_page_size = rte_log2_u32(page_size); - wq->cq = mlx5_devx_cmd_create_cq(sh->ctx, &cq_attr); - if (!wq->cq) { - rte_errno = errno; + ret = mlx5_devx_cq_create(sh->ctx, &wq->cq_obj, + log2above(MLX5_TXPP_CLKQ_SIZE), &cq_attr, + sh->numa_node); + if (ret) { DRV_LOG(ERR, "Failed to create CQ for Clock Queue."); goto error; } - wq->cq_dbrec = RTE_PTR_ADD(wq->cq_buf, umem_dbrec); wq->cq_ci = 0; /* Allocate memory buffer for Send Queue WQEs. */ if (sh->txpp.test) { @@ -578,7 +504,7 @@ sq_attr.static_sq_wq = 1; } sq_attr.state = MLX5_SQC_STATE_RST; - sq_attr.cqn = wq->cq->id; + sq_attr.cqn = wq->cq_obj.cq->id; sq_attr.packet_pacing_rate_limit_index = sh->txpp.pp_id; sq_attr.wq_attr.cd_slave = 1; sq_attr.wq_attr.uar_page = mlx5_os_get_devx_uar_page_id(sh->tx_uar); @@ -629,12 +555,13 @@ struct mlx5_txpp_wq *aq = &sh->txpp.rearm_queue; uint32_t arm_sn = aq->arm_sn << MLX5_CQ_SQN_OFFSET; uint32_t db_hi = arm_sn | MLX5_CQ_DBR_CMD_ALL | aq->cq_ci; - uint64_t db_be = rte_cpu_to_be_64(((uint64_t)db_hi << 32) | aq->cq->id); + uint64_t db_be = + rte_cpu_to_be_64(((uint64_t)db_hi << 32) | aq->cq_obj.cq->id); base_addr = mlx5_os_get_devx_uar_base_addr(sh->tx_uar); uint32_t *addr = RTE_PTR_ADD(base_addr, MLX5_CQ_DOORBELL); rte_compiler_barrier(); - aq->cq_dbrec[MLX5_CQ_ARM_DB] = rte_cpu_to_be_32(db_hi); + aq->cq_obj.db_rec[MLX5_CQ_ARM_DB] = rte_cpu_to_be_32(db_hi); rte_wmb(); #ifdef RTE_ARCH_64 *(uint64_t *)addr = db_be; @@ -732,7 +659,7 @@ mlx5_txpp_update_timestamp(struct mlx5_dev_ctx_shared *sh) { struct mlx5_txpp_wq *wq = &sh->txpp.clock_queue; - struct mlx5_cqe *cqe = (struct mlx5_cqe *)(uintptr_t)wq->cqes; + struct mlx5_cqe *cqe = (struct mlx5_cqe *)(uintptr_t)wq->cq_obj.cqes; union { rte_int128_t u128; struct mlx5_cqe_ts cts; @@ -807,7 +734,7 @@ do { volatile struct mlx5_cqe *cqe; - cqe = &wq->cqes[cq_ci & (MLX5_TXPP_REARM_CQ_SIZE - 1)]; + cqe = &wq->cq_obj.cqes[cq_ci & (MLX5_TXPP_REARM_CQ_SIZE - 1)]; ret = check_cqe(cqe, MLX5_TXPP_REARM_CQ_SIZE, cq_ci); switch (ret) { case MLX5_CQE_STATUS_ERR: @@ -839,7 +766,7 @@ } /* Update doorbell record to notify hardware. */ rte_compiler_barrier(); - *wq->cq_dbrec = rte_cpu_to_be_32(cq_ci); + *wq->cq_obj.db_rec = rte_cpu_to_be_32(cq_ci); rte_wmb(); wq->cq_ci = cq_ci; /* Fire new requests to Rearm Queue. */ @@ -934,9 +861,8 @@ } /* Subscribe CQ event to the event channel controlled by the driver. */ ret = mlx5_os_devx_subscribe_devx_event(sh->txpp.echan, - sh->txpp.rearm_queue.cq->obj, - sizeof(event_nums), - event_nums, 0); + sh->txpp.rearm_queue.cq_obj.cq->obj, + sizeof(event_nums), event_nums, 0); if (ret) { DRV_LOG(ERR, "Failed to subscribe CQE event."); rte_errno = errno; @@ -1138,7 +1064,8 @@ if (sh->txpp.refcnt) { struct mlx5_txpp_wq *wq = &sh->txpp.clock_queue; - struct mlx5_cqe *cqe = (struct mlx5_cqe *)(uintptr_t)wq->cqes; + struct mlx5_cqe *cqe = + (struct mlx5_cqe *)(uintptr_t)wq->cq_obj.cqes; union { rte_int128_t u128; struct mlx5_cqe_ts cts; From patchwork Wed Jan 6 08:19:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86032 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D3C51A09FF; Wed, 6 Jan 2021 09:21:25 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 476A61608E1; Wed, 6 Jan 2021 09:21:14 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 54FFE1608E0 for ; Wed, 6 Jan 2021 09:21:12 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:07 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgb009291; Wed, 6 Jan 2021 10:21:07 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:30 +0000 Message-Id: <1609921181-5019-9-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 08/19] net/mlx5: move ASO CQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Use common function for ASO CQ creation. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.h | 8 +--- drivers/net/mlx5/mlx5_flow_age.c | 82 +++++++++------------------------------- 2 files changed, 19 insertions(+), 71 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index ba2a8c4..f889180 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -467,13 +467,7 @@ struct mlx5_flow_counter_mng { struct mlx5_aso_cq { uint16_t log_desc_n; uint32_t cq_ci:24; - struct mlx5_devx_obj *cq; - struct mlx5dv_devx_umem *umem_obj; - union { - volatile void *umem_buf; - volatile struct mlx5_cqe *cqes; - }; - volatile uint32_t *db_rec; + struct mlx5_devx_cq cq_obj; uint64_t errors; }; diff --git a/drivers/net/mlx5/mlx5_flow_age.c b/drivers/net/mlx5/mlx5_flow_age.c index e867607..a75adc8 100644 --- a/drivers/net/mlx5/mlx5_flow_age.c +++ b/drivers/net/mlx5/mlx5_flow_age.c @@ -8,10 +8,12 @@ #include #include +#include #include "mlx5.h" #include "mlx5_flow.h" + /** * Destroy Completion Queue used for ASO access. * @@ -21,12 +23,8 @@ static void mlx5_aso_cq_destroy(struct mlx5_aso_cq *cq) { - if (cq->cq) - claim_zero(mlx5_devx_cmd_destroy(cq->cq)); - if (cq->umem_obj) - claim_zero(mlx5_glue->devx_umem_dereg(cq->umem_obj)); - if (cq->umem_buf) - mlx5_free((void *)(uintptr_t)cq->umem_buf); + if (cq->cq_obj.cq) + mlx5_devx_cq_destroy(&cq->cq_obj); memset(cq, 0, sizeof(*cq)); } @@ -43,60 +41,21 @@ * Socket to use for allocation. * @param[in] uar_page_id * UAR page ID to use. - * @param[in] eqn - * EQ number. * * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int mlx5_aso_cq_create(void *ctx, struct mlx5_aso_cq *cq, uint16_t log_desc_n, - int socket, int uar_page_id, uint32_t eqn) + int socket, int uar_page_id) { - struct mlx5_devx_cq_attr attr = { 0 }; - size_t pgsize = rte_mem_page_size(); - uint32_t umem_size; - uint16_t cq_size = 1 << log_desc_n; + struct mlx5_devx_cq_attr attr = { + .uar_page_id = uar_page_id, + }; cq->log_desc_n = log_desc_n; - umem_size = sizeof(struct mlx5_cqe) * cq_size + sizeof(*cq->db_rec) * 2; - cq->umem_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size, - 4096, socket); - if (!cq->umem_buf) { - DRV_LOG(ERR, "Failed to allocate memory for CQ."); - rte_errno = ENOMEM; - return -ENOMEM; - } - cq->umem_obj = mlx5_os_umem_reg(ctx, - (void *)(uintptr_t)cq->umem_buf, - umem_size, - IBV_ACCESS_LOCAL_WRITE); - if (!cq->umem_obj) { - DRV_LOG(ERR, "Failed to register umem for aso CQ."); - goto error; - } - attr.q_umem_valid = 1; - attr.db_umem_valid = 1; - attr.use_first_only = 0; - attr.overrun_ignore = 0; - attr.uar_page_id = uar_page_id; - attr.q_umem_id = mlx5_os_get_umem_id(cq->umem_obj); - attr.q_umem_offset = 0; - attr.db_umem_id = attr.q_umem_id; - attr.db_umem_offset = sizeof(struct mlx5_cqe) * cq_size; - attr.eqn = eqn; - attr.log_cq_size = log_desc_n; - attr.log_page_size = rte_log2_u32(pgsize); - cq->cq = mlx5_devx_cmd_create_cq(ctx, &attr); - if (!cq->cq) - goto error; - cq->db_rec = RTE_PTR_ADD(cq->umem_buf, (uintptr_t)attr.db_umem_offset); cq->cq_ci = 0; - memset((void *)(uintptr_t)cq->umem_buf, 0xFF, attr.db_umem_offset); - return 0; -error: - mlx5_aso_cq_destroy(cq); - return -1; + return mlx5_devx_cq_create(ctx, &cq->cq_obj, log_desc_n, &attr, socket); } /** @@ -195,8 +154,7 @@ mlx5_devx_cmd_destroy(sq->sq); sq->sq = NULL; } - if (sq->cq.cq) - mlx5_aso_cq_destroy(&sq->cq); + mlx5_aso_cq_destroy(&sq->cq); mlx5_aso_devx_dereg_mr(&sq->mr); memset(sq, 0, sizeof(*sq)); } @@ -247,8 +205,6 @@ * User Access Region object. * @param[in] pdn * Protection Domain number to use. - * @param[in] eqn - * EQ number. * @param[in] log_desc_n * Log of number of descriptors in queue. * @@ -257,8 +213,7 @@ */ static int mlx5_aso_sq_create(void *ctx, struct mlx5_aso_sq *sq, int socket, - void *uar, uint32_t pdn, - uint32_t eqn, uint16_t log_desc_n) + void *uar, uint32_t pdn, uint16_t log_desc_n) { struct mlx5_devx_create_sq_attr attr = { 0 }; struct mlx5_devx_modify_sq_attr modify_attr = { 0 }; @@ -272,7 +227,7 @@ sq_desc_n, &sq->mr, socket, pdn)) return -1; if (mlx5_aso_cq_create(ctx, &sq->cq, log_desc_n, socket, - mlx5_os_get_devx_uar_page_id(uar), eqn)) + mlx5_os_get_devx_uar_page_id(uar))) goto error; sq->log_desc_n = log_desc_n; sq->umem_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, wq_size + @@ -296,7 +251,7 @@ attr.tis_lst_sz = 0; attr.tis_num = 0; attr.user_index = 0xFFFF; - attr.cqn = sq->cq.cq->id; + attr.cqn = sq->cq.cq_obj.cq->id; wq_attr->uar_page = mlx5_os_get_devx_uar_page_id(uar); wq_attr->pd = pdn; wq_attr->wq_type = MLX5_WQ_TYPE_CYCLIC; @@ -348,8 +303,7 @@ mlx5_aso_queue_init(struct mlx5_dev_ctx_shared *sh) { return mlx5_aso_sq_create(sh->ctx, &sh->aso_age_mng->aso_sq, 0, - sh->tx_uar, sh->pdn, sh->eqn, - MLX5_ASO_QUEUE_LOG_DESC); + sh->tx_uar, sh->pdn, MLX5_ASO_QUEUE_LOG_DESC); } /** @@ -459,7 +413,7 @@ struct mlx5_aso_cq *cq = &sq->cq; uint32_t idx = cq->cq_ci & ((1 << cq->log_desc_n) - 1); volatile struct mlx5_err_cqe *cqe = - (volatile struct mlx5_err_cqe *)&cq->cqes[idx]; + (volatile struct mlx5_err_cqe *)&cq->cq_obj.cqes[idx]; cq->errors++; idx = rte_be_to_cpu_16(cqe->wqe_counter) & (1u << sq->log_desc_n); @@ -572,8 +526,8 @@ do { idx = next_idx; next_idx = (cq->cq_ci + 1) & mask; - rte_prefetch0(&cq->cqes[next_idx]); - cqe = &cq->cqes[idx]; + rte_prefetch0(&cq->cq_obj.cqes[next_idx]); + cqe = &cq->cq_obj.cqes[idx]; ret = check_cqe(cqe, cq_size, cq->cq_ci); /* * Be sure owner read is done before any other cookie field or @@ -593,7 +547,7 @@ mlx5_aso_age_action_update(sh, i); sq->tail += i; rte_io_wmb(); - cq->db_rec[0] = rte_cpu_to_be_32(cq->cq_ci); + cq->cq_obj.db_rec[0] = rte_cpu_to_be_32(cq->cq_ci); } return i; } From patchwork Wed Jan 6 08:19:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86033 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA710A09FF; Wed, 6 Jan 2021 09:21:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 89ED81608E7; Wed, 6 Jan 2021 09:21:15 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 5482B160830 for ; Wed, 6 Jan 2021 09:21:12 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:09 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgc009291; Wed, 6 Jan 2021 10:21:09 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:31 +0000 Message-Id: <1609921181-5019-10-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 09/19] net/mlx5: move Tx CQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for Tx CQ creation. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.h | 6 +- drivers/net/mlx5/mlx5_devx.c | 178 +++++++------------------------------------ 2 files changed, 29 insertions(+), 155 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index f889180..e61154b 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -847,11 +847,7 @@ struct mlx5_txq_obj { }; struct { struct rte_eth_dev *dev; - struct mlx5_devx_obj *cq_devx; - void *cq_umem; - void *cq_buf; - int64_t cq_dbrec_offset; - struct mlx5_devx_dbr_page *cq_dbrec_page; + struct mlx5_devx_cq cq_obj; struct mlx5_devx_obj *sq_devx; void *sq_umem; void *sq_buf; diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 5c5bea6..af0383c 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -15,6 +15,7 @@ #include #include +#include #include #include "mlx5.h" @@ -1141,28 +1142,6 @@ } /** - * Release DevX Tx CQ resources. - * - * @param txq_obj - * DevX Tx queue object. - */ -static void -mlx5_txq_release_devx_cq_resources(struct mlx5_txq_obj *txq_obj) -{ - if (txq_obj->cq_devx) - claim_zero(mlx5_devx_cmd_destroy(txq_obj->cq_devx)); - if (txq_obj->cq_umem) - claim_zero(mlx5_os_umem_dereg(txq_obj->cq_umem)); - if (txq_obj->cq_buf) - mlx5_free(txq_obj->cq_buf); - if (txq_obj->cq_dbrec_page) - claim_zero(mlx5_release_dbr(&txq_obj->txq_ctrl->priv->dbrpgs, - mlx5_os_get_umem_id - (txq_obj->cq_dbrec_page->umem), - txq_obj->cq_dbrec_offset)); -} - -/** * Destroy the Tx queue DevX object. * * @param txq_obj @@ -1172,124 +1151,8 @@ mlx5_txq_release_devx_resources(struct mlx5_txq_obj *txq_obj) { mlx5_txq_release_devx_sq_resources(txq_obj); - mlx5_txq_release_devx_cq_resources(txq_obj); -} - -/** - * Create a DevX CQ object and its resources for an Tx queue. - * - * @param dev - * Pointer to Ethernet device. - * @param idx - * Queue index in DPDK Tx queue array. - * - * @return - * Number of CQEs in CQ, 0 otherwise and rte_errno is set. - */ -static uint32_t -mlx5_txq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx) -{ - struct mlx5_priv *priv = dev->data->dev_private; - struct mlx5_txq_data *txq_data = (*priv->txqs)[idx]; - struct mlx5_txq_ctrl *txq_ctrl = - container_of(txq_data, struct mlx5_txq_ctrl, txq); - struct mlx5_txq_obj *txq_obj = txq_ctrl->obj; - struct mlx5_devx_cq_attr cq_attr = { 0 }; - struct mlx5_cqe *cqe; - size_t page_size; - size_t alignment; - uint32_t cqe_n; - uint32_t i; - int ret; - - MLX5_ASSERT(txq_data); - MLX5_ASSERT(txq_obj); - page_size = rte_mem_page_size(); - if (page_size == (size_t)-1) { - DRV_LOG(ERR, "Failed to get mem page size."); - rte_errno = ENOMEM; - return 0; - } - /* Allocate memory buffer for CQEs. */ - alignment = MLX5_CQE_BUF_ALIGNMENT; - if (alignment == (size_t)-1) { - DRV_LOG(ERR, "Failed to get CQE buf alignment."); - rte_errno = ENOMEM; - return 0; - } - /* Create the Completion Queue. */ - cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH + - 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; - cqe_n = 1UL << log2above(cqe_n); - if (cqe_n > UINT16_MAX) { - DRV_LOG(ERR, - "Port %u Tx queue %u requests to many CQEs %u.", - dev->data->port_id, txq_data->idx, cqe_n); - rte_errno = EINVAL; - return 0; - } - txq_obj->cq_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, - cqe_n * sizeof(struct mlx5_cqe), - alignment, - priv->sh->numa_node); - if (!txq_obj->cq_buf) { - DRV_LOG(ERR, - "Port %u Tx queue %u cannot allocate memory (CQ).", - dev->data->port_id, txq_data->idx); - rte_errno = ENOMEM; - return 0; - } - /* Register allocated buffer in user space with DevX. */ - txq_obj->cq_umem = mlx5_os_umem_reg(priv->sh->ctx, - (void *)txq_obj->cq_buf, - cqe_n * sizeof(struct mlx5_cqe), - IBV_ACCESS_LOCAL_WRITE); - if (!txq_obj->cq_umem) { - rte_errno = errno; - DRV_LOG(ERR, - "Port %u Tx queue %u cannot register memory (CQ).", - dev->data->port_id, txq_data->idx); - goto error; - } - /* Allocate doorbell record for completion queue. */ - txq_obj->cq_dbrec_offset = mlx5_get_dbr(priv->sh->ctx, - &priv->dbrpgs, - &txq_obj->cq_dbrec_page); - if (txq_obj->cq_dbrec_offset < 0) { - rte_errno = errno; - DRV_LOG(ERR, "Failed to allocate CQ door-bell."); - goto error; - } - cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(priv->sh->tx_uar); - cq_attr.eqn = priv->sh->eqn; - cq_attr.q_umem_valid = 1; - cq_attr.q_umem_offset = (uintptr_t)txq_obj->cq_buf % page_size; - cq_attr.q_umem_id = mlx5_os_get_umem_id(txq_obj->cq_umem); - cq_attr.db_umem_valid = 1; - cq_attr.db_umem_offset = txq_obj->cq_dbrec_offset; - cq_attr.db_umem_id = mlx5_os_get_umem_id(txq_obj->cq_dbrec_page->umem); - cq_attr.log_cq_size = rte_log2_u32(cqe_n); - cq_attr.log_page_size = rte_log2_u32(page_size); - /* Create completion queue object with DevX. */ - txq_obj->cq_devx = mlx5_devx_cmd_create_cq(priv->sh->ctx, &cq_attr); - if (!txq_obj->cq_devx) { - rte_errno = errno; - DRV_LOG(ERR, "Port %u Tx queue %u CQ creation failure.", - dev->data->port_id, idx); - goto error; - } - /* Initial fill CQ buffer with invalid CQE opcode. */ - cqe = (struct mlx5_cqe *)txq_obj->cq_buf; - for (i = 0; i < cqe_n; i++) { - cqe->op_own = (MLX5_CQE_INVALID << 4) | MLX5_CQE_OWNER_MASK; - ++cqe; - } - return cqe_n; -error: - ret = rte_errno; - mlx5_txq_release_devx_cq_resources(txq_obj); - rte_errno = ret; - return 0; + mlx5_devx_cq_destroy(&txq_obj->cq_obj); + memset(&txq_obj->cq_obj, 0, sizeof(txq_obj->cq_obj)); } /** @@ -1361,7 +1224,7 @@ sq_attr.tis_lst_sz = 1; sq_attr.tis_num = priv->sh->tis->id; sq_attr.state = MLX5_SQC_STATE_RST; - sq_attr.cqn = txq_obj->cq_devx->id; + sq_attr.cqn = txq_obj->cq_obj.cq->id; sq_attr.flush_in_error_en = 1; sq_attr.allow_multi_pkt_send_wqe = !!priv->config.mps; sq_attr.allow_swp = !!priv->config.swp; @@ -1425,8 +1288,11 @@ #else struct mlx5_dev_ctx_shared *sh = priv->sh; struct mlx5_txq_obj *txq_obj = txq_ctrl->obj; + struct mlx5_devx_cq_attr cq_attr = { + .uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar), + }; void *reg_addr; - uint32_t cqe_n; + uint32_t cqe_n, log_desc_n; uint32_t wqe_n; int ret = 0; @@ -1434,19 +1300,31 @@ MLX5_ASSERT(txq_obj); txq_obj->txq_ctrl = txq_ctrl; txq_obj->dev = dev; - cqe_n = mlx5_txq_create_devx_cq_resources(dev, idx); - if (!cqe_n) { - rte_errno = errno; + cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH + + 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; + log_desc_n = log2above(cqe_n); + cqe_n = 1UL << log_desc_n; + if (cqe_n > UINT16_MAX) { + DRV_LOG(ERR, "Port %u Tx queue %u requests to many CQEs %u.", + dev->data->port_id, txq_data->idx, cqe_n); + rte_errno = EINVAL; + return 0; + } + /* Create completion queue object with DevX. */ + ret = mlx5_devx_cq_create(sh->ctx, &txq_obj->cq_obj, log_desc_n, + &cq_attr, priv->sh->numa_node); + if (ret) { + DRV_LOG(ERR, "Port %u Tx queue %u CQ creation failure.", + dev->data->port_id, idx); goto error; } - txq_data->cqe_n = log2above(cqe_n); - txq_data->cqe_s = 1 << txq_data->cqe_n; + txq_data->cqe_n = log_desc_n; + txq_data->cqe_s = cqe_n; txq_data->cqe_m = txq_data->cqe_s - 1; - txq_data->cqes = (volatile struct mlx5_cqe *)txq_obj->cq_buf; + txq_data->cqes = txq_obj->cq_obj.cqes; txq_data->cq_ci = 0; txq_data->cq_pi = 0; - txq_data->cq_db = (volatile uint32_t *)(txq_obj->cq_dbrec_page->dbrs + - txq_obj->cq_dbrec_offset); + txq_data->cq_db = txq_obj->cq_obj.db_rec; *txq_data->cq_db = 0; /* Create Send Queue object with DevX. */ wqe_n = mlx5_txq_create_devx_sq_resources(dev, idx); From patchwork Wed Jan 6 08:19:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86034 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 410F4A09FF; Wed, 6 Jan 2021 09:21:45 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0F1DD16089F; Wed, 6 Jan 2021 09:21:18 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 097411608F5 for ; Wed, 6 Jan 2021 09:21:16 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:14 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgd009291; Wed, 6 Jan 2021 10:21:14 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:32 +0000 Message-Id: <1609921181-5019-11-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 10/19] net/mlx5: move Rx CQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for Rx CQ creation. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.c | 8 --- drivers/net/mlx5/mlx5.h | 3 +- drivers/net/mlx5/mlx5_devx.c | 142 +++++++++++++------------------------------ drivers/net/mlx5/mlx5_rxtx.h | 4 -- 4 files changed, 42 insertions(+), 115 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 91492c5..c0a36d6 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -936,14 +936,6 @@ struct mlx5_dev_ctx_shared * goto error; } if (sh->devx) { - /* Query the EQN for this core. */ - err = mlx5_glue->devx_query_eqn(sh->ctx, 0, &sh->eqn); - if (err) { - rte_errno = errno; - DRV_LOG(ERR, "Failed to query event queue number %d.", - rte_errno); - goto error; - } err = mlx5_os_get_pdn(sh->pd, &sh->pdn); if (err) { DRV_LOG(ERR, "Fail to extract pdn from PD"); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index e61154b..079cbca 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -683,7 +683,6 @@ struct mlx5_dev_ctx_shared { uint16_t bond_dev; /* Bond primary device id. */ uint32_t devx:1; /* Opened with DV. */ uint32_t flow_hit_aso_en:1; /* Flow Hit ASO is supported. */ - uint32_t eqn; /* Event Queue number. */ uint32_t max_port; /* Maximal IB device port index. */ void *ctx; /* Verbs/DV/DevX context. */ void *pd; /* Protection Domain. */ @@ -791,7 +790,7 @@ struct mlx5_rxq_obj { }; struct { struct mlx5_devx_obj *rq; /* DevX Rx Queue object. */ - struct mlx5_devx_obj *devx_cq; /* DevX CQ object. */ + struct mlx5_devx_cq cq_obj; /* DevX CQ object. */ void *devx_channel; }; }; diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index af0383c..913f169 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -172,30 +172,17 @@ } /** - * Release the resources allocated for the Rx CQ DevX object. + * Destroy the Rx queue DevX object. * - * @param rxq_ctrl - * DevX Rx queue object. + * @param rxq_obj + * Rxq object to destroy. */ static void -mlx5_rxq_release_devx_cq_resources(struct mlx5_rxq_ctrl *rxq_ctrl) +mlx5_rxq_release_devx_resources(struct mlx5_rxq_ctrl *rxq_ctrl) { - struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->cq_dbrec_page; - - if (rxq_ctrl->cq_umem) { - mlx5_os_umem_dereg(rxq_ctrl->cq_umem); - rxq_ctrl->cq_umem = NULL; - } - if (rxq_ctrl->rxq.cqes) { - rte_free((void *)(uintptr_t)rxq_ctrl->rxq.cqes); - rxq_ctrl->rxq.cqes = NULL; - } - if (dbr_page) { - claim_zero(mlx5_release_dbr(&rxq_ctrl->priv->dbrpgs, - mlx5_os_get_umem_id(dbr_page->umem), - rxq_ctrl->cq_dbr_offset)); - rxq_ctrl->cq_dbrec_page = NULL; - } + mlx5_rxq_release_devx_rq_resources(rxq_ctrl); + mlx5_devx_cq_destroy(&rxq_ctrl->obj->cq_obj); + memset(&rxq_ctrl->obj->cq_obj, 0, sizeof(rxq_ctrl->obj->cq_obj)); } /** @@ -213,14 +200,12 @@ mlx5_devx_modify_rq(rxq_obj, MLX5_RXQ_MOD_RDY2RST); claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq)); } else { - MLX5_ASSERT(rxq_obj->devx_cq); + MLX5_ASSERT(rxq_obj->cq_obj); claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq)); - claim_zero(mlx5_devx_cmd_destroy(rxq_obj->devx_cq)); if (rxq_obj->devx_channel) mlx5_os_devx_destroy_event_channel (rxq_obj->devx_channel); - mlx5_rxq_release_devx_rq_resources(rxq_obj->rxq_ctrl); - mlx5_rxq_release_devx_cq_resources(rxq_obj->rxq_ctrl); + mlx5_rxq_release_devx_resources(rxq_obj->rxq_ctrl); } } @@ -249,7 +234,7 @@ rte_errno = errno; return -rte_errno; } - if (out.event_resp.cookie != (uint64_t)(uintptr_t)rxq_obj->devx_cq) { + if (out.event_resp.cookie != (uint64_t)(uintptr_t)rxq_obj->cq_obj.cq) { rte_errno = EINVAL; return -rte_errno; } @@ -327,7 +312,7 @@ container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); struct mlx5_devx_create_rq_attr rq_attr = { 0 }; uint32_t wqe_n = 1 << (rxq_data->elts_n - rxq_data->sges_n); - uint32_t cqn = rxq_ctrl->obj->devx_cq->id; + uint32_t cqn = rxq_ctrl->obj->cq_obj.cq->id; struct mlx5_devx_dbr_page *dbr_page; int64_t dbr_offset; uint32_t wq_size = 0; @@ -410,31 +395,23 @@ * Queue index in DPDK Rx queue array. * * @return - * The DevX CQ object initialized, NULL otherwise and rte_errno is set. + * 0 on success, a negative errno value otherwise and rte_errno is set. */ -static struct mlx5_devx_obj * +static int mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx) { - struct mlx5_devx_obj *cq_obj = 0; + struct mlx5_devx_cq *cq_obj = 0; struct mlx5_devx_cq_attr cq_attr = { 0 }; struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_ctx_shared *sh = priv->sh; struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[idx]; struct mlx5_rxq_ctrl *rxq_ctrl = container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); - size_t page_size = rte_mem_page_size(); unsigned int cqe_n = mlx5_rxq_cqe_num(rxq_data); - struct mlx5_devx_dbr_page *dbr_page; - int64_t dbr_offset; - void *buf = NULL; - uint16_t event_nums[1] = {0}; uint32_t log_cqe_n; - uint32_t cq_size; + uint16_t event_nums[1] = { 0 }; int ret = 0; - if (page_size == (size_t)-1) { - DRV_LOG(ERR, "Failed to get page_size."); - goto error; - } if (priv->config.cqe_comp && !rxq_data->hw_timestamp && !rxq_data->lro) { cq_attr.cqe_comp_en = 1u; @@ -487,71 +464,37 @@ "Port %u Rx CQE compression is disabled for LRO.", dev->data->port_id); } + cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(sh->devx_rx_uar); log_cqe_n = log2above(cqe_n); - cq_size = sizeof(struct mlx5_cqe) * (1 << log_cqe_n); - buf = rte_calloc_socket(__func__, 1, cq_size, page_size, - rxq_ctrl->socket); - if (!buf) { - DRV_LOG(ERR, "Failed to allocate memory for CQ."); - goto error; - } - rxq_data->cqes = (volatile struct mlx5_cqe (*)[])(uintptr_t)buf; - rxq_ctrl->cq_umem = mlx5_os_umem_reg(priv->sh->ctx, buf, - cq_size, - IBV_ACCESS_LOCAL_WRITE); - if (!rxq_ctrl->cq_umem) { - DRV_LOG(ERR, "Failed to register umem for CQ."); - goto error; - } - /* Allocate CQ door-bell. */ - dbr_offset = mlx5_get_dbr(priv->sh->ctx, &priv->dbrpgs, &dbr_page); - if (dbr_offset < 0) { - DRV_LOG(ERR, "Failed to allocate CQ door-bell."); - goto error; - } - rxq_ctrl->cq_dbr_offset = dbr_offset; - rxq_ctrl->cq_dbrec_page = dbr_page; - rxq_data->cq_db = (uint32_t *)((uintptr_t)dbr_page->dbrs + - (uintptr_t)rxq_ctrl->cq_dbr_offset); - rxq_data->cq_uar = - mlx5_os_get_devx_uar_base_addr(priv->sh->devx_rx_uar); /* Create CQ using DevX API. */ - cq_attr.eqn = priv->sh->eqn; - cq_attr.uar_page_id = - mlx5_os_get_devx_uar_page_id(priv->sh->devx_rx_uar); - cq_attr.q_umem_id = mlx5_os_get_umem_id(rxq_ctrl->cq_umem); - cq_attr.q_umem_valid = 1; - cq_attr.log_cq_size = log_cqe_n; - cq_attr.log_page_size = rte_log2_u32(page_size); - cq_attr.db_umem_offset = rxq_ctrl->cq_dbr_offset; - cq_attr.db_umem_id = mlx5_os_get_umem_id(dbr_page->umem); - cq_attr.db_umem_valid = 1; - cq_obj = mlx5_devx_cmd_create_cq(priv->sh->ctx, &cq_attr); - if (!cq_obj) - goto error; + ret = mlx5_devx_cq_create(sh->ctx, &rxq_ctrl->obj->cq_obj, log_cqe_n, + &cq_attr, sh->numa_node); + if (ret) + return ret; + cq_obj = &rxq_ctrl->obj->cq_obj; + rxq_data->cqes = (volatile struct mlx5_cqe (*)[]) + (uintptr_t)cq_obj->cqes; + rxq_data->cq_db = cq_obj->db_rec; + rxq_data->cq_uar = mlx5_os_get_devx_uar_base_addr(sh->devx_rx_uar); rxq_data->cqe_n = log_cqe_n; - rxq_data->cqn = cq_obj->id; + rxq_data->cqn = cq_obj->cq->id; if (rxq_ctrl->obj->devx_channel) { ret = mlx5_os_devx_subscribe_devx_event - (rxq_ctrl->obj->devx_channel, - cq_obj->obj, - sizeof(event_nums), - event_nums, - (uint64_t)(uintptr_t)cq_obj); + (rxq_ctrl->obj->devx_channel, + cq_obj->cq->obj, + sizeof(event_nums), + event_nums, + (uint64_t)(uintptr_t)cq_obj->cq); if (ret) { DRV_LOG(ERR, "Fail to subscribe CQ to event channel."); - rte_errno = errno; - goto error; + ret = errno; + mlx5_devx_cq_destroy(cq_obj); + memset(cq_obj, 0, sizeof(*cq_obj)); + rte_errno = ret; + return -ret; } } - /* Initialise CQ to 1's to mark HW ownership for all CQEs. */ - memset((void *)(uintptr_t)rxq_data->cqes, 0xFF, cq_size); - return cq_obj; -error: - if (cq_obj) - mlx5_devx_cmd_destroy(cq_obj); - mlx5_rxq_release_devx_cq_resources(rxq_ctrl); - return NULL; + return 0; } /** @@ -655,8 +598,8 @@ tmpl->fd = mlx5_os_get_devx_channel_fd(tmpl->devx_channel); } /* Create CQ using DevX API. */ - tmpl->devx_cq = mlx5_rxq_create_devx_cq_resources(dev, idx); - if (!tmpl->devx_cq) { + ret = mlx5_rxq_create_devx_cq_resources(dev, idx); + if (ret) { DRV_LOG(ERR, "Failed to create CQ."); goto error; } @@ -682,12 +625,9 @@ ret = rte_errno; /* Save rte_errno before cleanup. */ if (tmpl->rq) claim_zero(mlx5_devx_cmd_destroy(tmpl->rq)); - if (tmpl->devx_cq) - claim_zero(mlx5_devx_cmd_destroy(tmpl->devx_cq)); if (tmpl->devx_channel) mlx5_os_devx_destroy_event_channel(tmpl->devx_channel); - mlx5_rxq_release_devx_rq_resources(rxq_ctrl); - mlx5_rxq_release_devx_cq_resources(rxq_ctrl); + mlx5_rxq_release_devx_resources(rxq_ctrl); rte_errno = ret; /* Restore rte_errno. */ return -rte_errno; } diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 1e9345a..aba9541 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -196,11 +196,7 @@ struct mlx5_rxq_ctrl { struct mlx5_devx_dbr_page *rq_dbrec_page; uint64_t rq_dbr_offset; /* Storing RQ door-bell information, needed when freeing door-bell. */ - struct mlx5_devx_dbr_page *cq_dbrec_page; - uint64_t cq_dbr_offset; - /* Storing CQ door-bell information, needed when freeing door-bell. */ void *wq_umem; /* WQ buffer registration info. */ - void *cq_umem; /* CQ buffer registration info. */ struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. */ uint32_t hairpin_status; /* Hairpin binding status. */ }; From patchwork Wed Jan 6 08:19:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86036 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2A94BA09FF; Wed, 6 Jan 2021 09:22:02 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7D5E41606D8; Wed, 6 Jan 2021 09:21:24 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 154DF1608C4 for ; Wed, 6 Jan 2021 09:21:22 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:17 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAge009291; Wed, 6 Jan 2021 10:21:17 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:33 +0000 Message-Id: <1609921181-5019-12-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 11/19] common/mlx5: enhance page size configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The PRM calculates page size in 4K, so need to reduce the log_wq_pg_sz attribute. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_devx_cmds.c | 53 ++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 30 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 59f0bcc..790a701 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -268,9 +268,8 @@ struct mlx5_devx_obj * MLX5_SET(mkc, mkc, mkey_7_0, attr->umem_id & 0xFF); MLX5_SET(mkc, mkc, translations_octword_size, translation_size); MLX5_SET(mkc, mkc, relaxed_ordering_write, - attr->relaxed_ordering_write); - MLX5_SET(mkc, mkc, relaxed_ordering_read, - attr->relaxed_ordering_read); + attr->relaxed_ordering_write); + MLX5_SET(mkc, mkc, relaxed_ordering_read, attr->relaxed_ordering_read); MLX5_SET64(mkc, mkc, start_addr, attr->addr); MLX5_SET64(mkc, mkc, len, attr->size); mkey->obj = mlx5_glue->devx_obj_create(ctx, in, in_size_dw * 4, out, @@ -308,7 +307,7 @@ struct mlx5_devx_obj * if (status) { int syndrome = MLX5_GET(query_flow_counter_out, out, syndrome); - DRV_LOG(ERR, "Bad devX status %x, syndrome = %x", status, + DRV_LOG(ERR, "Bad DevX status %x, syndrome = %x", status, syndrome); } return status; @@ -374,8 +373,7 @@ struct mlx5_devx_obj * syndrome = MLX5_GET(query_nic_vport_context_out, out, syndrome); if (status) { DRV_LOG(DEBUG, "Failed to query NIC vport context, " - "status %x, syndrome = %x", - status, syndrome); + "status %x, syndrome = %x", status, syndrome); return -1; } vctx = MLX5_ADDR_OF(query_nic_vport_context_out, out, @@ -662,8 +660,7 @@ struct mlx5_devx_obj * syndrome = MLX5_GET(query_hca_cap_out, out, syndrome); if (status) { DRV_LOG(DEBUG, "Failed to query devx HCA capabilities, " - "status %x, syndrome = %x", - status, syndrome); + "status %x, syndrome = %x", status, syndrome); return -1; } hcattr = MLX5_ADDR_OF(query_hca_cap_out, out, capability); @@ -683,11 +680,11 @@ struct mlx5_devx_obj * (cmd_hca_cap, hcattr, log_min_hairpin_wq_data_sz); attr->vhca_id = MLX5_GET(cmd_hca_cap, hcattr, vhca_id); attr->relaxed_ordering_write = MLX5_GET(cmd_hca_cap, hcattr, - relaxed_ordering_write); + relaxed_ordering_write); attr->relaxed_ordering_read = MLX5_GET(cmd_hca_cap, hcattr, - relaxed_ordering_read); + relaxed_ordering_read); attr->access_register_user = MLX5_GET(cmd_hca_cap, hcattr, - access_register_user); + access_register_user); attr->eth_net_offloads = MLX5_GET(cmd_hca_cap, hcattr, eth_net_offloads); attr->eth_virt = MLX5_GET(cmd_hca_cap, hcattr, eth_virt); @@ -738,8 +735,7 @@ struct mlx5_devx_obj * goto error; if (status) { DRV_LOG(DEBUG, "Failed to query devx QOS capabilities," - " status %x, syndrome = %x", - status, syndrome); + " status %x, syndrome = %x", status, syndrome); return -1; } hcattr = MLX5_ADDR_OF(query_hca_cap_out, out, capability); @@ -769,17 +765,14 @@ struct mlx5_devx_obj * MLX5_GET_HCA_CAP_OP_MOD_NIC_FLOW_TABLE | MLX5_HCA_CAP_OPMOD_GET_CUR); - rc = mlx5_glue->devx_general_cmd(ctx, - in, sizeof(in), - out, sizeof(out)); + rc = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, sizeof(out)); if (rc) goto error; status = MLX5_GET(query_hca_cap_out, out, status); syndrome = MLX5_GET(query_hca_cap_out, out, syndrome); if (status) { DRV_LOG(DEBUG, "Failed to query devx HCA capabilities, " - "status %x, syndrome = %x", - status, syndrome); + "status %x, syndrome = %x", status, syndrome); attr->log_max_ft_sampler_num = 0; return -1; } @@ -796,9 +789,7 @@ struct mlx5_devx_obj * MLX5_GET_HCA_CAP_OP_MOD_ETHERNET_OFFLOAD_CAPS | MLX5_HCA_CAP_OPMOD_GET_CUR); - rc = mlx5_glue->devx_general_cmd(ctx, - in, sizeof(in), - out, sizeof(out)); + rc = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, sizeof(out)); if (rc) { attr->eth_net_offloads = 0; goto error; @@ -807,8 +798,7 @@ struct mlx5_devx_obj * syndrome = MLX5_GET(query_hca_cap_out, out, syndrome); if (status) { DRV_LOG(DEBUG, "Failed to query devx HCA capabilities, " - "status %x, syndrome = %x", - status, syndrome); + "status %x, syndrome = %x", status, syndrome); attr->eth_net_offloads = 0; return -1; } @@ -927,7 +917,9 @@ struct mlx5_devx_obj * MLX5_SET(wq, wq_ctx, hw_counter, wq_attr->hw_counter); MLX5_SET(wq, wq_ctx, sw_counter, wq_attr->sw_counter); MLX5_SET(wq, wq_ctx, log_wq_stride, wq_attr->log_wq_stride); - MLX5_SET(wq, wq_ctx, log_wq_pg_sz, wq_attr->log_wq_pg_sz); + if (wq_attr->log_wq_pg_sz > MLX5_ADAPTER_PAGE_SHIFT) + MLX5_SET(wq, wq_ctx, log_wq_pg_sz, + wq_attr->log_wq_pg_sz - MLX5_ADAPTER_PAGE_SHIFT); MLX5_SET(wq, wq_ctx, log_wq_sz, wq_attr->log_wq_sz); MLX5_SET(wq, wq_ctx, dbr_umem_valid, wq_attr->dbr_umem_valid); MLX5_SET(wq, wq_ctx, wq_umem_valid, wq_attr->wq_umem_valid); @@ -1574,13 +1566,13 @@ struct mlx5_devx_obj * MLX5_SET(cqc, cqctx, cc, attr->use_first_only); MLX5_SET(cqc, cqctx, oi, attr->overrun_ignore); MLX5_SET(cqc, cqctx, log_cq_size, attr->log_cq_size); - MLX5_SET(cqc, cqctx, log_page_size, attr->log_page_size - - MLX5_ADAPTER_PAGE_SHIFT); + if (attr->log_page_size > MLX5_ADAPTER_PAGE_SHIFT) + MLX5_SET(cqc, cqctx, log_page_size, + attr->log_page_size - MLX5_ADAPTER_PAGE_SHIFT); MLX5_SET(cqc, cqctx, c_eqn, attr->eqn); MLX5_SET(cqc, cqctx, uar_page, attr->uar_page_id); MLX5_SET(cqc, cqctx, cqe_comp_en, !!attr->cqe_comp_en); - MLX5_SET(cqc, cqctx, mini_cqe_res_format, - attr->mini_cqe_res_format); + MLX5_SET(cqc, cqctx, mini_cqe_res_format, attr->mini_cqe_res_format); MLX5_SET(cqc, cqctx, mini_cqe_res_format_ext, attr->mini_cqe_res_format_ext); if (attr->q_umem_valid) { @@ -1809,8 +1801,9 @@ struct mlx5_devx_obj * if (attr->uar_index) { MLX5_SET(qpc, qpc, pm_state, MLX5_QP_PM_MIGRATED); MLX5_SET(qpc, qpc, uar_page, attr->uar_index); - MLX5_SET(qpc, qpc, log_page_size, attr->log_page_size - - MLX5_ADAPTER_PAGE_SHIFT); + if (attr->log_page_size > MLX5_ADAPTER_PAGE_SHIFT) + MLX5_SET(qpc, qpc, log_page_size, + attr->log_page_size - MLX5_ADAPTER_PAGE_SHIFT); if (attr->sq_size) { MLX5_ASSERT(RTE_IS_POWER_OF_2(attr->sq_size)); MLX5_SET(qpc, qpc, cqn_snd, attr->cqn); From patchwork Wed Jan 6 08:19:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86035 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3D04A09FF; Wed, 6 Jan 2021 09:21:53 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 408E91608C4; Wed, 6 Jan 2021 09:21:23 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 1606A1608E4 for ; Wed, 6 Jan 2021 09:21:22 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:20 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgf009291; Wed, 6 Jan 2021 10:21:20 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:34 +0000 Message-Id: <1609921181-5019-13-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 12/19] common/mlx5: share DevX SQ creation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The SQ object in DevX is created in several places and in several different drivers. In all places almost all the details are the same, and in particular the allocations of the required resources. Add a structure that contains all the resources, and provide creation and release functions for it. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common_devx.c | 119 ++++++++++++++++++++++++ drivers/common/mlx5/mlx5_common_devx.h | 27 +++++- drivers/common/mlx5/rte_common_mlx5_exports.def | 2 + drivers/common/mlx5/version.map | 2 + 4 files changed, 148 insertions(+), 2 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_devx.c b/drivers/common/mlx5/mlx5_common_devx.c index 3ec0dd5..58b75c3 100644 --- a/drivers/common/mlx5/mlx5_common_devx.c +++ b/drivers/common/mlx5/mlx5_common_devx.c @@ -153,3 +153,122 @@ rte_errno = ret; return -rte_errno; } + +/** + * Destroy DevX Send Queue. + * + * @param[in] sq + * DevX SQ to destroy. + */ +void +mlx5_devx_sq_destroy(struct mlx5_devx_sq *sq) +{ + if (sq->sq) + claim_zero(mlx5_devx_cmd_destroy(sq->sq)); + if (sq->umem_obj) + claim_zero(mlx5_os_umem_dereg(sq->umem_obj)); + if (sq->umem_buf) + mlx5_free((void *)(uintptr_t)sq->umem_buf); +} + +/** + * Create Send Queue using DevX API. + * + * Get a pointer to partially initialized attributes structure, and updates the + * following fields: + * wq_type + * wq_umem_valid + * wq_umem_id + * wq_umem_offset + * dbr_umem_valid + * dbr_umem_id + * dbr_addr + * log_wq_stride + * log_wq_sz + * log_wq_pg_sz + * All other fields are updated by caller. + * + * @param[in] ctx + * Context returned from mlx5 open_device() glue function. + * @param[in/out] sq_obj + * Pointer to SQ to create. + * @param[in] log_wqbb_n + * Log of number of WQBBs in queue. + * @param[in] attr + * Pointer to SQ attributes structure. + * @param[in] socket + * Socket to use for allocation. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_sq_create(void *ctx, struct mlx5_devx_sq *sq_obj, uint16_t log_wqbb_n, + struct mlx5_devx_create_sq_attr *attr, int socket) +{ + struct mlx5_devx_obj *sq = NULL; + struct mlx5dv_devx_umem *umem_obj = NULL; + void *umem_buf = NULL; + size_t alignment = MLX5_WQE_BUF_ALIGNMENT; + uint32_t umem_size, umem_dbrec; + uint16_t sq_size = 1 << log_wqbb_n; + int ret; + + if (alignment == (size_t)-1) { + DRV_LOG(ERR, "Failed to get WQE buf alignment."); + rte_errno = ENOMEM; + return -rte_errno; + } + /* Allocate memory buffer for WQEs and doorbell record. */ + umem_size = MLX5_WQE_SIZE * sq_size; + umem_dbrec = RTE_ALIGN(umem_size, MLX5_DBR_SIZE); + umem_size += MLX5_DBR_SIZE; + umem_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size, + alignment, socket); + if (!umem_buf) { + DRV_LOG(ERR, "Failed to allocate memory for SQ."); + rte_errno = ENOMEM; + return -rte_errno; + } + /* Register allocated buffer in user space with DevX. */ + umem_obj = mlx5_os_umem_reg(ctx, (void *)(uintptr_t)umem_buf, umem_size, + IBV_ACCESS_LOCAL_WRITE); + if (!umem_obj) { + DRV_LOG(ERR, "Failed to register umem for SQ."); + rte_errno = errno; + goto error; + } + /* Fill attributes for SQ object creation. */ + attr->wq_attr.wq_type = MLX5_WQ_TYPE_CYCLIC; + attr->wq_attr.wq_umem_valid = 1; + attr->wq_attr.wq_umem_id = mlx5_os_get_umem_id(umem_obj); + attr->wq_attr.wq_umem_offset = 0; + attr->wq_attr.dbr_umem_valid = 1; + attr->wq_attr.dbr_umem_id = attr->wq_attr.wq_umem_id; + attr->wq_attr.dbr_addr = umem_dbrec; + attr->wq_attr.log_wq_stride = rte_log2_u32(MLX5_WQE_SIZE); + attr->wq_attr.log_wq_sz = log_wqbb_n; + attr->wq_attr.log_wq_pg_sz = MLX5_LOG_PAGE_SIZE; + /* Create send queue object with DevX. */ + sq = mlx5_devx_cmd_create_sq(ctx, attr); + if (!sq) { + DRV_LOG(ERR, "Can't create DevX SQ object."); + rte_errno = ENOMEM; + goto error; + } + sq_obj->umem_buf = umem_buf; + sq_obj->umem_obj = umem_obj; + sq_obj->sq = sq; + sq_obj->db_rec = RTE_PTR_ADD(sq_obj->umem_buf, umem_dbrec); + return 0; +error: + ret = rte_errno; + if (umem_obj) + claim_zero(mlx5_os_umem_dereg(umem_obj)); + if (umem_buf) + mlx5_free((void *)(uintptr_t)umem_buf); + rte_errno = ret; + return -rte_errno; +} + + diff --git a/drivers/common/mlx5/mlx5_common_devx.h b/drivers/common/mlx5/mlx5_common_devx.h index 20d5da0..6b078b2 100644 --- a/drivers/common/mlx5/mlx5_common_devx.h +++ b/drivers/common/mlx5/mlx5_common_devx.h @@ -7,6 +7,9 @@ #include "mlx5_devx_cmds.h" +/* The standard page size */ +#define MLX5_LOG_PAGE_SIZE 12 + /* DevX Completion Queue structure. */ struct mlx5_devx_cq { struct mlx5_devx_obj *cq; /* The CQ DevX object. */ @@ -18,6 +21,18 @@ struct mlx5_devx_cq { volatile uint32_t *db_rec; /* The CQ doorbell record. */ }; +/* DevX Send Queue structure. */ +struct mlx5_devx_sq { + struct mlx5_devx_obj *sq; /* The SQ DevX object. */ + void *umem_obj; /* The SQ umem object. */ + union { + volatile void *umem_buf; + volatile struct mlx5_wqe *wqes; /* The SQ ring buffer. */ + }; + volatile uint32_t *db_rec; /* The SQ doorbell record. */ +}; + + /* mlx5_common_devx.c */ __rte_internal @@ -25,7 +40,15 @@ struct mlx5_devx_cq { __rte_internal int mlx5_devx_cq_create(void *ctx, struct mlx5_devx_cq *cq_obj, - uint16_t log_desc_n, struct mlx5_devx_cq_attr *attr, - int socket); + uint16_t log_desc_n, + struct mlx5_devx_cq_attr *attr, int socket); + +__rte_internal +void mlx5_devx_sq_destroy(struct mlx5_devx_sq *sq); + +__rte_internal +int mlx5_devx_sq_create(void *ctx, struct mlx5_devx_sq *sq_obj, + uint16_t log_wqbb_n, + struct mlx5_devx_create_sq_attr *attr, int socket); #endif /* RTE_PMD_MLX5_COMMON_DEVX_H_ */ diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def index d10db40..cfee96e 100644 --- a/drivers/common/mlx5/rte_common_mlx5_exports.def +++ b/drivers/common/mlx5/rte_common_mlx5_exports.def @@ -37,6 +37,8 @@ EXPORTS mlx5_devx_cq_create mlx5_devx_cq_destroy + mlx5_devx_sq_create + mlx5_devx_sq_destroy mlx5_get_dbr mlx5_glue diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 850202c..3414588 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -45,6 +45,8 @@ INTERNAL { mlx5_devx_cq_create; mlx5_devx_cq_destroy; + mlx5_devx_sq_create; + mlx5_devx_sq_destroy; mlx5_get_ifname_sysfs; mlx5_get_dbr; From patchwork Wed Jan 6 08:19:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86038 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 84FF8A09FF; Wed, 6 Jan 2021 09:22:18 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EA16B160906; Wed, 6 Jan 2021 09:21:29 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 1487C1608FF for ; Wed, 6 Jan 2021 09:21:27 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:22 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgg009291; Wed, 6 Jan 2021 10:21:22 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:35 +0000 Message-Id: <1609921181-5019-14-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 13/19] regex/mlx5: move DevX SQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for DevX SQ creation. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/regex/mlx5/mlx5_regex.h | 8 +- drivers/regex/mlx5/mlx5_regex_control.c | 153 ++++++++++--------------------- drivers/regex/mlx5/mlx5_regex_fastpath.c | 14 +-- 3 files changed, 55 insertions(+), 120 deletions(-) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 9f7a388..7e1b2a9 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -18,15 +18,10 @@ struct mlx5_regex_sq { uint16_t log_nb_desc; /* Log 2 number of desc for this object. */ - struct mlx5_devx_obj *obj; /* The SQ DevX object. */ - int64_t dbr_offset; /* Door bell record offset. */ - uint32_t dbr_umem; /* Door bell record umem id. */ - uint8_t *wqe; /* The SQ ring buffer. */ - struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ + struct mlx5_devx_sq sq_obj; /* The SQ DevX object. */ size_t pi, db_pi; size_t ci; uint32_t sqn; - uint32_t *dbr; }; struct mlx5_regex_cq { @@ -73,7 +68,6 @@ struct mlx5_regex_priv { uint32_t nb_engines; /* Number of RegEx engines. */ struct mlx5dv_devx_uar *uar; /* UAR object. */ struct ibv_pd *pd; - struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ struct mlx5_mr_share_cache mr_scache; /* Global shared MR cache. */ }; diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index ca6c0f5..df57fad 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -112,6 +112,27 @@ #endif /** + * Destroy the SQ object. + * + * @param qp + * Pointer to the QP element + * @param q_ind + * The index of the queue. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +regex_ctrl_destroy_sq(struct mlx5_regex_qp *qp, uint16_t q_ind) +{ + struct mlx5_regex_sq *sq = &qp->sqs[q_ind]; + + mlx5_devx_sq_destroy(&sq->sq_obj); + memset(sq, 0, sizeof(*sq)); + return 0; +} + +/** * create the SQ object. * * @param priv @@ -131,84 +152,42 @@ uint16_t q_ind, uint16_t log_nb_desc) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT - struct mlx5_devx_create_sq_attr attr = { 0 }; - struct mlx5_devx_modify_sq_attr modify_attr = { 0 }; - struct mlx5_devx_wq_attr *wq_attr = &attr.wq_attr; - struct mlx5_devx_dbr_page *dbr_page = NULL; + struct mlx5_devx_create_sq_attr attr = { + .user_index = q_ind, + .cqn = qp->cq.cq_obj.cq->id, + .wq_attr = (struct mlx5_devx_wq_attr){ + .uar_page = priv->uar->page_id, + }, + }; + struct mlx5_devx_modify_sq_attr modify_attr = { + .state = MLX5_SQC_STATE_RDY, + }; struct mlx5_regex_sq *sq = &qp->sqs[q_ind]; - void *buf = NULL; - uint32_t sq_size; uint32_t pd_num = 0; int ret; sq->log_nb_desc = log_nb_desc; - sq_size = 1 << sq->log_nb_desc; - sq->dbr_offset = mlx5_get_dbr(priv->ctx, &priv->dbrpgs, &dbr_page); - if (sq->dbr_offset < 0) { - DRV_LOG(ERR, "Can't allocate sq door bell record."); - rte_errno = ENOMEM; - goto error; - } - sq->dbr_umem = mlx5_os_get_umem_id(dbr_page->umem); - sq->dbr = (uint32_t *)((uintptr_t)dbr_page->dbrs + - (uintptr_t)sq->dbr_offset); - - buf = rte_calloc(NULL, 1, 64 * sq_size, 4096); - if (!buf) { - DRV_LOG(ERR, "Can't allocate wqe buffer."); - rte_errno = ENOMEM; - goto error; - } - sq->wqe = buf; - sq->wqe_umem = mlx5_glue->devx_umem_reg(priv->ctx, buf, 64 * sq_size, - 7); sq->ci = 0; sq->pi = 0; - if (!sq->wqe_umem) { - DRV_LOG(ERR, "Can't register wqe mem."); - rte_errno = ENOMEM; - goto error; - } - attr.state = MLX5_SQC_STATE_RST; - attr.tis_lst_sz = 0; - attr.tis_num = 0; - attr.user_index = q_ind; - attr.cqn = qp->cq.cq_obj.cq->id; - wq_attr->uar_page = priv->uar->page_id; - regex_get_pdn(priv->pd, &pd_num); - wq_attr->pd = pd_num; - wq_attr->wq_type = MLX5_WQ_TYPE_CYCLIC; - wq_attr->dbr_umem_id = sq->dbr_umem; - wq_attr->dbr_addr = sq->dbr_offset; - wq_attr->dbr_umem_valid = 1; - wq_attr->wq_umem_id = mlx5_os_get_umem_id(sq->wqe_umem); - wq_attr->wq_umem_offset = 0; - wq_attr->wq_umem_valid = 1; - wq_attr->log_wq_stride = 6; - wq_attr->log_wq_sz = sq->log_nb_desc; - sq->obj = mlx5_devx_cmd_create_sq(priv->ctx, &attr); - if (!sq->obj) { - DRV_LOG(ERR, "Can't create sq object."); - rte_errno = ENOMEM; - goto error; + ret = regex_get_pdn(priv->pd, &pd_num); + if (ret) + return ret; + attr.wq_attr.pd = pd_num; + ret = mlx5_devx_sq_create(priv->ctx, &sq->sq_obj, log_nb_desc, &attr, + SOCKET_ID_ANY); + if (ret) { + DRV_LOG(ERR, "Can't create SQ object."); + rte_errno = ENOMEM; + return -rte_errno; } - modify_attr.state = MLX5_SQC_STATE_RDY; - ret = mlx5_devx_cmd_modify_sq(sq->obj, &modify_attr); + ret = mlx5_devx_cmd_modify_sq(sq->sq_obj.sq, &modify_attr); if (ret) { - DRV_LOG(ERR, "Can't change sq state to ready."); - rte_errno = ENOMEM; - goto error; + DRV_LOG(ERR, "Can't change SQ state to ready."); + regex_ctrl_destroy_sq(qp, q_ind); + rte_errno = ENOMEM; + return -rte_errno; } - return 0; -error: - if (sq->wqe_umem) - mlx5_glue->devx_umem_dereg(sq->wqe_umem); - if (buf) - rte_free(buf); - if (sq->dbr_offset) - mlx5_release_dbr(&priv->dbrpgs, sq->dbr_umem, sq->dbr_offset); - return -rte_errno; #else (void)priv; (void)qp; @@ -220,44 +199,6 @@ } /** - * Destroy the SQ object. - * - * @param priv - * Pointer to the priv object. - * @param qp - * Pointer to the QP element - * @param q_ind - * The index of the queue. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -regex_ctrl_destroy_sq(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, - uint16_t q_ind) -{ - struct mlx5_regex_sq *sq = &qp->sqs[q_ind]; - - if (sq->wqe_umem) { - mlx5_glue->devx_umem_dereg(sq->wqe_umem); - sq->wqe_umem = NULL; - } - if (sq->wqe) { - rte_free((void *)(uintptr_t)sq->wqe); - sq->wqe = NULL; - } - if (sq->dbr_offset) { - mlx5_release_dbr(&priv->dbrpgs, sq->dbr_umem, sq->dbr_offset); - sq->dbr_offset = -1; - } - if (sq->obj) { - mlx5_devx_cmd_destroy(sq->obj); - sq->obj = NULL; - } - return 0; -} - -/** * Setup the qp. * * @param dev @@ -329,7 +270,7 @@ mlx5_mr_btree_free(&qp->mr_ctrl.cache_bh); err_btree: for (i = 0; i < nb_sq_config; i++) - regex_ctrl_destroy_sq(priv, qp, i); + regex_ctrl_destroy_sq(qp, i); regex_ctrl_destroy_cq(&qp->cq); err_cq: rte_free(qp->sqs); diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c index 255fd40..cd0f9bd 100644 --- a/drivers/regex/mlx5/mlx5_regex_fastpath.c +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -110,12 +110,12 @@ struct mlx5_regex_job { &priv->mr_scache, &qp->mr_ctrl, rte_pktmbuf_mtod(op->mbuf, uintptr_t), !!(op->mbuf->ol_flags & EXT_ATTACHED_MBUF)); - uint8_t *wqe = (uint8_t *)sq->wqe + wqe_offset; + uint8_t *wqe = (uint8_t *)(uintptr_t)sq->sq_obj.wqes + wqe_offset; int ds = 4; /* ctrl + meta + input + output */ set_wqe_ctrl_seg((struct mlx5_wqe_ctrl_seg *)wqe, sq->pi, - MLX5_OPCODE_MMO, MLX5_OPC_MOD_MMO_REGEX, sq->obj->id, - 0, ds, 0, 0); + MLX5_OPCODE_MMO, MLX5_OPC_MOD_MMO_REGEX, + sq->sq_obj.sq->id, 0, ds, 0, 0); set_regex_ctrl_seg(wqe + 12, 0, op->group_id0, op->group_id1, op->group_id2, op->group_id3, 0); @@ -137,12 +137,12 @@ struct mlx5_regex_job { { size_t wqe_offset = (sq->db_pi & (sq_size_get(sq) - 1)) * MLX5_SEND_WQE_BB; - uint8_t *wqe = (uint8_t *)sq->wqe + wqe_offset; + uint8_t *wqe = (uint8_t *)(uintptr_t)sq->sq_obj.wqes + wqe_offset; ((struct mlx5_wqe_ctrl_seg *)wqe)->fm_ce_se = MLX5_WQE_CTRL_CQ_UPDATE; uint64_t *doorbell_addr = (uint64_t *)((uint8_t *)uar->base_addr + 0x800); rte_io_wmb(); - sq->dbr[MLX5_SND_DBR] = rte_cpu_to_be_32((sq->db_pi + 1) & + sq->sq_obj.db_rec[MLX5_SND_DBR] = rte_cpu_to_be_32((sq->db_pi + 1) & MLX5_REGEX_MAX_WQE_INDEX); rte_wmb(); *doorbell_addr = *(volatile uint64_t *)wqe; @@ -301,7 +301,7 @@ struct mlx5_regex_job { uint32_t job_id; for (sqid = 0; sqid < queue->nb_obj; sqid++) { struct mlx5_regex_sq *sq = &queue->sqs[sqid]; - uint8_t *wqe = (uint8_t *)sq->wqe; + uint8_t *wqe = (uint8_t *)(uintptr_t)sq->sq_obj.wqes; for (entry = 0 ; entry < sq_size_get(sq); entry++) { job_id = sqid * sq_size_get(sq) + entry; struct mlx5_regex_job *job = &queue->jobs[job_id]; @@ -334,7 +334,7 @@ struct mlx5_regex_job { return -ENOMEM; qp->metadata = mlx5_glue->reg_mr(pd, ptr, - MLX5_REGEX_METADATA_SIZE*qp->nb_desc, + MLX5_REGEX_METADATA_SIZE * qp->nb_desc, IBV_ACCESS_LOCAL_WRITE); if (!qp->metadata) { DRV_LOG(ERR, "Failed to register metadata"); From patchwork Wed Jan 6 08:19:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86037 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C6AA5A09FF; Wed, 6 Jan 2021 09:22:10 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B66421608FF; Wed, 6 Jan 2021 09:21:28 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 15A32160903 for ; Wed, 6 Jan 2021 09:21:27 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:24 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgh009291; Wed, 6 Jan 2021 10:21:24 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:36 +0000 Message-Id: <1609921181-5019-15-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 14/19] net/mlx5: move rearm and clock queue SQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for DevX SQ creation for rearm and clock queue. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.h | 8 +-- drivers/net/mlx5/mlx5_txpp.c | 147 +++++++++++-------------------------------- 2 files changed, 36 insertions(+), 119 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 079cbca..0a0a943 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -615,15 +615,9 @@ struct mlx5_txpp_wq { uint32_t cq_ci:24; uint32_t arm_sn:2; /* Send Queue related data.*/ - struct mlx5_devx_obj *sq; - void *sq_umem; - union { - volatile void *sq_buf; - volatile struct mlx5_wqe *wqes; - }; + struct mlx5_devx_sq sq_obj; uint16_t sq_size; /* Number of WQEs in the queue. */ uint16_t sq_ci; /* Next WQE to execute. */ - volatile uint32_t *sq_dbrec; }; /* Tx packet pacing internal timestamp. */ diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c index bd679c2..b38482d 100644 --- a/drivers/net/mlx5/mlx5_txpp.c +++ b/drivers/net/mlx5/mlx5_txpp.c @@ -129,12 +129,7 @@ static void mlx5_txpp_destroy_send_queue(struct mlx5_txpp_wq *wq) { - if (wq->sq) - claim_zero(mlx5_devx_cmd_destroy(wq->sq)); - if (wq->sq_umem) - claim_zero(mlx5_os_umem_dereg(wq->sq_umem)); - if (wq->sq_buf) - mlx5_free((void *)(uintptr_t)wq->sq_buf); + mlx5_devx_sq_destroy(&wq->sq_obj); mlx5_devx_cq_destroy(&wq->cq_obj); memset(wq, 0, sizeof(*wq)); } @@ -163,6 +158,7 @@ mlx5_txpp_doorbell_rearm_queue(struct mlx5_dev_ctx_shared *sh, uint16_t ci) { struct mlx5_txpp_wq *wq = &sh->txpp.rearm_queue; + struct mlx5_wqe *wqe = (struct mlx5_wqe *)(uintptr_t)wq->sq_obj.wqes; union { uint32_t w32[2]; uint64_t w64; @@ -171,11 +167,11 @@ wq->sq_ci = ci + 1; cs.w32[0] = rte_cpu_to_be_32(rte_be_to_cpu_32 - (wq->wqes[ci & (wq->sq_size - 1)].ctrl[0]) | (ci - 1) << 8); - cs.w32[1] = wq->wqes[ci & (wq->sq_size - 1)].ctrl[1]; + (wqe[ci & (wq->sq_size - 1)].ctrl[0]) | (ci - 1) << 8); + cs.w32[1] = wqe[ci & (wq->sq_size - 1)].ctrl[1]; /* Update SQ doorbell record with new SQ ci. */ rte_compiler_barrier(); - *wq->sq_dbrec = rte_cpu_to_be_32(wq->sq_ci); + *wq->sq_obj.db_rec = rte_cpu_to_be_32(wq->sq_ci); /* Make sure the doorbell record is updated. */ rte_wmb(); /* Write to doorbel register to start processing. */ @@ -188,7 +184,7 @@ mlx5_txpp_fill_wqe_rearm_queue(struct mlx5_dev_ctx_shared *sh) { struct mlx5_txpp_wq *wq = &sh->txpp.rearm_queue; - struct mlx5_wqe *wqe = (struct mlx5_wqe *)(uintptr_t)wq->wqes; + struct mlx5_wqe *wqe = (struct mlx5_wqe *)(uintptr_t)wq->sq_obj.wqes; uint32_t i; for (i = 0; i < wq->sq_size; i += 2) { @@ -199,7 +195,7 @@ /* Build SEND_EN request with slave WQE index. */ cs = &wqe[i + 0].cseg; cs->opcode = RTE_BE32(MLX5_OPCODE_SEND_EN | 0); - cs->sq_ds = rte_cpu_to_be_32((wq->sq->id << 8) | 2); + cs->sq_ds = rte_cpu_to_be_32((wq->sq_obj.sq->id << 8) | 2); cs->flags = RTE_BE32(MLX5_COMP_ALWAYS << MLX5_COMP_MODE_OFFSET); cs->misc = RTE_BE32(0); @@ -207,11 +203,12 @@ index = (i * MLX5_TXPP_REARM / 2 + MLX5_TXPP_REARM) & ((1 << MLX5_WQ_INDEX_WIDTH) - 1); qs->max_index = rte_cpu_to_be_32(index); - qs->qpn_cqn = rte_cpu_to_be_32(sh->txpp.clock_queue.sq->id); + qs->qpn_cqn = + rte_cpu_to_be_32(sh->txpp.clock_queue.sq_obj.sq->id); /* Build WAIT request with slave CQE index. */ cs = &wqe[i + 1].cseg; cs->opcode = RTE_BE32(MLX5_OPCODE_WAIT | 0); - cs->sq_ds = rte_cpu_to_be_32((wq->sq->id << 8) | 2); + cs->sq_ds = rte_cpu_to_be_32((wq->sq_obj.sq->id << 8) | 2); cs->flags = RTE_BE32(MLX5_COMP_ONLY_ERR << MLX5_COMP_MODE_OFFSET); cs->misc = RTE_BE32(0); @@ -228,21 +225,23 @@ static int mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh) { - struct mlx5_devx_create_sq_attr sq_attr = { 0 }; + struct mlx5_devx_create_sq_attr sq_attr = { + .cd_master = 1, + .state = MLX5_SQC_STATE_RST, + .tis_lst_sz = 1, + .tis_num = sh->tis->id, + .wq_attr = (struct mlx5_devx_wq_attr){ + .pd = sh->pdn, + .uar_page = mlx5_os_get_devx_uar_page_id(sh->tx_uar), + }, + }; struct mlx5_devx_modify_sq_attr msq_attr = { 0 }; struct mlx5_devx_cq_attr cq_attr = { .uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar), }; struct mlx5_txpp_wq *wq = &sh->txpp.rearm_queue; - size_t page_size; - uint32_t umem_size, umem_dbrec; int ret; - page_size = rte_mem_page_size(); - if (page_size == (size_t)-1) { - DRV_LOG(ERR, "Failed to get mem page size"); - return -ENOMEM; - } /* Create completion queue object for Rearm Queue. */ ret = mlx5_devx_cq_create(sh->ctx, &wq->cq_obj, log2above(MLX5_TXPP_REARM_CQ_SIZE), &cq_attr, @@ -253,63 +252,25 @@ } wq->cq_ci = 0; wq->arm_sn = 0; - /* - * Allocate memory buffer for Send Queue WQEs. - * There should be no WQE leftovers in the cyclic queue. - */ wq->sq_size = MLX5_TXPP_REARM_SQ_SIZE; MLX5_ASSERT(wq->sq_size == (1 << log2above(wq->sq_size))); - umem_size = MLX5_WQE_SIZE * wq->sq_size; - umem_dbrec = RTE_ALIGN(umem_size, MLX5_DBR_SIZE); - umem_size += MLX5_DBR_SIZE; - wq->sq_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size, - page_size, sh->numa_node); - if (!wq->sq_buf) { - DRV_LOG(ERR, "Failed to allocate memory for Rearm Queue."); - rte_errno = ENOMEM; - goto error; - } - /* Register allocated buffer in user space with DevX. */ - wq->sq_umem = mlx5_os_umem_reg(sh->ctx, - (void *)(uintptr_t)wq->sq_buf, - umem_size, - IBV_ACCESS_LOCAL_WRITE); - if (!wq->sq_umem) { - rte_errno = errno; - DRV_LOG(ERR, "Failed to register umem for Rearm Queue."); - goto error; - } /* Create send queue object for Rearm Queue. */ - sq_attr.state = MLX5_SQC_STATE_RST; - sq_attr.tis_lst_sz = 1; - sq_attr.tis_num = sh->tis->id; sq_attr.cqn = wq->cq_obj.cq->id; - sq_attr.cd_master = 1; - sq_attr.wq_attr.uar_page = mlx5_os_get_devx_uar_page_id(sh->tx_uar); - sq_attr.wq_attr.wq_type = MLX5_WQ_TYPE_CYCLIC; - sq_attr.wq_attr.pd = sh->pdn; - sq_attr.wq_attr.log_wq_stride = rte_log2_u32(MLX5_WQE_SIZE); - sq_attr.wq_attr.log_wq_sz = rte_log2_u32(wq->sq_size); - sq_attr.wq_attr.dbr_umem_valid = 1; - sq_attr.wq_attr.dbr_addr = umem_dbrec; - sq_attr.wq_attr.dbr_umem_id = mlx5_os_get_umem_id(wq->sq_umem); - sq_attr.wq_attr.wq_umem_valid = 1; - sq_attr.wq_attr.wq_umem_id = mlx5_os_get_umem_id(wq->sq_umem); - sq_attr.wq_attr.wq_umem_offset = 0; - wq->sq = mlx5_devx_cmd_create_sq(sh->ctx, &sq_attr); - if (!wq->sq) { + /* There should be no WQE leftovers in the cyclic queue. */ + ret = mlx5_devx_sq_create(sh->ctx, &wq->sq_obj, + log2above(MLX5_TXPP_REARM_SQ_SIZE), &sq_attr, + sh->numa_node); + if (ret) { rte_errno = errno; DRV_LOG(ERR, "Failed to create SQ for Rearm Queue."); goto error; } - wq->sq_dbrec = RTE_PTR_ADD(wq->sq_buf, umem_dbrec + - MLX5_SND_DBR * sizeof(uint32_t)); /* Build the WQEs in the Send Queue before goto Ready state. */ mlx5_txpp_fill_wqe_rearm_queue(sh); /* Change queue state to ready. */ msq_attr.sq_state = MLX5_SQC_STATE_RST; msq_attr.state = MLX5_SQC_STATE_RDY; - ret = mlx5_devx_cmd_modify_sq(wq->sq, &msq_attr); + ret = mlx5_devx_cmd_modify_sq(wq->sq_obj.sq, &msq_attr); if (ret) { DRV_LOG(ERR, "Failed to set SQ ready state Rearm Queue."); goto error; @@ -326,7 +287,7 @@ mlx5_txpp_fill_wqe_clock_queue(struct mlx5_dev_ctx_shared *sh) { struct mlx5_txpp_wq *wq = &sh->txpp.clock_queue; - struct mlx5_wqe *wqe = (struct mlx5_wqe *)(uintptr_t)wq->wqes; + struct mlx5_wqe *wqe = (struct mlx5_wqe *)(uintptr_t)wq->sq_obj.wqes; struct mlx5_wqe_cseg *cs = &wqe->cseg; uint32_t wqe_size, opcode, i; uint8_t *dst; @@ -344,7 +305,7 @@ opcode = MLX5_OPCODE_NOP; } cs->opcode = rte_cpu_to_be_32(opcode | 0); /* Index is ignored. */ - cs->sq_ds = rte_cpu_to_be_32((wq->sq->id << 8) | + cs->sq_ds = rte_cpu_to_be_32((wq->sq_obj.sq->id << 8) | (wqe_size / MLX5_WSEG_SIZE)); cs->flags = RTE_BE32(MLX5_COMP_ALWAYS << MLX5_COMP_MODE_OFFSET); cs->misc = RTE_BE32(0); @@ -413,10 +374,11 @@ } wcopy: /* Duplicate the pattern to the next WQEs. */ - dst = (uint8_t *)(uintptr_t)wq->sq_buf; + dst = (uint8_t *)(uintptr_t)wq->sq_obj.umem_buf; for (i = 1; i < MLX5_TXPP_CLKQ_SIZE; i++) { dst += wqe_size; - rte_memcpy(dst, (void *)(uintptr_t)wq->sq_buf, wqe_size); + rte_memcpy(dst, (void *)(uintptr_t)wq->sq_obj.umem_buf, + wqe_size); } } @@ -432,15 +394,8 @@ .uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar), }; struct mlx5_txpp_wq *wq = &sh->txpp.clock_queue; - size_t page_size; - uint32_t umem_size, umem_dbrec; int ret; - page_size = rte_mem_page_size(); - if (page_size == (size_t)-1) { - DRV_LOG(ERR, "Failed to get mem page size"); - return -ENOMEM; - } sh->txpp.tsa = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, MLX5_TXPP_REARM_SQ_SIZE * sizeof(struct mlx5_txpp_ts), @@ -473,26 +428,6 @@ } /* There should not be WQE leftovers in the cyclic queue. */ MLX5_ASSERT(wq->sq_size == (1 << log2above(wq->sq_size))); - umem_size = MLX5_WQE_SIZE * wq->sq_size; - umem_dbrec = RTE_ALIGN(umem_size, MLX5_DBR_SIZE); - umem_size += MLX5_DBR_SIZE; - wq->sq_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size, - page_size, sh->numa_node); - if (!wq->sq_buf) { - DRV_LOG(ERR, "Failed to allocate memory for Clock Queue."); - rte_errno = ENOMEM; - goto error; - } - /* Register allocated buffer in user space with DevX. */ - wq->sq_umem = mlx5_os_umem_reg(sh->ctx, - (void *)(uintptr_t)wq->sq_buf, - umem_size, - IBV_ACCESS_LOCAL_WRITE); - if (!wq->sq_umem) { - rte_errno = errno; - DRV_LOG(ERR, "Failed to register umem for Clock Queue."); - goto error; - } /* Create send queue object for Clock Queue. */ if (sh->txpp.test) { sq_attr.tis_lst_sz = 1; @@ -503,37 +438,25 @@ sq_attr.non_wire = 1; sq_attr.static_sq_wq = 1; } - sq_attr.state = MLX5_SQC_STATE_RST; sq_attr.cqn = wq->cq_obj.cq->id; sq_attr.packet_pacing_rate_limit_index = sh->txpp.pp_id; sq_attr.wq_attr.cd_slave = 1; sq_attr.wq_attr.uar_page = mlx5_os_get_devx_uar_page_id(sh->tx_uar); - sq_attr.wq_attr.wq_type = MLX5_WQ_TYPE_CYCLIC; sq_attr.wq_attr.pd = sh->pdn; - sq_attr.wq_attr.log_wq_stride = rte_log2_u32(MLX5_WQE_SIZE); - sq_attr.wq_attr.log_wq_sz = rte_log2_u32(wq->sq_size); - sq_attr.wq_attr.dbr_umem_valid = 1; - sq_attr.wq_attr.dbr_addr = umem_dbrec; - sq_attr.wq_attr.dbr_umem_id = mlx5_os_get_umem_id(wq->sq_umem); - sq_attr.wq_attr.wq_umem_valid = 1; - sq_attr.wq_attr.wq_umem_id = mlx5_os_get_umem_id(wq->sq_umem); - /* umem_offset must be zero for static_sq_wq queue. */ - sq_attr.wq_attr.wq_umem_offset = 0; - wq->sq = mlx5_devx_cmd_create_sq(sh->ctx, &sq_attr); - if (!wq->sq) { + ret = mlx5_devx_sq_create(sh->ctx, &wq->sq_obj, log2above(wq->sq_size), + &sq_attr, sh->numa_node); + if (ret) { rte_errno = errno; DRV_LOG(ERR, "Failed to create SQ for Clock Queue."); goto error; } - wq->sq_dbrec = RTE_PTR_ADD(wq->sq_buf, umem_dbrec + - MLX5_SND_DBR * sizeof(uint32_t)); /* Build the WQEs in the Send Queue before goto Ready state. */ mlx5_txpp_fill_wqe_clock_queue(sh); /* Change queue state to ready. */ msq_attr.sq_state = MLX5_SQC_STATE_RST; msq_attr.state = MLX5_SQC_STATE_RDY; wq->sq_ci = 0; - ret = mlx5_devx_cmd_modify_sq(wq->sq, &msq_attr); + ret = mlx5_devx_cmd_modify_sq(wq->sq_obj.sq, &msq_attr); if (ret) { DRV_LOG(ERR, "Failed to set SQ ready state Clock Queue."); goto error; From patchwork Wed Jan 6 08:19:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86039 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D1472A09FF; Wed, 6 Jan 2021 09:22:28 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 824D9160905; Wed, 6 Jan 2021 09:21:33 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 1A47B160914 for ; Wed, 6 Jan 2021 09:21:32 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:26 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgi009291; Wed, 6 Jan 2021 10:21:26 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:37 +0000 Message-Id: <1609921181-5019-16-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 15/19] net/mlx5: move Tx SQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for Tx SQ creation. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.h | 8 +-- drivers/net/mlx5/mlx5_devx.c | 162 ++++++++++--------------------------------- 2 files changed, 40 insertions(+), 130 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 0a0a943..62d7c89 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -841,11 +841,9 @@ struct mlx5_txq_obj { struct { struct rte_eth_dev *dev; struct mlx5_devx_cq cq_obj; - struct mlx5_devx_obj *sq_devx; - void *sq_umem; - void *sq_buf; - int64_t sq_dbrec_offset; - struct mlx5_devx_dbr_page *sq_dbrec_page; + /* DevX CQ object and its resources. */ + struct mlx5_devx_sq sq_obj; + /* DevX SQ object and its resources. */ }; }; }; diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 913f169..96c44d4 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -115,7 +115,7 @@ else msq_attr.sq_state = MLX5_SQC_STATE_RDY; msq_attr.state = MLX5_SQC_STATE_RST; - ret = mlx5_devx_cmd_modify_sq(obj->sq_devx, &msq_attr); + ret = mlx5_devx_cmd_modify_sq(obj->sq_obj.sq, &msq_attr); if (ret) { DRV_LOG(ERR, "Cannot change the Tx SQ state to RESET" " %s", strerror(errno)); @@ -127,7 +127,7 @@ /* Change queue state to ready. */ msq_attr.sq_state = MLX5_SQC_STATE_RST; msq_attr.state = MLX5_SQC_STATE_RDY; - ret = mlx5_devx_cmd_modify_sq(obj->sq_devx, &msq_attr); + ret = mlx5_devx_cmd_modify_sq(obj->sq_obj.sq, &msq_attr); if (ret) { DRV_LOG(ERR, "Cannot change the Tx SQ state to READY" " %s", strerror(errno)); @@ -1052,36 +1052,6 @@ #if defined(HAVE_MLX5DV_DEVX_UAR_OFFSET) || !defined(HAVE_INFINIBAND_VERBS_H) /** - * Release DevX SQ resources. - * - * @param txq_obj - * DevX Tx queue object. - */ -static void -mlx5_txq_release_devx_sq_resources(struct mlx5_txq_obj *txq_obj) -{ - if (txq_obj->sq_devx) { - claim_zero(mlx5_devx_cmd_destroy(txq_obj->sq_devx)); - txq_obj->sq_devx = NULL; - } - if (txq_obj->sq_umem) { - claim_zero(mlx5_os_umem_dereg(txq_obj->sq_umem)); - txq_obj->sq_umem = NULL; - } - if (txq_obj->sq_buf) { - mlx5_free(txq_obj->sq_buf); - txq_obj->sq_buf = NULL; - } - if (txq_obj->sq_dbrec_page) { - claim_zero(mlx5_release_dbr(&txq_obj->txq_ctrl->priv->dbrpgs, - mlx5_os_get_umem_id - (txq_obj->sq_dbrec_page->umem), - txq_obj->sq_dbrec_offset)); - txq_obj->sq_dbrec_page = NULL; - } -} - -/** * Destroy the Tx queue DevX object. * * @param txq_obj @@ -1090,7 +1060,8 @@ static void mlx5_txq_release_devx_resources(struct mlx5_txq_obj *txq_obj) { - mlx5_txq_release_devx_sq_resources(txq_obj); + mlx5_devx_sq_destroy(&txq_obj->sq_obj); + memset(&txq_obj->sq_obj, 0, sizeof(txq_obj->sq_obj)); mlx5_devx_cq_destroy(&txq_obj->cq_obj); memset(&txq_obj->cq_obj, 0, sizeof(txq_obj->cq_obj)); } @@ -1102,100 +1073,39 @@ * Pointer to Ethernet device. * @param idx * Queue index in DPDK Tx queue array. + * @param[in] log_desc_n + * Log of number of descriptors in queue. * * @return - * Number of WQEs in SQ, 0 otherwise and rte_errno is set. + * 0 on success, a negative errno value otherwise and rte_errno is set. */ -static uint32_t -mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx) +static int +mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx, + uint16_t log_desc_n) { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_txq_data *txq_data = (*priv->txqs)[idx]; struct mlx5_txq_ctrl *txq_ctrl = container_of(txq_data, struct mlx5_txq_ctrl, txq); struct mlx5_txq_obj *txq_obj = txq_ctrl->obj; - struct mlx5_devx_create_sq_attr sq_attr = { 0 }; - size_t page_size; - uint32_t wqe_n; - int ret; + struct mlx5_devx_create_sq_attr sq_attr = { + .flush_in_error_en = 1, + .allow_multi_pkt_send_wqe = !!priv->config.mps, + .min_wqe_inline_mode = priv->config.hca_attr.vport_inline_mode, + .allow_swp = !!priv->config.swp, + .cqn = txq_obj->cq_obj.cq->id, + .tis_lst_sz = 1, + .tis_num = priv->sh->tis->id, + .wq_attr = (struct mlx5_devx_wq_attr){ + .pd = priv->sh->pdn, + .uar_page = + mlx5_os_get_devx_uar_page_id(priv->sh->tx_uar), + }, + }; - MLX5_ASSERT(txq_data); - MLX5_ASSERT(txq_obj); - page_size = rte_mem_page_size(); - if (page_size == (size_t)-1) { - DRV_LOG(ERR, "Failed to get mem page size."); - rte_errno = ENOMEM; - return 0; - } - wqe_n = RTE_MIN(1UL << txq_data->elts_n, - (uint32_t)priv->sh->device_attr.max_qp_wr); - txq_obj->sq_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, - wqe_n * sizeof(struct mlx5_wqe), - page_size, priv->sh->numa_node); - if (!txq_obj->sq_buf) { - DRV_LOG(ERR, - "Port %u Tx queue %u cannot allocate memory (SQ).", - dev->data->port_id, txq_data->idx); - rte_errno = ENOMEM; - goto error; - } - /* Register allocated buffer in user space with DevX. */ - txq_obj->sq_umem = mlx5_os_umem_reg - (priv->sh->ctx, - (void *)txq_obj->sq_buf, - wqe_n * sizeof(struct mlx5_wqe), - IBV_ACCESS_LOCAL_WRITE); - if (!txq_obj->sq_umem) { - rte_errno = errno; - DRV_LOG(ERR, - "Port %u Tx queue %u cannot register memory (SQ).", - dev->data->port_id, txq_data->idx); - goto error; - } - /* Allocate doorbell record for send queue. */ - txq_obj->sq_dbrec_offset = mlx5_get_dbr(priv->sh->ctx, - &priv->dbrpgs, - &txq_obj->sq_dbrec_page); - if (txq_obj->sq_dbrec_offset < 0) { - rte_errno = errno; - DRV_LOG(ERR, "Failed to allocate SQ door-bell."); - goto error; - } - sq_attr.tis_lst_sz = 1; - sq_attr.tis_num = priv->sh->tis->id; - sq_attr.state = MLX5_SQC_STATE_RST; - sq_attr.cqn = txq_obj->cq_obj.cq->id; - sq_attr.flush_in_error_en = 1; - sq_attr.allow_multi_pkt_send_wqe = !!priv->config.mps; - sq_attr.allow_swp = !!priv->config.swp; - sq_attr.min_wqe_inline_mode = priv->config.hca_attr.vport_inline_mode; - sq_attr.wq_attr.uar_page = - mlx5_os_get_devx_uar_page_id(priv->sh->tx_uar); - sq_attr.wq_attr.wq_type = MLX5_WQ_TYPE_CYCLIC; - sq_attr.wq_attr.pd = priv->sh->pdn; - sq_attr.wq_attr.log_wq_stride = rte_log2_u32(MLX5_WQE_SIZE); - sq_attr.wq_attr.log_wq_sz = log2above(wqe_n); - sq_attr.wq_attr.dbr_umem_valid = 1; - sq_attr.wq_attr.dbr_addr = txq_obj->sq_dbrec_offset; - sq_attr.wq_attr.dbr_umem_id = - mlx5_os_get_umem_id(txq_obj->sq_dbrec_page->umem); - sq_attr.wq_attr.wq_umem_valid = 1; - sq_attr.wq_attr.wq_umem_id = mlx5_os_get_umem_id(txq_obj->sq_umem); - sq_attr.wq_attr.wq_umem_offset = (uintptr_t)txq_obj->sq_buf % page_size; /* Create Send Queue object with DevX. */ - txq_obj->sq_devx = mlx5_devx_cmd_create_sq(priv->sh->ctx, &sq_attr); - if (!txq_obj->sq_devx) { - rte_errno = errno; - DRV_LOG(ERR, "Port %u Tx queue %u SQ creation failure.", - dev->data->port_id, idx); - goto error; - } - return wqe_n; -error: - ret = rte_errno; - mlx5_txq_release_devx_sq_resources(txq_obj); - rte_errno = ret; - return 0; + return mlx5_devx_sq_create(priv->sh->ctx, &txq_obj->sq_obj, log_desc_n, + &sq_attr, priv->sh->numa_node); } #endif @@ -1267,27 +1177,29 @@ txq_data->cq_db = txq_obj->cq_obj.db_rec; *txq_data->cq_db = 0; /* Create Send Queue object with DevX. */ - wqe_n = mlx5_txq_create_devx_sq_resources(dev, idx); - if (!wqe_n) { + wqe_n = RTE_MIN(1UL << txq_data->elts_n, + (uint32_t)priv->sh->device_attr.max_qp_wr); + log_desc_n = log2above(wqe_n); + ret = mlx5_txq_create_devx_sq_resources(dev, idx, log_desc_n); + if (ret) { + DRV_LOG(ERR, "Port %u Tx queue %u SQ creation failure.", + dev->data->port_id, idx); rte_errno = errno; goto error; } /* Create the Work Queue. */ - txq_data->wqe_n = log2above(wqe_n); + txq_data->wqe_n = log_desc_n; txq_data->wqe_s = 1 << txq_data->wqe_n; txq_data->wqe_m = txq_data->wqe_s - 1; - txq_data->wqes = (struct mlx5_wqe *)txq_obj->sq_buf; + txq_data->wqes = (struct mlx5_wqe *)(uintptr_t)txq_obj->sq_obj.wqes; txq_data->wqes_end = txq_data->wqes + txq_data->wqe_s; txq_data->wqe_ci = 0; txq_data->wqe_pi = 0; txq_data->wqe_comp = 0; txq_data->wqe_thres = txq_data->wqe_s / MLX5_TX_COMP_THRESH_INLINE_DIV; - txq_data->qp_db = (volatile uint32_t *) - (txq_obj->sq_dbrec_page->dbrs + - txq_obj->sq_dbrec_offset + - MLX5_SND_DBR * sizeof(uint32_t)); + txq_data->qp_db = txq_obj->sq_obj.db_rec; *txq_data->qp_db = 0; - txq_data->qp_num_8s = txq_obj->sq_devx->id << 8; + txq_data->qp_num_8s = txq_obj->sq_obj.sq->id << 8; /* Change Send Queue state to Ready-to-Send. */ ret = mlx5_devx_modify_sq(txq_obj, MLX5_TXQ_MOD_RST2RDY, 0); if (ret) { From patchwork Wed Jan 6 08:19:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86040 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2459EA09FF; Wed, 6 Jan 2021 09:22:37 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BEC36160918; Wed, 6 Jan 2021 09:21:34 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 1EC96160915 for ; Wed, 6 Jan 2021 09:21:32 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:29 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgj009291; Wed, 6 Jan 2021 10:21:29 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:38 +0000 Message-Id: <1609921181-5019-17-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 16/19] net/mlx5: move ASO SQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for ASO SQ creation. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common_devx.h | 1 + drivers/net/mlx5/mlx5.h | 8 +-- drivers/net/mlx5/mlx5_flow_age.c | 94 ++++++++++------------------------ 3 files changed, 30 insertions(+), 73 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_devx.h b/drivers/common/mlx5/mlx5_common_devx.h index 6b078b2..992ad8f 100644 --- a/drivers/common/mlx5/mlx5_common_devx.h +++ b/drivers/common/mlx5/mlx5_common_devx.h @@ -28,6 +28,7 @@ struct mlx5_devx_sq { union { volatile void *umem_buf; volatile struct mlx5_wqe *wqes; /* The SQ ring buffer. */ + volatile struct mlx5_aso_wqe *aso_wqes; }; volatile uint32_t *db_rec; /* The SQ doorbell record. */ }; diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 62d7c89..a3fd8d5 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -487,13 +487,7 @@ struct mlx5_aso_sq_elem { struct mlx5_aso_sq { uint16_t log_desc_n; struct mlx5_aso_cq cq; - struct mlx5_devx_obj *sq; - struct mlx5dv_devx_umem *wqe_umem; /* SQ buffer umem. */ - union { - volatile void *umem_buf; - volatile struct mlx5_aso_wqe *wqes; - }; - volatile uint32_t *db_rec; + struct mlx5_devx_sq sq_obj; volatile uint64_t *uar_addr; struct mlx5_aso_devx_mr mr; uint16_t pi; diff --git a/drivers/net/mlx5/mlx5_flow_age.c b/drivers/net/mlx5/mlx5_flow_age.c index a75adc8..3005afd 100644 --- a/drivers/net/mlx5/mlx5_flow_age.c +++ b/drivers/net/mlx5/mlx5_flow_age.c @@ -142,18 +142,7 @@ static void mlx5_aso_destroy_sq(struct mlx5_aso_sq *sq) { - if (sq->wqe_umem) { - mlx5_glue->devx_umem_dereg(sq->wqe_umem); - sq->wqe_umem = NULL; - } - if (sq->umem_buf) { - mlx5_free((void *)(uintptr_t)sq->umem_buf); - sq->umem_buf = NULL; - } - if (sq->sq) { - mlx5_devx_cmd_destroy(sq->sq); - sq->sq = NULL; - } + mlx5_devx_sq_destroy(&sq->sq_obj); mlx5_aso_cq_destroy(&sq->cq); mlx5_aso_devx_dereg_mr(&sq->mr); memset(sq, 0, sizeof(*sq)); @@ -174,7 +163,7 @@ uint64_t addr; /* All the next fields state should stay constant. */ - for (i = 0, wqe = &sq->wqes[0]; i < size; ++i, ++wqe) { + for (i = 0, wqe = &sq->sq_obj.aso_wqes[0]; i < size; ++i, ++wqe) { wqe->general_cseg.sq_ds = rte_cpu_to_be_32((sq->sqn << 8) | (sizeof(*wqe) >> 4)); wqe->aso_cseg.lkey = rte_cpu_to_be_32(sq->mr.mkey->id); @@ -215,12 +204,18 @@ mlx5_aso_sq_create(void *ctx, struct mlx5_aso_sq *sq, int socket, void *uar, uint32_t pdn, uint16_t log_desc_n) { - struct mlx5_devx_create_sq_attr attr = { 0 }; - struct mlx5_devx_modify_sq_attr modify_attr = { 0 }; - size_t pgsize = rte_mem_page_size(); - struct mlx5_devx_wq_attr *wq_attr = &attr.wq_attr; + struct mlx5_devx_create_sq_attr attr = { + .user_index = 0xFFFF, + .wq_attr = (struct mlx5_devx_wq_attr){ + .pd = pdn, + .uar_page = mlx5_os_get_devx_uar_page_id(uar), + }, + }; + struct mlx5_devx_modify_sq_attr modify_attr = { + .state = MLX5_SQC_STATE_RDY, + }; uint32_t sq_desc_n = 1 << log_desc_n; - uint32_t wq_size = sizeof(struct mlx5_aso_wqe) * sq_desc_n; + uint16_t log_wqbb_n; int ret; if (mlx5_aso_devx_reg_mr(ctx, (MLX5_ASO_AGE_ACTIONS_PER_POOL / 8) * @@ -230,58 +225,25 @@ mlx5_os_get_devx_uar_page_id(uar))) goto error; sq->log_desc_n = log_desc_n; - sq->umem_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, wq_size + - sizeof(*sq->db_rec) * 2, 4096, socket); - if (!sq->umem_buf) { - DRV_LOG(ERR, "Can't allocate wqe buffer."); - rte_errno = ENOMEM; - goto error; - } - sq->wqe_umem = mlx5_os_umem_reg(ctx, - (void *)(uintptr_t)sq->umem_buf, - wq_size + - sizeof(*sq->db_rec) * 2, - IBV_ACCESS_LOCAL_WRITE); - if (!sq->wqe_umem) { - DRV_LOG(ERR, "Failed to register umem for SQ."); - rte_errno = ENOMEM; - goto error; - } - attr.state = MLX5_SQC_STATE_RST; - attr.tis_lst_sz = 0; - attr.tis_num = 0; - attr.user_index = 0xFFFF; attr.cqn = sq->cq.cq_obj.cq->id; - wq_attr->uar_page = mlx5_os_get_devx_uar_page_id(uar); - wq_attr->pd = pdn; - wq_attr->wq_type = MLX5_WQ_TYPE_CYCLIC; - wq_attr->log_wq_pg_sz = rte_log2_u32(pgsize); - wq_attr->wq_umem_id = mlx5_os_get_umem_id(sq->wqe_umem); - wq_attr->wq_umem_offset = 0; - wq_attr->wq_umem_valid = 1; - wq_attr->log_wq_stride = 6; - wq_attr->log_wq_sz = rte_log2_u32(wq_size) - 6; - wq_attr->dbr_umem_id = wq_attr->wq_umem_id; - wq_attr->dbr_addr = wq_size; - wq_attr->dbr_umem_valid = 1; - sq->sq = mlx5_devx_cmd_create_sq(ctx, &attr); - if (!sq->sq) { - DRV_LOG(ERR, "Can't create sq object."); - rte_errno = ENOMEM; + /* for mlx5_aso_wqe that is twice the size of mlx5_wqe */ + log_wqbb_n = log_desc_n + 1; + ret = mlx5_devx_sq_create(ctx, &sq->sq_obj, log_wqbb_n, &attr, socket); + if (ret) { + DRV_LOG(ERR, "Can't create SQ object."); + rte_errno = ENOMEM; goto error; } - modify_attr.state = MLX5_SQC_STATE_RDY; - ret = mlx5_devx_cmd_modify_sq(sq->sq, &modify_attr); + ret = mlx5_devx_cmd_modify_sq(sq->sq_obj.sq, &modify_attr); if (ret) { - DRV_LOG(ERR, "Can't change sq state to ready."); - rte_errno = ENOMEM; + DRV_LOG(ERR, "Can't change SQ state to ready."); + rte_errno = ENOMEM; goto error; } sq->pi = 0; sq->head = 0; sq->tail = 0; - sq->sqn = sq->sq->id; - sq->db_rec = RTE_PTR_ADD(sq->umem_buf, (uintptr_t)(wq_attr->dbr_addr)); + sq->sqn = sq->sq_obj.sq->id; sq->uar_addr = mlx5_os_get_devx_uar_reg_addr(uar); mlx5_aso_init_sq(sq); return 0; @@ -345,8 +307,8 @@ return 0; sq->elts[start_head & mask].burst_size = max; do { - wqe = &sq->wqes[sq->head & mask]; - rte_prefetch0(&sq->wqes[(sq->head + 1) & mask]); + wqe = &sq->sq_obj.aso_wqes[sq->head & mask]; + rte_prefetch0(&sq->sq_obj.aso_wqes[(sq->head + 1) & mask]); /* Fill next WQE. */ rte_spinlock_lock(&mng->resize_sl); pool = mng->pools[sq->next]; @@ -371,7 +333,7 @@ wqe->general_cseg.flags = RTE_BE32(MLX5_COMP_ALWAYS << MLX5_COMP_MODE_OFFSET); rte_io_wmb(); - sq->db_rec[MLX5_SND_DBR] = rte_cpu_to_be_32(sq->pi); + sq->sq_obj.db_rec[MLX5_SND_DBR] = rte_cpu_to_be_32(sq->pi); rte_wmb(); *sq->uar_addr = *(volatile uint64_t *)wqe; /* Assume 64 bit ARCH.*/ rte_wmb(); @@ -418,7 +380,7 @@ cq->errors++; idx = rte_be_to_cpu_16(cqe->wqe_counter) & (1u << sq->log_desc_n); mlx5_aso_dump_err_objs((volatile uint32_t *)cqe, - (volatile uint32_t *)&sq->wqes[idx]); + (volatile uint32_t *)&sq->sq_obj.aso_wqes[idx]); } /** @@ -613,7 +575,7 @@ { int retries = 1024; - if (!sh->aso_age_mng->aso_sq.sq) + if (!sh->aso_age_mng->aso_sq.sq_obj.sq) return -EINVAL; rte_errno = 0; while (--retries) { From patchwork Wed Jan 6 08:19:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86041 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7F7FFA09FF; Wed, 6 Jan 2021 09:22:45 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ED6F81608D7; Wed, 6 Jan 2021 09:21:38 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 2937E1608D3 for ; Wed, 6 Jan 2021 09:21:37 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:34 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgk009291; Wed, 6 Jan 2021 10:21:33 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:39 +0000 Message-Id: <1609921181-5019-18-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 17/19] common/mlx5: share DevX RQ creation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The RQ object in DevX is used currently only in net driver, but it share for future. Add a structure that contains all the resources, and provide creation and release functions for it. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common_devx.c | 113 ++++++++++++++++++++++++ drivers/common/mlx5/mlx5_common_devx.h | 15 ++++ drivers/common/mlx5/rte_common_mlx5_exports.def | 2 + drivers/common/mlx5/version.map | 2 + 4 files changed, 132 insertions(+) diff --git a/drivers/common/mlx5/mlx5_common_devx.c b/drivers/common/mlx5/mlx5_common_devx.c index 58b75c3..d19be12 100644 --- a/drivers/common/mlx5/mlx5_common_devx.c +++ b/drivers/common/mlx5/mlx5_common_devx.c @@ -271,4 +271,117 @@ return -rte_errno; } +/** + * Destroy DevX Receive Queue. + * + * @param[in] rq + * DevX RQ to destroy. + */ +void +mlx5_devx_rq_destroy(struct mlx5_devx_rq *rq) +{ + if (rq->rq) + claim_zero(mlx5_devx_cmd_destroy(rq->rq)); + if (rq->umem_obj) + claim_zero(mlx5_os_umem_dereg(rq->umem_obj)); + if (rq->umem_buf) + mlx5_free((void *)(uintptr_t)rq->umem_buf); +} + +/** + * Create Receive Queue using DevX API. + * + * Get a pointer to partially initialized attributes structure, and updates the + * following fields: + * wq_umem_valid + * wq_umem_id + * wq_umem_offset + * dbr_umem_valid + * dbr_umem_id + * dbr_addr + * log_wq_pg_sz + * All other fields are updated by caller. + * + * @param[in] ctx + * Context returned from mlx5 open_device() glue function. + * @param[in/out] rq_obj + * Pointer to RQ to create. + * @param[in] wqe_size + * Size of WQE structure. + * @param[in] log_wqbb_n + * Log of number of WQBBs in queue. + * @param[in] attr + * Pointer to RQ attributes structure. + * @param[in] socket + * Socket to use for allocation. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, uint32_t wqe_size, + uint16_t log_wqbb_n, + struct mlx5_devx_create_rq_attr *attr, int socket) +{ + struct mlx5_devx_obj *rq = NULL; + struct mlx5dv_devx_umem *umem_obj = NULL; + void *umem_buf = NULL; + size_t alignment = MLX5_WQE_BUF_ALIGNMENT; + uint32_t umem_size, umem_dbrec; + uint16_t rq_size = 1 << log_wqbb_n; + int ret; + + if (alignment == (size_t)-1) { + DRV_LOG(ERR, "Failed to get WQE buf alignment."); + rte_errno = ENOMEM; + return -rte_errno; + } + /* Allocate memory buffer for WQEs and doorbell record. */ + umem_size = wqe_size * rq_size; + umem_dbrec = RTE_ALIGN(umem_size, MLX5_DBR_SIZE); + umem_size += MLX5_DBR_SIZE; + umem_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size, + alignment, socket); + if (!umem_buf) { + DRV_LOG(ERR, "Failed to allocate memory for RQ."); + rte_errno = ENOMEM; + return -rte_errno; + } + /* Register allocated buffer in user space with DevX. */ + umem_obj = mlx5_os_umem_reg(ctx, (void *)(uintptr_t)umem_buf, + umem_size, 0); + if (!umem_obj) { + DRV_LOG(ERR, "Failed to register umem for RQ."); + rte_errno = errno; + goto error; + } + /* Fill attributes for RQ object creation. */ + attr->wq_attr.wq_umem_valid = 1; + attr->wq_attr.wq_umem_id = mlx5_os_get_umem_id(umem_obj); + attr->wq_attr.wq_umem_offset = 0; + attr->wq_attr.dbr_umem_valid = 1; + attr->wq_attr.dbr_umem_id = attr->wq_attr.wq_umem_id; + attr->wq_attr.dbr_addr = umem_dbrec; + attr->wq_attr.log_wq_pg_sz = MLX5_LOG_PAGE_SIZE; + /* Create receive queue object with DevX. */ + rq = mlx5_devx_cmd_create_rq(ctx, attr, socket); + if (!rq) { + DRV_LOG(ERR, "Can't create DevX RQ object."); + rte_errno = ENOMEM; + goto error; + } + rq_obj->umem_buf = umem_buf; + rq_obj->umem_obj = umem_obj; + rq_obj->rq = rq; + rq_obj->db_rec = RTE_PTR_ADD(rq_obj->umem_buf, umem_dbrec); + return 0; +error: + ret = rte_errno; + if (umem_obj) + claim_zero(mlx5_os_umem_dereg(umem_obj)); + if (umem_buf) + mlx5_free((void *)(uintptr_t)umem_buf); + rte_errno = ret; + return -rte_errno; +} diff --git a/drivers/common/mlx5/mlx5_common_devx.h b/drivers/common/mlx5/mlx5_common_devx.h index 992ad8f..aad0184 100644 --- a/drivers/common/mlx5/mlx5_common_devx.h +++ b/drivers/common/mlx5/mlx5_common_devx.h @@ -33,6 +33,13 @@ struct mlx5_devx_sq { volatile uint32_t *db_rec; /* The SQ doorbell record. */ }; +/* DevX Receive Queue structure. */ +struct mlx5_devx_rq { + struct mlx5_devx_obj *rq; /* The RQ DevX object. */ + void *umem_obj; /* The RQ umem object. */ + volatile void *umem_buf; + volatile uint32_t *db_rec; /* The RQ doorbell record. */ +}; /* mlx5_common_devx.c */ @@ -52,4 +59,12 @@ int mlx5_devx_sq_create(void *ctx, struct mlx5_devx_sq *sq_obj, uint16_t log_wqbb_n, struct mlx5_devx_create_sq_attr *attr, int socket); +__rte_internal +void mlx5_devx_rq_destroy(struct mlx5_devx_rq *rq); + +__rte_internal +int mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, + uint32_t wqe_size, uint16_t log_wqbb_n, + struct mlx5_devx_create_rq_attr *attr, int socket); + #endif /* RTE_PMD_MLX5_COMMON_DEVX_H_ */ diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def index cfee96e..6e1ff50 100644 --- a/drivers/common/mlx5/rte_common_mlx5_exports.def +++ b/drivers/common/mlx5/rte_common_mlx5_exports.def @@ -37,6 +37,8 @@ EXPORTS mlx5_devx_cq_create mlx5_devx_cq_destroy + mlx5_devx_rq_create + mlx5_devx_rq_destroy mlx5_devx_sq_create mlx5_devx_sq_destroy diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 3414588..dac9411 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -45,6 +45,8 @@ INTERNAL { mlx5_devx_cq_create; mlx5_devx_cq_destroy; + mlx5_devx_rq_create; + mlx5_devx_rq_destroy; mlx5_devx_sq_create; mlx5_devx_sq_destroy; From patchwork Wed Jan 6 08:19:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86043 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C2246A09FF; Wed, 6 Jan 2021 09:23:02 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BF776160913; Wed, 6 Jan 2021 09:21:44 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 31B4A160922 for ; Wed, 6 Jan 2021 09:21:42 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:36 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgl009291; Wed, 6 Jan 2021 10:21:36 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:40 +0000 Message-Id: <1609921181-5019-19-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 18/19] net/mlx5: move Rx RQ creation to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using common function for Rx RQ creation. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.h | 4 +- drivers/net/mlx5/mlx5_devx.c | 177 +++++++++---------------------------------- drivers/net/mlx5/mlx5_rxtx.h | 4 - 3 files changed, 37 insertions(+), 148 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index a3fd8d5..3836a96 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -776,8 +776,9 @@ struct mlx5_rxq_obj { void *ibv_cq; /* Completion Queue. */ void *ibv_channel; }; + struct mlx5_devx_obj *rq; /* DevX RQ object for hairpin. */ struct { - struct mlx5_devx_obj *rq; /* DevX Rx Queue object. */ + struct mlx5_devx_rq rq_obj; /* DevX RQ object. */ struct mlx5_devx_cq cq_obj; /* DevX CQ object. */ void *devx_channel; }; @@ -954,7 +955,6 @@ struct mlx5_priv { /* Context for Verbs allocator. */ int nl_socket_rdma; /* Netlink socket (NETLINK_RDMA). */ int nl_socket_route; /* Netlink socket (NETLINK_ROUTE). */ - struct mlx5_dbr_page_list dbrpgs; /* Door-bell pages. */ struct mlx5_nl_vlan_vmwa_context *vmwa_context; /* VLAN WA context. */ struct mlx5_hlist *mreg_cp_tbl; /* Hash table of Rx metadata register copy table. */ diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 96c44d4..935cbd0 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -45,7 +45,7 @@ rq_attr.state = MLX5_RQC_STATE_RDY; rq_attr.vsd = (on ? 0 : 1); rq_attr.modify_bitmask = MLX5_MODIFY_RQ_IN_MODIFY_BITMASK_VSD; - return mlx5_devx_cmd_modify_rq(rxq_obj->rq, &rq_attr); + return mlx5_devx_cmd_modify_rq(rxq_obj->rq_obj.rq, &rq_attr); } /** @@ -85,7 +85,7 @@ default: break; } - return mlx5_devx_cmd_modify_rq(rxq_obj->rq, &rq_attr); + return mlx5_devx_cmd_modify_rq(rxq_obj->rq_obj.rq, &rq_attr); } /** @@ -145,44 +145,18 @@ } /** - * Release the resources allocated for an RQ DevX object. - * - * @param rxq_ctrl - * DevX Rx queue object. - */ -static void -mlx5_rxq_release_devx_rq_resources(struct mlx5_rxq_ctrl *rxq_ctrl) -{ - struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->rq_dbrec_page; - - if (rxq_ctrl->wq_umem) { - mlx5_os_umem_dereg(rxq_ctrl->wq_umem); - rxq_ctrl->wq_umem = NULL; - } - if (rxq_ctrl->rxq.wqes) { - mlx5_free((void *)(uintptr_t)rxq_ctrl->rxq.wqes); - rxq_ctrl->rxq.wqes = NULL; - } - if (dbr_page) { - claim_zero(mlx5_release_dbr(&rxq_ctrl->priv->dbrpgs, - mlx5_os_get_umem_id(dbr_page->umem), - rxq_ctrl->rq_dbr_offset)); - rxq_ctrl->rq_dbrec_page = NULL; - } -} - -/** * Destroy the Rx queue DevX object. * * @param rxq_obj * Rxq object to destroy. */ static void -mlx5_rxq_release_devx_resources(struct mlx5_rxq_ctrl *rxq_ctrl) +mlx5_rxq_release_devx_resources(struct mlx5_rxq_obj *rxq_obj) { - mlx5_rxq_release_devx_rq_resources(rxq_ctrl); - mlx5_devx_cq_destroy(&rxq_ctrl->obj->cq_obj); - memset(&rxq_ctrl->obj->cq_obj, 0, sizeof(rxq_ctrl->obj->cq_obj)); + mlx5_devx_rq_destroy(&rxq_obj->rq_obj); + memset(&rxq_obj->rq_obj, 0, sizeof(rxq_obj->rq_obj)); + mlx5_devx_cq_destroy(&rxq_obj->cq_obj); + memset(&rxq_obj->cq_obj, 0, sizeof(rxq_obj->cq_obj)); } /** @@ -195,17 +169,17 @@ mlx5_rxq_devx_obj_release(struct mlx5_rxq_obj *rxq_obj) { MLX5_ASSERT(rxq_obj); - MLX5_ASSERT(rxq_obj->rq); if (rxq_obj->rxq_ctrl->type == MLX5_RXQ_TYPE_HAIRPIN) { + MLX5_ASSERT(rxq_obj->rq); mlx5_devx_modify_rq(rxq_obj, MLX5_RXQ_MOD_RDY2RST); claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq)); } else { - MLX5_ASSERT(rxq_obj->cq_obj); - claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq)); + MLX5_ASSERT(rxq_obj->cq_obj.cq); + MLX5_ASSERT(rxq_obj->rq_obj.rq); + mlx5_rxq_release_devx_resources(rxq_obj); if (rxq_obj->devx_channel) mlx5_os_devx_destroy_event_channel (rxq_obj->devx_channel); - mlx5_rxq_release_devx_resources(rxq_obj->rxq_ctrl); } } @@ -247,52 +221,6 @@ } /** - * Fill common fields of create RQ attributes structure. - * - * @param rxq_data - * Pointer to Rx queue data. - * @param cqn - * CQ number to use with this RQ. - * @param rq_attr - * RQ attributes structure to fill.. - */ -static void -mlx5_devx_create_rq_attr_fill(struct mlx5_rxq_data *rxq_data, uint32_t cqn, - struct mlx5_devx_create_rq_attr *rq_attr) -{ - rq_attr->state = MLX5_RQC_STATE_RST; - rq_attr->vsd = (rxq_data->vlan_strip) ? 0 : 1; - rq_attr->cqn = cqn; - rq_attr->scatter_fcs = (rxq_data->crc_present) ? 1 : 0; -} - -/** - * Fill common fields of DevX WQ attributes structure. - * - * @param priv - * Pointer to device private data. - * @param rxq_ctrl - * Pointer to Rx queue control structure. - * @param wq_attr - * WQ attributes structure to fill.. - */ -static void -mlx5_devx_wq_attr_fill(struct mlx5_priv *priv, struct mlx5_rxq_ctrl *rxq_ctrl, - struct mlx5_devx_wq_attr *wq_attr) -{ - wq_attr->end_padding_mode = priv->config.hw_padding ? - MLX5_WQ_END_PAD_MODE_ALIGN : - MLX5_WQ_END_PAD_MODE_NONE; - wq_attr->pd = priv->sh->pdn; - wq_attr->dbr_addr = rxq_ctrl->rq_dbr_offset; - wq_attr->dbr_umem_id = - mlx5_os_get_umem_id(rxq_ctrl->rq_dbrec_page->umem); - wq_attr->dbr_umem_valid = 1; - wq_attr->wq_umem_id = mlx5_os_get_umem_id(rxq_ctrl->wq_umem); - wq_attr->wq_umem_valid = 1; -} - -/** * Create a RQ object using DevX. * * @param dev @@ -301,9 +229,9 @@ * Queue index in DPDK Rx queue array. * * @return - * The DevX RQ object initialized, NULL otherwise and rte_errno is set. + * 0 on success, a negative errno value otherwise and rte_errno is set. */ -static struct mlx5_devx_obj * +static int mlx5_rxq_create_devx_rq_resources(struct rte_eth_dev *dev, uint16_t idx) { struct mlx5_priv *priv = dev->data->dev_private; @@ -311,20 +239,15 @@ struct mlx5_rxq_ctrl *rxq_ctrl = container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); struct mlx5_devx_create_rq_attr rq_attr = { 0 }; - uint32_t wqe_n = 1 << (rxq_data->elts_n - rxq_data->sges_n); - uint32_t cqn = rxq_ctrl->obj->cq_obj.cq->id; - struct mlx5_devx_dbr_page *dbr_page; - int64_t dbr_offset; - uint32_t wq_size = 0; - uint32_t wqe_size = 0; - uint32_t log_wqe_size = 0; - void *buf = NULL; - struct mlx5_devx_obj *rq; + uint16_t log_desc_n = rxq_data->elts_n - rxq_data->sges_n; + uint32_t wqe_size, log_wqe_size; /* Fill RQ attributes. */ rq_attr.mem_rq_type = MLX5_RQC_MEM_RQ_TYPE_MEMORY_RQ_INLINE; rq_attr.flush_in_error_en = 1; - mlx5_devx_create_rq_attr_fill(rxq_data, cqn, &rq_attr); + rq_attr.vsd = (rxq_data->vlan_strip) ? 0 : 1; + rq_attr.cqn = rxq_ctrl->obj->cq_obj.cq->id; + rq_attr.scatter_fcs = (rxq_data->crc_present) ? 1 : 0; /* Fill WQ attributes for this RQ. */ if (mlx5_rxq_mprq_enabled(rxq_data)) { rq_attr.wq_attr.wq_type = MLX5_WQ_TYPE_CYCLIC_STRIDING_RQ; @@ -345,45 +268,17 @@ wqe_size = sizeof(struct mlx5_wqe_data_seg); } log_wqe_size = log2above(wqe_size) + rxq_data->sges_n; - rq_attr.wq_attr.log_wq_stride = log_wqe_size; - rq_attr.wq_attr.log_wq_sz = rxq_data->elts_n - rxq_data->sges_n; - /* Calculate and allocate WQ memory space. */ wqe_size = 1 << log_wqe_size; /* round up power of two.*/ - wq_size = wqe_n * wqe_size; - size_t alignment = MLX5_WQE_BUF_ALIGNMENT; - if (alignment == (size_t)-1) { - DRV_LOG(ERR, "Failed to get mem page size"); - rte_errno = ENOMEM; - return NULL; - } - buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, wq_size, - alignment, rxq_ctrl->socket); - if (!buf) - return NULL; - rxq_data->wqes = buf; - rxq_ctrl->wq_umem = mlx5_os_umem_reg(priv->sh->ctx, - buf, wq_size, 0); - if (!rxq_ctrl->wq_umem) - goto error; - /* Allocate RQ door-bell. */ - dbr_offset = mlx5_get_dbr(priv->sh->ctx, &priv->dbrpgs, &dbr_page); - if (dbr_offset < 0) { - DRV_LOG(ERR, "Failed to allocate RQ door-bell."); - goto error; - } - rxq_ctrl->rq_dbr_offset = dbr_offset; - rxq_ctrl->rq_dbrec_page = dbr_page; - rxq_data->rq_db = (uint32_t *)((uintptr_t)dbr_page->dbrs + - (uintptr_t)rxq_ctrl->rq_dbr_offset); + rq_attr.wq_attr.log_wq_stride = log_wqe_size; + rq_attr.wq_attr.log_wq_sz = log_desc_n; + rq_attr.wq_attr.end_padding_mode = priv->config.hw_padding ? + MLX5_WQ_END_PAD_MODE_ALIGN : + MLX5_WQ_END_PAD_MODE_NONE; + rq_attr.wq_attr.pd = priv->sh->pdn; /* Create RQ using DevX API. */ - mlx5_devx_wq_attr_fill(priv, rxq_ctrl, &rq_attr.wq_attr); - rq = mlx5_devx_cmd_create_rq(priv->sh->ctx, &rq_attr, rxq_ctrl->socket); - if (!rq) - goto error; - return rq; -error: - mlx5_rxq_release_devx_rq_resources(rxq_ctrl); - return NULL; + return mlx5_devx_rq_create(priv->sh->ctx, &rxq_ctrl->obj->rq_obj, + wqe_size, log_desc_n, &rq_attr, + rxq_ctrl->socket); } /** @@ -604,8 +499,8 @@ goto error; } /* Create RQ using DevX API. */ - tmpl->rq = mlx5_rxq_create_devx_rq_resources(dev, idx); - if (!tmpl->rq) { + ret = mlx5_rxq_create_devx_rq_resources(dev, idx); + if (ret) { DRV_LOG(ERR, "Port %u Rx queue %u RQ creation failure.", dev->data->port_id, idx); rte_errno = ENOMEM; @@ -615,19 +510,17 @@ ret = mlx5_devx_modify_rq(tmpl, MLX5_RXQ_MOD_RST2RDY); if (ret) goto error; + rxq_data->wqes = (void *)(uintptr_t)tmpl->rq_obj.umem_buf; + rxq_data->rq_db = (uint32_t *)(uintptr_t)tmpl->rq_obj.db_rec; rxq_data->cq_arm_sn = 0; - mlx5_rxq_initialize(rxq_data); rxq_data->cq_ci = 0; + mlx5_rxq_initialize(rxq_data); dev->data->rx_queue_state[idx] = RTE_ETH_QUEUE_STATE_STARTED; - rxq_ctrl->wqn = tmpl->rq->id; + rxq_ctrl->wqn = tmpl->rq_obj.rq->id; return 0; error: ret = rte_errno; /* Save rte_errno before cleanup. */ - if (tmpl->rq) - claim_zero(mlx5_devx_cmd_destroy(tmpl->rq)); - if (tmpl->devx_channel) - mlx5_os_devx_destroy_event_channel(tmpl->devx_channel); - mlx5_rxq_release_devx_resources(rxq_ctrl); + mlx5_rxq_devx_obj_release(tmpl); rte_errno = ret; /* Restore rte_errno. */ return -rte_errno; } @@ -671,7 +564,7 @@ struct mlx5_rxq_ctrl *rxq_ctrl = container_of(rxq, struct mlx5_rxq_ctrl, rxq); - rqt_attr->rq_list[i] = rxq_ctrl->obj->rq->id; + rqt_attr->rq_list[i] = rxq_ctrl->obj->rq_obj.rq->id; } MLX5_ASSERT(i > 0); for (j = 0; i != rqt_n; ++j, ++i) diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index aba9541..7756ed3 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -193,10 +193,6 @@ struct mlx5_rxq_ctrl { uint32_t flow_tunnels_n[MLX5_FLOW_TUNNEL]; /* Tunnels counters. */ uint32_t wqn; /* WQ number. */ uint16_t dump_file_n; /* Number of dump files. */ - struct mlx5_devx_dbr_page *rq_dbrec_page; - uint64_t rq_dbr_offset; - /* Storing RQ door-bell information, needed when freeing door-bell. */ - void *wq_umem; /* WQ buffer registration info. */ struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. */ uint32_t hairpin_status; /* Hairpin binding status. */ }; From patchwork Wed Jan 6 08:19:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 86042 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 60A07A09FF; Wed, 6 Jan 2021 09:22:55 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9B42C160922; Wed, 6 Jan 2021 09:21:43 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 31991160921 for ; Wed, 6 Jan 2021 09:21:42 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from michaelba@nvidia.com) with SMTP; 6 Jan 2021 10:21:38 +0200 Received: from nvidia.com (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 1068KAgm009291; Wed, 6 Jan 2021 10:21:38 +0200 From: Michael Baum To: dev@dpdk.org Cc: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Wed, 6 Jan 2021 08:19:41 +0000 Message-Id: <1609921181-5019-20-git-send-email-michaelba@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1609921181-5019-1-git-send-email-michaelba@nvidia.com> References: <1609231944-29274-2-git-send-email-michaelba@nvidia.com> <1609921181-5019-1-git-send-email-michaelba@nvidia.com> Subject: [dpdk-dev] [PATCH v3 19/19] common/mlx5: remove doorbell allocation API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The mlx5_devx_dbr_page structure was used to allocate and release the umem of the doorbells. Since doorbell and buffer have used same umem, this structure is useless. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common.c | 122 ------------------------ drivers/common/mlx5/mlx5_common.h | 23 ----- drivers/common/mlx5/rte_common_mlx5_exports.def | 3 - drivers/common/mlx5/version.map | 3 - 4 files changed, 151 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index a00ffcb..c26a2cf 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -8,12 +8,10 @@ #include #include -#include #include "mlx5_common.h" #include "mlx5_common_os.h" #include "mlx5_common_utils.h" -#include "mlx5_malloc.h" #include "mlx5_common_pci.h" int mlx5_common_logtype; @@ -126,126 +124,6 @@ static inline void mlx5_cpu_id(unsigned int level, } /** - * Allocate page of door-bells and register it using DevX API. - * - * @param [in] ctx - * Pointer to the device context. - * - * @return - * Pointer to new page on success, NULL otherwise. - */ -static struct mlx5_devx_dbr_page * -mlx5_alloc_dbr_page(void *ctx) -{ - struct mlx5_devx_dbr_page *page; - - /* Allocate space for door-bell page and management data. */ - page = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, - sizeof(struct mlx5_devx_dbr_page), - RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); - if (!page) { - DRV_LOG(ERR, "cannot allocate dbr page"); - return NULL; - } - /* Register allocated memory. */ - page->umem = mlx5_os_umem_reg(ctx, page->dbrs, - MLX5_DBR_PAGE_SIZE, 0); - if (!page->umem) { - DRV_LOG(ERR, "cannot umem reg dbr page"); - mlx5_free(page); - return NULL; - } - return page; -} - -/** - * Find the next available door-bell, allocate new page if needed. - * - * @param [in] ctx - * Pointer to device context. - * @param [in] head - * Pointer to the head of dbr pages list. - * @param [out] dbr_page - * Door-bell page containing the page data. - * - * @return - * Door-bell address offset on success, a negative error value otherwise. - */ -int64_t -mlx5_get_dbr(void *ctx, struct mlx5_dbr_page_list *head, - struct mlx5_devx_dbr_page **dbr_page) -{ - struct mlx5_devx_dbr_page *page = NULL; - uint32_t i, j; - - LIST_FOREACH(page, head, next) - if (page->dbr_count < MLX5_DBR_PER_PAGE) - break; - if (!page) { /* No page with free door-bell exists. */ - page = mlx5_alloc_dbr_page(ctx); - if (!page) /* Failed to allocate new page. */ - return (-1); - LIST_INSERT_HEAD(head, page, next); - } - /* Loop to find bitmap part with clear bit. */ - for (i = 0; - i < MLX5_DBR_BITMAP_SIZE && page->dbr_bitmap[i] == UINT64_MAX; - i++) - ; /* Empty. */ - /* Find the first clear bit. */ - MLX5_ASSERT(i < MLX5_DBR_BITMAP_SIZE); - j = rte_bsf64(~page->dbr_bitmap[i]); - page->dbr_bitmap[i] |= (UINT64_C(1) << j); - page->dbr_count++; - *dbr_page = page; - return (i * CHAR_BIT * sizeof(uint64_t) + j) * MLX5_DBR_SIZE; -} - -/** - * Release a door-bell record. - * - * @param [in] head - * Pointer to the head of dbr pages list. - * @param [in] umem_id - * UMEM ID of page containing the door-bell record to release. - * @param [in] offset - * Offset of door-bell record in page. - * - * @return - * 0 on success, a negative error value otherwise. - */ -int32_t -mlx5_release_dbr(struct mlx5_dbr_page_list *head, uint32_t umem_id, - uint64_t offset) -{ - struct mlx5_devx_dbr_page *page = NULL; - int ret = 0; - - LIST_FOREACH(page, head, next) - /* Find the page this address belongs to. */ - if (mlx5_os_get_umem_id(page->umem) == umem_id) - break; - if (!page) - return -EINVAL; - page->dbr_count--; - if (!page->dbr_count) { - /* Page not used, free it and remove from list. */ - LIST_REMOVE(page, next); - if (page->umem) - ret = -mlx5_os_umem_dereg(page->umem); - mlx5_free(page); - } else { - /* Mark in bitmap that this door-bell is not in use. */ - offset /= MLX5_DBR_SIZE; - int i = offset / 64; - int j = offset % 64; - - page->dbr_bitmap[i] &= ~(UINT64_C(1) << j); - } - return ret; -} - -/** * Allocate the User Access Region with DevX on specified device. * * @param [in] ctx diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index a484b74..e35188d 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -220,21 +220,6 @@ enum mlx5_class { }; #define MLX5_DBR_SIZE RTE_CACHE_LINE_SIZE -#define MLX5_DBR_PER_PAGE 64 -/* Must be >= CHAR_BIT * sizeof(uint64_t) */ -#define MLX5_DBR_PAGE_SIZE (MLX5_DBR_PER_PAGE * MLX5_DBR_SIZE) -/* Page size must be >= 512. */ -#define MLX5_DBR_BITMAP_SIZE (MLX5_DBR_PER_PAGE / (CHAR_BIT * sizeof(uint64_t))) - -struct mlx5_devx_dbr_page { - /* Door-bell records, must be first member in structure. */ - uint8_t dbrs[MLX5_DBR_PAGE_SIZE]; - LIST_ENTRY(mlx5_devx_dbr_page) next; /* Pointer to the next element. */ - void *umem; - uint32_t dbr_count; /* Number of door-bell records in use. */ - /* 1 bit marks matching door-bell is in use. */ - uint64_t dbr_bitmap[MLX5_DBR_BITMAP_SIZE]; -}; /* devX creation object */ struct mlx5_devx_obj { @@ -249,19 +234,11 @@ struct mlx5_klm { uint64_t address; }; -LIST_HEAD(mlx5_dbr_page_list, mlx5_devx_dbr_page); - __rte_internal void mlx5_translate_port_name(const char *port_name_in, struct mlx5_switch_info *port_info_out); void mlx5_glue_constructor(void); __rte_internal -int64_t mlx5_get_dbr(void *ctx, struct mlx5_dbr_page_list *head, - struct mlx5_devx_dbr_page **dbr_page); -__rte_internal -int32_t mlx5_release_dbr(struct mlx5_dbr_page_list *head, uint32_t umem_id, - uint64_t offset); -__rte_internal void *mlx5_devx_alloc_uar(void *ctx, int mapping); extern uint8_t haswell_broadwell_cpu; diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def index 6e1ff50..93bf4c4 100644 --- a/drivers/common/mlx5/rte_common_mlx5_exports.def +++ b/drivers/common/mlx5/rte_common_mlx5_exports.def @@ -42,7 +42,6 @@ EXPORTS mlx5_devx_sq_create mlx5_devx_sq_destroy - mlx5_get_dbr mlx5_glue mlx5_malloc_mem_select @@ -63,8 +62,6 @@ EXPORTS mlx5_pci_driver_register - mlx5_release_dbr - mlx5_malloc mlx5_realloc mlx5_free diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index dac9411..c666fdf 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -51,7 +51,6 @@ INTERNAL { mlx5_devx_sq_destroy; mlx5_get_ifname_sysfs; - mlx5_get_dbr; mlx5_mp_init_primary; mlx5_mp_uninit_primary; @@ -93,8 +92,6 @@ INTERNAL { mlx5_nl_vlan_vmwa_create; mlx5_nl_vlan_vmwa_delete; - mlx5_release_dbr; - mlx5_translate_port_name; mlx5_malloc_mem_select;