From patchwork Fri Jun 18 10:36:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 94424 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6DE71A0C46; Fri, 18 Jun 2021 12:41:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 79FC54116C; Fri, 18 Jun 2021 12:40:00 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 1548A41109 for ; Fri, 18 Jun 2021 12:39:58 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 15IAZ70l032210 for ; Fri, 18 Jun 2021 03:39:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=Ia97ei4W/BWF27NYztdn93aqyW6rRTJJlVA8HwUCmto=; b=aSPSPUcformiwOUS6rCS5ybksdoOUb5wSNtR9qofhOB5BAqTO1vPAjiiH5u6SJQ787sV oLknTuDt5rpxsi2bDWGrcdpGSnpmd8UiHll9qifMaOxmQqoJ5gHLWJA4cieJIXW5JiJ+ BIPHKShTdtR/qsXDlPKvKF1dZ4BI76d8tk9WbX7Peq+qedP+xbfT/l/iBg3XwVDfUZNr UCgwOEDPMxrPIfMq7ixFNTtkfhdV5fGeWCHMWC+IAG3zHddl5e/XSX5BNmC2AIgqg8dO Bsrj7U8kjSLmQBpdjnLjzbU7E3MKDbXVNt909yfrhxZa39ChU6kU29/0NS4fKRkpnjt1 mg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 398r750d0m-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 18 Jun 2021 03:39:58 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 18 Jun 2021 03:39:56 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 18 Jun 2021 03:39:56 -0700 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 6470C5B6AA8; Fri, 18 Jun 2021 03:39:16 -0700 (PDT) From: Nithin Dabilpuram To: CC: , , , , , , , , "Nithin Dabilpuram" Date: Fri, 18 Jun 2021 16:06:54 +0530 Message-ID: <20210618103741.26526-16-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210618103741.26526-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> <20210618103741.26526-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: Z-BL5cFfic4lNwfblV-MRYsNB3QnRVs7 X-Proofpoint-GUID: Z-BL5cFfic4lNwfblV-MRYsNB3QnRVs7 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-06-18_04:2021-06-18, 2021-06-18 signatures=0 Subject: [dpdk-dev] [PATCH v3 15/62] net/cnxk: add Rx queue setup and release X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Rx queue setup and release op for CN9K and CN10K SoC. Release is completely common while setup is platform dependent due to fast path Rx queue structure variation. Fastpath is platform dependent partly due to core cacheline size difference. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn10k_ethdev.h | 14 +++ drivers/net/cnxk/cn9k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn9k_ethdev.h | 14 +++ drivers/net/cnxk/cnxk_ethdev.c | 172 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ 9 files changed, 300 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index affbbd9..a9d2b03 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 836cc9f..6a8ca1f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 29bb24f..f761638 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -9,6 +9,7 @@ Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index d971bbd..b87c4e5 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,49 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -33,6 +76,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 1bf4a65..08e11bb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,4 +6,18 @@ #include +struct cn10k_eth_rxq { + uint64_t mbuf_initializer; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t data_off; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 7f3e910..ed9a813 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -44,6 +87,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 15d9397..6384609 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,4 +6,18 @@ #include +struct cn9k_eth_rxq { + uint64_t mbuf_initializer; + uint64_t data_off; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 002afdc..10e3ead 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,177 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint64_t +cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) +{ + uint16_t port_id = dev->eth_dev->data->port_id; + struct rte_mbuf mb_def; + uint64_t *tmp; + + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) % 8 != 0); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) - + offsetof(struct rte_mbuf, data_off) != + 2); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, nb_segs) - + offsetof(struct rte_mbuf, data_off) != + 4); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - + offsetof(struct rte_mbuf, data_off) != + 6); + mb_def.nb_segs = 1; + mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.port = port_id; + rte_mbuf_refcnt_set(&mb_def, 1); + + /* Prevent compiler reordering: rearm_data covers previous fields */ + rte_compiler_barrier(); + tmp = (uint64_t *)&mb_def.rearm_data; + + return *tmp; +} + +int +cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_mempool_ops *ops; + const char *platform_ops; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t first_skip; + int rc = -EINVAL; + size_t rxq_sz; + + /* Sanity checks */ + if (rx_conf->rx_deferred_start == 1) { + plt_err("Deferred Rx start is not supported"); + goto fail; + } + + platform_ops = rte_mbuf_platform_mempool_ops(); + /* This driver needs cnxk_npa mempool ops to work */ + ops = rte_mempool_get_ops(mp->ops_index); + if (strncmp(ops->name, platform_ops, RTE_MEMPOOL_OPS_NAMESIZE)) { + plt_err("mempool ops should be of cnxk_npa type"); + goto fail; + } + + if (mp->pool_id == 0) { + plt_err("Invalid pool_id"); + goto fail; + } + + /* Free memory prior to re-allocation if needed */ + if (eth_dev->data->rx_queues[qid] != NULL) { + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->rx_queue_release(eth_dev->data->rx_queues[qid]); + eth_dev->data->rx_queues[qid] = NULL; + } + + /* Setup ROC CQ */ + cq = &dev->cqs[qid]; + cq->qid = qid; + cq->nb_desc = nb_desc; + rc = roc_nix_cq_init(&dev->nix, cq); + if (rc) { + plt_err("Failed to init roc cq for rq=%d, rc=%d", qid, rc); + goto fail; + } + + /* Setup ROC RQ */ + rq = &dev->rqs[qid]; + rq->qid = qid; + rq->aura_handle = mp->pool_id; + rq->flow_tag_width = 32; + rq->sso_ena = false; + + /* Calculate first mbuf skip */ + first_skip = (sizeof(struct rte_mbuf)); + first_skip += RTE_PKTMBUF_HEADROOM; + first_skip += rte_pktmbuf_priv_size(mp); + rq->first_skip = first_skip; + rq->later_skip = sizeof(struct rte_mbuf); + rq->lpb_size = mp->elt_size; + + rc = roc_nix_rq_init(&dev->nix, rq, !!eth_dev->data->dev_started); + if (rc) { + plt_err("Failed to init roc rq for rq=%d, rc=%d", qid, rc); + goto cq_fini; + } + + /* Allocate and setup fast path rx queue */ + rc = -ENOMEM; + rxq_sz = sizeof(struct cnxk_eth_rxq_sp) + fp_rx_q_sz; + rxq_sp = plt_zmalloc(rxq_sz, PLT_CACHE_LINE_SIZE); + if (!rxq_sp) { + plt_err("Failed to alloc rx queue for rq=%d", qid); + goto rq_fini; + } + + /* Setup slow path fields */ + rxq_sp->dev = dev; + rxq_sp->qid = qid; + rxq_sp->qconf.conf.rx = *rx_conf; + rxq_sp->qconf.nb_desc = nb_desc; + rxq_sp->qconf.mp = mp; + + plt_nix_dbg("rq=%d pool=%s nb_desc=%d->%d", qid, mp->name, nb_desc, + cq->nb_desc); + + /* Store start of fast path area */ + eth_dev->data->rx_queues[qid] = rxq_sp + 1; + eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +rq_fini: + rc |= roc_nix_rq_fini(rq); +cq_fini: + rc |= roc_nix_cq_fini(cq); +fail: + return rc; +} + +static void +cnxk_nix_rx_queue_release(void *rxq) +{ + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t qid; + int rc; + + if (!rxq) + return; + + rxq_sp = cnxk_eth_rxq_to_sp(rxq); + dev = rxq_sp->dev; + qid = rxq_sp->qid; + + plt_nix_dbg("Releasing rxq %u", qid); + + /* Cleanup ROC RQ */ + rq = &dev->rqs[qid]; + rc = roc_nix_rq_fini(rq); + if (rc) + plt_err("Failed to cleanup rq, rc=%d", rc); + + /* Cleanup ROC CQ */ + cq = &dev->cqs[qid]; + rc = roc_nix_cq_fini(cq); + if (rc) + plt_err("Failed to cleanup cq, rc=%d", rc); + + /* Finally free fast path area */ + plt_free(rxq_sp); +} + uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level) @@ -590,6 +761,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .rx_queue_release = cnxk_nix_rx_queue_release, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index daa87af..4a7c2ca 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -10,6 +10,9 @@ #include #include #include +#include +#include +#include #include "roc_api.h" @@ -194,6 +197,12 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); + +uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss,