From patchwork Wed Jun 23 04:46:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 94711 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EE44DA0C41; Wed, 23 Jun 2021 06:49:07 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5AB5141102; Wed, 23 Jun 2021 06:48:10 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id D414A41101 for ; Wed, 23 Jun 2021 06:48:08 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 15N4jY9T026990 for ; Tue, 22 Jun 2021 21:48:08 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=CvuSxHSB91D6hg/xVgFenK6FmhB0XfBYDiJg0+9paLw=; b=C2h5sa3wi4Obkuv1sTFN7X1NswSSfx2+mM4UwKMBRKso84JBhVMwE64xE8wS0+1sIG3i zr1HoiFGRz9I5HZfI3encAWsNv9LFW706Ls2eP+Utne+OolCuXfnd4HJGW1ujDJ+X7IO OFZycyEkMXt3l4Znusvxa8rKtTcQFJ3YP+UpOyxLMYnmpQmeS1pceUM1TO6oz8lhnu0k FR/ms+GtjCJJTG8IOVVDuo5RZJZOjOgqxUBbg5rzZrgrwHLj73pTSNqS9H+1LOviG1BS rQPWnZ9Amb71xIHGB2FPrre3/F6E0A8JeYqe3/inTaAd5CSanVRbgwkqJBtx5peeauSS Jg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 39bx5j80pk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 22 Jun 2021 21:48:07 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 22 Jun 2021 21:48:06 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 22 Jun 2021 21:48:06 -0700 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 864015B693E; Tue, 22 Jun 2021 21:48:03 -0700 (PDT) From: Nithin Dabilpuram To: CC: , , , , , , , , "Nithin Dabilpuram" Date: Wed, 23 Jun 2021 10:16:15 +0530 Message-ID: <20210623044702.4240-16-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210623044702.4240-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> <20210623044702.4240-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: VbTZxOgA79mcwVOVXQZmcgnCFzgUQ_n_ X-Proofpoint-GUID: VbTZxOgA79mcwVOVXQZmcgnCFzgUQ_n_ X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-06-23_01:2021-06-22, 2021-06-23 signatures=0 Subject: [dpdk-dev] [PATCH v4 15/62] net/cnxk: add Rx queue setup and release X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Rx queue setup and release op for CN9K and CN10K SoC. Release is completely common while setup is platform dependent due to fast path Rx queue structure variation. Fastpath is platform dependent partly due to core cacheline size difference. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn10k_ethdev.h | 14 +++ drivers/net/cnxk/cn9k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn9k_ethdev.h | 14 +++ drivers/net/cnxk/cnxk_ethdev.c | 172 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ 9 files changed, 300 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index affbbd9..a9d2b03 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 836cc9f..6a8ca1f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 29bb24f..f761638 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -9,6 +9,7 @@ Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index d971bbd..b87c4e5 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,49 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -33,6 +76,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 1bf4a65..08e11bb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,4 +6,18 @@ #include +struct cn10k_eth_rxq { + uint64_t mbuf_initializer; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t data_off; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 2fb7c14..2ab035e 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -44,6 +87,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 15d9397..6384609 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,4 +6,18 @@ #include +struct cn9k_eth_rxq { + uint64_t mbuf_initializer; + uint64_t data_off; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index ea49809..2775fe4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,177 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint64_t +cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) +{ + uint16_t port_id = dev->eth_dev->data->port_id; + struct rte_mbuf mb_def; + uint64_t *tmp; + + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) % 8 != 0); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) - + offsetof(struct rte_mbuf, data_off) != + 2); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, nb_segs) - + offsetof(struct rte_mbuf, data_off) != + 4); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - + offsetof(struct rte_mbuf, data_off) != + 6); + mb_def.nb_segs = 1; + mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.port = port_id; + rte_mbuf_refcnt_set(&mb_def, 1); + + /* Prevent compiler reordering: rearm_data covers previous fields */ + rte_compiler_barrier(); + tmp = (uint64_t *)&mb_def.rearm_data; + + return *tmp; +} + +int +cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_mempool_ops *ops; + const char *platform_ops; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t first_skip; + int rc = -EINVAL; + size_t rxq_sz; + + /* Sanity checks */ + if (rx_conf->rx_deferred_start == 1) { + plt_err("Deferred Rx start is not supported"); + goto fail; + } + + platform_ops = rte_mbuf_platform_mempool_ops(); + /* This driver needs cnxk_npa mempool ops to work */ + ops = rte_mempool_get_ops(mp->ops_index); + if (strncmp(ops->name, platform_ops, RTE_MEMPOOL_OPS_NAMESIZE)) { + plt_err("mempool ops should be of cnxk_npa type"); + goto fail; + } + + if (mp->pool_id == 0) { + plt_err("Invalid pool_id"); + goto fail; + } + + /* Free memory prior to re-allocation if needed */ + if (eth_dev->data->rx_queues[qid] != NULL) { + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->rx_queue_release(eth_dev->data->rx_queues[qid]); + eth_dev->data->rx_queues[qid] = NULL; + } + + /* Setup ROC CQ */ + cq = &dev->cqs[qid]; + cq->qid = qid; + cq->nb_desc = nb_desc; + rc = roc_nix_cq_init(&dev->nix, cq); + if (rc) { + plt_err("Failed to init roc cq for rq=%d, rc=%d", qid, rc); + goto fail; + } + + /* Setup ROC RQ */ + rq = &dev->rqs[qid]; + rq->qid = qid; + rq->aura_handle = mp->pool_id; + rq->flow_tag_width = 32; + rq->sso_ena = false; + + /* Calculate first mbuf skip */ + first_skip = (sizeof(struct rte_mbuf)); + first_skip += RTE_PKTMBUF_HEADROOM; + first_skip += rte_pktmbuf_priv_size(mp); + rq->first_skip = first_skip; + rq->later_skip = sizeof(struct rte_mbuf); + rq->lpb_size = mp->elt_size; + + rc = roc_nix_rq_init(&dev->nix, rq, !!eth_dev->data->dev_started); + if (rc) { + plt_err("Failed to init roc rq for rq=%d, rc=%d", qid, rc); + goto cq_fini; + } + + /* Allocate and setup fast path rx queue */ + rc = -ENOMEM; + rxq_sz = sizeof(struct cnxk_eth_rxq_sp) + fp_rx_q_sz; + rxq_sp = plt_zmalloc(rxq_sz, PLT_CACHE_LINE_SIZE); + if (!rxq_sp) { + plt_err("Failed to alloc rx queue for rq=%d", qid); + goto rq_fini; + } + + /* Setup slow path fields */ + rxq_sp->dev = dev; + rxq_sp->qid = qid; + rxq_sp->qconf.conf.rx = *rx_conf; + rxq_sp->qconf.nb_desc = nb_desc; + rxq_sp->qconf.mp = mp; + + plt_nix_dbg("rq=%d pool=%s nb_desc=%d->%d", qid, mp->name, nb_desc, + cq->nb_desc); + + /* Store start of fast path area */ + eth_dev->data->rx_queues[qid] = rxq_sp + 1; + eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +rq_fini: + rc |= roc_nix_rq_fini(rq); +cq_fini: + rc |= roc_nix_cq_fini(cq); +fail: + return rc; +} + +static void +cnxk_nix_rx_queue_release(void *rxq) +{ + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t qid; + int rc; + + if (!rxq) + return; + + rxq_sp = cnxk_eth_rxq_to_sp(rxq); + dev = rxq_sp->dev; + qid = rxq_sp->qid; + + plt_nix_dbg("Releasing rxq %u", qid); + + /* Cleanup ROC RQ */ + rq = &dev->rqs[qid]; + rc = roc_nix_rq_fini(rq); + if (rc) + plt_err("Failed to cleanup rq, rc=%d", rc); + + /* Cleanup ROC CQ */ + cq = &dev->cqs[qid]; + rc = roc_nix_cq_fini(cq); + if (rc) + plt_err("Failed to cleanup cq, rc=%d", rc); + + /* Finally free fast path area */ + plt_free(rxq_sp); +} + uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level) @@ -602,6 +773,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .rx_queue_release = cnxk_nix_rx_queue_release, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index daa87af..4a7c2ca 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -10,6 +10,9 @@ #include #include #include +#include +#include +#include #include "roc_api.h" @@ -194,6 +197,12 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); + +uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss,