From patchwork Tue Mar 31 13:58:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 67489 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 90CABA0562; Tue, 31 Mar 2020 15:59:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6D8EB2C15; Tue, 31 Mar 2020 15:59:09 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id F38322BCE for ; Tue, 31 Mar 2020 15:59:07 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02VDtPQ0013975; Tue, 31 Mar 2020 06:59:06 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=P9VH89XtU76sPElNRYpoxlIxc3sZfSrNEemFLH4joxA=; b=xsELA8xdz3wscZyf3/czgF1xMTlv16SfDbhcPr0C9NfQ0QjcNS7rU7FfCn/yjyyHqyhQ 1dbOglPfKLRqC85Zm1trKe7FRV6MFFPCOB1Y6wviU0sCxzJUhVqEVVb9yzfTaDtjPsUo P9Nw/TcqbPCA9Bvdp+wixk8MyDAfPGcq8+PzxJEDLWoxaipPjI7L3salG18Iooe38HTI zILMAbU/QbLDR5oCGZBIDmkYcTErHKMNxDDHinwuZ9glwzsED9qATIueud1THiBJ82+c +U5B5d/dINUIWXQvhbMg4cfBIy9LWUiHB+RDrMGCKqBwxMCOIBM2nLfnFTxTJ9eSHjw0 Mg== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3023xp3ccq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 31 Mar 2020 06:59:06 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 31 Mar 2020 06:59:05 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 31 Mar 2020 06:59:04 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 31 Mar 2020 06:59:04 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.163.117]) by maili.marvell.com (Postfix) with ESMTP id 2AC2D3F703F; Tue, 31 Mar 2020 06:59:00 -0700 (PDT) From: To: , , Nithin Dabilpuram , Kiran Kumar K , "John McNamara" , Marko Kovacevic CC: , Pavan Nikhilesh Date: Tue, 31 Mar 2020 19:28:55 +0530 Message-ID: <20200331135856.4924-1-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200327095359.1934-1-pbhagavatula@marvell.com> References: <20200327095359.1934-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676 definitions=2020-03-31_04:2020-03-31, 2020-03-31 signatures=0 Subject: [dpdk-dev] [PATCH v5] net/octeontx2: add devargs to lock Rx/Tx ctx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Add device arguments to lock Rx/Tx contexts. Application can either choose to lock Rx or Tx contexts by using 'lock_rx_ctx' or 'lock_tx_ctx' respectively per each port. Example: -w 0002:02:00.0,lock_rx_ctx=1 -w 0002:03:00.0,lock_tx_ctx=1 Signed-off-by: Pavan Nikhilesh Reviewed-by: Andrzej Ostruszka --- Depends on http://patches.dpdk.org/patch/67178/ v5 Changes: - Remove redundant goto.(Andrzej) v4 Changes: - Fix return path using unnecessary goto.(Andrzej) - Fix datatype of values passed to devargs parser.(Andrzej) v3 Changes: - Split series into individual patches as targets are different. doc/guides/nics/octeontx2.rst | 16 ++ drivers/net/octeontx2/otx2_ethdev.c | 196 +++++++++++++++++++- drivers/net/octeontx2/otx2_ethdev.h | 2 + drivers/net/octeontx2/otx2_ethdev_devargs.c | 16 +- drivers/net/octeontx2/otx2_rss.c | 23 +++ 5 files changed, 244 insertions(+), 9 deletions(-) -- 2.17.1 diff --git a/doc/guides/nics/octeontx2.rst b/doc/guides/nics/octeontx2.rst index c2d87c9d0..df19443e3 100644 --- a/doc/guides/nics/octeontx2.rst +++ b/doc/guides/nics/octeontx2.rst @@ -209,6 +209,22 @@ Runtime Config Options With the above configuration, application can enable inline IPsec processing on 128 SAs (SPI 0-127). +- ``Lock Rx contexts in NDC cache`` + + Lock Rx contexts in NDC cache by using ``lock_rx_ctx`` parameter. + + For example:: + + -w 0002:02:00.0,lock_rx_ctx=1 + +- ``Lock Tx contexts in NDC cache`` + + Lock Tx contexts in NDC cache by using ``lock_tx_ctx`` parameter. + + For example:: + + -w 0002:02:00.0,lock_tx_ctx=1 + .. note:: Above devarg parameters are configurable per device, user needs to pass the diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index e60f4901c..49bf7ef9f 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -286,8 +286,7 @@ nix_cq_rq_init(struct rte_eth_dev *eth_dev, struct otx2_eth_dev *dev, NIX_CQ_ALIGN, dev->node); if (rz == NULL) { otx2_err("Failed to allocate mem for cq hw ring"); - rc = -ENOMEM; - goto fail; + return -ENOMEM; } memset(rz->addr, 0, rz->len); rxq->desc = (uintptr_t)rz->addr; @@ -336,7 +335,7 @@ nix_cq_rq_init(struct rte_eth_dev *eth_dev, struct otx2_eth_dev *dev, rc = otx2_mbox_process(mbox); if (rc) { otx2_err("Failed to init cq context"); - goto fail; + return rc; } aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); @@ -378,12 +377,44 @@ nix_cq_rq_init(struct rte_eth_dev *eth_dev, struct otx2_eth_dev *dev, rc = otx2_mbox_process(mbox); if (rc) { otx2_err("Failed to init rq context"); - goto fail; + return rc; + } + + if (dev->lock_rx_ctx) { + aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + aq->qidx = qid; + aq->ctype = NIX_AQ_CTYPE_CQ; + aq->op = NIX_AQ_INSTOP_LOCK; + + aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + if (!aq) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + otx2_mbox_msg_send(mbox, 0); + rc = otx2_mbox_wait_for_rsp(mbox, 0); + if (rc < 0) { + otx2_err("Failed to LOCK cq context"); + return rc; + } + + aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + if (!aq) { + otx2_err("Failed to LOCK rq context"); + return -ENOMEM; + } + } + aq->qidx = qid; + aq->ctype = NIX_AQ_CTYPE_RQ; + aq->op = NIX_AQ_INSTOP_LOCK; + rc = otx2_mbox_process(mbox); + if (rc < 0) { + otx2_err("Failed to LOCK rq context"); + return rc; + } } return 0; -fail: - return rc; } static int @@ -430,6 +461,40 @@ nix_cq_rq_uninit(struct rte_eth_dev *eth_dev, struct otx2_eth_rxq *rxq) return rc; } + if (dev->lock_rx_ctx) { + aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + aq->qidx = rxq->rq; + aq->ctype = NIX_AQ_CTYPE_CQ; + aq->op = NIX_AQ_INSTOP_UNLOCK; + + aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + if (!aq) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + otx2_mbox_msg_send(mbox, 0); + rc = otx2_mbox_wait_for_rsp(mbox, 0); + if (rc < 0) { + otx2_err("Failed to UNLOCK cq context"); + return rc; + } + + aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + if (!aq) { + otx2_err("Failed to UNLOCK rq context"); + return -ENOMEM; + } + } + aq->qidx = rxq->rq; + aq->ctype = NIX_AQ_CTYPE_RQ; + aq->op = NIX_AQ_INSTOP_UNLOCK; + rc = otx2_mbox_process(mbox); + if (rc < 0) { + otx2_err("Failed to UNLOCK rq context"); + return rc; + } + } + return 0; } @@ -715,6 +780,94 @@ nix_tx_offload_flags(struct rte_eth_dev *eth_dev) return flags; } +static int +nix_sqb_lock(struct rte_mempool *mp) +{ + struct otx2_npa_lf *npa_lf = otx2_intra_dev_get_cfg()->npa_lf; + struct npa_aq_enq_req *req; + int rc; + + req = otx2_mbox_alloc_msg_npa_aq_enq(npa_lf->mbox); + req->aura_id = npa_lf_aura_handle_to_aura(mp->pool_id); + req->ctype = NPA_AQ_CTYPE_AURA; + req->op = NPA_AQ_INSTOP_LOCK; + + req = otx2_mbox_alloc_msg_npa_aq_enq(npa_lf->mbox); + if (!req) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + otx2_mbox_msg_send(npa_lf->mbox, 0); + rc = otx2_mbox_wait_for_rsp(npa_lf->mbox, 0); + if (rc < 0) { + otx2_err("Failed to LOCK AURA context"); + return rc; + } + + req = otx2_mbox_alloc_msg_npa_aq_enq(npa_lf->mbox); + if (!req) { + otx2_err("Failed to LOCK POOL context"); + return -ENOMEM; + } + } + + req->aura_id = npa_lf_aura_handle_to_aura(mp->pool_id); + req->ctype = NPA_AQ_CTYPE_POOL; + req->op = NPA_AQ_INSTOP_LOCK; + + rc = otx2_mbox_process(npa_lf->mbox); + if (rc < 0) { + otx2_err("Unable to lock POOL in NDC"); + return rc; + } + + return 0; +} + +static int +nix_sqb_unlock(struct rte_mempool *mp) +{ + struct otx2_npa_lf *npa_lf = otx2_intra_dev_get_cfg()->npa_lf; + struct npa_aq_enq_req *req; + int rc; + + req = otx2_mbox_alloc_msg_npa_aq_enq(npa_lf->mbox); + req->aura_id = npa_lf_aura_handle_to_aura(mp->pool_id); + req->ctype = NPA_AQ_CTYPE_AURA; + req->op = NPA_AQ_INSTOP_UNLOCK; + + req = otx2_mbox_alloc_msg_npa_aq_enq(npa_lf->mbox); + if (!req) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + otx2_mbox_msg_send(npa_lf->mbox, 0); + rc = otx2_mbox_wait_for_rsp(npa_lf->mbox, 0); + if (rc < 0) { + otx2_err("Failed to UNLOCK AURA context"); + return rc; + } + + req = otx2_mbox_alloc_msg_npa_aq_enq(npa_lf->mbox); + if (!req) { + otx2_err("Failed to UNLOCK POOL context"); + return -ENOMEM; + } + } + req = otx2_mbox_alloc_msg_npa_aq_enq(npa_lf->mbox); + req->aura_id = npa_lf_aura_handle_to_aura(mp->pool_id); + req->ctype = NPA_AQ_CTYPE_POOL; + req->op = NPA_AQ_INSTOP_UNLOCK; + + rc = otx2_mbox_process(npa_lf->mbox); + if (rc < 0) { + otx2_err("Unable to UNLOCK AURA in NDC"); + return rc; + } + + return 0; +} + static int nix_sq_init(struct otx2_eth_txq *txq) { @@ -757,7 +910,20 @@ nix_sq_init(struct otx2_eth_txq *txq) /* Many to one reduction */ sq->sq.qint_idx = txq->sq % dev->qints; - return otx2_mbox_process(mbox); + rc = otx2_mbox_process(mbox); + if (rc < 0) + return rc; + + if (dev->lock_tx_ctx) { + sq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + sq->qidx = txq->sq; + sq->ctype = NIX_AQ_CTYPE_SQ; + sq->op = NIX_AQ_INSTOP_LOCK; + + rc = otx2_mbox_process(mbox); + } + + return rc; } static int @@ -800,6 +966,20 @@ nix_sq_uninit(struct otx2_eth_txq *txq) if (rc) return rc; + if (dev->lock_tx_ctx) { + /* Unlock sq */ + aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + aq->qidx = txq->sq; + aq->ctype = NIX_AQ_CTYPE_SQ; + aq->op = NIX_AQ_INSTOP_UNLOCK; + + rc = otx2_mbox_process(mbox); + if (rc < 0) + return rc; + + nix_sqb_unlock(txq->sqb_pool); + } + /* Read SQ and free sqb's */ aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); aq->qidx = txq->sq; @@ -921,6 +1101,8 @@ nix_alloc_sqb_pool(int port, struct otx2_eth_txq *txq, uint16_t nb_desc) } nix_sqb_aura_limit_cfg(txq->sqb_pool, txq->nb_sqb_bufs); + if (dev->lock_tx_ctx) + nix_sqb_lock(txq->sqb_pool); return 0; fail: diff --git a/drivers/net/octeontx2/otx2_ethdev.h b/drivers/net/octeontx2/otx2_ethdev.h index e5684f9f0..90ca8cbed 100644 --- a/drivers/net/octeontx2/otx2_ethdev.h +++ b/drivers/net/octeontx2/otx2_ethdev.h @@ -272,6 +272,8 @@ struct otx2_eth_dev { uint8_t max_mac_entries; uint8_t lf_tx_stats; uint8_t lf_rx_stats; + uint8_t lock_rx_ctx; + uint8_t lock_tx_ctx; uint16_t flags; uint16_t cints; uint16_t qints; diff --git a/drivers/net/octeontx2/otx2_ethdev_devargs.c b/drivers/net/octeontx2/otx2_ethdev_devargs.c index 5390eb217..b0480504c 100644 --- a/drivers/net/octeontx2/otx2_ethdev_devargs.c +++ b/drivers/net/octeontx2/otx2_ethdev_devargs.c @@ -124,6 +124,8 @@ parse_switch_header_type(const char *key, const char *value, void *extra_args) #define OTX2_FLOW_MAX_PRIORITY "flow_max_priority" #define OTX2_SWITCH_HEADER_TYPE "switch_header" #define OTX2_RSS_TAG_AS_XOR "tag_as_xor" +#define OTX2_LOCK_RX_CTX "lock_rx_ctx" +#define OTX2_LOCK_TX_CTX "lock_tx_ctx" int otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) @@ -134,9 +136,11 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) uint16_t switch_header_type = 0; uint16_t flow_max_priority = 3; uint16_t ipsec_in_max_spi = 1; - uint16_t scalar_enable = 0; uint16_t rss_tag_as_xor = 0; + uint16_t scalar_enable = 0; struct rte_kvargs *kvlist; + uint16_t lock_rx_ctx = 0; + uint16_t lock_tx_ctx = 0; if (devargs == NULL) goto null_devargs; @@ -161,6 +165,10 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) &parse_switch_header_type, &switch_header_type); rte_kvargs_process(kvlist, OTX2_RSS_TAG_AS_XOR, &parse_flag, &rss_tag_as_xor); + rte_kvargs_process(kvlist, OTX2_LOCK_RX_CTX, + &parse_flag, &lock_rx_ctx); + rte_kvargs_process(kvlist, OTX2_LOCK_TX_CTX, + &parse_flag, &lock_tx_ctx); otx2_parse_common_devargs(kvlist); rte_kvargs_free(kvlist); @@ -169,6 +177,8 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) dev->scalar_ena = scalar_enable; dev->rss_tag_as_xor = rss_tag_as_xor; dev->max_sqb_count = sqb_count; + dev->lock_rx_ctx = lock_rx_ctx; + dev->lock_tx_ctx = lock_tx_ctx; dev->rss_info.rss_size = rss_size; dev->npc_flow.flow_prealloc_size = flow_prealloc_size; dev->npc_flow.flow_max_priority = flow_max_priority; @@ -188,4 +198,6 @@ RTE_PMD_REGISTER_PARAM_STRING(net_octeontx2, OTX2_FLOW_MAX_PRIORITY "=<1-32>" OTX2_SWITCH_HEADER_TYPE "=" OTX2_RSS_TAG_AS_XOR "=1" - OTX2_NPA_LOCK_MASK "=<1-65535>"); + OTX2_NPA_LOCK_MASK "=<1-65535>" + OTX2_LOCK_RX_CTX "=1" + OTX2_LOCK_TX_CTX "=1"); diff --git a/drivers/net/octeontx2/otx2_rss.c b/drivers/net/octeontx2/otx2_rss.c index 7a8c8f3de..34005ef02 100644 --- a/drivers/net/octeontx2/otx2_rss.c +++ b/drivers/net/octeontx2/otx2_rss.c @@ -33,6 +33,29 @@ otx2_nix_rss_tbl_init(struct otx2_eth_dev *dev, req->qidx = (group * rss->rss_size) + idx; req->ctype = NIX_AQ_CTYPE_RSS; req->op = NIX_AQ_INSTOP_INIT; + + if (!dev->lock_rx_ctx) + continue; + + req = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + if (!req) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + otx2_mbox_msg_send(mbox, 0); + rc = otx2_mbox_wait_for_rsp(mbox, 0); + if (rc < 0) + return rc; + + req = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + if (!req) + return -ENOMEM; + } + req->rss.rq = ind_tbl[idx]; + /* Fill AQ info */ + req->qidx = (group * rss->rss_size) + idx; + req->ctype = NIX_AQ_CTYPE_RSS; + req->op = NIX_AQ_INSTOP_LOCK; } otx2_mbox_msg_send(mbox, 0);