From patchwork Mon Feb 7 07:29:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 106934 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 76A28A034F; Mon, 7 Feb 2022 08:30:25 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E0C1F41144; Mon, 7 Feb 2022 08:30:15 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 35E96410F7 for ; Mon, 7 Feb 2022 08:30:14 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 216MmlZn020123; Sun, 6 Feb 2022 23:30:10 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=Snfj/RAvtMHN0EWpKqL2KqW7lKc62YacB0oQe+Z1aMI=; b=XubpgonXhJetZjSOg/uhOE4iojr0eRW/YJ3A7wOlE0810OXYRpF89swj7Q+y0dPrqebK QaVKyHpjg++UEbb9cVAn4jvfWvYHL6aWUW1UpkF3IKVEiWyjwgoVDvlUbGgLpo2rSmob cXNe6vtLuIXuGvzEWaXVi4uyGZT6lw3b5XOnZL9i3NqgQH8Yg1klyyHD+NdqJw0M6ETp JQ9ygQ7V/dRTO1m1/aDGBpqw3AwQmOw93i3PKvjctpvaPpvniMsMzgzFhpGIijiu37rO NlXLsIF3TyHTLZ7+FIzvWF/OUfLOgLqH4qnDrSviKSRxkOE7z0+nWZKy20uvQpgOlrvT Rw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3e1smr4p2e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 23:30:09 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 6 Feb 2022 23:29:51 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sun, 6 Feb 2022 23:29:51 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id C49C83F704F; Sun, 6 Feb 2022 23:29:49 -0800 (PST) From: Nithin Dabilpuram To: , Nithin Dabilpuram , "Kiran Kumar K" , Sunil Kumar Kori , Satha Rao , Ray Kinsella CC: Subject: [PATCH 02/20] common/cnxk: realloc inline device XAQ AURA Date: Mon, 7 Feb 2022 12:59:14 +0530 Message-ID: <20220207072932.22409-2-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20220207072932.22409-1-ndabilpuram@marvell.com> References: <20220207072932.22409-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: pGJUAInS444-UBfGmA2VPuI4aUZAthgv X-Proofpoint-ORIG-GUID: pGJUAInS444-UBfGmA2VPuI4aUZAthgv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-07_02,2022-02-03_01,2021-12-02_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support to realloc inline device XAQ AURA with more buffers of new packet pool AURA. Signed-off-by: Nithin Dabilpuram --- drivers/common/cnxk/roc_nix_inl.h | 1 + drivers/common/cnxk/roc_nix_inl_dev.c | 110 +++++++++++++++++++++++++++++- drivers/common/cnxk/roc_nix_inl_dev_irq.c | 2 +- drivers/common/cnxk/roc_nix_inl_priv.h | 3 + drivers/common/cnxk/roc_platform.h | 1 + drivers/common/cnxk/version.map | 3 +- 6 files changed, 115 insertions(+), 5 deletions(-) diff --git a/drivers/common/cnxk/roc_nix_inl.h b/drivers/common/cnxk/roc_nix_inl.h index abbeac6..bbdcbc8 100644 --- a/drivers/common/cnxk/roc_nix_inl.h +++ b/drivers/common/cnxk/roc_nix_inl.h @@ -124,6 +124,7 @@ void __roc_api roc_nix_inl_dev_dump(struct roc_nix_inl_dev *roc_inl_dev); bool __roc_api roc_nix_inl_dev_is_probed(void); void __roc_api roc_nix_inl_dev_lock(void); void __roc_api roc_nix_inl_dev_unlock(void); +int __roc_api roc_nix_inl_dev_xaq_realloc(uint64_t aura_handle); /* NIX Inline Inbound API */ int __roc_api roc_nix_inl_inb_init(struct roc_nix *roc_nix); diff --git a/drivers/common/cnxk/roc_nix_inl_dev.c b/drivers/common/cnxk/roc_nix_inl_dev.c index dd93765..1d14f04 100644 --- a/drivers/common/cnxk/roc_nix_inl_dev.c +++ b/drivers/common/cnxk/roc_nix_inl_dev.c @@ -219,7 +219,6 @@ nix_inl_sso_setup(struct nix_inl_dev *inl_dev) struct sso_lf_alloc_rsp *sso_rsp; struct dev *dev = &inl_dev->dev; uint16_t hwgrp[1] = {0}; - uint32_t xae_cnt; int rc; /* Alloc SSOW LF */ @@ -240,8 +239,8 @@ nix_inl_sso_setup(struct nix_inl_dev *inl_dev) inl_dev->xae_waes = sso_rsp->xaq_wq_entries; inl_dev->iue = sso_rsp->in_unit_entries; - xae_cnt = inl_dev->iue; - rc = sso_hwgrp_init_xaq_aura(dev, &inl_dev->xaq, xae_cnt, + inl_dev->nb_xae = inl_dev->iue; + rc = sso_hwgrp_init_xaq_aura(dev, &inl_dev->xaq, inl_dev->nb_xae, inl_dev->xae_waes, inl_dev->xaq_buf_size, 1); if (rc) { @@ -518,6 +517,111 @@ nix_inl_lf_detach(struct nix_inl_dev *inl_dev) return mbox_process(dev->mbox); } +static int +nix_inl_dev_wait_for_sso_empty(struct nix_inl_dev *inl_dev) +{ + uintptr_t sso_base = inl_dev->sso_base; + int wait_ms = 3000; + + while (wait_ms > 0) { + /* Break when empty */ + if (!plt_read64(sso_base + SSO_LF_GGRP_XAQ_CNT) && + !plt_read64(sso_base + SSO_LF_GGRP_AQ_CNT)) + return 0; + + plt_delay_us(1000); + wait_ms -= 1; + } + + return -ETIMEDOUT; +} + +int +roc_nix_inl_dev_xaq_realloc(uint64_t aura_handle) +{ + struct idev_cfg *idev = idev_get_cfg(); + struct nix_inl_dev *inl_dev; + int rc, i; + + if (idev == NULL) + return 0; + + inl_dev = idev->nix_inl_dev; + /* Nothing to do if no inline device */ + if (!inl_dev) + return 0; + + if (!aura_handle) { + inl_dev->nb_xae = inl_dev->iue; + goto no_pool; + } + + /* Check if aura is already considered */ + for (i = 0; i < inl_dev->pkt_pools_cnt; i++) { + if (inl_dev->pkt_pools[i] == aura_handle) + return 0; + } + +no_pool: + /* Disable RQ if enabled */ + if (inl_dev->rq_refs) { + rc = nix_rq_ena_dis(&inl_dev->dev, &inl_dev->rq, false); + if (rc) { + plt_err("Failed to disable inline dev RQ, rc=%d", rc); + return rc; + } + } + + /* Wait for events to be removed */ + rc = nix_inl_dev_wait_for_sso_empty(inl_dev); + if (rc) { + plt_err("Timeout waiting for inline device event cleanup"); + goto exit; + } + + /* Disable HWGRP */ + plt_write64(0, inl_dev->sso_base + SSO_LF_GGRP_QCTL); + + inl_dev->pkt_pools_cnt++; + inl_dev->pkt_pools = + plt_realloc(inl_dev->pkt_pools, + sizeof(uint64_t *) * inl_dev->pkt_pools_cnt, 0); + if (!inl_dev->pkt_pools) + inl_dev->pkt_pools_cnt = 0; + else + inl_dev->pkt_pools[inl_dev->pkt_pools_cnt - 1] = aura_handle; + inl_dev->nb_xae += roc_npa_aura_op_limit_get(aura_handle); + + /* Realloc XAQ aura */ + rc = sso_hwgrp_init_xaq_aura(&inl_dev->dev, &inl_dev->xaq, + inl_dev->nb_xae, inl_dev->xae_waes, + inl_dev->xaq_buf_size, 1); + if (rc) { + plt_err("Failed to reinitialize xaq aura, rc=%d", rc); + return rc; + } + + /* Setup xaq for hwgrps */ + rc = sso_hwgrp_alloc_xaq(&inl_dev->dev, inl_dev->xaq.aura_handle, 1); + if (rc) { + plt_err("Failed to setup hwgrp xaq aura, rc=%d", rc); + return rc; + } + + /* Enable HWGRP */ + plt_write64(0x1, inl_dev->sso_base + SSO_LF_GGRP_QCTL); + +exit: + /* Renable RQ */ + if (inl_dev->rq_refs) { + rc = nix_rq_ena_dis(&inl_dev->dev, &inl_dev->rq, true); + if (rc) + plt_err("Failed to enable inline dev RQ, rc=%d", rc); + } + + return rc; +} + int roc_nix_inl_dev_init(struct roc_nix_inl_dev *roc_inl_dev) { diff --git a/drivers/common/cnxk/roc_nix_inl_dev_irq.c b/drivers/common/cnxk/roc_nix_inl_dev_irq.c index 848523b..d758e0c 100644 --- a/drivers/common/cnxk/roc_nix_inl_dev_irq.c +++ b/drivers/common/cnxk/roc_nix_inl_dev_irq.c @@ -72,7 +72,7 @@ nix_inl_sso_hwgrp_irq(void *param) if (intr & BIT(1)) nix_inl_sso_work_cb(inl_dev); - if (!(intr & BIT(1))) + if (intr & ~BIT(1)) plt_err("GGRP 0 GGRP_INT=0x%" PRIx64 "", intr); /* Clear interrupt */ diff --git a/drivers/common/cnxk/roc_nix_inl_priv.h b/drivers/common/cnxk/roc_nix_inl_priv.h index 2cdab6dc..17df23f 100644 --- a/drivers/common/cnxk/roc_nix_inl_priv.h +++ b/drivers/common/cnxk/roc_nix_inl_priv.h @@ -27,9 +27,12 @@ struct nix_inl_dev { uint32_t xaq_buf_size; uint32_t xae_waes; uint32_t iue; + uint32_t nb_xae; struct roc_sso_xaq_data xaq; roc_nix_inl_sso_work_cb_t work_cb; void *cb_args; + uint64_t *pkt_pools; + uint16_t pkt_pools_cnt; /* NIX data */ uint8_t lf_tx_stats; diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index c35a2b1..8eac24f 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -147,6 +147,7 @@ #define plt_intr_handle rte_intr_handle #define plt_zmalloc(sz, align) rte_zmalloc("cnxk", sz, align) +#define plt_realloc rte_realloc #define plt_free rte_free #define plt_read64(addr) rte_read64_relaxed((volatile void *)(addr)) diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 82b9fc1..617364f 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -132,11 +132,12 @@ INTERNAL { roc_nix_inl_dev_init; roc_nix_inl_dev_is_probed; roc_nix_inl_dev_lock; - roc_nix_inl_dev_unlock; roc_nix_inl_dev_rq; roc_nix_inl_dev_rq_get; roc_nix_inl_dev_rq_put; roc_nix_inl_dev_rq_limit_get; + roc_nix_inl_dev_unlock; + roc_nix_inl_dev_xaq_realloc; roc_nix_inl_inb_is_enabled; roc_nix_inl_inb_init; roc_nix_inl_inb_sa_base_get;