From patchwork Tue Feb 27 19:15:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harman Kalra X-Patchwork-Id: 137376 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3CBAF43C06; Tue, 27 Feb 2024 20:16:28 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7BD0142EE6; Tue, 27 Feb 2024 20:16:16 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id BABD642EE6 for ; Tue, 27 Feb 2024 20:16:14 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 41RFgUPG004861 for ; Tue, 27 Feb 2024 11:16:14 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type; s=pfpt0220; bh=/RdgkfBAVw9fVDU1mYhbQ dM8zcuwQuNtFe6sL4sfoT0=; b=deNLiNV3liytqsFXgO2cRUBCHmAeyLRuXtG6K PQhQwomf81xpmMFCwFONF66h702untObfED1S8HlKUjzhHD7Qlmd/ib54RoqsKgB fLTxixiLw7D03NLBXmg8m4WkwATFotFOff74bBAwQ3h1k2G3mtfYEfeI07QbnhWI LS+/oG0xdw9VBR0FXXwm+K6HG1/9dSaGZL3vDIFpSfYVesWso2EbO6E1YhAxqKOj 7dak3SuKxGBUy5du6sretPNv6T4zR4iySg932q4jX+YGGL5tH6NeUXsWwf1ISL5c x1rt3rd6T89lNZ3ICSB6psnlWgMraMUYOgE6trO7IcfzBxyHg== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3whjm694kq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 27 Feb 2024 11:16:14 -0800 (PST) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Tue, 27 Feb 2024 11:16:12 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1258.12 via Frontend Transport; Tue, 27 Feb 2024 11:16:12 -0800 Received: from localhost.localdomain (unknown [10.29.52.211]) by maili.marvell.com (Postfix) with ESMTP id 676673F718B; Tue, 27 Feb 2024 11:16:10 -0800 (PST) From: Harman Kalra To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra CC: Subject: [PATCH v4 03/23] net/cnxk: eswitch HW resource configuration Date: Wed, 28 Feb 2024 00:45:30 +0530 Message-ID: <20240227191550.137687-4-hkalra@marvell.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20240227191550.137687-1-hkalra@marvell.com> References: <20230811163419.165790-1-hkalra@marvell.com> <20240227191550.137687-1-hkalra@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: J31Gq-jbCq1EF8lRh1xB6hoyHC1onGBV X-Proofpoint-GUID: J31Gq-jbCq1EF8lRh1xB6hoyHC1onGBV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-27_06,2024-02-27_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Configuring the hardware resources used by the eswitch device. Signed-off-by: Harman Kalra --- drivers/net/cnxk/cnxk_eswitch.c | 216 ++++++++++++++++++++++++++++++++ 1 file changed, 216 insertions(+) diff --git a/drivers/net/cnxk/cnxk_eswitch.c b/drivers/net/cnxk/cnxk_eswitch.c index c4ea3063ae..810e7c9c25 100644 --- a/drivers/net/cnxk/cnxk_eswitch.c +++ b/drivers/net/cnxk/cnxk_eswitch.c @@ -6,6 +6,47 @@ #define CNXK_NIX_DEF_SQ_COUNT 512 +static int +eswitch_hw_rsrc_cleanup(struct cnxk_eswitch_dev *eswitch_dev, struct rte_pci_device *pci_dev) +{ + struct roc_nix *nix; + int rc = 0; + + nix = &eswitch_dev->nix; + + roc_nix_unregister_queue_irqs(nix); + roc_nix_tm_fini(nix); + rc = roc_nix_lf_free(nix); + if (rc) { + plt_err("Failed to cleanup sq, rc %d", rc); + goto exit; + } + + /* Check if this device is hosting common resource */ + nix = roc_idev_npa_nix_get(); + if (!nix || nix->pci_dev != pci_dev) { + rc = 0; + goto exit; + } + + /* Try nix fini now */ + rc = roc_nix_dev_fini(nix); + if (rc == -EAGAIN) { + plt_info("Common resource in use by other devices %s", pci_dev->name); + goto exit; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + goto exit; + } + + rte_free(eswitch_dev->txq); + rte_free(eswitch_dev->rxq); + rte_free(eswitch_dev->cxq); + +exit: + return rc; +} + static int cnxk_eswitch_dev_remove(struct rte_pci_device *pci_dev) { @@ -21,6 +62,9 @@ cnxk_eswitch_dev_remove(struct rte_pci_device *pci_dev) goto exit; } + /* Cleanup HW resources */ + eswitch_hw_rsrc_cleanup(eswitch_dev, pci_dev); + rte_free(eswitch_dev); exit: return rc; @@ -317,6 +361,170 @@ cnxk_eswitch_txq_setup(struct cnxk_eswitch_dev *eswitch_dev, uint16_t qid, uint1 return rc; } +static int +nix_lf_setup(struct cnxk_eswitch_dev *eswitch_dev) +{ + uint16_t nb_rxq, nb_txq, nb_cq; + struct roc_nix_fc_cfg fc_cfg; + struct roc_nix *nix; + uint64_t rx_cfg; + void *qs; + int rc; + + /* Initialize base roc nix */ + nix = &eswitch_dev->nix; + nix->pci_dev = eswitch_dev->pci_dev; + nix->hw_vlan_ins = true; + nix->reta_sz = ROC_NIX_RSS_RETA_SZ_256; + rc = roc_nix_dev_init(nix); + if (rc) { + plt_err("Failed to init nix eswitch device, rc=%d(%s)", rc, roc_error_msg_get(rc)); + goto fail; + } + + /* Get the representors count */ + rc = roc_nix_max_rep_count(&eswitch_dev->nix); + if (rc) { + plt_err("Failed to get rep cnt, rc=%d(%s)", rc, roc_error_msg_get(rc)); + goto free_cqs; + } + + /* Allocating an NIX LF */ + nb_rxq = CNXK_ESWITCH_MAX_RXQ; + nb_txq = CNXK_ESWITCH_MAX_TXQ; + nb_cq = CNXK_ESWITCH_MAX_RXQ; + rx_cfg = ROC_NIX_LF_RX_CFG_DIS_APAD; + rc = roc_nix_lf_alloc(nix, nb_rxq, nb_txq, rx_cfg); + if (rc) { + plt_err("lf alloc failed = %s(%d)", roc_error_msg_get(rc), rc); + goto dev_fini; + } + + if (nb_rxq) { + /* Allocate memory for eswitch rq's and cq's */ + qs = plt_zmalloc(sizeof(struct cnxk_eswitch_rxq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc eswitch rxq"); + goto lf_free; + } + eswitch_dev->rxq = qs; + } + + if (nb_txq) { + /* Allocate memory for roc sq's */ + qs = plt_zmalloc(sizeof(struct cnxk_eswitch_txq) * nb_txq, 0); + if (!qs) { + plt_err("Failed to alloc eswitch txq"); + goto free_rqs; + } + eswitch_dev->txq = qs; + } + + if (nb_cq) { + qs = plt_zmalloc(sizeof(struct cnxk_eswitch_cxq) * nb_cq, 0); + if (!qs) { + plt_err("Failed to alloc eswitch cxq"); + goto free_sqs; + } + eswitch_dev->cxq = qs; + } + + eswitch_dev->nb_rxq = nb_rxq; + eswitch_dev->nb_txq = nb_txq; + + /* Re-enable NIX LF error interrupts */ + roc_nix_err_intr_ena_dis(nix, true); + roc_nix_ras_intr_ena_dis(nix, true); + + rc = roc_nix_lso_fmt_setup(nix); + if (rc) { + plt_err("lso setup failed = %s(%d)", roc_error_msg_get(rc), rc); + goto free_cqs; + } + + rc = roc_nix_switch_hdr_set(nix, 0, 0, 0, 0); + if (rc) { + plt_err("switch hdr set failed = %s(%d)", roc_error_msg_get(rc), rc); + goto free_cqs; + } + + rc = roc_nix_tm_init(nix); + if (rc) { + plt_err("tm failed = %s(%d)", roc_error_msg_get(rc), rc); + goto free_cqs; + } + + /* Register queue IRQs */ + rc = roc_nix_register_queue_irqs(nix); + if (rc) { + plt_err("Failed to register queue interrupts rc=%d", rc); + goto tm_fini; + } + + /* Enable default tree */ + rc = roc_nix_tm_hierarchy_enable(nix, ROC_NIX_TM_DEFAULT, false); + if (rc) { + plt_err("tm default hierarchy enable failed = %s(%d)", roc_error_msg_get(rc), rc); + goto q_irq_fini; + } + + memset(&fc_cfg, 0, sizeof(struct roc_nix_fc_cfg)); + fc_cfg.rxchan_cfg.enable = false; + rc = roc_nix_fc_config_set(nix, &fc_cfg); + if (rc) { + plt_err("Failed to setup flow control, rc=%d(%s)", rc, roc_error_msg_get(rc)); + goto q_irq_fini; + } + + roc_nix_fc_mode_get(nix); + + return rc; +q_irq_fini: + roc_nix_unregister_queue_irqs(nix); +tm_fini: + roc_nix_tm_fini(nix); +free_cqs: + rte_free(eswitch_dev->cxq); +free_sqs: + rte_free(eswitch_dev->txq); +free_rqs: + rte_free(eswitch_dev->rxq); +lf_free: + roc_nix_lf_free(nix); +dev_fini: + roc_nix_dev_fini(nix); +fail: + return rc; +} + +static int +eswitch_hw_rsrc_setup(struct cnxk_eswitch_dev *eswitch_dev, struct rte_pci_device *pci_dev) +{ + struct roc_nix *nix; + int rc; + + nix = &eswitch_dev->nix; + rc = nix_lf_setup(eswitch_dev); + if (rc) { + plt_err("Failed to setup hw rsrc, rc=%d(%s)", rc, roc_error_msg_get(rc)); + goto fail; + } + + /* Initialize roc npc */ + eswitch_dev->npc.roc_nix = nix; + eswitch_dev->npc.flow_max_priority = 3; + eswitch_dev->npc.flow_prealloc_size = 1; + rc = roc_npc_init(&eswitch_dev->npc); + if (rc) + goto rsrc_cleanup; + + return rc; +rsrc_cleanup: + eswitch_hw_rsrc_cleanup(eswitch_dev, pci_dev); +fail: + return rc; +} + static int cnxk_eswitch_dev_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) { @@ -346,6 +554,12 @@ cnxk_eswitch_dev_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pc eswitch_dev = mz->addr; eswitch_dev->pci_dev = pci_dev; + + rc = eswitch_hw_rsrc_setup(eswitch_dev, pci_dev); + if (rc) { + plt_err("Failed to setup hw rsrc, rc=%d(%s)", rc, roc_error_msg_get(rc)); + goto free_mem; + } } /* Spinlock for synchronization between representors traffic and control @@ -354,6 +568,8 @@ cnxk_eswitch_dev_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pc rte_spinlock_init(&eswitch_dev->rep_lock); return rc; +free_mem: + rte_memzone_free(mz); fail: return rc; }