From patchwork Sat Mar 6 15:33:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88598 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8B745A0548; Sat, 6 Mar 2021 16:34:23 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5532E22A34F; Sat, 6 Mar 2021 16:34:20 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 8041D22A342 for ; Sat, 6 Mar 2021 16:34:18 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVumC029368 for ; Sat, 6 Mar 2021 07:34:17 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=phCYZCno6oYO2GBUlQ8sQuvHfr83x/WBR8ZxTc9EGZQ=; b=DOVKjspvZYSdVX8ZRwulxvg3IUMVjN6wGq9+IWnmjIugRGJ+Ng0b5GJWje+eQhLi51sP MzZR0XvLrw5frSXTXt76lMThYHIxdPNVlSw0NefcAxol3nHw97s/95pGKudX3qDtzWMU 0dCjXAS18Hk4p8BB9bKpbVKDios+sALjvCWVrru47MjqPopYAInhQw6+AeGALEjC7UWS tqnIQRrT0msISrgyzw8eyMuwyaRM59LsXNC3j07pDsBegaYLbOH4HtnsGuR/g1zUmXQf m2oqIjQ9TobYx5Dg36rPqkz5zDbFrIwPKqrIhYg3gTAPJXi1oIWE6NofIhHaiKGywMB5 Zg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurca6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:17 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:16 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:15 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 420973F7041; Sat, 6 Mar 2021 07:34:13 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:21 +0530 Message-ID: <20210306153404.10781-2-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 01/44] net/cnxk: add build infra and common probe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add build infrastructure and common probe and remove for cnxk driver which is used by both CN10K and CN9K SoC. Signed-off-by: Nithin Dabilpuram --- MAINTAINERS | 3 + doc/guides/nics/cnxk.rst | 29 +++++ doc/guides/nics/features/cnxk.ini | 9 ++ doc/guides/nics/features/cnxk_vec.ini | 9 ++ doc/guides/nics/features/cnxk_vf.ini | 9 ++ doc/guides/nics/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 + drivers/net/cnxk/cnxk_ethdev.c | 219 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++ drivers/net/cnxk/meson.build | 21 ++++ drivers/net/cnxk/version.map | 3 + drivers/net/meson.build | 1 + 12 files changed, 364 insertions(+) create mode 100644 doc/guides/nics/cnxk.rst create mode 100644 doc/guides/nics/features/cnxk.ini create mode 100644 doc/guides/nics/features/cnxk_vec.ini create mode 100644 doc/guides/nics/features/cnxk_vf.ini create mode 100644 drivers/net/cnxk/cnxk_ethdev.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.h create mode 100644 drivers/net/cnxk/meson.build create mode 100644 drivers/net/cnxk/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 67c179f..efabc3c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -745,6 +745,9 @@ M: Sunil Kumar Kori M: Satha Rao T: git://dpdk.org/next/dpdk-next-net-mrvl F: drivers/common/cnxk/ +F: drivers/net/cnxk/ +F: doc/guides/nics/cnxk.rst +F: doc/guides/nics/features/cnxk*.ini F: doc/guides/platform/cnxk.rst Marvell mvpp2 diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst new file mode 100644 index 0000000..ca21842 --- /dev/null +++ b/doc/guides/nics/cnxk.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2021 Marvell. + +CNXK Poll Mode driver +===================== + +The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver +support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K** +SoC family as well as for their virtual functions (VF) in SR-IOV context. + +More information can be found at `Marvell Official Website +`_. + +Features +-------- + +Features of the CNXK Ethdev PMD are: + +Prerequisites +------------- + +See :doc:`../platform/cnxk` for setup information. + + +Driver compilation and testing +------------------------------ + +Refer to the document :ref:`compiling and testing a PMD for a NIC ` +for details. diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini new file mode 100644 index 0000000..2c23464 --- /dev/null +++ b/doc/guides/nics/features/cnxk.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini new file mode 100644 index 0000000..de78516 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vec' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini new file mode 100644 index 0000000..9c96351 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vf' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 799697c..c1a04d9 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -19,6 +19,7 @@ Network Interface Controller Drivers axgbe bnx2x bnxt + cnxk cxgbe dpaa dpaa2 diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst index 9bbba65..a9c050e 100644 --- a/doc/guides/platform/cnxk.rst +++ b/doc/guides/platform/cnxk.rst @@ -141,6 +141,9 @@ HW Offload Drivers This section lists dataplane H/W block(s) available in CNXK SoC. +#. **Ethdev Driver** + See :doc:`../nics/cnxk` for NIX Ethdev driver information. + #. **Mempool Driver** See :doc:`../mempool/cnxk` for NPA mempool driver information. diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c new file mode 100644 index 0000000..6717410 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -0,0 +1,219 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include + +/* CNXK platform independent eth dev ops */ +struct eth_dev_ops cnxk_eth_dev_ops; + +static int +cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct rte_pci_device *pci_dev; + int rc, max_entries; + + eth_dev->dev_ops = &cnxk_eth_dev_ops; + + /* For secondary processes, the primary has done all the work */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + rte_eth_copy_pci_info(eth_dev, pci_dev); + eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; + + /* Initialize base roc nix */ + nix->pci_dev = pci_dev; + rc = roc_nix_dev_init(nix); + if (rc) { + plt_err("Failed to initialize roc nix rc=%d", rc); + goto error; + } + + dev->eth_dev = eth_dev; + + /* For vfs, returned max_entries will be 0. but to keep default mac + * address, one entry must be allocated. so setting up to 1. + */ + if (roc_nix_is_vf_or_sdp(nix)) + max_entries = 1; + else + max_entries = roc_nix_mac_max_entries_get(nix); + + if (max_entries <= 0) { + plt_err("Failed to get max entries for mac addr"); + rc = -ENOTSUP; + goto dev_fini; + } + + eth_dev->data->mac_addrs = + rte_zmalloc("mac_addr", max_entries * RTE_ETHER_ADDR_LEN, 0); + if (eth_dev->data->mac_addrs == NULL) { + plt_err("Failed to allocate memory for mac addr"); + rc = -ENOMEM; + goto dev_fini; + } + + dev->max_mac_entries = max_entries; + + /* Get mac address */ + rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to get mac addr, rc=%d", rc); + goto free_mac_addrs; + } + + /* Update the mac address */ + memcpy(eth_dev->data->mac_addrs, dev->mac_addr, RTE_ETHER_ADDR_LEN); + + if (!roc_nix_is_vf_or_sdp(nix)) { + /* Sync same MAC address to CGX/RPM table */ + rc = roc_nix_mac_addr_set(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to set mac addr, rc=%d", rc); + goto free_mac_addrs; + } + } + + /* Initialize roc npc */ + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 + " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, + eth_dev->data->port_id, roc_nix_get_pf(nix), + roc_nix_get_vf(nix), CNXK_ETH_DEV_PMD_VERSION, dev->hwcap, + dev->rx_offload_capa, dev->tx_offload_capa); + return 0; + +free_mac_addrs: + rte_free(eth_dev->data->mac_addrs); +dev_fini: + roc_nix_dev_fini(nix); +error: + plt_err("Failed to init nix eth_dev rc=%d", rc); + return rc; +} + +static int +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct roc_nix *nix = &dev->nix; + int rc, i; + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + roc_nix_npc_rx_ena_dis(nix, false); + + /* Free up SQs */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); + eth_dev->data->tx_queues[i] = NULL; + } + eth_dev->data->nb_tx_queues = 0; + + /* Free up RQ's and CQ's */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + dev_ops->rx_queue_release(eth_dev->data->rx_queues[i]); + eth_dev->data->rx_queues[i] = NULL; + } + eth_dev->data->nb_rx_queues = 0; + + /* Free tm resources */ + roc_nix_tm_fini(nix); + + /* Unregister queue irqs */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister cq irqs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Free nix lf resources */ + rc = roc_nix_lf_free(nix); + if (rc) + plt_err("Failed to free nix lf, rc=%d", rc); + + rte_free(eth_dev->data->mac_addrs); + eth_dev->data->mac_addrs = NULL; + + /* Check if mbox close is needed */ + if (!mbox_close) + return 0; + + rc = roc_nix_dev_fini(nix); + /* Can be freed later by PMD if NPA LF is in use */ + if (rc == -EAGAIN) { + eth_dev->data->dev_private = NULL; + return 0; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + } + + return rc; +} + +int +cnxk_nix_remove(struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct roc_nix *nix; + int rc = -EINVAL; + + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (eth_dev) { + /* Cleanup eth dev */ + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + rte_eth_dev_release_port(eth_dev); + } + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + /* Check if this device is hosting common resource */ + nix = roc_idev_npa_nix_get(); + if (nix->pci_dev != pci_dev) + return 0; + + /* Try nix fini now */ + rc = roc_nix_dev_fini(nix); + if (rc == -EAGAIN) { + plt_info("%s: common resource in use by other devices", + pci_dev->name); + goto exit; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + goto exit; + } + + /* Free device pointer as rte_ethdev does not have it anymore */ + rte_free(nix); +exit: + return rc; +} + +int +cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + int rc; + + RTE_SET_USED(pci_drv); + + rc = rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct cnxk_eth_dev), + cnxk_eth_dev_init); + + /* On error on secondary, recheck if port exists in primary or + * in mid of detach state. + */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY && rc) + if (!rte_eth_dev_allocated(pci_dev->device.name)) + return 0; + return rc; +} diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h new file mode 100644 index 0000000..0460d1e --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_ETHDEV_H__ +#define __CNXK_ETHDEV_H__ + +#include +#include + +#include +#include + +#include "roc_api.h" + +#define CNXK_ETH_DEV_PMD_VERSION "1.0" + +struct cnxk_eth_dev { + /* ROC NIX */ + struct roc_nix nix; + + /* Max macfilter entries */ + uint8_t max_mac_entries; + + uint16_t flags; + + /* Pointer back to rte */ + struct rte_eth_dev *eth_dev; + + /* HW capabilities / Limitations */ + union { + uint64_t hwcap; + }; + + /* Rx and Tx offload capabilities */ + uint64_t rx_offload_capa; + uint64_t tx_offload_capa; + uint32_t speed_capa; + + /* Default mac address */ + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; +}; + +static inline struct cnxk_eth_dev * +cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) +{ + return eth_dev->data->dev_private; +} + +/* Common ethdev ops */ +extern struct eth_dev_ops cnxk_eth_dev_ops; + +/* Ops */ +int cnxk_nix_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev); +int cnxk_nix_remove(struct rte_pci_device *pci_dev); + +#endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build new file mode 100644 index 0000000..77b2f18 --- /dev/null +++ b/drivers/net/cnxk/meson.build @@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2021 Marvell. +# + +if not dpdk_conf.get('RTE_ARCH_64') + build = false + reason = 'only supported on 64-bit' + subdir_done() +endif + +sources = files('cnxk_ethdev.c') + +deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] +deps += ['common_cnxk', 'mempool_cnxk'] + +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] +foreach flag: extra_flags + if cc.has_argument(flag) + cflags += flag + endif +endforeach diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map new file mode 100644 index 0000000..ee80c51 --- /dev/null +++ b/drivers/net/cnxk/version.map @@ -0,0 +1,3 @@ +INTERNAL { + local: *; +}; diff --git a/drivers/net/meson.build b/drivers/net/meson.build index fb9ff05..9f69842 100644 --- a/drivers/net/meson.build +++ b/drivers/net/meson.build @@ -10,6 +10,7 @@ drivers = ['af_packet', 'axgbe', 'bonding', 'bnx2x', 'bnxt', + 'cnxk', 'cxgbe', 'dpaa', 'dpaa2', 'e1000', From patchwork Sat Mar 6 15:33:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88599 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 29CE7A0548; Sat, 6 Mar 2021 16:34:34 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 122CA22A358; Sat, 6 Mar 2021 16:34:23 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 9929922A34B for ; Sat, 6 Mar 2021 16:34:21 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmqL029543 for ; Sat, 6 Mar 2021 07:34:20 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=4SLbwMRA8wkaqKApzdR2rt/+ngd3zXUyvarHCS4H1v4=; b=W6QcdKg5rkEawyeUyu+SFuapFRmHn4K2laXeG1NufYVmA3GO3ThTIbELeEDSWmICACE0 ynlb5laWpsrZIIMrfBJaWzTCnDdG6NE+MUSm2tgXaxouoLWos2U76uIVABi8FdI/UWXt rq+r1XiFMy6s/6WFd0QJOWB3VozHi1EOCH7XFsDAzactJQoiABvupZYBHHwr5m1paLse 8JeRzya7YKqMw5f2l8gey4Sgkz9QJ/seZWnGYohlntBkjEZ3Wsz00N/RdSVR4kbWTZyI EBHZsuaS8ioYrPxHEKS75nvz5uZ/kbVuDESnKAaurxHRQyN0PbAhLbfLmKu9QNERGLqh wg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcab-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:20 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:19 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:19 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 8A1CA3F7043; Sat, 6 Mar 2021 07:34:16 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:22 +0530 Message-ID: <20210306153404.10781-3-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 02/44] net/cnxk: add platform specific probe and remove X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add platform specific probe and remove callbacks for CN9K and CN10K which use common probe and remove functions. Register ethdev driver for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram --- drivers/net/cnxk/cn10k_ethdev.c | 64 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 9 +++++ drivers/net/cnxk/cn9k_ethdev.c | 82 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 9 +++++ drivers/net/cnxk/cnxk_ethdev.c | 42 +++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 19 ++++++++++ drivers/net/cnxk/meson.build | 5 +++ 7 files changed, 230 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_ethdev.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.h create mode 100644 drivers/net/cnxk/cn9k_ethdev.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.h diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c new file mode 100644 index 0000000..54711ea --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn10k_ethdev.h" + +static int +cn10k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 64) { + plt_err("Driver not compiled for CN10K"); + return -EFAULT; + } + + rc = plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + } + return 0; +} + +static const struct rte_pci_id cn10k_pci_nix_map[] = { + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_AF_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_AF_VF), + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn10k_pci_nix = { + .id_table = cn10k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn10k_nix_probe, + .remove = cn10k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn10k, cn10k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn10k, cn10k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn10k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h new file mode 100644 index 0000000..1bf4a65 --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_ETHDEV_H__ +#define __CN10K_ETHDEV_H__ + +#include + +#endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c new file mode 100644 index 0000000..bd97d5f --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn9k_ethdev.h" + +static int +cn9k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 128) { + plt_err("Driver not compiled for CN9K"); + return -EFAULT; + } + + rc = plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + /* Find eth dev allocated */ + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + + dev = cnxk_eth_pmd_priv(eth_dev); + /* Update capabilities already set for TSO. + * TSO not supported for earlier chip revisions + */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) + dev->tx_offload_capa &= ~(DEV_TX_OFFLOAD_TCP_TSO | + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | + DEV_TX_OFFLOAD_GRE_TNL_TSO); + + /* 50G and 100G to be supported for board version C0 + * and above of CN9K. + */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) { + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_50G; + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_100G; + } + + dev->hwcap = 0; + + /* Update HW erratas */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) + dev->cq_min_4k = 1; + return 0; +} + +static const struct rte_pci_id cn9k_pci_nix_map[] = { + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn9k_pci_nix = { + .id_table = cn9k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn9k_nix_probe, + .remove = cn9k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn9k, cn9k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn9k, cn9k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn9k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h new file mode 100644 index 0000000..15d9397 --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_ETHDEV_H__ +#define __CN9K_ETHDEV_H__ + +#include + +#endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 6717410..b836fc2 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -3,6 +3,40 @@ */ #include +static inline uint64_t +nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) +{ + uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; + + return capa; +} + +static inline uint64_t +nix_get_tx_offload_capa(struct cnxk_eth_dev *dev) +{ + RTE_SET_USED(dev); + return CNXK_NIX_TX_OFFLOAD_CAPA; +} + +static inline uint32_t +nix_get_speed_capa(struct cnxk_eth_dev *dev) +{ + uint32_t speed_capa; + + /* Auto negotiation disabled */ + speed_capa = ETH_LINK_SPEED_FIXED; + if (!roc_nix_is_vf_or_sdp(&dev->nix) && !roc_nix_is_lbk(&dev->nix)) { + speed_capa |= ETH_LINK_SPEED_1G | ETH_LINK_SPEED_10G | + ETH_LINK_SPEED_25G | ETH_LINK_SPEED_40G | + ETH_LINK_SPEED_50G | ETH_LINK_SPEED_100G; + } + + return speed_capa; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops; @@ -77,6 +111,14 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } } + /* Union of all capabilities supported by CNXK. + * Platform specific capabilities will be + * updated later. + */ + dev->rx_offload_capa = nix_get_rx_offload_capa(dev); + dev->tx_offload_capa = nix_get_tx_offload_capa(dev); + dev->speed_capa = nix_get_speed_capa(dev); + /* Initialize roc npc */ plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 0460d1e..ba2bfcd 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -14,6 +14,22 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +#define CNXK_NIX_TX_OFFLOAD_CAPA \ + (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ + DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_TX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_TX_OFFLOAD_TCP_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | \ + DEV_TX_OFFLOAD_SCTP_CKSUM | DEV_TX_OFFLOAD_TCP_TSO | \ + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | DEV_TX_OFFLOAD_GENEVE_TNL_TSO | \ + DEV_TX_OFFLOAD_GRE_TNL_TSO | DEV_TX_OFFLOAD_MULTI_SEGS | \ + DEV_TX_OFFLOAD_IPV4_CKSUM) + +#define CNXK_NIX_RX_OFFLOAD_CAPA \ + (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ + DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ + DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_RX_OFFLOAD_RSS_HASH) + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; @@ -28,6 +44,9 @@ struct cnxk_eth_dev { /* HW capabilities / Limitations */ union { + struct { + uint64_t cq_min_4k : 1; + }; uint64_t hwcap; }; diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 77b2f18..57dea5e 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,6 +10,11 @@ endif sources = files('cnxk_ethdev.c') +# CN9K +sources += files('cn9k_ethdev.c') +# CN10K +sources += files('cn10k_ethdev.c') + deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] From patchwork Sat Mar 6 15:33:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88600 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8E44CA0548; Sat, 6 Mar 2021 16:34:42 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 516C422A35B; Sat, 6 Mar 2021 16:34:27 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 360674014D for ; Sat, 6 Mar 2021 16:34:25 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVquf029350 for ; Sat, 6 Mar 2021 07:34:24 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=b1rIIEMu/wmydCmuI4rbYtMvcewvozjAExa3fxOPzRY=; b=jPl84677drliXLA4n0FCv9xDn6wW33UJa/PUIm/11qzWmiHC9GHJJGM3LLszjGkusytL 5teweFNQUMF7Kklz85p9LMUkmxeLPhD5yvqR4R7WI0D72r6N01rfEmpmP9cz5C7uPASZ rgfvzx6zPc8d7ufrubk6h/aPuUU/WOR0HO9Nn1vBkih9fDo0zSOHi2CboAuKiKC6M8NA OZrNVsL5dr/+3PX3a0lJG6d0AEzyTqCnZgRmfoqIUlIZWg3d8/zSn4sXJgvqTsAm8gxw Yn54z6gvnFIuTqvY4yXgTCuIp9owuqStLnSk+I/QlYjaiDQeX5dJTCdMJAAAtSfGfFtN Mg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcad-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:24 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:22 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:22 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:22 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id BFC3C3F703F; Sat, 6 Mar 2021 07:34:19 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:23 +0530 Message-ID: <20210306153404.10781-4-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 03/44] net/cnxk: add common devargs parsing function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add various devargs parsing command line arguments parsing functions supported by CN9K and CN10K. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/cnxk.rst | 94 +++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 7 ++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 166 +++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 278 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index ca21842..611ffb4 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -27,3 +27,97 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC ` for details. + +Runtime Config Options +---------------------- + +- ``Rx&Tx scalar mode enable`` (default ``0``) + + Ethdev supports both scalar and vector mode, it may be selected at runtime + using ``scalar_enable`` ``devargs`` parameter. + +- ``RSS reta size`` (default ``64``) + + RSS redirection table size may be configured during runtime using ``reta_size`` + ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,reta_size=256 + + With the above configuration, reta table of size 256 is populated. + +- ``Flow priority levels`` (default ``3``) + + RTE Flow priority levels can be configured during runtime using + ``flow_max_priority`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_max_priority=10 + + With the above configuration, priority level was set to 10 (0-9). Max + priority level supported is 32. + +- ``Reserve Flow entries`` (default ``8``) + + RTE flow entries can be pre allocated and the size of pre allocation can be + selected runtime using ``flow_prealloc_size`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_prealloc_size=4 + + With the above configuration, pre alloc size was set to 4. Max pre alloc + size supported is 32. + +- ``Max SQB buffer count`` (default ``512``) + + Send queue descriptor buffer count may be limited during runtime using + ``max_sqb_count`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,max_sqb_count=64 + + With the above configuration, each send queue's decscriptor buffer count is + limited to a maximum of 64 buffers. + +- ``Switch header enable`` (default ``none``) + + A port can be configured to a specific switch header type by using + ``switch_header`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,switch_header="higig2" + + With the above configuration, higig2 will be enabled on that port and the + traffic on this port should be higig2 traffic only. Supported switch header + types are "higig2", "dsa", "chlen90b" and "chlen24b". + +- ``RSS tag as XOR`` (default ``0``) + + The HW gives two options to configure the RSS adder i.e + + * ``rss_adder<7:0> = flow_tag<7:0> ^ flow_tag<15:8> ^ flow_tag<23:16> ^ flow_tag<31:24>`` + + * ``rss_adder<7:0> = flow_tag<7:0>`` + + Latter one aligns with standard NIC behavior vs former one is a legacy + RSS adder scheme used in OCTEON TX2 products. + + By default, the driver runs in the latter mode. + Setting this flag to 1 to select the legacy mode. + + For example to select the legacy mode(RSS tag adder as XOR):: + + -a 0002:02:00.0,tag_as_xor=1 + + + +.. note:: + + Above devarg parameters are configurable per device, user needs to pass the + parameters to all the PCIe devices if application requires to configure on + all the ethdev ports. diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index b836fc2..3a2309e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -58,6 +58,13 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) rte_eth_copy_pci_info(eth_dev, pci_dev); eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; + /* Parse devargs string */ + rc = cnxk_ethdev_parse_devargs(eth_dev->device->devargs, dev); + if (rc) { + plt_err("Failed to parse devargs rc=%d", rc); + goto error; + } + /* Initialize base roc nix */ nix->pci_dev = pci_dev; rc = roc_nix_dev_init(nix); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index ba2bfcd..97e3a15 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -9,11 +9,15 @@ #include #include +#include #include "roc_api.h" #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Max supported SQB count */ +#define CNXK_NIX_TX_MAX_SQB 512 + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -38,6 +42,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + bool scalar_ena; /* Pointer back to rte */ struct rte_eth_dev *eth_dev; @@ -73,4 +78,8 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +/* Devargs */ +int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, + struct cnxk_eth_dev *dev); + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c new file mode 100644 index 0000000..4af2803 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -0,0 +1,166 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include +#include + +#include "cnxk_ethdev.h" + +static int +parse_flow_max_priority(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the max priority to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flow_prealloc_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the prealloc size to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_reta_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + if (val <= ETH_RSS_RETA_SIZE_64) + val = ROC_NIX_RSS_RETA_SZ_64; + else if (val > ETH_RSS_RETA_SIZE_64 && val <= ETH_RSS_RETA_SIZE_128) + val = ROC_NIX_RSS_RETA_SZ_128; + else if (val > ETH_RSS_RETA_SIZE_128 && val <= ETH_RSS_RETA_SIZE_256) + val = ROC_NIX_RSS_RETA_SZ_256; + else + val = ROC_NIX_RSS_RETA_SZ_64; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flag(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + *(uint16_t *)extra_args = atoi(value); + + return 0; +} + +static int +parse_sqb_count(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_switch_header_type(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + if (strcmp(value, "higig2") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_HIGIG; + + if (strcmp(value, "dsa") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_EDSA; + + if (strcmp(value, "chlen90b") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_LEN_90B; + return 0; +} + +#define CNXK_RSS_RETA_SIZE "reta_size" +#define CNXK_SCL_ENABLE "scalar_enable" +#define CNXK_MAX_SQB_COUNT "max_sqb_count" +#define CNXK_FLOW_PREALLOC_SIZE "flow_prealloc_size" +#define CNXK_FLOW_MAX_PRIORITY "flow_max_priority" +#define CNXK_SWITCH_HEADER_TYPE "switch_header" +#define CNXK_RSS_TAG_AS_XOR "tag_as_xor" + +int +cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) +{ + uint16_t reta_sz = ROC_NIX_RSS_RETA_SZ_64; + uint16_t sqb_count = CNXK_NIX_TX_MAX_SQB; + uint16_t flow_prealloc_size = 8; + uint16_t switch_header_type = 0; + uint16_t flow_max_priority = 3; + uint16_t rss_tag_as_xor = 0; + uint16_t scalar_enable = 0; + struct rte_kvargs *kvlist; + + if (devargs == NULL) + goto null_devargs; + + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + goto exit; + + rte_kvargs_process(kvlist, CNXK_RSS_RETA_SIZE, &parse_reta_size, + &reta_sz); + rte_kvargs_process(kvlist, CNXK_SCL_ENABLE, &parse_flag, + &scalar_enable); + rte_kvargs_process(kvlist, CNXK_MAX_SQB_COUNT, &parse_sqb_count, + &sqb_count); + rte_kvargs_process(kvlist, CNXK_FLOW_PREALLOC_SIZE, + &parse_flow_prealloc_size, &flow_prealloc_size); + rte_kvargs_process(kvlist, CNXK_FLOW_MAX_PRIORITY, + &parse_flow_max_priority, &flow_max_priority); + rte_kvargs_process(kvlist, CNXK_SWITCH_HEADER_TYPE, + &parse_switch_header_type, &switch_header_type); + rte_kvargs_process(kvlist, CNXK_RSS_TAG_AS_XOR, &parse_flag, + &rss_tag_as_xor); + rte_kvargs_free(kvlist); + +null_devargs: + dev->scalar_ena = !!scalar_enable; + dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; + dev->nix.max_sqb_count = sqb_count; + dev->nix.reta_sz = reta_sz; + return 0; + +exit: + return -EINVAL; +} + +RTE_PMD_REGISTER_PARAM_STRING(net_cnxk, + CNXK_RSS_RETA_SIZE "=<64|128|256>" + CNXK_SCL_ENABLE "=1" + CNXK_MAX_SQB_COUNT "=<8-512>" + CNXK_FLOW_PREALLOC_SIZE "=<1-32>" + CNXK_FLOW_MAX_PRIORITY "=<1-32>" + CNXK_SWITCH_HEADER_TYPE "=" + CNXK_RSS_TAG_AS_XOR "=1"); diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 57dea5e..00c4722 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -8,7 +8,8 @@ if not dpdk_conf.get('RTE_ARCH_64') subdir_done() endif -sources = files('cnxk_ethdev.c') +sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_devargs.c') # CN9K sources += files('cn9k_ethdev.c') From patchwork Sat Mar 6 15:33:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88601 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 13FDFA0548; Sat, 6 Mar 2021 16:34:52 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 95EEA22A362; Sat, 6 Mar 2021 16:34:30 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 821FF22A360 for ; Sat, 6 Mar 2021 16:34:28 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVumE029368 for ; Sat, 6 Mar 2021 07:34:27 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=gBvNNGaysWKNJJ+HUfyRK5IOMRjgeUUwoqMz3OjXnME=; b=E+0jKrG54Titksf2QOCQArxJQZm61QnJfwDQ8M2CPwtEvpcMob6j78yh2yW+eOC2E2eq /IADz1PK49n6tZ0ccwIZ/gNYeRaQRlZJN/2TiIDLTHeitXvMvzXzPiY46VGpmowYhY01 VtPjReLkw2+m9ArQTN0VTosqyyOYujq0EiM/WtWtYpb2FQ7f1M8WoAiZpfwUvDG8xKV2 IPaXY/e47Xj2N2xhIY30KlXzMwtdm/kNv8RzOOHS+OH+SicSNd04GVLpU3kKIl2yEMSe zgyv9TrB7sWQ/sugwlTyFK0rYOzxA7HPOU5ZrAeZfzx6MTOP7LSgzG3p0z5Ab3yCkJ8i 2Q== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcag-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:27 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:26 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:25 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:25 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 076433F703F; Sat, 6 Mar 2021 07:34:22 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:24 +0530 Message-ID: <20210306153404.10781-5-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 04/44] net/cnxk: add common dev infos get support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support to retrieve dev infos get for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/cnxk.rst | 3 ++ doc/guides/nics/features/cnxk.ini | 4 ++ doc/guides/nics/features/cnxk_vec.ini | 4 ++ doc/guides/nics/features/cnxk_vf.ini | 3 ++ drivers/net/cnxk/cnxk_ethdev.c | 4 +- drivers/net/cnxk/cnxk_ethdev.h | 33 ++++++++++++++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 71 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 1 + 8 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 611ffb4..dfe2e7a 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,9 @@ Features Features of the CNXK Ethdev PMD are: +- SR-IOV VF +- Lock-free Tx queue + Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 2c23464..b426340 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index de78516..292ac1e 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 9c96351..bc2eb8a 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -4,6 +4,9 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 3a2309e..1567007 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -38,7 +38,9 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) } /* CNXK platform independent eth dev ops */ -struct eth_dev_ops cnxk_eth_dev_ops; +struct eth_dev_ops cnxk_eth_dev_ops = { + .dev_infos_get = cnxk_nix_info_get, +}; static int cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 97e3a15..8d9a7e0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,9 +15,40 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* VLAN tag inserted by NIX_TX_VTAG_ACTION. + * In Tx space is always reserved for this in FRS. + */ +#define CNXK_NIX_MAX_VTAG_INS 2 +#define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) + +/* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) + +#define CNXK_NIX_RX_MIN_DESC 16 +#define CNXK_NIX_RX_MIN_DESC_ALIGN 16 +#define CNXK_NIX_RX_NB_SEG_MAX 6 +#define CNXK_NIX_RX_DEFAULT_RING_SZ 4096 /* Max supported SQB count */ #define CNXK_NIX_TX_MAX_SQB 512 +/* If PTP is enabled additional SEND MEM DESC is required which + * takes 2 words, hence max 7 iova address are possible + */ +#if defined(RTE_LIBRTE_IEEE1588) +#define CNXK_NIX_TX_NB_SEG_MAX 7 +#else +#define CNXK_NIX_TX_NB_SEG_MAX 9 +#endif + +#define CNXK_NIX_RSS_L3_L4_SRC_DST \ + (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ + ETH_RSS_L4_DST_ONLY) + +#define CNXK_NIX_RSS_OFFLOAD \ + (ETH_RSS_PORT | ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP | \ + ETH_RSS_SCTP | ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD | \ + CNXK_NIX_RSS_L3_L4_SRC_DST | ETH_RSS_LEVEL_MASK | ETH_RSS_C_VLAN) + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -77,6 +108,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_info *dev_info); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c new file mode 100644 index 0000000..4a45956 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +int +cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) +{ + struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int max_rx_pktlen; + + max_rx_pktlen = (roc_nix_max_pkt_len(&dev->nix) + RTE_ETHER_CRC_LEN - + CNXK_NIX_MAX_VTAG_ACT_SIZE); + + devinfo->min_rx_bufsize = NIX_MIN_HW_FRS + RTE_ETHER_CRC_LEN; + devinfo->max_rx_pktlen = max_rx_pktlen; + devinfo->max_rx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_mac_addrs = dev->max_mac_entries; + devinfo->max_vfs = pci_dev->max_vfs; + devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; + + devinfo->rx_offload_capa = dev->rx_offload_capa; + devinfo->tx_offload_capa = dev->tx_offload_capa; + devinfo->rx_queue_offload_capa = 0; + devinfo->tx_queue_offload_capa = 0; + + devinfo->reta_size = dev->nix.reta_sz; + devinfo->hash_key_size = ROC_NIX_RSS_KEY_LEN; + devinfo->flow_type_rss_offloads = CNXK_NIX_RSS_OFFLOAD; + + devinfo->default_rxconf = (struct rte_eth_rxconf){ + .rx_drop_en = 0, + .offloads = 0, + }; + + devinfo->default_txconf = (struct rte_eth_txconf){ + .offloads = 0, + }; + + devinfo->default_rxportconf = (struct rte_eth_dev_portconf){ + .ring_size = CNXK_NIX_RX_DEFAULT_RING_SZ, + }; + + devinfo->rx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = CNXK_NIX_RX_MIN_DESC, + .nb_align = CNXK_NIX_RX_MIN_DESC_ALIGN, + .nb_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + }; + devinfo->rx_desc_lim.nb_max = + RTE_ALIGN_MUL_FLOOR(devinfo->rx_desc_lim.nb_max, + CNXK_NIX_RX_MIN_DESC_ALIGN); + + devinfo->tx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = 1, + .nb_align = 1, + .nb_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + }; + + devinfo->speed_capa = dev->speed_capa; + devinfo->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP | + RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 00c4722..45ccbe3 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -9,6 +9,7 @@ if not dpdk_conf.get('RTE_ARCH_64') endif sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c') # CN9K From patchwork Sat Mar 6 15:33:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88602 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 72B94A0548; Sat, 6 Mar 2021 16:35:03 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 48F1F22A309; Sat, 6 Mar 2021 16:34:33 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 89E9B22A367 for ; Sat, 6 Mar 2021 16:34:31 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FXJQi032533 for ; Sat, 6 Mar 2021 07:34:30 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=zPYIRCM+XQOtYwM/yDI1BAwHPbpllV82CqH8+rcXhes=; b=fb0UwKTt6ECk7w6gwBMtl4lcBpSjhKtPCvQJlsZqP+JJ7AONmE/LzahJWUita08nK6j3 YrYtGmif0dU2nSO0H0zCib+glwVOTXV8kOlxPybyaiWMfKook1qw0i3abrNxCEh03ky2 bsSXalOxb8MzCy9O/ouND5q5tFxL3ggPaRt0qu8CfuRggGMBAmhpKX6kkvGdEyafs2zC POrsPw8F8V5OvhhR3nnNZ/Hsvt0s1x5otXXCQMeOvFWHcuTWfnzJUvaglrORK+CoMw1h 7XyqrdLjdqbflbReCaQpnMp53LGOuLOR1iMTOWlYqg32sOJs53c87UOCkCK70Pni0reU ZQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcak-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:30 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:28 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:29 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 4F9AB3F703F; Sat, 6 Mar 2021 07:34:26 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:25 +0530 Message-ID: <20210306153404.10781-6-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 05/44] net/cnxk: add device configuration operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add device configuration op for CN9K and CN10K. Most of the device configuration is common between two platforms except for some supported offloads. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/cnxk.rst | 2 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cn10k_ethdev.c | 34 +++ drivers/net/cnxk/cn9k_ethdev.c | 45 +++ drivers/net/cnxk/cnxk_ethdev.c | 521 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 70 +++++ 8 files changed, 678 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index dfe2e7a..73eb62a 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -18,6 +18,8 @@ Features of the CNXK Ethdev PMD are: - SR-IOV VF - Lock-free Tx queue +- Multiple queues for TX and RX +- Receiver Side Scaling (RSS) Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b426340..96dba2a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 292ac1e..616991c 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index bc2eb8a..a0bd2f1 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 54711ea..9cf0f9e 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,38 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; +} + +static int cn10k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -26,6 +58,8 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index bd97d5f..4f50949 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_conf *conf = ð_dev->data->dev_conf; + struct rte_eth_txmode *txmode = &conf->txmode; + int rc; + + /* Platform specific checks */ + if ((roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) && + (txmode->offloads & DEV_TX_OFFLOAD_SCTP_CKSUM) && + ((txmode->offloads & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM) || + (txmode->offloads & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM))) { + plt_err("Outer IP and SCTP checksum unsupported"); + return -EINVAL; + } + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; +} + +static int cn9k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -27,6 +70,8 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 1567007..f141027 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,520 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint32_t +cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level) +{ + uint32_t flow_key_type[RSS_MAX_LEVELS][6] = { + {FLOW_KEY_TYPE_IPV4, FLOW_KEY_TYPE_IPV6, FLOW_KEY_TYPE_TCP, + FLOW_KEY_TYPE_UDP, FLOW_KEY_TYPE_SCTP, FLOW_KEY_TYPE_ETH_DMAC}, + {FLOW_KEY_TYPE_INNR_IPV4, FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_INNR_TCP, FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_INNR_SCTP, FLOW_KEY_TYPE_INNR_ETH_DMAC}, + {FLOW_KEY_TYPE_IPV4 | FLOW_KEY_TYPE_INNR_IPV4, + FLOW_KEY_TYPE_IPV6 | FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_TCP | FLOW_KEY_TYPE_INNR_TCP, + FLOW_KEY_TYPE_UDP | FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_SCTP | FLOW_KEY_TYPE_INNR_SCTP, + FLOW_KEY_TYPE_ETH_DMAC | FLOW_KEY_TYPE_INNR_ETH_DMAC} + }; + uint32_t flowkey_cfg = 0; + + dev->ethdev_rss_hf = ethdev_rss; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + + if (ethdev_rss & ETH_RSS_C_VLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VLAN; + + if (ethdev_rss & ETH_RSS_L3_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC; + + if (ethdev_rss & ETH_RSS_L3_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_DST; + + if (ethdev_rss & ETH_RSS_L4_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC; + + if (ethdev_rss & ETH_RSS_L4_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_DST; + + if (ethdev_rss & RSS_IPV4_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV4_INDEX]; + + if (ethdev_rss & RSS_IPV6_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV6_INDEX]; + + if (ethdev_rss & ETH_RSS_TCP) + flowkey_cfg |= flow_key_type[rss_level][RSS_TCP_INDEX]; + + if (ethdev_rss & ETH_RSS_UDP) + flowkey_cfg |= flow_key_type[rss_level][RSS_UDP_INDEX]; + + if (ethdev_rss & ETH_RSS_SCTP) + flowkey_cfg |= flow_key_type[rss_level][RSS_SCTP_INDEX]; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= flow_key_type[rss_level][RSS_DMAC_INDEX]; + + if (ethdev_rss & RSS_IPV6_EX_ENABLE) + flowkey_cfg |= FLOW_KEY_TYPE_IPV6_EXT; + + if (ethdev_rss & ETH_RSS_PORT) + flowkey_cfg |= FLOW_KEY_TYPE_PORT; + + if (ethdev_rss & ETH_RSS_NVGRE) + flowkey_cfg |= FLOW_KEY_TYPE_NVGRE; + + if (ethdev_rss & ETH_RSS_VXLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VXLAN; + + if (ethdev_rss & ETH_RSS_GENEVE) + flowkey_cfg |= FLOW_KEY_TYPE_GENEVE; + + if (ethdev_rss & ETH_RSS_GTPU) + flowkey_cfg |= FLOW_KEY_TYPE_GTPU; + + return flowkey_cfg; +} + +static void +nix_free_queue_mem(struct cnxk_eth_dev *dev) +{ + plt_free(dev->rqs); + plt_free(dev->cqs); + plt_free(dev->sqs); + dev->rqs = NULL; + dev->cqs = NULL; + dev->sqs = NULL; +} + +static int +nix_rss_default_setup(struct cnxk_eth_dev *dev) +{ + struct rte_eth_dev *eth_dev = dev->eth_dev; + uint8_t rss_hash_level; + uint32_t flowkey_cfg; + uint64_t rss_hf; + + rss_hf = eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_hash_level = ETH_RSS_LEVEL(rss_hf); + if (rss_hash_level) + rss_hash_level -= 1; + + flowkey_cfg = cnxk_rss_ethdev_to_nix(dev, rss_hf, rss_hash_level); + return roc_nix_rss_default_setup(&dev->nix, flowkey_cfg); +} + +static int +nix_store_queue_cfg_and_then_release(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = NULL; + struct cnxk_eth_qconf *rx_qconf = NULL; + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_txq_sp *txq_sp; + int i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + tx_qconf = malloc(nb_txq * sizeof(*tx_qconf)); + if (tx_qconf == NULL) { + plt_err("Failed to allocate memory for tx_qconf"); + goto fail; + } + + rx_qconf = malloc(nb_rxq * sizeof(*rx_qconf)); + if (rx_qconf == NULL) { + plt_err("Failed to allocate memory for rx_qconf"); + goto fail; + } + + txq = eth_dev->data->tx_queues; + for (i = 0; i < nb_txq; i++) { + if (txq[i] == NULL) { + tx_qconf[i].valid = false; + plt_info("txq[%d] is already released", i); + continue; + } + txq_sp = ((struct cnxk_eth_txq_sp *)txq[i]) - 1; + memcpy(&tx_qconf[i], &txq_sp->qconf, sizeof(*tx_qconf)); + tx_qconf[i].valid = true; + dev_ops->tx_queue_release(txq[i]); + eth_dev->data->tx_queues[i] = NULL; + } + + rxq = eth_dev->data->rx_queues; + for (i = 0; i < nb_rxq; i++) { + if (rxq[i] == NULL) { + rx_qconf[i].valid = false; + plt_info("rxq[%d] is already released", i); + continue; + } + rxq_sp = ((struct cnxk_eth_rxq_sp *)rxq[i]) - 1; + memcpy(&rx_qconf[i], &rxq_sp->qconf, sizeof(*rx_qconf)); + rx_qconf[i].valid = true; + dev_ops->rx_queue_release(rxq[i]); + eth_dev->data->rx_queues[i] = NULL; + } + + dev->tx_qconf = tx_qconf; + dev->rx_qconf = rx_qconf; + return 0; + +fail: + free(tx_qconf); + free(rx_qconf); + return -ENOMEM; +} + +static int +nix_restore_queue_cfg(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = dev->tx_qconf; + struct cnxk_eth_qconf *rx_qconf = dev->rx_qconf; + int rc, i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + rc = -ENOMEM; + /* Setup tx & rx queues with previous configuration so + * that the queues can be functional in cases like ports + * are started without re configuring queues. + * + * Usual re config sequence is like below: + * port_configure() { + * if(reconfigure) { + * queue_release() + * queue_setup() + * } + * queue_configure() { + * queue_release() + * queue_setup() + * } + * } + * port_start() + * + * In some application's control path, queue_configure() would + * NOT be invoked for TXQs/RXQs in port_configure(). + * In such cases, queues can be functional after start as the + * queues are already setup in port_configure(). + */ + for (i = 0; i < nb_txq; i++) { + if (!tx_qconf[i].valid) + continue; + rc = dev_ops->tx_queue_setup(eth_dev, i, tx_qconf[i].nb_desc, 0, + &tx_qconf[i].conf.tx); + if (rc) { + plt_err("Failed to setup tx queue rc=%d", rc); + txq = eth_dev->data->tx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->tx_queue_release(txq[i]); + goto fail; + } + } + + free(tx_qconf); + tx_qconf = NULL; + + for (i = 0; i < nb_rxq; i++) { + if (!rx_qconf[i].valid) + continue; + rc = dev_ops->rx_queue_setup(eth_dev, i, rx_qconf[i].nb_desc, 0, + &rx_qconf[i].conf.rx, + rx_qconf[i].mp); + if (rc) { + plt_err("Failed to setup rx queue rc=%d", rc); + rxq = eth_dev->data->rx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->rx_queue_release(rxq[i]); + goto tx_queue_release; + } + } + + free(rx_qconf); + rx_qconf = NULL; + + return 0; + +tx_queue_release: + txq = eth_dev->data->tx_queues; + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_release(txq[i]); +fail: + if (tx_qconf) + free(tx_qconf); + if (rx_qconf) + free(rx_qconf); + + return rc; +} + +static uint16_t +nix_eth_nop_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + RTE_SET_USED(queue); + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + return 0; +} + +static void +nix_set_nop_rxtx_function(struct rte_eth_dev *eth_dev) +{ + /* These dummy functions are required for supporting + * some applications which reconfigure queues without + * stopping tx burst and rx burst threads(eg kni app) + * When the queues context is saved, txq/rxqs are released + * which caused app crash since rx/tx burst is still + * on different lcores + */ + eth_dev->tx_pkt_burst = nix_eth_nop_burst; + eth_dev->rx_pkt_burst = nix_eth_nop_burst; + rte_mb(); +} + +int +cnxk_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + struct rte_eth_txmode *txmode = &conf->txmode; + char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix *nix = &dev->nix; + struct rte_ether_addr *ea; + uint8_t nb_rxq, nb_txq; + uint64_t rx_cfg; + void *qs; + int rc; + + rc = -EINVAL; + + /* Sanity checks */ + if (rte_eal_has_hugepages() == 0) { + plt_err("Huge page is not configured"); + goto fail_configure; + } + + if (conf->dcb_capability_en == 1) { + plt_err("dcb enable is not supported"); + goto fail_configure; + } + + if (conf->fdir_conf.mode != RTE_FDIR_MODE_NONE) { + plt_err("Flow director is not supported"); + goto fail_configure; + } + + if (rxmode->mq_mode != ETH_MQ_RX_NONE && + rxmode->mq_mode != ETH_MQ_RX_RSS) { + plt_err("Unsupported mq rx mode %d", rxmode->mq_mode); + goto fail_configure; + } + + if (txmode->mq_mode != ETH_MQ_TX_NONE) { + plt_err("Unsupported mq tx mode %d", txmode->mq_mode); + goto fail_configure; + } + + /* Free the resources allocated from the previous configure */ + if (dev->configured == 1) { + /* Unregister queue irq's */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister CQ irqs if present */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Set no-op functions */ + nix_set_nop_rxtx_function(eth_dev); + /* Store queue config for later */ + rc = nix_store_queue_cfg_and_then_release(eth_dev); + if (rc) + goto fail_configure; + roc_nix_tm_fini(nix); + roc_nix_lf_free(nix); + } + + dev->rx_offloads = rxmode->offloads; + dev->tx_offloads = txmode->offloads; + + /* Prepare rx cfg */ + rx_cfg = ROC_NIX_LF_RX_CFG_DIS_APAD; + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) { + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_OL4; + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_IL4; + } + rx_cfg |= (ROC_NIX_LF_RX_CFG_DROP_RE | ROC_NIX_LF_RX_CFG_L2_LEN_ERR | + ROC_NIX_LF_RX_CFG_LEN_IL4 | ROC_NIX_LF_RX_CFG_LEN_IL3 | + ROC_NIX_LF_RX_CFG_LEN_OL4 | ROC_NIX_LF_RX_CFG_LEN_OL3); + + nb_rxq = RTE_MAX(data->nb_rx_queues, 1); + nb_txq = RTE_MAX(data->nb_tx_queues, 1); + + /* Alloc a nix lf */ + rc = roc_nix_lf_alloc(nix, nb_rxq, nb_txq, rx_cfg); + if (rc) { + plt_err("Failed to init nix_lf rc=%d", rc); + goto fail_configure; + } + + nb_rxq = data->nb_rx_queues; + nb_txq = data->nb_tx_queues; + rc = -ENOMEM; + if (nb_rxq) { + /* Allocate memory for roc rq's and cq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_rq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc rqs"); + goto free_nix_lf; + } + dev->rqs = qs; + + qs = plt_zmalloc(sizeof(struct roc_nix_cq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc cqs"); + goto free_nix_lf; + } + dev->cqs = qs; + } + + if (nb_txq) { + /* Allocate memory for roc sq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_sq) * nb_txq, 0); + if (!qs) { + plt_err("Failed to alloc sqs"); + goto free_nix_lf; + } + dev->sqs = qs; + } + + /* Re-enable NIX LF error interrupts */ + roc_nix_err_intr_ena_dis(nix, true); + roc_nix_ras_intr_ena_dis(nix, true); + + if (nix->rx_ptp_ena) { + plt_err("Both PTP and switch header enabled"); + goto free_nix_lf; + } + + rc = roc_nix_lso_fmt_setup(nix); + if (rc) { + plt_err("failed to setup nix lso format fields, rc=%d", rc); + goto free_nix_lf; + } + + /* Configure RSS */ + rc = nix_rss_default_setup(dev); + if (rc) { + plt_err("Failed to configure rss rc=%d", rc); + goto free_nix_lf; + } + + /* Init the default TM scheduler hierarchy */ + rc = roc_nix_tm_init(nix); + if (rc) { + plt_err("Failed to init traffic manager, rc=%d", rc); + goto free_nix_lf; + } + + rc = roc_nix_tm_hierarchy_enable(nix, ROC_NIX_TM_DEFAULT, false); + if (rc) { + plt_err("Failed to enable default tm hierarchy, rc=%d", rc); + goto tm_fini; + } + + /* Register queue IRQs */ + rc = roc_nix_register_queue_irqs(nix); + if (rc) { + plt_err("Failed to register queue interrupts rc=%d", rc); + goto tm_fini; + } + + /* Register cq IRQs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) { + if (eth_dev->data->nb_rx_queues > dev->nix.cints) { + plt_err("Rx interrupt cannot be enabled, rxq > %d", + dev->nix.cints); + goto q_irq_fini; + } + /* Rx interrupt feature cannot work with vector mode because, + * vector mode does not process packets unless min 4 pkts are + * received, while cq interrupts are generated even for 1 pkt + * in the CQ. + */ + dev->scalar_ena = true; + + rc = roc_nix_register_cq_irqs(nix); + if (rc) { + plt_err("Failed to register CQ interrupts rc=%d", rc); + goto q_irq_fini; + } + } + + /* Configure loop back mode */ + rc = roc_nix_mac_loopback_enable(nix, + eth_dev->data->dev_conf.lpbk_mode); + if (rc) { + plt_err("Failed to configure cgx loop back mode rc=%d", rc); + goto cq_fini; + } + + /* + * Restore queue config when reconfigure followed by + * reconfigure and no queue configure invoked from application case. + */ + if (dev->configured == 1) { + rc = nix_restore_queue_cfg(eth_dev); + if (rc) + goto cq_fini; + } + + /* Update the mac address */ + ea = eth_dev->data->mac_addrs; + memcpy(ea, dev->mac_addr, RTE_ETHER_ADDR_LEN); + if (rte_is_zero_ether_addr(ea)) + rte_eth_random_addr((uint8_t *)ea); + + rte_ether_format_addr(ea_fmt, RTE_ETHER_ADDR_FMT_SIZE, ea); + + plt_nix_dbg("Configured port%d mac=%s nb_rxq=%d nb_txq=%d" + " rx_offloads=0x%" PRIx64 " tx_offloads=0x%" PRIx64 "", + eth_dev->data->port_id, ea_fmt, nb_rxq, nb_txq, + dev->rx_offloads, dev->tx_offloads); + + /* All good */ + dev->configured = 1; + dev->nb_rxq = data->nb_rx_queues; + dev->nb_txq = data->nb_tx_queues; + return 0; + +cq_fini: + roc_nix_unregister_cq_irqs(nix); +q_irq_fini: + roc_nix_unregister_queue_irqs(nix); +tm_fini: + roc_nix_tm_fini(nix); +free_nix_lf: + nix_free_queue_mem(dev); + rc |= roc_nix_lf_free(nix); +fail_configure: + dev->configured = 0; + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, @@ -76,6 +590,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } dev->eth_dev = eth_dev; + dev->configured = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. @@ -157,6 +672,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (rte_eal_process_type() != RTE_PROC_PRIMARY) return 0; + /* Clear the flag since we are closing down */ + dev->configured = 0; + roc_nix_npc_rx_ena_dis(nix, false); /* Free up SQs */ @@ -183,6 +701,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (eth_dev->data->dev_conf.intr_conf.rxq) roc_nix_unregister_cq_irqs(nix); + /* Free ROC RQ's, SQ's and CQ's memory */ + nix_free_queue_mem(dev); + /* Free nix lf resources */ rc = roc_nix_lf_free(nix); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 8d9a7e0..55da1da 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -65,10 +65,50 @@ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ DEV_RX_OFFLOAD_RSS_HASH) +#define RSS_IPV4_ENABLE \ + (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ + ETH_RSS_NONFRAG_IPV4_TCP | ETH_RSS_NONFRAG_IPV4_SCTP) + +#define RSS_IPV6_ENABLE \ + (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 | ETH_RSS_NONFRAG_IPV6_UDP | \ + ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_NONFRAG_IPV6_SCTP) + +#define RSS_IPV6_EX_ENABLE \ + (ETH_RSS_IPV6_EX | ETH_RSS_IPV6_TCP_EX | ETH_RSS_IPV6_UDP_EX) + +#define RSS_MAX_LEVELS 3 + +#define RSS_IPV4_INDEX 0 +#define RSS_IPV6_INDEX 1 +#define RSS_TCP_INDEX 2 +#define RSS_UDP_INDEX 3 +#define RSS_SCTP_INDEX 4 +#define RSS_DMAC_INDEX 5 + +struct cnxk_eth_qconf { + union { + struct rte_eth_txconf tx; + struct rte_eth_rxconf rx; + } conf; + struct rte_mempool *mp; + uint16_t nb_desc; + uint8_t valid; +}; + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC RQs, SQs and CQs */ + struct roc_nix_rq *rqs; + struct roc_nix_sq *sqs; + struct roc_nix_cq *cqs; + + /* Configured queue count */ + uint16_t nb_rxq; + uint16_t nb_txq; + uint8_t configured; + /* Max macfilter entries */ uint8_t max_mac_entries; @@ -90,11 +130,36 @@ struct cnxk_eth_dev { uint64_t rx_offload_capa; uint64_t tx_offload_capa; uint32_t speed_capa; + /* Configured Rx and Tx offloads */ + uint64_t rx_offloads; + uint64_t tx_offloads; + /* Platform specific offload flags */ + uint16_t rx_offload_flags; + uint16_t tx_offload_flags; + + /* ETHDEV RSS HF bitmask */ + uint64_t ethdev_rss_hf; + + /* Saved qconf before lf realloc */ + struct cnxk_eth_qconf *tx_qconf; + struct cnxk_eth_qconf *rx_qconf; /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; }; +struct cnxk_eth_rxq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + +struct cnxk_eth_txq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + static inline struct cnxk_eth_dev * cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) { @@ -110,6 +175,11 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_configure(struct rte_eth_dev *eth_dev); + +/* RSS */ +uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, From patchwork Sat Mar 6 15:33:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88603 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7280BA0548; Sat, 6 Mar 2021 16:35:13 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 71DEA22A37E; Sat, 6 Mar 2021 16:34:36 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 1317C22A36A for ; Sat, 6 Mar 2021 16:34:34 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVt9F029362 for ; Sat, 6 Mar 2021 07:34:34 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=C5PO1ATumrym2ITvJ+N3wnUHZHBm8QtWc+wrca44SjA=; b=V8r5pUirE9qwk5QaSuSOGnII2fwv6c8PGu63OunPpWrGvg3M9wPORfHcoCExnY7z1MF0 HN2e7Lm7PTcrHC1b04ejgi2JN1I78hstYsP8kHyfpnPlRZBUF1+Nj0GXZ6pGemBFBJ4Z ruBV4w06vGBWq15m5LTthAc1a5rIEURin2F0HPte64atnHxCDE82Dyohzoe91R0KIYgP d7n/ViQI/Se858LBwYI7+snrNIV0FkSy+6oS9WqWpxHU8bNDmOm9wbvAla5NA1cebGmm QiFemLCi2CIxy3nzQb8PMAycGzs1KOqJUPWJVMUBi8L8Ezu+ixtYFAmLGgDlH2Z9MVES bA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcat-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:34 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:32 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:32 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:32 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 914C03F7040; Sat, 6 Mar 2021 07:34:29 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:26 +0530 Message-ID: <20210306153404.10781-7-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 06/44] net/cnxk: add link status update support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add link status update callback to get current link status. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 7 +++ drivers/net/cnxk/cnxk_ethdev.h | 8 +++ drivers/net/cnxk/cnxk_link.c | 102 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 126 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_link.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 73eb62a..a982450 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -20,6 +20,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Link state information Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 96dba2a..affbbd9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 616991c..836cc9f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a0bd2f1..29bb24f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f141027..c07827c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -554,6 +554,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, + .link_update = cnxk_nix_link_update, }; static int @@ -589,6 +590,9 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) goto error; } + /* Register up msg callbacks */ + roc_nix_mac_link_cb_register(nix, cnxk_eth_dev_link_status_cb); + dev->eth_dev = eth_dev; dev->configured = 0; @@ -677,6 +681,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable link status events */ + roc_nix_mac_link_event_start_stop(nix, false); + /* Free up SQs */ for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 55da1da..6dad8ac 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,6 +15,9 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Used for struct cnxk_eth_dev::flags */ +#define CNXK_LINK_CFG_IN_PROGRESS_F BIT_ULL(0) + /* VLAN tag inserted by NIX_TX_VTAG_ACTION. * In Tx space is always reserved for this in FRS. */ @@ -181,6 +184,11 @@ int cnxk_nix_configure(struct rte_eth_dev *eth_dev); uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); +/* Link */ +void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, + struct roc_nix_link_info *link); +int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c new file mode 100644 index 0000000..0223d68 --- /dev/null +++ b/drivers/net/cnxk/cnxk_link.c @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +static inline int +nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) +{ + uint16_t wait = 1000; + + do { + rte_rmb(); + if (!(dev->flags & CNXK_LINK_CFG_IN_PROGRESS_F)) + break; + wait--; + rte_delay_ms(1); + } while (wait); + + return wait ? 0 : -1; +} + +static void +nix_link_status_print(struct rte_eth_dev *eth_dev, struct rte_eth_link *link) +{ + if (link && link->link_status) + plt_info("Port %d: Link Up - speed %u Mbps - %s", + (int)(eth_dev->data->port_id), + (uint32_t)link->link_speed, + link->link_duplex == ETH_LINK_FULL_DUPLEX + ? "full-duplex" + : "half-duplex"); + else + plt_info("Port %d: Link Down", (int)(eth_dev->data->port_id)); +} + +void +cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_link eth_link; + struct rte_eth_dev *eth_dev; + + if (!link || !nix) + return; + + eth_dev = dev->eth_dev; + if (!eth_dev || !eth_dev->data->dev_conf.intr_conf.lsc) + return; + + if (nix_wait_for_link_cfg(dev)) { + plt_err("Timeout waiting for link_cfg to complete"); + return; + } + + eth_link.link_status = link->status; + eth_link.link_speed = link->speed; + eth_link.link_autoneg = ETH_LINK_AUTONEG; + eth_link.link_duplex = link->full_duplex; + + /* Print link info */ + nix_link_status_print(eth_dev, ð_link); + + /* Update link info */ + rte_eth_linkstatus_set(eth_dev, ð_link); + + /* Set the flag and execute application callbacks */ + rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_INTR_LSC, NULL); +} + +int +cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_link_info info; + struct rte_eth_link link; + int rc; + + RTE_SET_USED(wait_to_complete); + memset(&link, 0, sizeof(struct rte_eth_link)); + + if (roc_nix_is_sdp(&dev->nix)) + return 0; + + if (roc_nix_is_lbk(&dev->nix)) { + link.link_status = ETH_LINK_UP; + link.link_speed = ETH_SPEED_NUM_100G; + link.link_autoneg = ETH_LINK_FIXED; + link.link_duplex = ETH_LINK_FULL_DUPLEX; + } else { + rc = roc_nix_mac_link_info_get(&dev->nix, &info); + if (rc) + return rc; + link.link_status = info.status; + link.link_speed = info.speed; + link.link_autoneg = ETH_LINK_AUTONEG; + if (info.full_duplex) + link.link_duplex = info.full_duplex; + } + + return rte_eth_linkstatus_set(eth_dev, &link); +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 45ccbe3..c0ec91a 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,7 +10,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', - 'cnxk_ethdev_devargs.c') + 'cnxk_ethdev_devargs.c', + 'cnxk_link.c') # CN9K sources += files('cn9k_ethdev.c') From patchwork Sat Mar 6 15:33:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88604 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B1A03A054F; Sat, 6 Mar 2021 16:35:23 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0D9C322A372; Sat, 6 Mar 2021 16:34:40 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 4643622A384 for ; Sat, 6 Mar 2021 16:34:38 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWlRR029538 for ; Sat, 6 Mar 2021 07:34:37 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=1JCiKM9k05I5puYx4+bAmmr811tOtmOnqniOSqalqgY=; b=bEmLWmonJo8MA70ejHTNgXPW0d2GM+cx+rgMkm3nuELvqf/HXhYdtDpMgaL8RkwzCQiU Cwt7FxXFgOnLzlqbZGGMJB71dIbog4l0YTuYuV5lbJVrLq7Qki+1u3WPw20lohPALGEw EzxQv5w1yNkZlaNmlaJ2dnkLeSocX4fB7nYW7hTA7KbgMrK144ODmOl29u8QirStavIU uyGaJbJIivpo/jt0oGFVbXkeM3jnSXcI2j+79i2mEv7aU4KQJtVPR/n/LO6qxUYEuZey WqMYmbLv3BmLDeOgcwaTe8O8XOy6dSVHhbD4n02S8IMy3U2rNHWk080il2BRgKh3oy2c 0g== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcb1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:37 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:36 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:35 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:35 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id DA9CC3F703F; Sat, 6 Mar 2021 07:34:32 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:27 +0530 Message-ID: <20210306153404.10781-8-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 07/44] net/cnxk: add Rx queue setup and release X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Rx queue setup and release op for CN9K and CN10K SoC. Release is completely common while setup is platform dependent due to fast path Rx queue structure variation. Fastpath is platform dependent partly due to core cacheline size difference. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn10k_ethdev.h | 14 +++ drivers/net/cnxk/cn9k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn9k_ethdev.h | 14 +++ drivers/net/cnxk/cnxk_ethdev.c | 172 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ 9 files changed, 300 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index affbbd9..a9d2b03 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 836cc9f..6a8ca1f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 29bb24f..f761638 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -9,6 +9,7 @@ Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 9cf0f9e..f7e2f7b 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,49 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -33,6 +76,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 1bf4a65..08e11bb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,4 +6,18 @@ #include +struct cn10k_eth_rxq { + uint64_t mbuf_initializer; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t data_off; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 4f50949..79c30aa 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -44,6 +87,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 15d9397..6384609 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,4 +6,18 @@ #include +struct cn9k_eth_rxq { + uint64_t mbuf_initializer; + uint64_t data_off; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index c07827c..dc41f78 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,177 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint64_t +cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) +{ + uint16_t port_id = dev->eth_dev->data->port_id; + struct rte_mbuf mb_def; + uint64_t *tmp; + + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) % 8 != 0); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) - + offsetof(struct rte_mbuf, data_off) != + 2); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, nb_segs) - + offsetof(struct rte_mbuf, data_off) != + 4); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - + offsetof(struct rte_mbuf, data_off) != + 6); + mb_def.nb_segs = 1; + mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.port = port_id; + rte_mbuf_refcnt_set(&mb_def, 1); + + /* Prevent compiler reordering: rearm_data covers previous fields */ + rte_compiler_barrier(); + tmp = (uint64_t *)&mb_def.rearm_data; + + return *tmp; +} + +int +cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_mempool_ops *ops; + const char *platform_ops; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t first_skip; + int rc = -EINVAL; + size_t rxq_sz; + + /* Sanity checks */ + if (rx_conf->rx_deferred_start == 1) { + plt_err("Deferred Rx start is not supported"); + goto fail; + } + + platform_ops = rte_mbuf_platform_mempool_ops(); + /* This driver needs cnxk_npa mempool ops to work */ + ops = rte_mempool_get_ops(mp->ops_index); + if (strncmp(ops->name, platform_ops, RTE_MEMPOOL_OPS_NAMESIZE)) { + plt_err("mempool ops should be of cnxk_npa type"); + goto fail; + } + + if (mp->pool_id == 0) { + plt_err("Invalid pool_id"); + goto fail; + } + + /* Free memory prior to re-allocation if needed */ + if (eth_dev->data->rx_queues[qid] != NULL) { + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->rx_queue_release(eth_dev->data->rx_queues[qid]); + eth_dev->data->rx_queues[qid] = NULL; + } + + /* Setup ROC CQ */ + cq = &dev->cqs[qid]; + cq->qid = qid; + cq->nb_desc = nb_desc; + rc = roc_nix_cq_init(&dev->nix, cq); + if (rc) { + plt_err("Failed to init roc cq for rq=%d, rc=%d", qid, rc); + goto fail; + } + + /* Setup ROC RQ */ + rq = &dev->rqs[qid]; + rq->qid = qid; + rq->aura_handle = mp->pool_id; + rq->flow_tag_width = 32; + rq->sso_ena = false; + + /* Calculate first mbuf skip */ + first_skip = (sizeof(struct rte_mbuf)); + first_skip += RTE_PKTMBUF_HEADROOM; + first_skip += rte_pktmbuf_priv_size(mp); + rq->first_skip = first_skip; + rq->later_skip = sizeof(struct rte_mbuf); + rq->lpb_size = mp->elt_size; + + rc = roc_nix_rq_init(&dev->nix, rq, !!eth_dev->data->dev_started); + if (rc) { + plt_err("Failed to init roc rq for rq=%d, rc=%d", qid, rc); + goto cq_fini; + } + + /* Allocate and setup fast path rx queue */ + rc = -ENOMEM; + rxq_sz = sizeof(struct cnxk_eth_rxq_sp) + fp_rx_q_sz; + rxq_sp = plt_zmalloc(rxq_sz, PLT_CACHE_LINE_SIZE); + if (!rxq_sp) { + plt_err("Failed to alloc rx queue for rq=%d", qid); + goto rq_fini; + } + + /* Setup slow path fields */ + rxq_sp->dev = dev; + rxq_sp->qid = qid; + rxq_sp->qconf.conf.rx = *rx_conf; + rxq_sp->qconf.nb_desc = nb_desc; + rxq_sp->qconf.mp = mp; + + plt_nix_dbg("rq=%d pool=%s nb_desc=%d->%d", qid, mp->name, nb_desc, + cq->nb_desc); + + /* Store start of fast path area */ + eth_dev->data->rx_queues[qid] = rxq_sp + 1; + eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +rq_fini: + rc |= roc_nix_rq_fini(rq); +cq_fini: + rc |= roc_nix_cq_fini(cq); +fail: + return rc; +} + +static void +cnxk_nix_rx_queue_release(void *rxq) +{ + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t qid; + int rc; + + if (!rxq) + return; + + rxq_sp = ((struct cnxk_eth_rxq_sp *)rxq) - 1; + dev = rxq_sp->dev; + qid = rxq_sp->qid; + + plt_nix_dbg("Releasing rxq %u", qid); + + /* Cleanup ROC RQ */ + rq = &dev->rqs[qid]; + rc = roc_nix_rq_fini(rq); + if (rc) + plt_err("Failed to cleanup rq, rc=%d", rc); + + /* Cleanup ROC CQ */ + cq = &dev->cqs[qid]; + rc = roc_nix_cq_fini(cq); + if (rc) + plt_err("Failed to cleanup cq, rc=%d", rc); + + /* Finally free fast path area */ + plt_free(rxq_sp); +} + uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level) @@ -555,6 +726,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .rx_queue_release = cnxk_nix_rx_queue_release, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 6dad8ac..e938c64 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -10,6 +10,9 @@ #include #include #include +#include +#include +#include #include "roc_api.h" @@ -179,6 +182,12 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); + +uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, From patchwork Sat Mar 6 15:33:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88605 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9099AA0548; Sat, 6 Mar 2021 16:35:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 45A1E22A386; Sat, 6 Mar 2021 16:34:43 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 81BAA22A386 for ; Sat, 6 Mar 2021 16:34:41 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmqN029543 for ; Sat, 6 Mar 2021 07:34:40 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=aQUxZJHrgQgfCqaNkekXn3+BWop/q1vK6XGuNScc5Bg=; b=N0iRO01IwmotLc7PNpZUyQ1C6xN8mZ2DCqKeTcqA3KsHD8lay4aewfYhXaR9BuzskNl0 45rl7bJaB24smca7NWkGcYwZFmOqsQgguZwj8JVRCuwMhKyJ1MqDkLJiTZIoHn/E42NL FRsYjRsm1xOlaxX4E4fs/KBXYUsE+rWhHWxoEkKkXguOlXBxCbHed0Lnd6W35ZgVSpo7 EYZQRw1hxduWbA0cXaK/V5Fm/bWm3PSPBh8nhO8pWHt2C4qi6TRw11UUDlTfFQoVV3XN nHnyl9ikRCXiiGw+twXNwe5h9V5/HTNNW7hyG/LllBfHjtwsqZfpbaHZ7yZT5Y5H/e3T Aw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcb6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:40 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:39 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:38 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:38 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 14F543F703F; Sat, 6 Mar 2021 07:34:35 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:28 +0530 Message-ID: <20210306153404.10781-9-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 08/44] net/cnxk: add Tx queue setup and release X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Tx queue setup and release for CN9K and CN10K. Release is common while setup is platform dependent due to differences in fast path Tx queue structures. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 71 +++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 12 +++++ drivers/net/cnxk/cn10k_tx.h | 13 +++++ drivers/net/cnxk/cn9k_ethdev.c | 69 ++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 10 ++++ drivers/net/cnxk/cn9k_tx.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev.c | 98 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 ++ 11 files changed, 292 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_tx.h create mode 100644 drivers/net/cnxk/cn9k_tx.h diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index a9d2b03..462d7c4 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 6a8ca1f..09e0d3a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index f761638..4a93a35 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -10,6 +10,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index f7e2f7b..e194b13 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,6 +2,76 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + union nix_send_hdr_w0_u send_hdr_w0; + union nix_send_sg_s sg_w0; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + send_hdr_w0.u = 0; + sg_w0.u = 0; + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr_w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[0]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + } else { + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr_w0.sizem1 = 1; + } + + send_hdr_w0.sq = qid; + sg_w0.subdc = NIX_SUBDC_SG; + sg_w0.segs = 1; + sg_w0.ld_type = NIX_SENDLDTYPE_LDD; + + txq->send_hdr_w0 = send_hdr_w0.u; + txq->sg_w0 = sg_w0.u; + + rte_wmb(); +} + +static int +cn10k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + /* Store lmt base in tx queue for easy access */ + txq->lmt_base = dev->nix.lmt_base; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + return 0; +} static int cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -76,6 +146,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 08e11bb..2157b16 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,6 +6,18 @@ #include +struct cn10k_eth_txq { + uint64_t send_hdr_w0; + uint64_t sg_w0; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + uintptr_t lmt_base; + rte_iova_t io_addr; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; + uint64_t cmd[4]; +} __plt_cache_aligned; + struct cn10k_eth_rxq { uint64_t mbuf_initializer; uintptr_t desc; diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h new file mode 100644 index 0000000..39d4755 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_TX_H__ +#define __CN10K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 79c30aa..e97ce15 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,6 +2,74 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr->w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[2]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + sg = (union nix_send_sg_s *)&txq->cmd[4]; + } else { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr->w0.sizem1 = 1; + sg = (union nix_send_sg_s *)&txq->cmd[2]; + } + + send_hdr->w0.sq = qid; + sg->subdc = NIX_SUBDC_SG; + sg->segs = 1; + sg->ld_type = NIX_SENDLDTYPE_LDD; + + rte_wmb(); +} + +static int +cn9k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + txq->lmt_addr = sq->lmt_addr; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + return 0; +} static int cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -87,6 +155,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 6384609..9ebf68f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,6 +6,16 @@ #include +struct cn9k_eth_txq { + uint64_t cmd[8]; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + void *lmt_addr; + rte_iova_t io_addr; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; +} __plt_cache_aligned; + struct cn9k_eth_rxq { uint64_t mbuf_initializer; uint64_t data_off; diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h new file mode 100644 index 0000000..bb6379b --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_TX_H__ +#define __CN9K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index dc41f78..5772345 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -66,6 +66,103 @@ cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) return *tmp; } +static inline uint8_t +nix_sq_max_sqe_sz(struct cnxk_eth_dev *dev) +{ + /* + * Maximum three segments can be supported with W8, Choose + * NIX_MAXSQESZ_W16 for multi segment offload. + */ + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + return NIX_MAXSQESZ_W16; + else + return NIX_MAXSQESZ_W8; +} + +int +cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_txq_sp *txq_sp; + struct roc_nix_sq *sq; + size_t txq_sz; + int rc; + + /* Free memory prior to re-allocation if needed. */ + if (eth_dev->data->tx_queues[qid] != NULL) { + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->tx_queue_release(eth_dev->data->tx_queues[qid]); + eth_dev->data->tx_queues[qid] = NULL; + } + + /* Setup ROC SQ */ + sq = &dev->sqs[qid]; + sq->qid = qid; + sq->nb_desc = nb_desc; + sq->max_sqe_sz = nix_sq_max_sqe_sz(dev); + + rc = roc_nix_sq_init(&dev->nix, sq); + if (rc) { + plt_err("Failed to init sq=%d, rc=%d", qid, rc); + return rc; + } + + rc = -ENOMEM; + txq_sz = sizeof(struct cnxk_eth_txq_sp) + fp_tx_q_sz; + txq_sp = plt_zmalloc(txq_sz, PLT_CACHE_LINE_SIZE); + if (!txq_sp) { + plt_err("Failed to alloc tx queue mem"); + rc |= roc_nix_sq_fini(sq); + return rc; + } + + txq_sp->dev = dev; + txq_sp->qid = qid; + txq_sp->qconf.conf.tx = *tx_conf; + txq_sp->qconf.nb_desc = nb_desc; + + plt_nix_dbg("sq=%d fc=%p offload=0x%" PRIx64 " lmt_addr=%p" + " nb_sqb_bufs=%d sqes_per_sqb_log2=%d", + qid, sq->fc, dev->tx_offloads, sq->lmt_addr, + sq->nb_sqb_bufs, sq->sqes_per_sqb_log2); + + /* Store start of fast path area */ + eth_dev->data->tx_queues[qid] = txq_sp + 1; + eth_dev->data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + return 0; +} + +static void +cnxk_nix_tx_queue_release(void *txq) +{ + struct cnxk_eth_txq_sp *txq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_sq *sq; + uint16_t qid; + int rc; + + if (!txq) + return; + + txq_sp = ((struct cnxk_eth_txq_sp *)txq) - 1; + dev = txq_sp->dev; + qid = txq_sp->qid; + + plt_nix_dbg("Releasing txq %u", qid); + + /* Cleanup ROC SQ */ + sq = &dev->sqs[qid]; + rc = roc_nix_sq_fini(sq); + if (rc) + plt_err("Failed to cleanup sq, rc=%d", rc); + + /* Finally free */ + plt_free(txq_sp); +} + int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, @@ -726,6 +823,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e938c64..90c8ff6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -182,6 +182,9 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf); int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, From patchwork Sat Mar 6 15:33:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88606 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BE445A0548; Sat, 6 Mar 2021 16:35:45 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E7DCB22A397; Sat, 6 Mar 2021 16:34:45 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id BEC3522A397 for ; Sat, 6 Mar 2021 16:34:44 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVt9H029362 for ; Sat, 6 Mar 2021 07:34:44 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=JSyL3CL8HkI+btyqhlKkimyXPkrw49DrHvZuip6ihaE=; b=Su7qTESYG0RdZUj4e0+gCskr7CTg0dHvfflgddcl3OHw4EhXvHVBxNZ/3sJ8JLJO0xSL 8Fohy3Dn8ZWUSGEBUMQBfgOtrV1ka6J2tP0tQMPMS6XdVJmEafRQv33YMFtzgTrbGu8n 98ufofxBLxuH2a4pPzdl27EykhNz6tVnYFkAhNOD2o/p+SzGnwZttIq24RNtm/z1MFiI zJnJSkvebC1wMjpYlWRXmaaESrt6GthetS5q1V9Waex1YscVO9qHDj9StA3QrR8DAHxu JfeCXpsqLDzCqj+tGXe4hwhk25lVDF7n5y33z5Sj2WRep863jmlPChD+J3CFP7b/85zq Fw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcb9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:43 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:42 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:41 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:41 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 394673F703F; Sat, 6 Mar 2021 07:34:38 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:29 +0530 Message-ID: <20210306153404.10781-10-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 09/44] net/cnxk: add packet type support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support for packet type lookup on Rx to translate HW specific types to RTE_PTYPE_* defines Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 21 +++ drivers/net/cnxk/cn10k_rx.h | 11 ++ drivers/net/cnxk/cn9k_ethdev.c | 21 +++ drivers/net/cnxk/cn9k_rx.h | 12 ++ drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 14 ++ drivers/net/cnxk/cnxk_lookup.c | 326 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 12 files changed, 413 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.h create mode 100644 drivers/net/cnxk/cn9k_rx.h create mode 100644 drivers/net/cnxk/cnxk_lookup.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index a982450..4f1b58c 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,7 @@ Features Features of the CNXK Ethdev PMD are: +- Packet type information - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 462d7c4..503582c 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 09e0d3a..9ad225a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 4a93a35..8c93ba7 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -13,6 +13,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index e194b13..efd5b67 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_rx.h" #include "cn10k_tx.h" +static int +cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, uint16_t qid) @@ -113,6 +130,9 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -148,6 +168,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h new file mode 100644 index 0000000..d3d1661 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_RX_H__ +#define __CN10K_RX_H__ + +#include + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index e97ce15..3f3de4f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_rx.h" #include "cn9k_tx.h" +static int +cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, uint16_t qid) @@ -111,6 +128,9 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -157,6 +177,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h new file mode 100644 index 0000000..95a1e69 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef __CN9K_RX_H__ +#define __CN9K_RX_H__ + +#include + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 5772345..96acf90 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -825,6 +825,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; static int @@ -865,6 +866,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->eth_dev = eth_dev; dev->configured = 0; + dev->ptype_disable = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 90c8ff6..6b7261c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,15 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +#define PTYPE_NON_TUNNEL_WIDTH 16 +#define PTYPE_TUNNEL_WIDTH 12 +#define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) +#define PTYPE_TUNNEL_ARRAY_SZ BIT(PTYPE_TUNNEL_WIDTH) +#define PTYPE_ARRAY_SZ \ + ((PTYPE_NON_TUNNEL_ARRAY_SZ + PTYPE_TUNNEL_ARRAY_SZ) * sizeof(uint16_t)) +/* Fastpath lookup */ +#define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -119,6 +128,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + uint8_t ptype_disable; bool scalar_ena; /* Pointer back to rte */ @@ -201,6 +211,10 @@ void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +/* Lookup configuration */ +const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); +void *cnxk_nix_fastpath_lookup_mem_get(void); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_lookup.c b/drivers/net/cnxk/cnxk_lookup.c new file mode 100644 index 0000000..a9e51e5 --- /dev/null +++ b/drivers/net/cnxk/cnxk_lookup.c @@ -0,0 +1,326 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include +#include + +#include "cnxk_ethdev.h" + +/* NIX_RX_PARSE_S's ERRCODE + ERRLEV (12 bits) */ +#define ERRCODE_ERRLEN_WIDTH 12 +#define ERR_ARRAY_SZ ((BIT(ERRCODE_ERRLEN_WIDTH)) * sizeof(uint32_t)) + +#define SA_TBL_SZ (RTE_MAX_ETHPORTS * sizeof(uint64_t)) +#define LOOKUP_ARRAY_SZ (PTYPE_ARRAY_SZ + ERR_ARRAY_SZ + SA_TBL_SZ) +const uint32_t * +cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev) +{ + RTE_SET_USED(eth_dev); + + static const uint32_t ptypes[] = { + RTE_PTYPE_L2_ETHER_QINQ, /* LB */ + RTE_PTYPE_L2_ETHER_VLAN, /* LB */ + RTE_PTYPE_L2_ETHER_TIMESYNC, /* LB */ + RTE_PTYPE_L2_ETHER_ARP, /* LC */ + RTE_PTYPE_L2_ETHER_NSH, /* LC */ + RTE_PTYPE_L2_ETHER_FCOE, /* LC */ + RTE_PTYPE_L2_ETHER_MPLS, /* LC */ + RTE_PTYPE_L3_IPV4, /* LC */ + RTE_PTYPE_L3_IPV4_EXT, /* LC */ + RTE_PTYPE_L3_IPV6, /* LC */ + RTE_PTYPE_L3_IPV6_EXT, /* LC */ + RTE_PTYPE_L4_TCP, /* LD */ + RTE_PTYPE_L4_UDP, /* LD */ + RTE_PTYPE_L4_SCTP, /* LD */ + RTE_PTYPE_L4_ICMP, /* LD */ + RTE_PTYPE_L4_IGMP, /* LD */ + RTE_PTYPE_TUNNEL_GRE, /* LD */ + RTE_PTYPE_TUNNEL_ESP, /* LD */ + RTE_PTYPE_TUNNEL_NVGRE, /* LD */ + RTE_PTYPE_TUNNEL_VXLAN, /* LE */ + RTE_PTYPE_TUNNEL_GENEVE, /* LE */ + RTE_PTYPE_TUNNEL_GTPC, /* LE */ + RTE_PTYPE_TUNNEL_GTPU, /* LE */ + RTE_PTYPE_TUNNEL_VXLAN_GPE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_GRE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_UDP, /* LE */ + RTE_PTYPE_INNER_L2_ETHER, /* LF */ + RTE_PTYPE_INNER_L3_IPV4, /* LG */ + RTE_PTYPE_INNER_L3_IPV6, /* LG */ + RTE_PTYPE_INNER_L4_TCP, /* LH */ + RTE_PTYPE_INNER_L4_UDP, /* LH */ + RTE_PTYPE_INNER_L4_SCTP, /* LH */ + RTE_PTYPE_INNER_L4_ICMP, /* LH */ + RTE_PTYPE_UNKNOWN, + }; + + return ptypes; +} + +/* + * +------------------ +------------------ + + * | | IL4 | IL3| IL2 | TU | L4 | L3 | L2 | + * +-------------------+-------------------+ + * + * +-------------------+------------------ + + * | | LH | LG | LF | LE | LD | LC | LB | + * +-------------------+-------------------+ + * + * ptype [LE - LD - LC - LB] = TU - L4 - L3 - T2 + * ptype_tunnel[LH - LG - LF] = IL4 - IL3 - IL2 - TU + * + */ +static void +nix_create_non_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lb, lc, ld, le; + uint16_t val; + uint32_t idx; + + for (idx = 0; idx < PTYPE_NON_TUNNEL_ARRAY_SZ; idx++) { + lb = idx & 0xF; + lc = (idx & 0xF0) >> 4; + ld = (idx & 0xF00) >> 8; + le = (idx & 0xF000) >> 12; + val = RTE_PTYPE_UNKNOWN; + + switch (lb) { + case NPC_LT_LB_STAG_QINQ: + val |= RTE_PTYPE_L2_ETHER_QINQ; + break; + case NPC_LT_LB_CTAG: + val |= RTE_PTYPE_L2_ETHER_VLAN; + break; + } + + switch (lc) { + case NPC_LT_LC_ARP: + val |= RTE_PTYPE_L2_ETHER_ARP; + break; + case NPC_LT_LC_NSH: + val |= RTE_PTYPE_L2_ETHER_NSH; + break; + case NPC_LT_LC_FCOE: + val |= RTE_PTYPE_L2_ETHER_FCOE; + break; + case NPC_LT_LC_MPLS: + val |= RTE_PTYPE_L2_ETHER_MPLS; + break; + case NPC_LT_LC_IP: + val |= RTE_PTYPE_L3_IPV4; + break; + case NPC_LT_LC_IP_OPT: + val |= RTE_PTYPE_L3_IPV4_EXT; + break; + case NPC_LT_LC_IP6: + val |= RTE_PTYPE_L3_IPV6; + break; + case NPC_LT_LC_IP6_EXT: + val |= RTE_PTYPE_L3_IPV6_EXT; + break; + case NPC_LT_LC_PTP: + val |= RTE_PTYPE_L2_ETHER_TIMESYNC; + break; + } + + switch (ld) { + case NPC_LT_LD_TCP: + val |= RTE_PTYPE_L4_TCP; + break; + case NPC_LT_LD_UDP: + val |= RTE_PTYPE_L4_UDP; + break; + case NPC_LT_LD_SCTP: + val |= RTE_PTYPE_L4_SCTP; + break; + case NPC_LT_LD_ICMP: + case NPC_LT_LD_ICMP6: + val |= RTE_PTYPE_L4_ICMP; + break; + case NPC_LT_LD_IGMP: + val |= RTE_PTYPE_L4_IGMP; + break; + case NPC_LT_LD_GRE: + val |= RTE_PTYPE_TUNNEL_GRE; + break; + case NPC_LT_LD_NVGRE: + val |= RTE_PTYPE_TUNNEL_NVGRE; + break; + } + + switch (le) { + case NPC_LT_LE_VXLAN: + val |= RTE_PTYPE_TUNNEL_VXLAN; + break; + case NPC_LT_LE_ESP: + val |= RTE_PTYPE_TUNNEL_ESP; + break; + case NPC_LT_LE_VXLANGPE: + val |= RTE_PTYPE_TUNNEL_VXLAN_GPE; + break; + case NPC_LT_LE_GENEVE: + val |= RTE_PTYPE_TUNNEL_GENEVE; + break; + case NPC_LT_LE_GTPC: + val |= RTE_PTYPE_TUNNEL_GTPC; + break; + case NPC_LT_LE_GTPU: + val |= RTE_PTYPE_TUNNEL_GTPU; + break; + case NPC_LT_LE_TU_MPLS_IN_GRE: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_GRE; + break; + case NPC_LT_LE_TU_MPLS_IN_UDP: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_UDP; + break; + } + ptype[idx] = val; + } +} + +#define TU_SHIFT(x) ((x) >> PTYPE_NON_TUNNEL_WIDTH) +static void +nix_create_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lf, lg, lh; + uint16_t val; + uint32_t idx; + + /* Skip non tunnel ptype array memory */ + ptype = ptype + PTYPE_NON_TUNNEL_ARRAY_SZ; + + for (idx = 0; idx < PTYPE_TUNNEL_ARRAY_SZ; idx++) { + lf = idx & 0xF; + lg = (idx & 0xF0) >> 4; + lh = (idx & 0xF00) >> 8; + val = RTE_PTYPE_UNKNOWN; + + switch (lf) { + case NPC_LT_LF_TU_ETHER: + val |= TU_SHIFT(RTE_PTYPE_INNER_L2_ETHER); + break; + } + switch (lg) { + case NPC_LT_LG_TU_IP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV4); + break; + case NPC_LT_LG_TU_IP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV6); + break; + } + switch (lh) { + case NPC_LT_LH_TU_TCP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_TCP); + break; + case NPC_LT_LH_TU_UDP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_UDP); + break; + case NPC_LT_LH_TU_SCTP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_SCTP); + break; + case NPC_LT_LH_TU_ICMP: + case NPC_LT_LH_TU_ICMP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_ICMP); + break; + } + + ptype[idx] = val; + } +} + +static void +nix_create_rx_ol_flags_array(void *mem) +{ + uint16_t idx, errcode, errlev; + uint32_t val, *ol_flags; + + /* Skip ptype array memory */ + ol_flags = (uint32_t *)((uint8_t *)mem + PTYPE_ARRAY_SZ); + + for (idx = 0; idx < BIT(ERRCODE_ERRLEN_WIDTH); idx++) { + errlev = idx & 0xf; + errcode = (idx & 0xff0) >> 4; + + val = PKT_RX_IP_CKSUM_UNKNOWN; + val |= PKT_RX_L4_CKSUM_UNKNOWN; + val |= PKT_RX_OUTER_L4_CKSUM_UNKNOWN; + + switch (errlev) { + case NPC_ERRLEV_RE: + /* Mark all errors as BAD checksum errors + * including Outer L2 length mismatch error + */ + if (errcode) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_L4_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LC: + if (errcode == NPC_EC_OIP4_CSUM || + errcode == NPC_EC_IP_FRAG_OFFSET_1) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_EIP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LG: + if (errcode == NPC_EC_IIP4_CSUM) + val |= PKT_RX_IP_CKSUM_BAD; + else + val |= PKT_RX_IP_CKSUM_GOOD; + break; + case NPC_ERRLEV_NIX: + if (errcode == NIX_RX_PERRCODE_OL4_CHK || + errcode == NIX_RX_PERRCODE_OL4_LEN || + errcode == NIX_RX_PERRCODE_OL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + val |= PKT_RX_OUTER_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL4_CHK || + errcode == NIX_RX_PERRCODE_IL4_LEN || + errcode == NIX_RX_PERRCODE_IL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL3_LEN || + errcode == NIX_RX_PERRCODE_OL3_LEN) { + val |= PKT_RX_IP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + } + ol_flags[idx] = val; + } +} + +void * +cnxk_nix_fastpath_lookup_mem_get(void) +{ + const char name[] = CNXK_NIX_FASTPATH_LOOKUP_MEM; + const struct rte_memzone *mz; + void *mem; + + mz = rte_memzone_lookup(name); + if (mz != NULL) + return mz->addr; + + /* Request for the first time */ + mz = rte_memzone_reserve_aligned(name, LOOKUP_ARRAY_SZ, SOCKET_ID_ANY, + 0, ROC_ALIGN); + if (mz != NULL) { + mem = mz->addr; + /* Form the ptype array lookup memory */ + nix_create_non_tunnel_ptype_array(mem); + nix_create_tunnel_ptype_array(mem); + /* Form the rx ol_flags based on errcode */ + nix_create_rx_ol_flags_array(mem); + return mem; + } + return NULL; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index c0ec91a..b6b6989 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -11,7 +11,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', - 'cnxk_link.c') + 'cnxk_link.c', + 'cnxk_lookup.c') # CN9K sources += files('cn9k_ethdev.c') From patchwork Sat Mar 6 15:33:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88607 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 853EEA0548; Sat, 6 Mar 2021 16:35:54 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 31A7022A393; Sat, 6 Mar 2021 16:34:49 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id C9BCB22A391 for ; Sat, 6 Mar 2021 16:34:47 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmqO029543 for ; Sat, 6 Mar 2021 07:34:47 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=1wgiLcLazGp01gst+R9JZM+FM45OQKqLR4umDag8JHY=; b=TW/h8Fqdw9lX6FBrNprdn91cZj/4yD3/DrjHur6I2p4d7kTJeQ9vK/C/V1XmZXIwiK2n f1eobAebwiO+TqSyXjw4e5pI7ficd9Ux6s0YYg0EL924JPxW3ruMkgec4noEzkYsVMqp 571q3oYkbacKv4xXVtZgu8Lz9ZrMqac7A/FCtzU1B5CuFEA0wQeZFW0ucg7yi/SJ1QjK /vxxKnVde30XdnjwU0qF3MV/W6vaSUEaBtM578PyiXR6morTPlRZKN8o2AXcSt3AE5jl ymbZ6UB6ol060gZFXIGmPedihvQ8EXyd4i8WhVhVp9rxLSVNid2xw+s/G8CF4TI6TrBT fQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcbe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:46 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:45 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:45 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:44 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 5E2F33F703F; Sat, 6 Mar 2021 07:34:42 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:30 +0530 Message-ID: <20210306153404.10781-11-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 10/44] net/cnxk: add queue start and stop support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Rx/Tx queue start and stop callbacks for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 16 ++++++ drivers/net/cnxk/cn9k_ethdev.c | 16 ++++++ drivers/net/cnxk/cnxk_ethdev.c | 92 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 1 + 7 files changed, 128 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 503582c..712f8d5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 9ad225a..82f2af0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8c93ba7..61fed11 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index efd5b67..1a9fcbb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -137,6 +137,21 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn10k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn10k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -168,6 +183,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 3f3de4f..3561632 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -135,6 +135,21 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn9k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn9k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -177,6 +192,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 96acf90..f1ba04f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -819,12 +819,104 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } +static int +cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc = -EINVAL; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, true); + if (rc) { + plt_err("Failed to enable sq aura fc, txq=%u, rc=%d", qid, rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +int +cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, false); + if (rc) { + plt_err("Failed to disable sqb aura fc, txq=%u, rc=%d", qid, + rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, true); + if (rc) { + plt_err("Failed to enable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, false); + if (rc) { + plt_err("Failed to disable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .tx_queue_start = cnxk_nix_tx_queue_start, + .rx_queue_start = cnxk_nix_rx_queue_start, + .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 6b7261c..7e79a8d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -199,6 +199,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); From patchwork Sat Mar 6 15:33:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88608 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 01999A0548; Sat, 6 Mar 2021 16:36:05 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B120522A3B1; Sat, 6 Mar 2021 16:34:51 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 79F8922A3A7 for ; Sat, 6 Mar 2021 16:34:50 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVumI029368 for ; Sat, 6 Mar 2021 07:34:49 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=MQjAt09ju1mvj06Hz71yYCABmB7OJHQpjema9gDuCBE=; b=ZY+OchuEvmOS/wCMexJP9ryWnIGW6rDDpIIUMQ26N7Vp5fMl1096s3COISXQ1kboayFe GbLkHRpn1ub5OWH/3o3SrW1HEbeNU1UT/sniNvjejFeFPQ2VqRA4D9ZtSG6GXnHnO04H FmG3iJ0lyYbBnji623JlUT11zOM/Sqp2yHg/bkmnj9pAC0gqRjlFzMKj+bu7baqyjZQI F+IebNKI/WYfd+bQY/h0XWmjQSwBoDeU0KZ34wlsyjmi3suWEviqzuaRq2BFb6kSCfAo 7RtYBy6IM4xHMm1ZX338Y1y5Scv1tu+J5GNBivbUZh4k0220YXb7sbKsmZ51l+52jfkj uA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcbf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:49 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:48 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:47 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:47 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 813513F703F; Sat, 6 Mar 2021 07:34:45 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:31 +0530 Message-ID: <20210306153404.10781-12-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 11/44] net/cnxk: add Rx support for cn9k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add Rx burst scalar version for CN9K. Signed-off-by: Jerin Jacob --- drivers/net/cnxk/cn9k_ethdev.h | 3 + drivers/net/cnxk/cn9k_rx.c | 124 +++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_rx.h | 152 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 + drivers/net/cnxk/meson.build | 3 +- 5 files changed, 284 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_rx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 9ebf68f..84dcc2c 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -30,4 +30,7 @@ struct cn9k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c new file mode 100644 index 0000000..1c05cf3 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.c @@ -0,0 +1,124 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +static inline uint16_t +nix_rx_nb_pkts(struct cn9k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn9k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define R(name, f3, f2, f1, f0, flags) \ + static uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 95a1e69..949fd95 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -7,6 +7,158 @@ #include +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->cn9k.pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = + nix_update_match_id(rx->cn9k.match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) #endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 7e79a8d..58c6710 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,9 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +/* Default mark value used when none is provided. */ +#define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff + #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index b6b6989..c2bfd94 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -15,7 +15,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_lookup.c') # CN9K -sources += files('cn9k_ethdev.c') +sources += files('cn9k_ethdev.c', + 'cn9k_rx.c') # CN10K sources += files('cn10k_ethdev.c') From patchwork Sat Mar 6 15:33:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88609 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2B2A9A0548; Sat, 6 Mar 2021 16:36:14 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E3E3922A3B5; Sat, 6 Mar 2021 16:34:55 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id CE8B222A3A2 for ; Sat, 6 Mar 2021 16:34:53 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVumJ029368 for ; Sat, 6 Mar 2021 07:34:53 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=3P4knlwMyDhv8TEJpTBrlabHdRMQk64clYkOQcu3QPA=; b=NYX0vh3bN7XL1sPin6MxgeO/1HFpKTN3G8B/1rDyq+6Pmw9w1SvAKNb+R9/EFqRRkSEl TY8JLRYQZEdS9OoeV4F8f869tKycJPUHsSHVFyQBJkR/Pgh5dz9S5GJK8K5kvIse1LA3 0Cp8GoxNdU4Q9krVUI8VXlCMiNy1ANsMabEdd06lnqGE6/Lzp6wtx8Fx2oX3NERAksZn JunYU+N/Ur5SqvptIkHw9qO9beqKuBILI9PtcNMA5wne5AxPt/XGHJ1vvn8kTl5OZdjS Jb0+or0uwzgLDvI35HQ/wq0gBBS0ofit6a4U8Ko69+m3DaEWz+pfZ1HYu/q1mKzwYR8m fA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcbn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:53 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:51 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:51 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:51 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 6A01E3F703F; Sat, 6 Mar 2021 07:34:48 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:32 +0530 Message-ID: <20210306153404.10781-13-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 12/44] net/cnxk: add Rx multi-segmented version for cn9k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Rx burst multi-segmented version for CN9K. Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh --- drivers/net/cnxk/cn9k_rx.c | 26 ++++++++++++++++++++ drivers/net/cnxk/cn9k_rx.h | 55 ++++++++++++++++++++++++++++++++++++++++-- drivers/net/cnxk/cnxk_ethdev.h | 3 +++ 3 files changed, 82 insertions(+), 2 deletions(-) diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 1c05cf3..5535735 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -88,6 +88,15 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ return nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } \ + \ + static uint16_t __rte_noinline __rte_hot \ + cn9k_nix_recv_pkts_mseg_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ } NIX_RX_FASTPATH_MODES @@ -110,6 +119,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, @@ -118,7 +129,22 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 949fd95..a6b245f 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -99,6 +99,53 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + + ((rx->cn9k.desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -133,8 +180,12 @@ cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } #define RSS_F NIX_RX_OFFLOAD_RSS_F diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 58c6710..a94f5eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -163,6 +163,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Rx burst for cleanup(Only Primary) */ + eth_rx_burst_t rx_pkt_burst_no_offload; + /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; }; From patchwork Sat Mar 6 15:33:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88610 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7ADBCA0548; Sat, 6 Mar 2021 16:36:23 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2107422A3B9; Sat, 6 Mar 2021 16:34:59 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id B3C2422A3AD for ; Sat, 6 Mar 2021 16:34:56 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmqP029543 for ; Sat, 6 Mar 2021 07:34:56 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=5o6e/ldbuI2S7r8nUrD3DMeFVH7wdWEzzSJCKTazFy0=; b=A7OheKxTwtuTv1G7Uucj0NRK9QxtDptPAcwApLqtEOJXs/o5FmhQPMLghOO8ACVAIrtq BgraD1ZOOx+i7S8KhWkS/fC1VEH5Mks2/CjfzB93SPz50RmINHS6wAmX+uke7hRkBnVm d4b6RtrSfc9nsigiUyNyzVjsc3p2oXitN/n7LQLIFiVgJwGJKDEOh3jqOkeaQIXv0vDR 4FemsGyugnHyHjpPmE8YlPOHFLFiduMKqxftTg5DcuW5l5vUOiyQqN0MB4NZqdLivztv WGRG9+4/vbeMr9vi6ueJBtwAYaYh8T6q18c2g0WdIU2jj+mx1eeocDClzowjrKEUKHN1 jg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcbs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:55 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:54 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:54 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 9D9453F703F; Sat, 6 Mar 2021 07:34:51 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:33 +0530 Message-ID: <20210306153404.10781-14-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 13/44] net/cnxk: add Rx vector version for cn9k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add Rx burst vector version for CN9K. Signed-off-by: Jerin Jacob Signed-off-by: Nithin Dabilpuram --- drivers/net/cnxk/cn9k_rx.c | 240 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 239 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 5535735..391f1e2 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include + #include "cn9k_ethdev.h" #include "cn9k_rx.h" @@ -83,6 +85,223 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += nix_recv_pkts(rx_queue, &rx_pkts[packets], pkts_left, + flags); + + return packets; +} + +#else + +static inline uint16_t +nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + #define R(name, f3, f2, f1, f0, flags) \ static uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ @@ -97,6 +316,14 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, { \ return nix_recv_pkts(rx_queue, rx_pkts, pkts, \ (flags) | NIX_RX_MULTI_SEG_F); \ + } \ + \ + static uint16_t __rte_noinline __rte_hot \ + cn9k_nix_recv_pkts_vec_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, (flags)); \ } NIX_RX_FASTPATH_MODES @@ -137,7 +364,18 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); From patchwork Sat Mar 6 15:33:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88611 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E7471A0548; Sat, 6 Mar 2021 16:36:34 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 96B0822A3C1; Sat, 6 Mar 2021 16:35:02 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id A83FC22A3C1 for ; Sat, 6 Mar 2021 16:35:00 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmlm029546 for ; Sat, 6 Mar 2021 07:34:59 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=oOk2FlsLVehZfnFYbl8aQW/rKvSoc/ooXL8nyXbieQU=; b=iuZlEK2P5+gXwkeEVDmlju6NG+9XGymzLvh52/5oIbPXiZ1FsM5h13owChT/UQIWa/Pi vVgnfdTHAocE0elFR4jrfBv7rp7bCP1ku+twrS89iSfyRtEXT0rS5Cp9GGdw18ax8h5K P7zDS1g38R9HjIWN0hmCImJwZ7V3UEDJzZ9o5IiwIbOeWEe5hl3LHJbJsQXL/T8CFeRx l8m2aHDUw9vP2Di3J3Pj6jJVV5YiQVWOF5senT3Zz8UQLYnFPl/u9lhulWW670rKJDUw MNeRLzjw2rw9GHci6cpZKiq0dNRvl5yYRQR1O0Xio/emcWG+c3YQraq1uKW/sP95cqXf Wg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcbt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:59 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:58 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:57 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:57 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id E45223F7041; Sat, 6 Mar 2021 07:34:54 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram , Harman Kalra Date: Sat, 6 Mar 2021 21:03:34 +0530 Message-ID: <20210306153404.10781-15-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 14/44] net/cnxk: add Tx support for cn9k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add Tx burst scalar version for CN9K. Signed-off-by: Jerin Jacob Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh Signed-off-by: Harman Kalra --- drivers/net/cnxk/cn9k_ethdev.h | 1 + drivers/net/cnxk/cn9k_tx.c | 103 ++++++++++++ drivers/net/cnxk/cn9k_tx.h | 357 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 71 ++++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 534 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_tx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 84dcc2c..cd0938f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -32,5 +32,6 @@ struct cn9k_eth_rxq { /* Rx and Tx routines */ void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c new file mode 100644 index 0000000..06e9618 --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.c @@ -0,0 +1,103 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +static __rte_always_inline uint16_t +nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + uint16_t i; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_one(cmd, lmt_addr, io_addr, flags); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, flags); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index bb6379b..5f915e8 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -4,10 +4,367 @@ #ifndef __CN9K_TX_H__ #define __CN9K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn9k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format += is_udp_tun ? 2 : 6; + + send_hdr_ext->w0.lso_format += + !!(ol_flags & PKT_TX_OUTER_IPV6) << 1; + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(++sg) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_one(uint64_t *cmd, void *lmt_addr, const rte_iova_t io_addr, + const uint32_t flags) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_prep_lmt(uint64_t *cmd, void *lmt_addr, const uint32_t flags) +{ + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeor(io_addr); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeorl(io_addr); +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + #endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a94f5eb..8facafc 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -103,6 +103,10 @@ /* Fastpath lookup */ #define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" +#define CNXK_NIX_UDP_TUN_BITMASK \ + ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ + (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -226,4 +230,71 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Inlines */ +static __rte_always_inline uint64_t +cnxk_pktmbuf_detach(struct rte_mbuf *m) +{ + struct rte_mempool *mp = m->pool; + uint32_t mbuf_size, buf_len; + struct rte_mbuf *md; + uint16_t priv_size; + uint16_t refcount; + + /* Update refcount of direct mbuf */ + md = rte_mbuf_from_indirect(m); + refcount = rte_mbuf_refcnt_update(md, -1); + + priv_size = rte_pktmbuf_priv_size(mp); + mbuf_size = (uint32_t)(sizeof(struct rte_mbuf) + priv_size); + buf_len = rte_pktmbuf_data_room_size(mp); + + m->priv_size = priv_size; + m->buf_addr = (char *)m + mbuf_size; + m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size; + m->buf_len = (uint16_t)buf_len; + rte_pktmbuf_reset_headroom(m); + m->data_len = 0; + m->ol_flags = 0; + m->next = NULL; + m->nb_segs = 1; + + /* Now indirect mbuf is safe to free */ + rte_pktmbuf_free(m); + + if (refcount == 0) { + rte_mbuf_refcnt_set(md, 1); + md->data_len = 0; + md->ol_flags = 0; + md->next = NULL; + md->nb_segs = 1; + return 0; + } else { + return 1; + } +} + +static __rte_always_inline uint64_t +cnxk_nix_prefree_seg(struct rte_mbuf *m) +{ + if (likely(rte_mbuf_refcnt_read(m) == 1)) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + m->next = NULL; + m->nb_segs = 1; + return 0; + } else if (rte_mbuf_refcnt_update(m, -1) == 0) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + rte_mbuf_refcnt_set(m, 1); + m->next = NULL; + m->nb_segs = 1; + return 0; + } + + /* Mbuf is having refcount more than 1 so need not to be freed */ + return 1; +} + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index c2bfd94..d152f08 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -16,7 +16,8 @@ sources = files('cnxk_ethdev.c', # CN9K sources += files('cn9k_ethdev.c', - 'cn9k_rx.c') + 'cn9k_rx.c', + 'cn9k_tx.c') # CN10K sources += files('cn10k_ethdev.c') From patchwork Sat Mar 6 15:33:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88612 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 83B33A0548; Sat, 6 Mar 2021 16:36:43 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AE1D022A3CE; Sat, 6 Mar 2021 16:35:04 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id AF3ED22A34F for ; Sat, 6 Mar 2021 16:35:03 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVquj029350 for ; Sat, 6 Mar 2021 07:35:03 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=TKg69XUYtp2TUJzdFhP/X9LyIgzZdhPjQJlsCIWvmQc=; b=Mnd+jwU2lumpogQao6DZjCgM1k8m/XvvPZuoHSBNg/AC3O6m6JZGXFSOYHb2nW7Q/3Zy 8mqoQYQ7BpAdZc32SLUUK/mt1xhgQXScWb112eBobGb0kY6BmCbv6S0JM3yeIBnFfR+o EzpgX4Byy6AX+Z4WnFnu6NJU8sVzw/kTyjKLC0FcQbo/g38G2vyWxJMnXddjVGaOj1xW IghGisZRdyxrYt7Vc/0L7w3xxsKnlBB6m7xTptUbN748hyZU1HlGL1vYJQgxBsL83F59 r2S8Msg8kEZQBpN5bECWJz2mr22oPjf9J2Ugzjg9HT3rXNQ4wzpuTGkGcB9j9y4SxqVo BQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcbw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:02 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:01 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:01 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:00 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 5CC8D3F7041; Sat, 6 Mar 2021 07:34:58 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:35 +0530 Message-ID: <20210306153404.10781-16-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 15/44] net/cnxk: add Tx multi-segment version for cn9k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Tx burst multi-segment version for CN9K. Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh --- drivers/net/cnxk/cn9k_tx.c | 70 +++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx.h | 105 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++ 3 files changed, 179 insertions(+) diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index 06e9618..a474eb5 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -55,6 +55,44 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + uint64_t i; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + uint16_t segdw; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags); + segdw = cn9k_nix_prepare_mseg(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_mseg_one(cmd, lmt_addr, io_addr, segdw); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + #define T(name, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -71,6 +109,25 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, NIX_TX_FASTPATH_MODES #undef T +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -89,6 +146,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, @@ -97,7 +156,18 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + pick_tx_func(eth_dev, nix_eth_tx_burst); + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); + rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index 5f915e8..d653b3c 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -294,6 +294,111 @@ cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) return roc_lmt_submit_ldeorl(io_addr); } +static __rte_always_inline uint16_t +cn9k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Commit changes to mbuf */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + rte_io_wmb(); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_prep_lmt(uint64_t *cmd, void *lmt_addr, uint16_t segdw) +{ + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one(uint64_t *cmd, void *lmt_addr, rte_iova_t io_addr, + uint16_t segdw) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one_release(uint64_t *cmd, void *lmt_addr, + rte_iova_t io_addr, uint16_t segdw) +{ + uint64_t lmt_status; + + rte_io_wmb(); + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 8facafc..2f31cba 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -46,6 +46,10 @@ #define CNXK_NIX_TX_NB_SEG_MAX 9 #endif +#define CNXK_NIX_TX_MSEG_SG_DWORDS \ + ((RTE_ALIGN_MUL_CEIL(CNXK_NIX_TX_NB_SEG_MAX, 3) / 3) + \ + CNXK_NIX_TX_NB_SEG_MAX) + #define CNXK_NIX_RSS_L3_L4_SRC_DST \ (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ ETH_RSS_L4_DST_ONLY) From patchwork Sat Mar 6 15:33:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88613 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F40BBA0548; Sat, 6 Mar 2021 16:36:53 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 37AB322A391; Sat, 6 Mar 2021 16:35:09 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 29AE722A3D3 for ; Sat, 6 Mar 2021 16:35:07 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmqT029543 for ; Sat, 6 Mar 2021 07:35:06 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=duPs3CRvzKB/MFN3qUDup41qZhbGx3ftD2U5BD5EY9s=; b=EOoiri0JDX0L7mUt9ta7oyZPKNnrNm79oYwl6L7sFCsoCABaFwLzkmzKG32sdL7bgOZQ rv75Qdn/yaAGxQVGhULN3iXeEaAnjCoZuUW/g6VYBgtHMFM0AfaOJ/UGqmmYYv3Fnvue PqTMsgFn0krkKTMSDDSkdEP1F9SGrEirkdnGo8tYvWKwpjSZ8HRMeBV6mOPqIFKyPFMj 9zNaWCbi91lIi6m/O8xtS/Jz3qOqB82tFij3UymIykIFResZemIhp2p3juMhdMz+xWMG dLFmyEqOXAjypN8aRBXSfAAafE1xS2qpnr093FLLDNzuRirmQCE9AVaE1TymZHHXzOZU NA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcc4-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:06 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:04 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:04 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 969313F703F; Sat, 6 Mar 2021 07:35:01 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:36 +0530 Message-ID: <20210306153404.10781-17-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 16/44] net/cnxk: add Tx vector version for cn9k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Tx burst vector version for CN9K. Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh --- drivers/net/cnxk/cn9k_tx.c | 951 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 950 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index a474eb5..300ccd2 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include + #include "cn9k_ethdev.h" #include "cn9k_tx.h" @@ -93,6 +95,921 @@ nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn9k_eth_txq *txq = tx_queue; + uint64_t *lmt_addr = txq->lmt_addr; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint64x2_t cmd00, cmd01; + uint64x2_t cmd10, cmd11; + uint64x2_t cmd20, cmd21; + uint64x2_t cmd30, cmd31; + uint64_t lmt_status, i; + uint16_t pkts_left; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + senddesc01_w0 = vld1q_dup_u64(&txq->cmd[0]); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->cmd[2]); + sgdesc23_w0 = sgdesc01_w0; + + for (i = 0; i < pkts; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + struct rte_mbuf *mbuf; + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } else { + struct rte_mbuf *mbuf; + /* Mark mempool object as "put" since + * it is freed by NIX + */ + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + RTE_SET_USED(mbuf); + } + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create first half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + } else { + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + } + + do { + vst1q_u64(lmt_addr, cmd00); + vst1q_u64(lmt_addr + 2, cmd01); + vst1q_u64(lmt_addr + 4, cmd10); + vst1q_u64(lmt_addr + 6, cmd11); + vst1q_u64(lmt_addr + 8, cmd20); + vst1q_u64(lmt_addr + 10, cmd21); + vst1q_u64(lmt_addr + 12, cmd30); + vst1q_u64(lmt_addr + 14, cmd31); + lmt_status = roc_lmt_submit_ldeor(io_addr); + + } while (lmt_status == 0); + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + if (unlikely(pkts_left)) + pkts += nix_xmit_pkts(tx_queue, tx_pkts, pkts_left, cmd, flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif + #define T(name, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -128,6 +1045,25 @@ NIX_TX_FASTPATH_MODES NIX_TX_FASTPATH_MODES #undef T +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -164,7 +1100,20 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); From patchwork Sat Mar 6 15:33:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88614 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7318AA0548; Sat, 6 Mar 2021 16:37:05 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D0FF422A3D3; Sat, 6 Mar 2021 16:35:11 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 69EF522A3D9 for ; Sat, 6 Mar 2021 16:35:10 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FXJQm032533 for ; Sat, 6 Mar 2021 07:35:09 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=OG14stIocYv7qUG92+F0iGyE5WRj9b5NHc5NBB33taQ=; b=kxu2S7YsGr0OOCNZZjJ/lgcbQzOSUHeVXrYpL0Z/9wr1Ig+OFKAOFtCg8oYVMbDJ+j2j TvRb6nQIuGd38aoItpHAbUUNanDT4TEPQcc6cuDKx+pd/T4U5ZKGjIDrsb4Z2ecp7/9V BTXfQqe0PJ7HX8blhTpQ8onbNvP66ul8CgOzAgeg7Ji4DcL6NHDDUeYRav/yI0FSlKvF atIHBbdGzu+SYWM7qLttcSdN22qbcmg85jz0ZlXGR9Nk3kRd7Rsabzi4sBf0Td0EWdfo 8CI3x12twj2odxv2kQyd3voON31epurI/XewYOCVjHaS+gXD+al5Yr98Z0+jrFuns9N8 vg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcc7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:09 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:07 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:07 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 1DC2A3F7041; Sat, 6 Mar 2021 07:35:04 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram , Harman Kalra Date: Sat, 6 Mar 2021 21:03:37 +0530 Message-ID: <20210306153404.10781-18-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 17/44] net/cnxk: add Rx support for cn10k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add Rx burst support for CN10K SoC. Signed-off-by: Jerin Jacob Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh Signed-off-by: Harman Kalra --- drivers/net/cnxk/cn10k_ethdev.h | 3 + drivers/net/cnxk/cn10k_rx.c | 123 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_rx.h | 151 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 279 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.c diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 2157b16..e4332d3 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -32,4 +32,7 @@ struct cn10k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c new file mode 100644 index 0000000..1ff1b04 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.c @@ -0,0 +1,123 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +static inline uint16_t +nix_rx_nb_pkts(struct cn10k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn10k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define R(name, f3, f2, f1, f0, flags) \ + static uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index d3d1661..f43f320 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -6,6 +6,157 @@ #include +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) #endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index d152f08..1ac6b85 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -19,7 +19,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_rx.c', 'cn9k_tx.c') # CN10K -sources += files('cn10k_ethdev.c') +sources += files('cn10k_ethdev.c', + 'cn10k_rx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] From patchwork Sat Mar 6 15:33:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88615 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DCC87A0548; Sat, 6 Mar 2021 16:37:13 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 289F722A3E6; Sat, 6 Mar 2021 16:35:16 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 19C6822A3DB for ; Sat, 6 Mar 2021 16:35:13 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVt9J029362 for ; Sat, 6 Mar 2021 07:35:13 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=E2e92dsbIIMPrOVHk3NKC9LQORRrX2HwQQY7VNh3GoA=; b=bIehCD7rhxYYYBSqFlsASfg/dtupIwp2mWNvxTc1sUS009bDqwC6fIMheLfbUApUid6t 8/WN1tGP8X4eU7vKT2MrIODjA3CdE7Z7l/yEH5GiB4bH2JK7RmDfzfoo8mz3ocRMKMiJ GvAAvqmV2DxjGioO02Wv+bftw+eyNFsWeOFbRObnNr/REfJ8vTip3yez0VJRcoyYeE/Z e+bDI0rxRPdy4RF01FCS6J5VGZley8bbf6LO4P3HcsgdGu7GxeKKtG12jW8CBxU1Qr0B oeVKp42D2hL+wcB8VSwoYspQpF6jGXeUbI/6TdYgdPyXRuUtMQhOPSEbuX+kBQ7UWLcS Tw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcc9-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:13 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:11 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:11 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 92E163F703F; Sat, 6 Mar 2021 07:35:08 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:38 +0530 Message-ID: <20210306153404.10781-19-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 18/44] net/cnxk: add Rx multi-segment version for cn10k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Rx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh --- doc/guides/nics/cnxk.rst | 2 ++ doc/guides/nics/features/cnxk.ini | 2 ++ doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 2 ++ drivers/net/cnxk/cn10k_rx.c | 27 ++++++++++++++++++ drivers/net/cnxk/cn10k_rx.h | 54 +++++++++++++++++++++++++++++++++-- 6 files changed, 86 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 4f1b58c..789ec29 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,11 +17,13 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Jumbo frames - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - Link state information +- Scatter-Gather IO support Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 712f8d5..23564b7 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 82f2af0..421048d 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 61fed11..e901fa2 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index 1ff1b04..b98e7a1 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -88,6 +88,15 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ return nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } \ + \ + static uint16_t __rte_noinline __rte_hot \ + cn10k_nix_recv_pkts_mseg_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ } NIX_RX_FASTPATH_MODES @@ -110,6 +119,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, @@ -118,6 +129,22 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index f43f320..7887a81 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -98,6 +98,52 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + ((rx->desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -131,8 +177,12 @@ cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } #define RSS_F NIX_RX_OFFLOAD_RSS_F From patchwork Sat Mar 6 15:33:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88616 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 56D5AA0548; Sat, 6 Mar 2021 16:37:22 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6513C22A3EF; Sat, 6 Mar 2021 16:35:19 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 1A36822A358 for ; Sat, 6 Mar 2021 16:35:16 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FYJGI001300 for ; Sat, 6 Mar 2021 07:35:16 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=rAHvnjndr8/eZe3zxEbkfprhtXAqakwNSqCPX64wN1c=; b=Le7c4W1vorEitQ3QSH56TrXhgS1gYNFgF6f0PD0cuzJThcNRSws0/0LEg9a7Q4gv64g4 lgGBlfmBYQdyJxhZ+0Qq7VGh7P3mt3Gldfs1wRSeX+bYNFriIpX+MRhiqpS2hdJxutmx mDSC4DrQPhy5XJQMuNTnLGsk8k+gMr6jLRluk2hxilJqaz0Pi7xqX5wd3wjZTZmLeKT+ DFLIsxbokVnuDdpLV7u56pV+Hsr5fqZK/vHsPa8dxmQlYDlOYd7ykmW9hpm77L5paRU3 t2F0VFhxMSiPa3PFPpQIDiFM6VADjEFv+rMBAV/vEaSFif8P1yHD+f4W3dLeFnS0tsOU KA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurccc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:16 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:14 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:14 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:14 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id C3A3A3F7044; Sat, 6 Mar 2021 07:35:11 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:39 +0530 Message-ID: <20210306153404.10781-20-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 19/44] net/cnxk: add Rx vector version for cn10k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add Rx burst vector version for CN10K. Signed-off-by: Jerin Jacob Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/cnxk.rst | 1 + drivers/net/cnxk/cn10k_rx.c | 240 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 240 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 789ec29..4187e9d 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Link state information - Scatter-Gather IO support +- Vector Poll mode driver Prerequisites ------------- diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index b98e7a1..2bc952d 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include + #include "cn10k_ethdev.h" #include "cn10k_rx.h" @@ -83,6 +85,223 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += nix_recv_pkts(rx_queue, &rx_pkts[packets], pkts_left, + flags); + + return packets; +} + +#else + +static inline uint16_t +nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + #define R(name, f3, f2, f1, f0, flags) \ static uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ @@ -97,6 +316,14 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, { \ return nix_recv_pkts(rx_queue, rx_pkts, pkts, \ (flags) | NIX_RX_MULTI_SEG_F); \ + } \ + \ + static uint16_t __rte_noinline __rte_hot \ + cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, (flags)); \ } NIX_RX_FASTPATH_MODES @@ -137,7 +364,18 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); From patchwork Sat Mar 6 15:33:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88617 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 415EFA0548; Sat, 6 Mar 2021 16:37:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 07EF322A3FA; Sat, 6 Mar 2021 16:35:22 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id D0BE422A3F3 for ; Sat, 6 Mar 2021 16:35:20 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FYJGJ001300 for ; Sat, 6 Mar 2021 07:35:20 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=FGyERPuYJeD69GVVP2JxholwTxXrxBuqZaRD7s9xdVc=; b=b/b8jXEiNl4b2ZkrmC77+UiIL/kK6e23bZI+w834FkiHmSdgfLhGBLtp1OUfyFXXKyXI Lq13ZGOHPSqo1v5cVW2bYqZ0hG++FHDZVP1rk44GJHXiDqgHbB45+tvlkFo5bvsjRjDK LMmn0cw2cVfQicB67pmttc61pPjiwrQSZvhOFAjech/h2qJK/t6ElXeQGSst5iJbPH50 4+9+2892DOXICuDWKsuJ101VS0+qsp+0ZzQ1GhTh1VaQleIldSPUMfKQOBeq7OYGnX/A TCF7/YWEgPFYALERr0k4NBNUtLH8lyaYQ6634Xh0NOBQ5wDRtNSOe+wJu3ZJB68IIiXd zQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcch-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:19 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:18 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:18 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:17 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 0AEB93F7040; Sat, 6 Mar 2021 07:35:14 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram , Harman Kalra Date: Sat, 6 Mar 2021 21:03:40 +0530 Message-ID: <20210306153404.10781-21-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 20/44] net/cnxk: add Tx support for cn10k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add Tx burst scalar version for CN10K. Signed-off-by: Jerin Jacob Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh Signed-off-by: Harman Kalra --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 7 + doc/guides/nics/features/cnxk_vec.ini | 6 + doc/guides/nics/features/cnxk_vf.ini | 7 + drivers/net/cnxk/cn10k_ethdev.h | 1 + drivers/net/cnxk/cn10k_tx.c | 174 +++++++++++++++++ drivers/net/cnxk/cn10k_tx.h | 358 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 556 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_tx.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 4187e9d..555730d 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -22,6 +22,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Inner and Outer Checksum offload - Link state information - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 23564b7..02be26b 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,11 +12,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 421048d..8c63853 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,10 +12,16 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index e901fa2..a1bd49b 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,11 +11,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index e4332d3..58c51ab 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -34,5 +34,6 @@ struct cn10k_eth_rxq { /* Rx and Tx routines */ void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c new file mode 100644 index 0000000..0fad4c0 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.c @@ -0,0 +1,174 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +static __rte_always_inline uint64_t +cn10k_nix_tx_steor_data(const uint16_t flags) +{ + const uint64_t dw_m1 = cn10k_nix_tx_ext_subs(flags) + 1; + uint64_t data = 0; + + /* This will be moved to addr area */ + data |= dw_m1; + /* 15 vector sizes for single seg */ + data |= dw_m1 << 19; + data |= dw_m1 << 22; + data |= dw_m1 << 25; + data |= dw_m1 << 28; + data |= dw_m1 << 31; + data |= dw_m1 << 34; + data |= dw_m1 << 37; + data |= dw_m1 << 40; + data |= dw_m1 << 43; + data |= dw_m1 << 46; + data |= dw_m1 << 49; + data |= dw_m1 << 52; + data |= dw_m1 << 55; + data |= dw_m1 << 58; + data |= dw_m1 << 61; + + return data; +} + +static __rte_always_inline uint16_t +nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + const rte_iova_t io_addr = txq->io_addr; + uintptr_t pa, lmt_addr = txq->lmt_base; + uint16_t lmt_id, burst, left, i; + uint64_t data; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + /* Get cmd skeleton */ + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + } + + /* Trigger LMTST */ + if (burst > 16) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (burst) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, flags); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 39d4755..ce1d4a0 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -4,10 +4,368 @@ #ifndef __CN10K_TX_H__ #define __CN10K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn10k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline void +cn10k_nix_tx_skeleton(const struct cn10k_eth_txq *txq, uint64_t *cmd, + const uint16_t flags) +{ + /* Send hdr */ + cmd[0] = txq->send_hdr_w0; + cmd[1] = 0; + cmd += 2; + + /* Send ext if present */ + if (flags & NIX_TX_NEED_EXT_HDR) { + *(__uint128_t *)cmd = *(const __uint128_t *)txq->cmd; + cmd += 2; + } + + /* Send sg */ + cmd[0] = txq->sg_w0; + cmd[1] = 0; +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, + const uint16_t flags) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format += is_udp_tun ? 2 : 6; + + send_hdr_ext->w0.lso_format += + !!(ol_flags & PKT_TX_OUTER_IPV6) << 1; + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(sg + 1) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } + + /* With minimal offloads, 'cmd' being local could be optimized out to + * registers. In other cases, 'cmd' will be in stack. Intent is + * 'cmd' stores content from txq->cmd which is copied only once. + */ + *((struct nix_send_hdr_s *)lmt_addr) = *send_hdr; + lmt_addr += 16; + if (flags & NIX_TX_NEED_EXT_HDR) { + *((struct nix_send_ext_s *)lmt_addr) = *send_hdr_ext; + lmt_addr += 16; + } + /* In case of multi-seg, sg template is stored here */ + *((union nix_send_sg_s *)lmt_addr) = *sg; + *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + #endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 1ac6b85..3aa9f70 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -20,7 +20,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx.c') # CN10K sources += files('cn10k_ethdev.c', - 'cn10k_rx.c') + 'cn10k_rx.c', + 'cn10k_tx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] From patchwork Sat Mar 6 15:33:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88618 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 50A46A0548; Sat, 6 Mar 2021 16:37:42 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 40F0622A3EC; Sat, 6 Mar 2021 16:35:25 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id E3B9922A37A for ; Sat, 6 Mar 2021 16:35:23 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVumM029368 for ; Sat, 6 Mar 2021 07:35:23 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=/i26B1zPNjdhzdF2qOzdeDLEc1WFKoMZdZ7yMz44HAA=; b=P7BXcHfhdnZXYLbE9MxxHOnPLZ8ndpSicsKOtZl84vfGS+NPJerz0ubaBnRzGsPEmJpv 4fT1B5R335rGo+wHktQ9fi1tuZbRo2lN6B4o3lvNo0nytKUZsNP3r8CPXLj11OO2nMxX B8J+CkB+bnVlFfKyKI/J3GxvyfWEIbpBJmpa/lO15iXO0k+4BOgDsRacsTTz4JfkAsjN M49tBEy2hTb6ppzetCy+QKPy7+RQ62zQaHtA3q4OQuc/kGuaoS1S+mhgkgEA+R7gOAnt pmMllqi21FZIUK2TVdWp1BVF3UpLhJ79WhLJiYhdFCZXjOOrU+GgCB0gfF4cccCN3Axn PA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurccn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:23 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:21 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:21 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:21 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 8B33C3F703F; Sat, 6 Mar 2021 07:35:18 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:41 +0530 Message-ID: <20210306153404.10781-22-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 21/44] net/cnxk: add Tx multi-segment version for cn10k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Tx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh --- drivers/net/cnxk/cn10k_tx.c | 124 ++++++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx.h | 71 +++++++++++++++++++++++++ 2 files changed, 195 insertions(+) diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index 0fad4c0..d170f31 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -125,6 +125,98 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t pa0, pa1, lmt_addr = txq->lmt_base; + const rte_iova_t io_addr = txq->io_addr; + uint16_t segdw, lmt_id, burst, left, i; + uint64_t data0, data1; + __uint128_t data128; + uint16_t shft; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + shft = 16; + data128 = 0; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags); + /* Store sg list directly on lmt line */ + segdw = cn10k_nix_prepare_mseg(tx_pkts[i], (uint64_t *)lmt_addr, + flags); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + data128 |= (((__uint128_t)(segdw - 1)) << shft); + shft += 3; + } + + data0 = (uint64_t)data128; + data1 = (uint64_t)(data128 >> 64); + /* Make data0 similar to data1 */ + data0 >>= 16; + /* Trigger LMTST */ + if (burst > 16) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= (15ULL << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + + pa1 = io_addr | (data1 & 0x7) << 4; + data1 &= ~0x7ULL; + data1 <<= 16; + data1 |= ((uint64_t)(burst - 17)) << 12; + data1 |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data1, pa1); + } else if (burst) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= ((burst - 1) << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + #define T(name, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ @@ -142,6 +234,25 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, NIX_TX_FASTPATH_MODES #undef T +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz)]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -160,6 +271,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, @@ -168,7 +281,18 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + pick_tx_func(eth_dev, nix_eth_tx_burst); + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); + rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index ce1d4a0..22f7a2b 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -295,6 +295,77 @@ cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); } +static __rte_always_inline uint16_t +cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Mark mempool object as "put" since it is freed by NIX + */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F From patchwork Sat Mar 6 15:33:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88619 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6A369A0548; Sat, 6 Mar 2021 16:37:53 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DBCF422A3F0; Sat, 6 Mar 2021 16:35:28 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 754B422A409 for ; Sat, 6 Mar 2021 16:35:27 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FXJQn032533 for ; Sat, 6 Mar 2021 07:35:26 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=aIg3mLfu1ZFflO31DEj59KIaLKd2K6SqETudZd8yEvQ=; b=M+fyCnsgrhc07WBRzaLvNBAq3ZEX+ixwsvoeaUYcqbcd2OjYWnzF3qaMPQ7yb+yyadzc jcd4Uue96m3lQw9MEbQibUcHqimSNVf1hjrFv7JO9YzKCcTMPihtSuMkVhg5hqR4k7CE FvdbP4AvDgeBLCzuq2OJb3wkEn0fhnLqNB+LSKsFRD6lDE9spZ7Pwuy4jQt7G04B/9FN nU0euZtvQFmqfm2/gzEmntc7YPQyr9Th2gJl9mRDeu75yrJmSe8TSxy548yU5ulG6R/K 7ItvkKSobri32uCrs/KUPk4cvbnN3GLv0Az/9s5CfSgsL8R9DhC9SF3q5YT5rhm5YMvN Mw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurccq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:26 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:25 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:24 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:24 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id CCC163F703F; Sat, 6 Mar 2021 07:35:21 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:42 +0530 Message-ID: <20210306153404.10781-23-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 22/44] net/cnxk: add Tx vector version for cn10k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Tx burst vector version for CN10K. Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh --- drivers/net/cnxk/cn10k_tx.c | 988 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 987 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index d170f31..c487c83 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include + #include "cn10k_ethdev.h" #include "cn10k_tx.h" @@ -217,6 +219,958 @@ nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3, data, pa; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint16_t left, scalar, burst, i, lmt_id; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t lmt_addr = txq->lmt_base; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint64x2_t cmd00, cmd01; + uint64x2_t cmd10, cmd11; + uint64x2_t cmd20, cmd21; + uint64x2_t cmd30, cmd31; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + scalar = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + senddesc01_w0 = vld1q_dup_u64(&txq->send_hdr_w0); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->sg_w0); + sgdesc23_w0 = sgdesc01_w0; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + for (i = 0; i < burst; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + struct rte_mbuf *mbuf; + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + } else { + struct rte_mbuf *mbuf; + /* Mark mempool object as "put" since + * it is freed by NIX + */ + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + RTE_SET_USED(mbuf); + } + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create first half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + } else { + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + } + + /* Store the prepared send desc to LMT lines */ + vst1q_u64((void *)lmt_addr, cmd00); + vst1q_u64((void *)(lmt_addr + 16), cmd01); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + vst1q_u64((void *)lmt_addr, cmd10); + vst1q_u64((void *)(lmt_addr + 16), cmd11); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + vst1q_u64((void *)lmt_addr, cmd20); + vst1q_u64((void *)(lmt_addr + 16), cmd21); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + vst1q_u64((void *)lmt_addr, cmd30); + vst1q_u64((void *)(lmt_addr + 16), cmd31); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + /* Trigger LMTST */ + if (burst > 16) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (burst) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Reset lmt base addr to start another burst */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + if (unlikely(scalar)) + pkts += nix_xmit_pkts(tx_queue, tx_pkts, scalar, cmd, flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif #define T(name, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ @@ -253,6 +1207,25 @@ NIX_TX_FASTPATH_MODES NIX_TX_FASTPATH_MODES #undef T +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -289,7 +1262,20 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); From patchwork Sat Mar 6 15:33:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88620 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6984AA0548; Sat, 6 Mar 2021 16:38:05 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9B54122A419; Sat, 6 Mar 2021 16:35:32 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id E007722A395 for ; Sat, 6 Mar 2021 16:35:30 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVt9L029362 for ; Sat, 6 Mar 2021 07:35:30 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=2Sv62L6bSyTwItBz6mLf4a5TGT47pJGWNlBa1RiL3E8=; b=M6z+/pRBMlf/RKqfG9KqszFsi2MG9l6I8Qv05J5VFPJ1fE+AlmxG7mpCqqMBLwnuzxsI lzorQ+PbGVYvVYAItToP1i/VvUxWwByDeVzqIf1F6eno5bGZw33ap3t2puyX5kAEwWSj k2vxISKd56FdAVmm6vEBVV10NawvsO9av0o1arAnNhtreBBm3QYCrISn237slcSo+CMW WZamwvLpMQXEvE/1B+epQMxvcdS82J7VXseUt22Q67cQQrHglH6kZwiJVNFOTPG9DkqF 9s4fmDUhUlK5awLVuTkmPMe1j/OhwWdWEifozm/gaKFl6z75EV/dPdXQjn8TSq4MPHOf rg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurccv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:29 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:28 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:28 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:28 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id F204D3F703F; Sat, 6 Mar 2021 07:35:24 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:43 +0530 Message-ID: <20210306153404.10781-24-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 23/44] net/cnxk: add device start and stop operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add device start and stop operation callbacks for CN9K and CN10K. Device stop is common for both platforms while device start as some platform dependent portion where the platform specific offload flags are recomputed and the right Rx/Tx burst function is chosen. Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/cnxk.rst | 84 ++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.c | 124 +++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.c | 127 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 90 ++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 2 + drivers/net/cnxk/cnxk_link.c | 11 ++++ 6 files changed, 438 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 555730d..42aa7a5 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -39,6 +39,58 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC ` for details. +#. Running testpmd: + + Follow instructions available in the document + :ref:`compiling and testing a PMD for a NIC ` + to run testpmd. + + Example output: + + .. code-block:: console + + .//app/dpdk-testpmd -c 0xc -a 0002:02:00.0 -- --portmask=0x1 --nb-cores=1 --port-topology=loop --rxq=1 --txq=1 + EAL: Detected 4 lcore(s) + EAL: Detected 1 NUMA nodes + EAL: Multi-process socket /var/run/dpdk/rte/mp_socket + EAL: Selected IOVA mode 'VA' + EAL: No available hugepages reported in hugepages-16777216kB + EAL: No available hugepages reported in hugepages-2048kB + EAL: Probing VFIO support... + EAL: VFIO support initialized + EAL: using IOMMU type 1 (Type 1) + [ 2003.202721] vfio-pci 0002:02:00.0: vfio_cap_init: hiding cap 0x14@0x98 + EAL: Probe PCI driver: net_cn10k (177d:a063) device: 0002:02:00.0 (socket 0) + PMD: RoC Model: cn10k + EAL: No legacy callbacks, legacy socket not created + testpmd: create a new mbuf pool : n=155456, size=2176, socket=0 + testpmd: preferred mempool ops selected: cn10k_mempool_ops + Configuring Port 0 (socket 0) + PMD: Port 0: Link Up - speed 25000 Mbps - full-duplex + + Port 0: link state change event + Port 0: 96:D4:99:72:A5:BF + Checking link statuses... + Done + No commandline core given, start packet forwarding + io packet forwarding - ports=1 - cores=1 - streams=1 - NUMA support enabled, MP allocation mode: native + Logical Core 3 (socket 0) forwards packets on 1 streams: + RX P=0/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00 + + io packet forwarding packets/burst=32 + nb forwarding cores=1 - nb forwarding ports=1 + port 0: RX queue number: 1 Tx queue number: 1 + Rx offloads=0x0 Tx offloads=0x10000 + RX queue: 0 + RX desc=4096 - RX free threshold=0 + RX threshold registers: pthresh=0 hthresh=0 wthresh=0 + RX Offloads=0x0 + TX queue: 0 + TX desc=512 - TX free threshold=0 + TX threshold registers: pthresh=0 hthresh=0 wthresh=0 + TX offloads=0x0 - TX RS bit threshold=0 + Press enter to exit + Runtime Config Options ---------------------- @@ -132,3 +184,35 @@ Runtime Config Options Above devarg parameters are configurable per device, user needs to pass the parameters to all the PCIe devices if application requires to configure on all the ethdev ports. + +Limitations +----------- + +``mempool_cnxk`` external mempool handler dependency +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NIC has inbuilt HW assisted external mempool manager. +``net_cnxk`` pmd only works with ``mempool_cnxk`` mempool handler +as it is performance wise most effective way for packet allocation and Tx buffer +recycling on OCTEON TX2 SoC platform. + +CRC stripping +~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by +the host interface irrespective of the offload configuration. + +Debugging Options +----------------- + +.. _table_cnxk_ethdev_debug_options: + +.. table:: cnxk ethdev debug options + + +---+------------+-------------------------------------------------------+ + | # | Component | EAL log command | + +===+============+=======================================================+ + | 1 | NIX | --log-level='pmd\.net.cnxk,8' | + +---+------------+-------------------------------------------------------+ + | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' | + +---+------------+-------------------------------------------------------+ diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 1a9fcbb..f9e0274 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -5,6 +5,98 @@ #include "cn10k_rx.h" #include "cn10k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn10k_eth_set_rx_function(eth_dev); return 0; } @@ -162,6 +255,10 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -169,6 +266,28 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -184,6 +303,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn10k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } @@ -221,6 +341,10 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (!eth_dev) return -ENOENT; + + /* Setup callbacks for secondary process */ + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); } return 0; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 3561632..c0ec0c6 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -5,6 +5,98 @@ #include "cn9k_rx.h" #include "cn9k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn9k_eth_set_rx_function(eth_dev); return 0; } @@ -171,6 +264,10 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -178,6 +275,28 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -193,6 +312,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn9k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } @@ -232,6 +352,13 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) if (!eth_dev) return -ENOENT; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* Setup callbacks for secondary process */ + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; + } + dev = cnxk_eth_pmd_priv(eth_dev); /* Update capabilities already set for TSO. * TSO not supported for earlier chip revisions diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f1ba04f..ba05711 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -908,12 +908,102 @@ cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) return rc; } +static int +cnxk_nix_dev_stop(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct rte_mbuf *rx_pkts[32]; + int count, i, j, rc; + void *rxq; + + /* Disable switch hdr pkind */ + roc_nix_switch_hdr_set(&dev->nix, 0); + + /* Stop link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) + roc_nix_mac_link_event_start_stop(&dev->nix, false); + + /* Disable Rx via NPC */ + roc_nix_npc_rx_ena_dis(&dev->nix, false); + + /* Stop rx queues and free up pkts pending */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = dev_ops->rx_queue_stop(eth_dev, i); + if (rc) + continue; + + rxq = eth_dev->data->rx_queues[i]; + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + while (count) { + for (j = 0; j < count; j++) + rte_pktmbuf_free(rx_pkts[j]); + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + } + } + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_stop(eth_dev, i); + + return 0; +} + +int +cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc, i; + + /* Start rx queues */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = cnxk_nix_rx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Enable Rx in NPC */ + rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); + if (rc) { + plt_err("Failed to enable NPC rx %d", rc); + return rc; + } + + cnxk_nix_toggle_flag_link_cfg(dev, true); + + /* Start link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) { + rc = roc_nix_mac_link_event_start_stop(&dev->nix, true); + if (rc) { + plt_err("Failed to start cgx link event %d", rc); + goto rx_disable; + } + } + + cnxk_nix_toggle_flag_link_cfg(dev, false); + + return 0; + +rx_disable: + roc_nix_npc_rx_ena_dis(&dev->nix, false); + cnxk_nix_toggle_flag_link_cfg(dev, false); + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_stop = cnxk_nix_dev_stop, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 2f31cba..984f4fe 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -214,6 +214,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); +int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); @@ -222,6 +223,7 @@ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); /* Link */ +void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c index 0223d68..8728eb1 100644 --- a/drivers/net/cnxk/cnxk_link.c +++ b/drivers/net/cnxk/cnxk_link.c @@ -4,6 +4,17 @@ #include "cnxk_ethdev.h" +void +cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set) +{ + if (set) + dev->flags |= CNXK_LINK_CFG_IN_PROGRESS_F; + else + dev->flags &= ~CNXK_LINK_CFG_IN_PROGRESS_F; + + rte_wmb(); +} + static inline int nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) { From patchwork Sat Mar 6 15:33:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88621 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CB973A0548; Sat, 6 Mar 2021 16:38:14 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DE6A922A41D; Sat, 6 Mar 2021 16:35:34 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 9548322A3F5 for ; Sat, 6 Mar 2021 16:35:33 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmlp029546 for ; Sat, 6 Mar 2021 07:35:32 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=61YRrDJHL6ywEzsnMQH0M/ZdXj3FuTc6cZfWWP+p15s=; b=Cg1K1ULaFrUB7OjAN1Ah6T3ricjYVNzquv48cERFXs4dUIPKy7MTJVsUzJUaOJPnigml d3qRdDUEzieuypDz0g5nF8oLIZkGepnVdYxQ1d3/rgsOsRWh++MNspwEQNPEiuumSR41 o9pZ5znUeufK6IPhMuAk4BNeYeMH2tPClm+gqbgVnQEN3KKJttgxq3xMYLYyHDF3gcsg /+f0QnODD8/1HNGYNJV2ShHAAwWygfo6F4geqt2AHb84SSgLc7KsD5JSWtu80R1iHUIA yUSLOD/YvHruLJnWhT/YB7KPHxPTXdTZy94XrDK6PEOjX9XNzBQKEDl+TVZhXdI92tXf 5A== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurccx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:32 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:31 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:31 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:30 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 8BCD63F703F; Sat, 6 Mar 2021 07:35:28 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:44 +0530 Message-ID: <20210306153404.10781-25-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 24/44] net/cnxk: add MAC address set ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori Default mac address set operation is implemented for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 32 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index ba05711..ed01087 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -999,6 +999,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 984f4fe..717a8d8 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -203,6 +203,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 4a45956..87cf4ee 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -69,3 +69,32 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; return 0; } + +int +cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + /* Update mac address at NPC */ + rc = roc_nix_npc_mac_addr_set(nix, addr->addr_bytes); + if (rc) + goto exit; + + /* Update mac address at CGX for PFs only */ + if (!roc_nix_is_vf_or_sdp(nix)) { + rc = roc_nix_mac_addr_set(nix, addr->addr_bytes); + if (rc) { + /* Rollback to previous mac address */ + roc_nix_npc_mac_addr_set(nix, dev->mac_addr); + goto exit; + } + } + + /* Update mac address to cnxk ethernet device */ + rte_memcpy(dev->mac_addr, addr->addr_bytes, RTE_ETHER_ADDR_LEN); + +exit: + return rc; +} From patchwork Sat Mar 6 15:33:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88622 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 61721A0548; Sat, 6 Mar 2021 16:38:23 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 12CB922A421; Sat, 6 Mar 2021 16:35:37 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 3586D22A421 for ; Sat, 6 Mar 2021 16:35:36 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWlRW029538 for ; Sat, 6 Mar 2021 07:35:35 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=tFdpnjZCnEQOOifRO/LCQBL50Y5D805l9EI4vlw+jG0=; b=cO3V4YCN7S9V/kgiXQbr+4BMK/uw/Yoh9rr0ocbrodYSJdYsCwXDP4+ojbF1L7NgySLz TkrbLleGTzzeELLt1an9kMJFD34nbYBuIvD54XttiPbGmN9BoK7zf3g/qmdyvk50d9fU qus7+cnRwxqCbISee93TkMbhF4LToaOXxVP7IJxpN3VQjaMK/vz76pw5e7tl9LHl1phT 5zcZbV6B9qKQh4tmVgvN2oWkbuvMFcvPZwrqXkrUELW/NT0m34IQ/htxlQTqJn3BAiRJ YBmq01a54zyDr4selERRYDoU7nNRPjsEmwRs2CmRURb1tkFD/VLtbPtpifLU/3IEx1gE 0A== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcd0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:35 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:33 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:33 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 738953F703F; Sat, 6 Mar 2021 07:35:31 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:45 +0530 Message-ID: <20210306153404.10781-26-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 25/44] net/cnxk: add MTU set device operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori This Patch implements mtu set dev op for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 51 +++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++- drivers/net/cnxk/cnxk_ethdev_ops.c | 77 ++++++++++++++++++++++++++++++++++- 7 files changed, 135 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 42aa7a5..6cb90a7 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Inner and Outer Checksum offload - Link state information +- MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 02be26b..6fef725 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 8c63853..79cb1e2 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a1bd49b..5cc9f3f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index ed01087..9040ce6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,50 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +static void +nix_enable_mseg_on_jumbo(struct cnxk_eth_rxq_sp *rxq) +{ + struct rte_pktmbuf_pool_private *mbp_priv; + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + uint32_t buffsz; + + dev = rxq->dev; + eth_dev = dev->eth_dev; + + /* Get rx buffer size */ + mbp_priv = rte_mempool_get_priv(rxq->qconf.mp); + buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; + + if (eth_dev->data->dev_conf.rxmode.max_rx_pkt_len > buffsz) { + dev->rx_offloads |= DEV_RX_OFFLOAD_SCATTER; + dev->tx_offloads |= DEV_TX_OFFLOAD_MULTI_SEGS; + } +} + +static int +nix_recalc_mtu(struct rte_eth_dev *eth_dev) +{ + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_eth_rxq_sp *rxq; + uint16_t mtu; + int rc; + + rxq = ((struct cnxk_eth_rxq_sp *)data->rx_queues[0]) - 1; + /* Setup scatter mode if needed by jumbo */ + nix_enable_mseg_on_jumbo(rxq); + + /* Setup MTU based on max_rx_pkt_len */ + mtu = data->dev_conf.rxmode.max_rx_pkt_len - CNXK_NIX_L2_OVERHEAD + + CNXK_NIX_MAX_VTAG_ACT_SIZE; + + rc = cnxk_nix_mtu_set(eth_dev, mtu); + if (rc) + plt_err("Failed to set default MTU size, rc=%d", rc); + + return rc; +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -955,6 +999,12 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); int rc, i; + if (eth_dev->data->nb_rx_queues != 0) { + rc = nix_recalc_mtu(eth_dev); + if (rc) + return rc; + } + /* Start rx queues */ for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { rc = cnxk_nix_rx_queue_start(eth_dev, i); @@ -999,6 +1049,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mtu_set = cnxk_nix_mtu_set, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 717a8d8..3838573 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -28,7 +28,9 @@ #define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) /* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ -#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + \ + RTE_ETHER_CRC_LEN + \ + CNXK_NIX_MAX_VTAG_ACT_SIZE) #define CNXK_NIX_RX_MIN_DESC 16 #define CNXK_NIX_RX_MIN_DESC_ALIGN 16 @@ -203,6 +205,7 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 87cf4ee..21b55c4 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -20,7 +20,8 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; devinfo->max_mac_addrs = dev->max_mac_entries; devinfo->max_vfs = pci_dev->max_vfs; - devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->max_mtu = devinfo->max_rx_pktlen - + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN); devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; devinfo->rx_offload_capa = dev->rx_offload_capa; @@ -98,3 +99,77 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) exit: return rc; } + +int +cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) +{ + uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + uint32_t buffsz; + + /* Check if MTU is within the allowed range */ + if ((frame_size - RTE_ETHER_CRC_LEN) < NIX_MIN_HW_FRS) { + plt_err("MTU is lesser than minimum"); + goto exit; + } + + if ((frame_size - RTE_ETHER_CRC_LEN) > + ((uint32_t)roc_nix_max_pkt_len(nix))) { + plt_err("MTU is greater than maximum"); + goto exit; + } + + buffsz = data->min_rx_buf_size - RTE_PKTMBUF_HEADROOM; + old_frame_size = data->mtu + CNXK_NIX_L2_OVERHEAD; + + /* Refuse MTU that requires the support of scattered packets + * when this feature has not been enabled before. + */ + if (data->dev_started && frame_size > buffsz && + !(dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER)) { + plt_err("Scatter offload is not enabled for mtu"); + goto exit; + } + + /* Check * >= max_frame */ + if ((dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) && + frame_size > (buffsz * CNXK_NIX_RX_NB_SEG_MAX)) { + plt_err("Greater than maximum supported packet length"); + goto exit; + } + + frame_size -= RTE_ETHER_CRC_LEN; + + /* Update mtu on Tx */ + rc = roc_nix_mac_mtu_set(nix, frame_size); + if (rc) { + plt_err("Failed to set MTU, rc=%d", rc); + goto exit; + } + + /* Sync same frame size on Rx */ + rc = roc_nix_mac_max_rx_len_set(nix, frame_size); + if (rc) { + /* Rollback to older mtu */ + roc_nix_mac_mtu_set(nix, + old_frame_size - RTE_ETHER_CRC_LEN); + plt_err("Failed to max Rx frame length, rc=%d", rc); + goto exit; + } + + frame_size += RTE_ETHER_CRC_LEN; + + if (frame_size > RTE_ETHER_MAX_LEN) + dev->rx_offloads |= DEV_RX_OFFLOAD_JUMBO_FRAME; + else + dev->rx_offloads &= ~DEV_RX_OFFLOAD_JUMBO_FRAME; + + /* Update max_rx_pkt_len */ + data->dev_conf.rxmode.max_rx_pkt_len = frame_size; + +exit: + return rc; +} From patchwork Sat Mar 6 15:33:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88623 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9B706A0548; Sat, 6 Mar 2021 16:38:34 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A87E422A433; Sat, 6 Mar 2021 16:35:40 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 5B89A22A429 for ; Sat, 6 Mar 2021 16:35:39 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmqW029543 for ; Sat, 6 Mar 2021 07:35:38 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=yRA6phGWpAKBFOOzwkblpDLNaN16FqVPmrTkrPFZfRQ=; b=WOBR/ix/mtIrpTLyrCo2veqDUF1MoAT0x14/fDhWEiJUx/5Q/LLlcxZXj1BOzFcV9cd9 1dNPcWr9DJP3VBhugbEat4CAA08nHS6aABKb+uWR2TpxZ/SybqQKKR0JJ5g4jIn9fHqA a27XmNb5BDfHLBHHvJ8/dqls7177bOeulNvA7aMGFYU0N0riad7eqDNXRwSGTWRZwMQX BXb+S3V8nHZeVharPtwGQcr2z6WEseH+dhxQEVFnRWaNJ3IsYNDnIG4UWQ+7pq6mJPeb fWNXbhSvSIkRFQO0XR1mt572fb7iugrTAy+DJAbZTWUoo7MPN1w0RprLe7sypD6+TmO+ JQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcd8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:38 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:37 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:36 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:36 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 5A3FE3F7044; Sat, 6 Mar 2021 07:35:34 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:46 +0530 Message-ID: <20210306153404.10781-27-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 26/44] net/cnxk: add promiscuous mode enable and disable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori Add device operations to enable and disable promisc mode for cn9k and cn10k. Signed-off-by: Sunil Kumar Kori --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 56 +++++++++++++++++++++++++++++++++++ 6 files changed, 63 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6cb90a7..364e511 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,6 +17,7 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Promiscuous mode - Jumbo frames - SR-IOV VF - Lock-free Tx queue diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 6fef725..9b2e163 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y TSO = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 79cb1e2..31471e0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -16,6 +16,7 @@ Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 9040ce6..8d16dec 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1060,6 +1060,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, + .promiscuous_enable = cnxk_nix_promisc_enable, + .promiscuous_disable = cnxk_nix_promisc_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 3838573..73aef34 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -208,6 +208,8 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); +int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 21b55c4..6feb3a9 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -173,3 +173,59 @@ cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) exit: return rc; } + +int +cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, false); + return rc; + } + + return 0; +} + +int +cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, false); + if (rc < 0) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, false); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, true); + return rc; + } + + return 0; +} From patchwork Sat Mar 6 15:33:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88624 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C962DA0548; Sat, 6 Mar 2021 16:38:42 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DF01022A436; Sat, 6 Mar 2021 16:35:43 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 7B32B22A3A1 for ; Sat, 6 Mar 2021 16:35:42 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmqX029543 for ; Sat, 6 Mar 2021 07:35:41 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=Sl0URkNLl6aos9ecfuCeqmembeXlqz1mncWcp5KZBoc=; b=LszMfV/XUf0fI5OWETa6e3SC4YCBU3GZRx2RpEYw31QNpvMmPpS1WgTO5nWYFNuWrQDT N0dUGCcznoK8Ve+GBlpZV9ACHOlRnEQ+B+HL7qKM3d+bv1Nkx+lYm6gl8AcutdiTyPK7 0EBJrGf/+VCmwQZMV4OMVJIwRo1xv+/7NOxN9M7LBbxdVFn50hT4eAo8wNx5V1lKSZ7z BZNII/cLjabbNOAtwvfk2yYc2B5OjZ3t4le3So7K8QjsPHGorE0bV1OHH0bthjdzPoL7 pCXwFpx09Wv7gkbVldbVQcaIzyzj5B7doyEkS+Qq0upqn6loT/iDnbnCpk4/6GGQdDC2 xA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcdb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:41 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:40 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:39 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:39 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 416153F703F; Sat, 6 Mar 2021 07:35:37 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:47 +0530 Message-ID: <20210306153404.10781-28-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 27/44] net/cnxk: add DMAC filter support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori DMAC filter support is added for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 44 ++++++++++++++++++++++++++++++++--- 6 files changed, 51 insertions(+), 3 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 364e511..ce33f17 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -23,6 +23,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- MAC filtering - Inner and Outer Checksum offload - Link state information - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 9b2e163..20d4d12 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 31471e0..e1de8ab 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 8d16dec..171418a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1050,6 +1050,8 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, + .mac_addr_add = cnxk_nix_mac_addr_add, + .mac_addr_remove = cnxk_nix_mac_addr_del, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 73aef34..38ac654 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -139,6 +139,7 @@ struct cnxk_eth_dev { /* Max macfilter entries */ uint8_t max_mac_entries; + bool dmac_filter_enable; uint16_t flags; uint8_t ptype_disable; @@ -206,6 +207,10 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); +int cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr, uint32_t index, + uint32_t pool); +void cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 6feb3a9..fc60576 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -101,6 +101,43 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) } int +cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, + uint32_t index, uint32_t pool) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + PLT_SET_USED(index); + PLT_SET_USED(pool); + + rc = roc_nix_mac_addr_add(nix, addr->addr_bytes); + if (rc < 0) { + plt_err("Failed to add mac address, rc=%d", rc); + return rc; + } + + /* Enable promiscuous mode at NIX level */ + roc_nix_npc_promisc_ena_dis(nix, true); + dev->dmac_filter_enable = true; + eth_dev->data->promiscuous = false; + + return 0; +} + +void +cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_mac_addr_del(nix, index); + if (rc) + plt_err("Failed to delete mac address, rc=%d", rc); +} + +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) { uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; @@ -212,8 +249,8 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (roc_nix_is_vf_or_sdp(nix)) return rc; - rc = roc_nix_npc_promisc_ena_dis(nix, false); - if (rc < 0) { + rc = roc_nix_npc_promisc_ena_dis(nix, dev->dmac_filter_enable); + if (rc) { plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, roc_error_msg_get(rc)); return rc; @@ -223,9 +260,10 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (rc) { plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, roc_error_msg_get(rc)); - roc_nix_npc_promisc_ena_dis(nix, true); + roc_nix_npc_promisc_ena_dis(nix, !dev->dmac_filter_enable); return rc; } + dev->dmac_filter_enable = false; return 0; } From patchwork Sat Mar 6 15:33:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88625 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3104A0548; Sat, 6 Mar 2021 16:38:50 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 27BEB22A431; Sat, 6 Mar 2021 16:35:46 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 3F9F522A3F3 for ; Sat, 6 Mar 2021 16:35:45 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVqul029350 for ; Sat, 6 Mar 2021 07:35:44 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=X5/qiJUaG1NBWyQ7YYLJTDEatnb5OLCUmdn0cF8KC9k=; b=ViCybJ6SL/gWIu9W2QqJpawpCJXHbbWeXWb3qS0IaZwUD65B7umj1rWN+KJA+4hG/ewX mbvIocb1rcoMDGEI6h5RisRuT/sGYjFizVycJ0D3WWSMRtCfXJtSTp0AlGUsi8ZlWc0Q HlHMZ+TvkCRr4/mpHvGZyaTkpTskPDs6NqZsmfXP1BCHfrCBalEhMXKrvaEQWq644Isn WZJQX61qG5fGwlgOGQCxkVrYSe5AxHTUycb2BXgsp9PMpsQRj3/Xpyf3Ajkr+7PSBO7c dthdRAKVW5esXM0waJGsO1pqeg9XkDABfRUWYVq+Vnh8fFKNMyNxDgaS+zhOlcy8FQcG cA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcde-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:44 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:43 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:42 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:42 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 2860D3F703F; Sat, 6 Mar 2021 07:35:39 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:48 +0530 Message-ID: <20210306153404.10781-29-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 28/44] net/cnxk: add all multicast enable/disable ethops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori L2 multicast packets can be allowed or blocked. Patch implements corresponding ethops. Signed-off-by: Sunil Kumar Kori --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 17 +++++++++++++++++ 5 files changed, 23 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 20d4d12..b41af2d 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e1de8ab..7fe8018 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 171418a..77a8c09 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1064,6 +1064,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, .promiscuous_enable = cnxk_nix_promisc_enable, .promiscuous_disable = cnxk_nix_promisc_disable, + .allmulticast_enable = cnxk_nix_allmulticast_enable, + .allmulticast_disable = cnxk_nix_allmulticast_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 38ac654..09031e9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -215,6 +215,8 @@ int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index fc60576..61ecbab 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -267,3 +267,20 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) dev->dmac_filter_enable = false; return 0; } + +int +cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, true, false); +} + +int +cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, false, + eth_dev->data->promiscuous); +} From patchwork Sat Mar 6 15:33:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88626 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 540A7A0548; Sat, 6 Mar 2021 16:38:59 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 60C1522A441; Sat, 6 Mar 2021 16:35:49 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 5158E22A3E5 for ; Sat, 6 Mar 2021 16:35:48 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmlr029546 for ; Sat, 6 Mar 2021 07:35:47 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=3lX4bJtKIkYBAMs4iycVqJxwAamosdmfptucoMJth/Y=; b=Y83/zizEO5RSP+dWEKnmKOl7KZXBKeu2wYrDrm0iGMnSJdtDKKe4KmomCaB1I9561DAl 4cYa2LdoR9zPPcl+nZijYOt9mTyhflGNjZci9NOUYFK7ZN6v8om7nFgFR8udNtzmJ7K2 DWs6H88igx0L7JXVMZ5NiS9QPFvXmRthXUhRHPY+XXCwyWt8sO/xR/FlcgSa61oCG9xi GkXaFbSQPx52Bogy9wRiTDK7bifoITrN5zTeoxyP4nnfxMfq6FyxYft0fNsXOwFxv0n6 JljA8mE7n+lLaamOMKEM6Y5Hd3ukmR5PAMjlKqbnCiCWZkzs9NjVb83a5fbydooF4oBb 7g== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcdr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:47 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:45 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:45 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:45 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 10BBF3F7040; Sat, 6 Mar 2021 07:35:42 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:49 +0530 Message-ID: <20210306153404.10781-30-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 29/44] net/cnxk: add Rx/Tx burst mode get ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori Patch implements ethdev operations to get Rx and Tx burst mode. Signed-off-by: Sunil Kumar Kori --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 127 ++++++++++++++++++++++++++++++++++ 6 files changed, 136 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b41af2d..298f167 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 7fe8018..a673cc1 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 5cc9f3f..335d082 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 77a8c09..28fcf8c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1066,6 +1066,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .promiscuous_disable = cnxk_nix_promisc_disable, .allmulticast_enable = cnxk_nix_allmulticast_enable, .allmulticast_disable = cnxk_nix_allmulticast_disable, + .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, + .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 09031e9..481ede9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -219,6 +219,10 @@ int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); +int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 61ecbab..7ae961a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -72,6 +72,133 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) } int +cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } rx_offload_map[] = { + {DEV_RX_OFFLOAD_VLAN_STRIP, " VLAN Strip,"}, + {DEV_RX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_LRO, " TCP LRO,"}, + {DEV_RX_OFFLOAD_QINQ_STRIP, " QinQ VLAN Strip,"}, + {DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_MACSEC_STRIP, " MACsec Strip,"}, + {DEV_RX_OFFLOAD_HEADER_SPLIT, " Header Split,"}, + {DEV_RX_OFFLOAD_VLAN_FILTER, " VLAN Filter,"}, + {DEV_RX_OFFLOAD_VLAN_EXTEND, " VLAN Extend,"}, + {DEV_RX_OFFLOAD_JUMBO_FRAME, " Jumbo Frame,"}, + {DEV_RX_OFFLOAD_SCATTER, " Scattered,"}, + {DEV_RX_OFFLOAD_TIMESTAMP, " Timestamp,"}, + {DEV_RX_OFFLOAD_SECURITY, " Security,"}, + {DEV_RX_OFFLOAD_KEEP_CRC, " Keep CRC,"}, + {DEV_RX_OFFLOAD_SCTP_CKSUM, " SCTP,"}, + {DEV_RX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_RX_OFFLOAD_RSS_HASH, " RSS,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Rx Offloads:", + "Scalar, Rx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Rx offload info */ + for (i = 0; i < RTE_DIM(rx_offload_map); i++) { + if (dev->rx_offloads & rx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + rx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int +cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } tx_offload_map[] = { + {DEV_TX_OFFLOAD_VLAN_INSERT, " VLAN Insert,"}, + {DEV_TX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_TX_OFFLOAD_SCTP_CKSUM, " SCTP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_TSO, " TCP TSO,"}, + {DEV_TX_OFFLOAD_UDP_TSO, " UDP TSO,"}, + {DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_QINQ_INSERT, " QinQ VLAN Insert,"}, + {DEV_TX_OFFLOAD_VXLAN_TNL_TSO, " VXLAN Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GRE_TNL_TSO, " GRE Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IPIP_TNL_TSO, " IP-in-IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GENEVE_TNL_TSO, " Geneve Tunnel TSO,"}, + {DEV_TX_OFFLOAD_MACSEC_INSERT, " MACsec Insert,"}, + {DEV_TX_OFFLOAD_MT_LOCKFREE, " Multi Thread Lockless Tx,"}, + {DEV_TX_OFFLOAD_MULTI_SEGS, " Scattered,"}, + {DEV_TX_OFFLOAD_MBUF_FAST_FREE, " H/W MBUF Free,"}, + {DEV_TX_OFFLOAD_SECURITY, " Security,"}, + {DEV_TX_OFFLOAD_UDP_TNL_TSO, " UDP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IP_TNL_TSO, " IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP, " Timestamp,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Tx Offloads:", + "Scalar, Tx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Tx offload info */ + for (i = 0; i < RTE_DIM(tx_offload_map); i++) { + if (dev->tx_offloads & tx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + tx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); From patchwork Sat Mar 6 15:33:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88627 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 59F76A0548; Sat, 6 Mar 2021 16:39:09 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E986222A447; Sat, 6 Mar 2021 16:35:52 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 13F1D22A447 for ; Sat, 6 Mar 2021 16:35:50 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVumP029368 for ; Sat, 6 Mar 2021 07:35:50 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=xfaaxrf4spgVNOPcnbwyvarsqA2kzW+2tiLL6oIkKvE=; b=kTUeLcp8WGEpPc9CA6ZK+C1Ap27pWsEHxFV7PD2tQ+bSe+9jwSXnb3am8jGWI4Eszli1 W144V+RDeDSx7SZEkilUuUxR/RFnR+EzvAeVa4F96fluwAfGGirBFz/dD/z8ll2H5aXu Hb1tycDD0GnR0lW0oC/URdqn+hNmcKqELStu9hFHAYveyLeIBboQJGJQ6BvEyGRPyyw9 H//Bu6I3IuBR+dJkOd5On5GDPoQNE2XI5ZtycDhXpql4aDaKyvN4YlQGYxtF/sXbHZsr ZNiW3+1LMK9IZffK68gVQKUvD9kJskhpuQADVJvED4yJr4WGss/hxDG4fswERwUWWN48 MQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcdw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:50 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:48 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:48 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:48 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id ED2AD3F703F; Sat, 6 Mar 2021 07:35:45 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:50 +0530 Message-ID: <20210306153404.10781-31-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 30/44] net/cnxk: add flow ctrl set/get ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori Patch implements set and get operations for flow control. Signed-off-by: Sunil Kumar Kori --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 74 +++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 95 +++++++++++++++++++++++++++++++++++ 6 files changed, 185 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index ce33f17..96b2c5d 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -26,6 +26,7 @@ Features of the CNXK Ethdev PMD are: - MAC filtering - Inner and Outer Checksum offload - Link state information +- Link flow control - MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 298f167..afd0f01 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -23,6 +23,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y Scattered Rx = Y L3 checksum offload = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index a673cc1..4bd11ce 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -22,6 +22,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y L3 checksum offload = Y L4 checksum offload = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 28fcf8c..0ffc45b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -81,6 +81,55 @@ nix_recalc_mtu(struct rte_eth_dev *eth_dev) return rc; } +static int +nix_init_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_conf = {0}; + int rc; + + /* Both Rx & Tx flow ctrl get enabled(RTE_FC_FULL) in HW + * by AF driver, update those info in PMD structure. + */ + rc = cnxk_nix_flow_ctrl_get(eth_dev, &fc_conf); + if (rc) + goto exit; + + fc->mode = fc_conf.mode; + fc->rx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_RX_PAUSE); + fc->tx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_TX_PAUSE); + +exit: + return rc; +} + +static int +nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_cfg = {0}; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + return 0; + + fc_cfg.mode = fc->mode; + + /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ + if (roc_model_is_cn96_Ax() && + (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { + fc_cfg.mode = + (fc_cfg.mode == RTE_FC_FULL || + fc_cfg.mode == RTE_FC_TX_PAUSE) ? + RTE_FC_TX_PAUSE : RTE_FC_NONE; + } + + return cnxk_nix_flow_ctrl_set(eth_dev, &fc_cfg); +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -640,6 +689,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct rte_eth_rxmode *rxmode = &conf->rxmode; struct rte_eth_txmode *txmode = &conf->txmode; char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix_fc_cfg fc_cfg = {0}; struct roc_nix *nix = &dev->nix; struct rte_ether_addr *ea; uint8_t nb_rxq, nb_txq; @@ -820,6 +870,21 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto cq_fini; } + /* Init flow control configuration */ + fc_cfg.cq_cfg_valid = false; + fc_cfg.rxchan_cfg.enable = true; + rc = roc_nix_fc_config_set(nix, &fc_cfg); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } + + /* Update flow control configuration to PMD */ + rc = nix_init_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } /* * Restore queue config when reconfigure followed by * reconfigure and no queue configure invoked from application case. @@ -1019,6 +1084,13 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } + /* Update Flow control configuration */ + rc = nix_update_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to enable flow control. error code(%d)", rc); + return rc; + } + /* Enable Rx in NPC */ rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); if (rc) { @@ -1068,6 +1140,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .allmulticast_disable = cnxk_nix_allmulticast_disable, .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, + .flow_ctrl_get = cnxk_nix_flow_ctrl_get, + .flow_ctrl_set = cnxk_nix_flow_ctrl_set, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 481ede9..77139d0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -113,6 +113,12 @@ ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) +struct cnxk_fc_cfg { + enum rte_eth_fc_mode mode; + uint8_t rx_pause; + uint8_t tx_pause; +}; + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -174,6 +180,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Flow control configuration */ + struct cnxk_fc_cfg fc_cfg; + /* Rx burst for cleanup(Only Primary) */ eth_rx_burst_t rx_pkt_burst_no_offload; @@ -223,6 +232,10 @@ int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); +int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 7ae961a..eac50a2 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -199,6 +199,101 @@ cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, } int +cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum rte_eth_fc_mode mode_map[] = { + RTE_FC_NONE, RTE_FC_RX_PAUSE, + RTE_FC_TX_PAUSE, RTE_FC_FULL + }; + struct roc_nix *nix = &dev->nix; + int mode; + + mode = roc_nix_fc_mode_get(nix); + if (mode < 0) + return mode; + + memset(fc_conf, 0, sizeof(struct rte_eth_fc_conf)); + fc_conf->mode = mode_map[mode]; + return 0; +} + +int +cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum roc_nix_fc_mode mode_map[] = { + ROC_NIX_FC_NONE, ROC_NIX_FC_RX, + ROC_NIX_FC_TX, ROC_NIX_FC_FULL + }; + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct roc_nix *nix = &dev->nix; + struct roc_nix_fc_cfg fc_cfg; + struct cnxk_eth_rxq_sp *rxq; + uint8_t rx_pause, tx_pause; + struct roc_nix_cq *cq; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) { + plt_err("Flow control configuration is not allowed on VFs"); + return -ENOTSUP; + } + + if (fc_conf->high_water || fc_conf->low_water || fc_conf->pause_time || + fc_conf->mac_ctrl_frame_fwd || fc_conf->autoneg) { + plt_info("Only MODE configuration is supported"); + return -EINVAL; + } + + if (fc_conf->mode == fc->mode) + return 0; + + rx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_RX_PAUSE); + tx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_TX_PAUSE); + + /* Check if TX pause frame is already enabled or not */ + if (fc->tx_pause ^ tx_pause) { + if (roc_model_is_cn96_Ax() && data->dev_started) { + /* On Ax, CQ should be in disabled state + * while setting flow control configuration. + */ + plt_info("Stop the port=%d for setting flow control", + data->port_id); + return 0; + } + + for (i = 0; i < data->nb_rx_queues; i++) { + memset(&fc_cfg, 0, sizeof(struct roc_nix_fc_cfg)); + rxq = ((struct cnxk_eth_rxq_sp *) + data->rx_queues[i]) - 1; + cq = &dev->cqs[rxq->qid]; + fc_cfg.cq_cfg_valid = true; + fc_cfg.cq_cfg.enable = tx_pause; + fc_cfg.cq_cfg.rq = rxq->qid; + fc_cfg.cq_cfg.cq_drop = cq->drop_thresh; + rc = roc_nix_fc_config_set(nix, &fc_cfg); + if (rc) + return rc; + } + } + + rc = roc_nix_fc_mode_set(nix, mode_map[fc_conf->mode]); + if (rc) + return rc; + + fc->rx_pause = rx_pause; + fc->tx_pause = tx_pause; + fc->mode = fc_conf->mode; + + return rc; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); From patchwork Sat Mar 6 15:33:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88628 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 06333A0548; Sat, 6 Mar 2021 16:39:18 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 24C2F22A425; Sat, 6 Mar 2021 16:35:55 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 9848122A3BC for ; Sat, 6 Mar 2021 16:35:53 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FXJQq032533 for ; Sat, 6 Mar 2021 07:35:52 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=coZuXfT4edBbOiWQ9jtNt5Xovow08wuevjO7eqHFV+8=; b=A9N2eXzsei+KAQovG242yBRwrXfKquRz8GBTPst8mHJ2RDpAtwrFuOmm9oLHMaSFlaQG k3r1i9nS8Xe7LkWZutvbjyhCDAHW3PnvYujXNduIG9UlrIa4OuiVByh0onvQnqImkx5e ufxBpP6XpOTecSjvgCyjxJ0cDeZxxqCRPvh45gZmnXzIh2XLk0Hq/YaratAWVbJF+nBu qtrleKImjBtQPiU4fgRpaZ+9wlV5dBcYbS10w+dbXvLBs063BmZ03+D3l7OJ3bBcc88U dpbTaj5kgZptQOywKFWhlJlXtcHzYeW54nBM+5Q7oVKzmkt5r4gb/rfnlnlaEcHv0LOW sA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurce1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:52 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:51 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:51 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id E262D3F703F; Sat, 6 Mar 2021 07:35:48 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:51 +0530 Message-ID: <20210306153404.10781-32-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 31/44] net/cnxk: add link up/down operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori Patch implements link up/down ethdev operations for cn9k and cn10k platform. Signed-off-by: Sunil Kumar Kori --- drivers/net/cnxk/cnxk_ethdev.c | 4 +++- drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 47 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 0ffc45b..cb7404f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -928,7 +928,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } -static int +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -1142,6 +1142,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, .flow_ctrl_get = cnxk_nix_flow_ctrl_get, .flow_ctrl_set = cnxk_nix_flow_ctrl_set, + .dev_set_link_up = cnxk_nix_set_link_up, + .dev_set_link_down = cnxk_nix_set_link_down, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 77139d0..6500433 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -236,6 +236,9 @@ int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); +int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); + int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, @@ -244,6 +247,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index eac50a2..37ba211 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -506,3 +506,50 @@ cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) return roc_nix_npc_mcast_config(&dev->nix, false, eth_dev->data->promiscuous); } + +int +cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + rc = roc_nix_mac_link_state_set(nix, true); + if (rc) + goto exit; + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + goto exit; + } + +exit: + return rc; +} + +int +cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_stop(eth_dev, i); + if (rc) + goto exit; + } + + rc = roc_nix_mac_link_state_set(nix, false); +exit: + return rc; +} From patchwork Sat Mar 6 15:33:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88629 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 76ABDA0548; Sat, 6 Mar 2021 16:39:26 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6996E22A3D0; Sat, 6 Mar 2021 16:35:58 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 7372E4014D for ; Sat, 6 Mar 2021 16:35:57 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmlu029546 for ; Sat, 6 Mar 2021 07:35:56 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=2d4vKv0pLBJkiR/F4oNmU7Sc172xFXqU/44Zotf5/Pc=; b=PfsZ8VjvuBGTFKEbhm82e3CJ81L7v6J6vztlyCiQIH1FbBGJogywcDVbg4I9nlMlw/S4 sv3NGzWttoDd+svV2NpZQ1drXCez4u7SmGa+u1Bj2Dn5Ujy5h6+918aRTcJGiLfZOYMR MmSJUer/xflokQiIv6cREkGAWxtXGOgAo7Kle8aeI61f7g2kRg7VviwPdLM4nHA71qx8 GLiw5TCVMR91D5SQXwUX4P3VWUP1aaAs+rhT9RJrjbfmc3sC6MYJorRrB4zUHFHE1WNV MAa+13ROoX9Gjioqzf+kITgdZCXkfPuXDYUoytXVI7u4Sg54BKFJfhDtBwPec2tjP56q qw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurce4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:56 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:55 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:54 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:54 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id E0BF53F7040; Sat, 6 Mar 2021 07:35:51 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:52 +0530 Message-ID: <20210306153404.10781-33-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 32/44] net/cnxk: add EEPROM module info get operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori Patch implements eeprom module info get ethops for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 39 +++++++++++++++++++++++++++++++++++ 6 files changed, 48 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index afd0f01..b1e8641 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -31,6 +31,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 4bd11ce..0f99634 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -29,6 +29,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 335d082..cecced9 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -26,6 +26,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index cb7404f..97d8e6d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1144,6 +1144,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .flow_ctrl_set = cnxk_nix_flow_ctrl_set, .dev_set_link_up = cnxk_nix_set_link_up, .dev_set_link_down = cnxk_nix_set_link_down, + .get_module_info = cnxk_nix_get_module_info, + .get_module_eeprom = cnxk_nix_get_module_eeprom, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 6500433..c4a562b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -238,6 +238,10 @@ int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); +int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo); +int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 37ba211..a1a963a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -553,3 +553,42 @@ cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) exit: return rc; } + +int +cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + modinfo->type = eeprom_info.sff_id; + modinfo->eeprom_len = ROC_NIX_EEPROM_SIZE; + return 0; +} + +int +cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + + if (!info->data || !info->length || + (info->offset + info->length > ROC_NIX_EEPROM_SIZE)) + return rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); + return 0; +} From patchwork Sat Mar 6 15:33:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88630 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 826C0A0548; Sat, 6 Mar 2021 16:39:35 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 91DF522A44F; Sat, 6 Mar 2021 16:36:01 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 739E422A3DE for ; Sat, 6 Mar 2021 16:36:00 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVquo029350 for ; Sat, 6 Mar 2021 07:35:59 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=WjU42tNIV+pSeo6MCwwlt5Hr9m2ZVifs0hldGu08s3c=; b=MhKoxFeCLyfeQOS5IkJHbgnDAUofdQNXlV1hwElV+93uRfmOFJShuWq9q4paTOdTnAr3 vxqdyym/ifoLryeFfqPErpRHbAjZNLR2lS3zULMz71ZRiRqTtE7H0XfFXKr2rq2PojUY E5x8s0YZZNSMrsHM1DjO22CV6NJ01jru8FRqS2i1Vh1WDlfKvFTcVf5NUNWGVDc43+J+ yNFThzfYhn1LiCFK0UeakgrslNHloCWGDpyB3aV18AA3+ah9H5DzneVFBzbM6r6JLh4h n9+YTam0Nb/jCVioGCwc4xpMftNoWiC2BLliZ+V7HBKT6+x8mNge6v2KN2p9DN/t+Vuo vQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurce6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:35:59 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:58 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:35:57 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:35:57 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 58B533F703F; Sat, 6 Mar 2021 07:35:55 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:53 +0530 Message-ID: <20210306153404.10781-34-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 33/44] net/cnxk: add Rx queue interrupt enable/disable ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori Application may choose to enable/disable interrupts on Rx queues so that application can select its processing if no packets are available on queues for a longer period. Signed-off-by: Sunil Kumar Kori --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 7 files changed, 29 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 96b2c5d..6a001d9 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -30,6 +30,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Support Rx interrupt Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b1e8641..e5669f5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 0f99634..dff0c9b 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index cecced9..b950d2f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 97d8e6d..bfcc456 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1146,6 +1146,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_set_link_down = cnxk_nix_set_link_down, .get_module_info = cnxk_nix_get_module_info, .get_module_eeprom = cnxk_nix_get_module_eeprom, + .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, + .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index c4a562b..76e1049 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -242,6 +242,10 @@ int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, struct rte_eth_dev_module_info *modinfo); int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, struct rte_dev_eeprom_info *info); +int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); +int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index a1a963a..34d4a42 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -592,3 +592,22 @@ cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); return 0; } + +int +cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_enable(&dev->nix, rx_queue_id); + return 0; +} + +int +cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); + return 0; +} From patchwork Sat Mar 6 15:33:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88631 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 498D7A0548; Sat, 6 Mar 2021 16:39:46 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2CFA822A3EE; Sat, 6 Mar 2021 16:36:05 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 4400C22A3EE for ; Sat, 6 Mar 2021 16:36:03 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmlv029546 for ; Sat, 6 Mar 2021 07:36:02 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=DYapsutOC5A23NV/7XqLZfJfwaUYGgMLYg6Uq67ymaY=; b=E+IkOk7VC528ztY5bW+34wfWDlUDmrLbc1Xpn7CEPeX3Dsk+Ch/tcUC/rBqtQyIpltOE hD+QSRUNMWV8p2RNTVL/aT/sW5XV5n4q+13WuJfaFWnyOdQaXdS3/b7ag59ncBOCvvV3 x9o8tlImw9OBw5JlgsKhrvHoSG9EMK+1C9XBOOoRx26ba24vFyJ6Ag6fcX5gdsohFYmZ OMPNWjI0QKKdTzJSK6kQDQCuF/D4fYw7MMWfJ4XXjm8kphzRNaNPQpax56jSBg/VoyF5 H60rynKmhm0pQK6ZynYPWnnFmk64GeXeJkT+izNLPfhUyALur0XMSKiUmVUpvD8+aV90 uw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurce8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:02 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:01 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:00 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:00 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 5F3653F703F; Sat, 6 Mar 2021 07:35:58 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:54 +0530 Message-ID: <20210306153404.10781-35-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 34/44] net/cnxk: add validation API for mempool ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori cn9k and cn10k supports platform specific mempool ops. This patch implements API to validate whether given mempool ops is supported or not. Signed-off-by: Sunil Kumar Kori --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 11 +++++++++++ 3 files changed, 13 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index bfcc456..8a76486 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1148,6 +1148,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .get_module_eeprom = cnxk_nix_get_module_eeprom, .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, + .pool_ops_supported = cnxk_nix_pool_ops_supported, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 76e1049..0b501f6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -246,6 +246,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); +int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 34d4a42..5b8bc53 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -611,3 +611,14 @@ cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); return 0; } + +int +cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) +{ + RTE_SET_USED(eth_dev); + + if (!strcmp(pool, rte_mbuf_platform_mempool_ops())) + return 0; + + return -ENOTSUP; +} From patchwork Sat Mar 6 15:33:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88632 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BEF88A0548; Sat, 6 Mar 2021 16:39:54 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6832B22A45A; Sat, 6 Mar 2021 16:36:09 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 0CCC422A430 for ; Sat, 6 Mar 2021 16:36:06 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FU4UV005224 for ; Sat, 6 Mar 2021 07:36:06 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=ZVOGOw/NuqDujgFmhfrZowHPqIY0bebnw2VaMoGHuwk=; b=ExG0Xkhnt2al/v4aW3GwxyZofv5m77vy2nMhUnrfHnIHsR7SMzc/SWG2UeLi71ZsgFlX 7ZyhsH5+d7Zix0GisCnCvOtCRcrlmN4pwwbmebCqaMf7NX2gIr3oREtj1WhJhW6m7FYu So6rfbIk/VyrARkM6zytg7PtwW1Hvel1ACh4/+5ltadocROffS2nxhSH4h9rMYjj5Vvv g/8zAL7vZGrAKr3t32UhHv9ctOXkXXsFU8LNJURB6Gh26sXSGtduvLrEKH0iyFBj+NEn /to+2YRmFcHd1lTk7ECVi1sk5pr0GLKRJXD8UmENljXkPBUURq2cnNTMA8A9irIW1nm3 NQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 374a4w06by-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:06 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:04 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:03 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:03 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 5E59B3F7040; Sat, 6 Mar 2021 07:36:01 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:55 +0530 Message-ID: <20210306153404.10781-36-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 35/44] net/cnxk: add port/queue stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Satha Rao This patch implements regular port statistics and queue mapping set api to get queue statistics Signed-off-by: Satha Rao --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 3 ++ drivers/net/cnxk/cnxk_ethdev.h | 8 ++++ drivers/net/cnxk/cnxk_stats.c | 85 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 105 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_stats.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6a001d9..c2a6fbb 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -25,6 +25,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - MAC filtering - Inner and Outer Checksum offload +- Port hardware statistics - Link state information - Link flow control - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index e5669f5..40952a9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -32,6 +32,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index dff0c9b..32035bb 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -30,6 +30,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index b950d2f..8060a68 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -27,6 +27,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 8a76486..a798b14 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1149,6 +1149,9 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, .pool_ops_supported = cnxk_nix_pool_ops_supported, + .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, + .stats_get = cnxk_nix_stats_get, + .stats_reset = cnxk_nix_stats_reset, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 0b501f6..5075e7c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -188,6 +188,10 @@ struct cnxk_eth_dev { /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; + + /* Per queue statistics counters */ + uint32_t txq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; + uint32_t rxq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; }; struct cnxk_eth_rxq_sp { @@ -271,6 +275,10 @@ void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx); +int cnxk_nix_stats_reset(struct rte_eth_dev *dev); +int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c new file mode 100644 index 0000000..24bff0b --- /dev/null +++ b/drivers/net/cnxk/cnxk_stats.c @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +int +cnxk_nix_stats_get(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct roc_nix_stats nix_stats; + int rc = 0, i; + + rc = roc_nix_stats_get(nix, &nix_stats); + if (rc) + goto exit; + + stats->opackets = nix_stats.tx_ucast; + stats->opackets += nix_stats.tx_mcast; + stats->opackets += nix_stats.tx_bcast; + stats->oerrors = nix_stats.tx_drop; + stats->obytes = nix_stats.tx_octs; + + stats->ipackets = nix_stats.rx_ucast; + stats->ipackets += nix_stats.rx_mcast; + stats->ipackets += nix_stats.rx_bcast; + stats->imissed = nix_stats.rx_drop; + stats->ibytes = nix_stats.rx_octs; + stats->ierrors = nix_stats.rx_err; + + for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS; i++) { + struct roc_nix_stats_queue qstats; + uint16_t qidx; + + if (dev->txq_stat_map[i] & (1U << 31)) { + qidx = dev->txq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 0, &qstats); + if (rc) + goto exit; + stats->q_opackets[i] = qstats.tx_pkts; + stats->q_obytes[i] = qstats.tx_octs; + stats->q_errors[i] = qstats.tx_drop_pkts; + } + + if (dev->rxq_stat_map[i] & (1U << 31)) { + qidx = dev->rxq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 1, &qstats); + if (rc) + goto exit; + stats->q_ipackets[i] = qstats.rx_pkts; + stats->q_ibytes[i] = qstats.rx_octs; + stats->q_errors[i] += qstats.rx_drop_pkts; + } + } +exit: + return rc; +} + +int +cnxk_nix_stats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_stats_reset(&dev->nix); +} + +int +cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (is_rx) { + if (queue_id >= dev->nb_rxq) + return -EINVAL; + dev->rxq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } else { + if (queue_id >= dev->nb_txq) + return -EINVAL; + dev->txq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } + + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 3aa9f70..0e141d7 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -12,7 +12,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', - 'cnxk_lookup.c') + 'cnxk_lookup.c', + 'cnxk_stats.c') # CN9K sources += files('cn9k_ethdev.c', From patchwork Sat Mar 6 15:33:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88633 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 21146A0548; Sat, 6 Mar 2021 16:40:04 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7BDC822A462; Sat, 6 Mar 2021 16:36:10 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 70E8522A45B for ; Sat, 6 Mar 2021 16:36:09 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FYJGN001300 for ; Sat, 6 Mar 2021 07:36:08 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=roHsQzMana4wWTe0QBm8UThCz3jI7aVl1kVhTpWNYj4=; b=eqoYQF/PAUS09xXSAtbS5cu1B51DRje+RfRv6F7HpOeb7NLkcv5u2VOq9gLOGRJVXdqn pras1sJJ27vbZWfMpIkvzATDqsxLD3pE9L6yfDsYrzucpNcL4xCc7/m9NBysUnxwVhp3 xxn/ybZOTbup/pA70SQxR53zzM6jvPeA+IeICxAhGhF6ZUOPyywHLaZZnylkZMehOYwc mcTD2SXzwtPfvk7L9a04kYEAAh/ClCHNfZuZUt/rhYYkU3TQwHVZXHT7KPrXQnh9U+S6 1/6hrSU90a2Wn7qbqmcvtTgmjF+ubXNxdp5CEotm1RRBPPh+GnYSszPHm1igF4U4MuX3 hA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurced-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:08 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:07 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:07 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:06 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 600FE3F703F; Sat, 6 Mar 2021 07:36:04 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:56 +0530 Message-ID: <20210306153404.10781-37-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 36/44] net/cnxk: add xstats apis X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Satha Rao Initial implementation of xstats operations. Signed-off-by: Satha Rao --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 5 ++ drivers/net/cnxk/cnxk_ethdev.h | 11 +++ drivers/net/cnxk/cnxk_stats.c | 132 ++++++++++++++++++++++++++++++++++ 6 files changed, 151 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 40952a9..192c15a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -34,6 +34,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 32035bb..e990480 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -32,6 +32,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8060a68..3a4417c 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -29,6 +29,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a798b14..a145aaa 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1152,6 +1152,11 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, .stats_get = cnxk_nix_stats_get, .stats_reset = cnxk_nix_stats_reset, + .xstats_get = cnxk_nix_xstats_get, + .xstats_get_names = cnxk_nix_xstats_get_names, + .xstats_reset = cnxk_nix_xstats_reset, + .xstats_get_by_id = cnxk_nix_xstats_get_by_id, + .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 5075e7c..dd05c24 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -279,6 +279,17 @@ int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, uint8_t stat_idx, uint8_t is_rx); int cnxk_nix_stats_reset(struct rte_eth_dev *dev); int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); +int cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat *xstats, unsigned int n); +int cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit); +int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit); +int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n); +int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c index 24bff0b..ce9f9f4 100644 --- a/drivers/net/cnxk/cnxk_stats.c +++ b/drivers/net/cnxk/cnxk_stats.c @@ -83,3 +83,135 @@ cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, return 0; } + +int +cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, struct rte_eth_xstat *xstats, + unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat roc_xstats[n]; + int size, i; + + size = roc_nix_xstats_get(&dev->nix, roc_xstats, n); + + /* If requested array do not have space then return with count */ + if (size < 0 || size > (int)n) + return size; + + for (i = 0; i < size; i++) { + xstats[i].id = roc_xstats[i].id; + xstats[i].value = roc_xstats[i].value; + } + + return size; +} + +int +cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat_name roc_xstats_name[limit]; + struct roc_nix *nix = &dev->nix; + int size, i; + + if ((int)limit < roc_nix_num_xstats_get(nix) && xstats_names == NULL) + return roc_nix_num_xstats_get(nix); + + size = roc_nix_xstats_names_get(nix, roc_xstats_name, limit); + + for (i = 0; i < size; i++) + strlcpy(xstats_names[i].name, roc_xstats_name[i].name, + sizeof(xstats_names[i].name)); + + return size; +} + +int +cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t xstat_cnt = roc_nix_num_xstats_get(&dev->nix), i; + struct roc_nix_xstat_name xnames[xstat_cnt]; + + if (limit < xstat_cnt && ids == NULL) + return xstat_cnt; + + if (limit > xstat_cnt) + return -EINVAL; + + if (xstats_names == NULL) + return -ENOMEM; + + roc_nix_xstats_names_get(&dev->nix, xnames, limit); + + for (i = 0; i < xstat_cnt; i++) { + if (ids[i] >= xstat_cnt) + return -EINVAL; + + strlcpy(xstats_names[i].name, xnames[ids[i]].name, + sizeof(xstats_names[i].name)); + } + + return limit; +} + +int +cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t xstat_cnt = roc_nix_num_xstats_get(&dev->nix), i; + struct roc_nix_xstat xstats[xstat_cnt]; + + if (n < xstat_cnt && ids == NULL) + return xstat_cnt; + + if (n > xstat_cnt) + return -EINVAL; + + if (values == NULL) + return -ENOMEM; + + roc_nix_xstats_get(&dev->nix, xstats, n); + + for (i = 0; i < xstat_cnt; i++) { + if (ids[i] >= xstat_cnt) + return -EINVAL; + values[i] = xstats[ids[i]].value; + } + + return n; +} + +int +cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0, i; + + rc = roc_nix_stats_reset(nix); + if (rc) + goto exit; + + /* Reset Rx Queues */ + for (i = 0; i < dev->nb_rxq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 1); + if (rc) + goto exit; + } + + /* Reset Tx Queues */ + for (i = 0; i < dev->nb_txq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 0); + if (rc) + goto exit; + } + +exit: + return rc; +} From patchwork Sat Mar 6 15:33:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88634 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 20311A0548; Sat, 6 Mar 2021 16:40:13 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A47AE22A40B; Sat, 6 Mar 2021 16:36:13 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id CE74522A40B for ; Sat, 6 Mar 2021 16:36:12 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FYUWK012457 for ; Sat, 6 Mar 2021 07:36:12 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=BNjFHpHij1qxkHJq1NR+gj/QJKUby79pDVG8ewU0e8w=; b=MeF15BNSMciMaiaFOnIjZTxpGh+/CCyHwF5vrwg9hVXz36TIoy46HIAeABJiLR5410kr Snd4jjrND3HwfunGpuo4KxZt6evM22W6Dbp3zdZ8m75JqLkE65kPJeayb8u2cQLDAPfZ pmPHi2NzLMQdMiteFEzKxS2WDguSGIfbMFcetpW/owozCL5B5Ag0CMiRXkDsbJQbgSHZ w0elq1QcVWLFo5VyfT+2r1wGZ3ANBlJOKVfoLmJYouWaG+2otexIos5ca6bsVoOQIuNW CEQv0z4Xi4ctmp+sZv+S8RuxtmoWDeArLXqkUmyAqFclXG8wNeNxa/5h/ZMQQ07jZhhb RQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 374a4w06c2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:12 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:10 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:09 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:09 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 648653F703F; Sat, 6 Mar 2021 07:36:07 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:57 +0530 Message-ID: <20210306153404.10781-38-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 37/44] net/cnxk: add rxq/txq info get operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Satha Rao Initial apis to get default queue information. Signed-off-by: Satha Rao --- drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 36 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a145aaa..24c51b4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1157,6 +1157,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .rxq_info_get = cnxk_nix_rxq_info_get, + .txq_info_get = cnxk_nix_txq_info_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index dd05c24..eeb6a53 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -290,6 +290,10 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo); +void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 5b8bc53..0bcba0c 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -622,3 +622,33 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } + +void +cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo) +{ + struct cnxk_eth_rxq_sp *rxq_sp = + ((struct cnxk_eth_rxq_sp *)eth_dev->data->rx_queues[qid]) - 1; + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->mp = rxq_sp->qconf.mp; + qinfo->scattered_rx = eth_dev->data->scattered_rx; + qinfo->nb_desc = rxq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &rxq_sp->qconf.conf.rx, sizeof(qinfo->conf)); +} + +void +cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo) +{ + struct cnxk_eth_txq_sp *txq_sp = + ((struct cnxk_eth_txq_sp *)eth_dev->data->tx_queues[qid]) - 1; + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->nb_desc = txq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); +} From patchwork Sat Mar 6 15:33:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88635 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B2B47A0548; Sat, 6 Mar 2021 16:40:24 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4CD1B22A475; Sat, 6 Mar 2021 16:36:17 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 5A41A22A475 for ; Sat, 6 Mar 2021 16:36:15 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FWmlw029546 for ; Sat, 6 Mar 2021 07:36:14 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=olqRUjlH2iLWFY02tX4g50mI5hD9teZdOQ5utdl1nz0=; b=YFCjA3YBwXP3sRD4WFXZFlD+oy/E7Wr6QKozGj/CqyTYzBiY6ALvhjIEOVLF93TFlbnF X/MW7RFl34tXduOMoIKlfhE8d/dWAVticsTYQOjzfbCuttTR7CZFSfC6bxmbWWOXI6wM mYuRojoh4oxHPI/iQO07RuKwsxF8aQdNwH12bzKHKoZvRBGFWba2eZ2DmLnaq3Z20olC P3ZCNI9ifocU0236rUi7shIyismOHL9vhqcanxtcsEtcvwf5BfDYKF/5c7EM5xx+A6OI ybk5f6SE/T3VSLIwJUlU6Vp6lKHtJIbPgy7EwrUszMJKwFgG77slf1Q8a1XEx4wT+sE9 5Q== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurceh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:14 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:13 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:12 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:12 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 62BBF3F7043; Sat, 6 Mar 2021 07:36:10 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:58 +0530 Message-ID: <20210306153404.10781-39-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 38/44] net/cnxk: add device close and reset operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori Patch implements device close and reset operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori --- drivers/net/cnxk/cnxk_ethdev.c | 35 ++++++++++++++++++++++++++++------- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 24c51b4..86dabad 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1119,6 +1119,9 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } +static int cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev); +static int cnxk_nix_dev_close(struct rte_eth_dev *eth_dev); + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, @@ -1130,6 +1133,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, .dev_stop = cnxk_nix_dev_stop, + .dev_close = cnxk_nix_dev_close, + .dev_reset = cnxk_nix_dev_reset, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, @@ -1270,7 +1275,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } static int -cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; @@ -1324,14 +1329,11 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs = NULL; - /* Check if mbox close is needed */ - if (!mbox_close) - return 0; - rc = roc_nix_dev_fini(nix); /* Can be freed later by PMD if NPA LF is in use */ if (rc == -EAGAIN) { - eth_dev->data->dev_private = NULL; + if (!reset) + eth_dev->data->dev_private = NULL; return 0; } else if (rc) { plt_err("Failed in nix dev fini, rc=%d", rc); @@ -1340,6 +1342,25 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) return rc; } +static int +cnxk_nix_dev_close(struct rte_eth_dev *eth_dev) +{ + cnxk_eth_dev_uninit(eth_dev, false); + return 0; +} + +static int +cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev) +{ + int rc; + + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + return cnxk_eth_dev_init(eth_dev); +} + int cnxk_nix_remove(struct rte_pci_device *pci_dev) { @@ -1350,7 +1371,7 @@ cnxk_nix_remove(struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (eth_dev) { /* Cleanup eth dev */ - rc = cnxk_eth_dev_uninit(eth_dev, true); + rc = cnxk_eth_dev_uninit(eth_dev, false); if (rc) return rc; From patchwork Sat Mar 6 15:33:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88636 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C2B01A0548; Sat, 6 Mar 2021 16:40:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7A4AD22A47A; Sat, 6 Mar 2021 16:36:20 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 63F9922A479 for ; Sat, 6 Mar 2021 16:36:18 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVumQ029368 for ; Sat, 6 Mar 2021 07:36:17 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=w+pzCli8lFrzXVRyamCsPHZp0neceNUiVfGTkN7x2s4=; b=IXXklNLJlM6OZQXrwQq/ItgELKiUXXR7/uA+COgknCOWLRGzsglreFczmQ3EbYckACh+ 3LiRQ1IwTfJUPbAUvoM3R9FFN8KK7T9V6UhLi2BhFAl5AZ6bEmS8H6y/QOzyW05MCWkx AzC8biUV7jE4KZxZjGqukKOKllvutSpvTUONPEJIPxkfgoGC0x2THcAjxx6RqcHC/xZu epVwAGBw1ohAc0O30PvrimbRk9S77yENumOIoGlYfD5xDXXlNlVDx893rzQdXOASDjLu bF9wEClDcLrUbE2SmlWKa4GXIIY8UoYd5swTyZE4HFpcondeuDkBt/5mmTNAFGwhqhV4 Ng== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurceq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:17 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:16 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:15 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:15 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 5DC263F703F; Sat, 6 Mar 2021 07:36:13 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:03:59 +0530 Message-ID: <20210306153404.10781-40-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 39/44] net/cnxk: add pending Tx mbuf cleanup operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sunil Kumar Kori Once mbufs are transmitted, mbufs are freed by H/W. No mbufs are accumalated as a pending mbuf. Hence operation is NOP for cnxk platform. Signed-off-by: Sunil Kumar Kori --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 10 ++++++++++ 3 files changed, 12 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 86dabad..5a2f90b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1164,6 +1164,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, + .tx_done_cleanup = cnxk_nix_tx_done_cleanup, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index eeb6a53..1ca52bc 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -251,6 +251,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); +int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 0bcba0c..ff8afac 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -652,3 +652,13 @@ cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); } + +/* It is a NOP for cnxk as HW frees the buffer on xmit */ +int +cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) +{ + RTE_SET_USED(txq); + RTE_SET_USED(free_cnt); + + return 0; +} From patchwork Sat Mar 6 15:34:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88637 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 327A5A0548; Sat, 6 Mar 2021 16:40:43 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A32B722A481; Sat, 6 Mar 2021 16:36:23 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9112F22A47E for ; Sat, 6 Mar 2021 16:36:21 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FU4UW005224 for ; Sat, 6 Mar 2021 07:36:21 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=wvtoWFtK95k1MHF1uScLslLA1oqKttquUr0hrlNRHa4=; b=KIiJmsw2sfAHFXhE75ydD1fXzF+It2lRjCt1ivBtmd9xEq2B1xU40nygMtR7oFQ8q8wg z7J0QZ+FteSWCb7RYTrUFAwDc7vCpl+5OTIPpd30q021sIYR7HUdJViy6OEC9xwd9KNq gfxoCMkSGnOJDaktqi7O3x0gbdCUcacqga+EVjAA50GAGtfauoVb8CNMYuRG184jN9IV I1pFHcKuSRUxTDGkMy9L17AaT0M9cyrAruvoSs0859FjCUqVBMLhBeHOAoEhXet48dgl S/udzN6ss9ctrD2KCO6j0Ro7U/S1RDIn9T2jEibJsKj0C3zrcR0kusEiS3StOkIu3ET0 Cw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 374a4w06cb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:21 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:18 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:18 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 698E03F704B; Sat, 6 Mar 2021 07:36:16 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:04:00 +0530 Message-ID: <20210306153404.10781-41-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 40/44] net/cnxk: add support to configure npc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Kiran Kumar K Adding support to configure NPC on device initialization. This involves reading the MKEX and initializing the necessary data. Signed-off-by: Kiran Kumar K --- drivers/net/cnxk/cnxk_ethdev.c | 25 ++++++++++++++++++++++--- drivers/net/cnxk/cnxk_ethdev.h | 3 +++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 3 +++ 3 files changed, 28 insertions(+), 3 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 5a2f90b..afe97f1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -8,7 +8,8 @@ nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) { uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; - if (roc_nix_is_vf_or_sdp(&dev->nix)) + if (roc_nix_is_vf_or_sdp(&dev->nix) || + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; return capa; @@ -120,6 +121,7 @@ nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ if (roc_model_is_cn96_Ax() && + dev->npc.switch_header_type != ROC_PRIV_FLAGS_HIGIG && (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { fc_cfg.mode = (fc_cfg.mode == RTE_FC_FULL || @@ -419,8 +421,10 @@ cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, dev->ethdev_rss_hf = ethdev_rss; - if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + if (ethdev_rss & ETH_RSS_L2_PAYLOAD && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_LEN_90B) { flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + } if (ethdev_rss & ETH_RSS_C_VLAN) flowkey_cfg |= FLOW_KEY_TYPE_VLAN; @@ -803,11 +807,18 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) roc_nix_err_intr_ena_dis(nix, true); roc_nix_ras_intr_ena_dis(nix, true); - if (nix->rx_ptp_ena) { + if (nix->rx_ptp_ena && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) { plt_err("Both PTP and switch header enabled"); goto free_nix_lf; } + rc = roc_nix_switch_hdr_set(nix, dev->npc.switch_header_type); + if (rc) { + plt_err("Failed to enable switch type nix_lf rc=%d", rc); + goto free_nix_lf; + } + rc = roc_nix_lso_fmt_setup(nix); if (rc) { plt_err("failed to setup nix lso format fields, rc=%d", rc); @@ -1259,6 +1270,11 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->speed_capa = nix_get_speed_capa(dev); /* Initialize roc npc */ + dev->npc.roc_nix = nix; + rc = roc_npc_init(&dev->npc); + if (rc) + goto free_mac_addrs; + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, eth_dev->data->port_id, roc_nix_get_pf(nix), @@ -1292,6 +1308,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable and free rte_flow entries */ + roc_npc_fini(&dev->npc); + /* Disable link status events */ roc_nix_mac_link_event_start_stop(nix, false); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 1ca52bc..e3b0bc1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -133,6 +133,9 @@ struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC NPC */ + struct roc_npc npc; + /* ROC RQs, SQs and CQs */ struct roc_nix_rq *rqs; struct roc_nix_sq *sqs; diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c index 4af2803..7fd06eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev_devargs.c +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -150,6 +150,9 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; dev->nix.max_sqb_count = sqb_count; dev->nix.reta_sz = reta_sz; + dev->npc.flow_prealloc_size = flow_prealloc_size; + dev->npc.flow_max_priority = flow_max_priority; + dev->npc.switch_header_type = switch_header_type; return 0; exit: From patchwork Sat Mar 6 15:34:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88638 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 98FEDA0548; Sat, 6 Mar 2021 16:40:52 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D139622A417; Sat, 6 Mar 2021 16:36:26 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id C294422A485 for ; Sat, 6 Mar 2021 16:36:24 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVt9Q029362 for ; Sat, 6 Mar 2021 07:36:24 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=EVdyY7SA+Z01FZenwvJRJ4Pvm8KoOHiEWKe/cH+mkz8=; b=Tkecpv01LfodsYbCLamEpEEdbmIl49u7MFi7dmJYVFZNGFCyOBFSmWeL61LxOf6kQpF3 gPctEawKL5dOfJ3bxsY+jVl5+bXCE8ax9hGDotHiABHbPUr3Aw21v3zKV10BL+hCMgqj /BTZYjqUH2fvGcT+6ao1ZHby0+IcGM17wMG+Ct23owLgOOCyIyY4nXITAXBfYk98JFq5 LRjrPIGvHyZXGS+099cNf668VI5zJDWPK1MEQT0rN0ndSo5hznuMtVV0DLnUEy8u4paA 9CQx+GSS4vBLFAd6d9ByCm1LkmABz/nr1fiI4133U86AJB86omMZlO1bm8sMRa6RW3fH KA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcew-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:23 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:22 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:21 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:21 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 648E23F7040; Sat, 6 Mar 2021 07:36:19 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:04:01 +0530 Message-ID: <20210306153404.10781-42-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 41/44] net/cnxk: add initial version of rte flow support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Kiran Kumar K Adding initial version of rte_flow support for cnxk family device. Supported rte_flow ops are flow_validate, flow_create, flow_crstroy, flow_flush, flow_query, flow_isolate. Signed-off-by: Kiran Kumar K --- doc/guides/nics/cnxk.rst | 118 ++++++++++++++ doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_rte_flow.c | 280 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_rte_flow.h | 69 +++++++++ drivers/net/cnxk/meson.build | 1 + 7 files changed, 471 insertions(+) create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index c2a6fbb..87401f0 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - MAC filtering +- Generic flow API - Inner and Outer Checksum offload - Port hardware statistics - Link state information @@ -222,3 +223,120 @@ Debugging Options +---+------------+-------------------------------------------------------+ | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' | +---+------------+-------------------------------------------------------+ + +RTE Flow Support +---------------- + +The OCTEON CN9K/CN10K SoC family NIC has support for the following patterns and +actions. + +Patterns: + +.. _table_cnxk_supported_flow_item_types: + +.. table:: Item types + + +----+--------------------------------+ + | # | Pattern Type | + +====+================================+ + | 1 | RTE_FLOW_ITEM_TYPE_ETH | + +----+--------------------------------+ + | 2 | RTE_FLOW_ITEM_TYPE_VLAN | + +----+--------------------------------+ + | 3 | RTE_FLOW_ITEM_TYPE_E_TAG | + +----+--------------------------------+ + | 4 | RTE_FLOW_ITEM_TYPE_IPV4 | + +----+--------------------------------+ + | 5 | RTE_FLOW_ITEM_TYPE_IPV6 | + +----+--------------------------------+ + | 6 | RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4| + +----+--------------------------------+ + | 7 | RTE_FLOW_ITEM_TYPE_MPLS | + +----+--------------------------------+ + | 8 | RTE_FLOW_ITEM_TYPE_ICMP | + +----+--------------------------------+ + | 9 | RTE_FLOW_ITEM_TYPE_UDP | + +----+--------------------------------+ + | 10 | RTE_FLOW_ITEM_TYPE_TCP | + +----+--------------------------------+ + | 11 | RTE_FLOW_ITEM_TYPE_SCTP | + +----+--------------------------------+ + | 12 | RTE_FLOW_ITEM_TYPE_ESP | + +----+--------------------------------+ + | 13 | RTE_FLOW_ITEM_TYPE_GRE | + +----+--------------------------------+ + | 14 | RTE_FLOW_ITEM_TYPE_NVGRE | + +----+--------------------------------+ + | 15 | RTE_FLOW_ITEM_TYPE_VXLAN | + +----+--------------------------------+ + | 16 | RTE_FLOW_ITEM_TYPE_GTPC | + +----+--------------------------------+ + | 17 | RTE_FLOW_ITEM_TYPE_GTPU | + +----+--------------------------------+ + | 18 | RTE_FLOW_ITEM_TYPE_GENEVE | + +----+--------------------------------+ + | 19 | RTE_FLOW_ITEM_TYPE_VXLAN_GPE | + +----+--------------------------------+ + | 20 | RTE_FLOW_ITEM_TYPE_IPV6_EXT | + +----+--------------------------------+ + | 21 | RTE_FLOW_ITEM_TYPE_VOID | + +----+--------------------------------+ + | 22 | RTE_FLOW_ITEM_TYPE_ANY | + +----+--------------------------------+ + | 23 | RTE_FLOW_ITEM_TYPE_GRE_KEY | + +----+--------------------------------+ + | 24 | RTE_FLOW_ITEM_TYPE_HIGIG2 | + +----+--------------------------------+ + +.. note:: + + ``RTE_FLOW_ITEM_TYPE_GRE_KEY`` works only when checksum and routing + bits in the GRE header are equal to 0. + +Actions: + +.. _table_cnxk_supported_ingress_action_types: + +.. table:: Ingress action types + + +----+-----------------------------------------+ + | # | Action Type | + +====+=========================================+ + | 1 | RTE_FLOW_ACTION_TYPE_VOID | + +----+-----------------------------------------+ + | 2 | RTE_FLOW_ACTION_TYPE_MARK | + +----+-----------------------------------------+ + | 3 | RTE_FLOW_ACTION_TYPE_FLAG | + +----+-----------------------------------------+ + | 4 | RTE_FLOW_ACTION_TYPE_COUNT | + +----+-----------------------------------------+ + | 5 | RTE_FLOW_ACTION_TYPE_DROP | + +----+-----------------------------------------+ + | 6 | RTE_FLOW_ACTION_TYPE_QUEUE | + +----+-----------------------------------------+ + | 7 | RTE_FLOW_ACTION_TYPE_RSS | + +----+-----------------------------------------+ + | 8 | RTE_FLOW_ACTION_TYPE_PF | + +----+-----------------------------------------+ + | 9 | RTE_FLOW_ACTION_TYPE_VF | + +----+-----------------------------------------+ + | 10 | RTE_FLOW_ACTION_TYPE_OF_POP_VLAN | + +----+-----------------------------------------+ + +.. _table_cnxk_supported_egress_action_types: + +.. table:: Egress action types + + +----+-----------------------------------------+ + | # | Action Type | + +====+=========================================+ + | 1 | RTE_FLOW_ACTION_TYPE_COUNT | + +----+-----------------------------------------+ + | 2 | RTE_FLOW_ACTION_TYPE_DROP | + +----+-----------------------------------------+ + | 3 | RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN | + +----+-----------------------------------------+ + | 4 | RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID | + +----+-----------------------------------------+ + | 5 | RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP | + +----+-----------------------------------------+ diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 192c15a..7b6d832 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -25,6 +25,7 @@ Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Flow control = Y +Flow API = Y Jumbo frame = Y Scattered Rx = Y L3 checksum offload = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e990480..ef37088 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -24,6 +24,7 @@ Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Flow control = Y +Flow API = Y Jumbo frame = Y L3 checksum offload = Y L4 checksum offload = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 3a4417c..69419d1 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -20,6 +20,7 @@ MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y +Flow API = Y Jumbo frame = Y Scattered Rx = Y L3 checksum offload = Y diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c new file mode 100644 index 0000000..344e01f --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -0,0 +1,280 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include + +static int +cnxk_map_actions(struct rte_eth_dev *dev, + const struct rte_flow_action actions[], + struct roc_npc_action in_actions[]) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + const struct rte_flow_action_count *act_count; + const struct rte_flow_action_queue *act_q; + int rq; + int i = 0; + + RTE_SET_USED(hw); + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_VOID: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VOID; + break; + + case RTE_FLOW_ACTION_TYPE_MARK: + in_actions[i].type = ROC_NPC_ACTION_TYPE_MARK; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_FLAG: + in_actions[i].type = ROC_NPC_ACTION_TYPE_FLAG; + break; + + case RTE_FLOW_ACTION_TYPE_COUNT: + act_count = (const struct rte_flow_action_count *) + actions->conf; + + if (act_count->shared == 1) { + plt_npc_dbg("Shared counter is not supported"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_COUNT; + break; + + case RTE_FLOW_ACTION_TYPE_DROP: + in_actions[i].type = ROC_NPC_ACTION_TYPE_DROP; + break; + + case RTE_FLOW_ACTION_TYPE_PF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_PF; + break; + + case RTE_FLOW_ACTION_TYPE_VF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VF; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_QUEUE: + act_q = (const struct rte_flow_action_queue *) + actions->conf; + rq = act_q->index; + if (rq >= dev->data->nb_rx_queues) { + plt_npc_dbg("Invalid queue index"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_QUEUE; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_RSS: + in_actions[i].type = ROC_NPC_ACTION_TYPE_RSS; + break; + + case RTE_FLOW_ACTION_TYPE_SECURITY: + in_actions[i].type = ROC_NPC_ACTION_TYPE_SEC; + break; + default: + plt_npc_dbg("Action is not supported = %d", + actions->type); + goto err_exit; + } + i++; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_END; + return 0; + +err_exit: + return -EINVAL; +} + +static int +cnxk_map_flow_data(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct roc_npc_attr *in_attr, + struct roc_npc_item_info in_pattern[], + struct roc_npc_action in_actions[]) +{ + int i = 0; + + in_attr->priority = attr->priority; + in_attr->ingress = attr->ingress; + in_attr->egress = attr->egress; + + while (pattern->type != RTE_FLOW_ITEM_TYPE_END) { + in_pattern[i].spec = pattern->spec; + in_pattern[i].last = pattern->last; + in_pattern[i].mask = pattern->mask; + in_pattern[i].type = term[pattern->type].item_type; + in_pattern[i].size = term[pattern->type].item_size; + pattern++; + i++; + } + in_pattern[i].type = ROC_NPC_ITEM_TYPE_END; + + return cnxk_map_actions(dev, actions, in_actions); +} + +static int +cnxk_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow flow; + int rc; + + rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return rc; + } + + return roc_npc_flow_parse(npc, &in_attr, in_pattern, in_actions, &flow); +} + +static struct rte_flow * +cnxk_flow_create(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct roc_npc *npc = &hw->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow *flow; + int errcode; + int rc; + + rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return NULL; + } + + flow = roc_npc_flow_create(npc, &in_attr, in_pattern, in_actions, + &errcode); + if (errcode != 0) { + rte_flow_error_set(error, errcode, errcode, NULL, + roc_error_msg_get(errcode)); + return NULL; + } + + return (struct rte_flow *)flow; +} + +static int +cnxk_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error) +{ + struct roc_npc_flow *in_flow = (struct roc_npc_flow *)flow; + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + int rc; + + rc = roc_npc_flow_destroy(npc, in_flow); + if (rc) + rte_flow_error_set(error, rc, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow Destroy failed"); + return rc; +} + +static int +cnxk_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + int rc; + + rc = roc_npc_mcam_free_all_resources(npc); + if (rc) { + rte_flow_error_set(error, EIO, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Failed to flush filter"); + return -rte_errno; + } + + return 0; +} + +static int +cnxk_flow_query(struct rte_eth_dev *dev, struct rte_flow *flow, + const struct rte_flow_action *action, void *data, + struct rte_flow_error *error) +{ + struct roc_npc_flow *in_flow = (struct roc_npc_flow *)flow; + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + struct rte_flow_query_count *query = data; + const char *errmsg = NULL; + int errcode = ENOTSUP; + int rc; + + if (action->type != RTE_FLOW_ACTION_TYPE_COUNT) { + errmsg = "Only COUNT is supported in query"; + goto err_exit; + } + + if (in_flow->ctr_id == NPC_COUNTER_NONE) { + errmsg = "Counter is not available"; + goto err_exit; + } + + rc = roc_npc_mcam_read_counter(npc, in_flow->ctr_id, &query->hits); + if (rc != 0) { + errcode = EIO; + errmsg = "Error reading flow counter"; + goto err_exit; + } + query->hits_set = 1; + query->bytes_set = 0; + + if (query->reset) + rc = roc_npc_mcam_clear_counter(npc, in_flow->ctr_id); + if (rc != 0) { + errcode = EIO; + errmsg = "Error clearing flow counter"; + goto err_exit; + } + + return 0; + +err_exit: + rte_flow_error_set(error, errcode, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, errmsg); + return -rte_errno; +} + +static int +cnxk_flow_isolate(struct rte_eth_dev *dev __rte_unused, int enable __rte_unused, + struct rte_flow_error *error) +{ + /* If we support, we need to un-install the default mcam + * entry for this port. + */ + + rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow isolation not supported"); + + return -rte_errno; +} + +const struct rte_flow_ops cnxk_flow_ops = { + .validate = cnxk_flow_validate, + .create = cnxk_flow_create, + .destroy = cnxk_flow_destroy, + .flush = cnxk_flow_flush, + .query = cnxk_flow_query, + .isolate = cnxk_flow_isolate, +}; diff --git a/drivers/net/cnxk/cnxk_rte_flow.h b/drivers/net/cnxk/cnxk_rte_flow.h new file mode 100644 index 0000000..3740226 --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_RTE_FLOW_H__ +#define __CNXK_RTE_FLOW_H__ + +#include +#include + +#include "cnxk_ethdev.h" +#include "roc_api.h" +#include "roc_npc_priv.h" + +struct cnxk_rte_flow_term_info { + uint16_t item_type; + uint16_t item_size; +}; + +struct cnxk_rte_flow_term_info term[] = { + [RTE_FLOW_ITEM_TYPE_ETH] = {ROC_NPC_ITEM_TYPE_ETH, + sizeof(struct rte_flow_item_eth)}, + [RTE_FLOW_ITEM_TYPE_VLAN] = {ROC_NPC_ITEM_TYPE_VLAN, + sizeof(struct rte_flow_item_vlan)}, + [RTE_FLOW_ITEM_TYPE_E_TAG] = {ROC_NPC_ITEM_TYPE_E_TAG, + sizeof(struct rte_flow_item_e_tag)}, + [RTE_FLOW_ITEM_TYPE_IPV4] = {ROC_NPC_ITEM_TYPE_IPV4, + sizeof(struct rte_flow_item_ipv4)}, + [RTE_FLOW_ITEM_TYPE_IPV6] = {ROC_NPC_ITEM_TYPE_IPV6, + sizeof(struct rte_flow_item_ipv6)}, + [RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4] = {ROC_NPC_ITEM_TYPE_ARP_ETH_IPV4, + sizeof(struct rte_flow_item_arp_eth_ipv4)}, + [RTE_FLOW_ITEM_TYPE_MPLS] = {ROC_NPC_ITEM_TYPE_MPLS, + sizeof(struct rte_flow_item_mpls)}, + [RTE_FLOW_ITEM_TYPE_ICMP] = {ROC_NPC_ITEM_TYPE_ICMP, + sizeof(struct rte_flow_item_icmp)}, + [RTE_FLOW_ITEM_TYPE_UDP] = {ROC_NPC_ITEM_TYPE_UDP, + sizeof(struct rte_flow_item_udp)}, + [RTE_FLOW_ITEM_TYPE_TCP] = {ROC_NPC_ITEM_TYPE_TCP, + sizeof(struct rte_flow_item_tcp)}, + [RTE_FLOW_ITEM_TYPE_SCTP] = {ROC_NPC_ITEM_TYPE_SCTP, + sizeof(struct rte_flow_item_sctp)}, + [RTE_FLOW_ITEM_TYPE_ESP] = {ROC_NPC_ITEM_TYPE_ESP, + sizeof(struct rte_flow_item_esp)}, + [RTE_FLOW_ITEM_TYPE_GRE] = {ROC_NPC_ITEM_TYPE_GRE, + sizeof(struct rte_flow_item_gre)}, + [RTE_FLOW_ITEM_TYPE_NVGRE] = {ROC_NPC_ITEM_TYPE_NVGRE, + sizeof(struct rte_flow_item_nvgre)}, + [RTE_FLOW_ITEM_TYPE_VXLAN] = {ROC_NPC_ITEM_TYPE_VXLAN, + sizeof(struct rte_flow_item_vxlan)}, + [RTE_FLOW_ITEM_TYPE_GTPC] = {ROC_NPC_ITEM_TYPE_GTPC, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GTPU] = {ROC_NPC_ITEM_TYPE_GTPU, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GENEVE] = {ROC_NPC_ITEM_TYPE_GENEVE, + sizeof(struct rte_flow_item_geneve)}, + [RTE_FLOW_ITEM_TYPE_VXLAN_GPE] = {ROC_NPC_ITEM_TYPE_VXLAN_GPE, + sizeof(struct rte_flow_item_vxlan_gpe)}, + [RTE_FLOW_ITEM_TYPE_IPV6_EXT] = {ROC_NPC_ITEM_TYPE_IPV6_EXT, + sizeof(struct rte_flow_item_ipv6_ext)}, + [RTE_FLOW_ITEM_TYPE_VOID] = {ROC_NPC_ITEM_TYPE_VOID, 0}, + [RTE_FLOW_ITEM_TYPE_ANY] = {ROC_NPC_ITEM_TYPE_ANY, 0}, + [RTE_FLOW_ITEM_TYPE_GRE_KEY] = {ROC_NPC_ITEM_TYPE_GRE_KEY, + sizeof(uint32_t)}, + [RTE_FLOW_ITEM_TYPE_HIGIG2] = { + ROC_NPC_ITEM_TYPE_HIGIG2, + sizeof(struct rte_flow_item_higig2_hdr)} +}; + +#endif /* __CNXK_RTE_FLOW_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 0e141d7..49aa016 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -13,6 +13,7 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', 'cnxk_lookup.c', + 'cnxk_rte_flow.c', 'cnxk_stats.c') # CN9K From patchwork Sat Mar 6 15:34:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88639 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2C805A0548; Sat, 6 Mar 2021 16:41:05 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 76C9122A492; Sat, 6 Mar 2021 16:36:29 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9187A22A486 for ; Sat, 6 Mar 2021 16:36:27 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FU4UX005224 for ; Sat, 6 Mar 2021 07:36:27 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=v6WMr9vq6YXOlgwHZIZAOF8p1lH6jvTLEHI2NXxcO50=; b=OF7K7yd+R7IfhtdjiwbJ6SUqEOb0xkKHfDGlR/unvpvCpbD/6tdE6tANufORjSY75x+A kvDEP8EM8hxszQV55n4O4mhDmSi38X+sQZ6P/tKBInxUGrzU/hjiYTLvZFjzCcRYUngH XwuLq/pdzLt/fuuB6jP75PHAz2wARH/IfQMJa38cB9/oQXhPocD/wGjNQtYB2L/lFvIw AQ9+J9qrpTZiaCl4638a3nnfGnk7JgESaUXWGWOy+w5j6bE3iAZ1KAfcAOdSkbYETGxj iADzD6IREietjLKgJ+N0kgPeOs+E5CJVniRVP6t+uHUdfa40AyC/OPVtQBi5LM/VJ+Ca uQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 374a4w06ce-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:27 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:24 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:24 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 5F7B83F703F; Sat, 6 Mar 2021 07:36:22 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:04:02 +0530 Message-ID: <20210306153404.10781-43-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 42/44] net/cnxk: add filter ctrl operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Satheesh Paul This patch adds filter_ctrl operation to enable rte_flow_ops. Signed-off-by: Satheesh Paul --- drivers/common/cnxk/roc_npc.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.c | 3 +++ drivers/net/cnxk/cnxk_ethdev.h | 6 +++++- drivers/net/cnxk/cnxk_ethdev_ops.c | 21 +++++++++++++++++++++ 4 files changed, 31 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index 0efe080..b862e23 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -645,6 +645,8 @@ roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, struct npc_flow_list *list; int rc; + npc->channel = roc_npc->channel; + flow = plt_zmalloc(sizeof(*flow), 0); if (flow == NULL) { *errcode = NPC_ERR_NO_MEM; diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index afe97f1..347428e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -773,6 +773,8 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto fail_configure; } + dev->npc.channel = roc_nix_get_base_chan(nix); + nb_rxq = data->nb_rx_queues; nb_txq = data->nb_tx_queues; rc = -ENOMEM; @@ -1176,6 +1178,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, + .filter_ctrl = cnxk_nix_filter_ctrl, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e3b0bc1..7cf7cf7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -218,6 +218,8 @@ cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) /* Common ethdev ops */ extern struct eth_dev_ops cnxk_eth_dev_ops; +extern const struct rte_flow_ops cnxk_flow_ops; + /* Ops */ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); @@ -255,7 +257,9 @@ int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); - +int cnxk_nix_filter_ctrl(struct rte_eth_dev *eth_dev, + enum rte_filter_type filter_type, + enum rte_filter_op filter_op, void *arg); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index ff8afac..00f1fe7 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -294,6 +294,27 @@ cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, } int +cnxk_nix_filter_ctrl(struct rte_eth_dev *eth_dev, + enum rte_filter_type filter_type, + enum rte_filter_op filter_op, void *arg) +{ + RTE_SET_USED(eth_dev); + + if (filter_type != RTE_ETH_FILTER_GENERIC) { + plt_err("Unsupported filter type %d", filter_type); + return -ENOTSUP; + } + + if (filter_op == RTE_ETH_FILTER_GET) { + *(const void **)arg = &cnxk_flow_ops; + return 0; + } + + plt_err("Invalid filter_op %d", filter_op); + return -EINVAL; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); From patchwork Sat Mar 6 15:34:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88640 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 637ECA0548; Sat, 6 Mar 2021 16:41:13 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A118F22A432; Sat, 6 Mar 2021 16:36:32 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 8D8BD22A432 for ; Sat, 6 Mar 2021 16:36:30 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVumT029368 for ; Sat, 6 Mar 2021 07:36:29 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=sMs3XtODVM4csIRQ733lAo9KIwkvbwwHIt/tmgbaReM=; b=NMH/aAWt+DprM8YMcSSrN4AGc253lsu/LDaHFQ/0iPNz1Sfq3Qwc3yv4bDac80q2Ci4a yUfFj5dw8u/PDxJE+Oh7HZPH1nx19XVCXVItlQGnZatBuboxF0oyFLs0kn7cWodQN253 lkOu3YBRXhFb1+06846dN3MoVcpMUKxS2qZOiszA0kL2ODxT6eUwt72mbXlc6js5tTzY ETNvMwfGLR+leQA0sr2nmOBK3Y/qdrSgpjFnDSaqHrVILQrIUkn6Ft0UzOm5s39ZbjDI 51PNaKhBOb9j6FMBgknsHHRm91JR3V94VyUU7QBCSt7XpW+NBI8M84d2wo4/YbWNb3hp Rw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcf6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:29 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:28 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:27 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:27 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 5F5183F7040; Sat, 6 Mar 2021 07:36:25 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:04:03 +0530 Message-ID: <20210306153404.10781-44-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 43/44] net/cnxk: add ethdev firmware version get X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Satha Rao Add callback to get ethdev firmware version. Signed-off-by: Satha Rao --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 6 files changed, 25 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 7b6d832..2c83bfb 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -36,6 +36,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index ef37088..c8ad253 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -34,6 +34,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 69419d1..4dbdfcb 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -31,6 +31,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 347428e..f006718 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1175,6 +1175,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .fw_version_get = cnxk_nix_fw_version_get, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 7cf7cf7..4b25593 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -298,6 +298,8 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +int cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size); void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo); void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 00f1fe7..bf89ede 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -644,6 +644,25 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } +int +cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const char *str = roc_npc_profile_name_get(&dev->npc); + uint32_t size = strlen(str) + 1; + + if (fw_size > size) + fw_size = size; + + strlcpy(fw_version, str, fw_size); + + if (fw_size < size) + return size; + + return 0; +} + void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo) From patchwork Sat Mar 6 15:34:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 88641 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 72B32A0548; Sat, 6 Mar 2021 16:41:21 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D713922A427; Sat, 6 Mar 2021 16:36:35 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 06D9C22A3B5 for ; Sat, 6 Mar 2021 16:36:33 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FU5V5005545 for ; Sat, 6 Mar 2021 07:36:33 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=8zLNRv5dljHpfIXQuxvl16932ECO0LuS+NjamLILYyo=; b=VFlivt0/qttlYoXn3W25/46SyiR5d6EbrwvXPzb6YVnwL8PS2t2bzS5Ag5CD3516AwsU U9HQB0JTLzlVpX582zS0feG0ZKlccCKmHr2IkWoQij2fcNZbFNuJBpNpxqG/MNqsmoiH dvaniz0M4xTMCuk0+OSnkj63bp/3ZcopmQ4WWYJm60S8kLIc4F/S8R8oPQbDPU76Kcr4 9HCxSgB+8LpDAutSbg6pwuk6zSiq7BSzQ5dNJ/CiyYpqe5n2CdfHN9HKiYRCO3/fsWYs a6mu/owDt85oZK0ZAL4gdZTSWGu7Wqn9UcjfZF27hU2nHr/5aX5RjAToATxMmRLCgJlb LQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 374a4w06cg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:36:33 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:31 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:36:31 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:36:30 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 67C543F703F; Sat, 6 Mar 2021 07:36:28 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Sat, 6 Mar 2021 21:04:04 +0530 Message-ID: <20210306153404.10781-45-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 44/44] net/cnxk: add get register operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Satha Rao With this patch implemented api to dump platform registers for debug purposes. Signed-off-by: Satha Rao --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 25 +++++++++++++++++++++++++ 7 files changed, 34 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 87401f0..98bcb51 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -32,6 +32,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Debug utilities - Context dump and error interrupt support - Support Rx interrupt Prerequisites diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 2c83bfb..d1c6f9d 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -38,6 +38,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index c8ad253..5f2478d 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -36,6 +36,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 4dbdfcb..3cbc369 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -33,6 +33,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f006718..99fb091 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1180,6 +1180,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, .filter_ctrl = cnxk_nix_filter_ctrl, + .get_reg = cnxk_nix_dev_get_reg, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 4b25593..74573f9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -313,6 +313,10 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Debug */ +int cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, + struct rte_dev_reg_info *regs); + /* Inlines */ static __rte_always_inline uint64_t cnxk_pktmbuf_detach(struct rte_mbuf *m) diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index bf89ede..41c6d37 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -702,3 +702,28 @@ cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) return 0; } + +int +cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint64_t *data = regs->data; + int rc = -ENOTSUP; + + if (data == NULL) { + rc = roc_nix_lf_get_reg_count(nix); + if (rc > 0) { + regs->length = rc; + regs->width = 8; + rc = 0; + } + return rc; + } + + if (!regs->length || + regs->length == (uint32_t)roc_nix_lf_get_reg_count(nix)) + return roc_nix_lf_reg_dump(nix, data); + + return rc; +}