From patchwork Sat Apr 3 14:17:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90520 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2AD8CA0548; Sat, 3 Apr 2021 16:18:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9FCCF40696; Sat, 3 Apr 2021 16:18:12 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id BBE4F4067B for ; Sat, 3 Apr 2021 16:18:10 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EIAn3004847 for ; Sat, 3 Apr 2021 07:18:10 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=ccNhPe3L+4NZbYSvmhmjkM3AR6F/mLObNHhvR0+MsYk=; b=XdMU9vjtkKQS01jWO2s+o9qzP6ZF0FRi0LDo6F0geDF1fMYRFKwocOiM8exTj3jZ8e6R f3lboTQP9m9ZHdGJsU9Tj0HCiQgCk7RyNgSldkrIg3atBjmmHBKQINZf47nI0/Y431KR OS6KuW4c0p7qiW319KnGnxIc3OeXCoy7OKtr1kLhDD4KnZRjOF8aKlvc2ZqsJ3p7Sfgh NKbl9WfoIh0UlhT5iLnX2ilZXcoRKuqI45Re0+I817kZROBua1eE+V3qC11BCH/BIU+m FV7zo2mtkAZaP41FXx8wCLUzt6axVxAIbXdMYv6XgdjJct4oHjUFTSMgKvQHX90jg/sz Ag== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37pqvt86du-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:10 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:08 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:08 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 6D2B03F703F; Sat, 3 Apr 2021 07:18:04 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 3 Apr 2021 19:47:41 +0530 Message-ID: <20210403141751.215926-1-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210305162149.2196166-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: JT52ts3sRwZ-MBUPKNrqpRRc5ZcGrw5d X-Proofpoint-ORIG-GUID: JT52ts3sRwZ-MBUPKNrqpRRc5ZcGrw5d X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 01/11] mempool/cnxk: add build infra and doc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the meson based build infrastructure for Marvell CNXK mempool driver along with stub implementations for mempool device probe. Also add Marvell CNXK mempool base documentation. Signed-off-by: Pavan Nikhilesh Signed-off-by: Jerin Jacob Signed-off-by: Nithin Dabilpuram Signed-off-by: Ashwin Sekhar T K --- MAINTAINERS | 6 +++ doc/guides/mempool/cnxk.rst | 55 ++++++++++++++++++++ doc/guides/mempool/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 ++ drivers/mempool/cnxk/cnxk_mempool.c | 78 +++++++++++++++++++++++++++++ drivers/mempool/cnxk/meson.build | 13 +++++ drivers/mempool/cnxk/version.map | 3 ++ drivers/mempool/meson.build | 3 +- 8 files changed, 161 insertions(+), 1 deletion(-) create mode 100644 doc/guides/mempool/cnxk.rst create mode 100644 drivers/mempool/cnxk/cnxk_mempool.c create mode 100644 drivers/mempool/cnxk/meson.build create mode 100644 drivers/mempool/cnxk/version.map diff --git a/MAINTAINERS b/MAINTAINERS index c837516d14..bae8b93030 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -501,6 +501,12 @@ M: Artem V. Andreev M: Andrew Rybchenko F: drivers/mempool/bucket/ +Marvell cnxk +M: Ashwin Sekhar T K +M: Pavan Nikhilesh +F: drivers/mempool/cnxk/ +F: doc/guides/mempool/cnxk.rst + Marvell OCTEON TX2 M: Jerin Jacob M: Nithin Dabilpuram diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst new file mode 100644 index 0000000000..e72a77c361 --- /dev/null +++ b/doc/guides/mempool/cnxk.rst @@ -0,0 +1,55 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2021 Marvell. + +CNXK NPA Mempool Driver +============================ + +The CNXK NPA PMD (**librte_mempool_cnxk**) provides mempool driver support for +the integrated mempool device found in **Marvell OCTEON CN9K/CN10K** SoC family. + +More information about CNXK SoC can be found at `Marvell Official Website +`_. + +Features +-------- + +CNXK NPA PMD supports: + +- Up to 128 NPA LFs +- 1M Pools per LF +- HW mempool manager +- Ethdev Rx buffer allocation in HW to save CPU cycles in the Rx path. +- Ethdev Tx buffer recycling in HW to save CPU cycles in the Tx path. + +Prerequisites and Compilation procedure +--------------------------------------- + + See :doc:`../platform/cnxk` for setup information. + +Pre-Installation Configuration +------------------------------ + + +Debugging Options +~~~~~~~~~~~~~~~~~ + +.. _table_cnxk_mempool_debug_options: + +.. table:: CNXK mempool debug options + + +---+------------+-------------------------------------------------------+ + | # | Component | EAL log command | + +===+============+=======================================================+ + | 1 | NPA | --log-level='pmd\.mempool.cnxk,8' | + +---+------------+-------------------------------------------------------+ + +Standalone mempool device +~~~~~~~~~~~~~~~~~~~~~~~~~ + + The ``usertools/dpdk-devbind.py`` script shall enumerate all the mempool + devices available in the system. In order to avoid, the end user to bind the + mempool device prior to use ethdev and/or eventdev device, the respective + driver configures an NPA LF and attach to the first probed ethdev or eventdev + device. In case, if end user need to run mempool as a standalone device + (without ethdev or eventdev), end user needs to bind a mempool device using + ``usertools/dpdk-devbind.py`` diff --git a/doc/guides/mempool/index.rst b/doc/guides/mempool/index.rst index a0e55467e6..ce53bc1ac7 100644 --- a/doc/guides/mempool/index.rst +++ b/doc/guides/mempool/index.rst @@ -11,6 +11,7 @@ application through the mempool API. :maxdepth: 2 :numbered: + cnxk octeontx octeontx2 ring diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst index 3b072877a1..9bbba65f2e 100644 --- a/doc/guides/platform/cnxk.rst +++ b/doc/guides/platform/cnxk.rst @@ -141,6 +141,9 @@ HW Offload Drivers This section lists dataplane H/W block(s) available in CNXK SoC. +#. **Mempool Driver** + See :doc:`../mempool/cnxk` for NPA mempool driver information. + Procedure to Setup Platform --------------------------- diff --git a/drivers/mempool/cnxk/cnxk_mempool.c b/drivers/mempool/cnxk/cnxk_mempool.c new file mode 100644 index 0000000000..947078c052 --- /dev/null +++ b/drivers/mempool/cnxk/cnxk_mempool.c @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "roc_api.h" + +static int +npa_remove(struct rte_pci_device *pci_dev) +{ + RTE_SET_USED(pci_dev); + + return 0; +} + +static int +npa_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + RTE_SET_USED(pci_drv); + RTE_SET_USED(pci_dev); + + return 0; +} + +static const struct rte_pci_id npa_pci_map[] = { + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_PF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KA, + }, + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_PF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KAS, + }, + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_VF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KA, + }, + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_VF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KAS, + }, + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver npa_pci = { + .id_table = npa_pci_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA, + .probe = npa_probe, + .remove = npa_remove, +}; + +RTE_PMD_REGISTER_PCI(mempool_cnxk, npa_pci); +RTE_PMD_REGISTER_PCI_TABLE(mempool_cnxk, npa_pci_map); +RTE_PMD_REGISTER_KMOD_DEP(mempool_cnxk, "vfio-pci"); diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build new file mode 100644 index 0000000000..0be0802373 --- /dev/null +++ b/drivers/mempool/cnxk/meson.build @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2021 Marvell. +# + +if not is_linux or not dpdk_conf.get('RTE_ARCH_64') + build = false + reason = 'only supported on 64-bit Linux' + subdir_done() +endif + +sources = files('cnxk_mempool.c') + +deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] diff --git a/drivers/mempool/cnxk/version.map b/drivers/mempool/cnxk/version.map new file mode 100644 index 0000000000..ee80c51721 --- /dev/null +++ b/drivers/mempool/cnxk/version.map @@ -0,0 +1,3 @@ +INTERNAL { + local: *; +}; diff --git a/drivers/mempool/meson.build b/drivers/mempool/meson.build index 4428813dae..a2814c1dfa 100644 --- a/drivers/mempool/meson.build +++ b/drivers/mempool/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -drivers = ['bucket', 'dpaa', 'dpaa2', 'octeontx', 'octeontx2', 'ring', 'stack'] +drivers = ['bucket', 'cnxk', 'dpaa', 'dpaa2', 'octeontx', 'octeontx2', 'ring', + 'stack'] std_deps = ['mempool'] From patchwork Sat Apr 3 14:17:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90521 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AFA04A0548; Sat, 3 Apr 2021 16:18:18 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C17C5140E10; Sat, 3 Apr 2021 16:18:15 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id EE057140E0D for ; Sat, 3 Apr 2021 16:18:13 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EFUeP002613 for ; Sat, 3 Apr 2021 07:18:13 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=1bsgS4Dw67ef090dZ1UOeuUAPzOoUny8P59x2ZxnnSM=; b=jO/k4mY8RlqZIOjFJJjyWy6xEF5e4TxdZ02HSygYXcFMtPhRnAfJdKhVl8EbffP+XTsw 6f62Vxc+k7+EJjJPenDKOoKHq8/c5UZHy/ge/APgHFE/BTabwlvp0bhTnpQ86kojkKzF ngEqLhohlOWQb9yLmxnQe9qDINJ+NV+GBRVU/Fx+PSBBynjrN8fcVT/pp+z7S4PHzogI ArEPGtFbgf0pDD9n8FskE5ZEo5VIcdChQsb1WtqXkzlW+NfqGkdz5f5mC8P4zYDmLA+C wIvhstjOGFkptIHHdyUTaw6hCc/LI82ilmiIzOYpBSxTztMRkm9SSR3cpSrhXuWQ5Y0d pQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 37pnqqrbxs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:13 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:11 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:11 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 9D3AC3F7040; Sat, 3 Apr 2021 07:18:08 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:42 +0530 Message-ID: <20210403141751.215926-2-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 7u8eU1O8oAJ3P9o8lMjnN2oJa6-lJWWA X-Proofpoint-GUID: 7u8eU1O8oAJ3P9o8lMjnN2oJa6-lJWWA X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 02/11] mempool/cnxk: add device probe/remove X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for CNXk mempool device probe and remove. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 23 +++++ drivers/mempool/cnxk/cnxk_mempool.c | 131 +++++++++++++++++++++++++++- 2 files changed, 150 insertions(+), 4 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index e72a77c361..907c19c841 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -30,6 +30,29 @@ Pre-Installation Configuration ------------------------------ +Runtime Config Options +~~~~~~~~~~~~~~~~~~~~~~ + +- ``Maximum number of mempools per application`` (default ``128``) + + The maximum number of mempools per application needs to be configured on + HW during mempool driver initialization. HW can support up to 1M mempools, + Since each mempool costs set of HW resources, the ``max_pools`` ``devargs`` + parameter is being introduced to configure the number of mempools required + for the application. + For example:: + + -a 0002:02:00.0,max_pools=512 + + With the above configuration, the driver will set up only 512 mempools for + the given application to save HW resources. + +.. note:: + + Since this configuration is per application, the end user needs to + provide ``max_pools`` parameter to the first PCIe device probed by the given + application. + Debugging Options ~~~~~~~~~~~~~~~~~ diff --git a/drivers/mempool/cnxk/cnxk_mempool.c b/drivers/mempool/cnxk/cnxk_mempool.c index 947078c052..703d15be42 100644 --- a/drivers/mempool/cnxk/cnxk_mempool.c +++ b/drivers/mempool/cnxk/cnxk_mempool.c @@ -15,21 +15,142 @@ #include "roc_api.h" +#define CNXK_NPA_DEV_NAME RTE_STR(cnxk_npa_dev_) +#define CNXK_NPA_DEV_NAME_LEN (sizeof(CNXK_NPA_DEV_NAME) + PCI_PRI_STR_SIZE) +#define CNXK_NPA_MAX_POOLS_PARAM "max_pools" + +static inline uint32_t +npa_aura_size_to_u32(uint8_t val) +{ + if (val == NPA_AURA_SZ_0) + return 128; + if (val >= NPA_AURA_SZ_MAX) + return BIT_ULL(20); + + return 1 << (val + 6); +} + static int -npa_remove(struct rte_pci_device *pci_dev) +parse_max_pools(const char *key, const char *value, void *extra_args) { - RTE_SET_USED(pci_dev); + RTE_SET_USED(key); + uint32_t val; + val = atoi(value); + if (val < npa_aura_size_to_u32(NPA_AURA_SZ_128)) + val = 128; + if (val > npa_aura_size_to_u32(NPA_AURA_SZ_1M)) + val = BIT_ULL(20); + + *(uint8_t *)extra_args = rte_log2_u32(val) - 6; return 0; } +static inline uint8_t +parse_aura_size(struct rte_devargs *devargs) +{ + uint8_t aura_sz = NPA_AURA_SZ_128; + struct rte_kvargs *kvlist; + + if (devargs == NULL) + goto exit; + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + goto exit; + + rte_kvargs_process(kvlist, CNXK_NPA_MAX_POOLS_PARAM, &parse_max_pools, + &aura_sz); + rte_kvargs_free(kvlist); +exit: + return aura_sz; +} + +static inline char * +npa_dev_to_name(struct rte_pci_device *pci_dev, char *name) +{ + snprintf(name, CNXK_NPA_DEV_NAME_LEN, CNXK_NPA_DEV_NAME PCI_PRI_FMT, + pci_dev->addr.domain, pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + + return name; +} + +static int +npa_init(struct rte_pci_device *pci_dev) +{ + char name[CNXK_NPA_DEV_NAME_LEN]; + const struct rte_memzone *mz; + struct roc_npa *dev; + int rc; + + rc = roc_plt_init(); + if (rc < 0) + goto error; + + rc = -ENOMEM; + mz = rte_memzone_reserve_aligned(npa_dev_to_name(pci_dev, name), + sizeof(*dev), SOCKET_ID_ANY, 0, + RTE_CACHE_LINE_SIZE); + if (mz == NULL) + goto error; + + dev = mz->addr; + dev->pci_dev = pci_dev; + + roc_idev_npa_maxpools_set(parse_aura_size(pci_dev->device.devargs)); + rc = roc_npa_dev_init(dev); + if (rc) + goto mz_free; + + return 0; + +mz_free: + rte_memzone_free(mz); +error: + plt_err("failed to initialize npa device rc=%d", rc); + return rc; +} + +static int +npa_fini(struct rte_pci_device *pci_dev) +{ + char name[CNXK_NPA_DEV_NAME_LEN]; + const struct rte_memzone *mz; + int rc; + + mz = rte_memzone_lookup(npa_dev_to_name(pci_dev, name)); + if (mz == NULL) + return -EINVAL; + + rc = roc_npa_dev_fini(mz->addr); + if (rc) { + if (rc != -EAGAIN) + plt_err("Failed to remove npa dev, rc=%d", rc); + return rc; + } + rte_memzone_free(mz); + + return 0; +} + +static int +npa_remove(struct rte_pci_device *pci_dev) +{ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + return npa_fini(pci_dev); +} + static int npa_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) { RTE_SET_USED(pci_drv); - RTE_SET_USED(pci_dev); - return 0; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + return npa_init(pci_dev); } static const struct rte_pci_id npa_pci_map[] = { @@ -76,3 +197,5 @@ static struct rte_pci_driver npa_pci = { RTE_PMD_REGISTER_PCI(mempool_cnxk, npa_pci); RTE_PMD_REGISTER_PCI_TABLE(mempool_cnxk, npa_pci_map); RTE_PMD_REGISTER_KMOD_DEP(mempool_cnxk, "vfio-pci"); +RTE_PMD_REGISTER_PARAM_STRING(mempool_cnxk, + CNXK_NPA_MAX_POOLS_PARAM "=<128-1048576>"); From patchwork Sat Apr 3 14:17:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90522 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D9A23A0548; Sat, 3 Apr 2021 16:18:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 215E4140E1E; Sat, 3 Apr 2021 16:18:19 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 34E67140E1D for ; Sat, 3 Apr 2021 16:18:18 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EHPxU004836 for ; Sat, 3 Apr 2021 07:18:17 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=6m2bw8SgR16WsjRk1Rmjs3hOVJ5bVx/JJBtsyFJiX5Y=; b=KNQgSA7pqXuQnZqNng3vGCwIvNqblilp+jSlwhhEMRzhGyF4RgEOiYBPDrP0jnbReWHk dtA95w4MtCKLYgJbc9NRiptnduirXPzykQvYhwrdrRkKnfxZo3LpOKycGqT+NPr4pT/M n1juVmk+s1t3GNCH1RLrRmJ0EM17nzId1HwEHVjtJZqnimOHPHsv2o+6n5H8/3RtqiU0 61U8RrpRcxwPp5Ij1J0/LHPIdRTDlOF9JQYtekPPJh08F95qPgpI/B4UKSFO2MtxZkpO DvtUhSoouvKB7XrpGmnGhV1Dcchzn+O1dlBQcJuwwvbliKADwoECCzNwBbAlzljnQQaG wQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 37pnqqrbxw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:17 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:15 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:15 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 768F53F703F; Sat, 3 Apr 2021 07:18:12 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:43 +0530 Message-ID: <20210403141751.215926-3-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: l8WKFfUDilMygYB9lPbQB6ct-8ZOkoZ0 X-Proofpoint-GUID: l8WKFfUDilMygYB9lPbQB6ct-8ZOkoZ0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 03/11] mempool/cnxk: add generic ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add generic CNXk mempool ops which will enqueue/dequeue from pool one element at a time. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cnxk_mempool.h | 26 ++++ drivers/mempool/cnxk/cnxk_mempool_ops.c | 171 ++++++++++++++++++++++++ drivers/mempool/cnxk/meson.build | 3 +- 3 files changed, 199 insertions(+), 1 deletion(-) create mode 100644 drivers/mempool/cnxk/cnxk_mempool.h create mode 100644 drivers/mempool/cnxk/cnxk_mempool_ops.c diff --git a/drivers/mempool/cnxk/cnxk_mempool.h b/drivers/mempool/cnxk/cnxk_mempool.h new file mode 100644 index 0000000000..099b7f6998 --- /dev/null +++ b/drivers/mempool/cnxk/cnxk_mempool.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef _CNXK_MEMPOOL_H_ +#define _CNXK_MEMPOOL_H_ + +#include + +unsigned int cnxk_mempool_get_count(const struct rte_mempool *mp); +ssize_t cnxk_mempool_calc_mem_size(const struct rte_mempool *mp, + uint32_t obj_num, uint32_t pg_shift, + size_t *min_chunk_size, size_t *align); +int cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, + void *vaddr, rte_iova_t iova, size_t len, + rte_mempool_populate_obj_cb_t *obj_cb, + void *obj_cb_arg); +int cnxk_mempool_alloc(struct rte_mempool *mp); +void cnxk_mempool_free(struct rte_mempool *mp); + +int __rte_hot cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, + unsigned int n); +int __rte_hot cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, + unsigned int n); + +#endif diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c new file mode 100644 index 0000000000..2ce1816c04 --- /dev/null +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -0,0 +1,171 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +#include "roc_api.h" +#include "cnxk_mempool.h" + +int __rte_hot +cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, unsigned int n) +{ + unsigned int index; + + /* Ensure mbuf init changes are written before the free pointers + * are enqueued to the stack. + */ + rte_io_wmb(); + for (index = 0; index < n; index++) + roc_npa_aura_op_free(mp->pool_id, 0, + (uint64_t)obj_table[index]); + + return 0; +} + +int __rte_hot +cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n) +{ + unsigned int index; + uint64_t obj; + + for (index = 0; index < n; index++, obj_table++) { + int retry = 4; + + /* Retry few times before failing */ + do { + obj = roc_npa_aura_op_alloc(mp->pool_id, 0); + } while (retry-- && (obj == 0)); + + if (obj == 0) { + cnxk_mempool_enq(mp, obj_table - index, index); + return -ENOENT; + } + *obj_table = (void *)obj; + } + + return 0; +} + +unsigned int +cnxk_mempool_get_count(const struct rte_mempool *mp) +{ + return (unsigned int)roc_npa_aura_op_available(mp->pool_id); +} + +ssize_t +cnxk_mempool_calc_mem_size(const struct rte_mempool *mp, uint32_t obj_num, + uint32_t pg_shift, size_t *min_chunk_size, + size_t *align) +{ + size_t total_elt_sz; + + /* Need space for one more obj on each chunk to fulfill + * alignment requirements. + */ + total_elt_sz = mp->header_size + mp->elt_size + mp->trailer_size; + return rte_mempool_op_calc_mem_size_helper( + mp, obj_num, pg_shift, total_elt_sz, min_chunk_size, align); +} + +int +cnxk_mempool_alloc(struct rte_mempool *mp) +{ + uint64_t aura_handle = 0; + struct npa_aura_s aura; + struct npa_pool_s pool; + uint32_t block_count; + size_t block_size; + int rc = -ERANGE; + + block_size = mp->elt_size + mp->header_size + mp->trailer_size; + block_count = mp->size; + if (mp->header_size % ROC_ALIGN != 0) { + plt_err("Header size should be multiple of %dB", ROC_ALIGN); + goto error; + } + + if (block_size % ROC_ALIGN != 0) { + plt_err("Block size should be multiple of %dB", ROC_ALIGN); + goto error; + } + + memset(&aura, 0, sizeof(struct npa_aura_s)); + memset(&pool, 0, sizeof(struct npa_pool_s)); + pool.nat_align = 1; + pool.buf_offset = mp->header_size / ROC_ALIGN; + + /* Use driver specific mp->pool_config to override aura config */ + if (mp->pool_config != NULL) + memcpy(&aura, mp->pool_config, sizeof(struct npa_aura_s)); + + rc = roc_npa_pool_create(&aura_handle, block_size, block_count, &aura, + &pool); + if (rc) { + plt_err("Failed to alloc pool or aura rc=%d", rc); + goto error; + } + + /* Store aura_handle for future queue operations */ + mp->pool_id = aura_handle; + plt_npa_dbg("block_sz=%lu block_count=%d aura_handle=0x%" PRIx64, + block_size, block_count, aura_handle); + + return 0; +error: + return rc; +} + +void +cnxk_mempool_free(struct rte_mempool *mp) +{ + int rc = 0; + + plt_npa_dbg("aura_handle=0x%" PRIx64, mp->pool_id); + rc = roc_npa_pool_destroy(mp->pool_id); + if (rc) + plt_err("Failed to free pool or aura rc=%d", rc); +} + +int +cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, + void *vaddr, rte_iova_t iova, size_t len, + rte_mempool_populate_obj_cb_t *obj_cb, void *obj_cb_arg) +{ + size_t total_elt_sz, off; + int num_elts; + + if (iova == RTE_BAD_IOVA) + return -EINVAL; + + total_elt_sz = mp->header_size + mp->elt_size + mp->trailer_size; + + /* Align object start address to a multiple of total_elt_sz */ + off = total_elt_sz - ((((uintptr_t)vaddr - 1) % total_elt_sz) + 1); + + if (len < off) + return -EINVAL; + + vaddr = (char *)vaddr + off; + iova += off; + len -= off; + num_elts = len / total_elt_sz; + + plt_npa_dbg("iova %" PRIx64 ", aligned iova %" PRIx64 "", iova - off, + iova); + plt_npa_dbg("length %" PRIu64 ", aligned length %" PRIu64 "", + (uint64_t)(len + off), (uint64_t)len); + plt_npa_dbg("element size %" PRIu64 "", (uint64_t)total_elt_sz); + plt_npa_dbg("requested objects %" PRIu64 ", possible objects %" PRIu64 + "", (uint64_t)max_objs, (uint64_t)num_elts); + + roc_npa_aura_op_range_set(mp->pool_id, iova, + iova + num_elts * total_elt_sz); + + if (roc_npa_pool_range_update_check(mp->pool_id) < 0) + return -EBUSY; + + return rte_mempool_op_populate_helper( + mp, RTE_MEMPOOL_POPULATE_F_ALIGN_OBJ, max_objs, vaddr, iova, + len, obj_cb, obj_cb_arg); +} diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build index 0be0802373..52244e728b 100644 --- a/drivers/mempool/cnxk/meson.build +++ b/drivers/mempool/cnxk/meson.build @@ -8,6 +8,7 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') subdir_done() endif -sources = files('cnxk_mempool.c') +sources = files('cnxk_mempool.c', + 'cnxk_mempool_ops.c') deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] From patchwork Sat Apr 3 14:17:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90523 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 40FC9A0548; Sat, 3 Apr 2021 16:18:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 53C53140E28; Sat, 3 Apr 2021 16:18:23 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id F2CBD4067B for ; Sat, 3 Apr 2021 16:18:21 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EILfc004873 for ; Sat, 3 Apr 2021 07:18:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=YDj3pmd2pY8RuBuOTPuwgHf+GNlnqYX2Zb6G6ftLwtw=; b=T4PGcf/8+Q/juJBAwmfJChVtbw6KXMPZmfj7+Ro5GhH/tB81i9T7KLR007Wo2YREdyK7 Mvml3OGsbf2/ntlRguky9ThwYbkisZirdxUB0oJbSp8GJrrzcFvJyHOxGZ3ncw8ZonSr SCdv74QtJCP5aSvBTNQkAuhsCdRuJRk4ymrZ8AV4s3fuhuQpfbZ8lTsoK9peYFGq6n9n qwZLfwq45kGqBTyOH1r3bO6wPM4gfy/AHNRjS0NOerwrZPkx65KBu8gNZlYoIMFWcdRt 9H+V+pGEI7eUG8wN26WoZMLMCsbOymgEu6QbSjYYCF5CczhQIFaQkI3JP7WRMWQgFfiN bA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37pqvt86e3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:21 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:19 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:19 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 500C13F704A; Sat, 3 Apr 2021 07:18:15 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:44 +0530 Message-ID: <20210403141751.215926-4-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: y-WLSTv-sKPHzMPUBb24saXylQVqISgF X-Proofpoint-ORIG-GUID: y-WLSTv-sKPHzMPUBb24saXylQVqISgF X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 04/11] mempool/cnxk: register lf init/fini callbacks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Register the CNXk mempool lf init/fini callbacks which will set the appropriate mempool ops to be used according to the platform. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cnxk_mempool_ops.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index 2ce1816c04..18c307288c 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -2,6 +2,7 @@ * Copyright(C) 2021 Marvell. */ +#include #include #include "roc_api.h" @@ -169,3 +170,23 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, mp, RTE_MEMPOOL_POPULATE_F_ALIGN_OBJ, max_objs, vaddr, iova, len, obj_cb, obj_cb_arg); } + +static int +cnxk_mempool_lf_init(void) +{ + if (roc_model_is_cn10k() || roc_model_is_cn9k()) + rte_mbuf_set_platform_mempool_ops("cnxk_mempool_ops"); + + return 0; +} + +static void +cnxk_mempool_lf_fini(void) +{ +} + +RTE_INIT(cnxk_mempool_ops_init) +{ + roc_npa_lf_init_cb_register(cnxk_mempool_lf_init); + roc_npa_lf_fini_cb_register(cnxk_mempool_lf_fini); +} From patchwork Sat Apr 3 14:17:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90524 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E9FAEA0548; Sat, 3 Apr 2021 16:18:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F40A8140E4C; Sat, 3 Apr 2021 16:18:26 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 85B78140E4C for ; Sat, 3 Apr 2021 16:18:25 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EFxgE002708 for ; Sat, 3 Apr 2021 07:18:24 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=vuVsJ73HZN0U24O4nw/oVDldW3t8VhUaNwGk9KKGzEU=; b=E6dEsUzXeDNlo7KnzBgn96JgSXp0/DptOPmXZnG9HoOtAF1j/UwtTTg621cixsggqi1n TkzDxS8SVAG9UjbwsWNzcIHsczWmXvJalJtJX0RGY6BwEYQJaRSVoqgpaoCq72hkfpc/ bacKXq+vsD0E+8A92QrUTKwaduoTpqFRYqkp7/WY3t99GFnCSTfceTK5eHykrlrkgApb N7cYGfkR8IgDzPsUfW1vr7kX+OQJa4wEf8INI3gpojY62kHs8hz3ofgY5i+ESVRJZlCf HUkuhM5S9/Yj2WjZUKpXZLJKEIR6ItN4twlGTZjVK3UIrvBzCDgX8dR7EWE3/7HAP7lP pw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 37pnqqrby5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:24 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:23 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:23 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 278C23F703F; Sat, 3 Apr 2021 07:18:19 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:45 +0530 Message-ID: <20210403141751.215926-5-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: ecOuZOD7kzfKgxIUU5QYyScHxv2lBLY9 X-Proofpoint-GUID: ecOuZOD7kzfKgxIUU5QYyScHxv2lBLY9 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 05/11] mempool/cnxk: add cn9k mempool ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Marvell CN9k mempool ops and implement CN9k mempool alloc which makes sure that the element size always occupy odd number of cachelines to ensure even distribution among of elements among L1D cache sets. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn9k_mempool_ops.c | 54 +++++++++++++++++++++++++ drivers/mempool/cnxk/cnxk_mempool_ops.c | 4 +- drivers/mempool/cnxk/meson.build | 3 +- 3 files changed, 59 insertions(+), 2 deletions(-) create mode 100644 drivers/mempool/cnxk/cn9k_mempool_ops.c diff --git a/drivers/mempool/cnxk/cn9k_mempool_ops.c b/drivers/mempool/cnxk/cn9k_mempool_ops.c new file mode 100644 index 0000000000..f5ac163af9 --- /dev/null +++ b/drivers/mempool/cnxk/cn9k_mempool_ops.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +#include "roc_api.h" +#include "cnxk_mempool.h" + +static int +cn9k_mempool_alloc(struct rte_mempool *mp) +{ + size_t block_size, padding; + + block_size = mp->elt_size + mp->header_size + mp->trailer_size; + /* Align header size to ROC_ALIGN */ + if (mp->header_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(mp->header_size, ROC_ALIGN) - + mp->header_size; + mp->header_size += padding; + block_size += padding; + } + + /* Align block size to ROC_ALIGN */ + if (block_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(block_size, ROC_ALIGN) - block_size; + mp->trailer_size += padding; + block_size += padding; + } + + /* + * Marvell CN9k has 8 sets, 41 ways L1D cache, VA<9:7> bits dictate the + * set selection. Add additional padding to ensure that the element size + * always occupies odd number of cachelines to ensure even distribution + * of elements among L1D cache sets. + */ + padding = ((block_size / ROC_ALIGN) % 2) ? 0 : ROC_ALIGN; + mp->trailer_size += padding; + + return cnxk_mempool_alloc(mp); +} + +static struct rte_mempool_ops cn9k_mempool_ops = { + .name = "cn9k_mempool_ops", + .alloc = cn9k_mempool_alloc, + .free = cnxk_mempool_free, + .enqueue = cnxk_mempool_enq, + .dequeue = cnxk_mempool_deq, + .get_count = cnxk_mempool_get_count, + .calc_mem_size = cnxk_mempool_calc_mem_size, + .populate = cnxk_mempool_populate, +}; + +MEMPOOL_REGISTER_OPS(cn9k_mempool_ops); diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index 18c307288c..45c45e9943 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -174,7 +174,9 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, static int cnxk_mempool_lf_init(void) { - if (roc_model_is_cn10k() || roc_model_is_cn9k()) + if (roc_model_is_cn9k()) + rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); + else if (roc_model_is_cn10k()) rte_mbuf_set_platform_mempool_ops("cnxk_mempool_ops"); return 0; diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build index 52244e728b..ff31893ff4 100644 --- a/drivers/mempool/cnxk/meson.build +++ b/drivers/mempool/cnxk/meson.build @@ -9,6 +9,7 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') endif sources = files('cnxk_mempool.c', - 'cnxk_mempool_ops.c') + 'cnxk_mempool_ops.c', + 'cn9k_mempool_ops.c') deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] From patchwork Sat Apr 3 14:17:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90525 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EBC25A0548; Sat, 3 Apr 2021 16:18:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 38BF4140E54; Sat, 3 Apr 2021 16:18:30 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 52904140E35 for ; Sat, 3 Apr 2021 16:18:29 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EHPxV004836 for ; Sat, 3 Apr 2021 07:18:28 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=mj8J/pQ7kUa3fsAvqWMQnyHPacc5py+6X7c4uteKjLQ=; b=ZCZPyeHSJ9wjVBDW0UK/r4+Lh5P0JJdq4IVcQqAYyOHXathg5Naz8Ff1cknNzAG9qpXT P49qo4ImwB16dNCfYKj0/c+JIZiwMFuTV0Myb5EcYwRWumNd3dda00jOKMDTa1+yadL+ JHjFHNPqdXRDvqEGYMYESLy8rZGU9jC853/qFK13k58CxmzLY3P/p+VqPC/+lcg6l2pw OFHuLop6EQc81CScrA7roYcwggL18g6eJFD3FKxdGqEwXMRR3wDFw7AXpR1EYZ4g8SyS VGpQmxgr/r4P+vb0BPR4AhM7MWJVs86XO8rhRtl9IPyB4hFEaTaP7wAr/Gu8Jk8VmOwf IQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 37pnqqrbya-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:28 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:27 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:27 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 0001D3F7040; Sat, 3 Apr 2021 07:18:23 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:46 +0530 Message-ID: <20210403141751.215926-6-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 7QFyBlfqNfjTQ7_JlAfgOGYXrrHPaImg X-Proofpoint-GUID: 7QFyBlfqNfjTQ7_JlAfgOGYXrrHPaImg X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 06/11] mempool/cnxk: add cn9k optimized mempool enqueue/dequeue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Marvell CN9k mempool enqueue/dequeue. Marvell CN9k supports burst dequeue which allows to dequeue up to 32 pointers using pipelined casp instructions. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 4 +++ drivers/mempool/cnxk/cn9k_mempool_ops.c | 39 +++++++++++++++++++++++-- 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index 907c19c841..f51532b101 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -21,6 +21,10 @@ CNXK NPA PMD supports: - Ethdev Rx buffer allocation in HW to save CPU cycles in the Rx path. - Ethdev Tx buffer recycling in HW to save CPU cycles in the Tx path. +CN9k NPA supports: + +- Burst alloc of up to 32 pointers. + Prerequisites and Compilation procedure --------------------------------------- diff --git a/drivers/mempool/cnxk/cn9k_mempool_ops.c b/drivers/mempool/cnxk/cn9k_mempool_ops.c index f5ac163af9..c0cdba640b 100644 --- a/drivers/mempool/cnxk/cn9k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn9k_mempool_ops.c @@ -7,6 +7,41 @@ #include "roc_api.h" #include "cnxk_mempool.h" +static int __rte_hot +cn9k_mempool_enq(struct rte_mempool *mp, void *const *obj_table, unsigned int n) +{ + /* Ensure mbuf init changes are written before the free pointers + * are enqueued to the stack. + */ + rte_io_wmb(); + roc_npa_aura_op_bulk_free(mp->pool_id, (const uint64_t *)obj_table, n, + 0); + + return 0; +} + +static inline int __rte_hot +cn9k_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n) +{ + unsigned int count; + + count = roc_npa_aura_op_bulk_alloc(mp->pool_id, (uint64_t *)obj_table, + n, 0, 1); + + if (unlikely(count != n)) { + /* If bulk alloc failed to allocate all pointers, try + * allocating remaining pointers with the default alloc + * with retry scheme. + */ + if (cnxk_mempool_deq(mp, &obj_table[count], n - count)) { + cn9k_mempool_enq(mp, obj_table, count); + return -ENOENT; + } + } + + return 0; +} + static int cn9k_mempool_alloc(struct rte_mempool *mp) { @@ -44,8 +79,8 @@ static struct rte_mempool_ops cn9k_mempool_ops = { .name = "cn9k_mempool_ops", .alloc = cn9k_mempool_alloc, .free = cnxk_mempool_free, - .enqueue = cnxk_mempool_enq, - .dequeue = cnxk_mempool_deq, + .enqueue = cn9k_mempool_enq, + .dequeue = cn9k_mempool_deq, .get_count = cnxk_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, .populate = cnxk_mempool_populate, From patchwork Sat Apr 3 14:17:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90526 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A6E32A0548; Sat, 3 Apr 2021 16:18:55 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 66442140E4E; Sat, 3 Apr 2021 16:18:34 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A91C4140E35 for ; Sat, 3 Apr 2021 16:18:33 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EIAn4004847 for ; Sat, 3 Apr 2021 07:18:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=QqNIxdn8SJf2PC7ogzo6A1necWZaLChTT+2t50nykqI=; b=kyryfDF4uTvlKct7l6FEMaCutKj4CO/rnQI3EYVn1GIuAHD+RJT3gmY/ws8+jWQtB4Hn QlV6Q9s8y5UE7FsGyzUw6B+baefeyq6elXZhc8D2uCI26x4R0FpQMLjzZ5RI3fVf6u5u 1GY9jX9eqNhzRyFcrx213rWWJUsQeSajAUxnK73bu885MvDc4/ee6x4brGH6PadHk8Rr tXx0hhH45Do6WYdcktW6F+MV//kAzmCBOhdTlYMwqkPtFCClKL2NVcluBp1bpsK+bvKw t906on3leEZq6yO/uiA6h3DxmDadEwdySxJOHPgh9nae87ClF+CqsaqzI8yFA1CGvyoX bg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37pqvt86eg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:33 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:31 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:31 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id CC1B33F7043; Sat, 3 Apr 2021 07:18:27 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:47 +0530 Message-ID: <20210403141751.215926-7-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: TPjHLI7Qzp8AnRaGyEa-Upz8T8gN6p-P X-Proofpoint-ORIG-GUID: TPjHLI7Qzp8AnRaGyEa-Upz8T8gN6p-P X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 07/11] mempool/cnxk: add cn10k mempool ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Marvell CN10k mempool ops and implement CN10k mempool alloc. CN10k has 64 bytes L1D cache line size. Hence the CN10k mempool alloc does not make the element size an odd multiple L1D cache line size as NPA requires the element sizes to be multiples of 128 bytes. Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 4 ++ drivers/mempool/cnxk/cn10k_mempool_ops.c | 52 ++++++++++++++++++++++++ drivers/mempool/cnxk/cnxk_mempool_ops.c | 2 +- drivers/mempool/cnxk/meson.build | 3 +- 4 files changed, 59 insertions(+), 2 deletions(-) create mode 100644 drivers/mempool/cnxk/cn10k_mempool_ops.c diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index f51532b101..783368e690 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -80,3 +80,7 @@ Standalone mempool device device. In case, if end user need to run mempool as a standalone device (without ethdev or eventdev), end user needs to bind a mempool device using ``usertools/dpdk-devbind.py`` + + Example command to run ``mempool_autotest`` test with standalone CN10K NPA device:: + + echo "mempool_autotest" | /app/test/dpdk-test -c 0xf0 --mbuf-pool-ops-name="cn10k_mempool_ops" diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c new file mode 100644 index 0000000000..9b63789006 --- /dev/null +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +#include "roc_api.h" +#include "cnxk_mempool.h" + +static int +cn10k_mempool_alloc(struct rte_mempool *mp) +{ + uint32_t block_size; + size_t padding; + + block_size = mp->elt_size + mp->header_size + mp->trailer_size; + /* Align header size to ROC_ALIGN */ + if (mp->header_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(mp->header_size, ROC_ALIGN) - + mp->header_size; + mp->header_size += padding; + block_size += padding; + } + + /* Align block size to ROC_ALIGN */ + if (block_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(block_size, ROC_ALIGN) - block_size; + mp->trailer_size += padding; + block_size += padding; + } + + return cnxk_mempool_alloc(mp); +} + +static void +cn10k_mempool_free(struct rte_mempool *mp) +{ + cnxk_mempool_free(mp); +} + +static struct rte_mempool_ops cn10k_mempool_ops = { + .name = "cn10k_mempool_ops", + .alloc = cn10k_mempool_alloc, + .free = cn10k_mempool_free, + .enqueue = cnxk_mempool_enq, + .dequeue = cnxk_mempool_deq, + .get_count = cnxk_mempool_get_count, + .calc_mem_size = cnxk_mempool_calc_mem_size, + .populate = cnxk_mempool_populate, +}; + +MEMPOOL_REGISTER_OPS(cn10k_mempool_ops); diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index 45c45e9943..0ec131a475 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -177,7 +177,7 @@ cnxk_mempool_lf_init(void) if (roc_model_is_cn9k()) rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); else if (roc_model_is_cn10k()) - rte_mbuf_set_platform_mempool_ops("cnxk_mempool_ops"); + rte_mbuf_set_platform_mempool_ops("cn10k_mempool_ops"); return 0; } diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build index ff31893ff4..3282b5e5a6 100644 --- a/drivers/mempool/cnxk/meson.build +++ b/drivers/mempool/cnxk/meson.build @@ -10,6 +10,7 @@ endif sources = files('cnxk_mempool.c', 'cnxk_mempool_ops.c', - 'cn9k_mempool_ops.c') + 'cn9k_mempool_ops.c', + 'cn10k_mempool_ops.c') deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] From patchwork Sat Apr 3 14:17:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90527 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8A01FA0548; Sat, 3 Apr 2021 16:19:01 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9ED80140E80; Sat, 3 Apr 2021 16:18:38 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 09539140E6C for ; Sat, 3 Apr 2021 16:18:36 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EFxgF002708 for ; Sat, 3 Apr 2021 07:18:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Ws3TSXZQhJEiVZwqVMCZMxrBI+w3V7U8T7SDRtSfEKo=; b=FwQtVPXYEHesaJ7WiXzwplqfr0jP4tYeTnnUbhzxry5Ab9OFd5Nl1143klO0WySCR1Va UQOdWSG9XejpcWW6l1Dmaobby+iInBIIP2C3KFvbgIilBjfbJBxs2aEK0PdhVqvnYk82 0ExszsOf5IVax0EdvPBXPH5itlGvmqaKghQLNbWAsCd4SNIHaQ9dirNCHPtZWz7zxWeG qTuI/0AK9/X58MAtak/3rYN16aiC4kGof1sCIn8dG9DGk8uBp7r0xNbZ9nL7R/z6NCh6 h/T2N4CpJgIHzI5rU0jQUQj6Cz7HohlZX9SYeb4hRIMCOXSlWTMBxxrMgnYnQtfmplVr jw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 37pnqqrbyq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:36 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:34 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:34 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id B9CAD3F703F; Sat, 3 Apr 2021 07:18:31 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:48 +0530 Message-ID: <20210403141751.215926-8-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 8MZ_tHVPafdpMrYjfpTVHimt6WHAAK6o X-Proofpoint-GUID: 8MZ_tHVPafdpMrYjfpTVHimt6WHAAK6o X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 08/11] mempool/cnxk: add batch op init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Marvell CN10k mempool supports batch enqueue/dequeue which can dequeue up to 512 pointers and enqueue up to 15 pointers using a single instruction. These batch operations require a DMA memory to enqueue/dequeue pointers. This patch adds the initialization of this DMA memory. Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 5 + drivers/mempool/cnxk/cn10k_mempool_ops.c | 122 ++++++++++++++++++++++- drivers/mempool/cnxk/cnxk_mempool.h | 3 + drivers/mempool/cnxk/cnxk_mempool_ops.c | 13 ++- 4 files changed, 138 insertions(+), 5 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index 783368e690..286ee29003 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -25,6 +25,11 @@ CN9k NPA supports: - Burst alloc of up to 32 pointers. +CN10k NPA supports: + +- Batch dequeue of up to 512 pointers with single instruction. +- Batch enqueue of up to 15 pointers with single instruction. + Prerequisites and Compilation procedure --------------------------------------- diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index 9b63789006..d34041528a 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -7,11 +7,117 @@ #include "roc_api.h" #include "cnxk_mempool.h" +#define BATCH_ALLOC_SZ ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS + +enum batch_op_status { + BATCH_ALLOC_OP_NOT_ISSUED = 0, + BATCH_ALLOC_OP_ISSUED = 1, + BATCH_ALLOC_OP_DONE +}; + +struct batch_op_mem { + unsigned int sz; + enum batch_op_status status; + uint64_t objs[BATCH_ALLOC_SZ] __rte_aligned(ROC_ALIGN); +}; + +struct batch_op_data { + uint64_t lmt_addr; + struct batch_op_mem mem[RTE_MAX_LCORE] __rte_aligned(ROC_ALIGN); +}; + +static struct batch_op_data **batch_op_data; + +#define BATCH_OP_DATA_GET(pool_id) \ + batch_op_data[roc_npa_aura_handle_to_aura(pool_id)] + +#define BATCH_OP_DATA_SET(pool_id, op_data) \ + do { \ + uint64_t aura = roc_npa_aura_handle_to_aura(pool_id); \ + batch_op_data[aura] = op_data; \ + } while (0) + +int +cn10k_mempool_lf_init(void) +{ + unsigned int maxpools, sz; + + maxpools = roc_idev_npa_maxpools_get(); + sz = maxpools * sizeof(struct batch_op_data *); + + batch_op_data = rte_zmalloc(NULL, sz, ROC_ALIGN); + if (!batch_op_data) + return -1; + + return 0; +} + +void +cn10k_mempool_lf_fini(void) +{ + if (!batch_op_data) + return; + + rte_free(batch_op_data); + batch_op_data = NULL; +} + +static int +batch_op_init(struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + int i; + + RTE_ASSERT(BATCH_OP_DATA_GET(mp->pool_id) == NULL); + op_data = rte_zmalloc(NULL, sizeof(struct batch_op_data), ROC_ALIGN); + if (op_data == NULL) + return -1; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + op_data->mem[i].sz = 0; + op_data->mem[i].status = BATCH_ALLOC_OP_NOT_ISSUED; + } + + op_data->lmt_addr = roc_idev_lmt_base_addr_get(); + BATCH_OP_DATA_SET(mp->pool_id, op_data); + + return 0; +} + +static void +batch_op_fini(struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + int i; + + op_data = BATCH_OP_DATA_GET(mp->pool_id); + + rte_wmb(); + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct batch_op_mem *mem = &op_data->mem[i]; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) { + mem->sz = roc_npa_aura_batch_alloc_extract( + mem->objs, mem->objs, BATCH_ALLOC_SZ); + mem->status = BATCH_ALLOC_OP_DONE; + } + if (mem->status == BATCH_ALLOC_OP_DONE) { + roc_npa_aura_op_bulk_free(mp->pool_id, mem->objs, + mem->sz, 1); + mem->status = BATCH_ALLOC_OP_NOT_ISSUED; + } + } + + rte_free(op_data); + BATCH_OP_DATA_SET(mp->pool_id, NULL); +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { uint32_t block_size; size_t padding; + int rc; block_size = mp->elt_size + mp->header_size + mp->trailer_size; /* Align header size to ROC_ALIGN */ @@ -29,12 +135,26 @@ cn10k_mempool_alloc(struct rte_mempool *mp) block_size += padding; } - return cnxk_mempool_alloc(mp); + rc = cnxk_mempool_alloc(mp); + if (rc) + return rc; + + rc = batch_op_init(mp); + if (rc) { + plt_err("Failed to init batch alloc mem rc=%d", rc); + goto error; + } + + return 0; +error: + cnxk_mempool_free(mp); + return rc; } static void cn10k_mempool_free(struct rte_mempool *mp) { + batch_op_fini(mp); cnxk_mempool_free(mp); } diff --git a/drivers/mempool/cnxk/cnxk_mempool.h b/drivers/mempool/cnxk/cnxk_mempool.h index 099b7f6998..6e54346e6a 100644 --- a/drivers/mempool/cnxk/cnxk_mempool.h +++ b/drivers/mempool/cnxk/cnxk_mempool.h @@ -23,4 +23,7 @@ int __rte_hot cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, int __rte_hot cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n); +int cn10k_mempool_lf_init(void); +void cn10k_mempool_lf_fini(void); + #endif diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index 0ec131a475..389c3622fd 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -174,17 +174,22 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, static int cnxk_mempool_lf_init(void) { - if (roc_model_is_cn9k()) + int rc = 0; + + if (roc_model_is_cn9k()) { rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); - else if (roc_model_is_cn10k()) + } else if (roc_model_is_cn10k()) { rte_mbuf_set_platform_mempool_ops("cn10k_mempool_ops"); - - return 0; + rc = cn10k_mempool_lf_init(); + } + return rc; } static void cnxk_mempool_lf_fini(void) { + if (roc_model_is_cn10k()) + cn10k_mempool_lf_fini(); } RTE_INIT(cnxk_mempool_ops_init) From patchwork Sat Apr 3 14:17:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90528 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 178E1A0548; Sat, 3 Apr 2021 16:19:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F342F140E1B; Sat, 3 Apr 2021 16:18:42 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 4F3A9140E6C for ; Sat, 3 Apr 2021 16:18:41 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EHL4f003697 for ; Sat, 3 Apr 2021 07:18:40 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=sgMBQhfl7FE1PTIvm1+6MzzQfeIN1WWnJsnQNX1UOPQ=; b=ZzMFwVxuPQYpkw6F97SVVkTZZw/HpFP2eIBIpR69ByMP1NYf3rn0sGottUoQX0nZz3ml Z+mP4IyUacJT8rVrxAiO/u74q0TGrQy2dmFXuVH48mGhRD/2WOTZxuJ6dQlKEtKfhGnX 8knc5Z4hrMJsyp1e4n3oVHLHZkyHARbSH3nnrchF0sh4ZxHcWO5jiHIPR5PxQJ4pPZv7 QbbafNOlR5+pEADZtgxlvKXuPvBlUGkYX7F5O+Bje2C/XfTgNJKAMYyoX9ojGY5K7j+1 qiDLWwQFOaxJMv8dSXb+oKQegNCZW46aohXWnYwawwk1Xq1N3D80yLoZZIdZLJpawxYa AQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37pqvt86ep-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:40 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:38 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:38 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 927F53F7040; Sat, 3 Apr 2021 07:18:35 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:49 +0530 Message-ID: <20210403141751.215926-9-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: QyjyHeJT_LigojxV-MFD4Z41QEXKeuLW X-Proofpoint-ORIG-GUID: QyjyHeJT_LigojxV-MFD4Z41QEXKeuLW X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 09/11] mempool/cnxk: add cn10k batch enqueue op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for Marvell CN10k mempool batch enqueue op. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn10k_mempool_ops.c | 28 +++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index d34041528a..2e3ec414da 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -112,6 +112,32 @@ batch_op_fini(struct rte_mempool *mp) BATCH_OP_DATA_SET(mp->pool_id, NULL); } +static int __rte_hot +cn10k_mempool_enq(struct rte_mempool *mp, void *const *obj_table, + unsigned int n) +{ + const uint64_t *ptr = (const uint64_t *)obj_table; + uint64_t lmt_addr = 0, lmt_id = 0; + struct batch_op_data *op_data; + + /* Ensure mbuf init changes are written before the free pointers are + * enqueued to the stack. + */ + rte_io_wmb(); + + if (n == 1) { + roc_npa_aura_op_free(mp->pool_id, 1, ptr[0]); + return 0; + } + + op_data = BATCH_OP_DATA_GET(mp->pool_id); + lmt_addr = op_data->lmt_addr; + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + roc_npa_aura_op_batch_free(mp->pool_id, ptr, n, 1, lmt_addr, lmt_id); + + return 0; +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { @@ -162,7 +188,7 @@ static struct rte_mempool_ops cn10k_mempool_ops = { .name = "cn10k_mempool_ops", .alloc = cn10k_mempool_alloc, .free = cn10k_mempool_free, - .enqueue = cnxk_mempool_enq, + .enqueue = cn10k_mempool_enq, .dequeue = cnxk_mempool_deq, .get_count = cnxk_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, From patchwork Sat Apr 3 14:17:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90529 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 30152A0548; Sat, 3 Apr 2021 16:19:16 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9DF1C140E98; Sat, 3 Apr 2021 16:18:45 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id B6528140EA3 for ; Sat, 3 Apr 2021 16:18:44 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EGEDU002778 for ; Sat, 3 Apr 2021 07:18:44 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=qzHnot9u4nzmOx50W2Q98Gr0jg5IIdX+SecRUnIUDlg=; b=JJkTiQMSfQARutzXndhWear/WXf//7iiSPOObODoh8AU3jwtq8y1sNmWCCf65nUWOWK7 aq3NQRGU/9IHlDkKNGifezRaki0aqPAt9//s+Sk7f51P57LV7AcppKxtCTQdNUPyv9ZR jwuYlWCqjN7WIpuA5V59NbtuWriK9bwNbkx8zpAca7OqUYBQghFdd8IkVDVbKJLTqgvq w4/P3YLJkniGafwwDJXdVCoXiKsW5/f+mOU+QD0igCBJBF90CNGYqnCKCcT+J8/bUNQR RtOunXGuw7PxHWyIw8E3BUpPJyKYAUDv/X8MrbdqvsOOZqph8fazFZCGPl7OsaiVXoOt pw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 37pnqqrc05-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:43 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:42 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:42 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 6A7183F703F; Sat, 3 Apr 2021 07:18:39 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:50 +0530 Message-ID: <20210403141751.215926-10-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: tbaNEdzIYI0T9NAk1m81S6k7ywS-mHPn X-Proofpoint-GUID: tbaNEdzIYI0T9NAk1m81S6k7ywS-mHPn X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 10/11] mempool/cnxk: add cn10k get count op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for Marvell CN10k get count op. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn10k_mempool_ops.c | 28 +++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index 2e3ec414da..16b2f6697f 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -138,6 +138,32 @@ cn10k_mempool_enq(struct rte_mempool *mp, void *const *obj_table, return 0; } +static unsigned int +cn10k_mempool_get_count(const struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + unsigned int count = 0; + int i; + + op_data = BATCH_OP_DATA_GET(mp->pool_id); + + rte_wmb(); + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct batch_op_mem *mem = &op_data->mem[i]; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) + count += roc_npa_aura_batch_alloc_count(mem->objs, + BATCH_ALLOC_SZ); + + if (mem->status == BATCH_ALLOC_OP_DONE) + count += mem->sz; + } + + count += cnxk_mempool_get_count(mp); + + return count; +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { @@ -190,7 +216,7 @@ static struct rte_mempool_ops cn10k_mempool_ops = { .free = cn10k_mempool_free, .enqueue = cn10k_mempool_enq, .dequeue = cnxk_mempool_deq, - .get_count = cnxk_mempool_get_count, + .get_count = cn10k_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, .populate = cnxk_mempool_populate, }; From patchwork Sat Apr 3 14:17:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90530 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A6C18A0548; Sat, 3 Apr 2021 16:19:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E4E22140E13; Sat, 3 Apr 2021 16:18:50 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 6CFE7140E13 for ; Sat, 3 Apr 2021 16:18:49 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 133EGjUU001351 for ; Sat, 3 Apr 2021 07:18:48 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=KLzdLHUU8xJiDZDLU+CBA7L3xjR6FgTuRLpxvNqkk8w=; b=jxZudAoqzfpidQ+6KXQuXIkIu7ymhtiumSQt4jGSjjqhXSGlhw+kQFnzkbWmQ22f2Efa atWWzv2LwlPQSrwr4JmdRSn9yInkVTzFBomPAcGssHIrRm7ExXpPkAgaTEMfIDaxa5VZ Lb+mFalXhcImrQYPXl8TphfFpnaN19VbdJuqzQsW7muhsuX/Y+rRq5MuKkcWfAP491w8 C6A8WrEdOxM1l4A5RnMRKxZqDDn8OSX0zU1daQ6aZ1WtwSEy4GnUZ2xlhxziD+r4IJqS 2gtWCmc862WJMz2UF+y7BmMLW42Rw/8Nw2N/kfyrtRvhOROiXgs3DBEbTvOmaWt8IxtM fA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37pqvt86f1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 03 Apr 2021 07:18:48 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 3 Apr 2021 07:18:46 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 3 Apr 2021 07:18:46 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 41E883F7040; Sat, 3 Apr 2021 07:18:42 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Sat, 3 Apr 2021 19:47:51 +0530 Message-ID: <20210403141751.215926-11-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210403141751.215926-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210403141751.215926-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: 1Im1UECAxSRIWiIBYltuC6O7I63MPXGT X-Proofpoint-ORIG-GUID: 1Im1UECAxSRIWiIBYltuC6O7I63MPXGT X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-03_05:2021-04-01, 2021-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v2 11/11] mempool/cnxk: add cn10k batch dequeue op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for Marvell CN10k mempool batch dequeue op. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn10k_mempool_ops.c | 72 +++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index 16b2f6697f..05f36ff263 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -164,6 +164,76 @@ cn10k_mempool_get_count(const struct rte_mempool *mp) return count; } +static int __rte_hot +cn10k_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n) +{ + struct batch_op_data *op_data; + struct batch_op_mem *mem; + unsigned int count = 0; + int tid, rc, retry; + bool loop = true; + + op_data = BATCH_OP_DATA_GET(mp->pool_id); + tid = rte_lcore_id(); + mem = &op_data->mem[tid]; + + /* Issue batch alloc */ + if (mem->status == BATCH_ALLOC_OP_NOT_ISSUED) { + rc = roc_npa_aura_batch_alloc_issue(mp->pool_id, mem->objs, + BATCH_ALLOC_SZ, 0, 1); + /* If issue fails, try falling back to default alloc */ + if (unlikely(rc)) + return cn10k_mempool_enq(mp, obj_table, n); + mem->status = BATCH_ALLOC_OP_ISSUED; + } + + retry = 4; + while (loop) { + unsigned int cur_sz; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) { + mem->sz = roc_npa_aura_batch_alloc_extract( + mem->objs, mem->objs, BATCH_ALLOC_SZ); + + /* If partial alloc reduce the retry count */ + retry -= (mem->sz != BATCH_ALLOC_SZ); + /* Break the loop if retry count exhausted */ + loop = !!retry; + mem->status = BATCH_ALLOC_OP_DONE; + } + + cur_sz = n - count; + if (cur_sz > mem->sz) + cur_sz = mem->sz; + + /* Dequeue the pointers */ + memcpy(&obj_table[count], &mem->objs[mem->sz - cur_sz], + cur_sz * sizeof(uintptr_t)); + mem->sz -= cur_sz; + count += cur_sz; + + /* Break loop if the required pointers has been dequeued */ + loop &= (count != n); + + /* Issue next batch alloc if pointers are exhausted */ + if (mem->sz == 0) { + rc = roc_npa_aura_batch_alloc_issue( + mp->pool_id, mem->objs, BATCH_ALLOC_SZ, 0, 1); + /* Break loop if issue failed and set status */ + loop &= !rc; + mem->status = !rc; + } + } + + if (unlikely(count != n)) { + /* No partial alloc allowed. Free up allocated pointers */ + cn10k_mempool_enq(mp, obj_table, count); + return -ENOENT; + } + + return 0; +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { @@ -215,7 +285,7 @@ static struct rte_mempool_ops cn10k_mempool_ops = { .alloc = cn10k_mempool_alloc, .free = cn10k_mempool_free, .enqueue = cn10k_mempool_enq, - .dequeue = cnxk_mempool_deq, + .dequeue = cn10k_mempool_deq, .get_count = cn10k_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, .populate = cnxk_mempool_populate,