From patchwork Tue Apr 6 15:11:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90731 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9789EA0546; Tue, 6 Apr 2021 17:11:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 816B41411D2; Tue, 6 Apr 2021 17:11:35 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id C3BB31411CC for ; Tue, 6 Apr 2021 17:11:33 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0UQm015128 for ; Tue, 6 Apr 2021 08:11:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=EYZhf57tKQj30Ckri4vtgcQzWPbSt7KUrcgdNUOBiYk=; b=VNrc5bGp4vahGhUTxlMXk7v5iUCd26ECBr0OwVZn0Z5vyl5EKy78fyFSxn1Zu1mFrO0r 2EkrzhDoc2rEue9O86M6e9Q5h2z/jaNU0VXhrTRxIoiExV6fxSQNrKuf6EIKsLDjsGgP 5B2k99dKUoBaT908lXjxRHz+CPm8kna/XfZGqUb7Qx5DweIX6TgvuI3WVde/2eb1ayoF kUkGBrJWqJ9S9tP3ezZcKgLEBGVHjYAniE4g/6DtJEeWVzvtYF+KmKSjBA+fEuBJ26ZQ HJT2bjmbP3TNNqp0kvzGfsaHz8gpfB4jibihs95DXaAFtliip+b4YYNCbTlDoabizh2/ qg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2pj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:33 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:31 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:30 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 6E2403F703F; Tue, 6 Apr 2021 08:11:28 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , , Nithin Dabilpuram Date: Tue, 6 Apr 2021 20:41:05 +0530 Message-ID: <20210406151115.1889455-2-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: ylkkzIOjEuY_2J3hDDEclrBo9O_kaMdx X-Proofpoint-ORIG-GUID: ylkkzIOjEuY_2J3hDDEclrBo9O_kaMdx X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 01/11] mempool/cnxk: add build infra and doc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the meson based build infrastructure for Marvell CNXK mempool driver along with stub implementations for mempool device probe. Also add Marvell CNXK mempool base documentation. Signed-off-by: Pavan Nikhilesh Signed-off-by: Jerin Jacob Signed-off-by: Nithin Dabilpuram Signed-off-by: Ashwin Sekhar T K --- MAINTAINERS | 6 +++ doc/guides/mempool/cnxk.rst | 55 ++++++++++++++++++++ doc/guides/mempool/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 ++ drivers/mempool/cnxk/cnxk_mempool.c | 78 +++++++++++++++++++++++++++++ drivers/mempool/cnxk/meson.build | 13 +++++ drivers/mempool/cnxk/version.map | 3 ++ drivers/mempool/meson.build | 3 +- 8 files changed, 161 insertions(+), 1 deletion(-) create mode 100644 doc/guides/mempool/cnxk.rst create mode 100644 drivers/mempool/cnxk/cnxk_mempool.c create mode 100644 drivers/mempool/cnxk/meson.build create mode 100644 drivers/mempool/cnxk/version.map diff --git a/MAINTAINERS b/MAINTAINERS index c837516d14..bae8b93030 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -501,6 +501,12 @@ M: Artem V. Andreev M: Andrew Rybchenko F: drivers/mempool/bucket/ +Marvell cnxk +M: Ashwin Sekhar T K +M: Pavan Nikhilesh +F: drivers/mempool/cnxk/ +F: doc/guides/mempool/cnxk.rst + Marvell OCTEON TX2 M: Jerin Jacob M: Nithin Dabilpuram diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst new file mode 100644 index 0000000000..e72a77c361 --- /dev/null +++ b/doc/guides/mempool/cnxk.rst @@ -0,0 +1,55 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2021 Marvell. + +CNXK NPA Mempool Driver +============================ + +The CNXK NPA PMD (**librte_mempool_cnxk**) provides mempool driver support for +the integrated mempool device found in **Marvell OCTEON CN9K/CN10K** SoC family. + +More information about CNXK SoC can be found at `Marvell Official Website +`_. + +Features +-------- + +CNXK NPA PMD supports: + +- Up to 128 NPA LFs +- 1M Pools per LF +- HW mempool manager +- Ethdev Rx buffer allocation in HW to save CPU cycles in the Rx path. +- Ethdev Tx buffer recycling in HW to save CPU cycles in the Tx path. + +Prerequisites and Compilation procedure +--------------------------------------- + + See :doc:`../platform/cnxk` for setup information. + +Pre-Installation Configuration +------------------------------ + + +Debugging Options +~~~~~~~~~~~~~~~~~ + +.. _table_cnxk_mempool_debug_options: + +.. table:: CNXK mempool debug options + + +---+------------+-------------------------------------------------------+ + | # | Component | EAL log command | + +===+============+=======================================================+ + | 1 | NPA | --log-level='pmd\.mempool.cnxk,8' | + +---+------------+-------------------------------------------------------+ + +Standalone mempool device +~~~~~~~~~~~~~~~~~~~~~~~~~ + + The ``usertools/dpdk-devbind.py`` script shall enumerate all the mempool + devices available in the system. In order to avoid, the end user to bind the + mempool device prior to use ethdev and/or eventdev device, the respective + driver configures an NPA LF and attach to the first probed ethdev or eventdev + device. In case, if end user need to run mempool as a standalone device + (without ethdev or eventdev), end user needs to bind a mempool device using + ``usertools/dpdk-devbind.py`` diff --git a/doc/guides/mempool/index.rst b/doc/guides/mempool/index.rst index a0e55467e6..ce53bc1ac7 100644 --- a/doc/guides/mempool/index.rst +++ b/doc/guides/mempool/index.rst @@ -11,6 +11,7 @@ application through the mempool API. :maxdepth: 2 :numbered: + cnxk octeontx octeontx2 ring diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst index d8fc00df40..489569ef4e 100644 --- a/doc/guides/platform/cnxk.rst +++ b/doc/guides/platform/cnxk.rst @@ -142,6 +142,9 @@ HW Offload Drivers This section lists dataplane H/W block(s) available in CNXK SoC. +#. **Mempool Driver** + See :doc:`../mempool/cnxk` for NPA mempool driver information. + Procedure to Setup Platform --------------------------- diff --git a/drivers/mempool/cnxk/cnxk_mempool.c b/drivers/mempool/cnxk/cnxk_mempool.c new file mode 100644 index 0000000000..947078c052 --- /dev/null +++ b/drivers/mempool/cnxk/cnxk_mempool.c @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "roc_api.h" + +static int +npa_remove(struct rte_pci_device *pci_dev) +{ + RTE_SET_USED(pci_dev); + + return 0; +} + +static int +npa_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + RTE_SET_USED(pci_drv); + RTE_SET_USED(pci_dev); + + return 0; +} + +static const struct rte_pci_id npa_pci_map[] = { + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_PF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KA, + }, + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_PF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KAS, + }, + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_VF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KA, + }, + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_VF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KAS, + }, + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver npa_pci = { + .id_table = npa_pci_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA, + .probe = npa_probe, + .remove = npa_remove, +}; + +RTE_PMD_REGISTER_PCI(mempool_cnxk, npa_pci); +RTE_PMD_REGISTER_PCI_TABLE(mempool_cnxk, npa_pci_map); +RTE_PMD_REGISTER_KMOD_DEP(mempool_cnxk, "vfio-pci"); diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build new file mode 100644 index 0000000000..0be0802373 --- /dev/null +++ b/drivers/mempool/cnxk/meson.build @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2021 Marvell. +# + +if not is_linux or not dpdk_conf.get('RTE_ARCH_64') + build = false + reason = 'only supported on 64-bit Linux' + subdir_done() +endif + +sources = files('cnxk_mempool.c') + +deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] diff --git a/drivers/mempool/cnxk/version.map b/drivers/mempool/cnxk/version.map new file mode 100644 index 0000000000..ee80c51721 --- /dev/null +++ b/drivers/mempool/cnxk/version.map @@ -0,0 +1,3 @@ +INTERNAL { + local: *; +}; diff --git a/drivers/mempool/meson.build b/drivers/mempool/meson.build index 4428813dae..a2814c1dfa 100644 --- a/drivers/mempool/meson.build +++ b/drivers/mempool/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -drivers = ['bucket', 'dpaa', 'dpaa2', 'octeontx', 'octeontx2', 'ring', 'stack'] +drivers = ['bucket', 'cnxk', 'dpaa', 'dpaa2', 'octeontx', 'octeontx2', 'ring', + 'stack'] std_deps = ['mempool'] From patchwork Tue Apr 6 15:11:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90732 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C2B2A0546; Tue, 6 Apr 2021 17:11:44 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C2E311411DD; Tue, 6 Apr 2021 17:11:37 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 864D81411D7 for ; Tue, 6 Apr 2021 17:11:36 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0T4q015121 for ; Tue, 6 Apr 2021 08:11:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=hFp3F8sTs/agzuR6lgFT2IrFAHA6g8ldQXi8Cl/qULU=; b=Qk/IahdQNaF4y3t9RLlb9Y/PYgDzr585Z4mvgNUElhQSGDpthZUNb7i+5KRE7wx0KqRc EUJzUMxu4+z4gW7zhT8hOSmRy4rPw3jBJzRZI+lZWKET1TGLJ+cliYGJQQ69IXUq2+eW 9qi05RbPZm4Zc68aQ/sIAKFvNK1O6jmO5bl9E5aNfT4clWza9DK6CgVk4lUsxH6pjBuk TxjssLcgM7kvQL+YDWrpCYyThuP4gxvYvGvjpDkzob1oy+kwsx1uI6EL845Vj9CLbxvG HFltrRep+k7nJuVdxBOzw+9IiwIClUrR2VzGvfcozZdP025+7ti0EqRxw6G9U+9GRpJs 3Q== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2pv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:35 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:33 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:33 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 75BBF3F7041; Tue, 6 Apr 2021 08:11:31 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:06 +0530 Message-ID: <20210406151115.1889455-3-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: B3BkF9jOu6YyzX-lrUlHFBNc1AB65reZ X-Proofpoint-ORIG-GUID: B3BkF9jOu6YyzX-lrUlHFBNc1AB65reZ X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 02/11] mempool/cnxk: add device probe/remove X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for CNXk mempool device probe and remove. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 23 +++++ drivers/mempool/cnxk/cnxk_mempool.c | 132 +++++++++++++++++++++++++++- 2 files changed, 151 insertions(+), 4 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index e72a77c361..907c19c841 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -30,6 +30,29 @@ Pre-Installation Configuration ------------------------------ +Runtime Config Options +~~~~~~~~~~~~~~~~~~~~~~ + +- ``Maximum number of mempools per application`` (default ``128``) + + The maximum number of mempools per application needs to be configured on + HW during mempool driver initialization. HW can support up to 1M mempools, + Since each mempool costs set of HW resources, the ``max_pools`` ``devargs`` + parameter is being introduced to configure the number of mempools required + for the application. + For example:: + + -a 0002:02:00.0,max_pools=512 + + With the above configuration, the driver will set up only 512 mempools for + the given application to save HW resources. + +.. note:: + + Since this configuration is per application, the end user needs to + provide ``max_pools`` parameter to the first PCIe device probed by the given + application. + Debugging Options ~~~~~~~~~~~~~~~~~ diff --git a/drivers/mempool/cnxk/cnxk_mempool.c b/drivers/mempool/cnxk/cnxk_mempool.c index 947078c052..dd4d74ca05 100644 --- a/drivers/mempool/cnxk/cnxk_mempool.c +++ b/drivers/mempool/cnxk/cnxk_mempool.c @@ -15,21 +15,143 @@ #include "roc_api.h" +#define CNXK_NPA_DEV_NAME RTE_STR(cnxk_npa_dev_) +#define CNXK_NPA_DEV_NAME_LEN (sizeof(CNXK_NPA_DEV_NAME) + PCI_PRI_STR_SIZE) +#define CNXK_NPA_MAX_POOLS_PARAM "max_pools" + +static inline uint32_t +npa_aura_size_to_u32(uint8_t val) +{ + if (val == NPA_AURA_SZ_0) + return 128; + if (val >= NPA_AURA_SZ_MAX) + return BIT_ULL(20); + + return 1 << (val + 6); +} + static int -npa_remove(struct rte_pci_device *pci_dev) +parse_max_pools(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + if (val < npa_aura_size_to_u32(NPA_AURA_SZ_128)) + val = 128; + if (val > npa_aura_size_to_u32(NPA_AURA_SZ_1M)) + val = BIT_ULL(20); + + *(uint8_t *)extra_args = rte_log2_u32(val) - 6; + return 0; +} + +static inline uint8_t +parse_aura_size(struct rte_devargs *devargs) +{ + uint8_t aura_sz = NPA_AURA_SZ_128; + struct rte_kvargs *kvlist; + + if (devargs == NULL) + goto exit; + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + goto exit; + + rte_kvargs_process(kvlist, CNXK_NPA_MAX_POOLS_PARAM, &parse_max_pools, + &aura_sz); + rte_kvargs_free(kvlist); +exit: + return aura_sz; +} + +static inline char * +npa_dev_to_name(struct rte_pci_device *pci_dev, char *name) +{ + snprintf(name, CNXK_NPA_DEV_NAME_LEN, CNXK_NPA_DEV_NAME PCI_PRI_FMT, + pci_dev->addr.domain, pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + + return name; +} + +static int +npa_init(struct rte_pci_device *pci_dev) { - RTE_SET_USED(pci_dev); + char name[CNXK_NPA_DEV_NAME_LEN]; + const struct rte_memzone *mz; + struct roc_npa *dev; + int rc = -ENOMEM; + + mz = rte_memzone_reserve_aligned(npa_dev_to_name(pci_dev, name), + sizeof(*dev), SOCKET_ID_ANY, 0, + RTE_CACHE_LINE_SIZE); + if (mz == NULL) + goto error; + + dev = mz->addr; + dev->pci_dev = pci_dev; + + roc_idev_npa_maxpools_set(parse_aura_size(pci_dev->device.devargs)); + rc = roc_npa_dev_init(dev); + if (rc) + goto mz_free; + + return 0; + +mz_free: + rte_memzone_free(mz); +error: + plt_err("failed to initialize npa device rc=%d", rc); + return rc; +} + +static int +npa_fini(struct rte_pci_device *pci_dev) +{ + char name[CNXK_NPA_DEV_NAME_LEN]; + const struct rte_memzone *mz; + int rc; + + mz = rte_memzone_lookup(npa_dev_to_name(pci_dev, name)); + if (mz == NULL) + return -EINVAL; + + rc = roc_npa_dev_fini(mz->addr); + if (rc) { + if (rc != -EAGAIN) + plt_err("Failed to remove npa dev, rc=%d", rc); + return rc; + } + rte_memzone_free(mz); return 0; } +static int +npa_remove(struct rte_pci_device *pci_dev) +{ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + return npa_fini(pci_dev); +} + static int npa_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) { + int rc; + RTE_SET_USED(pci_drv); - RTE_SET_USED(pci_dev); - return 0; + rc = roc_plt_init(); + if (rc < 0) + return rc; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + return npa_init(pci_dev); } static const struct rte_pci_id npa_pci_map[] = { @@ -76,3 +198,5 @@ static struct rte_pci_driver npa_pci = { RTE_PMD_REGISTER_PCI(mempool_cnxk, npa_pci); RTE_PMD_REGISTER_PCI_TABLE(mempool_cnxk, npa_pci_map); RTE_PMD_REGISTER_KMOD_DEP(mempool_cnxk, "vfio-pci"); +RTE_PMD_REGISTER_PARAM_STRING(mempool_cnxk, + CNXK_NPA_MAX_POOLS_PARAM "=<128-1048576>"); From patchwork Tue Apr 6 15:11:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90733 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 258F1A0546; Tue, 6 Apr 2021 17:11:51 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0B0D21411D1; Tue, 6 Apr 2021 17:11:42 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id D63871411B7 for ; Tue, 6 Apr 2021 17:11:39 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0X1R015149 for ; Tue, 6 Apr 2021 08:11:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=6m2bw8SgR16WsjRk1Rmjs3hOVJ5bVx/JJBtsyFJiX5Y=; b=MC6xLhj9VLziXCv6n1VYZEr2qqaFsIOR7UNKmnmBnbBrjbeuDN/glCD5m2fz41rPfvmA 3Sr26Ke9j3i9doLSLKEIW6W/Gl6WnV1Q7K6ibcEgpv5rG8pfoJBhC2s3vr8G/Onwb2OG Mpkbb7QVK44AJ/6fml1QXLgOWF2P9x/DdFd2b3VU/mVtb5u7g6oDOMPHszOYNL9pBThb fKHP6pva3/5ZtevSCjiq781Wdy0CllEzG0nivIuCxB/5BQhiO3coy5a6WQ6Dz2IMjtO8 tdxN0RJ8v6w+lf2A1/F/vZqSY0IGedeewqxYlW6hxMyYaNYNtcBBuWTYc+zkzEM+42rw bg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2q2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:38 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:36 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:36 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 3E5EC3F703F; Tue, 6 Apr 2021 08:11:34 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:07 +0530 Message-ID: <20210406151115.1889455-4-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: FQxvcMOy9lQ70Lk2URK_TSTRtIVJ0Uo6 X-Proofpoint-ORIG-GUID: FQxvcMOy9lQ70Lk2URK_TSTRtIVJ0Uo6 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 03/11] mempool/cnxk: add generic ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add generic CNXk mempool ops which will enqueue/dequeue from pool one element at a time. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cnxk_mempool.h | 26 ++++ drivers/mempool/cnxk/cnxk_mempool_ops.c | 171 ++++++++++++++++++++++++ drivers/mempool/cnxk/meson.build | 3 +- 3 files changed, 199 insertions(+), 1 deletion(-) create mode 100644 drivers/mempool/cnxk/cnxk_mempool.h create mode 100644 drivers/mempool/cnxk/cnxk_mempool_ops.c diff --git a/drivers/mempool/cnxk/cnxk_mempool.h b/drivers/mempool/cnxk/cnxk_mempool.h new file mode 100644 index 0000000000..099b7f6998 --- /dev/null +++ b/drivers/mempool/cnxk/cnxk_mempool.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef _CNXK_MEMPOOL_H_ +#define _CNXK_MEMPOOL_H_ + +#include + +unsigned int cnxk_mempool_get_count(const struct rte_mempool *mp); +ssize_t cnxk_mempool_calc_mem_size(const struct rte_mempool *mp, + uint32_t obj_num, uint32_t pg_shift, + size_t *min_chunk_size, size_t *align); +int cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, + void *vaddr, rte_iova_t iova, size_t len, + rte_mempool_populate_obj_cb_t *obj_cb, + void *obj_cb_arg); +int cnxk_mempool_alloc(struct rte_mempool *mp); +void cnxk_mempool_free(struct rte_mempool *mp); + +int __rte_hot cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, + unsigned int n); +int __rte_hot cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, + unsigned int n); + +#endif diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c new file mode 100644 index 0000000000..2ce1816c04 --- /dev/null +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -0,0 +1,171 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +#include "roc_api.h" +#include "cnxk_mempool.h" + +int __rte_hot +cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, unsigned int n) +{ + unsigned int index; + + /* Ensure mbuf init changes are written before the free pointers + * are enqueued to the stack. + */ + rte_io_wmb(); + for (index = 0; index < n; index++) + roc_npa_aura_op_free(mp->pool_id, 0, + (uint64_t)obj_table[index]); + + return 0; +} + +int __rte_hot +cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n) +{ + unsigned int index; + uint64_t obj; + + for (index = 0; index < n; index++, obj_table++) { + int retry = 4; + + /* Retry few times before failing */ + do { + obj = roc_npa_aura_op_alloc(mp->pool_id, 0); + } while (retry-- && (obj == 0)); + + if (obj == 0) { + cnxk_mempool_enq(mp, obj_table - index, index); + return -ENOENT; + } + *obj_table = (void *)obj; + } + + return 0; +} + +unsigned int +cnxk_mempool_get_count(const struct rte_mempool *mp) +{ + return (unsigned int)roc_npa_aura_op_available(mp->pool_id); +} + +ssize_t +cnxk_mempool_calc_mem_size(const struct rte_mempool *mp, uint32_t obj_num, + uint32_t pg_shift, size_t *min_chunk_size, + size_t *align) +{ + size_t total_elt_sz; + + /* Need space for one more obj on each chunk to fulfill + * alignment requirements. + */ + total_elt_sz = mp->header_size + mp->elt_size + mp->trailer_size; + return rte_mempool_op_calc_mem_size_helper( + mp, obj_num, pg_shift, total_elt_sz, min_chunk_size, align); +} + +int +cnxk_mempool_alloc(struct rte_mempool *mp) +{ + uint64_t aura_handle = 0; + struct npa_aura_s aura; + struct npa_pool_s pool; + uint32_t block_count; + size_t block_size; + int rc = -ERANGE; + + block_size = mp->elt_size + mp->header_size + mp->trailer_size; + block_count = mp->size; + if (mp->header_size % ROC_ALIGN != 0) { + plt_err("Header size should be multiple of %dB", ROC_ALIGN); + goto error; + } + + if (block_size % ROC_ALIGN != 0) { + plt_err("Block size should be multiple of %dB", ROC_ALIGN); + goto error; + } + + memset(&aura, 0, sizeof(struct npa_aura_s)); + memset(&pool, 0, sizeof(struct npa_pool_s)); + pool.nat_align = 1; + pool.buf_offset = mp->header_size / ROC_ALIGN; + + /* Use driver specific mp->pool_config to override aura config */ + if (mp->pool_config != NULL) + memcpy(&aura, mp->pool_config, sizeof(struct npa_aura_s)); + + rc = roc_npa_pool_create(&aura_handle, block_size, block_count, &aura, + &pool); + if (rc) { + plt_err("Failed to alloc pool or aura rc=%d", rc); + goto error; + } + + /* Store aura_handle for future queue operations */ + mp->pool_id = aura_handle; + plt_npa_dbg("block_sz=%lu block_count=%d aura_handle=0x%" PRIx64, + block_size, block_count, aura_handle); + + return 0; +error: + return rc; +} + +void +cnxk_mempool_free(struct rte_mempool *mp) +{ + int rc = 0; + + plt_npa_dbg("aura_handle=0x%" PRIx64, mp->pool_id); + rc = roc_npa_pool_destroy(mp->pool_id); + if (rc) + plt_err("Failed to free pool or aura rc=%d", rc); +} + +int +cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, + void *vaddr, rte_iova_t iova, size_t len, + rte_mempool_populate_obj_cb_t *obj_cb, void *obj_cb_arg) +{ + size_t total_elt_sz, off; + int num_elts; + + if (iova == RTE_BAD_IOVA) + return -EINVAL; + + total_elt_sz = mp->header_size + mp->elt_size + mp->trailer_size; + + /* Align object start address to a multiple of total_elt_sz */ + off = total_elt_sz - ((((uintptr_t)vaddr - 1) % total_elt_sz) + 1); + + if (len < off) + return -EINVAL; + + vaddr = (char *)vaddr + off; + iova += off; + len -= off; + num_elts = len / total_elt_sz; + + plt_npa_dbg("iova %" PRIx64 ", aligned iova %" PRIx64 "", iova - off, + iova); + plt_npa_dbg("length %" PRIu64 ", aligned length %" PRIu64 "", + (uint64_t)(len + off), (uint64_t)len); + plt_npa_dbg("element size %" PRIu64 "", (uint64_t)total_elt_sz); + plt_npa_dbg("requested objects %" PRIu64 ", possible objects %" PRIu64 + "", (uint64_t)max_objs, (uint64_t)num_elts); + + roc_npa_aura_op_range_set(mp->pool_id, iova, + iova + num_elts * total_elt_sz); + + if (roc_npa_pool_range_update_check(mp->pool_id) < 0) + return -EBUSY; + + return rte_mempool_op_populate_helper( + mp, RTE_MEMPOOL_POPULATE_F_ALIGN_OBJ, max_objs, vaddr, iova, + len, obj_cb, obj_cb_arg); +} diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build index 0be0802373..52244e728b 100644 --- a/drivers/mempool/cnxk/meson.build +++ b/drivers/mempool/cnxk/meson.build @@ -8,6 +8,7 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') subdir_done() endif -sources = files('cnxk_mempool.c') +sources = files('cnxk_mempool.c', + 'cnxk_mempool_ops.c') deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] From patchwork Tue Apr 6 15:11:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90734 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7DE84A0546; Tue, 6 Apr 2021 17:11:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A82791411F0; Tue, 6 Apr 2021 17:11:43 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 8C3711411D1 for ; Tue, 6 Apr 2021 17:11:41 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F07hp004267 for ; Tue, 6 Apr 2021 08:11:40 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=n6sxRHF0qyLReygfA6XpJiN5vRpsGIHobtURGk3feB4=; b=GkGD9kHP4WaQTCv+HVyht3OTSLk6mo/rhy18W7C5usfn6UXK36CpSoFHGIZ6Q6pVe+f1 J2UBQyBNjVO37W8W1w/hg5Upkq5QEu7YV/EW9AeKCuXOQ0Wf3rhnQVcIptXUtu6SU7Ii MGvpGyRWdaEq5OjZejJzySa20DQyuCXDPnhp4oyuvRuwwK6sVnZNONDFhPw4DoiPnWCr 87OSTS9rrqtlD7RbCyN9so+GJndG71oMFVi95xceh548DZIg0FgsfYERHrG6n5r43sy+ RgOg8hw1Vp3F9k3UMO4no5s4kuFVMYwaS89namuyDnyDVKOnjYxZQmj8oGQMza62xMU+ jg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 37r72p35gu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:40 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:39 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:39 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 0BFE13F7040; Tue, 6 Apr 2021 08:11:36 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:08 +0530 Message-ID: <20210406151115.1889455-5-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: vDRL1yyLjyENCBSgpNd2CZdpoleLZJsk X-Proofpoint-ORIG-GUID: vDRL1yyLjyENCBSgpNd2CZdpoleLZJsk X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 04/11] mempool/cnxk: register plt init callback X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Register the CNXk mempool plt init callback which will set the appropriate mempool ops to be used for the platform. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cnxk_mempool_ops.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index 2ce1816c04..e8f64be76b 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -2,6 +2,7 @@ * Copyright(C) 2021 Marvell. */ +#include #include #include "roc_api.h" @@ -169,3 +170,17 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, mp, RTE_MEMPOOL_POPULATE_F_ALIGN_OBJ, max_objs, vaddr, iova, len, obj_cb, obj_cb_arg); } + +static int +cnxk_mempool_plt_init(void) +{ + if (roc_model_is_cn10k() || roc_model_is_cn9k()) + rte_mbuf_set_platform_mempool_ops("cnxk_mempool_ops"); + + return 0; +} + +RTE_INIT(cnxk_mempool_ops_init) +{ + roc_plt_init_cb_register(cnxk_mempool_plt_init); +} From patchwork Tue Apr 6 15:11:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90735 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4BB2CA0546; Tue, 6 Apr 2021 17:12:06 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E14D21411F7; Tue, 6 Apr 2021 17:11:46 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A30CB1411E9 for ; Tue, 6 Apr 2021 17:11:44 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0RaI015105 for ; Tue, 6 Apr 2021 08:11:44 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=HpstXKB+6W62wiqjljt914QJoN0/ZmXdtWfYh1+iAfo=; b=NIq+ApJ7veqbv27MY/CC8yX8UG3um7HGU2lcV1JYQizQhzp6cqryRm4i7e0uCiyAfF6g 7414fQy3KBcl+kslqOmQyHC7nDDZ1jKcXScb/YU2HQIdF5J9oSYnX7P95qeIL60uyUxr BhKkPqnMZoC6OcMA9xs67TGYZFxxLkHHNgjUhnMlc7WfWBJMCtWOgAQ55KJ32KrXKAM8 NN3gb7qMLal+fA9J/uMqSKM33jmIm81lPRVgIQmXNxgGxmQ4xelO2JJWSJpmXuMeCRPV 3bkicd8BNll+1eO6rSV2Emxh+yAgv1c3s5E2kjHSDvKW7eh0wcj40aBaBOQAI6ji+Q5g 5g== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2qb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:44 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:42 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:42 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id C838B3F703F; Tue, 6 Apr 2021 08:11:39 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:09 +0530 Message-ID: <20210406151115.1889455-6-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: Vd330TE6iMYj_CS6ArV1Fyg02TJBXVfX X-Proofpoint-ORIG-GUID: Vd330TE6iMYj_CS6ArV1Fyg02TJBXVfX X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 05/11] mempool/cnxk: add cn9k mempool ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Marvell CN9k mempool ops and implement CN9k mempool alloc which makes sure that the element size always occupy odd number of cachelines to ensure even distribution among of elements among L1D cache sets. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn9k_mempool_ops.c | 54 +++++++++++++++++++++++++ drivers/mempool/cnxk/cnxk_mempool_ops.c | 4 +- drivers/mempool/cnxk/meson.build | 3 +- 3 files changed, 59 insertions(+), 2 deletions(-) create mode 100644 drivers/mempool/cnxk/cn9k_mempool_ops.c diff --git a/drivers/mempool/cnxk/cn9k_mempool_ops.c b/drivers/mempool/cnxk/cn9k_mempool_ops.c new file mode 100644 index 0000000000..f5ac163af9 --- /dev/null +++ b/drivers/mempool/cnxk/cn9k_mempool_ops.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +#include "roc_api.h" +#include "cnxk_mempool.h" + +static int +cn9k_mempool_alloc(struct rte_mempool *mp) +{ + size_t block_size, padding; + + block_size = mp->elt_size + mp->header_size + mp->trailer_size; + /* Align header size to ROC_ALIGN */ + if (mp->header_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(mp->header_size, ROC_ALIGN) - + mp->header_size; + mp->header_size += padding; + block_size += padding; + } + + /* Align block size to ROC_ALIGN */ + if (block_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(block_size, ROC_ALIGN) - block_size; + mp->trailer_size += padding; + block_size += padding; + } + + /* + * Marvell CN9k has 8 sets, 41 ways L1D cache, VA<9:7> bits dictate the + * set selection. Add additional padding to ensure that the element size + * always occupies odd number of cachelines to ensure even distribution + * of elements among L1D cache sets. + */ + padding = ((block_size / ROC_ALIGN) % 2) ? 0 : ROC_ALIGN; + mp->trailer_size += padding; + + return cnxk_mempool_alloc(mp); +} + +static struct rte_mempool_ops cn9k_mempool_ops = { + .name = "cn9k_mempool_ops", + .alloc = cn9k_mempool_alloc, + .free = cnxk_mempool_free, + .enqueue = cnxk_mempool_enq, + .dequeue = cnxk_mempool_deq, + .get_count = cnxk_mempool_get_count, + .calc_mem_size = cnxk_mempool_calc_mem_size, + .populate = cnxk_mempool_populate, +}; + +MEMPOOL_REGISTER_OPS(cn9k_mempool_ops); diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index e8f64be76b..d8ed37ec1a 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -174,7 +174,9 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, static int cnxk_mempool_plt_init(void) { - if (roc_model_is_cn10k() || roc_model_is_cn9k()) + if (roc_model_is_cn9k()) + rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); + else if (roc_model_is_cn10k()) rte_mbuf_set_platform_mempool_ops("cnxk_mempool_ops"); return 0; diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build index 52244e728b..ff31893ff4 100644 --- a/drivers/mempool/cnxk/meson.build +++ b/drivers/mempool/cnxk/meson.build @@ -9,6 +9,7 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') endif sources = files('cnxk_mempool.c', - 'cnxk_mempool_ops.c') + 'cnxk_mempool_ops.c', + 'cn9k_mempool_ops.c') deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] From patchwork Tue Apr 6 15:11:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90736 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 96CB1A0546; Tue, 6 Apr 2021 17:12:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 19A3A1411FE; Tue, 6 Apr 2021 17:11:50 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id B39601411F9 for ; Tue, 6 Apr 2021 17:11:47 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0ehg015459 for ; Tue, 6 Apr 2021 08:11:47 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=mj8J/pQ7kUa3fsAvqWMQnyHPacc5py+6X7c4uteKjLQ=; b=DOUbYRnrrroBvjNl0Bdk3s26iT/sxAB78Q3q1ALIaDDtp2CuGE1B6Duf5u5jHfQITFsj eZS+VJontB5g68G/esQJFABdy3TZWiFfjKV7QkIUZbZhNaVFYEEVWG3ylC5T9jh44p14 p36IkOmbLEp/Of4DlNPpLWGjU/SO8SCdqUmS/LY+oPP4eTYpXEE+IRzmTffXI2aN7EVh tfSg81tNHCnP/h0sNgcMIAtOb/ELqG1N3PYclSWV1Fba0noxC3AzdVJ9atXQLwBt7gFp Kz92/vknVgY/SzN3ioCg2F+PQd+2UBkMelJfgibN93LyXA3ZYvvrnqeUW0hrnlV1wqPD nQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2qf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:46 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:45 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:44 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 90D133F7040; Tue, 6 Apr 2021 08:11:42 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:10 +0530 Message-ID: <20210406151115.1889455-7-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: GnYwKv-OyFLiDY-GYY4VtANLItVEwe0u X-Proofpoint-ORIG-GUID: GnYwKv-OyFLiDY-GYY4VtANLItVEwe0u X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 06/11] mempool/cnxk: add cn9k optimized mempool enqueue/dequeue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Marvell CN9k mempool enqueue/dequeue. Marvell CN9k supports burst dequeue which allows to dequeue up to 32 pointers using pipelined casp instructions. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 4 +++ drivers/mempool/cnxk/cn9k_mempool_ops.c | 39 +++++++++++++++++++++++-- 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index 907c19c841..f51532b101 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -21,6 +21,10 @@ CNXK NPA PMD supports: - Ethdev Rx buffer allocation in HW to save CPU cycles in the Rx path. - Ethdev Tx buffer recycling in HW to save CPU cycles in the Tx path. +CN9k NPA supports: + +- Burst alloc of up to 32 pointers. + Prerequisites and Compilation procedure --------------------------------------- diff --git a/drivers/mempool/cnxk/cn9k_mempool_ops.c b/drivers/mempool/cnxk/cn9k_mempool_ops.c index f5ac163af9..c0cdba640b 100644 --- a/drivers/mempool/cnxk/cn9k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn9k_mempool_ops.c @@ -7,6 +7,41 @@ #include "roc_api.h" #include "cnxk_mempool.h" +static int __rte_hot +cn9k_mempool_enq(struct rte_mempool *mp, void *const *obj_table, unsigned int n) +{ + /* Ensure mbuf init changes are written before the free pointers + * are enqueued to the stack. + */ + rte_io_wmb(); + roc_npa_aura_op_bulk_free(mp->pool_id, (const uint64_t *)obj_table, n, + 0); + + return 0; +} + +static inline int __rte_hot +cn9k_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n) +{ + unsigned int count; + + count = roc_npa_aura_op_bulk_alloc(mp->pool_id, (uint64_t *)obj_table, + n, 0, 1); + + if (unlikely(count != n)) { + /* If bulk alloc failed to allocate all pointers, try + * allocating remaining pointers with the default alloc + * with retry scheme. + */ + if (cnxk_mempool_deq(mp, &obj_table[count], n - count)) { + cn9k_mempool_enq(mp, obj_table, count); + return -ENOENT; + } + } + + return 0; +} + static int cn9k_mempool_alloc(struct rte_mempool *mp) { @@ -44,8 +79,8 @@ static struct rte_mempool_ops cn9k_mempool_ops = { .name = "cn9k_mempool_ops", .alloc = cn9k_mempool_alloc, .free = cnxk_mempool_free, - .enqueue = cnxk_mempool_enq, - .dequeue = cnxk_mempool_deq, + .enqueue = cn9k_mempool_enq, + .dequeue = cn9k_mempool_deq, .get_count = cnxk_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, .populate = cnxk_mempool_populate, From patchwork Tue Apr 6 15:11:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90737 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E305AA0546; Tue, 6 Apr 2021 17:12:19 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 53841141204; Tue, 6 Apr 2021 17:11:51 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 69D001411FF for ; Tue, 6 Apr 2021 17:11:50 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0UQo015128 for ; Tue, 6 Apr 2021 08:11:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=hbRDLeqKGzlBzSKM8ZPiGEMZqOVSNja5lpHI6Om/QxY=; b=bSJMLrdJf0P5Tf3qP+cjlqAfbi1Q9I/QuvTm4DZyLPlO5J4/dc7rjWFe73kmhi+FqRyu dBi/9JT+JElhcjSI/p6YOPWxHNBF0Zqfb8kw5n4PMNAFPObwcNO9iQvgF3o2rJpdhmW0 HHq2CapnuYn8jUexorVtKV48Q3AtiUu8werRfTCYSDWXgRQ87/faTGEsIJp7TeLBTflW FNG3IVLzZ3tL1wFZlHA1Zbzsvd9J3yQcOLKR3lYEpvxxoXjnytgibh+CIBpOeiUX34+3 VtZ0ga3RhOWMi4QL47GrH6RBjuX2k9i/BLWH0vqVfPUQa9wnl8sNHsKhbRxHwBAo05UQ Ng== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2qm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:49 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:47 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:47 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 59A373F703F; Tue, 6 Apr 2021 08:11:45 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:11 +0530 Message-ID: <20210406151115.1889455-8-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: Gt162dKIuWWnK4vZdNwXGnt2L2t0rEo_ X-Proofpoint-ORIG-GUID: Gt162dKIuWWnK4vZdNwXGnt2L2t0rEo_ X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 07/11] mempool/cnxk: add cn10k mempool ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Marvell CN10k mempool ops and implement CN10k mempool alloc. CN10k has 64 bytes L1D cache line size. Hence the CN10k mempool alloc does not make the element size an odd multiple L1D cache line size as NPA requires the element sizes to be multiples of 128 bytes. Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 4 ++ drivers/mempool/cnxk/cn10k_mempool_ops.c | 52 ++++++++++++++++++++++++ drivers/mempool/cnxk/cnxk_mempool_ops.c | 2 +- drivers/mempool/cnxk/meson.build | 3 +- 4 files changed, 59 insertions(+), 2 deletions(-) create mode 100644 drivers/mempool/cnxk/cn10k_mempool_ops.c diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index f51532b101..783368e690 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -80,3 +80,7 @@ Standalone mempool device device. In case, if end user need to run mempool as a standalone device (without ethdev or eventdev), end user needs to bind a mempool device using ``usertools/dpdk-devbind.py`` + + Example command to run ``mempool_autotest`` test with standalone CN10K NPA device:: + + echo "mempool_autotest" | /app/test/dpdk-test -c 0xf0 --mbuf-pool-ops-name="cn10k_mempool_ops" diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c new file mode 100644 index 0000000000..9b63789006 --- /dev/null +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +#include "roc_api.h" +#include "cnxk_mempool.h" + +static int +cn10k_mempool_alloc(struct rte_mempool *mp) +{ + uint32_t block_size; + size_t padding; + + block_size = mp->elt_size + mp->header_size + mp->trailer_size; + /* Align header size to ROC_ALIGN */ + if (mp->header_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(mp->header_size, ROC_ALIGN) - + mp->header_size; + mp->header_size += padding; + block_size += padding; + } + + /* Align block size to ROC_ALIGN */ + if (block_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(block_size, ROC_ALIGN) - block_size; + mp->trailer_size += padding; + block_size += padding; + } + + return cnxk_mempool_alloc(mp); +} + +static void +cn10k_mempool_free(struct rte_mempool *mp) +{ + cnxk_mempool_free(mp); +} + +static struct rte_mempool_ops cn10k_mempool_ops = { + .name = "cn10k_mempool_ops", + .alloc = cn10k_mempool_alloc, + .free = cn10k_mempool_free, + .enqueue = cnxk_mempool_enq, + .dequeue = cnxk_mempool_deq, + .get_count = cnxk_mempool_get_count, + .calc_mem_size = cnxk_mempool_calc_mem_size, + .populate = cnxk_mempool_populate, +}; + +MEMPOOL_REGISTER_OPS(cn10k_mempool_ops); diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index d8ed37ec1a..42c02bf14e 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -177,7 +177,7 @@ cnxk_mempool_plt_init(void) if (roc_model_is_cn9k()) rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); else if (roc_model_is_cn10k()) - rte_mbuf_set_platform_mempool_ops("cnxk_mempool_ops"); + rte_mbuf_set_platform_mempool_ops("cn10k_mempool_ops"); return 0; } diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build index ff31893ff4..3282b5e5a6 100644 --- a/drivers/mempool/cnxk/meson.build +++ b/drivers/mempool/cnxk/meson.build @@ -10,6 +10,7 @@ endif sources = files('cnxk_mempool.c', 'cnxk_mempool_ops.c', - 'cn9k_mempool_ops.c') + 'cn9k_mempool_ops.c', + 'cn10k_mempool_ops.c') deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] From patchwork Tue Apr 6 15:11:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90738 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0EA29A0546; Tue, 6 Apr 2021 17:12:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B29691411EA; Tue, 6 Apr 2021 17:11:54 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 4424014120F for ; Tue, 6 Apr 2021 17:11:53 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0UQp015128 for ; Tue, 6 Apr 2021 08:11:52 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=gjRglxpZZtZ3Wnhw0WdVxE7hcJr6gVU2JmmEwXsrAuE=; b=BqkpsgC7XfsKSv+2Vj90coe3cxlMPack1ezI6FTGdqhUYaVpXxQNFiDG9P05sZgARRoB xK/4MLoNgRlZ6TyWxmwouFrkZnhw38ea8vqa7sPhXnYhg4FyafcDJ/7yJqhC4gC5a7xj MigUFiDfxmYAsgYxx+yIzV0eN9sMM/4Juojk1vty0PhOTAsQAtxiRKuaE+dT5Qlwu1Bo cB49FxcgjPFxFr6zhI6U6YLt//7YJjvNoHVdtzhiYNmZu2IH44N5AmtcwlYXvcPs2iHq gxS7CdQFGriMgC9t+rLN8dGC2UGWT8RRq7YruYwKguhJ5ACa/i2MdgCZGp+9Rx8aKguC vQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2qw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:52 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:50 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:50 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 232A23F7040; Tue, 6 Apr 2021 08:11:47 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:12 +0530 Message-ID: <20210406151115.1889455-9-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: wTS3vH7Fxh6eCPS04jGwNa05AJ_yeztJ X-Proofpoint-ORIG-GUID: wTS3vH7Fxh6eCPS04jGwNa05AJ_yeztJ X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 08/11] mempool/cnxk: add batch op init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Marvell CN10k mempool supports batch enqueue/dequeue which can dequeue up to 512 pointers and enqueue up to 15 pointers using a single instruction. These batch operations require a DMA memory to enqueue/dequeue pointers. This patch adds the initialization of this DMA memory. Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 5 + drivers/mempool/cnxk/cn10k_mempool_ops.c | 147 ++++++++++++++++++++++- drivers/mempool/cnxk/cnxk_mempool.h | 2 + drivers/mempool/cnxk/cnxk_mempool_ops.c | 11 +- 4 files changed, 160 insertions(+), 5 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index 783368e690..286ee29003 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -25,6 +25,11 @@ CN9k NPA supports: - Burst alloc of up to 32 pointers. +CN10k NPA supports: + +- Batch dequeue of up to 512 pointers with single instruction. +- Batch enqueue of up to 15 pointers with single instruction. + Prerequisites and Compilation procedure --------------------------------------- diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index 9b63789006..a3aef0ddb2 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -7,11 +7,136 @@ #include "roc_api.h" #include "cnxk_mempool.h" +#define BATCH_ALLOC_SZ ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS +#define BATCH_OP_DATA_TABLE_MZ_NAME "batch_op_data_table_mz" + +enum batch_op_status { + BATCH_ALLOC_OP_NOT_ISSUED = 0, + BATCH_ALLOC_OP_ISSUED = 1, + BATCH_ALLOC_OP_DONE +}; + +struct batch_op_mem { + unsigned int sz; + enum batch_op_status status; + uint64_t objs[BATCH_ALLOC_SZ] __rte_aligned(ROC_ALIGN); +}; + +struct batch_op_data { + uint64_t lmt_addr; + struct batch_op_mem mem[RTE_MAX_LCORE] __rte_aligned(ROC_ALIGN); +}; + +static struct batch_op_data **batch_op_data_tbl; + +static int +batch_op_data_table_create(void) +{ + const struct rte_memzone *mz; + + /* If table is already set, nothing to do */ + if (batch_op_data_tbl) + return 0; + + mz = rte_memzone_lookup(BATCH_OP_DATA_TABLE_MZ_NAME); + if (mz == NULL) { + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + unsigned int maxpools, sz; + + maxpools = roc_idev_npa_maxpools_get(); + sz = maxpools * sizeof(struct batch_op_data *); + + mz = rte_memzone_reserve_aligned( + BATCH_OP_DATA_TABLE_MZ_NAME, sz, SOCKET_ID_ANY, + 0, ROC_ALIGN); + } + if (mz == NULL) { + plt_err("Failed to reserve batch op data table"); + return -ENOMEM; + } + } + batch_op_data_tbl = mz->addr; + rte_wmb(); + return 0; +} + +static inline struct batch_op_data * +batch_op_data_get(uint64_t pool_id) +{ + uint64_t aura = roc_npa_aura_handle_to_aura(pool_id); + + return batch_op_data_tbl[aura]; +} + +static inline void +batch_op_data_set(uint64_t pool_id, struct batch_op_data *op_data) +{ + uint64_t aura = roc_npa_aura_handle_to_aura(pool_id); + + batch_op_data_tbl[aura] = op_data; +} + +static int +batch_op_init(struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + int i; + + op_data = batch_op_data_get(mp->pool_id); + /* The data should not have been allocated previously */ + RTE_ASSERT(op_data == NULL); + + op_data = rte_zmalloc(NULL, sizeof(struct batch_op_data), ROC_ALIGN); + if (op_data == NULL) + return -ENOMEM; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + op_data->mem[i].sz = 0; + op_data->mem[i].status = BATCH_ALLOC_OP_NOT_ISSUED; + } + + op_data->lmt_addr = roc_idev_lmt_base_addr_get(); + batch_op_data_set(mp->pool_id, op_data); + rte_wmb(); + + return 0; +} + +static void +batch_op_fini(struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + int i; + + op_data = batch_op_data_get(mp->pool_id); + + rte_wmb(); + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct batch_op_mem *mem = &op_data->mem[i]; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) { + mem->sz = roc_npa_aura_batch_alloc_extract( + mem->objs, mem->objs, BATCH_ALLOC_SZ); + mem->status = BATCH_ALLOC_OP_DONE; + } + if (mem->status == BATCH_ALLOC_OP_DONE) { + roc_npa_aura_op_bulk_free(mp->pool_id, mem->objs, + mem->sz, 1); + mem->status = BATCH_ALLOC_OP_NOT_ISSUED; + } + } + + rte_free(op_data); + batch_op_data_set(mp->pool_id, NULL); + rte_wmb(); +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { uint32_t block_size; size_t padding; + int rc; block_size = mp->elt_size + mp->header_size + mp->trailer_size; /* Align header size to ROC_ALIGN */ @@ -29,15 +154,35 @@ cn10k_mempool_alloc(struct rte_mempool *mp) block_size += padding; } - return cnxk_mempool_alloc(mp); + rc = cnxk_mempool_alloc(mp); + if (rc) + return rc; + + rc = batch_op_init(mp); + if (rc) { + plt_err("Failed to init batch alloc mem rc=%d", rc); + goto error; + } + + return 0; +error: + cnxk_mempool_free(mp); + return rc; } static void cn10k_mempool_free(struct rte_mempool *mp) { + batch_op_fini(mp); cnxk_mempool_free(mp); } +int +cn10k_mempool_plt_init(void) +{ + return batch_op_data_table_create(); +} + static struct rte_mempool_ops cn10k_mempool_ops = { .name = "cn10k_mempool_ops", .alloc = cn10k_mempool_alloc, diff --git a/drivers/mempool/cnxk/cnxk_mempool.h b/drivers/mempool/cnxk/cnxk_mempool.h index 099b7f6998..3405aa7663 100644 --- a/drivers/mempool/cnxk/cnxk_mempool.h +++ b/drivers/mempool/cnxk/cnxk_mempool.h @@ -23,4 +23,6 @@ int __rte_hot cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, int __rte_hot cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n); +int cn10k_mempool_plt_init(void); + #endif diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index 42c02bf14e..c7b75f026d 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -174,12 +174,15 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, static int cnxk_mempool_plt_init(void) { - if (roc_model_is_cn9k()) + int rc = 0; + + if (roc_model_is_cn9k()) { rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); - else if (roc_model_is_cn10k()) + } else if (roc_model_is_cn10k()) { rte_mbuf_set_platform_mempool_ops("cn10k_mempool_ops"); - - return 0; + rc = cn10k_mempool_plt_init(); + } + return rc; } RTE_INIT(cnxk_mempool_ops_init) From patchwork Tue Apr 6 15:11:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90739 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 070B7A0546; Tue, 6 Apr 2021 17:12:34 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 566F4141208; Tue, 6 Apr 2021 17:11:57 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id B8903141217 for ; Tue, 6 Apr 2021 17:11:55 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0SWR015108 for ; Tue, 6 Apr 2021 08:11:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=9z9awlTkX7mV62t3OZnm8/UldOgQxqaEUSwDUvz/BpE=; b=QBy8amtlwZlvIFdzyhYYC7NxO/pdVYWBk/Aw+SKPjlFTcBzjHaQg6I7pRMEb/DHQw7YH vtqBuuIJNUudUMoByJyR9AAkCx3uMprH15r7FPhI1CfCPiE04Y1p4zkQQW8nARrt+IRW +jdLNxJwl41pApNRIZ1HuTMiG2JM+XCOhhHTelhfx7RtsQ1W4eYU67NPoTUSyYRv6PWE qTEkEvx2l9cq8kKR9a1iPLp0B1o2rZodmrZ6GJZF2wHo15u8mkFGjY9ydRNQSOKdWhBv Z6XWCWK79S2BtIkhC/ecOazh9r4ZW3ah9TM13iKoIxesobHzzNksPy5CXjRo/0uNgFN6 Qg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2r5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:55 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:53 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:53 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id E14373F703F; Tue, 6 Apr 2021 08:11:50 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:13 +0530 Message-ID: <20210406151115.1889455-10-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: pMokuunppFHv8YGFlICl4XVZT8rgajXC X-Proofpoint-ORIG-GUID: pMokuunppFHv8YGFlICl4XVZT8rgajXC X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 09/11] mempool/cnxk: add cn10k batch enqueue op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for Marvell CN10k mempool batch enqueue op. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn10k_mempool_ops.c | 28 +++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index a3aef0ddb2..c225c227df 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -131,6 +131,32 @@ batch_op_fini(struct rte_mempool *mp) rte_wmb(); } +static int __rte_hot +cn10k_mempool_enq(struct rte_mempool *mp, void *const *obj_table, + unsigned int n) +{ + const uint64_t *ptr = (const uint64_t *)obj_table; + uint64_t lmt_addr = 0, lmt_id = 0; + struct batch_op_data *op_data; + + /* Ensure mbuf init changes are written before the free pointers are + * enqueued to the stack. + */ + rte_io_wmb(); + + if (n == 1) { + roc_npa_aura_op_free(mp->pool_id, 1, ptr[0]); + return 0; + } + + op_data = batch_op_data_get(mp->pool_id); + lmt_addr = op_data->lmt_addr; + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + roc_npa_aura_op_batch_free(mp->pool_id, ptr, n, 1, lmt_addr, lmt_id); + + return 0; +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { @@ -187,7 +213,7 @@ static struct rte_mempool_ops cn10k_mempool_ops = { .name = "cn10k_mempool_ops", .alloc = cn10k_mempool_alloc, .free = cn10k_mempool_free, - .enqueue = cnxk_mempool_enq, + .enqueue = cn10k_mempool_enq, .dequeue = cnxk_mempool_deq, .get_count = cnxk_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, From patchwork Tue Apr 6 15:11:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90740 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 65FA5A0546; Tue, 6 Apr 2021 17:12:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B03EC14121B; Tue, 6 Apr 2021 17:11:59 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 3C6FC14121F for ; Tue, 6 Apr 2021 17:11:58 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F06sZ004264 for ; Tue, 6 Apr 2021 08:11:57 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Yo4EdKWZyn6sdv5wu7Szwvf44Rp7ZMUo0Irel90M6wU=; b=Dri+GQnzDhOm7nWMJ+R2X3nU+OTf8gUUQvnmxIduYkPMlhzTA6UvPpSFORt5SvicUuBi ePX6KJk5SGRYp12v1MOzwnOVN0LxCB2oLhknVHx4M0HoG50/511QCLecfnmpidzr92OM hFbM9JDS+xA1gIcLmhu6e/c2rt2ZHkKG2kCpPmrQ4VZ3AEIe2T1BFzkGNWbzwOuNk+IW BIruBo8G6bDDraWN8Co1Phid6Ui4EFvF8nLhYEYSxpQKBsjo69MUBIc/OdNM4r9unoLn V2GRjTJTxdIt+xpGwIzYBgkKhmkkeIJrOexB4xaEWzq001g47+M4/CFHLxTkv6mDrN/4 9Q== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 37r72p35jj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:57 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:55 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:55 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id A8F3C3F7040; Tue, 6 Apr 2021 08:11:53 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:14 +0530 Message-ID: <20210406151115.1889455-11-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: xmYohuTOrua6meWfrwGv5xdO4KQVbFs1 X-Proofpoint-ORIG-GUID: xmYohuTOrua6meWfrwGv5xdO4KQVbFs1 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 10/11] mempool/cnxk: add cn10k get count op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for Marvell CN10k get count op. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn10k_mempool_ops.c | 28 +++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index c225c227df..d244a5e90f 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -157,6 +157,32 @@ cn10k_mempool_enq(struct rte_mempool *mp, void *const *obj_table, return 0; } +static unsigned int +cn10k_mempool_get_count(const struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + unsigned int count = 0; + int i; + + op_data = batch_op_data_get(mp->pool_id); + + rte_wmb(); + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct batch_op_mem *mem = &op_data->mem[i]; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) + count += roc_npa_aura_batch_alloc_count(mem->objs, + BATCH_ALLOC_SZ); + + if (mem->status == BATCH_ALLOC_OP_DONE) + count += mem->sz; + } + + count += cnxk_mempool_get_count(mp); + + return count; +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { @@ -215,7 +241,7 @@ static struct rte_mempool_ops cn10k_mempool_ops = { .free = cn10k_mempool_free, .enqueue = cn10k_mempool_enq, .dequeue = cnxk_mempool_deq, - .get_count = cnxk_mempool_get_count, + .get_count = cn10k_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, .populate = cnxk_mempool_populate, }; From patchwork Tue Apr 6 15:11:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90741 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D63DDA0546; Tue, 6 Apr 2021 17:12:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E2ECB14121F; Tue, 6 Apr 2021 17:12:02 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 3978B141229 for ; Tue, 6 Apr 2021 17:12:01 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0UQr015128 for ; Tue, 6 Apr 2021 08:12:00 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=K8SAd5qHCm7tqNs2/mmzRpMNzc/Y+z2mlYaJWuiSyfU=; b=NK+21m23VRJTCgVE5AXFyEJgromUDbzrEpnB7t4A/36j8tUuLuUIBAmapRDUjKYwkQDJ Ls+Eei8l75Il2k//g/KHfE410W2GmMEklbJbAeVzqthfnEQQEYc4aM/phNPJgvwH+c7S vCrD4Pq2K59ocuWBjOKo1bdBZ/qZtIJEhPo3TSA1V8YqghhF9E3md7dt9IA3H5tn3lRt qp3wzY4VuNnteUPB2EDRkyIxmExbWbpSqgdC/7xfeMt5kFxmltNKKEA1keHIcDp+0B1G w5y5iUey7mt7vZFID+wbuyjpR9+scBkwMKlImqmUA8hJUmQ07+9s9zHdzrKgbHRiTgTw Dg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2rh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:12:00 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:58 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:58 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 71CDD3F703F; Tue, 6 Apr 2021 08:11:56 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:15 +0530 Message-ID: <20210406151115.1889455-12-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: ZCvFhXI8Wo12qwxw1_a_0qy7nbeHCHF3 X-Proofpoint-ORIG-GUID: ZCvFhXI8Wo12qwxw1_a_0qy7nbeHCHF3 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 11/11] mempool/cnxk: add cn10k batch dequeue op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for Marvell CN10k mempool batch dequeue op. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn10k_mempool_ops.c | 72 +++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index d244a5e90f..95458b34b7 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -183,6 +183,76 @@ cn10k_mempool_get_count(const struct rte_mempool *mp) return count; } +static int __rte_hot +cn10k_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n) +{ + struct batch_op_data *op_data; + struct batch_op_mem *mem; + unsigned int count = 0; + int tid, rc, retry; + bool loop = true; + + op_data = batch_op_data_get(mp->pool_id); + tid = rte_lcore_id(); + mem = &op_data->mem[tid]; + + /* Issue batch alloc */ + if (mem->status == BATCH_ALLOC_OP_NOT_ISSUED) { + rc = roc_npa_aura_batch_alloc_issue(mp->pool_id, mem->objs, + BATCH_ALLOC_SZ, 0, 1); + /* If issue fails, try falling back to default alloc */ + if (unlikely(rc)) + return cn10k_mempool_enq(mp, obj_table, n); + mem->status = BATCH_ALLOC_OP_ISSUED; + } + + retry = 4; + while (loop) { + unsigned int cur_sz; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) { + mem->sz = roc_npa_aura_batch_alloc_extract( + mem->objs, mem->objs, BATCH_ALLOC_SZ); + + /* If partial alloc reduce the retry count */ + retry -= (mem->sz != BATCH_ALLOC_SZ); + /* Break the loop if retry count exhausted */ + loop = !!retry; + mem->status = BATCH_ALLOC_OP_DONE; + } + + cur_sz = n - count; + if (cur_sz > mem->sz) + cur_sz = mem->sz; + + /* Dequeue the pointers */ + memcpy(&obj_table[count], &mem->objs[mem->sz - cur_sz], + cur_sz * sizeof(uintptr_t)); + mem->sz -= cur_sz; + count += cur_sz; + + /* Break loop if the required pointers has been dequeued */ + loop &= (count != n); + + /* Issue next batch alloc if pointers are exhausted */ + if (mem->sz == 0) { + rc = roc_npa_aura_batch_alloc_issue( + mp->pool_id, mem->objs, BATCH_ALLOC_SZ, 0, 1); + /* Break loop if issue failed and set status */ + loop &= !rc; + mem->status = !rc; + } + } + + if (unlikely(count != n)) { + /* No partial alloc allowed. Free up allocated pointers */ + cn10k_mempool_enq(mp, obj_table, count); + return -ENOENT; + } + + return 0; +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { @@ -240,7 +310,7 @@ static struct rte_mempool_ops cn10k_mempool_ops = { .alloc = cn10k_mempool_alloc, .free = cn10k_mempool_free, .enqueue = cn10k_mempool_enq, - .dequeue = cnxk_mempool_deq, + .dequeue = cn10k_mempool_deq, .get_count = cn10k_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, .populate = cnxk_mempool_populate,