Cover Detail
Show a cover letter.
GET /api/covers/125632/?format=api
{ "id": 125632, "url": "http://patches.dpdk.org/api/covers/125632/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/cover/20230330062939.1206267-1-feifei.wang2@arm.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20230330062939.1206267-1-feifei.wang2@arm.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20230330062939.1206267-1-feifei.wang2@arm.com", "date": "2023-03-30T06:29:36", "name": "[v5,0/3] Recycle buffers from Tx to Rx", "submitter": { "id": 1771, "url": "http://patches.dpdk.org/api/people/1771/?format=api", "name": "Feifei Wang", "email": "feifei.wang2@arm.com" }, "mbox": "http://patches.dpdk.org/project/dpdk/cover/20230330062939.1206267-1-feifei.wang2@arm.com/mbox/", "series": [ { "id": 27580, "url": "http://patches.dpdk.org/api/series/27580/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=27580", "date": "2023-03-30T06:29:36", "name": "Recycle buffers from Tx to Rx", "version": 5, "mbox": "http://patches.dpdk.org/series/27580/mbox/" } ], "comments": "http://patches.dpdk.org/api/covers/125632/comments/", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 417824286D;\n\tThu, 30 Mar 2023 08:29:50 +0200 (CEST)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 01EE440E25;\n\tThu, 30 Mar 2023 08:29:50 +0200 (CEST)", "from foss.arm.com (foss.arm.com [217.140.110.172])\n by mails.dpdk.org (Postfix) with ESMTP id A686740685\n for <dev@dpdk.org>; Thu, 30 Mar 2023 08:29:48 +0200 (CEST)", "from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])\n by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5D9862F4;\n Wed, 29 Mar 2023 23:30:32 -0700 (PDT)", "from net-x86-dell-8268.shanghai.arm.com\n (net-x86-dell-8268.shanghai.arm.com [10.169.210.116])\n by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2ABA63F663;\n Wed, 29 Mar 2023 23:29:45 -0700 (PDT)" ], "From": "Feifei Wang <feifei.wang2@arm.com>", "To": "", "Cc": "dev@dpdk.org, konstantin.v.ananyev@yandex.ru, mb@smartsharesystems.com,\n nd@arm.com, Feifei Wang <feifei.wang2@arm.com>", "Subject": "[PATCH v5 0/3] Recycle buffers from Tx to Rx", "Date": "Thu, 30 Mar 2023 14:29:36 +0800", "Message-Id": "<20230330062939.1206267-1-feifei.wang2@arm.com>", "X-Mailer": "git-send-email 2.25.1", "In-Reply-To": "<20211224164613.32569-1-feifei.wang2@arm.com>", "References": "<20211224164613.32569-1-feifei.wang2@arm.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Currently, the transmit side frees the buffers into the lcore cache and\nthe receive side allocates buffers from the lcore cache. The transmit\nside typically frees 32 buffers resulting in 32*8=256B of stores to\nlcore cache. The receive side allocates 32 buffers and stores them in\nthe receive side software ring, resulting in 32*8=256B of stores and\n256B of load from the lcore cache.\n\nThis patch proposes a mechanism to avoid freeing to/allocating from\nthe lcore cache. i.e. the receive side will free the buffers from\ntransmit side directly into its software ring. This will avoid the 256B\nof loads and stores introduced by the lcore cache. It also frees up the\ncache lines used by the lcore cache. And we can call this mode as buffer\nrecycle mode.\n\nIn the latest version, buffer recycle mode is packaged as a separate API. \nThis allows for the users to change rxq/txq pairing in real time in data plane,\naccording to the analysis of the packet flow by the application, for example:\n-----------------------------------------------------------------------\nStep 1: upper application analyse the flow direction\nStep 2: rxq_buf_recycle_info = rte_eth_rx_buf_recycle_info_get(rx_portid, rx_queueid)\nStep 3: rte_eth_dev_buf_recycle(rx_portid, rx_queueid, tx_portid, tx_queueid, rxq_buf_recycle_info);\nStep 4: rte_eth_rx_burst(rx_portid,rx_queueid);\nStep 5: rte_eth_tx_burst(tx_portid,tx_queueid);\n-----------------------------------------------------------------------\nAbove can support user to change rxq/txq pairing at runtime and user does not need to\nknow the direction of flow in advance. This can effectively expand buffer recycle mode's\nuse scenarios.\n\nFurthermore, buffer recycle mode is no longer limited to the same pmd,\nit can support moving buffers between different vendor pmds, even can put the buffer\nanywhere into your Rx buffer ring as long as the address of the buffer ring can be provided.\nIn the latest version, we enable direct-rearm in i40e pmd and ixgbe pmd, and also try to\nuse i40e driver in Rx, ixgbe driver in Tx, and then achieve 7-9% performance improvement\nby buffer recycle mode.\n\nDifference between buffer recycle, ZC API used in mempool and general path\nFor general path: \n Rx: 32 pkts memcpy from mempool cache to rx_sw_ring\n Tx: 32 pkts memcpy from tx_sw_ring to temporary variable + 32 pkts memcpy from temporary variable to mempool cache\nFor ZC API used in mempool:\n Rx: 32 pkts memcpy from mempool cache to rx_sw_ring\n Tx: 32 pkts memcpy from tx_sw_ring to zero-copy mempool cache\n Refer link: http://patches.dpdk.org/project/dpdk/patch/20230221055205.22984-2-kamalakshitha.aligeri@arm.com/\nFor buffer recycle:\n Rx/Tx: 32 pkts memcpy from tx_sw_ring to rx_sw_ring\nThus we can see in the one loop, compared to general path, buffer recycle reduce 32+32=64 pkts memcpy;\nCompared to ZC API used in mempool, we can see buffer recycle reduce 32 pkts memcpy in each loop.\nSo, buffer recycle has its own benefits.\n\nTesting status:\n(1) dpdk l3fwd test with multiple drivers:\n port 0: 82599 NIC port 1: XL710 NIC\n-------------------------------------------------------------\n\t\tWithout fast free\tWith fast free\nThunderx2: +7.53%\t +13.54%\n-------------------------------------------------------------\n\n(2) dpdk l3fwd test with same driver:\n port 0 && 1: XL710 NIC\n-------------------------------------------------------------\n\t\tWithout fast free\tWith fast free\nAmpere altra: +12.61%\t\t +11.42%\nn1sdp:\t\t+8.30%\t\t\t+3.85%\nx86-sse:\t+8.43%\t\t\t+3.72%\n-------------------------------------------------------------\n\n(3) Performance comparison with ZC_mempool used\n port 0 && 1: XL710 NIC\n with fast free\n-------------------------------------------------------------\n\t\tWith recycle buffer\tWith zc_mempool\nAmpere altra:\t11.42%\t\t\t3.54%\n-------------------------------------------------------------\n\nV2:\n1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa)\n2. Add 'txq_data_get' API to get txq info for Rx (Konstantin)\n3. Use input parameter to enable direct rearm in l3fwd (Konstantin)\n4. Add condition detection for direct rearm API (Morten, Andrew Rybchenko)\n\nV3:\n1. Seperate Rx and Tx operation with two APIs in direct-rearm (Konstantin)\n2. Delete L3fwd change for direct rearm (Jerin)\n3. enable direct rearm in ixgbe driver in Arm\n\nv4:\n1. Rename direct-rearm as buffer recycle. Based on this, function name\nand variable name are changed to let this mode more general for all\ndrivers. (Konstantin, Morten)\n2. Add ring wrapping check (Konstantin)\n\nv5:\n1. some change for ethdev API (Morten)\n2. add support for avx2, sse, altivec path\n\nFeifei Wang (3):\n ethdev: add API for buffer recycle mode\n net/i40e: implement recycle buffer mode\n net/ixgbe: implement recycle buffer mode\n\n drivers/net/i40e/i40e_ethdev.c | 1 +\n drivers/net/i40e/i40e_ethdev.h | 2 +\n drivers/net/i40e/i40e_rxtx.c | 159 +++++++++++++++++++++\n drivers/net/i40e/i40e_rxtx.h | 4 +\n drivers/net/ixgbe/ixgbe_ethdev.c | 1 +\n drivers/net/ixgbe/ixgbe_ethdev.h | 3 +\n drivers/net/ixgbe/ixgbe_rxtx.c | 153 ++++++++++++++++++++\n drivers/net/ixgbe/ixgbe_rxtx.h | 4 +\n lib/ethdev/ethdev_driver.h | 10 ++\n lib/ethdev/ethdev_private.c | 2 +\n lib/ethdev/rte_ethdev.c | 33 +++++\n lib/ethdev/rte_ethdev.h | 230 +++++++++++++++++++++++++++++++\n lib/ethdev/rte_ethdev_core.h | 15 +-\n lib/ethdev/version.map | 6 +\n 14 files changed, 621 insertions(+), 2 deletions(-)" }