Cover Detail
Show a cover letter.
GET /api/covers/116921/?format=api
{ "id": 116921, "url": "http://patches.dpdk.org/api/covers/116921/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/cover/20220927024756.947272-1-feifei.wang2@arm.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20220927024756.947272-1-feifei.wang2@arm.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20220927024756.947272-1-feifei.wang2@arm.com", "date": "2022-09-27T02:47:53", "name": "[v2,0/3] Direct re-arming of buffers on receive side", "submitter": { "id": 1771, "url": "http://patches.dpdk.org/api/people/1771/?format=api", "name": "Feifei Wang", "email": "feifei.wang2@arm.com" }, "mbox": "http://patches.dpdk.org/project/dpdk/cover/20220927024756.947272-1-feifei.wang2@arm.com/mbox/", "series": [ { "id": 24846, "url": "http://patches.dpdk.org/api/series/24846/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=24846", "date": "2022-09-27T02:47:53", "name": "Direct re-arming of buffers on receive side", "version": 2, "mbox": "http://patches.dpdk.org/series/24846/mbox/" } ], "comments": "http://patches.dpdk.org/api/covers/116921/comments/", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 21C26A00C2;\n\tTue, 27 Sep 2022 04:48:12 +0200 (CEST)", "from [217.70.189.124] (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id C4F77410D0;\n\tTue, 27 Sep 2022 04:48:11 +0200 (CEST)", "from foss.arm.com (foss.arm.com [217.140.110.172])\n by mails.dpdk.org (Postfix) with ESMTP id A1D4240F19\n for <dev@dpdk.org>; Tue, 27 Sep 2022 04:48:10 +0200 (CEST)", "from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])\n by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7CFA81063;\n Mon, 26 Sep 2022 19:48:16 -0700 (PDT)", "from net-x86-dell-8268.shanghai.arm.com\n (net-x86-dell-8268.shanghai.arm.com [10.169.210.116])\n by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8F03B3F73D;\n Mon, 26 Sep 2022 19:48:08 -0700 (PDT)" ], "From": "Feifei Wang <feifei.wang2@arm.com>", "To": "", "Cc": "dev@dpdk.org,\n\tnd@arm.com,\n\tFeifei Wang <feifei.wang2@arm.com>", "Subject": "[PATCH v2 0/3] Direct re-arming of buffers on receive side", "Date": "Tue, 27 Sep 2022 10:47:53 +0800", "Message-Id": "<20220927024756.947272-1-feifei.wang2@arm.com>", "X-Mailer": "git-send-email 2.25.1", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Currently, the transmit side frees the buffers into the lcore cache and\nthe receive side allocates buffers from the lcore cache. The transmit\nside typically frees 32 buffers resulting in 32*8=256B of stores to\nlcore cache. The receive side allocates 32 buffers and stores them in\nthe receive side software ring, resulting in 32*8=256B of stores and\n256B of load from the lcore cache.\n\nThis patch proposes a mechanism to avoid freeing to/allocating from\nthe lcore cache. i.e. the receive side will free the buffers from\ntransmit side directly into it's software ring. This will avoid the 256B\nof loads and stores introduced by the lcore cache. It also frees up the\ncache lines used by the lcore cache.\n\nHowever, this solution poses several constraints:\n\n1)The receive queue needs to know which transmit queue it should take\nthe buffers from. The application logic decides which transmit port to\nuse to send out the packets. In many use cases the NIC might have a\nsingle port ([1], [2], [3]), in which case a given transmit queue is\nalways mapped to a single receive queue (1:1 Rx queue: Tx queue). This\nis easy to configure.\n\nIf the NIC has 2 ports (there are several references), then we will have\n1:2 (RX queue: TX queue) mapping which is still easy to configure.\nHowever, if this is generalized to 'N' ports, the configuration can be\nlong. More over the PMD would have to scan a list of transmit queues to\npull the buffers from.\n\n2)The other factor that needs to be considered is 'run-to-completion' vs\n'pipeline' models. In the run-to-completion model, the receive side and\nthe transmit side are running on the same lcore serially. In the pipeline\nmodel. The receive side and transmit side might be running on different\nlcores in parallel. This requires locking. This is not supported at this\npoint.\n\n3)Tx and Rx buffers must be from the same mempool. And we also must\nensure Tx buffer free number is equal to Rx buffer free number:\n(txq->tx_rs_thresh == RTE_I40E_RXQ_REARM_THRESH)\nThus, 'tx_next_dd' can be updated correctly in direct-rearm mode. This\nis due to tx_next_dd is a variable to compute tx sw-ring free location.\nIts value will be one more round than the position where next time free\nstarts.\n\nCurrent status in this patch:\n1)Two APIs are added for users to enable direct-rearm mode:\n In control plane, users can call 'rte_eth_txq_data_get' to get Tx sw_ring\n pointer and its txq_info (This avoid Rx load Tx data directly);\n\n In data plane, users can call 'rte_eth_rx_direct_rearm' to rearm Rx\n buffers and free Tx buffers at the same time (Currently it supports 1:1\n (rxq:txq) mapping:)\n-----------------------------------------------------------------------\n control plane:\n \trte_eth_txq_data_get(*txq_data);\n data plane:\n \tloop {\n \t\trte_eth_rx_direct_rearm(*txq_data){\n \t\t\tfor (i = 0; i <= 32; i++) {\n \t\t\t\trx.mbuf[i] = tx.mbuf[i];\n \t\t\t\tinitialize descs[i];\n \t\t\t}\n\t\t}\n\t\trte_eth_rx_burst;\n\t\trte_eth_tx_burst;\n \t}\n-----------------------------------------------------------------------\n2)The i40e driver is changed to do the direct re-arm of the receive\n side.\n3)L3fwd application is modified to enable direct rearm mode. Users can\n enable direct-rearm and map queues by input parameters.\n\nTesting status:\n1.The testing results for L3fwd are as follows:\n-------------------------------------------------------------------\nenabled direct rearm\n-------------------------------------------------------------------\nArm:\nN1SDP(neon path):\nwithout fast-free mode\t\twith fast-free mode\n\t+15.09%\t\t\t\t+4.2%\n\nAmpere Altra(neon path):\nwithout fast-free mode\t\twith fast-free mode\n\t+10.9%\t\t\t\t+14.6%\n-------------------------------------------------------------------\n\n2.The testing results for VPP-L3fwd are as follows:\n-------------------------------------------------------------------\nArm:\nN1SDP(neon path):\nwith direct re-arm mode enabled\n\t+4.5%\n\nAmpere Altra(neon path):\nwith direct re-arm mode enabled\n +6.5%\n-------------------------------------------------------------------\n\nReference:\n[1] https://store.nvidia.com/en-us/networking/store/product/MCX623105AN-CDAT/NVIDIAMCX623105ANCDATConnectX6DxENAdapterCard100GbECryptoDisabled/\n[2] https://www.intel.com/content/www/us/en/products/sku/192561/intel-ethernet-network-adapter-e810cqda1/specifications.html\n[3] https://www.broadcom.com/products/ethernet-connectivity/network-adapters/100gb-nic-ocp/n1100g\n\nV2:\n1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa)\n2. Add 'txq_data_get' API to get txq info for Rx (Konstantin)\n3. Use input parameter to enable direct rearm in l3fwd (Konstantin)\n4. Add condition detection for direct rearm API (Morten, Andrew Rybchenko)\n\nFeifei Wang (3):\n ethdev: add API for direct rearm mode\n net/i40e: enable direct rearm mode\n examples/l3fwd: enable direct rearm mode\n\n drivers/net/i40e/i40e_ethdev.c | 1 +\n drivers/net/i40e/i40e_ethdev.h | 2 +\n drivers/net/i40e/i40e_rxtx.c | 19 ++++++\n drivers/net/i40e/i40e_rxtx.h | 2 +\n drivers/net/i40e/i40e_rxtx_vec_neon.c | 93 ++++++++++++++++++++++++++\n examples/l3fwd/l3fwd.h | 12 ++++\n examples/l3fwd/l3fwd_lpm.c | 22 +++++++\n examples/l3fwd/main.c | 94 +++++++++++++++++++++++++-\n lib/ethdev/ethdev_driver.h | 9 +++\n lib/ethdev/ethdev_private.c | 1 +\n lib/ethdev/rte_ethdev.c | 37 +++++++++++\n lib/ethdev/rte_ethdev.h | 95 +++++++++++++++++++++++++++\n lib/ethdev/rte_ethdev_core.h | 5 ++\n lib/ethdev/version.map | 4 ++\n 14 files changed, 395 insertions(+), 1 deletion(-)" }