Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/26089/?format=api
{ "id": 26089, "url": "https://patches.dpdk.org/api/patches/26089/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/1498826274-1769-1-git-send-email-wei.guo.simon@gmail.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<1498826274-1769-1-git-send-email-wei.guo.simon@gmail.com>", "list_archive_url": "https://inbox.dpdk.org/dev/1498826274-1769-1-git-send-email-wei.guo.simon@gmail.com", "date": "2017-06-30T12:37:54", "name": "[dpdk-dev,RFC] app/testpmd: add option ring-bind-lcpu to bind Q with CPU", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "7a5a10828eb1070c682ca247c2807a5fd9f3cfbc", "submitter": { "id": 784, "url": "https://patches.dpdk.org/api/people/784/?format=api", "name": null, "email": "wei.guo.simon@gmail.com" }, "delegate": null, "mbox": "https://patches.dpdk.org/project/dpdk/patch/1498826274-1769-1-git-send-email-wei.guo.simon@gmail.com/mbox/", "series": [], "comments": "https://patches.dpdk.org/api/patches/26089/comments/", "check": "warning", "checks": "https://patches.dpdk.org/api/patches/26089/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 96BF14CC5;\n\tFri, 30 Jun 2017 14:38:11 +0200 (CEST)", "from mail-pf0-f193.google.com (mail-pf0-f193.google.com\n\t[209.85.192.193]) by dpdk.org (Postfix) with ESMTP id C64D7235\n\tfor <dev@dpdk.org>; Fri, 30 Jun 2017 14:38:09 +0200 (CEST)", "by mail-pf0-f193.google.com with SMTP id z6so16968172pfk.3\n\tfor <dev@dpdk.org>; Fri, 30 Jun 2017 05:38:09 -0700 (PDT)", "from simonLocalRHEL7.x64.cn.ibm.com ([116.230.182.250])\n\tby smtp.gmail.com with ESMTPSA id\n\t188sm13538617pgc.49.2017.06.30.05.38.07\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tFri, 30 Jun 2017 05:38:08 -0700 (PDT)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=from:to:cc:subject:date:message-id;\n\tbh=MSGhQMQs+FR2MV5q84NRiK1ljAXkhez9UwdcptWpSrE=;\n\tb=tvyQVwp2bPrq00dbNo/9ShgLTQH1Vpn8Sh9uiPbojFLuK+Uu2UaMPRnGpap/HdUiIs\n\tnsxEmvTzXPhs9NrhRQWZwkQaFI9VHNHCD2LanjdHkhZoEjBRbOuhtlvmdm+U9gsiE92s\n\tv+obL6IC5E7/V8Yodg5p5DOKrmSHDnK2jCcu6WXExMOoGhVk8BgABJxXb7bl+okO14bK\n\tD4rehI3B1sXk6WI5FhNmX7qz+AJfh5+0PQAdo7+8ryDBzc4YSYse4cVgPXmIhT2PRW/O\n\t90PT4uTvNCaGejQokHyBt/8mwQ0Pru6VQ1Kc2UR0BPlLoTCEe9pXAKd/ffrufw7PB+cM\n\tE+6g==", "X-Google-DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id;\n\tbh=MSGhQMQs+FR2MV5q84NRiK1ljAXkhez9UwdcptWpSrE=;\n\tb=EOSEKCplwYTWUy3q0bTXrDsdoBytEa/FeBKUqeMYnzj/FD/zhXovNh119fSQaE5hi4\n\teVCB8XMqFtTA9u7R5gH8+B4p7Ung6VJc2FAuYxmdwUZfOEshHCccarOoipXiAeyh4Arz\n\tXWcX6lwFdduSs2jPEuautuQiI8m2b/uRROMS1+9eYIwmmuJUW3yZt2q3991chqgLs4fM\n\tfajkwLonN1e5aFUcoC6GfwhvHV/QCYA58iGZ1sACxGTlIahLYkKzzHMs8K1akIcok3fa\n\tVA6J1DQTgNpSvLoJYn1ZpFmnHaSdsOBK9fqbzmX49nsIALRfCS8egm2mvRu8hpb4NICU\n\ts2BQ==", "X-Gm-Message-State": "AKS2vOxcOiXxTNa7jY1xAP0nYARy3gfSdWNl4Yl+e+XmmUiJ19Vpmt61\n\tA5KaDV+ul14ZZA==", "X-Received": "by 10.84.131.163 with SMTP id d32mr24181579pld.16.1498826288965; \n\tFri, 30 Jun 2017 05:38:08 -0700 (PDT)", "From": "wei.guo.simon@gmail.com", "To": "Jingjing Wu <jingjing.wu@intel.com>", "Cc": "dev@dpdk.org,\n\tSimon Guo <wei.guo.simon@gmail.com>", "Date": "Fri, 30 Jun 2017 20:37:54 +0800", "Message-Id": "<1498826274-1769-1-git-send-email-wei.guo.simon@gmail.com>", "X-Mailer": "git-send-email 1.8.3.1", "Subject": "[dpdk-dev] [RFC PATCH] app/testpmd: add option ring-bind-lcpu to\n\tbind Q with CPU", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "From: Simon Guo <wei.guo.simon@gmail.com>\n\nCurrently the rx/tx queue is allocated from the buffer pool on socket of:\n- port's socket if --port-numa-config specified\n- or ring-numa-config setting per port\n\nAll the above will \"bind\" queue to single socket per port configuration.\nBut it can actually archieve better performance if one port's queue can\nbe spread across multiple NUMA nodes, and the rx/tx queue is allocated\nper lcpu socket.\n\nWith this patch, testpmd can utilize the PCI-e bus bandwidth on another\nNUMA nodes. with 64bytes package, When running in PowerPC with Mellanox\nCX-4 card, single port(100G), with 8 cores, fw mode:\n- Without this patch: 52.5Mpps throughput\n- With this patch: 66Mpps throughput\n\nSigned-off-by: Simon Guo <wei.guo.simon@gmail.com>\n---\n app/test-pmd/parameters.c | 10 +++++\n app/test-pmd/testpmd.c | 102 ++++++++++++++++++++++++++++++++--------------\n app/test-pmd/testpmd.h | 1 +\n 3 files changed, 82 insertions(+), 31 deletions(-)", "diff": "diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c\nindex fbe6284..d3ab946 100644\n--- a/app/test-pmd/parameters.c\n+++ b/app/test-pmd/parameters.c\n@@ -130,6 +130,11 @@\n \t \"(flag: 1 for RX; 2 for TX; 3 for RX and TX).\\n\");\n \tprintf(\" --socket-num=N: set socket from which all memory is allocated \"\n \t \"in NUMA mode.\\n\");\n+\tprintf(\" --ring-bind-lcpu: \"\n+\t\t\" specify TX/RX rings will be allocated on local socket of lcpu.\"\n+\t\t\" It will overriden ring-numa-config or port-numa-config if success.\"\n+\t\t\" If local ring buffer is unavailable, it will use --ring-numa-config or port-numa-config instead.\"\n+\t\t\" It allows one port binds to multiple NUMA nodes. \\n\");\n \tprintf(\" --mbuf-size=N: set the data size of mbuf to N bytes.\\n\");\n \tprintf(\" --total-num-mbufs=N: set the number of mbufs to be allocated \"\n \t \"in mbuf pools.\\n\");\n@@ -563,6 +568,7 @@\n \t\t{ \"interactive\",\t\t0, 0, 0 },\n \t\t{ \"cmdline-file\",\t\t1, 0, 0 },\n \t\t{ \"auto-start\",\t\t\t0, 0, 0 },\n+\t\t{ \"ring-bind-lcpu\",\t\t0, 0, 0 },\n \t\t{ \"eth-peers-configfile\",\t1, 0, 0 },\n \t\t{ \"eth-peer\",\t\t\t1, 0, 0 },\n #endif\n@@ -674,6 +680,10 @@\n \t\t\t\tprintf(\"Auto-start selected\\n\");\n \t\t\t\tauto_start = 1;\n \t\t\t}\n+\t\t\tif (!strcmp(lgopts[opt_idx].name, \"ring-bind-lcpu\")) {\n+\t\t\t\tprintf(\"RingBuffer bind with local CPU selected\\n\");\n+\t\t\t\tring_bind_lcpu = 1;\n+\t\t\t}\n \t\t\tif (!strcmp(lgopts[opt_idx].name,\n \t\t\t\t \"eth-peers-configfile\")) {\n \t\t\t\tif (init_peer_eth_addrs(optarg) != 0)\ndiff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c\nindex b29328a..92eed8f 100644\n--- a/app/test-pmd/testpmd.c\n+++ b/app/test-pmd/testpmd.c\n@@ -98,6 +98,7 @@\n /* use master core for command line ? */\n uint8_t interactive = 0;\n uint8_t auto_start = 0;\n+uint8_t ring_bind_lcpu = 0;\n char cmdline_filename[PATH_MAX] = {0};\n \n /*\n@@ -1395,6 +1396,46 @@ static void eth_event_callback(uint8_t port_id,\n \treturn 1;\n }\n \n+static int find_local_socket(queueid_t qi, int is_rxq)\n+{\n+\t/*\n+\t * try to find the local mp with following logic:\n+\t * 1) Find the correct stream for the queue;\n+\t * 2) Find the correct lcore for the stream;\n+\t * 3) Find the correct socket for the lcore;\n+\t * 4) Find the correct mp for the scoket;\n+\t *\n+\t * If failed, failover to the old implementation.\n+\t */\n+\tint i, j, socket = NUMA_NO_CONFIG;\n+\n+\t/* find the stream based on queue no*/\n+\tfor (i = 0; i < nb_fwd_streams; i++) {\n+\t\tif (is_rxq) {\n+\t\t\tif (fwd_streams[i]->rx_queue == qi)\n+\t\t\t\tbreak;\n+\t\t} else {\n+\t\t\tif (fwd_streams[i]->tx_queue == qi)\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+\tif (i == nb_fwd_streams)\n+\t\treturn NUMA_NO_CONFIG;\n+\n+\t/* find the lcore based on stream idx */\n+\tfor (j = 0; j < nb_lcores; j++) {\n+\t\tif (fwd_lcores[j]->stream_idx == i)\n+\t\t\tbreak;\n+\t}\n+\tif (j == nb_lcores)\n+\t\treturn NUMA_NO_CONFIG;\n+\n+\t/* find the scoket for the lcore */\n+\tsocket = rte_lcore_to_socket_id(fwd_lcores_cpuids[j]);\n+\n+\treturn socket;\n+}\n+\n int\n start_port(portid_t pid)\n {\n@@ -1445,14 +1486,19 @@ static void eth_event_callback(uint8_t port_id,\n \t\t\tport->need_reconfig_queues = 0;\n \t\t\t/* setup tx queues */\n \t\t\tfor (qi = 0; qi < nb_txq; qi++) {\n+ int socket = port->socket_id;\n \t\t\t\tif ((numa_support) &&\n \t\t\t\t\t(txring_numa[pi] != NUMA_NO_CONFIG))\n-\t\t\t\t\tdiag = rte_eth_tx_queue_setup(pi, qi,\n-\t\t\t\t\t\tnb_txd,txring_numa[pi],\n-\t\t\t\t\t\t&(port->tx_conf));\n-\t\t\t\telse\n-\t\t\t\t\tdiag = rte_eth_tx_queue_setup(pi, qi,\n-\t\t\t\t\t\tnb_txd,port->socket_id,\n+ socket = txring_numa[pi];\n+\n+\t\t\t\tif (ring_bind_lcpu) {\n+\t\t\t\t\tint ret = find_local_socket(qi, 0);\n+\t\t\t\t\tif (ret != NUMA_NO_CONFIG)\n+\t\t\t\t\t\tsocket = ret;\n+\t\t\t\t}\n+\n+\t\t\t\tdiag = rte_eth_tx_queue_setup(pi, qi,\n+\t\t\t\t\t\tnb_txd, socket,\n \t\t\t\t\t\t&(port->tx_conf));\n \n \t\t\t\tif (diag == 0)\n@@ -1471,35 +1517,29 @@ static void eth_event_callback(uint8_t port_id,\n \t\t\t}\n \t\t\t/* setup rx queues */\n \t\t\tfor (qi = 0; qi < nb_rxq; qi++) {\n+\t\t\t\tint socket = port->socket_id;\n \t\t\t\tif ((numa_support) &&\n-\t\t\t\t\t(rxring_numa[pi] != NUMA_NO_CONFIG)) {\n-\t\t\t\t\tstruct rte_mempool * mp =\n-\t\t\t\t\t\tmbuf_pool_find(rxring_numa[pi]);\n-\t\t\t\t\tif (mp == NULL) {\n-\t\t\t\t\t\tprintf(\"Failed to setup RX queue:\"\n-\t\t\t\t\t\t\t\"No mempool allocation\"\n-\t\t\t\t\t\t\t\" on the socket %d\\n\",\n-\t\t\t\t\t\t\trxring_numa[pi]);\n-\t\t\t\t\t\treturn -1;\n-\t\t\t\t\t}\n-\n-\t\t\t\t\tdiag = rte_eth_rx_queue_setup(pi, qi,\n-\t\t\t\t\t nb_rxd,rxring_numa[pi],\n-\t\t\t\t\t &(port->rx_conf),mp);\n-\t\t\t\t} else {\n-\t\t\t\t\tstruct rte_mempool *mp =\n-\t\t\t\t\t\tmbuf_pool_find(port->socket_id);\n-\t\t\t\t\tif (mp == NULL) {\n-\t\t\t\t\t\tprintf(\"Failed to setup RX queue:\"\n+\t\t\t\t\t\t(rxring_numa[pi] != NUMA_NO_CONFIG))\n+\t\t\t\t\tsocket = rxring_numa[pi];\n+\n+\t\t\t\tif (ring_bind_lcpu) {\n+\t\t\t\t\tint ret = find_local_socket(qi, 1);\n+\t\t\t\t\tif (ret != NUMA_NO_CONFIG)\n+\t\t\t\t\t\tsocket = ret;\n+\t\t\t\t}\n+\n+\t\t\t\tstruct rte_mempool *mp =\n+\t\t\t\t\tmbuf_pool_find(socket);\n+\t\t\t\tif (mp == NULL) {\n+\t\t\t\t\tprintf(\"Failed to setup RX queue:\"\n \t\t\t\t\t\t\t\"No mempool allocation\"\n \t\t\t\t\t\t\t\" on the socket %d\\n\",\n-\t\t\t\t\t\t\tport->socket_id);\n-\t\t\t\t\t\treturn -1;\n-\t\t\t\t\t}\n-\t\t\t\t\tdiag = rte_eth_rx_queue_setup(pi, qi,\n-\t\t\t\t\t nb_rxd,port->socket_id,\n-\t\t\t\t\t &(port->rx_conf), mp);\n+\t\t\t\t\t\t\tsocket);\n+\t\t\t\t\treturn -1;\n \t\t\t\t}\n+\t\t\t\tdiag = rte_eth_rx_queue_setup(pi, qi,\n+\t\t\t\t\t\tnb_rxd,socket,\n+\t\t\t\t\t\t&(port->rx_conf), mp);\n \t\t\t\tif (diag == 0)\n \t\t\t\t\tcontinue;\n \ndiff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h\nindex 364502d..c45b2b8 100644\n--- a/app/test-pmd/testpmd.h\n+++ b/app/test-pmd/testpmd.h\n@@ -299,6 +299,7 @@ struct queue_stats_mappings {\n extern uint16_t verbose_level; /**< Drives messages being displayed, if any. */\n extern uint8_t interactive;\n extern uint8_t auto_start;\n+extern uint8_t ring_bind_lcpu;\n extern char cmdline_filename[PATH_MAX]; /**< offline commands file */\n extern uint8_t numa_support; /**< set by \"--numa\" parameter */\n extern uint16_t port_topology; /**< set by \"--port-topology\" parameter */\n", "prefixes": [ "dpdk-dev", "RFC" ] }