Cover Detail
Show a cover letter.
GET /api/covers/68190/?format=api
{ "id": 68190, "url": "http://patches.dpdk.org/api/covers/68190/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/cover/20200411141428.1987768-1-jerinj@marvell.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20200411141428.1987768-1-jerinj@marvell.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20200411141428.1987768-1-jerinj@marvell.com", "date": "2020-04-11T14:13:59", "name": "[v5,00/29] graph: introduce graph subsystem", "submitter": { "id": 1188, "url": "http://patches.dpdk.org/api/people/1188/?format=api", "name": "Jerin Jacob Kollanukkaran", "email": "jerinj@marvell.com" }, "mbox": "http://patches.dpdk.org/project/dpdk/cover/20200411141428.1987768-1-jerinj@marvell.com/mbox/", "series": [ { "id": 9314, "url": "http://patches.dpdk.org/api/series/9314/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=9314", "date": "2020-04-11T14:13:59", "name": "graph: introduce graph subsystem", "version": 5, "mbox": "http://patches.dpdk.org/series/9314/mbox/" } ], "comments": "http://patches.dpdk.org/api/covers/68190/comments/", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from dpdk.org (dpdk.org [92.243.14.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 5B0ADA059F;\n\tSat, 11 Apr 2020 16:14:24 +0200 (CEST)", "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id DB6FE1C1A0;\n\tSat, 11 Apr 2020 16:14:23 +0200 (CEST)", "from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com\n [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 253451C19F\n for <dev@dpdk.org>; Sat, 11 Apr 2020 16:14:21 +0200 (CEST)", "from pps.filterd (m0045849.ppops.net [127.0.0.1])\n by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id\n 03BEA2NU016787; Sat, 11 Apr 2020 07:14:19 -0700", "from sc-exch04.marvell.com ([199.233.58.184])\n by mx0a-0016f401.pphosted.com with ESMTP id 30bb8q8n26-1\n (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT);\n Sat, 11 Apr 2020 07:14:18 -0700", "from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH04.marvell.com\n (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2;\n Sat, 11 Apr 2020 07:14:17 -0700", "from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com\n (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2;\n Sat, 11 Apr 2020 07:14:16 -0700", "from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com\n (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend\n Transport; Sat, 11 Apr 2020 07:14:16 -0700", "from jerin-lab.marvell.com (jerin-lab.marvell.com [10.28.34.14])\n by maili.marvell.com (Postfix) with ESMTP id A33E43F7045;\n Sat, 11 Apr 2020 07:14:13 -0700 (PDT)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;\n h=from : to : cc :\n subject : date : message-id : in-reply-to : references : mime-version :\n content-type : content-transfer-encoding; s=pfpt0818;\n bh=EkvNAKuLzXoWjqNGB3Kz/4kycY8MD4Ac0Ea/wX4EkyA=;\n b=cnIKlR+vq08mTUAKn9g33Hk0klltAZsyk/Kv+brcvTkqXkwMeTfOHdxYkLocyX3PuiW7\n Uf7udMk+p9z+9BxwFo7JAhe6Loz1+QpedHIiXBW0egRL65th6Mhd2ROdpOfFUCcZYiZT\n DybK1T+cI1Q5X43lB3tu/yEDPzVQY3L+fB1NJJQeOfU78u3ewz46ct+O46URU3wJpFkU\n 1t/Gmd7mna9wdLC2vrAo02LGb9rChenWd/h76CIvflH318SeKPo003/b/WdfTUOGh++U\n VAHu2GPjz/C7uqY5qfaPITdxKs4L3nnTRFLdo+5D9A+zcYX8Y7MXk3m4YNuUR6Vrhamr tg==", "From": "<jerinj@marvell.com>", "To": "", "CC": "<dev@dpdk.org>, <thomas@monjalon.net>, <david.marchand@redhat.com>,\n <mdr@ashroe.eu>, <mattias.ronnblom@ericsson.com>,\n <kirankumark@marvell.com>, <pbhagavatula@marvell.com>,\n <ndabilpuram@marvell.com>, <xiao.w.wang@intel.com>, <amo@semihalf.com>,\n Jerin Jacob <jerinj@marvell.com>", "Date": "Sat, 11 Apr 2020 19:43:59 +0530", "Message-ID": "<20200411141428.1987768-1-jerinj@marvell.com>", "X-Mailer": "git-send-email 2.25.1", "In-Reply-To": "<20200405085613.1336841-1-jerinj@marvell.com>", "References": "<20200405085613.1336841-1-jerinj@marvell.com>", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=\"UTF-8\"", "Content-Transfer-Encoding": "8bit", "X-Proofpoint-Virus-Version": "vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676\n definitions=2020-04-11_04:2020-04-09,\n 2020-04-11 signatures=0", "Subject": "[dpdk-dev] [PATCH v5 00/29] graph: introduce graph subsystem", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "From: Jerin Jacob <jerinj@marvell.com>\n\nUsing graph traversal for packet processing is a proven architecture\nthat has been implemented in various open source libraries.\n\nGraph architecture for packet processing enables abstracting the data\nprocessing functions as “nodes” and “links” them together to create a\ncomplex “graph” to create reusable/modular data processing functions. \n\nThe patchset further includes performance enhancements and modularity\nto the DPDK as discussed in more detail below.\n\nv5..v4:\n------\nAddressed the following review comments from Andrzej Ostruszka.\n\n1) Addressed and comment in (http://mails.dpdk.org/archives/dev/2020-April/162184.html)\nand improved following function prototypes/return types and adjusted the\nimplementation\na) rte_graph_node_get\nb) rte_graph_max_count\nc) rte_graph_export\nd) rte_graph_destroy\n2) Updated UT and l3fwd-graph for updated function prototype\n3) bug fix in edge_update\n4) avoid reading graph_src_nodes_count() twice in rte_graph_create()\n5) Fix graph_mem_fixup_secondray typo\n6) Fixed Doxygen comments for rte_node_next_stream_put\n7) Updated the documentation to reflect the same.\n8) Removed RTE prefix from rte_node_mbuf_priv[1|2] * as they are\ninternal defines\n9) Limited next_hop id provided to LPM route add in\nlibrte_node/ip4_lookup.c to 24 bits ()\n10) Fixed pattern array overflow issue with l3fwd-graph/main.c by\nsplitting pattern\narray to default + non-default array. Updated doc with the same info.\n11) Fixed parsing issues in parse_config() in l3fwd-graph/main.c inline\nwith issues reported\nin l2fwd-event\n12)Removed next_hop field in l3fwd-graph/main.c main()\n13) Fixed graph create error check in l3fwd-graph/main.c main()\n\nv4..v3:\n-------\nAddressed the following review comments from Wang, Xiao W\n\n1) Remove unnecessary line from rte_graph.h\n2) Fix a typo from rte_graph.h\n3) Move NODE_ID_CHECK to 3rd patch where it is first used.\n4) Fixed bug in edge_update()\n\nv3..v2:\n-------\n1) refactor ipv4 node lookup by moving SSE and NEON specific code to\nlib/librte_node/ip4_lookup_sse.h and lib/librte_node/ip4_lookup_neon.h\n2) Add scalar version of process() function for ipv4 lookup to make\nthe node work on NON x86 and arm64 machines.\n\nv2..v1:\n------\n1) Added programmer guide/implementation documentation and l3fwd-graph doc\n\nRFC..v1:\n--------\n\n1) Split the patch to more logical ones for review.\n2) Added doxygen comments for the API\n3) Code cleanup\n4) Additional performance improvements.\nDelta between l3fwd and l3fwd-graph is negligible now.\n(~1%) on octeontx2.\n5) Added SIMD routines for x86 in additional to arm64.\n\nHosted in netlify for easy reference:\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nProgrammer’s Guide:\nhttps://dpdk-graph.netlify.com/doc/html/guides/prog_guide/graph_lib.html\n\nl3fwd-graph doc:\nhttps://dpdk-graph.netlify.com/doc/html/guides/sample_app_ug/l3_forward_graph.html\n\nAPI doc:\nhttps://dpdk-graph.netlify.com/doc/html/api/rte__graph_8h.html\nhttps://dpdk-graph.netlify.com/doc/html/api/rte__graph__worker_8h.html\nhttps://dpdk-graph.netlify.com/doc/html/api/rte__node__eth__api_8h.html\nhttps://dpdk-graph.netlify.com/doc/html/api/rte__node__ip4__api_8h.html\n\n2) Added the release notes for the this feature\n\n3) Fix build issues reported by CI for v1:\nhttp://mails.dpdk.org/archives/test-report/2020-March/121326.html\n\n\nAddional nodes planned for v20.08\n----------------------------------\n1) Packet classification node\n2) Support for IPV6 LPM node\n\n\nThis patchset contains\n-----------------------------\n1) The API definition to \"create\" nodes and \"link\" together to create a\n\"graph\" for packet processing. See, lib/librte_graph/rte_graph.h \n\n2) The Fast path API definition for the graph walker and enqueue\nfunction used by the workers. See, lib/librte_graph/rte_graph_worker.h\n\n3) Optimized SW implementation for (1) and (2). See, lib/librte_graph/\n\n4) Test case to verify the graph infrastructure functionality\nSee, app/test/test_graph.c\n \n5) Performance test cases to evaluate the cost of graph walker and nodes\nenqueue fast-path function for various combinations.\n\nSee app/test/test_graph_perf.c\n\n6) Packet processing nodes(Null, Rx, Tx, Pkt drop, IPV4 rewrite, IPv4\nlookup)\nusing graph infrastructure. See lib/librte_node/*\n\n7) An example application to showcase l3fwd\n(functionality same as existing examples/l3fwd) using graph\ninfrastructure and use packets processing nodes (item (6)). See examples/l3fwd-graph/.\n\nPerformance\n-----------\n1) Graph walk and node enqueue overhead can be tested with performance\ntest case application [1]\n# If all packets go from a node to another node (we call it as\n# \"homerun\") then it will be just a pointer swap for a burst of packets.\n# In the worst case, a couple of handful cycles to move an object from a\nnode to another node.\n\n2) Performance comparison with existing l3fwd (The complete static code\nwith out any nodes) vs modular l3fwd-graph with 5 nodes\n(ip4_lookup, ip4_rewrite, ethdev_tx, ethdev_rx, pkt_drop).\nHere is graphical representation of the l3fwd-graph as Graphviz dot\nfile: \nhttp://bit.ly/39UPPGm\n\n# l3fwd-graph performance is -1.2% wrt static l3fwd.\n\n# We have simulated the similar test with existing librte_pipeline\n# application [4].\nip_pipline application is -48.62% wrt static l3fwd.\n\nThe above results are on octeontx2. It may vary on other platforms.\nThe platforms with higher L1 and L2 caches will have further better\nperformance.\n\n\nTested architectures:\n--------------------\n1) AArch64\n2) X86\n\n\nIdentified tweaking for better performance on different targets\n---------------------------------------------------------------\n1) Test with various burst size values (256, 128, 64, 32) using\nCONFIG_RTE_GRAPH_BURST_SIZE config option.\nBased on our testing, on x86 and arm64 servers, The sweet spot is 256\nburst size.\nWhile on arm64 embedded SoCs, it is either 64 or 128.\n\n2) Disable node statistics (use CONFIG_RTE_LIBRTE_GRAPH_STATS config\noption)\nif not needed.\n\n3) Use arm64 optimized memory copy for arm64 architecture by\nselecting CONFIG_RTE_ARCH_ARM64_MEMCPY. \n\nCommands to run tests\n---------------------\n\n[1] \nperf test:\necho \"graph_perf_autotest\" | sudo ./build/app/test/dpdk-test -c 0x30\n\n[2]\nfunctionality test:\necho \"graph_autotest\" | sudo ./build/app/test/dpdk-test -c 0x30\n\n[3]\nl3fwd-graph:\n./l3fwd-graph -c 0x100 -- -p 0x3 --config=\"(0, 0, 8)\" -P\n\n[4]\n# ./ip_pipeline --c 0xff0000 -- -s route.cli\n\nRoute.cli: (Copy paste to the shell to avoid dos format issues)\n\nhttps://pastebin.com/raw/B4Ktx7TT\n\nJerin Jacob (13):\n graph: define the public API for graph support\n graph: implement node registration\n graph: implement node operations\n graph: implement node debug routines\n graph: implement internal graph operation helpers\n graph: populate fastpath memory for graph reel\n graph: implement create and destroy APIs\n graph: implement graph operation APIs\n graph: implement Graphviz export\n graph: implement debug routines\n graph: implement stats support\n graph: implement fastpath API routines\n doc: add graph library programmer's guide guide\n\nKiran Kumar K (2):\n graph: add unit test case\n node: add ipv4 rewrite node\n\nNithin Dabilpuram (11):\n node: add log infra and null node\n node: add ethdev Rx node\n node: add ethdev Tx node\n node: add ethdev Rx and Tx node ctrl API\n node: ipv4 lookup for arm64\n node: add ipv4 rewrite and lookup ctrl API\n node: add packet drop node\n l3fwd-graph: add graph based l3fwd skeleton\n l3fwd-graph: add ethdev configuration changes\n l3fwd-graph: add graph config and main loop\n doc: add l3fwd graph application user guide\n\nPavan Nikhilesh (3):\n graph: add performance testcase\n node: add generic ipv4 lookup node\n node: ipv4 lookup for x86\n\n MAINTAINERS | 14 +\n app/test/Makefile | 7 +\n app/test/meson.build | 12 +-\n app/test/test_graph.c | 819 ++++\n app/test/test_graph_perf.c | 1057 ++++++\n config/common_base | 12 +\n config/rte_config.h | 4 +\n doc/api/doxy-api-index.md | 5 +\n doc/api/doxy-api.conf.in | 2 +\n doc/guides/prog_guide/graph_lib.rst | 397 ++\n .../prog_guide/img/anatomy_of_a_node.svg | 1078 ++++++\n .../prog_guide/img/graph_mem_layout.svg | 702 ++++\n doc/guides/prog_guide/img/link_the_nodes.svg | 3330 +++++++++++++++++\n doc/guides/prog_guide/index.rst | 1 +\n doc/guides/rel_notes/release_20_05.rst | 32 +\n doc/guides/sample_app_ug/index.rst | 1 +\n doc/guides/sample_app_ug/intro.rst | 4 +\n doc/guides/sample_app_ug/l3_forward_graph.rst | 334 ++\n examples/Makefile | 3 +\n examples/l3fwd-graph/Makefile | 58 +\n examples/l3fwd-graph/main.c | 1126 ++++++\n examples/l3fwd-graph/meson.build | 13 +\n examples/meson.build | 6 +-\n lib/Makefile | 6 +\n lib/librte_graph/Makefile | 28 +\n lib/librte_graph/graph.c | 587 +++\n lib/librte_graph/graph_debug.c | 84 +\n lib/librte_graph/graph_ops.c | 169 +\n lib/librte_graph/graph_populate.c | 234 ++\n lib/librte_graph/graph_private.h | 347 ++\n lib/librte_graph/graph_stats.c | 406 ++\n lib/librte_graph/meson.build | 11 +\n lib/librte_graph/node.c | 421 +++\n lib/librte_graph/rte_graph.h | 668 ++++\n lib/librte_graph/rte_graph_version.map | 47 +\n lib/librte_graph/rte_graph_worker.h | 510 +++\n lib/librte_node/Makefile | 32 +\n lib/librte_node/ethdev_ctrl.c | 115 +\n lib/librte_node/ethdev_rx.c | 221 ++\n lib/librte_node/ethdev_rx_priv.h | 81 +\n lib/librte_node/ethdev_tx.c | 86 +\n lib/librte_node/ethdev_tx_priv.h | 62 +\n lib/librte_node/ip4_lookup.c | 215 ++\n lib/librte_node/ip4_lookup_neon.h | 238 ++\n lib/librte_node/ip4_lookup_sse.h | 244 ++\n lib/librte_node/ip4_rewrite.c | 326 ++\n lib/librte_node/ip4_rewrite_priv.h | 77 +\n lib/librte_node/log.c | 14 +\n lib/librte_node/meson.build | 10 +\n lib/librte_node/node_private.h | 79 +\n lib/librte_node/null.c | 23 +\n lib/librte_node/pkt_drop.c | 26 +\n lib/librte_node/rte_node_eth_api.h | 64 +\n lib/librte_node/rte_node_ip4_api.h | 78 +\n lib/librte_node/rte_node_version.map | 9 +\n lib/meson.build | 5 +-\n meson.build | 1 +\n mk/rte.app.mk | 2 +\n 58 files changed, 14538 insertions(+), 5 deletions(-)\n create mode 100644 app/test/test_graph.c\n create mode 100644 app/test/test_graph_perf.c\n create mode 100644 doc/guides/prog_guide/graph_lib.rst\n create mode 100644 doc/guides/prog_guide/img/anatomy_of_a_node.svg\n create mode 100644 doc/guides/prog_guide/img/graph_mem_layout.svg\n create mode 100644 doc/guides/prog_guide/img/link_the_nodes.svg\n create mode 100644 doc/guides/sample_app_ug/l3_forward_graph.rst\n create mode 100644 examples/l3fwd-graph/Makefile\n create mode 100644 examples/l3fwd-graph/main.c\n create mode 100644 examples/l3fwd-graph/meson.build\n create mode 100644 lib/librte_graph/Makefile\n create mode 100644 lib/librte_graph/graph.c\n create mode 100644 lib/librte_graph/graph_debug.c\n create mode 100644 lib/librte_graph/graph_ops.c\n create mode 100644 lib/librte_graph/graph_populate.c\n create mode 100644 lib/librte_graph/graph_private.h\n create mode 100644 lib/librte_graph/graph_stats.c\n create mode 100644 lib/librte_graph/meson.build\n create mode 100644 lib/librte_graph/node.c\n create mode 100644 lib/librte_graph/rte_graph.h\n create mode 100644 lib/librte_graph/rte_graph_version.map\n create mode 100644 lib/librte_graph/rte_graph_worker.h\n create mode 100644 lib/librte_node/Makefile\n create mode 100644 lib/librte_node/ethdev_ctrl.c\n create mode 100644 lib/librte_node/ethdev_rx.c\n create mode 100644 lib/librte_node/ethdev_rx_priv.h\n create mode 100644 lib/librte_node/ethdev_tx.c\n create mode 100644 lib/librte_node/ethdev_tx_priv.h\n create mode 100644 lib/librte_node/ip4_lookup.c\n create mode 100644 lib/librte_node/ip4_lookup_neon.h\n create mode 100644 lib/librte_node/ip4_lookup_sse.h\n create mode 100644 lib/librte_node/ip4_rewrite.c\n create mode 100644 lib/librte_node/ip4_rewrite_priv.h\n create mode 100644 lib/librte_node/log.c\n create mode 100644 lib/librte_node/meson.build\n create mode 100644 lib/librte_node/node_private.h\n create mode 100644 lib/librte_node/null.c\n create mode 100644 lib/librte_node/pkt_drop.c\n create mode 100644 lib/librte_node/rte_node_eth_api.h\n create mode 100644 lib/librte_node/rte_node_ip4_api.h\n create mode 100644 lib/librte_node/rte_node_version.map" }