Cover Detail
Show a cover letter.
GET /api/covers/62419/?format=api
{ "id": 62419, "url": "http://patches.dpdk.org/api/covers/62419/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/cover/1572940915-29416-1-git-send-email-viacheslavo@mellanox.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<1572940915-29416-1-git-send-email-viacheslavo@mellanox.com>", "list_archive_url": "https://inbox.dpdk.org/dev/1572940915-29416-1-git-send-email-viacheslavo@mellanox.com", "date": "2019-11-05T08:01:35", "name": "[00/20] net/mlx5: implement extensive metadata feature", "submitter": { "id": 1102, "url": "http://patches.dpdk.org/api/people/1102/?format=api", "name": "Slava Ovsiienko", "email": "viacheslavo@mellanox.com" }, "mbox": "http://patches.dpdk.org/project/dpdk/cover/1572940915-29416-1-git-send-email-viacheslavo@mellanox.com/mbox/", "series": [ { "id": 7242, "url": "http://patches.dpdk.org/api/series/7242/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=7242", "date": "2019-11-05T08:01:35", "name": "net/mlx5: implement extensive metadata feature", "version": 1, "mbox": "http://patches.dpdk.org/series/7242/mbox/" } ], "comments": "http://patches.dpdk.org/api/covers/62419/comments/", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from dpdk.org (dpdk.org [92.243.14.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id A4D9EA0352;\n\tTue, 5 Nov 2019 09:02:01 +0100 (CET)", "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 8ACA837B4;\n\tTue, 5 Nov 2019 09:02:01 +0100 (CET)", "from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129])\n by dpdk.org (Postfix) with ESMTP id 694643772\n for <dev@dpdk.org>; Tue, 5 Nov 2019 09:01:59 +0100 (CET)", "from Internal Mail-Server by MTLPINE1 (envelope-from\n viacheslavo@mellanox.com)\n with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:01:57 +0200", "from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx\n [10.210.16.104])\n by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA581veG026076;\n Tue, 5 Nov 2019 10:01:57 +0200", "from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1])\n by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA581vO3030749;\n Tue, 5 Nov 2019 08:01:57 GMT", "(from viacheslavo@localhost)\n by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA581vnK030748;\n Tue, 5 Nov 2019 08:01:57 GMT" ], "X-Authentication-Warning": "pegasus11.mtr.labs.mlnx: viacheslavo set sender to\n viacheslavo@mellanox.com using -f", "From": "Viacheslav Ovsiienko <viacheslavo@mellanox.com>", "To": "dev@dpdk.org", "Cc": "matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net,\n orika@mellanox.com, Yongseok Koh <yskoh@mellanox.com>", "Date": "Tue, 5 Nov 2019 08:01:35 +0000", "Message-Id": "<1572940915-29416-1-git-send-email-viacheslavo@mellanox.com>", "X-Mailer": "git-send-email 1.8.3.1", "Subject": "[dpdk-dev] [PATCH 00/20] net/mlx5: implement extensive metadata\n\tfeature", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "The modern networks operate on the base of the packet switching\napproach, and in-network environment data are transmitted as the\npackets. Within the host besides the data, actually transmitted\non the wire as packets, there might some out-of-band data helping\nto process packets. These data are named as metadata, exist on\na per-packet basis and are attached to each packet as some extra\ndedicated storage (in meaning it besides the packet data itself).\n\nIn the DPDK network data are represented as mbuf structure chains\nand go along the application/DPDK datapath. From the other side,\nDPDK provides Flow API to control the flow engine. Being precise,\nthere are two kinds of metadata in the DPDK, the one is purely\nsoftware metadata (as fields of mbuf - flags, packet types, data\nlength, etc.), and the other is metadata within flow engine.\nIn this scope, we cover the second type (flow engine metadata) only.\n\nThe flow engine metadata is some extra data, supported on the\nper-packet basis and usually handled by hardware inside flow\nengine.\n\nInitially, there were proposed two metadata related actions:\n\n- RTE_FLOW_ACTION_TYPE_FLAG\n- RTE_FLOW_ACTION_TYPE_MARK\n\nThese actions set the special flag in the packet metadata, MARK\naction stores some specified value in the metadata storage, and,\non the packet receiving PMD puts the flag and value to the mbuf\nand applications can see the packet was threated inside flow engine\naccording to the appropriate RTE flow(s). MARK and FLAG are like\nsome kind of gateway to transfer some per-packet information from\nthe flow engine to the application via receiving datapath. Also,\nthere is the item of type RTE_FLOW_ITEM_TYPE_MARK provided. It\nallows us to extend the flow match pattern with the capability\nto match the metadata values set by MARK/FLAG actions on other\nflows.\n\nFrom the datapath point of view, the MARK and FLAG are related\nto the receiving side only. It would useful to have the same gateway\non the transmitting side and there was the feature of type\nRTE_FLOW_ITEM_TYPE_META was proposed. The application can fill\nthe field in mbuf and this value will be transferred to some field\nin the packet metadata inside the flow engine.\n\nIt did not matter whether these metadata fields are shared because\nof MARK and META items belonged to different domains (receiving and\ntransmitting) and could be vendor-specific.\n\nSo far, so good, DPDK proposes some entities to control metadata\ninside the flow engine and gateways to exchange these values on\na per-packet basis via datapath.\n\nAs we can see, the MARK and META means are not symmetric, there\nis absent action which would allow us to set META value on the\ntransmitting path. So, the action of type:\n\n- RTE_FLOW_ACTION_TYPE_SET_META is proposed.\n\nThe next, applications raise the new requirements for packet\nmetadata. The flow engines are getting more complex, internal\nswitches are introduced, multiple ports might be supported within\nthe same flow engine namespace. From the DPDK points of view, it\nmeans the packets might be sent on one eth_dev port and received\non the other one, and the packet path inside the flow engine\nentirely belongs to the same hardware device. The simplest example\nis SR-IOV with PF, VFs and the representors. And there is a\nbrilliant opportunity to provide some out-of-band channel to\ntransfer some extra data from one port to another one, besides\nthe packet data itself. And applications would like to use this\nopportunity.\n\nImproving the metadata definitions it is proposed to:\n- suppose MARK and META metadata fields not shared, dedicated\n- extend applying area for MARK and META items/actions for all\n flow engine domains - transmitting and receiving\n - allow MARK and META metadata to be preserved while crossing\n the flow domains (from transmit origin through flow database\n inside (E-)switch to receiving side domain), in simple words,\n to allow metadata to convey the packet thought entire flow\n engine space.\n\nAnother new proposed feature is transient per-packet storage\ninside the flow engine. It might have a lot of use cases.\nFor example, if there is VXLAN tunneled traffic and some flow\nperforms VXLAN decapsulation and wishes to save information\nregarding the dropped header it could use this temporary\ntransient storage. The tools to maintain this storage are\ntraditional (for DPDK rte_flow API):\n\n- RTE_FLOW_ACTION_TYPE_SET_TAG - to set value\n- RTE_FLOW_ACTION_TYPE_SET_ITEM - to match on\n\nThere are primary properties of the proposed storage:\n- the storage is presented as an array of 32-bit opaque values\n- the size of array (or even bitmap of available indices) is\n vendor specific and is subject to run-time trial\n- it is transient, it means it exists only inside flow engine,\n no gateways for interacting with datapath, applications have\n way neither to specify these data on transmitting nor to get\n these data on receiving\n\nThis patchset implements the abovementioned extensive metadata\nfeature in the mlx5 PMD.\n\nThe patchset must be applied after public RTE API updates:\n\n[1] http://patches.dpdk.org/patch/62354/\n[2] http://patches.dpdk.org/patch/62355/\n\nSigned-off-by: Yongseok Koh <yskoh@mellanox.com>\nSigned-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>\n\nViacheslav Ovsiienko (20):\n net/mlx5: convert internal tag endianness\n net/mlx5: update modify header action translator\n net/mlx5: add metadata register copy\n net/mlx5: refactor flow structure\n net/mlx5: update flow functions\n net/mlx5: update meta register matcher set\n net/mlx5: rename structure and function\n net/mlx5: check metadata registers availability\n net/mlx5: add devarg for extensive metadata support\n net/mlx5: adjust shared register according to mask\n net/mlx5: check the maximal modify actions number\n net/mlx5: update metadata register id query\n net/mlx5: add flow tag support\n net/mlx5: extend flow mark support\n net/mlx5: extend flow meta data support\n net/mlx5: add meta data support to Rx datapath\n net/mlx5: add simple hash table\n net/mlx5: introduce flow splitters chain\n net/mlx5: split Rx flows to provide metadata copy\n net/mlx5: add metadata register copy table\n\n doc/guides/nics/mlx5.rst | 49 +\n drivers/net/mlx5/mlx5.c | 135 ++-\n drivers/net/mlx5/mlx5.h | 19 +-\n drivers/net/mlx5/mlx5_defs.h | 7 +\n drivers/net/mlx5/mlx5_ethdev.c | 8 +-\n drivers/net/mlx5/mlx5_flow.c | 1178 ++++++++++++++++++++++-\n drivers/net/mlx5/mlx5_flow.h | 108 ++-\n drivers/net/mlx5/mlx5_flow_dv.c | 1544 ++++++++++++++++++++++++------\n drivers/net/mlx5/mlx5_flow_verbs.c | 55 +-\n drivers/net/mlx5/mlx5_prm.h | 45 +-\n drivers/net/mlx5/mlx5_rxtx.c | 6 +\n drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 25 +-\n drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 23 +\n drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 27 +-\n drivers/net/mlx5/mlx5_utils.h | 115 ++-\n 15 files changed, 2922 insertions(+), 422 deletions(-)" }