From patchwork Wed Nov 6 17:37:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62579 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 118DDA04AD; Wed, 6 Nov 2019 18:38:01 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9604B1D409; Wed, 6 Nov 2019 18:38:00 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 3E6D11D408 for ; Wed, 6 Nov 2019 18:37:58 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 6 Nov 2019 19:37:56 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA6HbuqP026585; Wed, 6 Nov 2019 19:37:56 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA6HbuaJ021130; Wed, 6 Nov 2019 17:37:56 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA6HbuWb021129; Wed, 6 Nov 2019 17:37:56 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Wed, 6 Nov 2019 17:37:34 +0000 Message-Id: <1573061873-20898-1-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v2 00/19] net/mlx5: implement extensive metadata feature X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The modern networks operate on the base of the packet switching approach, and in-network environment data are transmitted as the packets. Within the host besides the data, actually transmitted on the wire as packets, there might some out-of-band data helping to process packets. These data are named as metadata, exist on a per-packet basis and are attached to each packet as some extra dedicated storage (in meaning it besides the packet data itself). In the DPDK network data are represented as mbuf structure chains and go along the application/DPDK datapath. From the other side, DPDK provides Flow API to control the flow engine. Being precise, there are two kinds of metadata in the DPDK, the one is purely software metadata (as fields of mbuf - flags, packet types, data length, etc.), and the other is metadata within flow engine. In this scope, we cover the second type (flow engine metadata) only. The flow engine metadata is some extra data, supported on the per-packet basis and usually handled by hardware inside flow engine. Initially, there were proposed two metadata related actions: - RTE_FLOW_ACTION_TYPE_FLAG - RTE_FLOW_ACTION_TYPE_MARK These actions set the special flag in the packet metadata, MARK action stores some specified value in the metadata storage, and, on the packet receiving PMD puts the flag and value to the mbuf and applications can see the packet was threated inside flow engine according to the appropriate RTE flow(s). MARK and FLAG are like some kind of gateway to transfer some per-packet information from the flow engine to the application via receiving datapath. Also, there is the item of type RTE_FLOW_ITEM_TYPE_MARK provided. It allows us to extend the flow match pattern with the capability to match the metadata values set by MARK/FLAG actions on other flows. From the datapath point of view, the MARK and FLAG are related to the receiving side only. It would useful to have the same gateway on the transmitting side and there was the feature of type RTE_FLOW_ITEM_TYPE_META was proposed. The application can fill the field in mbuf and this value will be transferred to some field in the packet metadata inside the flow engine. It did not matter whether these metadata fields are shared because of MARK and META items belonged to different domains (receiving and transmitting) and could be vendor-specific. So far, so good, DPDK proposes some entities to control metadata inside the flow engine and gateways to exchange these values on a per-packet basis via datapath. As we can see, the MARK and META means are not symmetric, there is absent action which would allow us to set META value on the transmitting path. So, the action of type: - RTE_FLOW_ACTION_TYPE_SET_META is proposed. The next, applications raise the new requirements for packet metadata. The flow engines are getting more complex, internal switches are introduced, multiple ports might be supported within the same flow engine namespace. From the DPDK points of view, it means the packets might be sent on one eth_dev port and received on the other one, and the packet path inside the flow engine entirely belongs to the same hardware device. The simplest example is SR-IOV with PF, VFs and the representors. And there is a brilliant opportunity to provide some out-of-band channel to transfer some extra data from one port to another one, besides the packet data itself. And applications would like to use this opportunity. Improving the metadata definitions it is proposed to: - suppose MARK and META metadata fields not shared, dedicated - extend applying area for MARK and META items/actions for all flow engine domains - transmitting and receiving - allow MARK and META metadata to be preserved while crossing the flow domains (from transmit origin through flow database inside (E-)switch to receiving side domain), in simple words, to allow metadata to convey the packet thought entire flow engine space. Another new proposed feature is transient per-packet storage inside the flow engine. It might have a lot of use cases. For example, if there is VXLAN tunneled traffic and some flow performs VXLAN decapsulation and wishes to save information regarding the dropped header it could use this temporary transient storage. The tools to maintain this storage are traditional (for DPDK rte_flow API): - RTE_FLOW_ACTION_TYPE_SET_TAG - to set value - RTE_FLOW_ACTION_TYPE_SET_ITEM - to match on There are primary properties of the proposed storage: - the storage is presented as an array of 32-bit opaque values - the size of array (or even bitmap of available indices) is vendor specific and is subject to run-time trial - it is transient, it means it exists only inside flow engine, no gateways for interacting with datapath, applications have way neither to specify these data on transmitting nor to get these data on receiving This patchset implements the abovementioned extensive metadata feature in the mlx5 PMD. The patchset must be applied after hashed list patch: [1] http://patches.dpdk.org/patch/62539/ Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- v2: - fix: metadata endianess - fix: infinite loop in header modify update routine - fix: reg_c_3 is reserved for split shared tag - fix: vport mask and value endianess - hash list implementation removed - rebased v1: http://patches.dpdk.org/cover/62419/ Viacheslav Ovsiienko (19): net/mlx5: convert internal tag endianness net/mlx5: update modify header action translator net/mlx5: add metadata register copy net/mlx5: refactor flow structure net/mlx5: update flow functions net/mlx5: update meta register matcher set net/mlx5: rename structure and function net/mlx5: check metadata registers availability net/mlx5: add devarg for extensive metadata support net/mlx5: adjust shared register according to mask net/mlx5: check the maximal modify actions number net/mlx5: update metadata register id query net/mlx5: add flow tag support net/mlx5: extend flow mark support net/mlx5: extend flow meta data support net/mlx5: add meta data support to Rx datapath net/mlx5: introduce flow splitters chain net/mlx5: split Rx flows to provide metadata copy net/mlx5: add metadata register copy table doc/guides/nics/mlx5.rst | 49 + drivers/net/mlx5/mlx5.c | 152 ++- drivers/net/mlx5/mlx5.h | 19 +- drivers/net/mlx5/mlx5_defs.h | 8 + drivers/net/mlx5/mlx5_ethdev.c | 8 +- drivers/net/mlx5/mlx5_flow.c | 1201 ++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 108 ++- drivers/net/mlx5/mlx5_flow_dv.c | 1566 ++++++++++++++++++++++++------ drivers/net/mlx5/mlx5_flow_verbs.c | 55 +- drivers/net/mlx5/mlx5_prm.h | 45 +- drivers/net/mlx5/mlx5_rxtx.c | 5 + drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 25 +- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 23 + drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 27 +- 14 files changed, 2868 insertions(+), 423 deletions(-)