Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/56095/?format=api
{ "id": 56095, "url": "http://patches.dpdk.org/api/patches/56095/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/patch/1562257767-19035-6-git-send-email-viacheslavo@mellanox.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<1562257767-19035-6-git-send-email-viacheslavo@mellanox.com>", "list_archive_url": "https://inbox.dpdk.org/dev/1562257767-19035-6-git-send-email-viacheslavo@mellanox.com", "date": "2019-07-04T16:29:25", "name": "[5/7] net/mlx5: introduce Tx burst routine template", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "d1b13c847703ef062635bb3de21e809fbc70e4c2", "submitter": { "id": 1102, "url": "http://patches.dpdk.org/api/people/1102/?format=api", "name": "Slava Ovsiienko", "email": "viacheslavo@mellanox.com" }, "delegate": { "id": 3268, "url": "http://patches.dpdk.org/api/users/3268/?format=api", "username": "rasland", "first_name": "Raslan", "last_name": "Darawsheh", "email": "rasland@nvidia.com" }, "mbox": "http://patches.dpdk.org/project/dpdk/patch/1562257767-19035-6-git-send-email-viacheslavo@mellanox.com/mbox/", "series": [ { "id": 5342, "url": "http://patches.dpdk.org/api/series/5342/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=5342", "date": "2019-07-04T16:29:20", "name": "net/mlx5: consolidate Tx datapath", "version": 1, "mbox": "http://patches.dpdk.org/series/5342/mbox/" } ], "comments": "http://patches.dpdk.org/api/patches/56095/comments/", "check": "success", "checks": "http://patches.dpdk.org/api/patches/56095/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id B24B131FC;\n\tThu, 4 Jul 2019 18:30:13 +0200 (CEST)", "from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129])\n\tby dpdk.org (Postfix) with ESMTP id 1365B3195\n\tfor <dev@dpdk.org>; Thu, 4 Jul 2019 18:30:12 +0200 (CEST)", "from Internal Mail-Server by MTLPINE2 (envelope-from\n\tviacheslavo@mellanox.com)\n\twith ESMTPS (AES256-SHA encrypted); 4 Jul 2019 19:30:09 +0300", "from pegasus12.mtr.labs.mlnx. (pegasus12.mtr.labs.mlnx\n\t[10.210.17.40])\n\tby labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x64GTYPq002908;\n\tThu, 4 Jul 2019 19:30:09 +0300" ], "From": "Viacheslav Ovsiienko <viacheslavo@mellanox.com>", "To": "dev@dpdk.org", "Cc": "yskoh@mellanox.com", "Date": "Thu, 4 Jul 2019 16:29:25 +0000", "Message-Id": "<1562257767-19035-6-git-send-email-viacheslavo@mellanox.com>", "X-Mailer": "git-send-email 1.8.3.1", "In-Reply-To": "<1562257767-19035-1-git-send-email-viacheslavo@mellanox.com>", "References": "<1562257767-19035-1-git-send-email-viacheslavo@mellanox.com>", "Subject": "[dpdk-dev] [PATCH 5/7] net/mlx5: introduce Tx burst routine template", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "Mellanox NICs support the wide set of Tx offloads. The supported\noffloads are reported by the mlx5 PMD in rte_eth_dev_info tx_offload_capa\nfield. An application may choose any combination of supported offloads\nand configure the device appropriately. Some of Tx offloads may be\nnot requested by application, or ever all of them may be omitted.\nMost of the Tx offloads require some code branches in tx_burst routine\nto support ones. If Tx offload is not requested the tx_burst routine\ncode may be significantly simplified and consume less CPU cycles.\n\nFor example, if application does not engage TSO offload this code\ncan be omitted, if multi-segment packet is not supposed the tx_burst\nmay assume single mbuf packets only, etc.\n\nCurrently, the mlx5 PMD implements multiple tx_burst subroutines\nfor most common combinations of requested Tx offloads, each branch\nhas its own dedicated implementation. It is not very easy to update,\nsupport and develop such kind of code - multiple branches impose\nthe multiple points to process. Also many of frequently requested\noffload combinations are not supported yet. That leads to selecting of\nnot completely matching tx_burst routine and harms the performance.\n\nThis patch introduces the new approach for tx_burst code. It is proposed\nto develop the unified template for tx_burst routine, which supports\nall the Tx offloads and takes the compile time defined parameter\ndescribing the supposed set of supported offloads. On the base\nof this template, the compiler is able to generate multiple tx_burst\nroutines highly optimized for the statically specified set of Tx offloads.\nNext, in runtime, at Tx queue configuration the best matching optimized\nimplementation of tx_burst is chosen.\n\nThis patch intentionally omits the template internal implementation,\nbut just introduces the template itself to emboss the approach of\nthe multiple specially tuned tx_burst routines.\n\nSigned-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>\n---\n drivers/net/mlx5/mlx5_rxtx.c | 511 ++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 507 insertions(+), 4 deletions(-)", "diff": "diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c\nindex 13f9431..af6f705 100644\n--- a/drivers/net/mlx5/mlx5_rxtx.c\n+++ b/drivers/net/mlx5/mlx5_rxtx.c\n@@ -1,6 +1,6 @@\n /* SPDX-License-Identifier: BSD-3-Clause\n * Copyright 2015 6WIND S.A.\n- * Copyright 2015 Mellanox Technologies, Ltd\n+ * Copyright 2015-2019 Mellanox Technologies, Ltd\n */\n \n #include <assert.h>\n@@ -34,6 +34,57 @@\n #include \"mlx5_defs.h\"\n #include \"mlx5_prm.h\"\n \n+/* TX burst subroutines return codes. */\n+enum mlx5_txcmp_code {\n+\tMLX5_TXCMP_CODE_EXIT = 0,\n+\tMLX5_TXCMP_CODE_ERROR,\n+\tMLX5_TXCMP_CODE_SINGLE,\n+\tMLX5_TXCMP_CODE_MULTI,\n+\tMLX5_TXCMP_CODE_TSO,\n+\tMLX5_TXCMP_CODE_EMPW,\n+};\n+\n+/*\n+ * These defines are used to configure Tx burst routine option set\n+ * supported at compile time. The not specified options are optimized out\n+ * out due to if conditions can be explicitly calculated at compile time.\n+ * The offloads with bigger runtime check (require more CPU cycles to\n+ * skip) overhead should have the bigger index - this is needed to\n+ * select the better matching routine function if no exact match and\n+ * some offloads are not actually requested.\n+ */\n+#define MLX5_TXOFF_CONFIG_MULTI (1u << 0) /* Multi-segment packets.*/\n+#define MLX5_TXOFF_CONFIG_TSO (1u << 1) /* TCP send offload supported.*/\n+#define MLX5_TXOFF_CONFIG_SWP (1u << 2) /* Tunnels/SW Parser offloads.*/\n+#define MLX5_TXOFF_CONFIG_CSUM (1u << 3) /* Check Sums offloaded. */\n+#define MLX5_TXOFF_CONFIG_INLINE (1u << 4) /* Data inlining supported. */\n+#define MLX5_TXOFF_CONFIG_VLAN (1u << 5) /* VLAN insertion supported.*/\n+#define MLX5_TXOFF_CONFIG_METADATA (1u << 6) /* Flow metadata. */\n+#define MLX5_TXOFF_CONFIG_EMPW (1u << 8) /* Enhanced MPW supported.*/\n+\n+/* The most common offloads groups. */\n+#define MLX5_TXOFF_CONFIG_NONE 0\n+#define MLX5_TXOFF_CONFIG_FULL (MLX5_TXOFF_CONFIG_MULTI | \\\n+\t\t\t\tMLX5_TXOFF_CONFIG_TSO | \\\n+\t\t\t\tMLX5_TXOFF_CONFIG_SWP | \\\n+\t\t\t\tMLX5_TXOFF_CONFIG_CSUM | \\\n+\t\t\t\tMLX5_TXOFF_CONFIG_INLINE | \\\n+\t\t\t\tMLX5_TXOFF_CONFIG_VLAN | \\\n+\t\t\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+#define MLX5_TXOFF_CONFIG(mask) (olx & MLX5_TXOFF_CONFIG_##mask)\n+\n+#define MLX5_TXOFF_DECL(func, olx) \\\n+static uint16_t mlx5_tx_burst_##func(void *txq, \\\n+\t\t\t\t struct rte_mbuf **pkts, \\\n+\t\t\t\t uint16_t pkts_n) \\\n+{ \\\n+\treturn mlx5_tx_burst_tmpl((struct mlx5_txq_data *restrict)txq, \\\n+\t\t pkts, pkts_n, (olx)); \\\n+}\n+\n+#define MLX5_TXOFF_INFO(func, olx) {mlx5_tx_burst_##func, olx},\n+\n static __rte_always_inline uint32_t\n rxq_cq_to_pkt_type(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cqe);\n \n@@ -1531,7 +1582,323 @@\n }\n \n /**\n- * Configure the TX function to use.\n+ * DPDK Tx callback template. This is configured template\n+ * used to generate routines optimized for specified offload setup.\n+ * One of this generated functions is chosen at SQ configuration\n+ * time.\n+ *\n+ * @param txq\n+ * Generic pointer to TX queue structure.\n+ * @param[in] pkts\n+ * Packets to transmit.\n+ * @param pkts_n\n+ * Number of packets in array.\n+ * @param olx\n+ * Configured offloads mask, presents the bits of MLX5_TXOFF_CONFIG_xxx\n+ * values. Should be static to take compile time static configuration\n+ * advantages.\n+ *\n+ * @return\n+ * Number of packets successfully transmitted (<= pkts_n).\n+ */\n+static __rte_always_inline uint16_t\n+mlx5_tx_burst_tmpl(struct mlx5_txq_data *restrict txq,\n+\t\t struct rte_mbuf **restrict pkts,\n+\t\t uint16_t pkts_n,\n+\t\t unsigned int olx)\n+{\n+\t(void)txq;\n+\t(void)pkts;\n+\t(void)pkts_n;\n+\t(void)olx;\n+\treturn 0;\n+}\n+\n+/* Generate routines with Enhanced Multi-Packet Write support. */\n+MLX5_TXOFF_DECL(full_empw,\n+\t\tMLX5_TXOFF_CONFIG_FULL | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(none_empw,\n+\t\tMLX5_TXOFF_CONFIG_NONE | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(md_empw,\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(mt_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(mtsc_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(mti_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(mtv_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(mtiv_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(sc_empw,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(sci_empw,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(scv_empw,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(sciv_empw,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(i_empw,\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(v_empw,\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_DECL(iv_empw,\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+/* Generate routines without Enhanced Multi-Packet Write support. */\n+MLX5_TXOFF_DECL(full,\n+\t\tMLX5_TXOFF_CONFIG_FULL)\n+\n+MLX5_TXOFF_DECL(none,\n+\t\tMLX5_TXOFF_CONFIG_NONE)\n+\n+MLX5_TXOFF_DECL(md,\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_DECL(mt,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_DECL(mtsc,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_DECL(mti,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+\n+MLX5_TXOFF_DECL(mtv,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+\n+MLX5_TXOFF_DECL(mtiv,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_DECL(sc,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_DECL(sci,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+\n+MLX5_TXOFF_DECL(scv,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+\n+MLX5_TXOFF_DECL(sciv,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_DECL(i,\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_DECL(v,\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_DECL(iv,\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+/*\n+ * Array of declared and compiled Tx burst function and corresponding\n+ * supported offloads set. The array is used to select the Tx burst\n+ * function for specified offloads set at Tx queue configuration time.\n+ */\n+const struct {\n+\teth_tx_burst_t func;\n+\tunsigned int olx;\n+} txoff_func[] = {\n+MLX5_TXOFF_INFO(full_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(none_empw,\n+\t\tMLX5_TXOFF_CONFIG_NONE | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(md_empw,\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(mt_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(mtsc_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(mti_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(mtv_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(mtiv_empw,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(sc_empw,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(sci_empw,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(scv_empw,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(sciv_empw,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(i_empw,\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(v_empw,\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(iv_empw,\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW)\n+\n+MLX5_TXOFF_INFO(full,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(none,\n+\t\tMLX5_TXOFF_CONFIG_NONE)\n+\n+MLX5_TXOFF_INFO(md,\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(mt,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(mtsc,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(mti,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+\n+MLX5_TXOFF_INFO(mtv,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(mtiv,\n+\t\tMLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(sc,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(sci,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(scv,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(sciv,\n+\t\tMLX5_TXOFF_CONFIG_SWP |\tMLX5_TXOFF_CONFIG_CSUM |\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(i,\n+\t\tMLX5_TXOFF_CONFIG_INLINE |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(v,\n+\t\tMLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+\n+MLX5_TXOFF_INFO(iv,\n+\t\tMLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN |\n+\t\tMLX5_TXOFF_CONFIG_METADATA)\n+};\n+\n+/**\n+ * Configure the Tx function to use. The routine checks configured\n+ * Tx offloads for the device and selects appropriate Tx burst\n+ * routine. There are multiple Tx burst routines compiled from\n+ * the same template in the most optimal way for the dedicated\n+ * Tx offloads set.\n *\n * @param dev\n * Pointer to private data structure.\n@@ -1542,8 +1909,144 @@\n eth_tx_burst_t\n mlx5_select_tx_function(struct rte_eth_dev *dev)\n {\n-\t(void)dev;\n-\treturn removed_tx_burst;\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_dev_config *config = &priv->config;\n+\tuint64_t tx_offloads = dev->data->dev_conf.txmode.offloads;\n+\tunsigned int diff = 0, olx = 0, i, m;\n+\n+\tstatic_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=\n+\t\t MLX5_DSEG_MAX, \"invalid WQE max size\");\n+\tstatic_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,\n+\t\t \"invalid WQE Control Segment size\");\n+\tstatic_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,\n+\t\t \"invalid WQE Ethernet Segment size\");\n+\tstatic_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,\n+\t\t \"invalid WQE Data Segment size\");\n+\tstatic_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,\n+\t\t \"invalid WQE size\");\n+\tassert(priv);\n+\tif (tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) {\n+\t\t/* We should support Multi-Segment Packets. */\n+\t\tolx |= MLX5_TXOFF_CONFIG_MULTI;\n+\t}\n+\tif (tx_offloads & (DEV_TX_OFFLOAD_TCP_TSO |\n+\t\t\t DEV_TX_OFFLOAD_VXLAN_TNL_TSO |\n+\t\t\t DEV_TX_OFFLOAD_GRE_TNL_TSO |\n+\t\t\t DEV_TX_OFFLOAD_IP_TNL_TSO |\n+\t\t\t DEV_TX_OFFLOAD_UDP_TNL_TSO)) {\n+\t\t/* We should support TCP Send Offload. */\n+\t\tolx |= MLX5_TXOFF_CONFIG_TSO;\n+\t}\n+\tif (tx_offloads & (DEV_TX_OFFLOAD_IP_TNL_TSO |\n+\t\t\t DEV_TX_OFFLOAD_UDP_TNL_TSO |\n+\t\t\t DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM)) {\n+\t\t/* We should support Software Parser for Tunnels. */\n+\t\tolx |= MLX5_TXOFF_CONFIG_SWP;\n+\t}\n+\tif (tx_offloads & (DEV_TX_OFFLOAD_IPV4_CKSUM |\n+\t\t\t DEV_TX_OFFLOAD_UDP_CKSUM |\n+\t\t\t DEV_TX_OFFLOAD_TCP_CKSUM |\n+\t\t\t DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM)) {\n+\t\t/* We should support IP/TCP/UDP Checksums. */\n+\t\tolx |= MLX5_TXOFF_CONFIG_CSUM;\n+\t}\n+\tif (tx_offloads & DEV_TX_OFFLOAD_VLAN_INSERT) {\n+\t\t/* We should support VLAN insertion. */\n+\t\tolx |= MLX5_TXOFF_CONFIG_VLAN;\n+\t}\n+\tif (config->tx_inline) {\n+\t\t/*\n+\t\t * Data inlining is enabled by default.\n+\t\t * Required inline data are disabled by default.\n+\t\t */\n+\t\tolx |= MLX5_TXOFF_CONFIG_INLINE;\n+\t}\n+\tif (config->mps == MLX5_MPW_ENHANCED &&\n+\t config->txq_inline_min <= 0) {\n+\t\t/*\n+\t\t * The NIC supports Enhanced Multi-Packet Write.\n+\t\t * We do not support legacy MPW due to its\n+\t\t * hardware related problems, so we just ignore\n+\t\t * legacy MLX5_MPW settings. There should be no\n+\t\t * minimal required inline data.\n+\t\t */\n+\t\tolx |= MLX5_TXOFF_CONFIG_EMPW;\n+\t}\n+\tif (tx_offloads & DEV_TX_OFFLOAD_MATCH_METADATA) {\n+\t\t/* We should support Flow metadata. */\n+\t\tolx |= MLX5_TXOFF_CONFIG_METADATA;\n+\t}\n+\t/*\n+\t * Scan the routines table to find the minimal\n+\t * satisfying routine with requested offloads.\n+\t */\n+\tm = RTE_DIM(txoff_func);\n+\tfor (i = 0; i < RTE_DIM(txoff_func); i++) {\n+\t\tunsigned int tmp;\n+\n+\t\ttmp = txoff_func[i].olx;\n+\t\tif (tmp == olx) {\n+\t\t\t/* Meets requested offloads exactly.*/\n+\t\t\tm = i;\n+\t\t\tbreak;\n+\t\t}\n+\t\tif ((tmp & olx) != olx) {\n+\t\t\t/* Does not meet requested offloads at all. */\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif ((olx ^ tmp) & MLX5_TXOFF_CONFIG_EMPW)\n+\t\t\t/* Do not enable eMPW if not configured. */\n+\t\t\tcontinue;\n+\t\tif ((olx ^ tmp) & MLX5_TXOFF_CONFIG_INLINE)\n+\t\t\t/* Do not enable inlining if not configured. */\n+\t\t\tcontinue;\n+\t\t/*\n+\t\t * Some routine meets the requirements.\n+\t\t * Check whether it has minimal amount\n+\t\t * of not requested offloads.\n+\t\t */\n+\t\ttmp = __builtin_popcountl(tmp & ~olx);\n+\t\tif (m >= RTE_DIM(txoff_func) || tmp < diff) {\n+\t\t\t/* First or better match, save and continue. */\n+\t\t\tm = i;\n+\t\t\tdiff = tmp;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (tmp == diff) {\n+\t\t\ttmp = txoff_func[i].olx ^ txoff_func[m].olx;\n+\t\t\tif (__builtin_ffsl(txoff_func[i].olx & ~tmp) <\n+\t\t\t __builtin_ffsl(txoff_func[m].olx & ~tmp)) {\n+\t\t\t\t/* Lighter not requested offload. */\n+\t\t\t\tm = i;\n+\t\t\t}\n+\t\t}\n+\t}\n+\tif (m >= RTE_DIM(txoff_func)) {\n+\t\tDRV_LOG(DEBUG, \"port %u has no selected Tx function\"\n+\t\t\t \" for requested offloads %04X\",\n+\t\t\t\tdev->data->port_id, olx);\n+\t\treturn NULL;\n+\t}\n+\tDRV_LOG(DEBUG, \"port %u has selected Tx function\"\n+\t\t \" supporting offloads %04X/%04X\",\n+\t\t\tdev->data->port_id, olx, txoff_func[m].olx);\n+\tif (txoff_func[m].olx & MLX5_TXOFF_CONFIG_MULTI)\n+\t\tDRV_LOG(DEBUG, \"\\tMULTI (multi segment)\");\n+\tif (txoff_func[m].olx & MLX5_TXOFF_CONFIG_TSO)\n+\t\tDRV_LOG(DEBUG, \"\\tTSO (TCP send offload)\");\n+\tif (txoff_func[m].olx & MLX5_TXOFF_CONFIG_SWP)\n+\t\tDRV_LOG(DEBUG, \"\\tSWP (software parser)\");\n+\tif (txoff_func[m].olx & MLX5_TXOFF_CONFIG_CSUM)\n+\t\tDRV_LOG(DEBUG, \"\\tCSUM (checksum offload)\");\n+\tif (txoff_func[m].olx & MLX5_TXOFF_CONFIG_INLINE)\n+\t\tDRV_LOG(DEBUG, \"\\tINLIN (inline data)\");\n+\tif (txoff_func[m].olx & MLX5_TXOFF_CONFIG_VLAN)\n+\t\tDRV_LOG(DEBUG, \"\\tVLANI (VLAN insertion)\");\n+\tif (txoff_func[m].olx & MLX5_TXOFF_CONFIG_METADATA)\n+\t\tDRV_LOG(DEBUG, \"\\tMETAD (tx Flow metadata)\");\n+\tif (txoff_func[m].olx & MLX5_TXOFF_CONFIG_EMPW)\n+\t\tDRV_LOG(DEBUG, \"\\tEMPW (Enhanced MPW)\");\n+\treturn txoff_func[m].func;\n }\n \n \n", "prefixes": [ "5/7" ] }