Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/62695/?format=api
{ "id": 62695, "url": "http://patches.dpdk.org/api/patches/62695/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/patch/1573146604-17803-19-git-send-email-viacheslavo@mellanox.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<1573146604-17803-19-git-send-email-viacheslavo@mellanox.com>", "list_archive_url": "https://inbox.dpdk.org/dev/1573146604-17803-19-git-send-email-viacheslavo@mellanox.com", "date": "2019-11-07T17:10:03", "name": "[v3,18/19] net/mlx5: split Rx flows to provide metadata copy", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": true, "hash": "a8c1a2238b2b4ddc6f367e938e2091aff155b24a", "submitter": { "id": 1102, "url": "http://patches.dpdk.org/api/people/1102/?format=api", "name": "Slava Ovsiienko", "email": "viacheslavo@mellanox.com" }, "delegate": { "id": 3268, "url": "http://patches.dpdk.org/api/users/3268/?format=api", "username": "rasland", "first_name": "Raslan", "last_name": "Darawsheh", "email": "rasland@nvidia.com" }, "mbox": "http://patches.dpdk.org/project/dpdk/patch/1573146604-17803-19-git-send-email-viacheslavo@mellanox.com/mbox/", "series": [ { "id": 7336, "url": "http://patches.dpdk.org/api/series/7336/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=7336", "date": "2019-11-07T17:09:46", "name": "net/mlx5: implement extensive metadata feature", "version": 3, "mbox": "http://patches.dpdk.org/series/7336/mbox/" } ], "comments": "http://patches.dpdk.org/api/patches/62695/comments/", "check": "fail", "checks": "http://patches.dpdk.org/api/patches/62695/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from dpdk.org (dpdk.org [92.243.14.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 8A8A2A034E;\n\tThu, 7 Nov 2019 18:12:58 +0100 (CET)", "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id AB94C1C039;\n\tThu, 7 Nov 2019 18:10:49 +0100 (CET)", "from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129])\n by dpdk.org (Postfix) with ESMTP id 6332E1BFDD\n for <dev@dpdk.org>; Thu, 7 Nov 2019 18:10:33 +0100 (CET)", "from Internal Mail-Server by MTLPINE1 (envelope-from\n viacheslavo@mellanox.com)\n with ESMTPS (AES256-SHA encrypted); 7 Nov 2019 19:10:32 +0200", "from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx\n [10.210.16.104])\n by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA7HAW0K024367;\n Thu, 7 Nov 2019 19:10:32 +0200", "from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1])\n by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA7HAWio017978;\n Thu, 7 Nov 2019 17:10:32 GMT", "(from viacheslavo@localhost)\n by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA7HAWw8017977;\n Thu, 7 Nov 2019 17:10:32 GMT" ], "X-Authentication-Warning": "pegasus11.mtr.labs.mlnx: viacheslavo set sender to\n viacheslavo@mellanox.com using -f", "From": "Viacheslav Ovsiienko <viacheslavo@mellanox.com>", "To": "dev@dpdk.org", "Cc": "matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net,\n orika@mellanox.com, Yongseok Koh <yskoh@mellanox.com>", "Date": "Thu, 7 Nov 2019 17:10:03 +0000", "Message-Id": "<1573146604-17803-19-git-send-email-viacheslavo@mellanox.com>", "X-Mailer": "git-send-email 1.8.3.1", "In-Reply-To": "<1573146604-17803-1-git-send-email-viacheslavo@mellanox.com>", "References": "<1572940915-29416-1-git-send-email-viacheslavo@mellanox.com>\n <1573146604-17803-1-git-send-email-viacheslavo@mellanox.com>", "Subject": "[dpdk-dev] [PATCH v3 18/19] net/mlx5: split Rx flows to provide\n\tmetadata copy", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "Values set by MARK and SET_META actions should be carried over\nto the VF representor in case of flow miss on Tx path. However,\nas not all metadata registers are preserved across the different\ndomains (NIC Rx/Tx and E-Switch FDB), as a workaround, those\nvalues should be carried by reg_c's which are preserved across\ndomains and copied to STE flow_tag (MARK) and reg_b (META) fields\nin the last stage of flow steering, in order to scatter those\nvalues to flow_tag and flow_table_metadata of CQE.\n\nWhile reg_c[meta] can be copied to reg_b simply by modify-header\naction (it is supported by hardware), it is not possible to copy\nreg_c[mark] to the STE flow_tag as flow_tag is not a metadata\nregister and this is not supported by hardware. Instead, it should\nbe manually set by a flow per MARK ID. For this purpose, there\nshould be a dedicated flow table - RX_CP_TBL and all the Rx flow\nshould pass by the table to properly copy values.\n\nAs the last action of Rx flow steering must be a terminal action\nsuch as QUEUE, RSS or DROP, if a user flow has Q/RSS action, the\nflow must be split in order to pass by the RX_CP_TBL. And the\nremained Q/RSS action will be performed by another dedicated\naction table - RX_ACT_TBL.\n\nFor example, for an ingress flow:\n pattern,\n actions_having_QRSS\nit must be split into two flows. The first one is,\n pattern,\n actions_except_QRSS / copy (reg_c[2] := flow_id) / jump to RX_CP_TBL\nand the second one in RX_ACT_TBL.\n (if reg_c[2] == flow_id),\n action_QRSS\nwhere flow_id is uniquely allocated and managed identifier.\n\nThis patch implements the Rx flow splitting and build the RX_ACT_TBL.\nAlso, per each egress flow on NIC Tx, a copy action (reg_c[]= reg_a)\nshould be added in order to transfer metadata from WQE.\n\nSigned-off-by: Yongseok Koh <yskoh@mellanox.com>\nSigned-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>\nAcked-by: Matan Azrad <matan@mellanox.com>\n---\n drivers/net/mlx5/mlx5.c | 8 +\n drivers/net/mlx5/mlx5.h | 1 +\n drivers/net/mlx5/mlx5_flow.c | 428 ++++++++++++++++++++++++++++++++++++++++++-\n drivers/net/mlx5/mlx5_flow.h | 1 +\n 4 files changed, 436 insertions(+), 2 deletions(-)", "diff": "diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c\nindex fb7b94b..6359bc9 100644\n--- a/drivers/net/mlx5/mlx5.c\n+++ b/drivers/net/mlx5/mlx5.c\n@@ -2411,6 +2411,12 @@ struct mlx5_flow_id_pool *\n \t\terr = mlx5_alloc_shared_dr(priv);\n \t\tif (err)\n \t\t\tgoto error;\n+\t\tpriv->qrss_id_pool = mlx5_flow_id_pool_alloc();\n+\t\tif (!priv->qrss_id_pool) {\n+\t\t\tDRV_LOG(ERR, \"can't create flow id pool\");\n+\t\t\terr = ENOMEM;\n+\t\t\tgoto error;\n+\t\t}\n \t}\n \t/* Supported Verbs flow priority number detection. */\n \terr = mlx5_flow_discover_priorities(eth_dev);\n@@ -2463,6 +2469,8 @@ struct mlx5_flow_id_pool *\n \t\t\tclose(priv->nl_socket_rdma);\n \t\tif (priv->vmwa_context)\n \t\t\tmlx5_vlan_vmwa_exit(priv->vmwa_context);\n+\t\tif (priv->qrss_id_pool)\n+\t\t\tmlx5_flow_id_pool_release(priv->qrss_id_pool);\n \t\tif (own_domain_id)\n \t\t\tclaim_zero(rte_eth_switch_domain_free(priv->domain_id));\n \t\trte_free(priv);\ndiff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h\nindex 92d445a..9c1a88a 100644\n--- a/drivers/net/mlx5/mlx5.h\n+++ b/drivers/net/mlx5/mlx5.h\n@@ -733,6 +733,7 @@ struct mlx5_priv {\n \tuint32_t nl_sn; /* Netlink message sequence number. */\n \tLIST_HEAD(dbrpage, mlx5_devx_dbr_page) dbrpgs; /* Door-bell pages. */\n \tstruct mlx5_vlan_vmwa_context *vmwa_context; /* VLAN WA context. */\n+\tstruct mlx5_flow_id_pool *qrss_id_pool;\n #ifndef RTE_ARCH_64\n \trte_spinlock_t uar_lock_cq; /* CQs share a common distinct UAR */\n \trte_spinlock_t uar_lock[MLX5_UAR_PAGE_NUM_MAX];\ndiff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c\nindex d97a0b2..2f6ace0 100644\n--- a/drivers/net/mlx5/mlx5_flow.c\n+++ b/drivers/net/mlx5/mlx5_flow.c\n@@ -2222,6 +2222,49 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,\n \treturn 0;\n }\n \n+/* Allocate unique ID for the split Q/RSS subflows. */\n+static uint32_t\n+flow_qrss_get_id(struct rte_eth_dev *dev)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tuint32_t qrss_id, ret;\n+\n+\tret = mlx5_flow_id_get(priv->qrss_id_pool, &qrss_id);\n+\tif (ret)\n+\t\treturn 0;\n+\tassert(qrss_id);\n+\treturn qrss_id;\n+}\n+\n+/* Free unique ID for the split Q/RSS subflows. */\n+static void\n+flow_qrss_free_id(struct rte_eth_dev *dev, uint32_t qrss_id)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\n+\tif (qrss_id)\n+\t\tmlx5_flow_id_release(priv->qrss_id_pool, qrss_id);\n+}\n+\n+/**\n+ * Release resource related QUEUE/RSS action split.\n+ *\n+ * @param dev\n+ * Pointer to Ethernet device.\n+ * @param flow\n+ * Flow to release id's from.\n+ */\n+static void\n+flow_mreg_split_qrss_release(struct rte_eth_dev *dev,\n+\t\t\t struct rte_flow *flow)\n+{\n+\tstruct mlx5_flow *dev_flow;\n+\n+\tLIST_FOREACH(dev_flow, &flow->dev_flows, next)\n+\t\tif (dev_flow->qrss_id)\n+\t\t\tflow_qrss_free_id(dev, dev_flow->qrss_id);\n+}\n+\n static int\n flow_null_validate(struct rte_eth_dev *dev __rte_unused,\n \t\t const struct rte_flow_attr *attr __rte_unused,\n@@ -2511,6 +2554,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,\n \tconst struct mlx5_flow_driver_ops *fops;\n \tenum mlx5_flow_drv_type type = flow->drv_type;\n \n+\tflow_mreg_split_qrss_release(dev, flow);\n \tassert(type > MLX5_FLOW_TYPE_MIN && type < MLX5_FLOW_TYPE_MAX);\n \tfops = flow_get_drv_ops(type);\n \tfops->destroy(dev, flow);\n@@ -2581,6 +2625,41 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,\n }\n \n /**\n+ * Get QUEUE/RSS action from the action list.\n+ *\n+ * @param[in] actions\n+ * Pointer to the list of actions.\n+ * @param[out] qrss\n+ * Pointer to the return pointer.\n+ * @param[out] qrss_type\n+ * Pointer to the action type to return. RTE_FLOW_ACTION_TYPE_END is returned\n+ * if no QUEUE/RSS is found.\n+ *\n+ * @return\n+ * Total number of actions.\n+ */\n+static int\n+flow_parse_qrss_action(const struct rte_flow_action actions[],\n+\t\t const struct rte_flow_action **qrss)\n+{\n+\tint actions_n = 0;\n+\n+\tfor (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) {\n+\t\tswitch (actions->type) {\n+\t\tcase RTE_FLOW_ACTION_TYPE_QUEUE:\n+\t\tcase RTE_FLOW_ACTION_TYPE_RSS:\n+\t\t\t*qrss = actions;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tbreak;\n+\t\t}\n+\t\tactions_n++;\n+\t}\n+\t/* Count RTE_FLOW_ACTION_TYPE_END. */\n+\treturn actions_n + 1;\n+}\n+\n+/**\n * Check if the flow should be splited due to hairpin.\n * The reason for the split is that in current HW we can't\n * support encap on Rx, so if a flow have encap we move it\n@@ -2832,6 +2911,351 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,\n }\n \n /**\n+ * Split action list having QUEUE/RSS for metadata register copy.\n+ *\n+ * Once Q/RSS action is detected in user's action list, the flow action\n+ * should be split in order to copy metadata registers, which will happen in\n+ * RX_CP_TBL like,\n+ * - CQE->flow_tag := reg_c[1] (MARK)\n+ * - CQE->flow_table_metadata (reg_b) := reg_c[0] (META)\n+ * The Q/RSS action will be performed on RX_ACT_TBL after passing by RX_CP_TBL.\n+ * This is because the last action of each flow must be a terminal action\n+ * (QUEUE, RSS or DROP).\n+ *\n+ * Flow ID must be allocated to identify actions in the RX_ACT_TBL and it is\n+ * stored and kept in the mlx5_flow structure per each sub_flow.\n+ *\n+ * The Q/RSS action is replaced with,\n+ * - SET_TAG, setting the allocated flow ID to reg_c[2].\n+ * And the following JUMP action is added at the end,\n+ * - JUMP, to RX_CP_TBL.\n+ *\n+ * A flow to perform remained Q/RSS action will be created in RX_ACT_TBL by\n+ * flow_create_split_metadata() routine. The flow will look like,\n+ * - If flow ID matches (reg_c[2]), perform Q/RSS.\n+ *\n+ * @param dev\n+ * Pointer to Ethernet device.\n+ * @param[out] split_actions\n+ * Pointer to store split actions to jump to CP_TBL.\n+ * @param[in] actions\n+ * Pointer to the list of original flow actions.\n+ * @param[in] qrss\n+ * Pointer to the Q/RSS action.\n+ * @param[in] actions_n\n+ * Number of original actions.\n+ * @param[out] error\n+ * Perform verbose error reporting if not NULL.\n+ *\n+ * @return\n+ * non-zero unique flow_id on success, otherwise 0 and\n+ * error/rte_error are set.\n+ */\n+static uint32_t\n+flow_mreg_split_qrss_prep(struct rte_eth_dev *dev,\n+\t\t\t struct rte_flow_action *split_actions,\n+\t\t\t const struct rte_flow_action *actions,\n+\t\t\t const struct rte_flow_action *qrss,\n+\t\t\t int actions_n, struct rte_flow_error *error)\n+{\n+\tstruct mlx5_rte_flow_action_set_tag *set_tag;\n+\tstruct rte_flow_action_jump *jump;\n+\tconst int qrss_idx = qrss - actions;\n+\tuint32_t flow_id;\n+\tint ret = 0;\n+\n+\t/*\n+\t * Given actions will be split\n+\t * - Replace QUEUE/RSS action with SET_TAG to set flow ID.\n+\t * - Add jump to mreg CP_TBL.\n+\t * As a result, there will be one more action.\n+\t */\n+\t++actions_n;\n+\t/*\n+\t * Allocate the new subflow ID. This one is unique within\n+\t * device and not shared with representors. Otherwise,\n+\t * we would have to resolve multi-thread access synch\n+\t * issue. Each flow on the shared device is appended\n+\t * with source vport identifier, so the resulting\n+\t * flows will be unique in the shared (by master and\n+\t * representors) domain even if they have coinciding\n+\t * IDs.\n+\t */\n+\tflow_id = flow_qrss_get_id(dev);\n+\tif (!flow_id)\n+\t\treturn rte_flow_error_set(error, ENOMEM,\n+\t\t\t\t\t RTE_FLOW_ERROR_TYPE_ACTION,\n+\t\t\t\t\t NULL, \"can't allocate id \"\n+\t\t\t\t\t \"for split Q/RSS subflow\");\n+\t/* Internal SET_TAG action to set flow ID. */\n+\tset_tag = (void *)(split_actions + actions_n);\n+\t*set_tag = (struct mlx5_rte_flow_action_set_tag){\n+\t\t.data = flow_id,\n+\t};\n+\tret = mlx5_flow_get_reg_id(dev, MLX5_COPY_MARK, 0, error);\n+\tif (ret < 0)\n+\t\treturn ret;\n+\tset_tag->id = ret;\n+\t/* JUMP action to jump to mreg copy table (CP_TBL). */\n+\tjump = (void *)(set_tag + 1);\n+\t*jump = (struct rte_flow_action_jump){\n+\t\t.group = MLX5_FLOW_MREG_CP_TABLE_GROUP,\n+\t};\n+\t/* Construct new actions array. */\n+\tmemcpy(split_actions, actions, sizeof(*split_actions) * actions_n);\n+\t/* Replace QUEUE/RSS action. */\n+\tsplit_actions[qrss_idx] = (struct rte_flow_action){\n+\t\t.type = MLX5_RTE_FLOW_ACTION_TYPE_TAG,\n+\t\t.conf = set_tag,\n+\t};\n+\tsplit_actions[actions_n - 2] = (struct rte_flow_action){\n+\t\t.type = RTE_FLOW_ACTION_TYPE_JUMP,\n+\t\t.conf = jump,\n+\t};\n+\tsplit_actions[actions_n - 1] = (struct rte_flow_action){\n+\t\t.type = RTE_FLOW_ACTION_TYPE_END,\n+\t};\n+\treturn flow_id;\n+}\n+\n+/**\n+ * Extend the given action list for Tx metadata copy.\n+ *\n+ * Copy the given action list to the ext_actions and add flow metadata register\n+ * copy action in order to copy reg_a set by WQE to reg_c[0].\n+ *\n+ * @param[out] ext_actions\n+ * Pointer to the extended action list.\n+ * @param[in] actions\n+ * Pointer to the list of actions.\n+ * @param[in] actions_n\n+ * Number of actions in the list.\n+ * @param[out] error\n+ * Perform verbose error reporting if not NULL.\n+ *\n+ * @return\n+ * 0 on success, negative value otherwise\n+ */\n+static int\n+flow_mreg_tx_copy_prep(struct rte_eth_dev *dev,\n+\t\t struct rte_flow_action *ext_actions,\n+\t\t const struct rte_flow_action *actions,\n+\t\t int actions_n, struct rte_flow_error *error)\n+{\n+\tstruct mlx5_flow_action_copy_mreg *cp_mreg =\n+\t\t(struct mlx5_flow_action_copy_mreg *)\n+\t\t\t(ext_actions + actions_n + 1);\n+\tint ret;\n+\n+\tret = mlx5_flow_get_reg_id(dev, MLX5_METADATA_RX, 0, error);\n+\tif (ret < 0)\n+\t\treturn ret;\n+\tcp_mreg->dst = ret;\n+\tret = mlx5_flow_get_reg_id(dev, MLX5_METADATA_TX, 0, error);\n+\tif (ret < 0)\n+\t\treturn ret;\n+\tcp_mreg->src = ret;\n+\tmemcpy(ext_actions, actions,\n+\t\t\tsizeof(*ext_actions) * actions_n);\n+\text_actions[actions_n - 1] = (struct rte_flow_action){\n+\t\t.type = MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG,\n+\t\t.conf = cp_mreg,\n+\t};\n+\text_actions[actions_n] = (struct rte_flow_action){\n+\t\t.type = RTE_FLOW_ACTION_TYPE_END,\n+\t};\n+\treturn 0;\n+}\n+\n+/**\n+ * The splitting for metadata feature.\n+ *\n+ * - Q/RSS action on NIC Rx should be split in order to pass by\n+ * the mreg copy table (RX_CP_TBL) and then it jumps to the\n+ * action table (RX_ACT_TBL) which has the split Q/RSS action.\n+ *\n+ * - All the actions on NIC Tx should have a mreg copy action to\n+ * copy reg_a from WQE to reg_c[0].\n+ *\n+ * @param dev\n+ * Pointer to Ethernet device.\n+ * @param[in] flow\n+ * Parent flow structure pointer.\n+ * @param[in] attr\n+ * Flow rule attributes.\n+ * @param[in] items\n+ * Pattern specification (list terminated by the END pattern item).\n+ * @param[in] actions\n+ * Associated actions (list terminated by the END action).\n+ * @param[in] external\n+ * This flow rule is created by request external to PMD.\n+ * @param[out] error\n+ * Perform verbose error reporting if not NULL.\n+ * @return\n+ * 0 on success, negative value otherwise\n+ */\n+static int\n+flow_create_split_metadata(struct rte_eth_dev *dev,\n+\t\t\t struct rte_flow *flow,\n+\t\t\t const struct rte_flow_attr *attr,\n+\t\t\t const struct rte_flow_item items[],\n+\t\t\t const struct rte_flow_action actions[],\n+\t\t\t bool external, struct rte_flow_error *error)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_dev_config *config = &priv->config;\n+\tconst struct rte_flow_action *qrss = NULL;\n+\tstruct rte_flow_action *ext_actions = NULL;\n+\tstruct mlx5_flow *dev_flow = NULL;\n+\tuint32_t qrss_id = 0;\n+\tsize_t act_size;\n+\tint actions_n;\n+\tint ret;\n+\n+\t/* Check whether extensive metadata feature is engaged. */\n+\tif (!config->dv_flow_en ||\n+\t config->dv_xmeta_en == MLX5_XMETA_MODE_LEGACY ||\n+\t !mlx5_flow_ext_mreg_supported(dev))\n+\t\treturn flow_create_split_inner(dev, flow, NULL, attr, items,\n+\t\t\t\t\t actions, external, error);\n+\tactions_n = flow_parse_qrss_action(actions, &qrss);\n+\tif (qrss) {\n+\t\t/* Exclude hairpin flows from splitting. */\n+\t\tif (qrss->type == RTE_FLOW_ACTION_TYPE_QUEUE) {\n+\t\t\tconst struct rte_flow_action_queue *queue;\n+\n+\t\t\tqueue = qrss->conf;\n+\t\t\tif (mlx5_rxq_get_type(dev, queue->index) ==\n+\t\t\t MLX5_RXQ_TYPE_HAIRPIN)\n+\t\t\t\tqrss = NULL;\n+\t\t} else if (qrss->type == RTE_FLOW_ACTION_TYPE_RSS) {\n+\t\t\tconst struct rte_flow_action_rss *rss;\n+\n+\t\t\trss = qrss->conf;\n+\t\t\tif (mlx5_rxq_get_type(dev, rss->queue[0]) ==\n+\t\t\t MLX5_RXQ_TYPE_HAIRPIN)\n+\t\t\t\tqrss = NULL;\n+\t\t}\n+\t}\n+\tif (qrss) {\n+\t\t/*\n+\t\t * Q/RSS action on NIC Rx should be split in order to pass by\n+\t\t * the mreg copy table (RX_CP_TBL) and then it jumps to the\n+\t\t * action table (RX_ACT_TBL) which has the split Q/RSS action.\n+\t\t */\n+\t\tact_size = sizeof(struct rte_flow_action) * (actions_n + 1) +\n+\t\t\t sizeof(struct rte_flow_action_set_tag) +\n+\t\t\t sizeof(struct rte_flow_action_jump);\n+\t\text_actions = rte_zmalloc(__func__, act_size, 0);\n+\t\tif (!ext_actions)\n+\t\t\treturn rte_flow_error_set(error, ENOMEM,\n+\t\t\t\t\t\t RTE_FLOW_ERROR_TYPE_ACTION,\n+\t\t\t\t\t\t NULL, \"no memory to split \"\n+\t\t\t\t\t\t \"metadata flow\");\n+\t\t/*\n+\t\t * Create the new actions list with removed Q/RSS action\n+\t\t * and appended set tag and jump to register copy table\n+\t\t * (RX_CP_TBL). We should preallocate unique tag ID here\n+\t\t * in advance, because it is needed for set tag action.\n+\t\t */\n+\t\tqrss_id = flow_mreg_split_qrss_prep(dev, ext_actions, actions,\n+\t\t\t\t\t\t qrss, actions_n, error);\n+\t\tif (!qrss_id) {\n+\t\t\tret = -rte_errno;\n+\t\t\tgoto exit;\n+\t\t}\n+\t} else if (attr->egress && !attr->transfer) {\n+\t\t/*\n+\t\t * All the actions on NIC Tx should have a metadata register\n+\t\t * copy action to copy reg_a from WQE to reg_c[meta]\n+\t\t */\n+\t\tact_size = sizeof(struct rte_flow_action) * (actions_n + 1) +\n+\t\t\t sizeof(struct mlx5_flow_action_copy_mreg);\n+\t\text_actions = rte_zmalloc(__func__, act_size, 0);\n+\t\tif (!ext_actions)\n+\t\t\treturn rte_flow_error_set(error, ENOMEM,\n+\t\t\t\t\t\t RTE_FLOW_ERROR_TYPE_ACTION,\n+\t\t\t\t\t\t NULL, \"no memory to split \"\n+\t\t\t\t\t\t \"metadata flow\");\n+\t\t/* Create the action list appended with copy register. */\n+\t\tret = flow_mreg_tx_copy_prep(dev, ext_actions, actions,\n+\t\t\t\t\t actions_n, error);\n+\t\tif (ret < 0)\n+\t\t\tgoto exit;\n+\t}\n+\t/* Add the unmodified original or prefix subflow. */\n+\tret = flow_create_split_inner(dev, flow, &dev_flow, attr, items,\n+\t\t\t\t ext_actions ? ext_actions : actions,\n+\t\t\t\t external, error);\n+\tif (ret < 0)\n+\t\tgoto exit;\n+\tassert(dev_flow);\n+\tif (qrss_id) {\n+\t\tconst struct rte_flow_attr q_attr = {\n+\t\t\t.group = MLX5_FLOW_MREG_ACT_TABLE_GROUP,\n+\t\t\t.ingress = 1,\n+\t\t};\n+\t\t/* Internal PMD action to set register. */\n+\t\tstruct mlx5_rte_flow_item_tag q_tag_spec = {\n+\t\t\t.data = qrss_id,\n+\t\t\t.id = 0,\n+\t\t};\n+\t\tstruct rte_flow_item q_items[] = {\n+\t\t\t{\n+\t\t\t\t.type = MLX5_RTE_FLOW_ITEM_TYPE_TAG,\n+\t\t\t\t.spec = &q_tag_spec,\n+\t\t\t\t.last = NULL,\n+\t\t\t\t.mask = NULL,\n+\t\t\t},\n+\t\t\t{\n+\t\t\t\t.type = RTE_FLOW_ITEM_TYPE_END,\n+\t\t\t},\n+\t\t};\n+\t\tstruct rte_flow_action q_actions[] = {\n+\t\t\t{\n+\t\t\t\t.type = qrss->type,\n+\t\t\t\t.conf = qrss->conf,\n+\t\t\t},\n+\t\t\t{\n+\t\t\t\t.type = RTE_FLOW_ACTION_TYPE_END,\n+\t\t\t},\n+\t\t};\n+\t\tuint64_t hash_fields = dev_flow->hash_fields;\n+\t\t/*\n+\t\t * Put unique id in prefix flow due to it is destroyed after\n+\t\t * prefix flow and id will be freed after there is no actual\n+\t\t * flows with this id and identifier reallocation becomes\n+\t\t * possible (for example, for other flows in other threads).\n+\t\t */\n+\t\tdev_flow->qrss_id = qrss_id;\n+\t\tqrss_id = 0;\n+\t\tdev_flow = NULL;\n+\t\tret = mlx5_flow_get_reg_id(dev, MLX5_COPY_MARK, 0, error);\n+\t\tif (ret < 0)\n+\t\t\tgoto exit;\n+\t\tq_tag_spec.id = ret;\n+\t\t/* Add suffix subflow to execute Q/RSS. */\n+\t\tret = flow_create_split_inner(dev, flow, &dev_flow,\n+\t\t\t\t\t &q_attr, q_items, q_actions,\n+\t\t\t\t\t external, error);\n+\t\tif (ret < 0)\n+\t\t\tgoto exit;\n+\t\tassert(dev_flow);\n+\t\tdev_flow->hash_fields = hash_fields;\n+\t}\n+\n+exit:\n+\t/*\n+\t * We do not destroy the partially created sub_flows in case of error.\n+\t * These ones are included into parent flow list and will be destroyed\n+\t * by flow_drv_destroy.\n+\t */\n+\tflow_qrss_free_id(dev, qrss_id);\n+\trte_free(ext_actions);\n+\treturn ret;\n+}\n+\n+/**\n * Split the flow to subflow set. The splitters might be linked\n * in the chain, like this:\n * flow_create_split_outer() calls:\n@@ -2876,8 +3300,8 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,\n {\n \tint ret;\n \n-\tret = flow_create_split_inner(dev, flow, NULL, attr, items,\n-\t\t\t\t actions, external, error);\n+\tret = flow_create_split_metadata(dev, flow, attr, items,\n+\t\t\t\t\t actions, external, error);\n \tassert(ret <= 0);\n \treturn ret;\n }\ndiff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h\nindex ef16aef..c71938b 100644\n--- a/drivers/net/mlx5/mlx5_flow.h\n+++ b/drivers/net/mlx5/mlx5_flow.h\n@@ -500,6 +500,7 @@ struct mlx5_flow {\n #endif\n \t\tstruct mlx5_flow_verbs verbs;\n \t};\n+\tuint32_t qrss_id; /**< Uniqie Q/RSS suffix subflow tag. */\n \tbool external; /**< true if the flow is created external to PMD. */\n };\n \n", "prefixes": [ "v3", "18/19" ] }