Patch Detail

HTTP 200 OK
Allow: GET, PUT, PATCH, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "id": 91532,
    "url": "https://patches.dpdk.org/api/patches/91532/?format=api",
    "web_url": "https://patches.dpdk.org/project/dpdk/patch/20210415050929.2082949-2-lizh@nvidia.com/",
    "project": {
        "id": 1,
        "url": "https://patches.dpdk.org/api/projects/1/?format=api",
        "name": "DPDK",
        "link_name": "dpdk",
        "list_id": "dev.dpdk.org",
        "list_email": "dev@dpdk.org",
        "web_url": "http://core.dpdk.org",
        "scm_url": "git://dpdk.org/dpdk",
        "webscm_url": "http://git.dpdk.org/dpdk",
        "list_archive_url": "https://inbox.dpdk.org/dev",
        "list_archive_url_format": "https://inbox.dpdk.org/dev/{}",
        "commit_url_format": ""
    },
    "msgid": "<20210415050929.2082949-2-lizh@nvidia.com>",
    "list_archive_url": "https://inbox.dpdk.org/dev/20210415050929.2082949-2-lizh@nvidia.com",
    "date": "2021-04-15T05:09:26",
    "name": "[v6,1/4] net/mlx5: support meter policy operations",
    "commit_ref": null,
    "pull_url": null,
    "state": "superseded",
    "archived": true,
    "hash": "34f421b8208d7cf11046b6dcd202340e691fc1f7",
    "submitter": {
        "id": 1967,
        "url": "https://patches.dpdk.org/api/people/1967/?format=api",
        "name": "Li Zhang",
        "email": "lizh@nvidia.com"
    },
    "delegate": {
        "id": 3268,
        "url": "https://patches.dpdk.org/api/users/3268/?format=api",
        "username": "rasland",
        "first_name": "Raslan",
        "last_name": "Darawsheh",
        "email": "rasland@nvidia.com"
    },
    "mbox": "https://patches.dpdk.org/project/dpdk/patch/20210415050929.2082949-2-lizh@nvidia.com/mbox/",
    "series": [
        {
            "id": 16391,
            "url": "https://patches.dpdk.org/api/series/16391/?format=api",
            "web_url": "https://patches.dpdk.org/project/dpdk/list/?series=16391",
            "date": "2021-04-15T05:09:25",
            "name": "net/mlx5: support meter policy operations",
            "version": 6,
            "mbox": "https://patches.dpdk.org/series/16391/mbox/"
        }
    ],
    "comments": "https://patches.dpdk.org/api/patches/91532/comments/",
    "check": "warning",
    "checks": "https://patches.dpdk.org/api/patches/91532/checks/",
    "tags": {},
    "related": [],
    "headers": {
        "Return-Path": "<dev-bounces@dpdk.org>",
        "X-Original-To": "patchwork@inbox.dpdk.org",
        "Delivered-To": "patchwork@inbox.dpdk.org",
        "Received": [
            "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 9AEC3A0A0C;\n\tThu, 15 Apr 2021 07:10:01 +0200 (CEST)",
            "from [217.70.189.124] (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 1178B161FC7;\n\tThu, 15 Apr 2021 07:09:51 +0200 (CEST)",
            "from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129])\n by mails.dpdk.org (Postfix) with ESMTP id 7D79C161FA9\n for <dev@dpdk.org>; Thu, 15 Apr 2021 07:09:40 +0200 (CEST)",
            "from Internal Mail-Server by MTLPINE1 (envelope-from\n lizh@nvidia.com)\n with SMTP; 15 Apr 2021 08:09:35 +0300",
            "from nvidia.com (c-235-17-1-009.mtl.labs.mlnx [10.235.17.9])\n by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 13F59YFb027304;\n Thu, 15 Apr 2021 08:09:34 +0300"
        ],
        "From": "Li Zhang <lizh@nvidia.com>",
        "To": "dekelp@nvidia.com, orika@nvidia.com, viacheslavo@nvidia.com,\n matan@nvidia.com, shahafs@nvidia.com",
        "Cc": "dev@dpdk.org, thomas@monjalon.net, rasland@nvidia.com, roniba@nvidia.com",
        "Date": "Thu, 15 Apr 2021 08:09:26 +0300",
        "Message-Id": "<20210415050929.2082949-2-lizh@nvidia.com>",
        "X-Mailer": "git-send-email 2.21.0",
        "In-Reply-To": "<20210415050929.2082949-1-lizh@nvidia.com>",
        "References": "<20210401081624.1482490-1-lizh@nvidia.com>\n <20210415050929.2082949-1-lizh@nvidia.com>",
        "MIME-Version": "1.0",
        "Content-Transfer-Encoding": "8bit",
        "Subject": "[dpdk-dev] [PATCH v6 1/4] net/mlx5: support meter policy operations",
        "X-BeenThere": "dev@dpdk.org",
        "X-Mailman-Version": "2.1.29",
        "Precedence": "list",
        "List-Id": "DPDK patches and discussions <dev.dpdk.org>",
        "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>",
        "List-Archive": "<http://mails.dpdk.org/archives/dev/>",
        "List-Post": "<mailto:dev@dpdk.org>",
        "List-Help": "<mailto:dev-request@dpdk.org?subject=help>",
        "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>",
        "Errors-To": "dev-bounces@dpdk.org",
        "Sender": "\"dev\" <dev-bounces@dpdk.org>"
    },
    "content": "MLX5 PMD checks the validation of actions in policy while add\na new meter policy, if pass the validation, allocates the new\npolicy object from the meter policy indexed memory pool.\n\nIt is common to use the same policy for multiple meters.\nMLX5 PMD supports two types of policy: termination policy and\nno-termination policy.\n\nImplement the next policy operations:\nvalidate:\nThe driver doesn't support to configure actions in the flow\nafter the meter action except one case when the meter policy\nis configured to do nothing in GREEN\\YELLOW and only DROP action\nin RED, this special policy is called non-terminated policy\nand is handed as a singleton object internally.\n\nFor all the terminated policies, the next actions are supported:\nGREEN - QUEUE, RSS, PORT_ID, JUMP, DROP, MARK and SET_TAG.\nYELLOW - not supported at all -> must be empty.\nRED - must include DROP action.\n\nHence, in ingress case, for example,\nQUEUE\\RSS\\JUMP must be configured as last action for GREEN color.\n\nAll the above limitations will be validated.\n\ncreate:\nValidate the policy configuration.\nPrepare the related tables and actions.\n\ndestroy:\nRelease the created policy resources.\n\nSigned-off-by: Li Zhang <lizh@nvidia.com>\nAcked-by: Matan Azrad <matan@nvidia.com>\n---\n drivers/net/mlx5/linux/mlx5_os.c   |    2 +-\n drivers/net/mlx5/mlx5.c            |   77 +-\n drivers/net/mlx5/mlx5.h            |  158 +++-\n drivers/net/mlx5/mlx5_flow.c       |  192 +++-\n drivers/net/mlx5/mlx5_flow.h       |   73 +-\n drivers/net/mlx5/mlx5_flow_aso.c   |   10 +-\n drivers/net/mlx5/mlx5_flow_dv.c    | 1412 +++++++++++++++++++++++++++-\n drivers/net/mlx5/mlx5_flow_meter.c |  445 ++++++++-\n drivers/net/mlx5/mlx5_trigger.c    |    1 +\n 9 files changed, 2242 insertions(+), 128 deletions(-)",
    "diff": "diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c\nindex bec288c840..276283d492 100644\n--- a/drivers/net/mlx5/linux/mlx5_os.c\n+++ b/drivers/net/mlx5/linux/mlx5_os.c\n@@ -1299,7 +1299,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,\n \t\t\tlog_obj_size <=\n \t\t\tconfig->hca_attr.qos.log_meter_aso_max_alloc) {\n \t\t\t\tsh->meter_aso_en = 1;\n-\t\t\t\terr = mlx5_aso_flow_mtrs_mng_init(priv);\n+\t\t\t\terr = mlx5_aso_flow_mtrs_mng_init(priv->sh);\n \t\t\t\tif (err) {\n \t\t\t\t\terr = -err;\n \t\t\t\t\tgoto error;\ndiff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c\nindex 572db1bc67..b4cdaa7131 100644\n--- a/drivers/net/mlx5/mlx5.c\n+++ b/drivers/net/mlx5/mlx5.c\n@@ -350,6 +350,20 @@ static const struct mlx5_indexed_pool_config mlx5_ipool_cfg[] = {\n \t\t.free = mlx5_free,\n \t\t.type = \"mlx5_shared_action_rss\",\n \t},\n+\t[MLX5_IPOOL_MTR_POLICY] = {\n+\t\t/**\n+\t\t * The ipool index should grow continually from small to big,\n+\t\t * for policy idx, so not set grow_trunk to avoid policy index\n+\t\t * not jump continually.\n+\t\t */\n+\t\t.size = sizeof(struct mlx5_flow_meter_sub_policy),\n+\t\t.trunk_size = 64,\n+\t\t.need_lock = 1,\n+\t\t.release_mem_en = 1,\n+\t\t.malloc = mlx5_malloc,\n+\t\t.free = mlx5_free,\n+\t\t.type = \"mlx5_meter_policy_ipool\",\n+\t},\n };\n \n \n@@ -569,27 +583,25 @@ mlx5_flow_counters_mng_close(struct mlx5_dev_ctx_shared *sh)\n  *   Pointer to mlx5_dev_ctx_shared object to free\n  */\n int\n-mlx5_aso_flow_mtrs_mng_init(struct mlx5_priv *priv)\n+mlx5_aso_flow_mtrs_mng_init(struct mlx5_dev_ctx_shared *sh)\n {\n-\tif (!priv->mtr_idx_tbl) {\n-\t\tpriv->mtr_idx_tbl = mlx5_l3t_create(MLX5_L3T_TYPE_DWORD);\n-\t\tif (!priv->mtr_idx_tbl) {\n-\t\t\tDRV_LOG(ERR, \"fail to create meter lookup table.\");\n-\t\t\trte_errno = ENOMEM;\n-\t\t\treturn -ENOMEM;\n-\t\t}\n-\t}\n-\tif (!priv->sh->mtrmng) {\n-\t\tpriv->sh->mtrmng = mlx5_malloc(MLX5_MEM_ZERO,\n-\t\t\tsizeof(*priv->sh->mtrmng),\n+\tif (!sh->mtrmng) {\n+\t\tsh->mtrmng = mlx5_malloc(MLX5_MEM_ZERO,\n+\t\t\tsizeof(*sh->mtrmng),\n \t\t\tRTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);\n-\t\tif (!priv->sh->mtrmng) {\n-\t\t\tDRV_LOG(ERR, \"mlx5_aso_mtr_pools_mng allocation was failed.\");\n+\t\tif (!sh->mtrmng) {\n+\t\t\tDRV_LOG(ERR,\n+\t\t\t\"meter management allocation was failed.\");\n \t\t\trte_errno = ENOMEM;\n \t\t\treturn -ENOMEM;\n \t\t}\n-\t\trte_spinlock_init(&priv->sh->mtrmng->mtrsl);\n-\t\tLIST_INIT(&priv->sh->mtrmng->meters);\n+\t\tif (sh->meter_aso_en) {\n+\t\t\trte_spinlock_init(&sh->mtrmng->pools_mng.mtrsl);\n+\t\t\tLIST_INIT(&sh->mtrmng->pools_mng.meters);\n+\t\t\tsh->mtrmng->policy_idx_tbl =\n+\t\t\t\tmlx5_l3t_create(MLX5_L3T_TYPE_DWORD);\n+\t\t}\n+\t\tsh->mtrmng->def_policy_id = MLX5_INVALID_POLICY_ID;\n \t}\n \treturn 0;\n }\n@@ -605,31 +617,34 @@ static void\n mlx5_aso_flow_mtrs_mng_close(struct mlx5_dev_ctx_shared *sh)\n {\n \tstruct mlx5_aso_mtr_pool *mtr_pool;\n-\tstruct mlx5_aso_mtr_pools_mng *mtrmng = sh->mtrmng;\n+\tstruct mlx5_flow_mtr_mng *mtrmng = sh->mtrmng;\n \tuint32_t idx;\n #ifdef HAVE_MLX5_DR_CREATE_ACTION_ASO\n \tstruct mlx5_aso_mtr *aso_mtr;\n \tint i;\n #endif /* HAVE_MLX5_DR_CREATE_ACTION_ASO */\n \n-\tmlx5_aso_queue_uninit(sh, ASO_OPC_MOD_POLICER);\n-\tidx = mtrmng->n_valid;\n-\twhile (idx--) {\n-\t\tmtr_pool = mtrmng->pools[idx];\n+\tif (sh->meter_aso_en) {\n+\t\tmlx5_aso_queue_uninit(sh, ASO_OPC_MOD_POLICER);\n+\t\tidx = mtrmng->pools_mng.n_valid;\n+\t\twhile (idx--) {\n+\t\t\tmtr_pool = mtrmng->pools_mng.pools[idx];\n #ifdef HAVE_MLX5_DR_CREATE_ACTION_ASO\n-\t\tfor (i = 0; i < MLX5_ASO_MTRS_PER_POOL; i++) {\n-\t\t\taso_mtr = &mtr_pool->mtrs[i];\n-\t\t\tif (aso_mtr->fm.meter_action)\n-\t\t\t\tclaim_zero(mlx5_glue->destroy_flow_action\n-\t\t\t\t\t\t(aso_mtr->fm.meter_action));\n-\t\t}\n+\t\t\tfor (i = 0; i < MLX5_ASO_MTRS_PER_POOL; i++) {\n+\t\t\t\taso_mtr = &mtr_pool->mtrs[i];\n+\t\t\t\tif (aso_mtr->fm.meter_action)\n+\t\t\t\t\tclaim_zero\n+\t\t\t\t\t(mlx5_glue->destroy_flow_action\n+\t\t\t\t\t(aso_mtr->fm.meter_action));\n+\t\t\t}\n #endif /* HAVE_MLX5_DR_CREATE_ACTION_ASO */\n-\t\tclaim_zero(mlx5_devx_cmd_destroy\n+\t\t\tclaim_zero(mlx5_devx_cmd_destroy\n \t\t\t\t\t\t(mtr_pool->devx_obj));\n-\t\tmtrmng->n_valid--;\n-\t\tmlx5_free(mtr_pool);\n+\t\t\tmtrmng->pools_mng.n_valid--;\n+\t\t\tmlx5_free(mtr_pool);\n+\t\t}\n+\t\tmlx5_free(sh->mtrmng->pools_mng.pools);\n \t}\n-\tmlx5_free(sh->mtrmng->pools);\n \tmlx5_free(sh->mtrmng);\n \tsh->mtrmng = NULL;\n }\ndiff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h\nindex a8e11023cc..cb7c75aa2e 100644\n--- a/drivers/net/mlx5/mlx5.h\n+++ b/drivers/net/mlx5/mlx5.h\n@@ -56,6 +56,7 @@ enum mlx5_ipool_index {\n \tMLX5_IPOOL_RTE_FLOW, /* Pool for rte_flow. */\n \tMLX5_IPOOL_RSS_EXPANTION_FLOW_ID, /* Pool for Queue/RSS flow ID. */\n \tMLX5_IPOOL_RSS_SHARED_ACTIONS, /* Pool for RSS shared actions. */\n+\tMLX5_IPOOL_MTR_POLICY, /* Pool for meter policy resource. */\n \tMLX5_IPOOL_MAX,\n };\n \n@@ -580,9 +581,126 @@ struct mlx5_dev_shared_port {\n \t/* Aging information for per port. */\n };\n \n+/*\n+ * Max number of actions per DV flow.\n+ * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED\n+ * in rdma-core file providers/mlx5/verbs.c.\n+ */\n+#define MLX5_DV_MAX_NUMBER_OF_ACTIONS 8\n+\n /*ASO flow meter structures*/\n /* Modify this value if enum rte_mtr_color changes. */\n #define RTE_MTR_DROPPED RTE_COLORS\n+/* Yellow is not supported. */\n+#define MLX5_MTR_RTE_COLORS (RTE_COLOR_GREEN + 1)\n+/* table_id 22 bits in mlx5_flow_tbl_key so limit policy number. */\n+#define MLX5_MAX_SUB_POLICY_TBL_NUM 0x3FFFFF\n+#define MLX5_INVALID_POLICY_ID UINT32_MAX\n+/* Suffix table_id on MLX5_FLOW_TABLE_LEVEL_METER. */\n+#define MLX5_MTR_TABLE_ID_SUFFIX 1\n+/* Drop table_id on MLX5_FLOW_TABLE_LEVEL_METER. */\n+#define MLX5_MTR_TABLE_ID_DROP 2\n+\n+enum mlx5_meter_domain {\n+\tMLX5_MTR_DOMAIN_INGRESS,\n+\tMLX5_MTR_DOMAIN_EGRESS,\n+\tMLX5_MTR_DOMAIN_TRANSFER,\n+\tMLX5_MTR_DOMAIN_MAX,\n+};\n+#define MLX5_MTR_DOMAIN_INGRESS_BIT  (1 << MLX5_MTR_DOMAIN_INGRESS)\n+#define MLX5_MTR_DOMAIN_EGRESS_BIT   (1 << MLX5_MTR_DOMAIN_EGRESS)\n+#define MLX5_MTR_DOMAIN_TRANSFER_BIT (1 << MLX5_MTR_DOMAIN_TRANSFER)\n+#define MLX5_MTR_ALL_DOMAIN_BIT      (MLX5_MTR_DOMAIN_INGRESS_BIT | \\\n+\t\t\t\t\tMLX5_MTR_DOMAIN_EGRESS_BIT | \\\n+\t\t\t\t\tMLX5_MTR_DOMAIN_TRANSFER_BIT)\n+\n+/*\n+ * Meter sub-policy structure.\n+ * Each RSS TIR in meter policy need its own sub-policy resource.\n+ */\n+struct mlx5_flow_meter_sub_policy {\n+\tuint32_t main_policy_id:1;\n+\t/* Main policy id is same as this sub_policy id. */\n+\tuint32_t idx:31;\n+\t/* Index to sub_policy ipool entity. */\n+\tvoid *main_policy;\n+\t/* Point to struct mlx5_flow_meter_policy. */\n+\tstruct mlx5_flow_tbl_resource *tbl_rsc;\n+\t/* The sub-policy table resource. */\n+\tuint32_t rix_hrxq[MLX5_MTR_RTE_COLORS];\n+\t/* Index to TIR resource. */\n+\tstruct mlx5_flow_tbl_resource *jump_tbl[MLX5_MTR_RTE_COLORS];\n+\t/* Meter jump/drop table. */\n+\tstruct mlx5_flow_dv_matcher *color_matcher[RTE_COLORS];\n+\t/* Matcher for Color. */\n+\tvoid *color_rule[RTE_COLORS];\n+\t/* Meter green/yellow/drop rule. */\n+};\n+\n+struct mlx5_meter_policy_acts {\n+\tuint8_t actions_n;\n+\t/* Number of actions. */\n+\tvoid *dv_actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];\n+\t/* Action list. */\n+};\n+\n+struct mlx5_meter_policy_action_container {\n+\tuint32_t rix_mark;\n+\t/* Index to the mark action. */\n+\tstruct mlx5_flow_dv_modify_hdr_resource *modify_hdr;\n+\t/* Pointer to modify header resource in cache. */\n+\tuint8_t fate_action;\n+\t/* Fate action type. */\n+\tunion {\n+\t\tstruct rte_flow_action *rss;\n+\t\t/* Rss action configuration. */\n+\t\tuint32_t rix_port_id_action;\n+\t\t/* Index to port ID action resource. */\n+\t\tvoid *dr_jump_action[MLX5_MTR_DOMAIN_MAX];\n+\t\t/* Jump/drop action per color. */\n+\t};\n+};\n+\n+/* Flow meter policy parameter structure. */\n+struct mlx5_flow_meter_policy {\n+\tuint32_t is_rss:1;\n+\t/* Is RSS policy table. */\n+\tuint32_t ingress:1;\n+\t/* Rule applies to ingress domain. */\n+\tuint32_t egress:1;\n+\t/* Rule applies to egress domain. */\n+\tuint32_t transfer:1;\n+\t/* Rule applies to transfer domain. */\n+\trte_spinlock_t sl;\n+\tuint32_t ref_cnt;\n+\t/* Use count. */\n+\tstruct mlx5_meter_policy_action_container act_cnt[MLX5_MTR_RTE_COLORS];\n+\t/* Policy actions container. */\n+\tvoid *dr_drop_action[MLX5_MTR_DOMAIN_MAX];\n+\t/* drop action for red color. */\n+\tuint16_t sub_policy_num;\n+\t/* Count sub policy tables, 3 bits per domain. */\n+\tstruct mlx5_flow_meter_sub_policy **sub_policys[MLX5_MTR_DOMAIN_MAX];\n+\t/* Sub policy table array must be the end of struct. */\n+};\n+\n+/* The maximum sub policy is relate to struct mlx5_rss_hash_fields[]. */\n+#define MLX5_MTR_RSS_MAX_SUB_POLICY 7\n+#define MLX5_MTR_SUB_POLICY_NUM_SHIFT  3\n+#define MLX5_MTR_SUB_POLICY_NUM_MASK  0x7\n+#define MLX5_MTRS_DEFAULT_RULE_PRIORITY 0xFFFF\n+\n+/* Flow meter default policy parameter structure.\n+ * Policy index 0 is reserved by default policy table.\n+ * Action per color as below:\n+ * green - do nothing, yellow - do nothing, red - drop\n+ */\n+struct mlx5_flow_meter_def_policy {\n+\tstruct mlx5_flow_meter_sub_policy sub_policy;\n+\t/* Policy rules jump to other tables. */\n+\tvoid *dr_jump_action[RTE_COLORS];\n+\t/* Jump action per color. */\n+};\n \n /* Meter table structure. */\n struct mlx5_meter_domain_info {\n@@ -746,6 +864,28 @@ struct mlx5_aso_mtr_pools_mng {\n \tstruct mlx5_aso_mtr_pool **pools; /* ASO flow meter pool array. */\n };\n \n+/* Meter management structure for global flow meter resource. */\n+struct mlx5_flow_mtr_mng {\n+\tstruct mlx5_aso_mtr_pools_mng pools_mng;\n+\t/* Pools management structure for ASO flow meter pools. */\n+\tstruct mlx5_flow_meter_def_policy *def_policy[MLX5_MTR_DOMAIN_MAX];\n+\t/* Default policy table. */\n+\tuint32_t def_policy_id;\n+\t/* Default policy id. */\n+\tuint32_t def_policy_ref_cnt;\n+\t/** def_policy meter use count. */\n+\tstruct mlx5_l3t_tbl *policy_idx_tbl;\n+\t/* Policy index lookup table. */\n+\tstruct mlx5_flow_tbl_resource *drop_tbl[MLX5_MTR_DOMAIN_MAX];\n+\t/* Meter drop table. */\n+\tstruct mlx5_flow_dv_matcher *drop_matcher[MLX5_MTR_DOMAIN_MAX];\n+\t/* Matcher meter in drop table. */\n+\tstruct mlx5_flow_dv_matcher *def_matcher[MLX5_MTR_DOMAIN_MAX];\n+\t/* Default matcher in drop table. */\n+\tvoid *def_rule[MLX5_MTR_DOMAIN_MAX];\n+\t/* Default rule in drop table. */\n+};\n+\n /* Table key of the hash organization. */\n union mlx5_flow_tbl_key {\n \tstruct {\n@@ -772,9 +912,9 @@ struct mlx5_flow_tbl_resource {\n #define MLX5_FLOW_MREG_ACT_TABLE_GROUP (MLX5_MAX_TABLES - 1)\n #define MLX5_FLOW_MREG_CP_TABLE_GROUP (MLX5_MAX_TABLES - 2)\n /* Tables for metering splits should be added here. */\n-#define MLX5_FLOW_TABLE_LEVEL_SUFFIX (MLX5_MAX_TABLES - 3)\n-#define MLX5_FLOW_TABLE_LEVEL_METER (MLX5_MAX_TABLES - 4)\n-#define MLX5_MAX_TABLES_EXTERNAL MLX5_FLOW_TABLE_LEVEL_METER\n+#define MLX5_FLOW_TABLE_LEVEL_METER (MLX5_MAX_TABLES - 3)\n+#define MLX5_FLOW_TABLE_LEVEL_POLICY (MLX5_MAX_TABLES - 4)\n+#define MLX5_MAX_TABLES_EXTERNAL MLX5_FLOW_TABLE_LEVEL_POLICY\n #define MLX5_MAX_TABLES_FDB UINT16_MAX\n #define MLX5_FLOW_TABLE_FACTOR 10\n \n@@ -935,8 +1075,8 @@ struct mlx5_dev_ctx_shared {\n \tstruct mlx5_geneve_tlv_option_resource *geneve_tlv_option_resource;\n \t/* Management structure for geneve tlv option */\n \trte_spinlock_t geneve_tlv_opt_sl; /* Lock for geneve tlv resource */\n-\tstruct mlx5_aso_mtr_pools_mng *mtrmng;\n-\t/* Meter pools management structure. */\n+\tstruct mlx5_flow_mtr_mng *mtrmng;\n+\t/* Meter management structure. */\n \tstruct mlx5_dev_shared_port port[]; /* per device port data array. */\n };\n \n@@ -1237,7 +1377,7 @@ int mlx5_hairpin_cap_get(struct rte_eth_dev *dev,\n bool mlx5_flex_parser_ecpri_exist(struct rte_eth_dev *dev);\n int mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev);\n int mlx5_flow_aso_age_mng_init(struct mlx5_dev_ctx_shared *sh);\n-int mlx5_aso_flow_mtrs_mng_init(struct mlx5_priv *priv);\n+int mlx5_aso_flow_mtrs_mng_init(struct mlx5_dev_ctx_shared *sh);\n \n /* mlx5_ethdev.c */\n \n@@ -1463,6 +1603,12 @@ int mlx5_flow_meter_attach(struct mlx5_priv *priv,\n \t\t\t   struct rte_flow_error *error);\n void mlx5_flow_meter_detach(struct mlx5_priv *priv,\n \t\t\t    struct mlx5_flow_meter_info *fm);\n+struct mlx5_flow_meter_policy *mlx5_flow_meter_policy_find\n+\t\t(struct rte_eth_dev *dev,\n+\t\tuint32_t policy_id,\n+\t\tuint32_t *policy_idx);\n+int mlx5_flow_meter_flush(struct rte_eth_dev *dev,\n+\t\t\t  struct rte_mtr_error *error);\n \n /* mlx5_os.c */\n struct rte_pci_driver;\ndiff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c\nindex ee2c351649..81e0e3b7a9 100644\n--- a/drivers/net/mlx5/mlx5_flow.c\n+++ b/drivers/net/mlx5/mlx5_flow.c\n@@ -1028,7 +1028,7 @@ flow_rxq_tunnel_ptype_update(struct mlx5_rxq_ctrl *rxq_ctrl)\n  * @param[in] dev_handle\n  *   Pointer to device flow handle structure.\n  */\n-static void\n+void\n flow_drv_rxq_flags_set(struct rte_eth_dev *dev,\n \t\t       struct mlx5_flow_handle *dev_handle)\n {\n@@ -4479,8 +4479,8 @@ flow_meter_split_prep(struct rte_eth_dev *dev,\n \t\tactions_pre++;\n \t\tjump_data = (struct rte_flow_action_jump *)actions_pre;\n \t\tjump_data->group = attr->transfer ?\n-\t\t\t\t(MLX5_FLOW_TABLE_LEVEL_METER - 1) :\n-\t\t\t\t MLX5_FLOW_TABLE_LEVEL_METER;\n+\t\t\t\t(MLX5_FLOW_TABLE_LEVEL_POLICY - 1) :\n+\t\t\t\t MLX5_FLOW_TABLE_LEVEL_POLICY;\n \t\thw_mtr_action->conf = jump_data;\n \t\tactions_pre = (struct rte_flow_action *)(jump_data + 1);\n \t} else {\n@@ -5079,8 +5079,8 @@ flow_create_split_metadata(struct rte_eth_dev *dev,\n \tif (qrss) {\n \t\t/* Check if it is in meter suffix table. */\n \t\tmtr_sfx = attr->group == (attr->transfer ?\n-\t\t\t  (MLX5_FLOW_TABLE_LEVEL_SUFFIX - 1) :\n-\t\t\t  MLX5_FLOW_TABLE_LEVEL_SUFFIX);\n+\t\t\t  (MLX5_FLOW_TABLE_LEVEL_METER - 1) :\n+\t\t\t  MLX5_FLOW_TABLE_LEVEL_METER);\n \t\t/*\n \t\t * Q/RSS action on NIC Rx should be split in order to pass by\n \t\t * the mreg copy table (RX_CP_TBL) and then it jumps to the\n@@ -5344,8 +5344,8 @@ flow_create_split_meter(struct rte_eth_dev *dev,\n \t\tdev_flow->handle->is_meter_flow_id = 1;\n \t\t/* Setting the sfx group atrr. */\n \t\tsfx_attr.group = sfx_attr.transfer ?\n-\t\t\t\t(MLX5_FLOW_TABLE_LEVEL_SUFFIX - 1) :\n-\t\t\t\t MLX5_FLOW_TABLE_LEVEL_SUFFIX;\n+\t\t\t\t(MLX5_FLOW_TABLE_LEVEL_METER - 1) :\n+\t\t\t\t MLX5_FLOW_TABLE_LEVEL_METER;\n \t\tflow_split_info->prefix_layers =\n \t\t\t\tflow_get_prefix_layer_flags(dev_flow);\n \t\tflow_split_info->prefix_mark = dev_flow->handle->mark;\n@@ -6608,6 +6608,169 @@ mlx5_flow_ops_get(struct rte_eth_dev *dev __rte_unused,\n \treturn 0;\n }\n \n+/**\n+ * Validate meter policy actions.\n+ * Dispatcher for action type specific validation.\n+ *\n+ * @param[in] dev\n+ *   Pointer to the Ethernet device structure.\n+ * @param[in] action\n+ *   The meter policy action object to validate.\n+ * @param[in] attr\n+ *   Attributes of flow to determine steering domain.\n+ * @param[out] is_rss\n+ *   Is RSS or not.\n+ * @param[out] domain_bitmap\n+ *   Domain bitmap.\n+ * @param[out] is_def_policy\n+ *   Is default policy or not.\n+ * @param[out] error\n+ *   Perform verbose error reporting if not NULL. Initialized in case of\n+ *   error only.\n+ *\n+ * @return\n+ *   0 on success, otherwise negative errno value.\n+ */\n+int\n+mlx5_flow_validate_mtr_acts(struct rte_eth_dev *dev,\n+\t\t\tconst struct rte_flow_action *actions[RTE_COLORS],\n+\t\t\tstruct rte_flow_attr *attr,\n+\t\t\tbool *is_rss,\n+\t\t\tuint8_t *domain_bitmap,\n+\t\t\tbool *is_def_policy,\n+\t\t\tstruct rte_mtr_error *error)\n+{\n+\tconst struct mlx5_flow_driver_ops *fops;\n+\n+\tfops = flow_get_drv_ops(MLX5_FLOW_TYPE_DV);\n+\treturn fops->validate_mtr_acts(dev, actions, attr,\n+\t\t\tis_rss, domain_bitmap, is_def_policy, error);\n+}\n+\n+/**\n+ * Destroy the meter table set.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ * @param[in] mtr_policy\n+ *   Meter policy struct.\n+ */\n+void\n+mlx5_flow_destroy_mtr_acts(struct rte_eth_dev *dev,\n+\t\t      struct mlx5_flow_meter_policy *mtr_policy)\n+{\n+\tconst struct mlx5_flow_driver_ops *fops;\n+\n+\tfops = flow_get_drv_ops(MLX5_FLOW_TYPE_DV);\n+\tfops->destroy_mtr_acts(dev, mtr_policy);\n+}\n+\n+/**\n+ * Create policy action, lock free,\n+ * (mutex should be acquired by caller).\n+ * Dispatcher for action type specific call.\n+ *\n+ * @param[in] dev\n+ *   Pointer to the Ethernet device structure.\n+ * @param[in] mtr_policy\n+ *   Meter policy struct.\n+ * @param[in] action\n+ *   Action specification used to create meter actions.\n+ * @param[out] error\n+ *   Perform verbose error reporting if not NULL. Initialized in case of\n+ *   error only.\n+ *\n+ * @return\n+ *   0 on success, otherwise negative errno value.\n+ */\n+int\n+mlx5_flow_create_mtr_acts(struct rte_eth_dev *dev,\n+\t\t      struct mlx5_flow_meter_policy *mtr_policy,\n+\t\t      const struct rte_flow_action *actions[RTE_COLORS],\n+\t\t      struct rte_mtr_error *error)\n+{\n+\tconst struct mlx5_flow_driver_ops *fops;\n+\n+\tfops = flow_get_drv_ops(MLX5_FLOW_TYPE_DV);\n+\treturn fops->create_mtr_acts(dev, mtr_policy, actions, error);\n+}\n+\n+/**\n+ * Create policy rules, lock free,\n+ * (mutex should be acquired by caller).\n+ * Dispatcher for action type specific call.\n+ *\n+ * @param[in] dev\n+ *   Pointer to the Ethernet device structure.\n+ * @param[in] mtr_policy\n+ *   Meter policy struct.\n+ *\n+ * @return\n+ *   0 on success, -1 otherwise.\n+ */\n+int\n+mlx5_flow_create_policy_rules(struct rte_eth_dev *dev,\n+\t\t\t     struct mlx5_flow_meter_policy *mtr_policy)\n+{\n+\tconst struct mlx5_flow_driver_ops *fops;\n+\n+\tfops = flow_get_drv_ops(MLX5_FLOW_TYPE_DV);\n+\treturn fops->create_policy_rules(dev, mtr_policy);\n+}\n+\n+/**\n+ * Destroy policy rules, lock free,\n+ * (mutex should be acquired by caller).\n+ * Dispatcher for action type specific call.\n+ *\n+ * @param[in] dev\n+ *   Pointer to the Ethernet device structure.\n+ * @param[in] mtr_policy\n+ *   Meter policy struct.\n+ */\n+void\n+mlx5_flow_destroy_policy_rules(struct rte_eth_dev *dev,\n+\t\t\t     struct mlx5_flow_meter_policy *mtr_policy)\n+{\n+\tconst struct mlx5_flow_driver_ops *fops;\n+\n+\tfops = flow_get_drv_ops(MLX5_FLOW_TYPE_DV);\n+\tfops->destroy_policy_rules(dev, mtr_policy);\n+}\n+\n+/**\n+ * Destroy the default policy table set.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ */\n+void\n+mlx5_flow_destroy_def_policy(struct rte_eth_dev *dev)\n+{\n+\tconst struct mlx5_flow_driver_ops *fops;\n+\n+\tfops = flow_get_drv_ops(MLX5_FLOW_TYPE_DV);\n+\tfops->destroy_def_policy(dev);\n+}\n+\n+/**\n+ * Destroy the default policy table set.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ *\n+ * @return\n+ *   0 on success, -1 otherwise.\n+ */\n+int\n+mlx5_flow_create_def_policy(struct rte_eth_dev *dev)\n+{\n+\tconst struct mlx5_flow_driver_ops *fops;\n+\n+\tfops = flow_get_drv_ops(MLX5_FLOW_TYPE_DV);\n+\treturn fops->create_def_policy(dev);\n+}\n+\n /**\n  * Create the needed meter and suffix tables.\n  *\n@@ -6647,6 +6810,21 @@ mlx5_flow_destroy_mtr_tbls(struct rte_eth_dev *dev,\n \treturn fops->destroy_mtr_tbls(dev, tbls);\n }\n \n+/**\n+ * Destroy the global meter drop table.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ */\n+void\n+mlx5_flow_destroy_mtr_drop_tbls(struct rte_eth_dev *dev)\n+{\n+\tconst struct mlx5_flow_driver_ops *fops;\n+\n+\tfops = flow_get_drv_ops(MLX5_FLOW_TYPE_DV);\n+\tfops->destroy_mtr_drop_tbls(dev);\n+}\n+\n /**\n  * Allocate the needed aso flow meter id.\n  *\ndiff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h\nindex 7fa15eef7b..51d040c4d2 100644\n--- a/drivers/net/mlx5/mlx5_flow.h\n+++ b/drivers/net/mlx5/mlx5_flow.h\n@@ -691,13 +691,6 @@ struct mlx5_flow_handle {\n #define MLX5_FLOW_HANDLE_VERBS_SIZE (sizeof(struct mlx5_flow_handle))\n #endif\n \n-/*\n- * Max number of actions per DV flow.\n- * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED\n- * in rdma-core file providers/mlx5/verbs.c.\n- */\n-#define MLX5_DV_MAX_NUMBER_OF_ACTIONS 8\n-\n /** Device flow structure only for DV flow creation. */\n struct mlx5_flow_dv_workspace {\n \tuint32_t group; /**< The group index. */\n@@ -1099,6 +1092,7 @@ typedef struct mlx5_meter_domains_infos *(*mlx5_flow_create_mtr_tbls_t)\n \t\t\t\t\t    (struct rte_eth_dev *dev);\n typedef int (*mlx5_flow_destroy_mtr_tbls_t)(struct rte_eth_dev *dev,\n \t\t\t\t\tstruct mlx5_meter_domains_infos *tbls);\n+typedef void (*mlx5_flow_destroy_mtr_drop_tbls_t)(struct rte_eth_dev *dev);\n typedef uint32_t (*mlx5_flow_mtr_alloc_t)\n \t\t\t\t\t    (struct rte_eth_dev *dev);\n typedef void (*mlx5_flow_mtr_free_t)(struct rte_eth_dev *dev,\n@@ -1144,6 +1138,32 @@ typedef int (*mlx5_flow_sync_domain_t)\n \t\t\t(struct rte_eth_dev *dev,\n \t\t\t uint32_t domains,\n \t\t\t uint32_t flags);\n+typedef int (*mlx5_flow_validate_mtr_acts_t)\n+\t\t\t(struct rte_eth_dev *dev,\n+\t\t\t const struct rte_flow_action *actions[RTE_COLORS],\n+\t\t\t struct rte_flow_attr *attr,\n+\t\t\t bool *is_rss,\n+\t\t\t uint8_t *domain_bitmap,\n+\t\t\t bool *is_def_policy,\n+\t\t\t struct rte_mtr_error *error);\n+typedef int (*mlx5_flow_create_mtr_acts_t)\n+\t\t\t(struct rte_eth_dev *dev,\n+\t\t      struct mlx5_flow_meter_policy *mtr_policy,\n+\t\t      const struct rte_flow_action *actions[RTE_COLORS],\n+\t\t      struct rte_mtr_error *error);\n+typedef void (*mlx5_flow_destroy_mtr_acts_t)\n+\t\t\t(struct rte_eth_dev *dev,\n+\t\t      struct mlx5_flow_meter_policy *mtr_policy);\n+typedef int (*mlx5_flow_create_policy_rules_t)\n+\t\t\t(struct rte_eth_dev *dev,\n+\t\t\t  struct mlx5_flow_meter_policy *mtr_policy);\n+typedef void (*mlx5_flow_destroy_policy_rules_t)\n+\t\t\t(struct rte_eth_dev *dev,\n+\t\t\t  struct mlx5_flow_meter_policy *mtr_policy);\n+typedef int (*mlx5_flow_create_def_policy_t)\n+\t\t\t(struct rte_eth_dev *dev);\n+typedef void (*mlx5_flow_destroy_def_policy_t)\n+\t\t\t(struct rte_eth_dev *dev);\n \n struct mlx5_flow_driver_ops {\n \tmlx5_flow_validate_t validate;\n@@ -1155,8 +1175,16 @@ struct mlx5_flow_driver_ops {\n \tmlx5_flow_query_t query;\n \tmlx5_flow_create_mtr_tbls_t create_mtr_tbls;\n \tmlx5_flow_destroy_mtr_tbls_t destroy_mtr_tbls;\n+\tmlx5_flow_destroy_mtr_drop_tbls_t destroy_mtr_drop_tbls;\n \tmlx5_flow_mtr_alloc_t create_meter;\n \tmlx5_flow_mtr_free_t free_meter;\n+\tmlx5_flow_validate_mtr_acts_t validate_mtr_acts;\n+\tmlx5_flow_create_mtr_acts_t create_mtr_acts;\n+\tmlx5_flow_destroy_mtr_acts_t destroy_mtr_acts;\n+\tmlx5_flow_create_policy_rules_t create_policy_rules;\n+\tmlx5_flow_destroy_policy_rules_t destroy_policy_rules;\n+\tmlx5_flow_create_def_policy_t create_def_policy;\n+\tmlx5_flow_destroy_def_policy_t destroy_def_policy;\n \tmlx5_flow_counter_alloc_t counter_alloc;\n \tmlx5_flow_counter_free_t counter_free;\n \tmlx5_flow_counter_query_t counter_query;\n@@ -1233,12 +1261,13 @@ static inline struct mlx5_aso_mtr *\n mlx5_aso_meter_by_idx(struct mlx5_priv *priv, uint32_t idx)\n {\n \tstruct mlx5_aso_mtr_pool *pool;\n-\tstruct mlx5_aso_mtr_pools_mng *mtrmng = priv->sh->mtrmng;\n+\tstruct mlx5_aso_mtr_pools_mng *pools_mng =\n+\t\t\t\t&priv->sh->mtrmng->pools_mng;\n \n \t/* Decrease to original index. */\n \tidx--;\n-\tMLX5_ASSERT(idx / MLX5_ASO_MTRS_PER_POOL < mtrmng->n);\n-\tpool = mtrmng->pools[idx / MLX5_ASO_MTRS_PER_POOL];\n+\tMLX5_ASSERT(idx / MLX5_ASO_MTRS_PER_POOL < pools_mng->n);\n+\tpool = pools_mng->pools[idx / MLX5_ASO_MTRS_PER_POOL];\n \treturn &pool->mtrs[idx % MLX5_ASO_MTRS_PER_POOL];\n }\n \n@@ -1384,8 +1413,7 @@ struct mlx5_meter_domains_infos *mlx5_flow_create_mtr_tbls\n \t\t\t\t\t(struct rte_eth_dev *dev);\n int mlx5_flow_destroy_mtr_tbls(struct rte_eth_dev *dev,\n \t\t\t       struct mlx5_meter_domains_infos *tbl);\n-int mlx5_flow_meter_flush(struct rte_eth_dev *dev,\n-\t\t\t  struct rte_mtr_error *error);\n+void mlx5_flow_destroy_mtr_drop_tbls(struct rte_eth_dev *dev);\n int mlx5_flow_dv_discover_counter_offset_support(struct rte_eth_dev *dev);\n int mlx5_shared_action_flush(struct rte_eth_dev *dev);\n void mlx5_release_tunnel_hub(struct mlx5_dev_ctx_shared *sh, uint16_t port_id);\n@@ -1486,4 +1514,25 @@ int mlx5_flow_os_set_specific_workspace(struct mlx5_flow_workspace *data);\n void mlx5_flow_os_release_workspace(void);\n uint32_t mlx5_flow_mtr_alloc(struct rte_eth_dev *dev);\n void mlx5_flow_mtr_free(struct rte_eth_dev *dev, uint32_t mtr_idx);\n+int mlx5_flow_validate_mtr_acts(struct rte_eth_dev *dev,\n+\t\t\tconst struct rte_flow_action *actions[RTE_COLORS],\n+\t\t\tstruct rte_flow_attr *attr,\n+\t\t\tbool *is_rss,\n+\t\t\tuint8_t *domain_bitmap,\n+\t\t\tbool *is_def_policy,\n+\t\t\tstruct rte_mtr_error *error);\n+void mlx5_flow_destroy_mtr_acts(struct rte_eth_dev *dev,\n+\t\t      struct mlx5_flow_meter_policy *mtr_policy);\n+int mlx5_flow_create_mtr_acts(struct rte_eth_dev *dev,\n+\t\t      struct mlx5_flow_meter_policy *mtr_policy,\n+\t\t      const struct rte_flow_action *actions[RTE_COLORS],\n+\t\t      struct rte_mtr_error *error);\n+int mlx5_flow_create_policy_rules(struct rte_eth_dev *dev,\n+\t\t\t     struct mlx5_flow_meter_policy *mtr_policy);\n+void mlx5_flow_destroy_policy_rules(struct rte_eth_dev *dev,\n+\t\t\t     struct mlx5_flow_meter_policy *mtr_policy);\n+int mlx5_flow_create_def_policy(struct rte_eth_dev *dev);\n+void mlx5_flow_destroy_def_policy(struct rte_eth_dev *dev);\n+void flow_drv_rxq_flags_set(struct rte_eth_dev *dev,\n+\t\t       struct mlx5_flow_handle *dev_handle);\n #endif /* RTE_PMD_MLX5_FLOW_H_ */\ndiff --git a/drivers/net/mlx5/mlx5_flow_aso.c b/drivers/net/mlx5/mlx5_flow_aso.c\nindex 62d2df054b..20cd4fe18c 100644\n--- a/drivers/net/mlx5/mlx5_flow_aso.c\n+++ b/drivers/net/mlx5/mlx5_flow_aso.c\n@@ -311,11 +311,11 @@ mlx5_aso_queue_init(struct mlx5_dev_ctx_shared *sh,\n \t\tmlx5_aso_age_init_sq(&sh->aso_age_mng->aso_sq);\n \t\tbreak;\n \tcase ASO_OPC_MOD_POLICER:\n-\t\tif (mlx5_aso_sq_create(sh->ctx, &sh->mtrmng->sq, 0,\n+\t\tif (mlx5_aso_sq_create(sh->ctx, &sh->mtrmng->pools_mng.sq, 0,\n \t\t\t\t  sh->tx_uar, sh->pdn, MLX5_ASO_QUEUE_LOG_DESC,\n \t\t\t\t  sh->sq_ts_format))\n \t\t\treturn -1;\n-\t\tmlx5_aso_mtr_init_sq(&sh->mtrmng->sq);\n+\t\tmlx5_aso_mtr_init_sq(&sh->mtrmng->pools_mng.sq);\n \t\tbreak;\n \tdefault:\n \t\tDRV_LOG(ERR, \"Unknown ASO operation mode\");\n@@ -342,7 +342,7 @@ mlx5_aso_queue_uninit(struct mlx5_dev_ctx_shared *sh,\n \t\tsq = &sh->aso_age_mng->aso_sq;\n \t\tbreak;\n \tcase ASO_OPC_MOD_POLICER:\n-\t\tsq = &sh->mtrmng->sq;\n+\t\tsq = &sh->mtrmng->pools_mng.sq;\n \t\tbreak;\n \tdefault:\n \t\tDRV_LOG(ERR, \"Unknown ASO operation mode\");\n@@ -798,7 +798,7 @@ int\n mlx5_aso_meter_update_by_wqe(struct mlx5_dev_ctx_shared *sh,\n \t\t\tstruct mlx5_aso_mtr *mtr)\n {\n-\tstruct mlx5_aso_sq *sq = &sh->mtrmng->sq;\n+\tstruct mlx5_aso_sq *sq = &sh->mtrmng->pools_mng.sq;\n \tuint32_t poll_wqe_times = MLX5_MTR_POLL_WQE_CQE_TIMES;\n \n \tdo {\n@@ -830,7 +830,7 @@ int\n mlx5_aso_mtr_wait(struct mlx5_dev_ctx_shared *sh,\n \t\t\tstruct mlx5_aso_mtr *mtr)\n {\n-\tstruct mlx5_aso_sq *sq = &sh->mtrmng->sq;\n+\tstruct mlx5_aso_sq *sq = &sh->mtrmng->pools_mng.sq;\n \tuint32_t poll_cqe_times = MLX5_MTR_POLL_WQE_CQE_TIMES;\n \n \tif (__atomic_load_n(&mtr->state, __ATOMIC_RELAXED) ==\ndiff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c\nindex af3397fb55..759731333d 100644\n--- a/drivers/net/mlx5/mlx5_flow_dv.c\n+++ b/drivers/net/mlx5/mlx5_flow_dv.c\n@@ -21,6 +21,8 @@\n #include <rte_gtp.h>\n #include <rte_eal_paging.h>\n #include <rte_mpls.h>\n+#include <rte_mtr.h>\n+#include <rte_mtr_driver.h>\n \n #include <mlx5_glue.h>\n #include <mlx5_devx_cmds.h>\n@@ -184,6 +186,31 @@ flow_dv_attr_init(const struct rte_flow_item *item, union flow_dv_attr *attr,\n \tattr->valid = 1;\n }\n \n+/**\n+ * Convert rte_mtr_color to mlx5 color.\n+ *\n+ * @param[in] rcol\n+ *   rte_mtr_color.\n+ *\n+ * @return\n+ *   mlx5 color.\n+ */\n+static int\n+rte_col_2_mlx5_col(enum rte_color rcol)\n+{\n+\tswitch (rcol) {\n+\tcase RTE_COLOR_GREEN:\n+\t\treturn MLX5_FLOW_COLOR_GREEN;\n+\tcase RTE_COLOR_YELLOW:\n+\t\treturn MLX5_FLOW_COLOR_YELLOW;\n+\tcase RTE_COLOR_RED:\n+\t\treturn MLX5_FLOW_COLOR_RED;\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\treturn MLX5_FLOW_COLOR_UNDEFINED;\n+}\n+\n struct field_modify_info {\n \tuint32_t size; /* Size of field in protocol header, in bytes. */\n \tuint32_t offset; /* Offset of field in protocol header, in bytes. */\n@@ -4695,10 +4722,6 @@ flow_dv_validate_action_jump(struct rte_eth_dev *dev,\n \t\t\t\t\t  RTE_FLOW_ERROR_TYPE_ACTION, NULL,\n \t\t\t\t\t  \"can't have 2 fate actions in\"\n \t\t\t\t\t  \" same flow\");\n-\tif (action_flags & MLX5_FLOW_ACTION_METER)\n-\t\treturn rte_flow_error_set(error, ENOTSUP,\n-\t\t\t\t\t  RTE_FLOW_ERROR_TYPE_ACTION, NULL,\n-\t\t\t\t\t  \"jump with meter not support\");\n \tif (!action->conf)\n \t\treturn rte_flow_error_set(error, EINVAL,\n \t\t\t\t\t  RTE_FLOW_ERROR_TYPE_ACTION_CONF,\n@@ -5928,9 +5951,10 @@ static int\n flow_dv_mtr_container_resize(struct rte_eth_dev *dev)\n {\n \tstruct mlx5_priv *priv = dev->data->dev_private;\n-\tstruct mlx5_aso_mtr_pools_mng *mtrmng = priv->sh->mtrmng;\n-\tvoid *old_pools = mtrmng->pools;\n-\tuint32_t resize = mtrmng->n + MLX5_MTRS_CONTAINER_RESIZE;\n+\tstruct mlx5_aso_mtr_pools_mng *pools_mng =\n+\t\t\t\t&priv->sh->mtrmng->pools_mng;\n+\tvoid *old_pools = pools_mng->pools;\n+\tuint32_t resize = pools_mng->n + MLX5_MTRS_CONTAINER_RESIZE;\n \tuint32_t mem_size = sizeof(struct mlx5_aso_mtr_pool *) * resize;\n \tvoid *pools = mlx5_malloc(MLX5_MEM_ZERO, mem_size, 0, SOCKET_ID_ANY);\n \n@@ -5938,16 +5962,16 @@ flow_dv_mtr_container_resize(struct rte_eth_dev *dev)\n \t\trte_errno = ENOMEM;\n \t\treturn -ENOMEM;\n \t}\n-\tif (!mtrmng->n)\n+\tif (!pools_mng->n)\n \t\tif (mlx5_aso_queue_init(priv->sh, ASO_OPC_MOD_POLICER)) {\n \t\t\tmlx5_free(pools);\n \t\t\treturn -ENOMEM;\n \t\t}\n \tif (old_pools)\n-\t\tmemcpy(pools, old_pools, mtrmng->n *\n+\t\tmemcpy(pools, old_pools, pools_mng->n *\n \t\t\t\t       sizeof(struct mlx5_aso_mtr_pool *));\n-\tmtrmng->n = resize;\n-\tmtrmng->pools = pools;\n+\tpools_mng->n = resize;\n+\tpools_mng->pools = pools;\n \tif (old_pools)\n \t\tmlx5_free(old_pools);\n \treturn 0;\n@@ -5970,7 +5994,8 @@ flow_dv_mtr_pool_create(struct rte_eth_dev *dev,\n \t\t\t     struct mlx5_aso_mtr **mtr_free)\n {\n \tstruct mlx5_priv *priv = dev->data->dev_private;\n-\tstruct mlx5_aso_mtr_pools_mng *mtrmng = priv->sh->mtrmng;\n+\tstruct mlx5_aso_mtr_pools_mng *pools_mng =\n+\t\t\t\t&priv->sh->mtrmng->pools_mng;\n \tstruct mlx5_aso_mtr_pool *pool = NULL;\n \tstruct mlx5_devx_obj *dcs = NULL;\n \tuint32_t i;\n@@ -5990,17 +6015,17 @@ flow_dv_mtr_pool_create(struct rte_eth_dev *dev,\n \t\treturn NULL;\n \t}\n \tpool->devx_obj = dcs;\n-\tpool->index = mtrmng->n_valid;\n-\tif (pool->index == mtrmng->n && flow_dv_mtr_container_resize(dev)) {\n+\tpool->index = pools_mng->n_valid;\n+\tif (pool->index == pools_mng->n && flow_dv_mtr_container_resize(dev)) {\n \t\tmlx5_free(pool);\n \t\tclaim_zero(mlx5_devx_cmd_destroy(dcs));\n \t\treturn NULL;\n \t}\n-\tmtrmng->pools[pool->index] = pool;\n-\tmtrmng->n_valid++;\n+\tpools_mng->pools[pool->index] = pool;\n+\tpools_mng->n_valid++;\n \tfor (i = 1; i < MLX5_ASO_MTRS_PER_POOL; ++i) {\n \t\tpool->mtrs[i].offset = i;\n-\t\tLIST_INSERT_HEAD(&mtrmng->meters,\n+\t\tLIST_INSERT_HEAD(&pools_mng->meters,\n \t\t\t\t\t\t&pool->mtrs[i], next);\n \t}\n \tpool->mtrs[0].offset = 0;\n@@ -6020,15 +6045,16 @@ static void\n flow_dv_aso_mtr_release_to_pool(struct rte_eth_dev *dev, uint32_t mtr_idx)\n {\n \tstruct mlx5_priv *priv = dev->data->dev_private;\n-\tstruct mlx5_aso_mtr_pools_mng *mtrmng = priv->sh->mtrmng;\n+\tstruct mlx5_aso_mtr_pools_mng *pools_mng =\n+\t\t\t\t&priv->sh->mtrmng->pools_mng;\n \tstruct mlx5_aso_mtr *aso_mtr = mlx5_aso_meter_by_idx(priv, mtr_idx);\n \n \tMLX5_ASSERT(aso_mtr);\n-\trte_spinlock_lock(&mtrmng->mtrsl);\n+\trte_spinlock_lock(&pools_mng->mtrsl);\n \tmemset(&aso_mtr->fm, 0, sizeof(struct mlx5_flow_meter_info));\n \taso_mtr->state = ASO_METER_FREE;\n-\tLIST_INSERT_HEAD(&mtrmng->meters, aso_mtr, next);\n-\trte_spinlock_unlock(&mtrmng->mtrsl);\n+\tLIST_INSERT_HEAD(&pools_mng->meters, aso_mtr, next);\n+\trte_spinlock_unlock(&pools_mng->mtrsl);\n }\n \n /**\n@@ -6045,7 +6071,8 @@ flow_dv_mtr_alloc(struct rte_eth_dev *dev)\n {\n \tstruct mlx5_priv *priv = dev->data->dev_private;\n \tstruct mlx5_aso_mtr *mtr_free = NULL;\n-\tstruct mlx5_aso_mtr_pools_mng *mtrmng = priv->sh->mtrmng;\n+\tstruct mlx5_aso_mtr_pools_mng *pools_mng =\n+\t\t\t\t&priv->sh->mtrmng->pools_mng;\n \tstruct mlx5_aso_mtr_pool *pool;\n \tuint32_t mtr_idx = 0;\n \n@@ -6055,16 +6082,16 @@ flow_dv_mtr_alloc(struct rte_eth_dev *dev)\n \t}\n \t/* Allocate the flow meter memory. */\n \t/* Get free meters from management. */\n-\trte_spinlock_lock(&mtrmng->mtrsl);\n-\tmtr_free = LIST_FIRST(&mtrmng->meters);\n+\trte_spinlock_lock(&pools_mng->mtrsl);\n+\tmtr_free = LIST_FIRST(&pools_mng->meters);\n \tif (mtr_free)\n \t\tLIST_REMOVE(mtr_free, next);\n \tif (!mtr_free && !flow_dv_mtr_pool_create(dev, &mtr_free)) {\n-\t\trte_spinlock_unlock(&mtrmng->mtrsl);\n+\t\trte_spinlock_unlock(&pools_mng->mtrsl);\n \t\treturn 0;\n \t}\n \tmtr_free->state = ASO_METER_WAIT;\n-\trte_spinlock_unlock(&mtrmng->mtrsl);\n+\trte_spinlock_unlock(&pools_mng->mtrsl);\n \tpool = container_of(mtr_free,\n \t\t\tstruct mlx5_aso_mtr_pool,\n \t\t\tmtrs[mtr_free->offset]);\n@@ -13390,6 +13417,556 @@ flow_dv_action_query(struct rte_eth_dev *dev,\n \t}\n }\n \n+/**\n+ * Destroy the meter sub policy table rules.\n+ * Lock free, (mutex should be acquired by caller).\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ * @param[in] sub_policy\n+ *   Pointer to meter sub policy table.\n+ */\n+static void\n+__flow_dv_destroy_sub_policy_rules(struct rte_eth_dev *dev,\n+\t\t\t     struct mlx5_flow_meter_sub_policy *sub_policy)\n+{\n+\tstruct mlx5_flow_tbl_data_entry *tbl;\n+\tint i;\n+\n+\tfor (i = 0; i < RTE_COLORS; i++) {\n+\t\tif (sub_policy->color_rule[i]) {\n+\t\t\tclaim_zero(mlx5_flow_os_destroy_flow\n+\t\t\t\t(sub_policy->color_rule[i]));\n+\t\t\tsub_policy->color_rule[i] = NULL;\n+\t\t}\n+\t\tif (sub_policy->color_matcher[i]) {\n+\t\t\ttbl = container_of(sub_policy->color_matcher[i]->tbl,\n+\t\t\t\ttypeof(*tbl), tbl);\n+\t\t\tmlx5_cache_unregister(&tbl->matchers,\n+\t\t\t\t      &sub_policy->color_matcher[i]->entry);\n+\t\t\tsub_policy->color_matcher[i] = NULL;\n+\t\t}\n+\t}\n+\tfor (i = 0; i < MLX5_MTR_RTE_COLORS; i++) {\n+\t\tif (sub_policy->rix_hrxq[i]) {\n+\t\t\tmlx5_hrxq_release(dev, sub_policy->rix_hrxq[i]);\n+\t\t\tsub_policy->rix_hrxq[i] = 0;\n+\t\t}\n+\t\tif (sub_policy->jump_tbl[i]) {\n+\t\t\tflow_dv_tbl_resource_release(MLX5_SH(dev),\n+\t\t\tsub_policy->jump_tbl[i]);\n+\t\t\tsub_policy->jump_tbl[i] = NULL;\n+\t\t}\n+\t}\n+\tif (sub_policy->tbl_rsc) {\n+\t\tflow_dv_tbl_resource_release(MLX5_SH(dev),\n+\t\t\tsub_policy->tbl_rsc);\n+\t\tsub_policy->tbl_rsc = NULL;\n+\t}\n+}\n+\n+/**\n+ * Destroy policy rules, lock free,\n+ * (mutex should be acquired by caller).\n+ * Dispatcher for action type specific call.\n+ *\n+ * @param[in] dev\n+ *   Pointer to the Ethernet device structure.\n+ * @param[in] mtr_policy\n+ *   Meter policy struct.\n+ */\n+static void\n+flow_dv_destroy_policy_rules(struct rte_eth_dev *dev,\n+\t\t      struct mlx5_flow_meter_policy *mtr_policy)\n+{\n+\tuint32_t i, j;\n+\tstruct mlx5_flow_meter_sub_policy *sub_policy;\n+\tuint16_t sub_policy_num;\n+\n+\tfor (i = 0; i < MLX5_MTR_DOMAIN_MAX; i++) {\n+\t\tsub_policy_num = (mtr_policy->sub_policy_num >>\n+\t\t\t(MLX5_MTR_SUB_POLICY_NUM_SHIFT * i)) &\n+\t\t\tMLX5_MTR_SUB_POLICY_NUM_MASK;\n+\t\tfor (j = 0; j < sub_policy_num; j++) {\n+\t\t\tsub_policy = mtr_policy->sub_policys[i][j];\n+\t\t\tif (sub_policy)\n+\t\t\t\t__flow_dv_destroy_sub_policy_rules\n+\t\t\t\t\t\t(dev, sub_policy);\n+\t\t}\n+\t}\n+}\n+\n+/**\n+ * Destroy policy action, lock free,\n+ * (mutex should be acquired by caller).\n+ * Dispatcher for action type specific call.\n+ *\n+ * @param[in] dev\n+ *   Pointer to the Ethernet device structure.\n+ * @param[in] mtr_policy\n+ *   Meter policy struct.\n+ */\n+static void\n+flow_dv_destroy_mtr_policy_acts(struct rte_eth_dev *dev,\n+\t\t      struct mlx5_flow_meter_policy *mtr_policy)\n+{\n+\tstruct rte_flow_action *rss_action;\n+\tstruct mlx5_flow_handle dev_handle;\n+\tuint32_t i, j;\n+\n+\tfor (i = 0; i < MLX5_MTR_RTE_COLORS; i++) {\n+\t\tif (mtr_policy->act_cnt[i].rix_mark) {\n+\t\t\tflow_dv_tag_release(dev,\n+\t\t\t\tmtr_policy->act_cnt[i].rix_mark);\n+\t\t\tmtr_policy->act_cnt[i].rix_mark = 0;\n+\t\t}\n+\t\tif (mtr_policy->act_cnt[i].modify_hdr) {\n+\t\t\tdev_handle.dvh.modify_hdr =\n+\t\t\t\tmtr_policy->act_cnt[i].modify_hdr;\n+\t\t\tflow_dv_modify_hdr_resource_release(dev, &dev_handle);\n+\t\t}\n+\t\tswitch (mtr_policy->act_cnt[i].fate_action) {\n+\t\tcase MLX5_FLOW_FATE_SHARED_RSS:\n+\t\t\trss_action = mtr_policy->act_cnt[i].rss;\n+\t\t\tmlx5_free(rss_action);\n+\t\t\tbreak;\n+\t\tcase MLX5_FLOW_FATE_PORT_ID:\n+\t\t\tif (mtr_policy->act_cnt[i].rix_port_id_action) {\n+\t\t\t\tflow_dv_port_id_action_resource_release(dev,\n+\t\t\t\tmtr_policy->act_cnt[i].rix_port_id_action);\n+\t\t\t\tmtr_policy->act_cnt[i].rix_port_id_action = 0;\n+\t\t\t}\n+\t\t\tbreak;\n+\t\tcase MLX5_FLOW_FATE_DROP:\n+\t\tcase MLX5_FLOW_FATE_JUMP:\n+\t\t\tfor (j = 0; j < MLX5_MTR_DOMAIN_MAX; j++)\n+\t\t\t\tmtr_policy->act_cnt[i].dr_jump_action[j] =\n+\t\t\t\t\t\tNULL;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\t/*Queue action do nothing*/\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tfor (j = 0; j < MLX5_MTR_DOMAIN_MAX; j++)\n+\t\tmtr_policy->dr_drop_action[j] = NULL;\n+}\n+\n+/**\n+ * Create policy action per domain, lock free,\n+ * (mutex should be acquired by caller).\n+ * Dispatcher for action type specific call.\n+ *\n+ * @param[in] dev\n+ *   Pointer to the Ethernet device structure.\n+ * @param[in] mtr_policy\n+ *   Meter policy struct.\n+ * @param[in] action\n+ *   Action specification used to create meter actions.\n+ * @param[out] error\n+ *   Perform verbose error reporting if not NULL. Initialized in case of\n+ *   error only.\n+ *\n+ * @return\n+ *   0 on success, otherwise negative errno value.\n+ */\n+static int\n+__flow_dv_create_domain_policy_acts(struct rte_eth_dev *dev,\n+\t\t\tstruct mlx5_flow_meter_policy *mtr_policy,\n+\t\t\tconst struct rte_flow_action *actions[RTE_COLORS],\n+\t\t\tenum mlx5_meter_domain domain,\n+\t\t\tstruct rte_mtr_error *error)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct rte_flow_error flow_err;\n+\tconst struct rte_flow_action *act;\n+\tuint64_t action_flags = 0;\n+\tstruct mlx5_flow_handle dh;\n+\tstruct mlx5_flow dev_flow;\n+\tstruct mlx5_flow_dv_port_id_action_resource port_id_action;\n+\tint i, ret;\n+\tuint8_t egress, transfer;\n+\tstruct mlx5_meter_policy_action_container *act_cnt = NULL;\n+\tunion {\n+\t\tstruct mlx5_flow_dv_modify_hdr_resource res;\n+\t\tuint8_t len[sizeof(struct mlx5_flow_dv_modify_hdr_resource) +\n+\t\t\t    sizeof(struct mlx5_modification_cmd) *\n+\t\t\t    (MLX5_MAX_MODIFY_NUM + 1)];\n+\t} mhdr_dummy;\n+\n+\tegress = (domain == MLX5_MTR_DOMAIN_EGRESS) ? 1 : 0;\n+\ttransfer = (domain == MLX5_MTR_DOMAIN_TRANSFER) ? 1 : 0;\n+\tmemset(&dh, 0, sizeof(struct mlx5_flow_handle));\n+\tmemset(&dev_flow, 0, sizeof(struct mlx5_flow));\n+\tmemset(&port_id_action, 0,\n+\t\tsizeof(struct mlx5_flow_dv_port_id_action_resource));\n+\tdev_flow.handle = &dh;\n+\tdev_flow.dv.port_id_action = &port_id_action;\n+\tdev_flow.external = true;\n+\tfor (i = 0; i < RTE_COLORS; i++) {\n+\t\tif (i < MLX5_MTR_RTE_COLORS)\n+\t\t\tact_cnt = &mtr_policy->act_cnt[i];\n+\t\tfor (act = actions[i];\n+\t\t\tact && act->type != RTE_FLOW_ACTION_TYPE_END;\n+\t\t\tact++) {\n+\t\t\tswitch (act->type) {\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_MARK:\n+\t\t\t{\n+\t\t\t\tuint32_t tag_be = mlx5_flow_mark_set\n+\t\t\t\t\t(((const struct rte_flow_action_mark *)\n+\t\t\t\t\t(act->conf))->id);\n+\n+\t\t\t\tif (i >= MLX5_MTR_RTE_COLORS)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL,\n+\t\t\t\t\t  \"cannot create policy \"\n+\t\t\t\t\t  \"mark action for this color\");\n+\t\t\t\tdev_flow.handle->mark = 1;\n+\t\t\t\tif (flow_dv_tag_resource_register(dev, tag_be,\n+\t\t\t\t\t\t  &dev_flow, &flow_err))\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL,\n+\t\t\t\t\t\"cannot setup policy mark action\");\n+\t\t\t\tMLX5_ASSERT(dev_flow.dv.tag_resource);\n+\t\t\t\tact_cnt->rix_mark =\n+\t\t\t\t\tdev_flow.handle->dvh.rix_tag;\n+\t\t\t\tif (action_flags & MLX5_FLOW_ACTION_QUEUE) {\n+\t\t\t\t\tdev_flow.handle->rix_hrxq =\n+\t\t\tmtr_policy->sub_policys[domain][0]->rix_hrxq[i];\n+\t\t\t\t\tflow_drv_rxq_flags_set(dev,\n+\t\t\t\t\t\tdev_flow.handle);\n+\t\t\t\t}\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_MARK;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_SET_TAG:\n+\t\t\t{\n+\t\t\t\tstruct mlx5_flow_dv_modify_hdr_resource\n+\t\t\t\t\t*mhdr_res = &mhdr_dummy.res;\n+\n+\t\t\t\tif (i >= MLX5_MTR_RTE_COLORS)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL,\n+\t\t\t\t\t  \"cannot create policy \"\n+\t\t\t\t\t  \"set tag action for this color\");\n+\t\t\t\tmemset(mhdr_res, 0, sizeof(*mhdr_res));\n+\t\t\t\tmhdr_res->ft_type = transfer ?\n+\t\t\t\t\tMLX5DV_FLOW_TABLE_TYPE_FDB :\n+\t\t\t\t\tegress ?\n+\t\t\t\t\tMLX5DV_FLOW_TABLE_TYPE_NIC_TX :\n+\t\t\t\t\tMLX5DV_FLOW_TABLE_TYPE_NIC_RX;\n+\t\t\t\tif (flow_dv_convert_action_set_tag\n+\t\t\t\t(dev, mhdr_res,\n+\t\t\t\t(const struct rte_flow_action_set_tag *)\n+\t\t\t\tact->conf,  &flow_err))\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"cannot convert policy \"\n+\t\t\t\t\t\"set tag action\");\n+\t\t\t\tif (!mhdr_res->actions_num)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"cannot find policy \"\n+\t\t\t\t\t\"set tag action\");\n+\t\t\t\t/* create modify action if needed. */\n+\t\t\t\tdev_flow.dv.group = 1;\n+\t\t\t\tif (flow_dv_modify_hdr_resource_register\n+\t\t\t\t\t(dev, mhdr_res, &dev_flow, &flow_err))\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"cannot register policy \"\n+\t\t\t\t\t\"set tag action\");\n+\t\t\t\tact_cnt->modify_hdr =\n+\t\t\t\tdev_flow.handle->dvh.modify_hdr;\n+\t\t\t\tif (action_flags & MLX5_FLOW_ACTION_QUEUE) {\n+\t\t\t\t\tdev_flow.handle->rix_hrxq =\n+\t\t\t\tmtr_policy->sub_policys[domain][0]->rix_hrxq[i];\n+\t\t\t\t\tflow_drv_rxq_flags_set(dev,\n+\t\t\t\t\t\tdev_flow.handle);\n+\t\t\t\t}\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_SET_TAG;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_DROP:\n+\t\t\t{\n+\t\t\t\tstruct mlx5_flow_mtr_mng *mtrmng =\n+\t\t\t\t\t\tpriv->sh->mtrmng;\n+\t\t\t\tstruct mlx5_flow_tbl_data_entry *tbl_data;\n+\n+\t\t\t\t/*\n+\t\t\t\t * Create the drop table with\n+\t\t\t\t * METER DROP level.\n+\t\t\t\t */\n+\t\t\t\tif (!mtrmng->drop_tbl[domain]) {\n+\t\t\t\t\tmtrmng->drop_tbl[domain] =\n+\t\t\t\t\tflow_dv_tbl_resource_get(dev,\n+\t\t\t\t\tMLX5_FLOW_TABLE_LEVEL_METER,\n+\t\t\t\t\tegress, transfer, false, NULL, 0,\n+\t\t\t\t\t0, MLX5_MTR_TABLE_ID_DROP, &flow_err);\n+\t\t\t\t\tif (!mtrmng->drop_tbl[domain])\n+\t\t\t\t\t\treturn -rte_mtr_error_set\n+\t\t\t\t\t(error, ENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL,\n+\t\t\t\t\t\"Failed to create meter drop table\");\n+\t\t\t\t}\n+\t\t\t\ttbl_data = container_of\n+\t\t\t\t(mtrmng->drop_tbl[domain],\n+\t\t\t\tstruct mlx5_flow_tbl_data_entry, tbl);\n+\t\t\t\tif (i < MLX5_MTR_RTE_COLORS) {\n+\t\t\t\t\tact_cnt->dr_jump_action[domain] =\n+\t\t\t\t\t\ttbl_data->jump.action;\n+\t\t\t\t\tact_cnt->fate_action =\n+\t\t\t\t\t\tMLX5_FLOW_FATE_DROP;\n+\t\t\t\t}\n+\t\t\t\tif (i == RTE_COLOR_RED)\n+\t\t\t\t\tmtr_policy->dr_drop_action[domain] =\n+\t\t\t\t\t\ttbl_data->jump.action;\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_DROP;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_QUEUE:\n+\t\t\t{\n+\t\t\t\tstruct mlx5_hrxq *hrxq;\n+\t\t\t\tuint32_t hrxq_idx;\n+\t\t\t\tstruct mlx5_flow_rss_desc rss_desc;\n+\t\t\t\tstruct mlx5_flow_meter_sub_policy *sub_policy =\n+\t\t\t\tmtr_policy->sub_policys[domain][0];\n+\n+\t\t\t\tif (i >= MLX5_MTR_RTE_COLORS)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"cannot create policy \"\n+\t\t\t\t\t\"fate queue for this color\");\n+\t\t\t\tmemset(&rss_desc, 0,\n+\t\t\t\t\tsizeof(struct mlx5_flow_rss_desc));\n+\t\t\t\trss_desc.queue_num = 1;\n+\t\t\t\trss_desc.const_q = act->conf;\n+\t\t\t\thrxq = flow_dv_hrxq_prepare(dev, &dev_flow,\n+\t\t\t\t\t\t    &rss_desc, &hrxq_idx);\n+\t\t\t\tif (!hrxq)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL,\n+\t\t\t\t\t\"cannot create policy fate queue\");\n+\t\t\t\tsub_policy->rix_hrxq[i] = hrxq_idx;\n+\t\t\t\tact_cnt->fate_action =\n+\t\t\t\t\tMLX5_FLOW_FATE_QUEUE;\n+\t\t\t\tdev_flow.handle->fate_action =\n+\t\t\t\t\tMLX5_FLOW_FATE_QUEUE;\n+\t\t\t\tif (action_flags & MLX5_FLOW_ACTION_MARK ||\n+\t\t\t\t    action_flags & MLX5_FLOW_ACTION_SET_TAG) {\n+\t\t\t\t\tdev_flow.handle->rix_hrxq = hrxq_idx;\n+\t\t\t\t\tflow_drv_rxq_flags_set(dev,\n+\t\t\t\t\t\tdev_flow.handle);\n+\t\t\t\t}\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_QUEUE;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_RSS:\n+\t\t\t{\n+\t\t\t\tint rss_size;\n+\n+\t\t\t\tif (i >= MLX5_MTR_RTE_COLORS)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL,\n+\t\t\t\t\t  \"cannot create policy \"\n+\t\t\t\t\t  \"rss action for this color\");\n+\t\t\t\t/*\n+\t\t\t\t * Save RSS conf into policy struct\n+\t\t\t\t * for translate stage.\n+\t\t\t\t */\n+\t\t\t\trss_size = (int)rte_flow_conv\n+\t\t\t\t\t(RTE_FLOW_CONV_OP_ACTION,\n+\t\t\t\t\tNULL, 0, act, &flow_err);\n+\t\t\t\tif (rss_size <= 0)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, \"Get the wrong \"\n+\t\t\t\t\t  \"rss action struct size\");\n+\t\t\t\tact_cnt->rss = mlx5_malloc(MLX5_MEM_ZERO,\n+\t\t\t\t\t\trss_size, 0, SOCKET_ID_ANY);\n+\t\t\t\tif (!act_cnt->rss)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL,\n+\t\t\t\t\t  \"Fail to malloc rss action memory\");\n+\t\t\t\tret = rte_flow_conv(RTE_FLOW_CONV_OP_ACTION,\n+\t\t\t\t\tact_cnt->rss, rss_size,\n+\t\t\t\t\tact, &flow_err);\n+\t\t\t\tif (ret < 0)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, \"Fail to save \"\n+\t\t\t\t\t  \"rss action into policy struct\");\n+\t\t\t\tact_cnt->fate_action =\n+\t\t\t\t\tMLX5_FLOW_FATE_SHARED_RSS;\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_RSS;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_PORT_ID:\n+\t\t\t{\n+\t\t\t\tstruct mlx5_flow_dv_port_id_action_resource\n+\t\t\t\t\tport_id_resource;\n+\t\t\t\tuint32_t port_id = 0;\n+\n+\t\t\t\tif (i >= MLX5_MTR_RTE_COLORS)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"cannot create policy \"\n+\t\t\t\t\t\"port action for this color\");\n+\t\t\t\tmemset(&port_id_resource, 0,\n+\t\t\t\t\tsizeof(port_id_resource));\n+\t\t\t\tif (flow_dv_translate_action_port_id(dev, act,\n+\t\t\t\t\t\t&port_id, &flow_err))\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"cannot translate \"\n+\t\t\t\t\t\"policy port action\");\n+\t\t\t\tport_id_resource.port_id = port_id;\n+\t\t\t\tif (flow_dv_port_id_action_resource_register\n+\t\t\t\t\t(dev, &port_id_resource,\n+\t\t\t\t\t&dev_flow, &flow_err))\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"cannot setup \"\n+\t\t\t\t\t\"policy port action\");\n+\t\t\t\tact_cnt->rix_port_id_action =\n+\t\t\t\t\tdev_flow.handle->rix_port_id_action;\n+\t\t\t\tact_cnt->fate_action =\n+\t\t\t\t\tMLX5_FLOW_FATE_PORT_ID;\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_PORT_ID;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_JUMP:\n+\t\t\t{\n+\t\t\t\tuint32_t jump_group = 0;\n+\t\t\t\tuint32_t table = 0;\n+\t\t\t\tstruct mlx5_flow_tbl_data_entry *tbl_data;\n+\t\t\t\tstruct flow_grp_info grp_info = {\n+\t\t\t\t\t.external = !!dev_flow.external,\n+\t\t\t\t\t.transfer = !!transfer,\n+\t\t\t\t\t.fdb_def_rule = !!priv->fdb_def_rule,\n+\t\t\t\t\t.std_tbl_fix = 0,\n+\t\t\t\t\t.skip_scale = dev_flow.skip_scale &\n+\t\t\t\t\t(1 << MLX5_SCALE_FLOW_GROUP_BIT),\n+\t\t\t\t};\n+\t\t\t\tstruct mlx5_flow_meter_sub_policy *sub_policy =\n+\t\t\t\tmtr_policy->sub_policys[domain][0];\n+\n+\t\t\t\tif (i >= MLX5_MTR_RTE_COLORS)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL,\n+\t\t\t\t\t  \"cannot create policy \"\n+\t\t\t\t\t  \"jump action for this color\");\n+\t\t\t\tjump_group =\n+\t\t\t\t((const struct rte_flow_action_jump *)\n+\t\t\t\t\t\t\tact->conf)->group;\n+\t\t\t\tif (mlx5_flow_group_to_table(dev, NULL,\n+\t\t\t\t\t\t       jump_group,\n+\t\t\t\t\t\t       &table,\n+\t\t\t\t\t\t       &grp_info, &flow_err))\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"cannot setup \"\n+\t\t\t\t\t\"policy jump action\");\n+\t\t\t\tsub_policy->jump_tbl[i] =\n+\t\t\t\tflow_dv_tbl_resource_get(dev,\n+\t\t\t\t\ttable, egress,\n+\t\t\t\t\ttransfer,\n+\t\t\t\t\t!!dev_flow.external,\n+\t\t\t\t\tNULL, jump_group, 0,\n+\t\t\t\t\t0, &flow_err);\n+\t\t\t\tif\n+\t\t\t\t(!sub_policy->jump_tbl[i])\n+\t\t\t\t\treturn  -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"cannot create jump action.\");\n+\t\t\t\ttbl_data = container_of\n+\t\t\t\t(sub_policy->jump_tbl[i],\n+\t\t\t\tstruct mlx5_flow_tbl_data_entry, tbl);\n+\t\t\t\tact_cnt->dr_jump_action[domain] =\n+\t\t\t\t\ttbl_data->jump.action;\n+\t\t\t\tact_cnt->fate_action =\n+\t\t\t\t\tMLX5_FLOW_FATE_JUMP;\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_JUMP;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tdefault:\n+\t\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, \"action type not supported\");\n+\t\t\t}\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+/**\n+ * Create policy action per domain, lock free,\n+ * (mutex should be acquired by caller).\n+ * Dispatcher for action type specific call.\n+ *\n+ * @param[in] dev\n+ *   Pointer to the Ethernet device structure.\n+ * @param[in] mtr_policy\n+ *   Meter policy struct.\n+ * @param[in] action\n+ *   Action specification used to create meter actions.\n+ * @param[out] error\n+ *   Perform verbose error reporting if not NULL. Initialized in case of\n+ *   error only.\n+ *\n+ * @return\n+ *   0 on success, otherwise negative errno value.\n+ */\n+static int\n+flow_dv_create_mtr_policy_acts(struct rte_eth_dev *dev,\n+\t\t      struct mlx5_flow_meter_policy *mtr_policy,\n+\t\t      const struct rte_flow_action *actions[RTE_COLORS],\n+\t\t      struct rte_mtr_error *error)\n+{\n+\tint ret, i;\n+\tuint16_t sub_policy_num;\n+\n+\tfor (i = 0; i < MLX5_MTR_DOMAIN_MAX; i++) {\n+\t\tsub_policy_num = (mtr_policy->sub_policy_num >>\n+\t\t\t(MLX5_MTR_SUB_POLICY_NUM_SHIFT * i)) &\n+\t\t\tMLX5_MTR_SUB_POLICY_NUM_MASK;\n+\t\tif (sub_policy_num) {\n+\t\t\tret = __flow_dv_create_domain_policy_acts(dev,\n+\t\t\t\tmtr_policy, actions,\n+\t\t\t\t(enum mlx5_meter_domain)i, error);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n /**\n  * Query a dv flow  rule for its statistics via devx.\n  *\n@@ -13570,9 +14147,484 @@ flow_dv_destroy_mtr_tbl(struct rte_eth_dev *dev,\n \treturn 0;\n }\n \n+static void\n+flow_dv_destroy_mtr_drop_tbls(struct rte_eth_dev *dev)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_flow_mtr_mng *mtrmng = priv->sh->mtrmng;\n+\tstruct mlx5_flow_tbl_data_entry *tbl;\n+\tint i;\n+\n+\tfor (i = 0; i < MLX5_MTR_DOMAIN_MAX; i++) {\n+\t\tif (mtrmng->def_rule[i]) {\n+\t\t\tclaim_zero(mlx5_flow_os_destroy_flow\n+\t\t\t\t\t(mtrmng->def_rule[i]));\n+\t\t\tmtrmng->def_rule[i] = NULL;\n+\t\t}\n+\t\tif (mtrmng->def_matcher[i]) {\n+\t\t\ttbl = container_of(mtrmng->def_matcher[i]->tbl,\n+\t\t\t\tstruct mlx5_flow_tbl_data_entry, tbl);\n+\t\t\tmlx5_cache_unregister(&tbl->matchers,\n+\t\t\t\t      &mtrmng->def_matcher[i]->entry);\n+\t\t\tmtrmng->def_matcher[i] = NULL;\n+\t\t}\n+\t\tif (mtrmng->drop_matcher[i]) {\n+\t\t\ttbl = container_of(mtrmng->drop_matcher[i]->tbl,\n+\t\t\t\tstruct mlx5_flow_tbl_data_entry, tbl);\n+\t\t\tmlx5_cache_unregister(&tbl->matchers,\n+\t\t\t\t      &mtrmng->drop_matcher[i]->entry);\n+\t\t\tmtrmng->drop_matcher[i] = NULL;\n+\t\t}\n+\t\tif (mtrmng->drop_tbl[i]) {\n+\t\t\tflow_dv_tbl_resource_release(MLX5_SH(dev),\n+\t\t\t\tmtrmng->drop_tbl[i]);\n+\t\t\tmtrmng->drop_tbl[i] = NULL;\n+\t\t}\n+\t}\n+}\n+\n /* Number of meter flow actions, count and jump or count and drop. */\n #define METER_ACTIONS 2\n \n+static void\n+__flow_dv_destroy_domain_def_policy(struct rte_eth_dev *dev,\n+\t\t\t      enum mlx5_meter_domain domain)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_flow_meter_def_policy *def_policy =\n+\t\t\tpriv->sh->mtrmng->def_policy[domain];\n+\n+\t__flow_dv_destroy_sub_policy_rules(dev, &def_policy->sub_policy);\n+\tmlx5_free(def_policy);\n+\tpriv->sh->mtrmng->def_policy[domain] = NULL;\n+}\n+\n+/**\n+ * Destroy the default policy table set.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ */\n+static void\n+flow_dv_destroy_def_policy(struct rte_eth_dev *dev)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tint i;\n+\n+\tfor (i = 0; i < MLX5_MTR_DOMAIN_MAX; i++)\n+\t\tif (priv->sh->mtrmng->def_policy[i])\n+\t\t\t__flow_dv_destroy_domain_def_policy(dev,\n+\t\t\t\t\t(enum mlx5_meter_domain)i);\n+\tpriv->sh->mtrmng->def_policy_id = MLX5_INVALID_POLICY_ID;\n+}\n+\n+static int\n+__flow_dv_create_policy_flow(struct rte_eth_dev *dev,\n+\t\t\tuint32_t color_reg_c_idx,\n+\t\t\tenum rte_color color, void *matcher_object,\n+\t\t\tint actions_n, void *actions,\n+\t\t\tbool is_default_policy, void **rule,\n+\t\t\tconst struct rte_flow_attr *attr)\n+{\n+\tint ret;\n+\tstruct mlx5_flow_dv_match_params value = {\n+\t\t.size = sizeof(value.buf) -\n+\t\t\tMLX5_ST_SZ_BYTES(fte_match_set_misc4),\n+\t};\n+\tstruct mlx5_flow_dv_match_params matcher = {\n+\t\t.size = sizeof(matcher.buf) -\n+\t\t\tMLX5_ST_SZ_BYTES(fte_match_set_misc4),\n+\t};\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\n+\tif (!is_default_policy && (priv->representor || priv->master)) {\n+\t\tif (flow_dv_translate_item_port_id(dev, matcher.buf,\n+\t\t\t\t\t\t   value.buf, NULL, attr)) {\n+\t\t\tDRV_LOG(ERR,\n+\t\t\t\"Failed to create meter policy flow with port.\");\n+\t\t\treturn -1;\n+\t\t}\n+\t}\n+\tflow_dv_match_meta_reg(matcher.buf, value.buf,\n+\t\t\t\t(enum modify_reg)color_reg_c_idx,\n+\t\t\t\trte_col_2_mlx5_col(color),\n+\t\t\t\tUINT32_MAX);\n+\tret = mlx5_flow_os_create_flow(matcher_object,\n+\t\t\t(void *)&value, actions_n, actions, rule);\n+\tif (ret) {\n+\t\tDRV_LOG(ERR, \"Failed to create meter policy flow.\");\n+\t\treturn -1;\n+\t}\n+\treturn 0;\n+}\n+\n+static int\n+__flow_dv_create_policy_matcher(struct rte_eth_dev *dev,\n+\t\t\tuint32_t color_reg_c_idx,\n+\t\t\tuint16_t priority,\n+\t\t\tstruct mlx5_flow_meter_sub_policy *sub_policy,\n+\t\t\tconst struct rte_flow_attr *attr,\n+\t\t\tbool is_default_policy,\n+\t\t\tstruct rte_flow_error *error)\n+{\n+\tstruct mlx5_cache_entry *entry;\n+\tstruct mlx5_flow_tbl_resource *tbl_rsc = sub_policy->tbl_rsc;\n+\tstruct mlx5_flow_dv_matcher matcher = {\n+\t\t.mask = {\n+\t\t\t.size = sizeof(matcher.mask.buf) -\n+\t\t\t\tMLX5_ST_SZ_BYTES(fte_match_set_misc4),\n+\t\t},\n+\t\t.tbl = tbl_rsc,\n+\t};\n+\tstruct mlx5_flow_dv_match_params value = {\n+\t\t.size = sizeof(value.buf) -\n+\t\t\tMLX5_ST_SZ_BYTES(fte_match_set_misc4),\n+\t};\n+\tstruct mlx5_flow_cb_ctx ctx = {\n+\t\t.error = error,\n+\t\t.data = &matcher,\n+\t};\n+\tstruct mlx5_flow_tbl_data_entry *tbl_data;\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tuint32_t color_mask = (UINT32_C(1) << MLX5_MTR_COLOR_BITS) - 1;\n+\n+\tif (!is_default_policy && (priv->representor || priv->master)) {\n+\t\tif (flow_dv_translate_item_port_id(dev, matcher.mask.buf,\n+\t\t\t\t\t\t   value.buf, NULL, attr)) {\n+\t\t\tDRV_LOG(ERR,\n+\t\t\t\"Failed to register meter drop matcher with port.\");\n+\t\t\treturn -1;\n+\t\t}\n+\t}\n+\ttbl_data = container_of(tbl_rsc, struct mlx5_flow_tbl_data_entry, tbl);\n+\tif (priority < RTE_COLOR_RED)\n+\t\tflow_dv_match_meta_reg(matcher.mask.buf, value.buf,\n+\t\t\t(enum modify_reg)color_reg_c_idx, 0, color_mask);\n+\tmatcher.priority = priority;\n+\tmatcher.crc = rte_raw_cksum((const void *)matcher.mask.buf,\n+\t\t\t\t\tmatcher.mask.size);\n+\tentry = mlx5_cache_register(&tbl_data->matchers, &ctx);\n+\tif (!entry) {\n+\t\tDRV_LOG(ERR, \"Failed to register meter drop matcher.\");\n+\t\treturn -1;\n+\t}\n+\tsub_policy->color_matcher[priority] =\n+\t\tcontainer_of(entry, struct mlx5_flow_dv_matcher, entry);\n+\treturn 0;\n+}\n+\n+/**\n+ * Create the policy rules per domain.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ * @param[in] sub_policy\n+ *    Pointer to sub policy table..\n+ * @param[in] egress\n+ *   Direction of the table.\n+ * @param[in] transfer\n+ *   E-Switch or NIC flow.\n+ * @param[in] acts\n+ *   Pointer to policy action list per color.\n+ *\n+ * @return\n+ *   0 on success, -1 otherwise.\n+ */\n+static int\n+__flow_dv_create_domain_policy_rules(struct rte_eth_dev *dev,\n+\t\tstruct mlx5_flow_meter_sub_policy *sub_policy,\n+\t\tuint8_t egress, uint8_t transfer, bool is_default_policy,\n+\t\tstruct mlx5_meter_policy_acts acts[RTE_COLORS])\n+{\n+\tstruct rte_flow_error flow_err;\n+\tuint32_t color_reg_c_idx;\n+\tstruct rte_flow_attr attr = {\n+\t\t.group = MLX5_FLOW_TABLE_LEVEL_POLICY,\n+\t\t.priority = 0,\n+\t\t.ingress = 0,\n+\t\t.egress = !!egress,\n+\t\t.transfer = !!transfer,\n+\t\t.reserved = 0,\n+\t};\n+\tint i;\n+\tint ret = mlx5_flow_get_reg_id(dev, MLX5_MTR_COLOR, 0, &flow_err);\n+\n+\tif (ret < 0)\n+\t\treturn -1;\n+\t/* Create policy table with POLICY level. */\n+\tif (!sub_policy->tbl_rsc)\n+\t\tsub_policy->tbl_rsc = flow_dv_tbl_resource_get(dev,\n+\t\t\t\tMLX5_FLOW_TABLE_LEVEL_POLICY,\n+\t\t\t\tegress, transfer, false, NULL, 0, 0,\n+\t\t\t\tsub_policy->idx, &flow_err);\n+\tif (!sub_policy->tbl_rsc) {\n+\t\tDRV_LOG(ERR,\n+\t\t\t\"Failed to create meter sub policy table.\");\n+\t\treturn -1;\n+\t}\n+\t/* Prepare matchers. */\n+\tcolor_reg_c_idx = ret;\n+\tfor (i = 0; i < RTE_COLORS; i++) {\n+\t\tif (i == RTE_COLOR_YELLOW || !acts[i].actions_n)\n+\t\t\tcontinue;\n+\t\tattr.priority = i;\n+\t\tif (!sub_policy->color_matcher[i]) {\n+\t\t\t/* Create matchers for Color. */\n+\t\t\tif (__flow_dv_create_policy_matcher(dev,\n+\t\t\t\tcolor_reg_c_idx, i, sub_policy,\n+\t\t\t\t&attr, is_default_policy, &flow_err))\n+\t\t\t\treturn -1;\n+\t\t}\n+\t\t/* Create flow, matching color. */\n+\t\tif (acts[i].actions_n)\n+\t\t\tif (__flow_dv_create_policy_flow(dev,\n+\t\t\t\tcolor_reg_c_idx, (enum rte_color)i,\n+\t\t\t\tsub_policy->color_matcher[i]->matcher_object,\n+\t\t\t\tacts[i].actions_n,\n+\t\t\t\tacts[i].dv_actions,\n+\t\t\t\tis_default_policy,\n+\t\t\t\t&sub_policy->color_rule[i],\n+\t\t\t\t&attr))\n+\t\t\t\treturn -1;\n+\t}\n+\treturn 0;\n+}\n+\n+static int\n+__flow_dv_create_policy_acts_rules(struct rte_eth_dev *dev,\n+\t\t\tstruct mlx5_flow_meter_policy *mtr_policy,\n+\t\t\tstruct mlx5_flow_meter_sub_policy *sub_policy,\n+\t\t\tuint32_t domain)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_meter_policy_acts acts[RTE_COLORS];\n+\tstruct mlx5_flow_dv_tag_resource *tag;\n+\tstruct mlx5_flow_dv_port_id_action_resource *port_action;\n+\tstruct mlx5_hrxq *hrxq;\n+\tuint8_t egress, transfer;\n+\tint i;\n+\n+\tfor (i = 0; i < RTE_COLORS; i++) {\n+\t\tacts[i].actions_n = 0;\n+\t\tif (i == RTE_COLOR_YELLOW)\n+\t\t\tcontinue;\n+\t\tif (i == RTE_COLOR_RED) {\n+\t\t\t/* Only support drop on red. */\n+\t\t\tacts[i].dv_actions[0] =\n+\t\t\tmtr_policy->dr_drop_action[domain];\n+\t\t\tacts[i].actions_n = 1;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (mtr_policy->act_cnt[i].rix_mark) {\n+\t\t\ttag = mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_TAG],\n+\t\t\t\t\tmtr_policy->act_cnt[i].rix_mark);\n+\t\t\tif (!tag) {\n+\t\t\t\tDRV_LOG(ERR, \"Failed to find \"\n+\t\t\t\t\"mark action for policy.\");\n+\t\t\t\treturn -1;\n+\t\t\t}\n+\t\t\tacts[i].dv_actions[acts[i].actions_n] =\n+\t\t\t\t\t\ttag->action;\n+\t\t\tacts[i].actions_n++;\n+\t\t}\n+\t\tif (mtr_policy->act_cnt[i].modify_hdr) {\n+\t\t\tacts[i].dv_actions[acts[i].actions_n] =\n+\t\t\tmtr_policy->act_cnt[i].modify_hdr->action;\n+\t\t\tacts[i].actions_n++;\n+\t\t}\n+\t\tif (mtr_policy->act_cnt[i].fate_action) {\n+\t\t\tswitch (mtr_policy->act_cnt[i].fate_action) {\n+\t\t\tcase MLX5_FLOW_FATE_PORT_ID:\n+\t\t\t\tport_action = mlx5_ipool_get\n+\t\t\t\t\t(priv->sh->ipool[MLX5_IPOOL_PORT_ID],\n+\t\t\t\tmtr_policy->act_cnt[i].rix_port_id_action);\n+\t\t\t\tif (!port_action) {\n+\t\t\t\t\tDRV_LOG(ERR, \"Failed to find \"\n+\t\t\t\t\t\t\"port action for policy.\");\n+\t\t\t\t\treturn -1;\n+\t\t\t\t}\n+\t\t\t\tacts[i].dv_actions[acts[i].actions_n] =\n+\t\t\t\tport_action->action;\n+\t\t\t\tacts[i].actions_n++;\n+\t\t\t\tbreak;\n+\t\t\tcase MLX5_FLOW_FATE_DROP:\n+\t\t\tcase MLX5_FLOW_FATE_JUMP:\n+\t\t\t\tacts[i].dv_actions[acts[i].actions_n] =\n+\t\t\t\tmtr_policy->act_cnt[i].dr_jump_action[domain];\n+\t\t\t\tacts[i].actions_n++;\n+\t\t\t\tbreak;\n+\t\t\tcase MLX5_FLOW_FATE_SHARED_RSS:\n+\t\t\tcase MLX5_FLOW_FATE_QUEUE:\n+\t\t\t\thrxq = mlx5_ipool_get\n+\t\t\t\t(priv->sh->ipool[MLX5_IPOOL_HRXQ],\n+\t\t\t\tsub_policy->rix_hrxq[i]);\n+\t\t\t\tif (!hrxq) {\n+\t\t\t\t\tDRV_LOG(ERR, \"Failed to find \"\n+\t\t\t\t\t\t\"queue action for policy.\");\n+\t\t\t\t\treturn -1;\n+\t\t\t\t}\n+\t\t\t\tacts[i].dv_actions[acts[i].actions_n] =\n+\t\t\t\thrxq->action;\n+\t\t\t\tacts[i].actions_n++;\n+\t\t\t\tbreak;\n+\t\t\tdefault:\n+\t\t\t\t/*Queue action do nothing*/\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t}\n+\tegress = (domain == MLX5_MTR_DOMAIN_EGRESS) ? 1 : 0;\n+\ttransfer = (domain == MLX5_MTR_DOMAIN_TRANSFER) ? 1 : 0;\n+\tif (__flow_dv_create_domain_policy_rules(dev, sub_policy,\n+\t\t\t\tegress, transfer, false, acts)) {\n+\t\tDRV_LOG(ERR,\n+\t\t\"Failed to create policy rules per domain.\");\n+\t\treturn -1;\n+\t}\n+\treturn 0;\n+}\n+\n+/**\n+ * Create the policy rules.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ * @param[in,out] mtr_policy\n+ *   Pointer to meter policy table.\n+ *\n+ * @return\n+ *   0 on success, -1 otherwise.\n+ */\n+static int\n+flow_dv_create_policy_rules(struct rte_eth_dev *dev,\n+\t\t\t     struct mlx5_flow_meter_policy *mtr_policy)\n+{\n+\tint i;\n+\tuint16_t sub_policy_num;\n+\n+\tfor (i = 0; i < MLX5_MTR_DOMAIN_MAX; i++) {\n+\t\tsub_policy_num = (mtr_policy->sub_policy_num >>\n+\t\t\t(MLX5_MTR_SUB_POLICY_NUM_SHIFT * i)) &\n+\t\t\tMLX5_MTR_SUB_POLICY_NUM_MASK;\n+\t\tif (!sub_policy_num)\n+\t\t\tcontinue;\n+\t\t/* Prepare actions list and create policy rules. */\n+\t\tif (__flow_dv_create_policy_acts_rules(dev, mtr_policy,\n+\t\t\tmtr_policy->sub_policys[i][0], i)) {\n+\t\t\tDRV_LOG(ERR,\n+\t\t\t\"Failed to create policy action list per domain.\");\n+\t\t\treturn -1;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static int\n+__flow_dv_create_domain_def_policy(struct rte_eth_dev *dev, uint32_t domain)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_flow_mtr_mng *mtrmng = priv->sh->mtrmng;\n+\tstruct mlx5_flow_meter_def_policy *def_policy;\n+\tstruct mlx5_flow_tbl_resource *jump_tbl;\n+\tstruct mlx5_flow_tbl_data_entry *tbl_data;\n+\tuint8_t egress, transfer;\n+\tstruct rte_flow_error error;\n+\tstruct mlx5_meter_policy_acts acts[RTE_COLORS];\n+\tint ret;\n+\n+\tegress = (domain == MLX5_MTR_DOMAIN_EGRESS) ? 1 : 0;\n+\ttransfer = (domain == MLX5_MTR_DOMAIN_TRANSFER) ? 1 : 0;\n+\tdef_policy = mtrmng->def_policy[domain];\n+\tif (!def_policy) {\n+\t\tdef_policy = mlx5_malloc(MLX5_MEM_ZERO,\n+\t\t\tsizeof(struct mlx5_flow_meter_def_policy),\n+\t\t\tRTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);\n+\t\tif (!def_policy) {\n+\t\t\tDRV_LOG(ERR, \"Failed to alloc \"\n+\t\t\t\t\t\"default policy table.\");\n+\t\t\tgoto def_policy_error;\n+\t\t}\n+\t\tmtrmng->def_policy[domain] = def_policy;\n+\t\t/* Create the meter suffix table with SUFFIX level. */\n+\t\tjump_tbl = flow_dv_tbl_resource_get(dev,\n+\t\t\t\tMLX5_FLOW_TABLE_LEVEL_METER,\n+\t\t\t\tegress, transfer, false, NULL, 0,\n+\t\t\t\t0, MLX5_MTR_TABLE_ID_SUFFIX, &error);\n+\t\tif (!jump_tbl) {\n+\t\t\tDRV_LOG(ERR,\n+\t\t\t\t\"Failed to create meter suffix table.\");\n+\t\t\tgoto def_policy_error;\n+\t\t}\n+\t\tdef_policy->sub_policy.jump_tbl[RTE_COLOR_GREEN] = jump_tbl;\n+\t\ttbl_data = container_of(jump_tbl,\n+\t\t\t\tstruct mlx5_flow_tbl_data_entry, tbl);\n+\t\tdef_policy->dr_jump_action[RTE_COLOR_GREEN] =\n+\t\t\t\t\t\ttbl_data->jump.action;\n+\t\tacts[RTE_COLOR_GREEN].dv_actions[0] =\n+\t\t\t\t\t\ttbl_data->jump.action;\n+\t\tacts[RTE_COLOR_GREEN].actions_n = 1;\n+\t\t/* Create jump action to the drop table. */\n+\t\tif (!mtrmng->drop_tbl[domain]) {\n+\t\t\tmtrmng->drop_tbl[domain] = flow_dv_tbl_resource_get\n+\t\t\t\t(dev, MLX5_FLOW_TABLE_LEVEL_METER,\n+\t\t\t\tegress, transfer, false, NULL, 0,\n+\t\t\t\t0, MLX5_MTR_TABLE_ID_DROP, &error);\n+\t\t\tif (!mtrmng->drop_tbl[domain]) {\n+\t\t\t\tDRV_LOG(ERR, \"Failed to create \"\n+\t\t\t\t\"meter drop table for default policy.\");\n+\t\t\t\tgoto def_policy_error;\n+\t\t\t}\n+\t\t}\n+\t\ttbl_data = container_of(mtrmng->drop_tbl[domain],\n+\t\t\t\tstruct mlx5_flow_tbl_data_entry, tbl);\n+\t\tdef_policy->dr_jump_action[RTE_COLOR_RED] =\n+\t\t\t\t\t\ttbl_data->jump.action;\n+\t\tacts[RTE_COLOR_RED].dv_actions[0] = tbl_data->jump.action;\n+\t\tacts[RTE_COLOR_RED].actions_n = 1;\n+\t\t/* Create default policy rules. */\n+\t\tret = __flow_dv_create_domain_policy_rules(dev,\n+\t\t\t\t\t&def_policy->sub_policy,\n+\t\t\t\t\tegress, transfer, true, acts);\n+\t\tif (ret) {\n+\t\t\tDRV_LOG(ERR, \"Failed to create \"\n+\t\t\t\t\"default policy rules.\");\n+\t\t\t\tgoto def_policy_error;\n+\t\t}\n+\t}\n+\treturn 0;\n+def_policy_error:\n+\t__flow_dv_destroy_domain_def_policy(dev,\n+\t\t\t(enum mlx5_meter_domain)domain);\n+\treturn -1;\n+}\n+\n+/**\n+ * Create the default policy table set.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ * @return\n+ *   0 on success, -1 otherwise.\n+ */\n+static int\n+flow_dv_create_def_policy(struct rte_eth_dev *dev)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tint i;\n+\n+\t/* Non-termination policy table. */\n+\tfor (i = 0; i < MLX5_MTR_DOMAIN_MAX; i++) {\n+\t\tif (!priv->config.dv_esw_en && i == MLX5_MTR_DOMAIN_TRANSFER)\n+\t\t\tcontinue;\n+\t\tif (__flow_dv_create_domain_def_policy(dev, i)) {\n+\t\t\tDRV_LOG(ERR,\n+\t\t\t\"Failed to create default policy\");\n+\t\t\treturn -1;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n /**\n  * Create specify domain meter table and suffix table.\n  *\n@@ -13602,19 +14654,11 @@ flow_dv_prepare_mtr_tables(struct rte_eth_dev *dev,\n \t\tdtb = &mtb->egress;\n \telse\n \t\tdtb = &mtb->ingress;\n-\t/* Create the meter table with METER level. */\n-\tdtb->tbl = flow_dv_tbl_resource_get(dev, MLX5_FLOW_TABLE_LEVEL_METER,\n-\t\t\t\t\t    egress, transfer, false, NULL, 0,\n-\t\t\t\t\t    0, 0, &error);\n-\tif (!dtb->tbl) {\n-\t\tDRV_LOG(ERR, \"Failed to create meter policer table.\");\n-\t\treturn -1;\n-\t}\n \t/* Create the meter suffix table with SUFFIX level. */\n \tdtb->sfx_tbl = flow_dv_tbl_resource_get(dev,\n-\t\t\t\t\t    MLX5_FLOW_TABLE_LEVEL_SUFFIX,\n-\t\t\t\t\t    egress, transfer, false, NULL, 0,\n-\t\t\t\t\t    0, 0, &error);\n+\t\t\t\t\tMLX5_FLOW_TABLE_LEVEL_METER,\n+\t\t\t\t\tegress, transfer, false, NULL, 0,\n+\t\t\t\t\t0, MLX5_MTR_TABLE_ID_SUFFIX, &error);\n \tif (!dtb->sfx_tbl) {\n \t\tDRV_LOG(ERR, \"Failed to create meter suffix table.\");\n \t\treturn -1;\n@@ -13933,6 +14977,292 @@ flow_dv_action_validate(struct rte_eth_dev *dev,\n \t}\n }\n \n+/**\n+ * Validate meter policy actions.\n+ * Dispatcher for action type specific validation.\n+ *\n+ * @param[in] dev\n+ *   Pointer to the Ethernet device structure.\n+ * @param[in] action\n+ *   The meter policy action object to validate.\n+ * @param[in] attr\n+ *   Attributes of flow to determine steering domain.\n+ * @param[out] error\n+ *   Perform verbose error reporting if not NULL. Initialized in case of\n+ *   error only.\n+ *\n+ * @return\n+ *   0 on success, otherwise negative errno value.\n+ */\n+static int\n+flow_dv_validate_mtr_policy_acts(struct rte_eth_dev *dev,\n+\t\t\tconst struct rte_flow_action *actions[RTE_COLORS],\n+\t\t\tstruct rte_flow_attr *attr,\n+\t\t\tbool *is_rss,\n+\t\t\tuint8_t *domain_bitmap,\n+\t\t\tbool *is_def_policy,\n+\t\t\tstruct rte_mtr_error *error)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_dev_config *dev_conf = &priv->config;\n+\tconst struct rte_flow_action *act;\n+\tuint64_t action_flags = 0;\n+\tint actions_n;\n+\tint i, ret;\n+\tstruct rte_flow_error flow_err;\n+\tuint8_t domain_color[RTE_COLORS] = {0};\n+\tuint8_t def_domain = MLX5_MTR_ALL_DOMAIN_BIT;\n+\n+\tif (!priv->config.dv_esw_en)\n+\t\tdef_domain &= ~MLX5_MTR_DOMAIN_TRANSFER_BIT;\n+\t*domain_bitmap = def_domain;\n+\tif (actions[RTE_COLOR_YELLOW] &&\n+\t\tactions[RTE_COLOR_YELLOW]->type != RTE_FLOW_ACTION_TYPE_END)\n+\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\tNULL,\n+\t\t\t\t\"Yellow color does not support any action.\");\n+\tif (actions[RTE_COLOR_YELLOW] &&\n+\t\tactions[RTE_COLOR_YELLOW]->type != RTE_FLOW_ACTION_TYPE_DROP)\n+\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\tNULL, \"Red color only supports drop action.\");\n+\t/*\n+\t * Check default policy actions:\n+\t * Green/Yellow: no action, Red: drop action\n+\t */\n+\tif ((!actions[RTE_COLOR_GREEN] ||\n+\t\tactions[RTE_COLOR_GREEN]->type == RTE_FLOW_ACTION_TYPE_END)) {\n+\t\t*is_def_policy = true;\n+\t\treturn 0;\n+\t}\n+\tflow_err.message = NULL;\n+\tfor (i = 0; i < RTE_COLORS; i++) {\n+\t\tact = actions[i];\n+\t\tfor (action_flags = 0, actions_n = 0;\n+\t\t\tact && act->type != RTE_FLOW_ACTION_TYPE_END;\n+\t\t\tact++) {\n+\t\t\tif (actions_n == MLX5_DV_MAX_NUMBER_OF_ACTIONS)\n+\t\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, \"too many actions\");\n+\t\t\tswitch (act->type) {\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_PORT_ID:\n+\t\t\t\tif (!priv->config.dv_esw_en)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"PORT action validate check\"\n+\t\t\t\t\t\" fail for ESW disable\");\n+\t\t\t\tret = flow_dv_validate_action_port_id(dev,\n+\t\t\t\t\t\taction_flags,\n+\t\t\t\t\t\tact, attr, &flow_err);\n+\t\t\t\tif (ret)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, flow_err.message ?\n+\t\t\t\t\tflow_err.message :\n+\t\t\t\t\t\"PORT action validate check fail\");\n+\t\t\t\t++actions_n;\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_PORT_ID;\n+\t\t\t\tbreak;\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_MARK:\n+\t\t\t\tret = flow_dv_validate_action_mark(dev, act,\n+\t\t\t\t\t\t\t   action_flags,\n+\t\t\t\t\t\t\t   attr, &flow_err);\n+\t\t\t\tif (ret < 0)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, flow_err.message ?\n+\t\t\t\t\tflow_err.message :\n+\t\t\t\t\t\"Mark action validate check fail\");\n+\t\t\t\tif (dev_conf->dv_xmeta_en !=\n+\t\t\t\t\tMLX5_XMETA_MODE_LEGACY)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"Extend MARK action is \"\n+\t\t\t\t\t\"not supported. Please try use \"\n+\t\t\t\t\t\"default policy for meter.\");\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_MARK;\n+\t\t\t\t++actions_n;\n+\t\t\t\tbreak;\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_SET_TAG:\n+\t\t\t\tret = flow_dv_validate_action_set_tag(dev,\n+\t\t\t\t\t\t\tact, action_flags,\n+\t\t\t\t\t\t\tattr, &flow_err);\n+\t\t\t\tif (ret)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, flow_err.message ?\n+\t\t\t\t\tflow_err.message :\n+\t\t\t\t\t\"Set tag action validate check fail\");\n+\t\t\t\t/*\n+\t\t\t\t * Count all modify-header actions\n+\t\t\t\t * as one action.\n+\t\t\t\t */\n+\t\t\t\tif (!(action_flags &\n+\t\t\t\t\tMLX5_FLOW_MODIFY_HDR_ACTIONS))\n+\t\t\t\t\t++actions_n;\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_SET_TAG;\n+\t\t\t\tbreak;\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_DROP:\n+\t\t\t\tret = mlx5_flow_validate_action_drop\n+\t\t\t\t\t(action_flags,\n+\t\t\t\t\tattr, &flow_err);\n+\t\t\t\tif (ret < 0)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, flow_err.message ?\n+\t\t\t\t\tflow_err.message :\n+\t\t\t\t\t\"Drop action validate check fail\");\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_DROP;\n+\t\t\t\t++actions_n;\n+\t\t\t\tbreak;\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_QUEUE:\n+\t\t\t\t/*\n+\t\t\t\t * Check whether extensive\n+\t\t\t\t * metadata feature is engaged.\n+\t\t\t\t */\n+\t\t\t\tif (dev_conf->dv_flow_en &&\n+\t\t\t\t\t(dev_conf->dv_xmeta_en !=\n+\t\t\t\t\tMLX5_XMETA_MODE_LEGACY) &&\n+\t\t\t\t\tmlx5_flow_ext_mreg_supported(dev))\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, \"Queue action with meta \"\n+\t\t\t\t\t  \"is not supported. Please try use \"\n+\t\t\t\t\t  \"default policy for meter.\");\n+\t\t\t\tret = mlx5_flow_validate_action_queue(act,\n+\t\t\t\t\t\t\taction_flags, dev,\n+\t\t\t\t\t\t\tattr, &flow_err);\n+\t\t\t\tif (ret < 0)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, flow_err.message ?\n+\t\t\t\t\t  flow_err.message :\n+\t\t\t\t\t  \"Queue action validate check fail\");\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_QUEUE;\n+\t\t\t\t++actions_n;\n+\t\t\t\tbreak;\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_RSS:\n+\t\t\t\tif (dev_conf->dv_flow_en &&\n+\t\t\t\t\t(dev_conf->dv_xmeta_en !=\n+\t\t\t\t\tMLX5_XMETA_MODE_LEGACY) &&\n+\t\t\t\t\tmlx5_flow_ext_mreg_supported(dev))\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, \"RSS action with meta \"\n+\t\t\t\t\t  \"is not supported. Please try use \"\n+\t\t\t\t\t  \"default policy for meter.\");\n+\t\t\t\tret = mlx5_validate_action_rss(dev, act,\n+\t\t\t\t\t\t&flow_err);\n+\t\t\t\tif (ret < 0)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, flow_err.message ?\n+\t\t\t\t\t  flow_err.message :\n+\t\t\t\t\t  \"RSS action validate check fail\");\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_RSS;\n+\t\t\t\t++actions_n;\n+\t\t\t\t*is_rss = true;\n+\t\t\t\tbreak;\n+\t\t\tcase RTE_FLOW_ACTION_TYPE_JUMP:\n+\t\t\t\tret = flow_dv_validate_action_jump(dev,\n+\t\t\t\t\tNULL, act, action_flags,\n+\t\t\t\t\tattr, true, &flow_err);\n+\t\t\t\tif (ret)\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t  ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, flow_err.message ?\n+\t\t\t\t\t  flow_err.message :\n+\t\t\t\t\t  \"Jump action validate check fail\");\n+\t\t\t\t++actions_n;\n+\t\t\t\taction_flags |= MLX5_FLOW_ACTION_JUMP;\n+\t\t\t\tbreak;\n+\t\t\tdefault:\n+\t\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL,\n+\t\t\t\t\t\"Doesn't support optional action\");\n+\t\t\t}\n+\t\t}\n+\t\t/* Yellow is not supported, just skip. */\n+\t\tif (i == RTE_COLOR_YELLOW)\n+\t\t\tcontinue;\n+\t\tif (action_flags & MLX5_FLOW_ACTION_PORT_ID)\n+\t\t\tdomain_color[i] = MLX5_MTR_DOMAIN_TRANSFER_BIT;\n+\t\telse if ((action_flags &\n+\t\t\t(MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_QUEUE)) ||\n+\t\t\t(action_flags & MLX5_FLOW_ACTION_MARK))\n+\t\t\t/*\n+\t\t\t * Only support MLX5_XMETA_MODE_LEGACY\n+\t\t\t * so MARK action only in ingress domain.\n+\t\t\t */\n+\t\t\tdomain_color[i] = MLX5_MTR_DOMAIN_INGRESS_BIT;\n+\t\telse\n+\t\t\tdomain_color[i] = def_domain;\n+\t\t/*\n+\t\t * Validate the drop action mutual exclusion\n+\t\t * with other actions. Drop action is mutually-exclusive\n+\t\t * with any other action, except for Count action.\n+\t\t */\n+\t\tif ((action_flags & MLX5_FLOW_ACTION_DROP) &&\n+\t\t\t(action_flags & ~MLX5_FLOW_ACTION_DROP)) {\n+\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\tNULL, \"Drop action is mutually-exclusive \"\n+\t\t\t\t\"with any other action\");\n+\t\t}\n+\t\t/* Eswitch has few restrictions on using items and actions */\n+\t\tif (domain_color[i] & MLX5_MTR_DOMAIN_TRANSFER_BIT) {\n+\t\t\tif (!mlx5_flow_ext_mreg_supported(dev) &&\n+\t\t\t\taction_flags & MLX5_FLOW_ACTION_MARK)\n+\t\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"unsupported action MARK\");\n+\t\t\tif (action_flags & MLX5_FLOW_ACTION_QUEUE)\n+\t\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"unsupported action QUEUE\");\n+\t\t\tif (action_flags & MLX5_FLOW_ACTION_RSS)\n+\t\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"unsupported action RSS\");\n+\t\t\tif (!(action_flags & MLX5_FLOW_FATE_ESWITCH_ACTIONS))\n+\t\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"no fate action is found\");\n+\t\t} else {\n+\t\t\tif (!(action_flags & MLX5_FLOW_FATE_ACTIONS) &&\n+\t\t\t\t(domain_color[i] &\n+\t\t\t\tMLX5_MTR_DOMAIN_INGRESS_BIT)) {\n+\t\t\t\tif ((domain_color[i] &\n+\t\t\t\t\tMLX5_MTR_DOMAIN_EGRESS_BIT))\n+\t\t\t\t\tdomain_color[i] =\n+\t\t\t\t\tMLX5_MTR_DOMAIN_EGRESS_BIT;\n+\t\t\t\telse\n+\t\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\tENOTSUP,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\tNULL, \"no fate action is found\");\n+\t\t\t}\n+\t\t}\n+\t\tif (domain_color[i] != def_domain)\n+\t\t\t*domain_bitmap = domain_color[i];\n+\t}\n+\treturn 0;\n+}\n+\n static int\n flow_dv_sync_domain(struct rte_eth_dev *dev, uint32_t domains, uint32_t flags)\n {\n@@ -13968,8 +15298,16 @@ const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops = {\n \t.query = flow_dv_query,\n \t.create_mtr_tbls = flow_dv_create_mtr_tbl,\n \t.destroy_mtr_tbls = flow_dv_destroy_mtr_tbl,\n+\t.destroy_mtr_drop_tbls = flow_dv_destroy_mtr_drop_tbls,\n \t.create_meter = flow_dv_mtr_alloc,\n \t.free_meter = flow_dv_aso_mtr_release_to_pool,\n+\t.validate_mtr_acts = flow_dv_validate_mtr_policy_acts,\n+\t.create_mtr_acts = flow_dv_create_mtr_policy_acts,\n+\t.destroy_mtr_acts = flow_dv_destroy_mtr_policy_acts,\n+\t.create_policy_rules = flow_dv_create_policy_rules,\n+\t.destroy_policy_rules = flow_dv_destroy_policy_rules,\n+\t.create_def_policy = flow_dv_create_def_policy,\n+\t.destroy_def_policy = flow_dv_destroy_def_policy,\n \t.counter_alloc = flow_dv_counter_allocate,\n \t.counter_free = flow_dv_counter_free,\n \t.counter_query = flow_dv_counter_query,\ndiff --git a/drivers/net/mlx5/mlx5_flow_meter.c b/drivers/net/mlx5/mlx5_flow_meter.c\nindex af0a1c18cb..b38ff77210 100644\n--- a/drivers/net/mlx5/mlx5_flow_meter.c\n+++ b/drivers/net/mlx5/mlx5_flow_meter.c\n@@ -38,6 +38,12 @@ mlx5_flow_meter_action_create(struct mlx5_priv *priv,\n \tuint32_t cbs_cir = rte_be_to_cpu_32(srtcm->cbs_cir);\n \tuint32_t ebs_eir = rte_be_to_cpu_32(srtcm->ebs_eir);\n \tuint32_t val;\n+\tenum mlx5_meter_domain domain =\n+\t\tfm->transfer ? MLX5_MTR_DOMAIN_TRANSFER :\n+\t\t\tfm->egress ? MLX5_MTR_DOMAIN_EGRESS :\n+\t\t\t\tMLX5_MTR_DOMAIN_INGRESS;\n+\tstruct mlx5_flow_meter_def_policy *def_policy =\n+\t\tpriv->sh->mtrmng->def_policy[domain];\n \n \tmemset(fmp, 0, MLX5_ST_SZ_BYTES(flow_meter_parameters));\n \tMLX5_SET(flow_meter_parameters, fmp, valid, 1);\n@@ -57,10 +63,7 @@ mlx5_flow_meter_action_create(struct mlx5_priv *priv,\n \tMLX5_SET(flow_meter_parameters, fmp, ebs_exponent, val);\n \tval = (ebs_eir >> ASO_DSEG_EBS_MAN_OFFSET) & ASO_DSEG_MAN_MASK;\n \tMLX5_SET(flow_meter_parameters, fmp, ebs_mantissa, val);\n-\tmtr_init.next_table =\n-\t\tfm->transfer ? fm->mfts->transfer.tbl->obj :\n-\t\t\tfm->egress ? fm->mfts->egress.tbl->obj :\n-\t\t\t\tfm->mfts->ingress.tbl->obj;\n+\tmtr_init.next_table = def_policy->sub_policy.tbl_rsc->obj;\n \tmtr_init.reg_c_index = priv->mtr_color_reg - REG_C_0;\n \tmtr_init.flow_meter_parameter = fmp;\n \tmtr_init.flow_meter_parameter_sz =\n@@ -317,7 +320,7 @@ mlx5_flow_mtr_cap_get(struct rte_eth_dev *dev,\n \t\t\t\t\t  \"Meter is not supported\");\n \tmemset(cap, 0, sizeof(*cap));\n \tif (priv->sh->meter_aso_en)\n-\t    /* 2 meters per one ASO cache line. */\n+\t\t/* 2 meters per one ASO cache line. */\n \t\tcap->n_max = 1 << (qattr->log_max_num_meter_aso + 1);\n \telse\n \t\tcap->n_max = 1 << qattr->log_max_flow_meter;\n@@ -435,6 +438,347 @@ mlx5_flow_meter_profile_delete(struct rte_eth_dev *dev,\n \treturn 0;\n }\n \n+/**\n+ * Find policy by id.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ * @param policy_id\n+ *   Policy id.\n+ *\n+ * @return\n+ *   Pointer to the policy found on success, NULL otherwise.\n+ */\n+struct mlx5_flow_meter_policy *\n+mlx5_flow_meter_policy_find(struct rte_eth_dev *dev,\n+\t\t\t    uint32_t policy_id,\n+\t\t\t    uint32_t *policy_idx)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_flow_meter_sub_policy *sub_policy = NULL;\n+\tunion mlx5_l3t_data data;\n+\n+\tif (policy_id > MLX5_MAX_SUB_POLICY_TBL_NUM ||\n+\t\t!priv->sh->mtrmng->policy_idx_tbl)\n+\t\treturn NULL;\n+\tif (mlx5_l3t_get_entry(priv->sh->mtrmng->policy_idx_tbl,\n+\t\t\t\tpolicy_id, &data) ||\n+\t\t\t\t!data.dword)\n+\t\treturn NULL;\n+\tif (policy_idx)\n+\t\t*policy_idx = data.dword;\n+\tsub_policy = mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_MTR_POLICY],\n+\t\t\t\t\tdata.dword);\n+\t/* Remove reference taken by the mlx5_l3t_get_entry. */\n+\tmlx5_l3t_clear_entry(priv->sh->mtrmng->policy_idx_tbl,\n+\t\t\t\tpolicy_id);\n+\tif (sub_policy)\n+\t\tif (sub_policy->main_policy_id)\n+\t\t\treturn sub_policy->main_policy;\n+\treturn NULL;\n+}\n+\n+/**\n+ * Callback to check MTR policy action validate\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ * @param[in] actions\n+ *   Pointer to meter policy action detail.\n+ * @param[out] error\n+ *   Pointer to the error structure.\n+ *\n+ * @return\n+ *   0 on success, a negative errno value otherwise and rte_errno is set.\n+ */\n+static int\n+mlx5_flow_meter_policy_validate(struct rte_eth_dev *dev,\n+\tstruct rte_mtr_meter_policy_params *policy,\n+\tstruct rte_mtr_error *error)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct rte_flow_attr attr = { .transfer =\n+\t\t\tpriv->config.dv_esw_en ? 1 : 0};\n+\tbool is_rss = false;\n+\tbool is_def_policy = false;\n+\tuint8_t domain_bitmap;\n+\tint ret;\n+\n+\tif (!priv->mtr_en || !priv->sh->meter_aso_en)\n+\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\tNULL, \"meter policy unsupported.\");\n+\tret = mlx5_flow_validate_mtr_acts(dev, policy->actions, &attr,\n+\t\t\t&is_rss, &domain_bitmap, &is_def_policy, error);\n+\tif (ret)\n+\t\treturn ret;\n+\treturn 0;\n+}\n+\n+static int\n+__mlx5_flow_meter_policy_delete(struct rte_eth_dev *dev,\n+\t\t\tuint32_t policy_id,\n+\t\t\tstruct mlx5_flow_meter_policy *mtr_policy,\n+\t\t\tstruct rte_mtr_error *error)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_flow_meter_sub_policy *sub_policy;\n+\tuint32_t i, j;\n+\tuint16_t sub_policy_num;\n+\n+\trte_spinlock_lock(&mtr_policy->sl);\n+\tif (mtr_policy->ref_cnt) {\n+\t\trte_spinlock_unlock(&mtr_policy->sl);\n+\t\treturn -rte_mtr_error_set(error, EBUSY,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID,\n+\t\t\t\t NULL,\n+\t\t\t\t\"Meter policy object is being used.\");\n+\t}\n+\tmlx5_flow_destroy_policy_rules(dev, mtr_policy);\n+\tmlx5_flow_destroy_mtr_acts(dev, mtr_policy);\n+\tfor (i = 0; i < MLX5_MTR_DOMAIN_MAX; i++) {\n+\t\tsub_policy_num = (mtr_policy->sub_policy_num >>\n+\t\t\t(MLX5_MTR_SUB_POLICY_NUM_SHIFT * i)) &\n+\t\t\tMLX5_MTR_SUB_POLICY_NUM_MASK;\n+\t\tif (sub_policy_num) {\n+\t\t\tfor (j = 0; j < sub_policy_num; j++) {\n+\t\t\t\tsub_policy = mtr_policy->sub_policys[i][j];\n+\t\t\t\tif (sub_policy)\n+\t\t\t\t\tmlx5_ipool_free\n+\t\t\t\t\t(priv->sh->ipool[MLX5_IPOOL_MTR_POLICY],\n+\t\t\t\t\tsub_policy->idx);\n+\t\t\t}\n+\t\t}\n+\t}\n+\tif (priv->sh->mtrmng->policy_idx_tbl) {\n+\t\tif (mlx5_l3t_clear_entry(priv->sh->mtrmng->policy_idx_tbl,\n+\t\t\t\t\tpolicy_id)) {\n+\t\t\trte_spinlock_unlock(&mtr_policy->sl);\n+\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID, NULL,\n+\t\t\t\t\"Fail to delete policy in index table.\");\n+\t\t}\n+\t}\n+\trte_spinlock_unlock(&mtr_policy->sl);\n+\treturn 0;\n+}\n+\n+/**\n+ * Callback to add MTR policy.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ * @param[out] policy_id\n+ *   Pointer to policy id\n+ * @param[in] actions\n+ *   Pointer to meter policy action detail.\n+ * @param[out] error\n+ *   Pointer to the error structure.\n+ *\n+ * @return\n+ *   0 on success, a negative errno value otherwise and rte_errno is set.\n+ */\n+static int\n+mlx5_flow_meter_policy_add(struct rte_eth_dev *dev,\n+\t\t\tuint32_t policy_id,\n+\t\t\tstruct rte_mtr_meter_policy_params *policy,\n+\t\t\tstruct rte_mtr_error *error)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct rte_flow_attr attr = { .transfer =\n+\t\t\tpriv->config.dv_esw_en ? 1 : 0};\n+\tuint32_t sub_policy_idx = 0;\n+\tuint32_t policy_idx = 0;\n+\tstruct mlx5_flow_meter_policy *mtr_policy = NULL;\n+\tstruct mlx5_flow_meter_sub_policy *sub_policy;\n+\tbool is_rss = false;\n+\tbool is_def_policy = false;\n+\tuint32_t i;\n+\tint ret;\n+\tuint32_t policy_size = sizeof(struct mlx5_flow_meter_policy);\n+\tuint16_t sub_policy_num;\n+\tuint8_t domain_bitmap = 0;\n+\tunion mlx5_l3t_data data;\n+\n+\tif (!priv->mtr_en)\n+\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\t\t  RTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\t\t  NULL, \"meter policy unsupported.\");\n+\tif (policy_id == MLX5_INVALID_POLICY_ID)\n+\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID, NULL,\n+\t\t\t\"policy ID is invalid. \");\n+\tif (policy_id == priv->sh->mtrmng->def_policy_id)\n+\t\treturn -rte_mtr_error_set(error, EEXIST,\n+\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID, NULL,\n+\t\t\t\"policy ID exists. \");\n+\tmtr_policy = mlx5_flow_meter_policy_find(dev, policy_id,\n+\t\t\t\t&policy_idx);\n+\tif (mtr_policy)\n+\t\treturn -rte_mtr_error_set(error, EEXIST,\n+\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID, NULL,\n+\t\t\t\"policy ID exists. \");\n+\tret = mlx5_flow_validate_mtr_acts(dev, policy->actions, &attr,\n+\t\t\t&is_rss, &domain_bitmap, &is_def_policy, error);\n+\tif (ret)\n+\t\treturn ret;\n+\tif (!domain_bitmap)\n+\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\tNULL, \"fail to find policy domain.\");\n+\tif (is_def_policy) {\n+\t\tif (priv->sh->mtrmng->def_policy_id != MLX5_INVALID_POLICY_ID)\n+\t\t\treturn -rte_mtr_error_set(error, EEXIST,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID,\n+\t\t\t\tNULL, \"a policy with similar actions \"\n+\t\t\t\t\"is already configured\");\n+\t\tif (mlx5_flow_create_def_policy(dev))\n+\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY,\n+\t\t\t\tNULL,\n+\t\t\t\t\"fail to create non-terminated policy.\");\n+\t\tpriv->sh->mtrmng->def_policy_id = policy_id;\n+\t\treturn 0;\n+\t}\n+\tif (!priv->sh->meter_aso_en)\n+\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY, NULL,\n+\t\t\t\"no ASO capability to support the policy \");\n+\tfor (i = 0; i < MLX5_MTR_DOMAIN_MAX; i++) {\n+\t\tif (!(domain_bitmap & (1 << i)))\n+\t\t\tcontinue;\n+\t\tif (is_rss) {\n+\t\t\tpolicy_size +=\n+\t\t\tsizeof(struct mlx5_flow_meter_sub_policy *) *\n+\t\t\tMLX5_MTR_RSS_MAX_SUB_POLICY;\n+\t\t\tbreak;\n+\t\t}\n+\t\tpolicy_size += sizeof(struct mlx5_flow_meter_sub_policy *);\n+\t}\n+\tmtr_policy = mlx5_malloc(MLX5_MEM_ZERO, policy_size,\n+\t\t\t RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);\n+\tif (!mtr_policy)\n+\t\treturn -rte_mtr_error_set(error, ENOMEM,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY, NULL,\n+\t\t\t\t\"Memory alloc failed for meter policy.\");\n+\tpolicy_size = sizeof(struct mlx5_flow_meter_policy);\n+\tfor (i = 0; i < MLX5_MTR_DOMAIN_MAX; i++) {\n+\t\tif (!(domain_bitmap & (1 << i)))\n+\t\t\tcontinue;\n+\t\tmtr_policy->ingress = (i == MLX5_MTR_DOMAIN_INGRESS) ? 1 : 0;\n+\t\tmtr_policy->egress = (i == MLX5_MTR_DOMAIN_EGRESS) ? 1 : 0;\n+\t\tmtr_policy->transfer = (i == MLX5_MTR_DOMAIN_TRANSFER) ? 1 : 0;\n+\t\tsub_policy = mlx5_ipool_zmalloc\n+\t\t\t\t(priv->sh->ipool[MLX5_IPOOL_MTR_POLICY],\n+\t\t\t\t&sub_policy_idx);\n+\t\tif (!sub_policy)\n+\t\t\tgoto policy_add_err;\n+\t\tif (sub_policy_idx > MLX5_MAX_SUB_POLICY_TBL_NUM)\n+\t\t\tgoto policy_add_err;\n+\t\tsub_policy->idx = sub_policy_idx;\n+\t\tsub_policy->main_policy = mtr_policy;\n+\t\tif (!policy_idx) {\n+\t\t\tpolicy_idx = sub_policy_idx;\n+\t\t\tsub_policy->main_policy_id = 1;\n+\t\t}\n+\t\tmtr_policy->sub_policys[i] =\n+\t\t(struct mlx5_flow_meter_sub_policy **)\n+\t\t\t((uint8_t *)mtr_policy + policy_size);\n+\t\tmtr_policy->sub_policys[i][0] = sub_policy;\n+\t\tsub_policy_num = (mtr_policy->sub_policy_num >>\n+\t\t\t(MLX5_MTR_SUB_POLICY_NUM_SHIFT * i)) &\n+\t\t\tMLX5_MTR_SUB_POLICY_NUM_MASK;\n+\t\tsub_policy_num++;\n+\t\tmtr_policy->sub_policy_num &= ~(MLX5_MTR_SUB_POLICY_NUM_MASK <<\n+\t\t\t(MLX5_MTR_SUB_POLICY_NUM_SHIFT * i));\n+\t\tmtr_policy->sub_policy_num |=\n+\t\t\t(sub_policy_num & MLX5_MTR_SUB_POLICY_NUM_MASK) <<\n+\t\t\t(MLX5_MTR_SUB_POLICY_NUM_SHIFT * i);\n+\t\tif (is_rss) {\n+\t\t\tmtr_policy->is_rss = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t\tpolicy_size += sizeof(struct mlx5_flow_meter_sub_policy  *);\n+\t}\n+\trte_spinlock_init(&mtr_policy->sl);\n+\tret = mlx5_flow_create_mtr_acts(dev, mtr_policy,\n+\t\t\t\t\tpolicy->actions, error);\n+\tif (ret)\n+\t\tgoto policy_add_err;\n+\tif (!is_rss) {\n+\t\t/* Create policy rules in HW. */\n+\t\tret = mlx5_flow_create_policy_rules(dev, mtr_policy);\n+\t\tif (ret)\n+\t\t\tgoto policy_add_err;\n+\t}\n+\tdata.dword = policy_idx;\n+\tif (!priv->sh->mtrmng->policy_idx_tbl) {\n+\t\tpriv->sh->mtrmng->policy_idx_tbl =\n+\t\t\t\tmlx5_l3t_create(MLX5_L3T_TYPE_DWORD);\n+\t\tif (!priv->sh->mtrmng->policy_idx_tbl)\n+\t\t\tgoto policy_add_err;\n+\t}\n+\tif (mlx5_l3t_set_entry(priv->sh->mtrmng->policy_idx_tbl,\n+\t\t\t\tpolicy_id, &data))\n+\t\tgoto policy_add_err;\n+\treturn 0;\n+policy_add_err:\n+\tif (mtr_policy) {\n+\t\tret = __mlx5_flow_meter_policy_delete(dev, policy_id,\n+\t\t\tmtr_policy, error);\n+\t\tmlx5_free(mtr_policy);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n+\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\t  RTE_MTR_ERROR_TYPE_UNSPECIFIED,\n+\t\t\t\t  NULL, \"Failed to create devx policy.\");\n+}\n+\n+/**\n+ * Callback to delete MTR policy.\n+ *\n+ * @param[in] dev\n+ *   Pointer to Ethernet device.\n+ * @param[in] policy_id\n+ *   Meter policy id.\n+ * @param[out] error\n+ *   Pointer to the error structure.\n+ *\n+ * @return\n+ *   0 on success, a negative errno value otherwise and rte_errno is set.\n+ */\n+static int\n+mlx5_flow_meter_policy_delete(struct rte_eth_dev *dev,\n+\t\t\t  uint32_t policy_id,\n+\t\t\t  struct rte_mtr_error *error)\n+{\n+\tstruct mlx5_priv *priv = dev->data->dev_private;\n+\tstruct mlx5_flow_meter_policy *mtr_policy;\n+\tuint32_t policy_idx;\n+\tint ret;\n+\n+\tif (policy_id == priv->sh->mtrmng->def_policy_id) {\n+\t\tif (priv->sh->mtrmng->def_policy_ref_cnt > 0)\n+\t\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID, NULL,\n+\t\t\t\t\"Meter policy object is being used.\");\n+\t\tpriv->sh->mtrmng->def_policy_id = MLX5_INVALID_POLICY_ID;\n+\t\treturn 0;\n+\t}\n+\tmtr_policy = mlx5_flow_meter_policy_find(dev, policy_id, &policy_idx);\n+\tif (!mtr_policy)\n+\t\treturn -rte_mtr_error_set(error, ENOTSUP,\n+\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID, NULL,\n+\t\t\t\"Meter policy id is invalid. \");\n+\tret = __mlx5_flow_meter_policy_delete(dev, policy_id, mtr_policy,\n+\t\t\t\t\t\terror);\n+\tif (ret)\n+\t\treturn ret;\n+\tmlx5_free(mtr_policy);\n+\treturn 0;\n+}\n+\n /**\n  * Modify the flow meter action.\n  *\n@@ -942,6 +1286,9 @@ static const struct rte_mtr_ops mlx5_flow_mtr_ops = {\n \t.capabilities_get = mlx5_flow_mtr_cap_get,\n \t.meter_profile_add = mlx5_flow_meter_profile_add,\n \t.meter_profile_delete = mlx5_flow_meter_profile_delete,\n+\t.meter_policy_validate = mlx5_flow_meter_policy_validate,\n+\t.meter_policy_add = mlx5_flow_meter_policy_add,\n+\t.meter_policy_delete = mlx5_flow_meter_policy_delete,\n \t.destroy = mlx5_flow_meter_destroy,\n \t.meter_enable = mlx5_flow_meter_enable,\n \t.meter_disable = mlx5_flow_meter_disable,\n@@ -989,22 +1336,32 @@ mlx5_flow_meter_find(struct mlx5_priv *priv, uint32_t meter_id,\n \tstruct mlx5_legacy_flow_meter *legacy_fm;\n \tstruct mlx5_legacy_flow_meters *fms = &priv->flow_meters;\n \tstruct mlx5_aso_mtr *aso_mtr;\n-\tstruct mlx5_aso_mtr_pools_mng *mtrmng = priv->sh->mtrmng;\n+\tstruct mlx5_aso_mtr_pools_mng *pools_mng =\n+\t\t\t\t&priv->sh->mtrmng->pools_mng;\n \tunion mlx5_l3t_data data;\n \n \tif (priv->sh->meter_aso_en) {\n-\t\trte_spinlock_lock(&mtrmng->mtrsl);\n-\t\tif (mlx5_l3t_get_entry(priv->mtr_idx_tbl, meter_id, &data) ||\n-\t\t\t!data.dword) {\n-\t\t\trte_spinlock_unlock(&mtrmng->mtrsl);\n-\t\t\treturn NULL;\n+\t\trte_spinlock_lock(&pools_mng->mtrsl);\n+\t\tif (priv->mtr_idx_tbl) {\n+\t\t\tif (mlx5_l3t_get_entry(priv->mtr_idx_tbl,\n+\t\t\t\tmeter_id, &data) ||\n+\t\t\t\t!data.dword) {\n+\t\t\t\trte_spinlock_unlock(&pools_mng->mtrsl);\n+\t\t\t\treturn NULL;\n+\t\t\t}\n+\t\t\tif (mtr_idx)\n+\t\t\t\t*mtr_idx = data.dword;\n+\t\t\taso_mtr = mlx5_aso_meter_by_idx(priv, data.dword);\n+\t\t\t/* Remove reference taken by the mlx5_l3t_get_entry. */\n+\t\t\tmlx5_l3t_clear_entry(priv->mtr_idx_tbl, meter_id);\n+\t\t} else {\n+\t\t\tif (mtr_idx)\n+\t\t\t\t*mtr_idx = meter_id;\n+\t\t\taso_mtr = mlx5_aso_meter_by_idx(priv, meter_id);\n \t\t}\n-\t\tif (mtr_idx)\n-\t\t\t*mtr_idx = data.dword;\n-\t\taso_mtr = mlx5_aso_meter_by_idx(priv, data.dword);\n-\t\t/* Remove reference taken by the mlx5_l3t_get_entry. */\n-\t\tmlx5_l3t_clear_entry(priv->mtr_idx_tbl, meter_id);\n-\t\trte_spinlock_unlock(&mtrmng->mtrsl);\n+\t\trte_spinlock_unlock(&pools_mng->mtrsl);\n+\t\tif (!aso_mtr || aso_mtr->state == ASO_METER_FREE)\n+\t\t\treturn NULL;\n \t\treturn &aso_mtr->fm;\n \t}\n \tTAILQ_FOREACH(legacy_fm, fms, next)\n@@ -1169,30 +1526,31 @@ int\n mlx5_flow_meter_flush(struct rte_eth_dev *dev, struct rte_mtr_error *error)\n {\n \tstruct mlx5_priv *priv = dev->data->dev_private;\n-\tstruct mlx5_aso_mtr_pools_mng *mtrmng = priv->sh->mtrmng;\n+\tstruct mlx5_aso_mtr_pools_mng *pools_mng =\n+\t\t\t\t&priv->sh->mtrmng->pools_mng;\n \tstruct mlx5_legacy_flow_meters *fms = &priv->flow_meters;\n \tstruct mlx5_mtr_profiles *fmps = &priv->flow_meter_profiles;\n \tstruct mlx5_flow_meter_profile *fmp;\n \tstruct mlx5_legacy_flow_meter *legacy_fm;\n \tstruct mlx5_flow_meter_info *fm;\n \tstruct mlx5_aso_mtr_pool *mtr_pool;\n+\tstruct mlx5_flow_meter_sub_policy *sub_policy;\n \tvoid *tmp;\n-\tuint32_t i, offset, mtr_idx;\n+\tuint32_t i, offset, mtr_idx, policy_idx;\n+\tvoid *entry;\n \n+\tif (!priv->mtr_en)\n+\t\treturn 0;\n \tif (priv->sh->meter_aso_en) {\n-\t\ti = mtrmng->n_valid;\n+\t\ti = pools_mng->n_valid;\n \t\twhile (i--) {\n-\t\t\tmtr_pool = mtrmng->pools[i];\n+\t\t\tmtr_pool = pools_mng->pools[i];\n \t\t\tfor (offset = 0; offset < MLX5_ASO_MTRS_PER_POOL;\n \t\t\t\toffset++) {\n \t\t\t\tfm = &mtr_pool->mtrs[offset].fm;\n \t\t\t\tmtr_idx = MLX5_MAKE_MTR_IDX(i, offset);\n-\t\t\t\tif (mlx5_flow_meter_params_flush(dev,\n-\t\t\t\t\t\tfm, mtr_idx))\n-\t\t\t\t\treturn -rte_mtr_error_set\n-\t\t\t\t\t(error, EINVAL,\n-\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_PROFILE_ID,\n-\t\t\t\t\tNULL, \"MTR object meter profile invalid.\");\n+\t\t\t\t(void)mlx5_flow_meter_params_flush(dev,\n+\t\t\t\t\t\tfm, mtr_idx);\n \t\t\t}\n \t\t}\n \t} else {\n@@ -1200,9 +1558,35 @@ mlx5_flow_meter_flush(struct rte_eth_dev *dev, struct rte_mtr_error *error)\n \t\t\tfm = &legacy_fm->fm;\n \t\t\tif (mlx5_flow_meter_params_flush(dev, fm, 0))\n \t\t\t\treturn -rte_mtr_error_set(error, EINVAL,\n-\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_PROFILE_ID,\n-\t\t\t\t\tNULL, \"MTR object meter profile invalid.\");\n+\t\t\t\tRTE_MTR_ERROR_TYPE_METER_PROFILE_ID,\n+\t\t\t\tNULL, \"MTR object meter profile invalid.\");\n+\t\t}\n+\t}\n+\tif (priv->sh->mtrmng->policy_idx_tbl) {\n+\t\tMLX5_L3T_FOREACH(priv->sh->mtrmng->policy_idx_tbl,\n+\t\t\t\t\ti, entry) {\n+\t\t\tpolicy_idx = *(uint32_t *)entry;\n+\t\t\tsub_policy = mlx5_ipool_get\n+\t\t\t\t(priv->sh->ipool[MLX5_IPOOL_MTR_POLICY],\n+\t\t\t\tpolicy_idx);\n+\t\t\tif (!sub_policy)\n+\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t\tEINVAL,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID,\n+\t\t\t\t\t\tNULL, \"MTR object \"\n+\t\t\t\t\t\t\"meter policy invalid.\");\n+\t\t\tif (__mlx5_flow_meter_policy_delete(dev, i,\n+\t\t\t\t\t\tsub_policy->main_policy,\n+\t\t\t\t\t\terror))\n+\t\t\t\treturn -rte_mtr_error_set(error,\n+\t\t\t\t\t\tEINVAL,\n+\t\t\t\t\tRTE_MTR_ERROR_TYPE_METER_POLICY_ID,\n+\t\t\t\t\t\tNULL, \"MTR object \"\n+\t\t\t\t\t\t\"meter policy invalid.\");\n+\t\t\tmlx5_free(sub_policy->main_policy);\n \t\t}\n+\t\tmlx5_l3t_destroy(priv->sh->mtrmng->policy_idx_tbl);\n+\t\tpriv->sh->mtrmng->policy_idx_tbl = NULL;\n \t}\n \tTAILQ_FOREACH_SAFE(fmp, fmps, next, tmp) {\n \t\t/* Check unused. */\n@@ -1211,5 +1595,8 @@ mlx5_flow_meter_flush(struct rte_eth_dev *dev, struct rte_mtr_error *error)\n \t\tTAILQ_REMOVE(&priv->flow_meter_profiles, fmp, next);\n \t\tmlx5_free(fmp);\n \t}\n+\t/* Delete default policy table. */\n+\tmlx5_flow_destroy_def_policy(dev);\n+\tmlx5_flow_destroy_mtr_drop_tbls(dev);\n \treturn 0;\n }\ndiff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c\nindex 94dd56709e..90d918849a 100644\n--- a/drivers/net/mlx5/mlx5_trigger.c\n+++ b/drivers/net/mlx5/mlx5_trigger.c\n@@ -1179,6 +1179,7 @@ mlx5_dev_stop(struct rte_eth_dev *dev)\n \tmlx5_traffic_disable(dev);\n \t/* All RX queue flags will be cleared in the flush interface. */\n \tmlx5_flow_list_flush(dev, &priv->flows, true);\n+\tmlx5_flow_meter_flush(dev, NULL);\n \tmlx5_rx_intr_vec_disable(dev);\n \tpriv->sh->port[priv->dev_port - 1].ih_port_id = RTE_MAX_ETHPORTS;\n \tpriv->sh->port[priv->dev_port - 1].devx_ih_port_id = RTE_MAX_ETHPORTS;\n",
    "prefixes": [
        "v6",
        "1/4"
    ]
}