From patchwork Thu Apr 4 09:54:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 52269 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8F91B1B112; Thu, 4 Apr 2019 11:54:10 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70085.outbound.protection.outlook.com [40.107.7.85]) by dpdk.org (Postfix) with ESMTP id 194941B0F7 for ; Thu, 4 Apr 2019 11:54:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MQekMAaAiykr28ELAo4jU3LKAA31kx5Z1UaDfObl/nY=; b=qMr47z289qwE0fo8OJGQ4xtNBjjKw9/R798Pr4e/4yHtKm2ok9qOLPguJi8ggxPXtUxCAWzbf8xOyc8Ylc0acuCnWHq4NtuTTQzZ9JqX6ROAMYqC5vSRGMp93Wx/szZLTLXd7ACYPVQsC1lE/jWtjkSPBQrly7LqkAlByfrGBhc= Received: from AM4PR05MB3425.eurprd05.prod.outlook.com (10.171.190.15) by AM4PR05MB3139.eurprd05.prod.outlook.com (10.171.186.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.22; Thu, 4 Apr 2019 09:54:06 +0000 Received: from AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827]) by AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827%4]) with mapi id 15.20.1771.014; Thu, 4 Apr 2019 09:54:06 +0000 From: Ori Kam To: Matan Azrad , Yongseok Koh , Shahaf Shuler CC: "dev@dpdk.org" , Ori Kam , Slava Ovsiienko Thread-Topic: [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Thread-Index: AQHU6sxXvdqR4PNy4EGhKBhNp77JIQ== Date: Thu, 4 Apr 2019 09:54:06 +0000 Message-ID: <1554371628-170844-2-git-send-email-orika@mellanox.com> References: <1553790741-69362-1-git-send-email-orika@mellanox.com> <1554371628-170844-1-git-send-email-orika@mellanox.com> In-Reply-To: <1554371628-170844-1-git-send-email-orika@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: LO2P265CA0363.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a3::15) To AM4PR05MB3425.eurprd05.prod.outlook.com (2603:10a6:205:b::15) authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 1.8.3.1 x-originating-ip: [37.142.13.130] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 03055bca-19d1-4de1-1379-08d6b8e379e5 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600139)(711020)(4605104)(4618075)(2017052603328)(7193020); SRVR:AM4PR05MB3139; x-ms-traffictypediagnostic: AM4PR05MB3139: x-microsoft-antispam-prvs: x-forefront-prvs: 0997523C40 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(376002)(39860400002)(396003)(366004)(346002)(51234002)(199004)(189003)(76176011)(53936002)(186003)(6636002)(2616005)(446003)(25786009)(26005)(52116002)(36756003)(6486002)(6512007)(476003)(5660300002)(256004)(107886003)(2906002)(3846002)(53946003)(6116002)(97736004)(68736007)(486006)(4326008)(478600001)(6436002)(110136005)(105586002)(99286004)(86362001)(305945005)(66066001)(11346002)(30864003)(316002)(102836004)(8676002)(81166006)(8936002)(14454004)(106356001)(6506007)(81156014)(71200400001)(386003)(7736002)(50226002)(71190400001)(54906003)(4720700003)(579004); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3139; H:AM4PR05MB3425.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 6BSEStSF1cRBhGLVnIpa9S+tsCn+PcOb7V97Ne63n7rv64Pgo4SFq5+ofMfaGgKd6hbAMsnFPYy7Lv5pE6jLkNW3hOdnen5qfalMNbocrSFMC5GSL9apBF+o1voVPv8BMPJKJ+DqZWzHVWNM0miViHzUXLvxBv6bgS+Xagl4+dciyxAhY/Vp+9RRhLh/qnoSvYOs8aetQL4Eim6QNNQniGrytlgJWHYLSq99bk5PLULG3tVKZKf8cAsI6yfdo8eLH8O1zjupeFtGcjHHKWSAoWawUc3NeMaw7WNgkd/9kjhpa2EJTV/2SmArVnav7n60ESZcXAOeSVDlNqSbR7qC0Nxu9qwc1V4CXzzdZwiUBy9lNI3D7/PS3leThexKCxk07byL5te2O9GLBTvCw0ItkvOQkNEYlJJbc0K9E+dL7iM= MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 03055bca-19d1-4de1-1379-08d6b8e379e5 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Apr 2019 09:54:06.4568 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3139 Subject: [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is the first patch of a series that is designed to enable the Direct Rules API. The main difference between Direct Verbs and Direct Rules from API prespective, is that in Direct Rules each action has it's own create function and the object itself is of type void. In this patch I'm adding functions to generate actions that currenlty are done without create action, and I'm changing the action type to be void *, so in next patches only the glue functions will need to change. Signed-off-by: Ori Kam Acked-by: Shahaf Shuler --- drivers/net/mlx5/Makefile | 2 +- drivers/net/mlx5/meson.build | 2 +- drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_flow.h | 17 +++- drivers/net/mlx5/mlx5_flow_dv.c | 184 +++++++++++++++++++++++++++++----------- drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------ drivers/net/mlx5/mlx5_glue.h | 26 +++--- 7 files changed, 280 insertions(+), 94 deletions(-) diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index ccd179c..0d20f0f 100644 --- a/drivers/net/mlx5/Makefile +++ b/drivers/net/mlx5/Makefile @@ -8,7 +8,7 @@ include $(RTE_SDK)/mk/rte.vars.mk LIB = librte_pmd_mlx5.a LIB_GLUE = $(LIB_GLUE_BASE).$(LIB_GLUE_VERSION) LIB_GLUE_BASE = librte_pmd_mlx5_glue.so -LIB_GLUE_VERSION = 19.02.0 +LIB_GLUE_VERSION = 19.05.0 # Sources. SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5.c diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build index a429655..a4c684e 100644 --- a/drivers/net/mlx5/meson.build +++ b/drivers/net/mlx5/meson.build @@ -4,7 +4,7 @@ pmd_dlopen = (get_option('ibverbs_link') == 'dlopen') LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so' -LIB_GLUE_VERSION = '19.02.0' +LIB_GLUE_VERSION = '19.05.0' LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION if pmd_dlopen dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 47a7d75..784bf9b 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -309,6 +309,8 @@ struct mlx5_priv { LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers; LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps; LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds; + LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags; + /* Tags resources cache. */ uint32_t link_speed_capa; /* Link speed capabilities. */ struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */ struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */ diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index e1e798b..4f69ae2 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource { LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next; /* Pointer to next element. */ rte_atomic32_t refcnt; /**< Reference counter. */ - struct ibv_flow_action *verbs_action; + void *verbs_action; /**< Verbs encap/decap action object. */ uint8_t buf[MLX5_ENCAP_MAX_LEN]; size_t size; @@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource { uint8_t ft_type; }; +/* Tag resource structure. */ +struct mlx5_flow_dv_tag_resource { + LIST_ENTRY(mlx5_flow_dv_tag_resource) next; + /* Pointer to next element. */ + rte_atomic32_t refcnt; /**< Reference counter. */ + void *action; + /**< Verbs tag action object. */ + uint32_t tag; /**< the tag value. */ +}; + /* Number of modification commands. */ #define MLX5_MODIFY_NUM 8 @@ -259,7 +269,7 @@ struct mlx5_flow_dv { /**< Pointer to modify header resource in cache. */ struct ibv_flow *flow; /**< Installed flow. */ #ifdef HAVE_IBV_FLOW_DV_SUPPORT - struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS]; + void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS]; /**< Action list. */ #endif int actions_n; /**< number of actions. */ @@ -332,6 +342,7 @@ struct mlx5_flow_counter { }; uint64_t hits; /**< Number of packets matched by the rule. */ uint64_t bytes; /**< Number of bytes matched by the rule. */ + void *action; /**< Pointer to the dv action. */ }; /* Flow structure. */ @@ -339,6 +350,8 @@ struct rte_flow { TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ enum mlx5_flow_drv_type drv_type; /**< Drvier type. */ struct mlx5_flow_counter *counter; /**< Holds flow counter. */ + struct mlx5_flow_dv_tag_resource *tag_resource; + /**< pointer to the tag action. */ struct rte_flow_action_rss rss;/**< RSS context. */ uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */ uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */ diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index ad84dea..d9e2ac9 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -32,7 +32,6 @@ #include "mlx5_prm.h" #include "mlx5_glue.h" #include "mlx5_flow.h" - #ifdef HAVE_IBV_FLOW_DV_SUPPORT #ifndef HAVE_IBV_FLOW_DEVX_COUNTERS @@ -1537,6 +1536,11 @@ struct field_modify_info modify_tcp[] = { .id = id, .dcs = dcs, }; + tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0); + if (!tmpl.action) { + ret = errno; + goto error_exit; + } *cnt = tmpl; LIST_INSERT_HEAD(&priv->flow_counters, cnt, next); return cnt; @@ -2828,6 +2832,97 @@ struct field_modify_info modify_tcp[] = { } /** + * Find existing tag resource or create and register a new one. + * + * @param dev[in, out] + * Pointer to rte_eth_dev structure. + * @param[in, out] resource + * Pointer to tag resource. + * @parm[in, out] dev_flow + * Pointer to the dev_flow. + * @param[out] error + * pointer to error structure. + * + * @return + * 0 on success otherwise -errno and errno is set. + */ +static int +flow_dv_tag_resource_register + (struct rte_eth_dev *dev, + struct mlx5_flow_dv_tag_resource *resource, + struct mlx5_flow *dev_flow, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_flow_dv_tag_resource *cache_resource; + + /* Lookup a matching resource from cache. */ + LIST_FOREACH(cache_resource, &priv->tags, next) { + if (resource->tag == cache_resource->tag) { + DRV_LOG(DEBUG, "tag resource %p: refcnt %d++", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + rte_atomic32_inc(&cache_resource->refcnt); + dev_flow->flow->tag_resource = cache_resource; + return 0; + } + } + /* Register new resource. */ + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0); + if (!cache_resource) + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "cannot allocate resource memory"); + *cache_resource = *resource; + cache_resource->action = mlx5_glue->dv_create_flow_action_tag + (resource->tag); + if (!cache_resource->action) { + rte_free(cache_resource); + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create action"); + } + rte_atomic32_init(&cache_resource->refcnt); + rte_atomic32_inc(&cache_resource->refcnt); + LIST_INSERT_HEAD(&priv->tags, cache_resource, next); + dev_flow->flow->tag_resource = cache_resource; + DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + return 0; +} + +/** + * Release the tag. + * + * @param dev + * Pointer to Ethernet device. + * @param flow + * Pointer to mlx5_flow. + * + * @return + * 1 while a reference on it exists, 0 when freed. + */ +static int +flow_dv_tag_release(struct rte_eth_dev *dev, + struct mlx5_flow_dv_tag_resource *tag) +{ + assert(tag); + DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--", + dev->data->port_id, (void *)tag, + rte_atomic32_read(&tag->refcnt)); + if (rte_atomic32_dec_and_test(&tag->refcnt)) { + claim_zero(mlx5_glue->destroy_flow_action(tag->action)); + LIST_REMOVE(tag, next); + DRV_LOG(DEBUG, "port %u tag %p: removed", + dev->data->port_id, (void *)tag); + rte_free(tag); + return 0; + } + return 1; +} + +/** * Fill the flow with DV spec. * * @param[in] dev @@ -2872,6 +2967,7 @@ struct field_modify_info modify_tcp[] = { MLX5DV_FLOW_TABLE_TYPE_NIC_RX }; union flow_dv_attr flow_attr = { .attr = 0 }; + struct mlx5_flow_dv_tag_resource tag_resource; if (priority == MLX5_FLOW_PRIO_RSVD) priority = priv->config.flow_prio - 1; @@ -2886,26 +2982,29 @@ struct field_modify_info modify_tcp[] = { case RTE_FLOW_ACTION_TYPE_VOID: break; case RTE_FLOW_ACTION_TYPE_FLAG: - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_TAG; - dev_flow->dv.actions[actions_n].tag_value = + tag_resource.tag = mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT); - actions_n++; + if (!flow->tag_resource) + if (flow_dv_tag_resource_register + (dev, &tag_resource, dev_flow, error)) + return errno; + dev_flow->dv.actions[actions_n++] = + flow->tag_resource->action; action_flags |= MLX5_FLOW_ACTION_FLAG; break; case RTE_FLOW_ACTION_TYPE_MARK: - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_TAG; - dev_flow->dv.actions[actions_n].tag_value = - mlx5_flow_mark_set - (((const struct rte_flow_action_mark *) - (actions->conf))->id); - actions_n++; + tag_resource.tag = mlx5_flow_mark_set + (((const struct rte_flow_action_mark *) + (actions->conf))->id); + if (!flow->tag_resource) + if (flow_dv_tag_resource_register + (dev, &tag_resource, dev_flow, error)) + return errno; + dev_flow->dv.actions[actions_n++] = + flow->tag_resource->action; action_flags |= MLX5_FLOW_ACTION_MARK; break; case RTE_FLOW_ACTION_TYPE_DROP: - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_DROP; action_flags |= MLX5_FLOW_ACTION_DROP; break; case RTE_FLOW_ACTION_TYPE_QUEUE: @@ -2933,17 +3032,13 @@ struct field_modify_info modify_tcp[] = { rte_errno = ENOTSUP; goto cnt_err; } - flow->counter = - flow_dv_counter_new(dev, - count->shared, count->id); + flow->counter = flow_dv_counter_new(dev, count->shared, + count->id); if (flow->counter == NULL) goto cnt_err; - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_COUNTERS_DEVX; - dev_flow->dv.actions[actions_n].obj = - flow->counter->dcs->obj; + dev_flow->dv.actions[actions_n++] = + flow->counter->action; action_flags |= MLX5_FLOW_ACTION_COUNT; - ++actions_n; break; cnt_err: if (rte_errno == ENOTSUP) @@ -2964,11 +3059,8 @@ struct field_modify_info modify_tcp[] = { if (flow_dv_create_action_l2_encap(dev, actions, dev_flow, error)) return -rte_errno; - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; - dev_flow->dv.actions[actions_n].action = + dev_flow->dv.actions[actions_n++] = dev_flow->dv.encap_decap->verbs_action; - actions_n++; action_flags |= actions->type == RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ? MLX5_FLOW_ACTION_VXLAN_ENCAP : @@ -2979,11 +3071,8 @@ struct field_modify_info modify_tcp[] = { if (flow_dv_create_action_l2_decap(dev, dev_flow, error)) return -rte_errno; - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; - dev_flow->dv.actions[actions_n].action = + dev_flow->dv.actions[actions_n++] = dev_flow->dv.encap_decap->verbs_action; - actions_n++; action_flags |= actions->type == RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ? MLX5_FLOW_ACTION_VXLAN_DECAP : @@ -2995,9 +3084,7 @@ struct field_modify_info modify_tcp[] = { if (flow_dv_create_action_raw_encap (dev, actions, dev_flow, attr, error)) return -rte_errno; - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; - dev_flow->dv.actions[actions_n].action = + dev_flow->dv.actions[actions_n++] = dev_flow->dv.encap_decap->verbs_action; } else { /* Handle encap without preceding decap. */ @@ -3005,12 +3092,9 @@ struct field_modify_info modify_tcp[] = { dev_flow, error)) return -rte_errno; - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; - dev_flow->dv.actions[actions_n].action = + dev_flow->dv.actions[actions_n++] = dev_flow->dv.encap_decap->verbs_action; } - actions_n++; action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP; break; case RTE_FLOW_ACTION_TYPE_RAW_DECAP: @@ -3025,11 +3109,8 @@ struct field_modify_info modify_tcp[] = { dev_flow, error)) return -rte_errno; - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; - dev_flow->dv.actions[actions_n].action = + dev_flow->dv.actions[actions_n++] = dev_flow->dv.encap_decap->verbs_action; - actions_n++; } /* If decap is followed by encap, handle it at encap. */ action_flags |= MLX5_FLOW_ACTION_RAW_DECAP; @@ -3098,11 +3179,8 @@ struct field_modify_info modify_tcp[] = { dev_flow, error)) return -rte_errno; - dev_flow->dv.actions[actions_n].type = - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; - dev_flow->dv.actions[actions_n].action = + dev_flow->dv.actions[actions_n++] = dev_flow->dv.modify_hdr->verbs_action; - actions_n++; } break; default: @@ -3277,9 +3355,9 @@ struct field_modify_info modify_tcp[] = { "cannot get drop hash queue"); goto error; } - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP; - dv->actions[n].qp = dv->hrxq->qp; - n++; + dv->actions[n++] = + mlx5_glue->dv_create_flow_action_dest_ibv_qp + (dv->hrxq->qp); } else if (flow->actions & (MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) { struct mlx5_hrxq *hrxq; @@ -3304,9 +3382,9 @@ struct field_modify_info modify_tcp[] = { goto error; } dv->hrxq = hrxq; - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP; - dv->actions[n].qp = hrxq->qp; - n++; + dv->actions[n++] = + mlx5_glue->dv_create_flow_action_dest_ibv_qp + (dv->hrxq->qp); } dv->flow = mlx5_glue->dv_create_flow(dv->matcher->matcher_object, @@ -3484,6 +3562,10 @@ struct field_modify_info modify_tcp[] = { flow_dv_counter_release(flow->counter); flow->counter = NULL; } + if (flow->tag_resource) { + flow_dv_tag_release(dev, flow->tag_resource); + flow->tag_resource = NULL; + } while (!LIST_EMPTY(&flow->dev_flows)) { dev_flow = LIST_FIRST(&flow->dev_flows); LIST_REMOVE(dev_flow, next); diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c index c817d86..4b5aade 100644 --- a/drivers/net/mlx5/mlx5_glue.c +++ b/drivers/net/mlx5/mlx5_glue.c @@ -175,10 +175,20 @@ } static int -mlx5_glue_destroy_flow_action(struct ibv_flow_action *action) +mlx5_glue_destroy_flow_action(void *action) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT - return ibv_destroy_flow_action(action); + struct mlx5dv_flow_action_attr *attr = action; + int res = 0; + switch (attr->type) { + case MLX5DV_FLOW_ACTION_TAG: + break; + default: + res = ibv_destroy_flow_action(attr->action); + break; + } + free(action); + return res; #else (void)action; return ENOTSUP; @@ -430,16 +440,23 @@ mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher, struct mlx5dv_flow_match_parameters *match_value, size_t num_actions, - struct mlx5dv_flow_action_attr *actions_attr) + void *actions[]) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT + struct mlx5dv_flow_action_attr actions_attr[8]; + + if (num_actions > 8) + return NULL; + for (size_t i = 0; i < num_actions; i++) + actions_attr[i] = + *((struct mlx5dv_flow_action_attr *)(actions[i])); return mlx5dv_create_flow(matcher, match_value, num_actions, actions_attr); #else (void)matcher; (void)match_value; (void)num_actions; - (void)actions_attr; + (void)actions; return NULL; #endif } @@ -455,31 +472,45 @@ #endif } -static struct ibv_flow_action * -mlx5_glue_dv_create_flow_action_packet_reformat - (struct ibv_context *ctx, - size_t data_sz, - void *data, - enum mlx5dv_flow_action_packet_reformat_type reformat_type, - enum mlx5dv_flow_table_type ft_type) +static void * +mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT - return mlx5dv_create_flow_action_packet_reformat(ctx, - data_sz, - data, - reformat_type, - ft_type); + struct mlx5dv_flow_action_attr *action; + + (void)offset; + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX; + action->obj = counter_obj; + return action; #else - (void)ctx; - (void)data_sz; - (void)data; - (void)reformat_type; - (void)ft_type; + (void)counter_obj; + (void)offset; return NULL; #endif } -static struct ibv_flow_action * +static void * +mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT + struct mlx5dv_flow_action_attr *action; + + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP; + action->obj = qp; + return action; +#else + (void)qp; + return NULL; +#endif +} + +static void * mlx5_glue_dv_create_flow_action_modify_header (struct ibv_context *ctx, size_t actions_sz, @@ -487,8 +518,15 @@ enum mlx5dv_flow_table_type ft_type) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT - return mlx5dv_create_flow_action_modify_header(ctx, actions_sz, - actions, ft_type); + struct mlx5dv_flow_action_attr *action; + + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; + action->action = mlx5dv_create_flow_action_modify_header + (ctx, actions_sz, actions, ft_type); + return action; #else (void)ctx; (void)actions_sz; @@ -498,6 +536,50 @@ #endif } +static void * +mlx5_glue_dv_create_flow_action_packet_reformat + (struct ibv_context *ctx, + size_t data_sz, + void *data, + enum mlx5dv_flow_action_packet_reformat_type reformat_type, + enum mlx5dv_flow_table_type ft_type) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT + struct mlx5dv_flow_action_attr *action; + + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; + action->action = mlx5dv_create_flow_action_packet_reformat + (ctx, data_sz, data, reformat_type, ft_type); + return action; +#else + (void)ctx; + (void)data_sz; + (void)data; + (void)reformat_type; + (void)ft_type; + return NULL; +#endif +} + +static void * +mlx5_glue_dv_create_flow_action_tag(uint32_t tag) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT + struct mlx5dv_flow_action_attr *action; + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_TAG; + action->tag_value = tag; + return action; +#endif + (void)tag; + return NULL; +} + static struct ibv_context * mlx5_glue_dv_open_device(struct ibv_device *device) { @@ -645,10 +727,15 @@ .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher, .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher, .dv_create_flow = mlx5_glue_dv_create_flow, - .dv_create_flow_action_packet_reformat = - mlx5_glue_dv_create_flow_action_packet_reformat, + .dv_create_flow_action_counter = + mlx5_glue_dv_create_flow_action_counter, + .dv_create_flow_action_dest_ibv_qp = + mlx5_glue_dv_create_flow_action_dest_ibv_qp, .dv_create_flow_action_modify_header = - mlx5_glue_dv_create_flow_action_modify_header, + mlx5_glue_dv_create_flow_action_modify_header, + .dv_create_flow_action_packet_reformat = + mlx5_glue_dv_create_flow_action_packet_reformat, + .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag, .dv_open_device = mlx5_glue_dv_open_device, .devx_obj_create = mlx5_glue_devx_obj_create, .devx_obj_destroy = mlx5_glue_devx_obj_destroy, diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index b118960..32487ea 100644 --- a/drivers/net/mlx5/mlx5_glue.h +++ b/drivers/net/mlx5/mlx5_glue.h @@ -55,6 +55,10 @@ enum mlx5dv_flow_table_type { flow_table_type = 0, }; #endif +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS +#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0 +#endif + #ifndef HAVE_IBV_DEVX_OBJ struct mlx5dv_devx_obj; #endif @@ -98,7 +102,7 @@ struct mlx5_glue { struct ibv_flow *(*create_flow)(struct ibv_qp *qp, struct ibv_flow_attr *flow); int (*destroy_flow)(struct ibv_flow *flow_id); - int (*destroy_flow_action)(struct ibv_flow_action *action); + int (*destroy_flow_action)(void *action); struct ibv_qp *(*create_qp)(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr); struct ibv_qp *(*create_qp_ex) @@ -160,19 +164,17 @@ struct mlx5_glue { int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher); struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher, struct mlx5dv_flow_match_parameters *match_value, - size_t num_actions, - struct mlx5dv_flow_action_attr *actions_attr); - struct ibv_flow_action *(*dv_create_flow_action_packet_reformat) - (struct ibv_context *ctx, - size_t data_sz, - void *data, + size_t num_actions, void *actions[]); + void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset); + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp); + void *(*dv_create_flow_action_modify_header) + (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[], + enum mlx5dv_flow_table_type ft_type); + void *(*dv_create_flow_action_packet_reformat) + (struct ibv_context *ctx, size_t data_sz, void *data, enum mlx5dv_flow_action_packet_reformat_type reformat_type, enum mlx5dv_flow_table_type ft_type); - struct ibv_flow_action *(*dv_create_flow_action_modify_header) - (struct ibv_context *ctx, - size_t actions_sz, - uint64_t actions[], - enum mlx5dv_flow_table_type ft_type); + void *(*dv_create_flow_action_tag)(uint32_t tag); struct ibv_context *(*dv_open_device)(struct ibv_device *device); struct mlx5dv_devx_obj *(*devx_obj_create) (struct ibv_context *ctx, From patchwork Thu Apr 4 09:54:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 52270 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5A2021B120; Thu, 4 Apr 2019 11:54:13 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70049.outbound.protection.outlook.com [40.107.7.49]) by dpdk.org (Postfix) with ESMTP id 5AAA11B10C for ; Thu, 4 Apr 2019 11:54:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IzUgarIxJJdK0noIyAbFbgknc3PfpTp8yP6k5mrjvK4=; b=doqrve5bErY6y6g/3kzRdMFFIaj8pjqxmKrXuyT+aJd5/PxybtbEhOkmervzWzO/Lp756agFrNnBzsd1LAYtGg1tJ/lhzLYF7H49WKxAASyIX9kbUuhLjoCS9F/RhLJ/vkiYih9HBz7bn8AQzCCZhqOAfn2esOTGDvqzSUlCJk4= Received: from AM4PR05MB3425.eurprd05.prod.outlook.com (10.171.190.15) by AM4PR05MB3282.eurprd05.prod.outlook.com (10.171.187.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.19; Thu, 4 Apr 2019 09:54:07 +0000 Received: from AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827]) by AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827%4]) with mapi id 15.20.1771.014; Thu, 4 Apr 2019 09:54:07 +0000 From: Ori Kam To: Matan Azrad , Yongseok Koh , Shahaf Shuler CC: "dev@dpdk.org" , Ori Kam , Slava Ovsiienko Thread-Topic: [PATCH v4 2/3] net/mlx5: add Direct Rules API Thread-Index: AQHU6sxY/0kUf/04f0Gf0SFatvgkXg== Date: Thu, 4 Apr 2019 09:54:07 +0000 Message-ID: <1554371628-170844-3-git-send-email-orika@mellanox.com> References: <1553790741-69362-1-git-send-email-orika@mellanox.com> <1554371628-170844-1-git-send-email-orika@mellanox.com> In-Reply-To: <1554371628-170844-1-git-send-email-orika@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: LO2P265CA0363.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a3::15) To AM4PR05MB3425.eurprd05.prod.outlook.com (2603:10a6:205:b::15) authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 1.8.3.1 x-originating-ip: [37.142.13.130] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 11a6af96-a61c-4491-4c57-08d6b8e37a93 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM4PR05MB3282; x-ms-traffictypediagnostic: AM4PR05MB3282: x-microsoft-antispam-prvs: x-forefront-prvs: 0997523C40 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(366004)(39860400002)(136003)(346002)(396003)(199004)(189003)(256004)(446003)(386003)(97736004)(102836004)(6506007)(6116002)(6436002)(8676002)(316002)(11346002)(53936002)(2906002)(81156014)(107886003)(478600001)(5660300002)(54906003)(81166006)(2616005)(53946003)(476003)(110136005)(6486002)(8936002)(6512007)(50226002)(76176011)(52116002)(14454004)(86362001)(36756003)(26005)(14444005)(486006)(71200400001)(4326008)(4720700003)(3846002)(186003)(66066001)(68736007)(25786009)(305945005)(106356001)(99286004)(6636002)(30864003)(105586002)(7736002)(71190400001)(559001)(579004); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3282; H:AM4PR05MB3425.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: zou5FQ1q8anvnfpHIbbjc5pgjVKR+TXEigIL8WnNz7RKLSlxhFGKNPqO+ZYZxtYM1ENAuEVmosqa/NGKRYHufE63ZYnY63SkC1uPCUUqSi/h6fKPk3feRnrVr2hjI2I0hFUyMmkumxFwavdN6gEcYxDDN4dVRzTTZVbnGsykkuJctTr9Oj6dSeDcyRSENUL2JdWVZKO9VkjVj72aa80tl/HUUPOYYMxsxvg+uGZzYXFqwn3rTpyW86QrGy65KtVN49aiciOdDhZULbs9xPHNTP+dh0qR+J9IH1V1XjZahmSeJtcMV8Y0Oc0Y7FrJoww3TaH1OemeEyYeS8ZhaJ4juECnHXTS9J/6DoElEEqiGkz1oscv4bS/Rr4DKXgMVGfQqMDr1xoaviIKFl8kGQvhzWGCfzKiYeYv7HPq9mPR/Oc= MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 11a6af96-a61c-4491-4c57-08d6b8e37a93 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Apr 2019 09:54:07.7017 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3282 Subject: [dpdk-dev] [PATCH v4 2/3] net/mlx5: add Direct Rules API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adds calls to the Direct Rules API inside the glue functions. Due to difference in parameters between the Direct Rules and Direct Verbs some of the glue functions API was updated. Signed-off-by: Ori Kam Acked-by: Shahaf Shuler --- drivers/net/mlx5/Makefile | 5 ++ drivers/net/mlx5/mlx5.c | 16 ++++ drivers/net/mlx5/mlx5.h | 15 ++++ drivers/net/mlx5/mlx5_flow.c | 1 + drivers/net/mlx5/mlx5_flow.h | 6 +- drivers/net/mlx5/mlx5_flow_dv.c | 103 ++++++++++++++++++++---- drivers/net/mlx5/mlx5_glue.c | 170 ++++++++++++++++++++++++++++++++++------ drivers/net/mlx5/mlx5_glue.h | 31 +++++--- drivers/net/mlx5/mlx5_prm.h | 24 +++++- 9 files changed, 318 insertions(+), 53 deletions(-) diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index 0d20f0f..93bc869 100644 --- a/drivers/net/mlx5/Makefile +++ b/drivers/net/mlx5/Makefile @@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh func mlx5dv_create_flow_action_packet_reformat \ $(AUTOCONF_OUTPUT) $Q sh -- '$<' '$@' \ + HAVE_MLX5DV_DR \ + infiniband/mlx5dv.h \ + enum MLX5DV_DR_NS_TYPE_TERMINATING \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ HAVE_IBV_DEVX_OBJ \ infiniband/mlx5dv.h \ func mlx5dv_devx_obj_create \ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 4044505..65aa9cf 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1483,6 +1483,22 @@ struct mlx5_dev_spawn_data { priv->tcf_context = NULL; } } +#ifdef HAVE_MLX5DV_DR + priv->rx_ns = mlx5dv_dr_create_ns + (sh->ctx, MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS); + if (priv->rx_ns == NULL) { + DRV_LOG(ERR, "mlx5dv_dr_create_ns failed"); + err = errno; + goto error; + } + priv->tx_ns = mlx5dv_dr_create_ns(sh->ctx, + MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS); + if (priv->tx_ns == NULL) { + DRV_LOG(ERR, "mlx5dv_dr_create_ns failed"); + err = errno; + goto error; + } +#endif TAILQ_INIT(&priv->flows); TAILQ_INIT(&priv->ctrl_flows); /* Hint libmlx5 to use PMD allocator for data plane resources */ diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 784bf9b..d4963cb 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -259,6 +259,15 @@ struct mlx5_ibv_shared { struct mlx5_ibv_shared_port port[]; /* per device port data array. */ }; +/* Table structure. */ +struct mlx5_flow_tbl_resource { + void *obj; /**< Pointer to DR table object. */ + rte_atomic32_t refcnt; /**< Reference counter. */ +}; + +#define MLX5_MAX_TABLES 1024 +#define MLX5_GROUP_FACTOR 1 + struct mlx5_priv { LIST_ENTRY(mlx5_priv) mem_event_cb; /**< Called by memory event callback. */ @@ -326,6 +335,12 @@ struct mlx5_priv { /* UAR same-page access control required in 32bit implementations. */ #endif struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */ + void *rx_ns; /* RX Direct Rules name space handle. */ + struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES]; + /* RX Direct Rules tables. */ + void *tx_ns; /* TX Direct Rules name space handle. */ + struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES]; + /* TX Direct Rules tables/ */ }; #define PORT_ID(priv) ((priv)->dev_data->port_id) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index bc6a7c1..9dc492a 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -2084,6 +2084,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void *)); flow = rte_calloc(__func__, 1, flow_size, 0); flow->drv_type = flow_get_drv_type(dev, attr); + flow->ingress = attr->ingress; assert(flow->drv_type > MLX5_FLOW_TYPE_MIN && flow->drv_type < MLX5_FLOW_TYPE_MAX); flow->queue = (void *)(flow + 1); diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 4f69ae2..8ba37a0 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher { uint16_t crc; /**< CRC of key. */ uint16_t priority; /**< Priority of matcher. */ uint8_t egress; /**< Egress matcher. */ + uint32_t group; /**< The matcher group. */ struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */ }; @@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource { size_t size; uint8_t reformat_type; uint8_t ft_type; + uint64_t flags; /**< Flags for RDMA API. */ }; /* Tag resource structure. */ @@ -348,7 +350,7 @@ struct mlx5_flow_counter { /* Flow structure. */ struct rte_flow { TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ - enum mlx5_flow_drv_type drv_type; /**< Drvier type. */ + enum mlx5_flow_drv_type drv_type; /**< Driver type. */ struct mlx5_flow_counter *counter; /**< Holds flow counter. */ struct mlx5_flow_dv_tag_resource *tag_resource; /**< pointer to the tag action. */ @@ -360,6 +362,8 @@ struct rte_flow { uint64_t actions; /**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */ struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */ + uint8_t ingress; /**< 1 if the flow is ingress. */ + uint32_t group; /**< The group index. */ }; typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev, diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index d9e2ac9..dd5b541 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -809,11 +809,20 @@ struct field_modify_info modify_tcp[] = { { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_flow_dv_encap_decap_resource *cache_resource; + struct rte_flow *flow = dev_flow->flow; + struct mlx5dv_dr_ns *ns; + + resource->flags = flow->group ? 0 : 1; + if (flow->ingress) + ns = priv->rx_ns; + else + ns = priv->tx_ns; /* Lookup a matching resource from cache. */ LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) { if (resource->reformat_type == cache_resource->reformat_type && resource->ft_type == cache_resource->ft_type && + resource->flags == cache_resource->flags && resource->size == cache_resource->size && !memcmp((const void *)resource->buf, (const void *)cache_resource->buf, @@ -835,10 +844,10 @@ struct field_modify_info modify_tcp[] = { *cache_resource = *resource; cache_resource->verbs_action = mlx5_glue->dv_create_flow_action_packet_reformat - (priv->sh->ctx, cache_resource->size, - (cache_resource->size ? cache_resource->buf : NULL), - cache_resource->reformat_type, - cache_resource->ft_type); + (priv->sh->ctx, cache_resource->reformat_type, + cache_resource->ft_type, ns, cache_resource->flags, + cache_resource->size, + (cache_resource->size ? cache_resource->buf : NULL)); if (!cache_resource->verbs_action) { rte_free(cache_resource); return rte_flow_error_set(error, ENOMEM, @@ -1442,6 +1451,10 @@ struct field_modify_info modify_tcp[] = { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_flow_dv_modify_hdr_resource *cache_resource; + struct mlx5dv_dr_ns *ns = + resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ? + priv->tx_ns : priv->rx_ns; + /* Lookup a matching resource from cache. */ LIST_FOREACH(cache_resource, &priv->modify_cmds, next) { if (resource->ft_type == cache_resource->ft_type && @@ -1467,11 +1480,11 @@ struct field_modify_info modify_tcp[] = { *cache_resource = *resource; cache_resource->verbs_action = mlx5_glue->dv_create_flow_action_modify_header - (priv->sh->ctx, + (priv->sh->ctx, cache_resource->ft_type, + ns, 0, cache_resource->actions_num * sizeof(cache_resource->actions[0]), - (uint64_t *)cache_resource->actions, - cache_resource->ft_type); + (uint64_t *)cache_resource->actions); if (!cache_resource->verbs_action) { rte_free(cache_resource); return rte_flow_error_set(error, ENOMEM, @@ -1596,11 +1609,13 @@ struct field_modify_info modify_tcp[] = { struct mlx5_priv *priv = dev->data->dev_private; uint32_t priority_max = priv->config.flow_prio - 1; +#ifdef HAVE_MLX5DV_DR if (attributes->group) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_GROUP, NULL, "groups is not supported"); +#endif if (attributes->priority != MLX5_FLOW_PRIO_RSVD && attributes->priority >= priority_max) return rte_flow_error_set(error, ENOTSUP, @@ -2173,11 +2188,13 @@ struct field_modify_info modify_tcp[] = { * Flow pattern to translate. * @param[in] inner * Item is inner pattern. + * @param[in] group + * The group to insert the rule. */ static void flow_dv_translate_item_ipv4(void *matcher, void *key, const struct rte_flow_item *item, - int inner) + int inner, uint32_t group) { const struct rte_flow_item_ipv4 *ipv4_m = item->mask; const struct rte_flow_item_ipv4 *ipv4_v = item->spec; @@ -2204,7 +2221,10 @@ struct field_modify_info modify_tcp[] = { outer_headers); headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers); } - MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf); + if (group == 0) + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf); + else + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x4); MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 4); if (!ipv4_v) return; @@ -2246,11 +2266,13 @@ struct field_modify_info modify_tcp[] = { * Flow pattern to translate. * @param[in] inner * Item is inner pattern. + * @param[in] group + * The group to insert the rule. */ static void flow_dv_translate_item_ipv6(void *matcher, void *key, const struct rte_flow_item *item, - int inner) + int inner, uint32_t group) { const struct rte_flow_item_ipv6 *ipv6_m = item->mask; const struct rte_flow_item_ipv6 *ipv6_v = item->spec; @@ -2287,7 +2309,10 @@ struct field_modify_info modify_tcp[] = { outer_headers); headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers); } - MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf); + if (group == 0) + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf); + else + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x6); MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 6); if (!ipv6_v) return; @@ -2727,7 +2752,11 @@ struct field_modify_info modify_tcp[] = { match_criteria_enable |= (!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) << MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT; - +#ifdef HAVE_MLX5DV_DR + match_criteria_enable |= + (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) << + MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT; +#endif return match_criteria_enable; } @@ -2758,12 +2787,14 @@ struct field_modify_info modify_tcp[] = { .type = IBV_FLOW_ATTR_NORMAL, .match_mask = (void *)&matcher->mask, }; + struct mlx5_flow_tbl_resource *tbl = NULL; /* Lookup from cache. */ LIST_FOREACH(cache_matcher, &priv->matchers, next) { if (matcher->crc == cache_matcher->crc && matcher->priority == cache_matcher->priority && matcher->egress == cache_matcher->egress && + matcher->group == cache_matcher->group && !memcmp((const void *)matcher->mask.buf, (const void *)cache_matcher->mask.buf, cache_matcher->mask.size)) { @@ -2778,6 +2809,27 @@ struct field_modify_info modify_tcp[] = { return 0; } } +#ifdef HAVE_MLX5DV_DR + if (matcher->egress) { + tbl = &priv->tx_tbl[matcher->group]; + if (!tbl->obj) + tbl->obj = mlx5_glue->dr_create_flow_tbl + (priv->tx_ns, + matcher->group * MLX5_GROUP_FACTOR); + } else { + tbl = &priv->rx_tbl[matcher->group]; + if (!tbl->obj) + tbl->obj = mlx5_glue->dr_create_flow_tbl + (priv->rx_ns, + matcher->group * MLX5_GROUP_FACTOR); + } + if (!tbl->obj) + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create table"); + + rte_atomic32_inc(&tbl->refcnt); +#endif /* Register new matcher. */ cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0); if (!cache_matcher) @@ -2791,9 +2843,16 @@ struct field_modify_info modify_tcp[] = { if (matcher->egress) dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS; cache_matcher->matcher_object = - mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr); + mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr, + tbl->obj); if (!cache_matcher->matcher_object) { rte_free(cache_matcher); +#ifdef HAVE_MLX5DV_DR + if (rte_atomic32_dec_and_test(&tbl->refcnt)) { + mlx5_glue->dr_destroy_flow_tbl(tbl->obj); + tbl->obj = NULL; + } +#endif return rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, "cannot create matcher"); @@ -2805,6 +2864,7 @@ struct field_modify_info modify_tcp[] = { cache_matcher->priority, cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher, rte_atomic32_read(&cache_matcher->refcnt)); + rte_atomic32_inc(&tbl->refcnt); return 0; } @@ -3226,7 +3286,7 @@ struct field_modify_info modify_tcp[] = { break; case RTE_FLOW_ITEM_TYPE_IPV4: flow_dv_translate_item_ipv4(match_mask, match_value, - items, tunnel); + items, tunnel, attr->group); matcher.priority = MLX5_PRIORITY_MAP_L3; dev_flow->dv.hash_fields |= mlx5_flow_hashfields_adjust @@ -3238,7 +3298,7 @@ struct field_modify_info modify_tcp[] = { break; case RTE_FLOW_ITEM_TYPE_IPV6: flow_dv_translate_item_ipv6(match_mask, match_value, - items, tunnel); + items, tunnel, attr->group); matcher.priority = MLX5_PRIORITY_MAP_L3; dev_flow->dv.hash_fields |= mlx5_flow_hashfields_adjust @@ -3316,6 +3376,7 @@ struct field_modify_info modify_tcp[] = { matcher.priority = mlx5_flow_adjust_priority(dev, priority, matcher.priority); matcher.egress = attr->egress; + matcher.group = attr->group; if (flow_dv_matcher_register(dev, &matcher, dev_flow, error)) return -rte_errno; return 0; @@ -3431,6 +3492,8 @@ struct field_modify_info modify_tcp[] = { struct mlx5_flow *flow) { struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher; + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_flow_tbl_resource *tbl; assert(matcher->matcher_object); DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--", @@ -3440,6 +3503,14 @@ struct field_modify_info modify_tcp[] = { claim_zero(mlx5_glue->dv_destroy_flow_matcher (matcher->matcher_object)); LIST_REMOVE(matcher, next); + if (matcher->egress) + tbl = &priv->tx_tbl[matcher->group]; + else + tbl = &priv->rx_tbl[matcher->group]; + if (rte_atomic32_dec_and_test(&tbl->refcnt)) { + mlx5_glue->dr_destroy_flow_tbl(tbl->obj); + tbl->obj = NULL; + } rte_free(matcher); DRV_LOG(DEBUG, "port %u matcher %p: removed", dev->data->port_id, (void *)matcher); @@ -3529,7 +3600,7 @@ struct field_modify_info modify_tcp[] = { LIST_FOREACH(dev_flow, &flow->dev_flows, next) { dv = &dev_flow->dv; if (dv->flow) { - claim_zero(mlx5_glue->destroy_flow(dv->flow)); + claim_zero(mlx5_glue->dv_destroy_flow(dv->flow)); dv->flow = NULL; } if (dv->hrxq) { diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c index 4b5aade..b0b144c 100644 --- a/drivers/net/mlx5/mlx5_glue.c +++ b/drivers/net/mlx5/mlx5_glue.c @@ -178,6 +178,9 @@ mlx5_glue_destroy_flow_action(void *action) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_destroy_action(action); +#else struct mlx5dv_flow_action_attr *attr = action; int res = 0; switch (attr->type) { @@ -189,6 +192,7 @@ } free(action); return res; +#endif #else (void)action; return ENOTSUP; @@ -365,6 +369,53 @@ return ibv_cq_ex_to_cq(cq); } +static void * +mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_create_ft(ns, level); +#else + (void)ns; + (void)level; + return NULL; +#endif +} + +static int +mlx5_glue_dr_destroy_flow_tbl(void *tbl) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_destroy_ft(tbl); +#else + (void)tbl; + return 0; +#endif +} + +static void * +mlx5_glue_dr_create_ns(struct ibv_context *ctx, + enum mlx5dv_dr_ns_domain domain) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_create_ns(ctx, domain); +#else + (void)ctx; + (void)domain; + return NULL; +#endif +} + +static int +mlx5_glue_dr_destroy_ns(void *ns) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_destroy_ns(ns); +#else + (void)ns; + return 0; +#endif +} + static struct ibv_cq_ex * mlx5_glue_dv_create_cq(struct ibv_context *context, struct ibv_cq_init_attr_ex *cq_attr, @@ -423,26 +474,40 @@ #endif } -static struct mlx5dv_flow_matcher * +static void * mlx5_glue_dv_create_flow_matcher(struct ibv_context *context, - struct mlx5dv_flow_matcher_attr *matcher_attr) + struct mlx5dv_flow_matcher_attr *matcher_attr, + void *tbl) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + (void)context; + return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority, + matcher_attr->match_criteria_enable, + matcher_attr->match_mask); +#else + (void)tbl; return mlx5dv_create_flow_matcher(context, matcher_attr); +#endif #else (void)context; (void)matcher_attr; + (void)tbl; return NULL; #endif } -static struct ibv_flow * -mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher, - struct mlx5dv_flow_match_parameters *match_value, +static void * +mlx5_glue_dv_create_flow(void *matcher, + void *match_value, size_t num_actions, void *actions[]) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_create_rule(matcher, match_value, num_actions, + (struct mlx5dv_dr_action **)actions); +#else struct mlx5dv_flow_action_attr actions_attr[8]; if (num_actions > 8) @@ -452,6 +517,7 @@ *((struct mlx5dv_flow_action_attr *)(actions[i])); return mlx5dv_create_flow(matcher, match_value, num_actions, actions_attr); +#endif #else (void)matcher; (void)match_value; @@ -461,21 +527,13 @@ #endif } -static int -mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher *matcher) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT - return mlx5dv_destroy_flow_matcher(matcher); -#else - (void)matcher; - return 0; -#endif -} - static void * mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_create_action_devx_counter(counter_obj, offset); +#else struct mlx5dv_flow_action_attr *action; (void)offset; @@ -485,6 +543,7 @@ action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX; action->obj = counter_obj; return action; +#endif #else (void)counter_obj; (void)offset; @@ -496,6 +555,9 @@ mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_create_action_dest_ibv_qp(qp); +#else struct mlx5dv_flow_action_attr *action; action = malloc(sizeof(*action)); @@ -504,6 +566,7 @@ action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP; action->obj = qp; return action; +#endif #else (void)qp; return NULL; @@ -513,13 +576,22 @@ static void * mlx5_glue_dv_create_flow_action_modify_header (struct ibv_context *ctx, + enum mlx5dv_flow_table_type ft_type, + void *ns, uint64_t flags, size_t actions_sz, - uint64_t actions[], - enum mlx5dv_flow_table_type ft_type) + uint64_t actions[]) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + (void)ctx; + (void)ft_type; + return mlx5dv_dr_create_action_modify_header(ns, flags, actions_sz, + actions); +#else struct mlx5dv_flow_action_attr *action; + (void)ns; + (void)flags; action = malloc(sizeof(*action)); if (!action) return NULL; @@ -527,11 +599,14 @@ action->action = mlx5dv_create_flow_action_modify_header (ctx, actions_sz, actions, ft_type); return action; +#endif #else (void)ctx; + (void)ft_type; + (void)ns; + (void)flags; (void)actions_sz; (void)actions; - (void)ft_type; return NULL; #endif } @@ -539,12 +614,20 @@ static void * mlx5_glue_dv_create_flow_action_packet_reformat (struct ibv_context *ctx, - size_t data_sz, - void *data, enum mlx5dv_flow_action_packet_reformat_type reformat_type, - enum mlx5dv_flow_table_type ft_type) + enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns, + uint32_t flags, size_t data_sz, void *data) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + (void)ctx; + (void)ft_type; + return mlx5dv_dr_create_action_packet_reformat(ns, flags, + reformat_type, data_sz, + data); +#else + (void)ns; + (void)flags; struct mlx5dv_flow_action_attr *action; action = malloc(sizeof(*action)); @@ -554,12 +637,15 @@ action->action = mlx5dv_create_flow_action_packet_reformat (ctx, data_sz, data, reformat_type, ft_type); return action; +#endif #else (void)ctx; - (void)data_sz; - (void)data; (void)reformat_type; (void)ft_type; + (void)ns; + (void)flags; + (void)data_sz; + (void)data; return NULL; #endif } @@ -568,6 +654,9 @@ mlx5_glue_dv_create_flow_action_tag(uint32_t tag) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_create_action_tag(tag); +#else struct mlx5dv_flow_action_attr *action; action = malloc(sizeof(*action)); if (!action) @@ -576,10 +665,36 @@ action->tag_value = tag; return action; #endif +#endif (void)tag; return NULL; } +static int +mlx5_glue_dv_destroy_flow(void *flow_id) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_destroy_rule(flow_id); +#else + return ibv_destroy_flow(flow_id); +#endif +} + +static int +mlx5_glue_dv_destroy_flow_matcher(void *matcher) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_destroy_matcher(matcher); +#else + return mlx5dv_destroy_flow_matcher(matcher); +#endif +#else + (void)matcher; + return 0; +#endif +} + static struct ibv_context * mlx5_glue_dv_open_device(struct ibv_device *device) { @@ -718,6 +833,10 @@ .get_async_event = mlx5_glue_get_async_event, .port_state_str = mlx5_glue_port_state_str, .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq, + .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl, + .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl, + .dr_create_ns = mlx5_glue_dr_create_ns, + .dr_destroy_ns = mlx5_glue_dr_destroy_ns, .dv_create_cq = mlx5_glue_dv_create_cq, .dv_create_wq = mlx5_glue_dv_create_wq, .dv_query_device = mlx5_glue_dv_query_device, @@ -725,7 +844,6 @@ .dv_init_obj = mlx5_glue_dv_init_obj, .dv_create_qp = mlx5_glue_dv_create_qp, .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher, - .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher, .dv_create_flow = mlx5_glue_dv_create_flow, .dv_create_flow_action_counter = mlx5_glue_dv_create_flow_action_counter, @@ -736,6 +854,8 @@ .dv_create_flow_action_packet_reformat = mlx5_glue_dv_create_flow_action_packet_reformat, .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag, + .dv_destroy_flow = mlx5_glue_dv_destroy_flow, + .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher, .dv_open_device = mlx5_glue_dv_open_device, .devx_obj_create = mlx5_glue_devx_obj_create, .devx_obj_destroy = mlx5_glue_devx_obj_destroy, diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index 32487ea..eb29ffa 100644 --- a/drivers/net/mlx5/mlx5_glue.h +++ b/drivers/net/mlx5/mlx5_glue.h @@ -63,6 +63,11 @@ struct mlx5dv_devx_obj; #endif +#ifndef HAVE_MLX5DV_DR +struct mlx5dv_dr_ns; +enum mlx5dv_dr_ns_domain { unused, }; +#endif + /* LIB_GLUE_VERSION must be updated every time this structure is modified. */ struct mlx5_glue { const char *version; @@ -140,6 +145,11 @@ struct mlx5_glue { struct ibv_async_event *event); const char *(*port_state_str)(enum ibv_port_state port_state); struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq); + void *(*dr_create_flow_tbl)(void *ns, uint32_t level); + int (*dr_destroy_flow_tbl)(void *tbl); + void *(*dr_create_ns)(struct ibv_context *ctx, + enum mlx5dv_dr_ns_domain domain); + int (*dr_destroy_ns)(void *ns); struct ibv_cq_ex *(*dv_create_cq) (struct ibv_context *context, struct ibv_cq_init_attr_ex *cq_attr, @@ -158,23 +168,26 @@ struct mlx5_glue { (struct ibv_context *context, struct ibv_qp_init_attr_ex *qp_init_attr_ex, struct mlx5dv_qp_init_attr *dv_qp_init_attr); - struct mlx5dv_flow_matcher *(*dv_create_flow_matcher) + void *(*dv_create_flow_matcher) (struct ibv_context *context, - struct mlx5dv_flow_matcher_attr *matcher_attr); - int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher); - struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher, - struct mlx5dv_flow_match_parameters *match_value, + struct mlx5dv_flow_matcher_attr *matcher_attr, + void *tbl); + void *(*dv_create_flow)(void *matcher, void *match_value, size_t num_actions, void *actions[]); void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset); void *(*dv_create_flow_action_dest_ibv_qp)(void *qp); void *(*dv_create_flow_action_modify_header) - (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[], - enum mlx5dv_flow_table_type ft_type); + (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type, + void *ns, uint64_t flags, size_t actions_sz, + uint64_t actions[]); void *(*dv_create_flow_action_packet_reformat) - (struct ibv_context *ctx, size_t data_sz, void *data, + (struct ibv_context *ctx, enum mlx5dv_flow_action_packet_reformat_type reformat_type, - enum mlx5dv_flow_table_type ft_type); + enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns, + uint32_t flags, size_t data_sz, void *data); void *(*dv_create_flow_action_tag)(uint32_t tag); + int (*dv_destroy_flow)(void *flow); + int (*dv_destroy_flow_matcher)(void *matcher); struct ibv_context *(*dv_open_device)(struct ibv_device *device); struct mlx5dv_devx_obj *(*devx_obj_create) (struct ibv_context *ctx, diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index da1219e..b15266f 100644 --- a/drivers/net/mlx5/mlx5_prm.h +++ b/drivers/net/mlx5/mlx5_prm.h @@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits { u8 reserved_at_1a0[0x60]; }; +struct mlx5_ifc_fte_match_set_misc3_bits { + u8 inner_tcp_seq_num[0x20]; + u8 outer_tcp_seq_num[0x20]; + u8 inner_tcp_ack_num[0x20]; + u8 outer_tcp_ack_num[0x20]; + u8 reserved_at_auto1[0x8]; + u8 outer_vxlan_gpe_vni[0x18]; + u8 outer_vxlan_gpe_next_protocol[0x8]; + u8 outer_vxlan_gpe_flags[0x8]; + u8 reserved_at_a8[0x10]; + u8 icmp_header_data[0x20]; + u8 icmpv6_header_data[0x20]; + u8 icmp_type[0x8]; + u8 icmp_code[0x8]; + u8 icmpv6_type[0x8]; + u8 icmpv6_code[0x8]; + u8 reserved_at_1a0[0xe0]; +}; + /* Flow matcher. */ struct mlx5_ifc_fte_match_param_bits { struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers; struct mlx5_ifc_fte_match_set_misc_bits misc_parameters; struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers; struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2; - u8 reserved_at_800[0x800]; + struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3; }; enum { MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT, MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT, MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT, - MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT + MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT, + MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT }; enum { From patchwork Thu Apr 4 09:54:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ori Kam X-Patchwork-Id: 52271 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A87971B136; Thu, 4 Apr 2019 11:54:15 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70049.outbound.protection.outlook.com [40.107.7.49]) by dpdk.org (Postfix) with ESMTP id A48CF1B114 for ; Thu, 4 Apr 2019 11:54:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZIXeWTG8IWs0ds+dFr5gPGbukNMmbv6QK0eLk3C7v+Y=; b=rXTVI1y65cMPngxKixB9OB4jAc4gqZyGLRfm0p+buPrVguRytpp8gD3ecKWCiPLrD+NbzmVDRvjNgU461wAOUuXjTw8mfMOraMseVROA3R66R94F7JVT6Q0edjtclOh6k/YAzM8db4tQ2YkFCb+iJ0yi7QKY3ZmHtlsTiCId5TQ= Received: from AM4PR05MB3425.eurprd05.prod.outlook.com (10.171.190.15) by AM4PR05MB3282.eurprd05.prod.outlook.com (10.171.187.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.19; Thu, 4 Apr 2019 09:54:08 +0000 Received: from AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827]) by AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827%4]) with mapi id 15.20.1771.014; Thu, 4 Apr 2019 09:54:08 +0000 From: Ori Kam To: Matan Azrad , Yongseok Koh , Shahaf Shuler CC: "dev@dpdk.org" , Ori Kam , Slava Ovsiienko Thread-Topic: [PATCH v4 3/3] net/mlx5: add jump action support for NIC Thread-Index: AQHU6sxYVemT+nideEqytG2zNK8Spg== Date: Thu, 4 Apr 2019 09:54:08 +0000 Message-ID: <1554371628-170844-4-git-send-email-orika@mellanox.com> References: <1553790741-69362-1-git-send-email-orika@mellanox.com> <1554371628-170844-1-git-send-email-orika@mellanox.com> In-Reply-To: <1554371628-170844-1-git-send-email-orika@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: LO2P265CA0363.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a3::15) To AM4PR05MB3425.eurprd05.prod.outlook.com (2603:10a6:205:b::15) authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 1.8.3.1 x-originating-ip: [37.142.13.130] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 82662b30-3303-435a-8f1c-08d6b8e37b46 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM4PR05MB3282; x-ms-traffictypediagnostic: AM4PR05MB3282: x-microsoft-antispam-prvs: x-forefront-prvs: 0997523C40 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(366004)(39860400002)(136003)(346002)(396003)(199004)(189003)(256004)(446003)(386003)(97736004)(102836004)(6506007)(6116002)(6436002)(8676002)(316002)(11346002)(53936002)(2906002)(81156014)(107886003)(478600001)(5660300002)(54906003)(81166006)(2616005)(53946003)(476003)(110136005)(6486002)(8936002)(6512007)(50226002)(76176011)(52116002)(14454004)(86362001)(36756003)(26005)(14444005)(486006)(71200400001)(4326008)(4720700003)(3846002)(186003)(66066001)(68736007)(25786009)(305945005)(106356001)(99286004)(6636002)(30864003)(105586002)(7736002)(71190400001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3282; H:AM4PR05MB3425.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: xHqE2gRZ43H/zo4dgsZ+72AyNWZtLrX6o5H6Ibim6nfqqoOjBcblfELWWNpnRapeiwsa+j3TrdvzGy9rs/QmdPSssYcP0WbANqhDpRPmZtXNrZm2ahiCe2PIP2PmBY9WU/jMabelfGcZ+qtlZL1cXFc1uPoYQ/SPikYvn036fJCTmAwJFlIk2HB9/ZHZtx5S1r190DJMo1nTHW7VmvP+k3zKBrVaDuv6JL5NPOeo1no36ImGdHguoW0GQYj8UMdDNvX7cTMFgGfJ8q5G+apKl2AiG1h+j+9FnBY9ButPkVONUbHSkSTzpnqyl/U6zuRX8a5DpMR0JTQYW9H/n9r7Hq5DHp9o0cwzgo95pO0JuX1clKxXyE5ebA2FE8UlAoBJ1Av9dRM+VxCxN2B02Pt6hNY4bQtRs6BATuvArdqLLHM= MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 82662b30-3303-435a-8f1c-08d6b8e37b46 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Apr 2019 09:54:08.7884 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3282 Subject: [dpdk-dev] [PATCH v4 3/3] net/mlx5: add jump action support for NIC X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When using Direct Rules we can add actions to jump between tables. This is extra useful since rule insertion rate is much higher on other tables compared to table zero. if no group is selected the rule is added to group 0. Signed-off-by: Ori Kam Acked-by: Shahaf Shuler --- v4: * Fix compilation issue. --- drivers/net/mlx5/mlx5.h | 6 + drivers/net/mlx5/mlx5_flow.h | 15 ++- drivers/net/mlx5/mlx5_flow_dv.c | 280 +++++++++++++++++++++++++++++++++++----- drivers/net/mlx5/mlx5_glue.c | 13 ++ drivers/net/mlx5/mlx5_glue.h | 1 + 5 files changed, 284 insertions(+), 31 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index d4963cb..5a7597e 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -319,6 +319,12 @@ struct mlx5_priv { LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps; LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds; LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags; + LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl; + /* Pointer to next element. */ + rte_atomic32_t refcnt; /**< Reference counter. */ + struct ibv_flow_action *verbs_action; + /**< Verbs modify header action object. */ + uint8_t ft_type; /**< Flow table type, Rx or Tx. */ /* Tags resources cache. */ uint32_t link_speed_capa; /* Link speed capabilities. */ struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */ diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 8ba37a0..622e305 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -115,7 +115,8 @@ #define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27) #define MLX5_FLOW_FATE_ACTIONS \ - (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS) + (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \ + MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP) #define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \ MLX5_FLOW_ACTION_NVGRE_ENCAP | \ @@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource { /**< Modification actions. */ }; +/* Jump action resource structure. */ +struct mlx5_flow_dv_jump_tbl_resource { + LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next; + /* Pointer to next element. */ + rte_atomic32_t refcnt; /**< Reference counter. */ + void *action; /**< Pointer to the rdma core action. */ + uint8_t ft_type; /**< Flow table type, Rx or Tx. */ + struct mlx5_flow_tbl_resource *tbl; /**< The target table. */ +}; + /* * Max number of actions per DV flow. * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED @@ -270,6 +281,8 @@ struct mlx5_flow_dv { struct mlx5_flow_dv_modify_hdr_resource *modify_hdr; /**< Pointer to modify header resource in cache. */ struct ibv_flow *flow; /**< Installed flow. */ + struct mlx5_flow_dv_jump_tbl_resource *jump; + /**< Pointer to the jump action resource. */ #ifdef HAVE_IBV_FLOW_DV_SUPPORT void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS]; /**< Action list. */ diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index dd5b541..10c6eee 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -865,6 +865,68 @@ struct field_modify_info modify_tcp[] = { } /** + * Find existing table jump resource or create and register a new one. + * + * @param dev[in, out] + * Pointer to rte_eth_dev structure. + * @param[in, out] resource + * Pointer to jump table resource. + * @parm[in, out] dev_flow + * Pointer to the dev_flow. + * @param[out] error + * pointer to error structure. + * + * @return + * 0 on success otherwise -errno and errno is set. + */ +static int +flow_dv_jump_tbl_resource_register + (struct rte_eth_dev *dev, + struct mlx5_flow_dv_jump_tbl_resource *resource, + struct mlx5_flow *dev_flow, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_flow_dv_jump_tbl_resource *cache_resource; + + /* Lookup a matching resource from cache. */ + LIST_FOREACH(cache_resource, &priv->jump_tbl, next) { + if (resource->tbl == cache_resource->tbl) { + DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + rte_atomic32_inc(&cache_resource->refcnt); + dev_flow->dv.jump = cache_resource; + return 0; + } + } + /* Register new jump table resource. */ + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0); + if (!cache_resource) + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "cannot allocate resource memory"); + *cache_resource = *resource; + cache_resource->action = + mlx5_glue->dr_create_flow_action_dest_flow_tbl + (resource->tbl->obj); + if (!cache_resource->action) { + rte_free(cache_resource); + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create action"); + } + rte_atomic32_init(&cache_resource->refcnt); + rte_atomic32_inc(&cache_resource->refcnt); + LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next); + dev_flow->dv.jump = cache_resource; + DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + return 0; +} + +/** * Get the size of specific rte_flow_item_type * * @param[in] item_type @@ -1427,6 +1489,37 @@ struct field_modify_info modify_tcp[] = { } /** + * Validate jump action. + * + * @param[in] action + * Pointer to the modify action. + * @param[in] group + * The group of the current flow. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_action_jump(const struct rte_flow_action *action, + uint32_t group, + struct rte_flow_error *error) +{ + if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, "action configuration not set"); + if (group >= ((const struct rte_flow_action_jump *)action->conf)->group) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "target group must be higher then" + " the current flow group"); + return 0; +} + + +/** * Find existing modify-header resource or create and register a new one. * * @param dev[in, out] @@ -1609,7 +1702,7 @@ struct field_modify_info modify_tcp[] = { struct mlx5_priv *priv = dev->data->dev_private; uint32_t priority_max = priv->config.flow_prio - 1; -#ifdef HAVE_MLX5DV_DR +#ifndef HAVE_MLX5DV_DR if (attributes->group) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_GROUP, @@ -1981,6 +2074,14 @@ struct field_modify_info modify_tcp[] = { MLX5_FLOW_ACTION_SET_TTL : MLX5_FLOW_ACTION_DEC_TTL; break; + case RTE_FLOW_ACTION_TYPE_JUMP: + ret = flow_dv_validate_action_jump(actions, + attr->group, error); + if (ret) + return ret; + ++actions_n; + action_flags |= MLX5_FLOW_ACTION_JUMP; + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -2760,6 +2861,82 @@ struct field_modify_info modify_tcp[] = { return match_criteria_enable; } + +/** + * Get a flow table. + * + * @param dev[in, out] + * Pointer to rte_eth_dev structure. + * @param[in] table_id + * Table id to use. + * @param[in] egress + * Direction of the table. + * @param[out] error + * pointer to error structure. + * + * @return + * Returns tables resource based on the index, NULL in case of failed. + */ +static struct mlx5_flow_tbl_resource * +flow_dv_tbl_resource_get(struct rte_eth_dev *dev, + uint32_t table_id, uint8_t egress, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_flow_tbl_resource *tbl; + +#ifdef HAVE_MLX5DV_DR + if (egress) { + tbl = &priv->tx_tbl[table_id]; + if (!tbl->obj) + tbl->obj = mlx5_glue->dr_create_flow_tbl + (priv->tx_ns, table_id); + } else { + tbl = &priv->rx_tbl[table_id]; + if (!tbl->obj) + tbl->obj = mlx5_glue->dr_create_flow_tbl + (priv->rx_ns, table_id); + } + if (!tbl->obj) { + rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create table"); + return NULL; + } + rte_atomic32_inc(&tbl->refcnt); + return tbl; +#else + (void)error; + (void)tbl; + if (egress) + return &priv->tx_tbl[table_id]; + else + return &priv->rx_tbl[table_id]; +#endif +} + +/** + * Release a flow table. + * + * @param[in] tbl + * Table resource to be released. + * + * @return + * Returns 0 if table was released, else return 1; + */ +static int +flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl) +{ + if (!tbl) + return 0; + if (rte_atomic32_dec_and_test(&tbl->refcnt)) { + mlx5_glue->dr_destroy_flow_tbl(tbl->obj); + tbl->obj = NULL; + return 0; + } + return 1; +} + /** * Register the flow matcher. * @@ -2809,33 +2986,20 @@ struct field_modify_info modify_tcp[] = { return 0; } } -#ifdef HAVE_MLX5DV_DR - if (matcher->egress) { - tbl = &priv->tx_tbl[matcher->group]; - if (!tbl->obj) - tbl->obj = mlx5_glue->dr_create_flow_tbl - (priv->tx_ns, - matcher->group * MLX5_GROUP_FACTOR); - } else { - tbl = &priv->rx_tbl[matcher->group]; - if (!tbl->obj) - tbl->obj = mlx5_glue->dr_create_flow_tbl - (priv->rx_ns, - matcher->group * MLX5_GROUP_FACTOR); - } - if (!tbl->obj) - return rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, "cannot create table"); - - rte_atomic32_inc(&tbl->refcnt); -#endif /* Register new matcher. */ cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0); if (!cache_matcher) return rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, "cannot allocate matcher memory"); + tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR, + matcher->egress, error); + if (!tbl) { + rte_free(cache_matcher); + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create table"); + } *cache_matcher = *matcher; dv_attr.match_criteria_enable = flow_dv_matcher_enable(cache_matcher->mask.buf); @@ -2848,10 +3012,7 @@ struct field_modify_info modify_tcp[] = { if (!cache_matcher->matcher_object) { rte_free(cache_matcher); #ifdef HAVE_MLX5DV_DR - if (rte_atomic32_dec_and_test(&tbl->refcnt)) { - mlx5_glue->dr_destroy_flow_tbl(tbl->obj); - tbl->obj = NULL; - } + flow_dv_tbl_resource_release(tbl); #endif return rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, @@ -3037,6 +3198,9 @@ struct field_modify_info modify_tcp[] = { const struct rte_flow_action *action = actions; const struct rte_flow_action_count *count = action->conf; const uint8_t *rss_key; + const struct rte_flow_action_jump *jump_data; + struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource; + struct mlx5_flow_tbl_resource *tbl; switch (actions->type) { case RTE_FLOW_ACTION_TYPE_VOID: @@ -3175,6 +3339,31 @@ struct field_modify_info modify_tcp[] = { /* If decap is followed by encap, handle it at encap. */ action_flags |= MLX5_FLOW_ACTION_RAW_DECAP; break; + case RTE_FLOW_ACTION_TYPE_JUMP: + jump_data = action->conf; + tbl = flow_dv_tbl_resource_get(dev, jump_data->group * + MLX5_GROUP_FACTOR, + attr->egress, error); + if (!tbl) + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "cannot create jump action."); + jump_tbl_resource.tbl = tbl; + if (flow_dv_jump_tbl_resource_register + (dev, &jump_tbl_resource, dev_flow, error)) { + flow_dv_tbl_resource_release(tbl); + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "cannot create jump action."); + } + dev_flow->dv.actions[actions_n++] = + dev_flow->dv.jump->action; + action_flags |= MLX5_FLOW_ACTION_JUMP; + break; case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC: case RTE_FLOW_ACTION_TYPE_SET_MAC_DST: if (flow_dv_convert_action_modify_mac(&res, actions, @@ -3507,10 +3696,7 @@ struct field_modify_info modify_tcp[] = { tbl = &priv->tx_tbl[matcher->group]; else tbl = &priv->rx_tbl[matcher->group]; - if (rte_atomic32_dec_and_test(&tbl->refcnt)) { - mlx5_glue->dr_destroy_flow_tbl(tbl->obj); - tbl->obj = NULL; - } + flow_dv_tbl_resource_release(tbl); rte_free(matcher); DRV_LOG(DEBUG, "port %u matcher %p: removed", dev->data->port_id, (void *)matcher); @@ -3551,6 +3737,38 @@ struct field_modify_info modify_tcp[] = { } /** + * Release an jump to table action resource. + * + * @param flow + * Pointer to mlx5_flow. + * + * @return + * 1 while a reference on it exists, 0 when freed. + */ +static int +flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow) +{ + struct mlx5_flow_dv_jump_tbl_resource *cache_resource = + flow->dv.jump; + + assert(cache_resource->action); + DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) { + claim_zero(mlx5_glue->destroy_flow_action + (cache_resource->action)); + LIST_REMOVE(cache_resource, next); + flow_dv_tbl_resource_release(cache_resource->tbl); + rte_free(cache_resource); + DRV_LOG(DEBUG, "jump table resource %p: removed", + (void *)cache_resource); + return 0; + } + return 1; +} + +/** * Release a modify-header resource. * * @param flow @@ -3646,6 +3864,8 @@ struct field_modify_info modify_tcp[] = { flow_dv_encap_decap_resource_release(dev_flow); if (dev_flow->dv.modify_hdr) flow_dv_modify_hdr_resource_release(dev_flow); + if (dev_flow->dv.jump) + flow_dv_jump_tbl_resource_release(dev_flow); rte_free(dev_flow); } } diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c index b0b144c..f5a6c2e 100644 --- a/drivers/net/mlx5/mlx5_glue.c +++ b/drivers/net/mlx5/mlx5_glue.c @@ -370,6 +370,17 @@ } static void * +mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_create_action_dest_flow_table(tbl); +#else + (void)tbl; + return NULL; +#endif +} + +static void * mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level) { #ifdef HAVE_MLX5DV_DR @@ -833,6 +844,8 @@ .get_async_event = mlx5_glue_get_async_event, .port_state_str = mlx5_glue_port_state_str, .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq, + .dr_create_flow_action_dest_flow_tbl = + mlx5_glue_dr_create_flow_action_dest_flow_tbl, .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl, .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl, .dr_create_ns = mlx5_glue_dr_create_ns, diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index eb29ffa..058e9b1 100644 --- a/drivers/net/mlx5/mlx5_glue.h +++ b/drivers/net/mlx5/mlx5_glue.h @@ -145,6 +145,7 @@ struct mlx5_glue { struct ibv_async_event *event); const char *(*port_state_str)(enum ibv_port_state port_state); struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq); + void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl); void *(*dr_create_flow_tbl)(void *ns, uint32_t level); int (*dr_destroy_flow_tbl)(void *tbl); void *(*dr_create_ns)(struct ibv_context *ctx,