From patchwork Fri Jun 8 13:57:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 40848 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D81B95F6E; Fri, 8 Jun 2018 15:58:19 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 32E735F3B for ; Fri, 8 Jun 2018 15:58:18 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from xuemingl@mellanox.com) with ESMTPS (AES256-SHA encrypted); 8 Jun 2018 17:00:35 +0300 Received: from dev-r630-06.mtbc.labs.mlnx (dev-r630-06.mtbc.labs.mlnx [10.12.205.180]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w58DwGiD000396; Fri, 8 Jun 2018 16:58:16 +0300 Received: from dev-r630-06.mtbc.labs.mlnx (localhost [127.0.0.1]) by dev-r630-06.mtbc.labs.mlnx (8.14.7/8.14.7) with ESMTP id w58DwFHj023620; Fri, 8 Jun 2018 21:58:15 +0800 Received: (from xuemingl@localhost) by dev-r630-06.mtbc.labs.mlnx (8.14.7/8.14.7/Submit) id w58DwCKQ023618; Fri, 8 Jun 2018 21:58:12 +0800 From: Xueming Li To: Shahaf Shuler , Nelio Laranjeiro Cc: Xueming Li , dev@dpdk.org Date: Fri, 8 Jun 2018 21:57:51 +0800 Message-Id: <20180608135751.23516-1-xuemingl@mellanox.com> X-Mailer: git-send-email 2.13.3 Subject: [dpdk-dev] [RFC v1] net/mlx5: support multiple flow priorities X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch supports multiple rte_flow priorities on NICs that support 16 or more Verbs priorities. Control flow always consume 3 Verbs priorities even in isolated mode. For NICs that support 16 or more Verbs priorities, each rte_flow priority consumes 6 Verbs priorities, higher 3 for tunnels, lower 3 for non-tunnels. NICs that have 8 Verbs priorities only support 1 user prioirty: User tunnel Verbs flows: 0 - 2 User non-tunnel Verbs flows: 2 - 4 control Verbs flows: 5 - 8 Signed-off-by: Xueming Li --- drivers/net/mlx5/mlx5.c | 8 +++++- drivers/net/mlx5/mlx5.h | 10 +++++-- drivers/net/mlx5/mlx5_flow.c | 67 ++++++++++++++++++++++++++------------------ 3 files changed, 54 insertions(+), 31 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index c933e274f..6897cc546 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1189,12 +1189,18 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, /* Supported Verbs flow priority number detection. */ if (verb_priorities == 0) verb_priorities = mlx5_get_max_verbs_prio(eth_dev); - if (verb_priorities < MLX5_VERBS_FLOW_PRIO_8) { + if (verb_priorities < MLX5_VERBS_FLOW_MIN_PRIOS) { DRV_LOG(ERR, "port %u wrong Verbs flow priorities: %u", eth_dev->data->port_id, verb_priorities); goto port_error; } priv->config.max_verbs_prio = verb_priorities; + if (verb_priorities == MLX5_VERBS_FLOW_MIN_PRIOS) + priv->config.ctrl_flow_prio = 1; + else + priv->config.ctrl_flow_prio = + (verb_priorities - MLX5_VERBS_FLOW_CTRL_PRIOS) / + MLX5_VERBS_FLOW_USER_PRIOS; /* * Once the device is added to the list of memory event * callback, its global MR cache table cannot be expanded diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 997b04a33..514501cc3 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -115,6 +115,7 @@ struct mlx5_dev_config { /* Rx queue count threshold to enable MPRQ. */ } mprq; /* Configurations for Multi-Packet RQ. */ unsigned int max_verbs_prio; /* Number of Verb flow priorities. */ + unsigned int ctrl_flow_prio; /* Control flow priority. */ unsigned int tso_max_payload_sz; /* Maximum TCP payload for TSO. */ unsigned int ind_table_max_size; /* Maximum indirection table size. */ int txq_inline; /* Maximum packet size for inlining. */ @@ -131,8 +132,13 @@ enum mlx5_verbs_alloc_type { MLX5_VERBS_ALLOC_TYPE_RX_QUEUE, }; -/* 8 Verbs priorities. */ -#define MLX5_VERBS_FLOW_PRIO_8 8 +/* Minimal hardware Verbs priorities. */ +#define MLX5_VERBS_FLOW_MIN_PRIOS 8 +/* Verbs priorities used by control flow. */ +#define MLX5_VERBS_FLOW_CTRL_PRIOS 3 +/* Verbs priorities used by each user flow. */ +#define MLX5_VERBS_FLOW_USER_PRIOS 6 + /** * Verbs allocator needs a context to know in the callback which kind of diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 994be05be..0e4b6f179 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -31,9 +31,6 @@ #include "mlx5_prm.h" #include "mlx5_glue.h" -/* Flow priority for control plane flows. */ -#define MLX5_CTRL_FLOW_PRIORITY 1 - /* Internet Protocol versions. */ #define MLX5_IPV4 4 #define MLX5_IPV6 6 @@ -619,6 +616,8 @@ mlx5_flow_item_validate(const struct rte_flow_item *item, /** * Extract attribute to the parser. * + * @param dev + * Pointer to Ethernet device. * @param[in] attr * Flow rule attributes. * @param[out] error @@ -628,9 +627,12 @@ mlx5_flow_item_validate(const struct rte_flow_item *item, * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -mlx5_flow_convert_attributes(const struct rte_flow_attr *attr, +mlx5_flow_convert_attributes(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, struct rte_flow_error *error) { + struct priv *priv = dev->data->dev_private; + if (attr->group) { rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_GROUP, @@ -638,11 +640,11 @@ mlx5_flow_convert_attributes(const struct rte_flow_attr *attr, "groups are not supported"); return -rte_errno; } - if (attr->priority && attr->priority != MLX5_CTRL_FLOW_PRIORITY) { + if (attr->priority > priv->config.ctrl_flow_prio) { rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, NULL, - "priorities are not supported"); + "priority not supported"); return -rte_errno; } if (attr->egress) { @@ -1039,23 +1041,33 @@ mlx5_flow_update_priority(struct rte_eth_dev *dev, unsigned int i; uint16_t priority; - /* 8 priorities >= 16 priorities - * Control flow: 4-7 8-15 - * User normal flow: 1-3 4-7 - * User tunnel flow: 0-2 0-3 - */ - priority = attr->priority * MLX5_VERBS_FLOW_PRIO_8; - if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_PRIO_8) - priority /= 2; - /* - * Lower non-tunnel flow Verbs priority 1 if only support 8 Verbs - * priorities, lower 4 otherwise. - */ - if (!parser->inner) { - if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_PRIO_8) - priority += 1; + if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_MIN_PRIOS) { + /* + * Verbs priority mapping: + * user tunnel flows: 0 - 2 + * user non-tunnel flows: 2 - 4 + * control flows: 5 - 8 + */ + if (attr->priority >= priv->config.ctrl_flow_prio) + priority = MLX5_VERBS_FLOW_MIN_PRIOS - + MLX5_VERBS_FLOW_CTRL_PRIOS; + else if (!parser->inner) + priority = (MLX5_VERBS_FLOW_MIN_PRIOS - + MLX5_VERBS_FLOW_CTRL_PRIOS - + MLX5_VERBS_FLOW_USER_PRIOS / 2); else - priority += MLX5_VERBS_FLOW_PRIO_8 / 2; + priority = 0; + } else { + /* + * Verbs priority mapping: + * user flows * 6, control flows 3. + * User flow mapping: + * tunnel 3, non-tunnel 3. + */ + priority = attr->priority * MLX5_VERBS_FLOW_USER_PRIOS; + if (attr->priority != priv->config.ctrl_flow_prio && + !parser->inner) + priority += MLX5_VERBS_FLOW_USER_PRIOS / 2; } if (parser->drop) { parser->queue[HASH_RXQ_ETH].ibv_attr->priority = priority + @@ -1285,7 +1297,7 @@ mlx5_flow_convert(struct rte_eth_dev *dev, .layer = HASH_RXQ_ETH, .mark_id = MLX5_FLOW_MARK_DEFAULT, }; - ret = mlx5_flow_convert_attributes(attr, error); + ret = mlx5_flow_convert_attributes(dev, attr, error); if (ret) return ret; ret = mlx5_flow_convert_actions(dev, actions, error, parser); @@ -3019,7 +3031,7 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_attr attr = { .ingress = 1, - .priority = MLX5_CTRL_FLOW_PRIORITY, + .priority = priv->config.ctrl_flow_prio, }; struct rte_flow_item items[] = { { @@ -3739,7 +3751,7 @@ unsigned int mlx5_get_max_verbs_prio(struct rte_eth_dev *dev) { struct priv *priv = dev->data->dev_private; - unsigned int verb_priorities = MLX5_VERBS_FLOW_PRIO_8; + unsigned int verb_priorities = MLX5_VERBS_FLOW_MIN_PRIOS; struct { struct ibv_flow_attr attr; struct ibv_flow_spec_eth eth; @@ -3773,8 +3785,7 @@ mlx5_get_max_verbs_prio(struct rte_eth_dev *dev) break; } } while (1); - DRV_LOG(DEBUG, "port %u Verbs flow priorities: %d," - " user flow priorities: %d", - dev->data->port_id, verb_priorities, MLX5_CTRL_FLOW_PRIORITY); + DRV_LOG(DEBUG, "port %u Verbs flow priorities: %d", + dev->data->port_id, verb_priorities); return verb_priorities; }