From patchwork Wed Jul 11 07:22:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42795 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7FE441B4B5; Wed, 11 Jul 2018 09:23:22 +0200 (CEST) Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by dpdk.org (Postfix) with ESMTP id 8BEAA1B466 for ; Wed, 11 Jul 2018 09:23:19 +0200 (CEST) Received: by mail-wr1-f42.google.com with SMTP id b15-v6so17029680wrv.10 for ; Wed, 11 Jul 2018 00:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ax8P3r+VBpEqhnaCODZO+9MA5wBs5COlrbAOevoOZu4=; b=Z5K55JgGOfohighcwiwOpV0ZAz2ghsIo9Zrlk7SoCPKlrndIf6loLHDB3vrLt93SaQ GwMHFEWPjkZUUYbKYLZNgsmPdhVkh1pSgEGn/8Sh5nCNBQhPnlf3+xMJHHFJyN9030fl /elGgvBD7mUF6kV82CGN9g1Hl65QLBFx8ULlPNHohkqVNBmAvi3NFqFU8Q4Z02DFsWEC yJzYmVFpBZ/Ys3J4JS9yrfwCR9VcWgz7ip6DYwXm19aKX8+KAda1fqv/XfKR5RZMfQ/Q fW1RIykhZAzLbrJnHuAWYSXauC5Z3p+lVsAKDWA1WJrNDTclkl7HkOpUUmczLw7wOAuO WElA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ax8P3r+VBpEqhnaCODZO+9MA5wBs5COlrbAOevoOZu4=; b=VCS2M86wS/73Emb4j8bSg20c07IkmF5kTlCpSlq/4LW2w3kyeSa4KoR6+0g3+PHRiw OpzlcNlXggn3EX5UIfYuEgBpbrgTN+Ax3NE+1Vmx+R5bladI5HU97RaEzvdZyIiwjfZQ GMlLObIIzzJMRV5+GKo3P2jkkfXg1mDh2m6CGo/b+noFpsdIWNx9m4bf5nvEHMftACyy FzWmq47qeevQNEAezCW2HL9ci0zzWVRynZ0p16T7TzpkBmGZIpoaT39O65+jgBlf3sf7 2xe5BR0yNLtgdo8TqTyhBY08XwS77KXQAu8bkbFkbflv8LMa3V5ji9pU5oZ/xOAN7M1s twBQ== X-Gm-Message-State: AOUpUlGOtffbkBGhtcgTc6RrPVaJGWgGbC9T2/RcFlavZbNV9/HjlQVM Rddl0zqkZ8Ef3hJdloLpcSodLX9ZAg== X-Google-Smtp-Source: AAOMgpc23QNbV2Ecv5VqmqjEowyr033waoN4m5gSyF02CuPrmYszWkGZCsJge5Lef9xeuao9eiDAlg== X-Received: by 2002:a5d:4b50:: with SMTP id w16-v6mr1183751wrs.87.1531293797884; Wed, 11 Jul 2018 00:23:17 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:17 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:34 +0200 Message-Id: <40c457fa16871ead7eac9f6d594f8ecf7fbbc6b6.1531293415.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 01/21] net/mlx5: remove flow support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This start a series to re-work the flow engine in mlx5 to easily support flow conversion to Verbs or TC. This is necessary to handle both regular flows and representors flows. As the full file needs to be clean-up to re-write all items/actions processing, this patch starts to disable the regular code and only let the PMD to start in isolated mode. After this patch flow API will not be usable. Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 3095 +--------------------------------- drivers/net/mlx5/mlx5_rxtx.h | 1 - 2 files changed, 80 insertions(+), 3016 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 45207d70e..a45cb06e1 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -31,2406 +31,49 @@ #include "mlx5_prm.h" #include "mlx5_glue.h" -/* Flow priority for control plane flows. */ -#define MLX5_CTRL_FLOW_PRIORITY 1 - -/* Internet Protocol versions. */ -#define MLX5_IPV4 4 -#define MLX5_IPV6 6 -#define MLX5_GRE 47 - -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT -struct ibv_flow_spec_counter_action { - int dummy; -}; -#endif - -/* Dev ops structure defined in mlx5.c */ -extern const struct eth_dev_ops mlx5_dev_ops; -extern const struct eth_dev_ops mlx5_dev_ops_isolate; - -/** Structure give to the conversion functions. */ -struct mlx5_flow_data { - struct rte_eth_dev *dev; /** Ethernet device. */ - struct mlx5_flow_parse *parser; /** Parser context. */ - struct rte_flow_error *error; /** Error context. */ -}; - -static int -mlx5_flow_create_eth(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_vlan(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_ipv4(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_ipv6(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_udp(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_tcp(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_vxlan(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_vxlan_gpe(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_gre(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_mpls(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -struct mlx5_flow_parse; - -static void -mlx5_flow_create_copy(struct mlx5_flow_parse *parser, void *src, - unsigned int size); - -static int -mlx5_flow_create_flag_mark(struct mlx5_flow_parse *parser, uint32_t mark_id); - -static int -mlx5_flow_create_count(struct rte_eth_dev *dev, struct mlx5_flow_parse *parser); - -/* Hash RX queue types. */ -enum hash_rxq_type { - HASH_RXQ_TCPV4, - HASH_RXQ_UDPV4, - HASH_RXQ_IPV4, - HASH_RXQ_TCPV6, - HASH_RXQ_UDPV6, - HASH_RXQ_IPV6, - HASH_RXQ_ETH, - HASH_RXQ_TUNNEL, -}; - -/* Initialization data for hash RX queue. */ -struct hash_rxq_init { - uint64_t hash_fields; /* Fields that participate in the hash. */ - uint64_t dpdk_rss_hf; /* Matching DPDK RSS hash fields. */ - unsigned int flow_priority; /* Flow priority to use. */ - unsigned int ip_version; /* Internet protocol. */ -}; - -/* Initialization data for hash RX queues. */ -const struct hash_rxq_init hash_rxq_init[] = { - [HASH_RXQ_TCPV4] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV4 | - IBV_RX_HASH_DST_IPV4 | - IBV_RX_HASH_SRC_PORT_TCP | - IBV_RX_HASH_DST_PORT_TCP), - .dpdk_rss_hf = ETH_RSS_NONFRAG_IPV4_TCP, - .flow_priority = 0, - .ip_version = MLX5_IPV4, - }, - [HASH_RXQ_UDPV4] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV4 | - IBV_RX_HASH_DST_IPV4 | - IBV_RX_HASH_SRC_PORT_UDP | - IBV_RX_HASH_DST_PORT_UDP), - .dpdk_rss_hf = ETH_RSS_NONFRAG_IPV4_UDP, - .flow_priority = 0, - .ip_version = MLX5_IPV4, - }, - [HASH_RXQ_IPV4] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV4 | - IBV_RX_HASH_DST_IPV4), - .dpdk_rss_hf = (ETH_RSS_IPV4 | - ETH_RSS_FRAG_IPV4), - .flow_priority = 1, - .ip_version = MLX5_IPV4, - }, - [HASH_RXQ_TCPV6] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV6 | - IBV_RX_HASH_DST_IPV6 | - IBV_RX_HASH_SRC_PORT_TCP | - IBV_RX_HASH_DST_PORT_TCP), - .dpdk_rss_hf = ETH_RSS_NONFRAG_IPV6_TCP, - .flow_priority = 0, - .ip_version = MLX5_IPV6, - }, - [HASH_RXQ_UDPV6] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV6 | - IBV_RX_HASH_DST_IPV6 | - IBV_RX_HASH_SRC_PORT_UDP | - IBV_RX_HASH_DST_PORT_UDP), - .dpdk_rss_hf = ETH_RSS_NONFRAG_IPV6_UDP, - .flow_priority = 0, - .ip_version = MLX5_IPV6, - }, - [HASH_RXQ_IPV6] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV6 | - IBV_RX_HASH_DST_IPV6), - .dpdk_rss_hf = (ETH_RSS_IPV6 | - ETH_RSS_FRAG_IPV6), - .flow_priority = 1, - .ip_version = MLX5_IPV6, - }, - [HASH_RXQ_ETH] = { - .hash_fields = 0, - .dpdk_rss_hf = 0, - .flow_priority = 2, - }, -}; - -/* Number of entries in hash_rxq_init[]. */ -const unsigned int hash_rxq_init_n = RTE_DIM(hash_rxq_init); - -/** Structure for holding counter stats. */ -struct mlx5_flow_counter_stats { - uint64_t hits; /**< Number of packets matched by the rule. */ - uint64_t bytes; /**< Number of bytes matched by the rule. */ -}; - -/** Structure for Drop queue. */ -struct mlx5_hrxq_drop { - struct ibv_rwq_ind_table *ind_table; /**< Indirection table. */ - struct ibv_qp *qp; /**< Verbs queue pair. */ - struct ibv_wq *wq; /**< Verbs work queue. */ - struct ibv_cq *cq; /**< Verbs completion queue. */ -}; - -/* Flows structures. */ -struct mlx5_flow { - uint64_t hash_fields; /**< Fields that participate in the hash. */ - struct ibv_flow_attr *ibv_attr; /**< Pointer to Verbs attributes. */ - struct ibv_flow *ibv_flow; /**< Verbs flow. */ - struct mlx5_hrxq *hrxq; /**< Hash Rx queues. */ -}; - -/* Drop flows structures. */ -struct mlx5_flow_drop { - struct ibv_flow_attr *ibv_attr; /**< Pointer to Verbs attributes. */ - struct ibv_flow *ibv_flow; /**< Verbs flow. */ -}; - -struct rte_flow { - TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ - uint32_t mark:1; /**< Set if the flow is marked. */ - uint32_t drop:1; /**< Drop queue. */ - struct rte_flow_action_rss rss_conf; /**< RSS configuration */ - uint16_t (*queues)[]; /**< Queues indexes to use. */ - uint8_t rss_key[40]; /**< copy of the RSS key. */ - uint32_t tunnel; /**< Tunnel type of RTE_PTYPE_TUNNEL_XXX. */ - struct ibv_counter_set *cs; /**< Holds the counters for the rule. */ - struct mlx5_flow_counter_stats counter_stats;/**> 12) - -const uint32_t ptype_ext[] = { - [PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN)] = RTE_PTYPE_TUNNEL_VXLAN | - RTE_PTYPE_L4_UDP, - [PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN_GPE)] = RTE_PTYPE_TUNNEL_VXLAN_GPE | - RTE_PTYPE_L4_UDP, - [PTYPE_IDX(RTE_PTYPE_TUNNEL_GRE)] = RTE_PTYPE_TUNNEL_GRE, - [PTYPE_IDX(RTE_PTYPE_TUNNEL_MPLS_IN_GRE)] = - RTE_PTYPE_TUNNEL_MPLS_IN_GRE, - [PTYPE_IDX(RTE_PTYPE_TUNNEL_MPLS_IN_UDP)] = - RTE_PTYPE_TUNNEL_MPLS_IN_GRE | RTE_PTYPE_L4_UDP, -}; - -/** Structure to generate a simple graph of layers supported by the NIC. */ -struct mlx5_flow_items { - /** List of possible actions for these items. */ - const enum rte_flow_action_type *const actions; - /** Bit-masks corresponding to the possibilities for the item. */ - const void *mask; - /** - * Default bit-masks to use when item->mask is not provided. When - * \default_mask is also NULL, the full supported bit-mask (\mask) is - * used instead. - */ - const void *default_mask; - /** Bit-masks size in bytes. */ - const unsigned int mask_sz; - /** - * Conversion function from rte_flow to NIC specific flow. - * - * @param item - * rte_flow item to convert. - * @param default_mask - * Default bit-masks to use when item->mask is not provided. - * @param data - * Internal structure to store the conversion. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is - * set. - */ - int (*convert)(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - /** Size in bytes of the destination structure. */ - const unsigned int dst_sz; - /** List of possible following items. */ - const enum rte_flow_item_type *const items; -}; - -/** Valid action for this PMD. */ -static const enum rte_flow_action_type valid_actions[] = { - RTE_FLOW_ACTION_TYPE_DROP, - RTE_FLOW_ACTION_TYPE_QUEUE, - RTE_FLOW_ACTION_TYPE_MARK, - RTE_FLOW_ACTION_TYPE_FLAG, -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT - RTE_FLOW_ACTION_TYPE_COUNT, -#endif - RTE_FLOW_ACTION_TYPE_END, -}; - -/** Graph of supported items and associated actions. */ -static const struct mlx5_flow_items mlx5_flow_items[] = { - [RTE_FLOW_ITEM_TYPE_END] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_VXLAN, - RTE_FLOW_ITEM_TYPE_VXLAN_GPE, - RTE_FLOW_ITEM_TYPE_GRE), - }, - [RTE_FLOW_ITEM_TYPE_ETH] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_VLAN, - RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_eth){ - .dst.addr_bytes = "\xff\xff\xff\xff\xff\xff", - .src.addr_bytes = "\xff\xff\xff\xff\xff\xff", - .type = -1, - }, - .default_mask = &rte_flow_item_eth_mask, - .mask_sz = sizeof(struct rte_flow_item_eth), - .convert = mlx5_flow_create_eth, - .dst_sz = sizeof(struct ibv_flow_spec_eth), - }, - [RTE_FLOW_ITEM_TYPE_VLAN] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_vlan){ - .tci = -1, - .inner_type = -1, - }, - .default_mask = &rte_flow_item_vlan_mask, - .mask_sz = sizeof(struct rte_flow_item_vlan), - .convert = mlx5_flow_create_vlan, - .dst_sz = 0, - }, - [RTE_FLOW_ITEM_TYPE_IPV4] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_UDP, - RTE_FLOW_ITEM_TYPE_TCP, - RTE_FLOW_ITEM_TYPE_GRE), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_ipv4){ - .hdr = { - .src_addr = -1, - .dst_addr = -1, - .type_of_service = -1, - .next_proto_id = -1, - }, - }, - .default_mask = &rte_flow_item_ipv4_mask, - .mask_sz = sizeof(struct rte_flow_item_ipv4), - .convert = mlx5_flow_create_ipv4, - .dst_sz = sizeof(struct ibv_flow_spec_ipv4_ext), - }, - [RTE_FLOW_ITEM_TYPE_IPV6] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_UDP, - RTE_FLOW_ITEM_TYPE_TCP, - RTE_FLOW_ITEM_TYPE_GRE), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_ipv6){ - .hdr = { - .src_addr = { - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - }, - .dst_addr = { - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - }, - .vtc_flow = -1, - .proto = -1, - .hop_limits = -1, - }, - }, - .default_mask = &rte_flow_item_ipv6_mask, - .mask_sz = sizeof(struct rte_flow_item_ipv6), - .convert = mlx5_flow_create_ipv6, - .dst_sz = sizeof(struct ibv_flow_spec_ipv6), - }, - [RTE_FLOW_ITEM_TYPE_UDP] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_VXLAN, - RTE_FLOW_ITEM_TYPE_VXLAN_GPE, - RTE_FLOW_ITEM_TYPE_MPLS), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_udp){ - .hdr = { - .src_port = -1, - .dst_port = -1, - }, - }, - .default_mask = &rte_flow_item_udp_mask, - .mask_sz = sizeof(struct rte_flow_item_udp), - .convert = mlx5_flow_create_udp, - .dst_sz = sizeof(struct ibv_flow_spec_tcp_udp), - }, - [RTE_FLOW_ITEM_TYPE_TCP] = { - .actions = valid_actions, - .mask = &(const struct rte_flow_item_tcp){ - .hdr = { - .src_port = -1, - .dst_port = -1, - }, - }, - .default_mask = &rte_flow_item_tcp_mask, - .mask_sz = sizeof(struct rte_flow_item_tcp), - .convert = mlx5_flow_create_tcp, - .dst_sz = sizeof(struct ibv_flow_spec_tcp_udp), - }, - [RTE_FLOW_ITEM_TYPE_GRE] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6, - RTE_FLOW_ITEM_TYPE_MPLS), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_gre){ - .protocol = -1, - }, - .default_mask = &rte_flow_item_gre_mask, - .mask_sz = sizeof(struct rte_flow_item_gre), - .convert = mlx5_flow_create_gre, -#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT - .dst_sz = sizeof(struct ibv_flow_spec_gre), -#else - .dst_sz = sizeof(struct ibv_flow_spec_tunnel), -#endif - }, - [RTE_FLOW_ITEM_TYPE_MPLS] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_mpls){ - .label_tc_s = "\xff\xff\xf0", - }, - .default_mask = &rte_flow_item_mpls_mask, - .mask_sz = sizeof(struct rte_flow_item_mpls), - .convert = mlx5_flow_create_mpls, -#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT - .dst_sz = sizeof(struct ibv_flow_spec_mpls), -#endif - }, - [RTE_FLOW_ITEM_TYPE_VXLAN] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_IPV4, /* For L3 VXLAN. */ - RTE_FLOW_ITEM_TYPE_IPV6), /* For L3 VXLAN. */ - .actions = valid_actions, - .mask = &(const struct rte_flow_item_vxlan){ - .vni = "\xff\xff\xff", - }, - .default_mask = &rte_flow_item_vxlan_mask, - .mask_sz = sizeof(struct rte_flow_item_vxlan), - .convert = mlx5_flow_create_vxlan, - .dst_sz = sizeof(struct ibv_flow_spec_tunnel), - }, - [RTE_FLOW_ITEM_TYPE_VXLAN_GPE] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_vxlan_gpe){ - .vni = "\xff\xff\xff", - }, - .default_mask = &rte_flow_item_vxlan_gpe_mask, - .mask_sz = sizeof(struct rte_flow_item_vxlan_gpe), - .convert = mlx5_flow_create_vxlan_gpe, - .dst_sz = sizeof(struct ibv_flow_spec_tunnel), - }, -}; - -/** Structure to pass to the conversion function. */ -struct mlx5_flow_parse { - uint32_t inner; /**< Verbs value, set once tunnel is encountered. */ - uint32_t create:1; - /**< Whether resources should remain after a validate. */ - uint32_t drop:1; /**< Target is a drop queue. */ - uint32_t mark:1; /**< Mark is present in the flow. */ - uint32_t count:1; /**< Count is present in the flow. */ - uint32_t mark_id; /**< Mark identifier. */ - struct rte_flow_action_rss rss_conf; /**< RSS configuration */ - uint16_t queues[RTE_MAX_QUEUES_PER_PORT]; /**< Queues indexes to use. */ - uint8_t rss_key[40]; /**< copy of the RSS key. */ - enum hash_rxq_type layer; /**< Last pattern layer detected. */ - enum hash_rxq_type out_layer; /**< Last outer pattern layer detected. */ - uint32_t tunnel; /**< Tunnel type of RTE_PTYPE_TUNNEL_XXX. */ - struct ibv_counter_set *cs; /**< Holds the counter set for the rule */ - struct { - struct ibv_flow_attr *ibv_attr; - /**< Pointer to Verbs attributes. */ - unsigned int offset; - /**< Current position or total size of the attribute. */ - uint64_t hash_fields; /**< Verbs hash fields. */ - } queue[RTE_DIM(hash_rxq_init)]; -}; - -static const struct rte_flow_ops mlx5_flow_ops = { - .validate = mlx5_flow_validate, - .create = mlx5_flow_create, - .destroy = mlx5_flow_destroy, - .flush = mlx5_flow_flush, -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT - .query = mlx5_flow_query, -#else - .query = NULL, -#endif - .isolate = mlx5_flow_isolate, -}; - -/* Convert FDIR request to Generic flow. */ -struct mlx5_fdir { - struct rte_flow_attr attr; - struct rte_flow_action actions[2]; - struct rte_flow_item items[4]; - struct rte_flow_item_eth l2; - struct rte_flow_item_eth l2_mask; - union { - struct rte_flow_item_ipv4 ipv4; - struct rte_flow_item_ipv6 ipv6; - } l3; - union { - struct rte_flow_item_ipv4 ipv4; - struct rte_flow_item_ipv6 ipv6; - } l3_mask; - union { - struct rte_flow_item_udp udp; - struct rte_flow_item_tcp tcp; - } l4; - union { - struct rte_flow_item_udp udp; - struct rte_flow_item_tcp tcp; - } l4_mask; - struct rte_flow_action_queue queue; -}; - -/* Verbs specification header. */ -struct ibv_spec_header { - enum ibv_flow_spec_type type; - uint16_t size; -}; - -/** - * Check item is fully supported by the NIC matching capability. - * - * @param item[in] - * Item specification. - * @param mask[in] - * Bit-masks covering supported fields to compare with spec, last and mask in - * \item. - * @param size - * Bit-Mask size in bytes. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_item_validate(const struct rte_flow_item *item, - const uint8_t *mask, unsigned int size) -{ - unsigned int i; - const uint8_t *spec = item->spec; - const uint8_t *last = item->last; - const uint8_t *m = item->mask ? item->mask : mask; - - if (!spec && (item->mask || last)) - goto error; - if (!spec) - return 0; - /* - * Single-pass check to make sure that: - * - item->mask is supported, no bits are set outside mask. - * - Both masked item->spec and item->last are equal (no range - * supported). - */ - for (i = 0; i < size; i++) { - if (!m[i]) - continue; - if ((m[i] | mask[i]) != mask[i]) - goto error; - if (last && ((spec[i] & m[i]) != (last[i] & m[i]))) - goto error; - } - return 0; -error: - rte_errno = ENOTSUP; - return -rte_errno; -} - -/** - * Extract attribute to the parser. - * - * @param[in] attr - * Flow rule attributes. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert_attributes(const struct rte_flow_attr *attr, - struct rte_flow_error *error) -{ - if (attr->group) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_GROUP, - NULL, - "groups are not supported"); - return -rte_errno; - } - if (attr->priority && attr->priority != MLX5_CTRL_FLOW_PRIORITY) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, - NULL, - "priorities are not supported"); - return -rte_errno; - } - if (attr->egress) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, - NULL, - "egress is not supported"); - return -rte_errno; - } - if (attr->transfer) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER, - NULL, - "transfer is not supported"); - return -rte_errno; - } - if (!attr->ingress) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_INGRESS, - NULL, - "only ingress is supported"); - return -rte_errno; - } - return 0; -} - -/** - * Extract actions request to the parser. - * - * @param dev - * Pointer to Ethernet device. - * @param[in] actions - * Associated actions (list terminated by the END action). - * @param[out] error - * Perform verbose error reporting if not NULL. - * @param[in, out] parser - * Internal parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert_actions(struct rte_eth_dev *dev, - const struct rte_flow_action actions[], - struct rte_flow_error *error, - struct mlx5_flow_parse *parser) -{ - enum { FATE = 1, MARK = 2, COUNT = 4, }; - uint32_t overlap = 0; - struct priv *priv = dev->data->dev_private; - - for (; actions->type != RTE_FLOW_ACTION_TYPE_END; ++actions) { - if (actions->type == RTE_FLOW_ACTION_TYPE_VOID) { - continue; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_DROP) { - if (overlap & FATE) - goto exit_action_overlap; - overlap |= FATE; - parser->drop = 1; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_QUEUE) { - const struct rte_flow_action_queue *queue = - (const struct rte_flow_action_queue *) - actions->conf; - - if (overlap & FATE) - goto exit_action_overlap; - overlap |= FATE; - if (!queue || (queue->index > (priv->rxqs_n - 1))) - goto exit_action_not_supported; - parser->queues[0] = queue->index; - parser->rss_conf = (struct rte_flow_action_rss){ - .queue_num = 1, - .queue = parser->queues, - }; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_RSS) { - const struct rte_flow_action_rss *rss = - (const struct rte_flow_action_rss *) - actions->conf; - const uint8_t *rss_key; - uint32_t rss_key_len; - uint16_t n; - - if (overlap & FATE) - goto exit_action_overlap; - overlap |= FATE; - if (rss->func && - rss->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "the only supported RSS hash" - " function is Toeplitz"); - return -rte_errno; - } -#ifndef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - if (parser->rss_conf.level > 1) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "a nonzero RSS encapsulation" - " level is not supported"); - return -rte_errno; - } -#endif - if (parser->rss_conf.level > 2) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "RSS encapsulation level" - " > 1 is not supported"); - return -rte_errno; - } - if (rss->types & MLX5_RSS_HF_MASK) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "unsupported RSS type" - " requested"); - return -rte_errno; - } - if (rss->key_len) { - rss_key_len = rss->key_len; - rss_key = rss->key; - } else { - rss_key_len = rss_hash_default_key_len; - rss_key = rss_hash_default_key; - } - if (rss_key_len != RTE_DIM(parser->rss_key)) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "RSS hash key must be" - " exactly 40 bytes long"); - return -rte_errno; - } - if (!rss->queue_num) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "no valid queues"); - return -rte_errno; - } - if (rss->queue_num > RTE_DIM(parser->queues)) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "too many queues for RSS" - " context"); - return -rte_errno; - } - for (n = 0; n < rss->queue_num; ++n) { - if (rss->queue[n] >= priv->rxqs_n) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "queue id > number of" - " queues"); - return -rte_errno; - } - } - parser->rss_conf = (struct rte_flow_action_rss){ - .func = RTE_ETH_HASH_FUNCTION_DEFAULT, - .level = rss->level ? rss->level : 1, - .types = rss->types, - .key_len = rss_key_len, - .queue_num = rss->queue_num, - .key = memcpy(parser->rss_key, rss_key, - sizeof(*rss_key) * rss_key_len), - .queue = memcpy(parser->queues, rss->queue, - sizeof(*rss->queue) * - rss->queue_num), - }; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_MARK) { - const struct rte_flow_action_mark *mark = - (const struct rte_flow_action_mark *) - actions->conf; - - if (overlap & MARK) - goto exit_action_overlap; - overlap |= MARK; - if (!mark) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "mark must be defined"); - return -rte_errno; - } else if (mark->id >= MLX5_FLOW_MARK_MAX) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "mark must be between 0" - " and 16777199"); - return -rte_errno; - } - parser->mark = 1; - parser->mark_id = mark->id; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_FLAG) { - if (overlap & MARK) - goto exit_action_overlap; - overlap |= MARK; - parser->mark = 1; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_COUNT && - priv->config.flow_counter_en) { - if (overlap & COUNT) - goto exit_action_overlap; - overlap |= COUNT; - parser->count = 1; - } else { - goto exit_action_not_supported; - } - } - /* When fate is unknown, drop traffic. */ - if (!(overlap & FATE)) - parser->drop = 1; - if (parser->drop && parser->mark) - parser->mark = 0; - if (!parser->rss_conf.queue_num && !parser->drop) { - rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, "no valid action"); - return -rte_errno; - } - return 0; -exit_action_not_supported: - rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, - actions, "action not supported"); - return -rte_errno; -exit_action_overlap: - rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, - actions, "overlapping actions are not supported"); - return -rte_errno; -} - -/** - * Validate items. - * - * @param[in] items - * Pattern specification (list terminated by the END pattern item). - * @param[out] error - * Perform verbose error reporting if not NULL. - * @param[in, out] parser - * Internal parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert_items_validate(struct rte_eth_dev *dev, - const struct rte_flow_item items[], - struct rte_flow_error *error, - struct mlx5_flow_parse *parser) -{ - struct priv *priv = dev->data->dev_private; - const struct mlx5_flow_items *cur_item = mlx5_flow_items; - unsigned int i; - unsigned int last_voids = 0; - int ret = 0; - - /* Initialise the offsets to start after verbs attribute. */ - for (i = 0; i != hash_rxq_init_n; ++i) - parser->queue[i].offset = sizeof(struct ibv_flow_attr); - for (; items->type != RTE_FLOW_ITEM_TYPE_END; ++items) { - const struct mlx5_flow_items *token = NULL; - unsigned int n; - - if (items->type == RTE_FLOW_ITEM_TYPE_VOID) { - last_voids++; - continue; - } - for (i = 0; - cur_item->items && - cur_item->items[i] != RTE_FLOW_ITEM_TYPE_END; - ++i) { - if (cur_item->items[i] == items->type) { - token = &mlx5_flow_items[items->type]; - break; - } - } - if (!token) { - ret = -ENOTSUP; - goto exit_item_not_supported; - } - cur_item = token; - ret = mlx5_flow_item_validate(items, - (const uint8_t *)cur_item->mask, - cur_item->mask_sz); - if (ret) - goto exit_item_not_supported; - if (IS_TUNNEL(items->type)) { - if (parser->tunnel && - !((items - last_voids - 1)->type == - RTE_FLOW_ITEM_TYPE_GRE && items->type == - RTE_FLOW_ITEM_TYPE_MPLS)) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - items, - "Cannot recognize multiple" - " tunnel encapsulations."); - return -rte_errno; - } - if (items->type == RTE_FLOW_ITEM_TYPE_MPLS && - !priv->config.mpls_en) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - items, - "MPLS not supported or" - " disabled in firmware" - " configuration."); - return -rte_errno; - } - if (!priv->config.tunnel_en && - parser->rss_conf.level > 1) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - items, - "RSS on tunnel is not supported"); - return -rte_errno; - } - parser->inner = IBV_FLOW_SPEC_INNER; - parser->tunnel = flow_ptype[items->type]; - } - if (parser->drop) { - parser->queue[HASH_RXQ_ETH].offset += cur_item->dst_sz; - } else { - for (n = 0; n != hash_rxq_init_n; ++n) - parser->queue[n].offset += cur_item->dst_sz; - } - last_voids = 0; - } - if (parser->drop) { - parser->queue[HASH_RXQ_ETH].offset += - sizeof(struct ibv_flow_spec_action_drop); - } - if (parser->mark) { - for (i = 0; i != hash_rxq_init_n; ++i) - parser->queue[i].offset += - sizeof(struct ibv_flow_spec_action_tag); - } - if (parser->count) { - unsigned int size = sizeof(struct ibv_flow_spec_counter_action); - - for (i = 0; i != hash_rxq_init_n; ++i) - parser->queue[i].offset += size; - } - return 0; -exit_item_not_supported: - return rte_flow_error_set(error, -ret, RTE_FLOW_ERROR_TYPE_ITEM, - items, "item not supported"); -} - -/** - * Allocate memory space to store verbs flow attributes. - * - * @param[in] size - * Amount of byte to allocate. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * A verbs flow attribute on success, NULL otherwise and rte_errno is set. - */ -static struct ibv_flow_attr * -mlx5_flow_convert_allocate(unsigned int size, struct rte_flow_error *error) -{ - struct ibv_flow_attr *ibv_attr; - - ibv_attr = rte_calloc(__func__, 1, size, 0); - if (!ibv_attr) { - rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot allocate verbs spec attributes"); - return NULL; - } - return ibv_attr; -} - -/** - * Make inner packet matching with an higher priority from the non Inner - * matching. - * - * @param dev - * Pointer to Ethernet device. - * @param[in, out] parser - * Internal parser structure. - * @param attr - * User flow attribute. - */ -static void -mlx5_flow_update_priority(struct rte_eth_dev *dev, - struct mlx5_flow_parse *parser, - const struct rte_flow_attr *attr) -{ - struct priv *priv = dev->data->dev_private; - unsigned int i; - uint16_t priority; - - /* 8 priorities >= 16 priorities - * Control flow: 4-7 8-15 - * User normal flow: 1-3 4-7 - * User tunnel flow: 0-2 0-3 - */ - priority = attr->priority * MLX5_VERBS_FLOW_PRIO_8; - if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_PRIO_8) - priority /= 2; - /* - * Lower non-tunnel flow Verbs priority 1 if only support 8 Verbs - * priorities, lower 4 otherwise. - */ - if (!parser->inner) { - if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_PRIO_8) - priority += 1; - else - priority += MLX5_VERBS_FLOW_PRIO_8 / 2; - } - if (parser->drop) { - parser->queue[HASH_RXQ_ETH].ibv_attr->priority = priority + - hash_rxq_init[HASH_RXQ_ETH].flow_priority; - return; - } - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - parser->queue[i].ibv_attr->priority = priority + - hash_rxq_init[i].flow_priority; - } -} - -/** - * Finalise verbs flow attributes. - * - * @param[in, out] parser - * Internal parser structure. - */ -static void -mlx5_flow_convert_finalise(struct mlx5_flow_parse *parser) -{ - unsigned int i; - uint32_t inner = parser->inner; - - /* Don't create extra flows for outer RSS. */ - if (parser->tunnel && parser->rss_conf.level < 2) - return; - /* - * Fill missing layers in verbs specifications, or compute the correct - * offset to allocate the memory space for the attributes and - * specifications. - */ - for (i = 0; i != hash_rxq_init_n - 1; ++i) { - union { - struct ibv_flow_spec_ipv4_ext ipv4; - struct ibv_flow_spec_ipv6 ipv6; - struct ibv_flow_spec_tcp_udp udp_tcp; - struct ibv_flow_spec_eth eth; - } specs; - void *dst; - uint16_t size; - - if (i == parser->layer) - continue; - if (parser->layer == HASH_RXQ_ETH || - parser->layer == HASH_RXQ_TUNNEL) { - if (hash_rxq_init[i].ip_version == MLX5_IPV4) { - size = sizeof(struct ibv_flow_spec_ipv4_ext); - specs.ipv4 = (struct ibv_flow_spec_ipv4_ext){ - .type = inner | IBV_FLOW_SPEC_IPV4_EXT, - .size = size, - }; - } else { - size = sizeof(struct ibv_flow_spec_ipv6); - specs.ipv6 = (struct ibv_flow_spec_ipv6){ - .type = inner | IBV_FLOW_SPEC_IPV6, - .size = size, - }; - } - if (parser->queue[i].ibv_attr) { - dst = (void *)((uintptr_t) - parser->queue[i].ibv_attr + - parser->queue[i].offset); - memcpy(dst, &specs, size); - ++parser->queue[i].ibv_attr->num_of_specs; - } - parser->queue[i].offset += size; - } - if ((i == HASH_RXQ_UDPV4) || (i == HASH_RXQ_TCPV4) || - (i == HASH_RXQ_UDPV6) || (i == HASH_RXQ_TCPV6)) { - size = sizeof(struct ibv_flow_spec_tcp_udp); - specs.udp_tcp = (struct ibv_flow_spec_tcp_udp) { - .type = inner | ((i == HASH_RXQ_UDPV4 || - i == HASH_RXQ_UDPV6) ? - IBV_FLOW_SPEC_UDP : - IBV_FLOW_SPEC_TCP), - .size = size, - }; - if (parser->queue[i].ibv_attr) { - dst = (void *)((uintptr_t) - parser->queue[i].ibv_attr + - parser->queue[i].offset); - memcpy(dst, &specs, size); - ++parser->queue[i].ibv_attr->num_of_specs; - } - parser->queue[i].offset += size; - } - } -} - -/** - * Update flows according to pattern and RSS hash fields. - * - * @param[in, out] parser - * Internal parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert_rss(struct mlx5_flow_parse *parser) -{ - unsigned int i; - enum hash_rxq_type start; - enum hash_rxq_type layer; - int outer = parser->tunnel && parser->rss_conf.level < 2; - uint64_t rss = parser->rss_conf.types; - - layer = outer ? parser->out_layer : parser->layer; - if (layer == HASH_RXQ_TUNNEL) - layer = HASH_RXQ_ETH; - if (outer) { - /* Only one hash type for outer RSS. */ - if (rss && layer == HASH_RXQ_ETH) { - start = HASH_RXQ_TCPV4; - } else if (rss && layer != HASH_RXQ_ETH && - !(rss & hash_rxq_init[layer].dpdk_rss_hf)) { - /* If RSS not match L4 pattern, try L3 RSS. */ - if (layer < HASH_RXQ_IPV4) - layer = HASH_RXQ_IPV4; - else if (layer > HASH_RXQ_IPV4 && layer < HASH_RXQ_IPV6) - layer = HASH_RXQ_IPV6; - start = layer; - } else { - start = layer; - } - /* Scan first valid hash type. */ - for (i = start; rss && i <= layer; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - if (hash_rxq_init[i].dpdk_rss_hf & rss) - break; - } - if (rss && i <= layer) - parser->queue[layer].hash_fields = - hash_rxq_init[i].hash_fields; - /* Trim unused hash types. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser->queue[i].ibv_attr && i != layer) { - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - } - } - } else { - /* Expand for inner or normal RSS. */ - if (rss && (layer == HASH_RXQ_ETH || layer == HASH_RXQ_IPV4)) - start = HASH_RXQ_TCPV4; - else if (rss && layer == HASH_RXQ_IPV6) - start = HASH_RXQ_TCPV6; - else - start = layer; - /* For L4 pattern, try L3 RSS if no L4 RSS. */ - /* Trim unused hash types. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - if (i < start || i > layer) { - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - continue; - } - if (!rss) - continue; - if (hash_rxq_init[i].dpdk_rss_hf & rss) { - parser->queue[i].hash_fields = - hash_rxq_init[i].hash_fields; - } else if (i != layer) { - /* Remove unused RSS expansion. */ - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - } else if (layer < HASH_RXQ_IPV4 && - (hash_rxq_init[HASH_RXQ_IPV4].dpdk_rss_hf & - rss)) { - /* Allow IPv4 RSS on L4 pattern. */ - parser->queue[i].hash_fields = - hash_rxq_init[HASH_RXQ_IPV4] - .hash_fields; - } else if (i > HASH_RXQ_IPV4 && i < HASH_RXQ_IPV6 && - (hash_rxq_init[HASH_RXQ_IPV6].dpdk_rss_hf & - rss)) { - /* Allow IPv4 RSS on L4 pattern. */ - parser->queue[i].hash_fields = - hash_rxq_init[HASH_RXQ_IPV6] - .hash_fields; - } - } - } - return 0; -} - -/** - * Validate and convert a flow supported by the NIC. - * - * @param dev - * Pointer to Ethernet device. - * @param[in] attr - * Flow rule attributes. - * @param[in] pattern - * Pattern specification (list terminated by the END pattern item). - * @param[in] actions - * Associated actions (list terminated by the END action). - * @param[out] error - * Perform verbose error reporting if not NULL. - * @param[in, out] parser - * Internal parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert(struct rte_eth_dev *dev, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], - struct rte_flow_error *error, - struct mlx5_flow_parse *parser) -{ - const struct mlx5_flow_items *cur_item = mlx5_flow_items; - unsigned int i; - int ret; - - /* First step. Validate the attributes, items and actions. */ - *parser = (struct mlx5_flow_parse){ - .create = parser->create, - .layer = HASH_RXQ_ETH, - .mark_id = MLX5_FLOW_MARK_DEFAULT, - }; - ret = mlx5_flow_convert_attributes(attr, error); - if (ret) - return ret; - ret = mlx5_flow_convert_actions(dev, actions, error, parser); - if (ret) - return ret; - ret = mlx5_flow_convert_items_validate(dev, items, error, parser); - if (ret) - return ret; - mlx5_flow_convert_finalise(parser); - /* - * Second step. - * Allocate the memory space to store verbs specifications. - */ - if (parser->drop) { - unsigned int offset = parser->queue[HASH_RXQ_ETH].offset; - - parser->queue[HASH_RXQ_ETH].ibv_attr = - mlx5_flow_convert_allocate(offset, error); - if (!parser->queue[HASH_RXQ_ETH].ibv_attr) - goto exit_enomem; - parser->queue[HASH_RXQ_ETH].offset = - sizeof(struct ibv_flow_attr); - } else { - for (i = 0; i != hash_rxq_init_n; ++i) { - unsigned int offset; - - offset = parser->queue[i].offset; - parser->queue[i].ibv_attr = - mlx5_flow_convert_allocate(offset, error); - if (!parser->queue[i].ibv_attr) - goto exit_enomem; - parser->queue[i].offset = sizeof(struct ibv_flow_attr); - } - } - /* Third step. Conversion parse, fill the specifications. */ - parser->inner = 0; - parser->tunnel = 0; - parser->layer = HASH_RXQ_ETH; - for (; items->type != RTE_FLOW_ITEM_TYPE_END; ++items) { - struct mlx5_flow_data data = { - .dev = dev, - .parser = parser, - .error = error, - }; - - if (items->type == RTE_FLOW_ITEM_TYPE_VOID) - continue; - cur_item = &mlx5_flow_items[items->type]; - ret = cur_item->convert(items, - (cur_item->default_mask ? - cur_item->default_mask : - cur_item->mask), - &data); - if (ret) - goto exit_free; - } - if (!parser->drop) { - /* RSS check, remove unused hash types. */ - ret = mlx5_flow_convert_rss(parser); - if (ret) - goto exit_free; - /* Complete missing specification. */ - mlx5_flow_convert_finalise(parser); - } - mlx5_flow_update_priority(dev, parser, attr); - if (parser->mark) - mlx5_flow_create_flag_mark(parser, parser->mark_id); - if (parser->count && parser->create) { - mlx5_flow_create_count(dev, parser); - if (!parser->cs) - goto exit_count_error; - } -exit_free: - /* Only verification is expected, all resources should be released. */ - if (!parser->create) { - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser->queue[i].ibv_attr) { - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - } - } - } - return ret; -exit_enomem: - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser->queue[i].ibv_attr) { - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - } - } - rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, "cannot allocate verbs spec attributes"); - return -rte_errno; -exit_count_error: - rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, "cannot create counter"); - return -rte_errno; -} - -/** - * Copy the specification created into the flow. - * - * @param parser - * Internal parser structure. - * @param src - * Create specification. - * @param size - * Size in bytes of the specification to copy. - */ -static void -mlx5_flow_create_copy(struct mlx5_flow_parse *parser, void *src, - unsigned int size) -{ - unsigned int i; - void *dst; - - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - dst = (void *)((uintptr_t)parser->queue[i].ibv_attr + - parser->queue[i].offset); - memcpy(dst, src, size); - ++parser->queue[i].ibv_attr->num_of_specs; - parser->queue[i].offset += size; - } -} - -/** - * Convert Ethernet item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_eth(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_eth *spec = item->spec; - const struct rte_flow_item_eth *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - const unsigned int eth_size = sizeof(struct ibv_flow_spec_eth); - struct ibv_flow_spec_eth eth = { - .type = parser->inner | IBV_FLOW_SPEC_ETH, - .size = eth_size, - }; - - parser->layer = HASH_RXQ_ETH; - if (spec) { - unsigned int i; - - if (!mask) - mask = default_mask; - memcpy(ð.val.dst_mac, spec->dst.addr_bytes, ETHER_ADDR_LEN); - memcpy(ð.val.src_mac, spec->src.addr_bytes, ETHER_ADDR_LEN); - eth.val.ether_type = spec->type; - memcpy(ð.mask.dst_mac, mask->dst.addr_bytes, ETHER_ADDR_LEN); - memcpy(ð.mask.src_mac, mask->src.addr_bytes, ETHER_ADDR_LEN); - eth.mask.ether_type = mask->type; - /* Remove unwanted bits from values. */ - for (i = 0; i < ETHER_ADDR_LEN; ++i) { - eth.val.dst_mac[i] &= eth.mask.dst_mac[i]; - eth.val.src_mac[i] &= eth.mask.src_mac[i]; - } - eth.val.ether_type &= eth.mask.ether_type; - } - mlx5_flow_create_copy(parser, ð, eth_size); - return 0; -} - -/** - * Convert VLAN item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_vlan(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_vlan *spec = item->spec; - const struct rte_flow_item_vlan *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - struct ibv_flow_spec_eth *eth; - const unsigned int eth_size = sizeof(struct ibv_flow_spec_eth); - const char *msg = "VLAN cannot be empty"; - - if (spec) { - unsigned int i; - if (!mask) - mask = default_mask; - - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - - eth = (void *)((uintptr_t)parser->queue[i].ibv_attr + - parser->queue[i].offset - eth_size); - eth->val.vlan_tag = spec->tci; - eth->mask.vlan_tag = mask->tci; - eth->val.vlan_tag &= eth->mask.vlan_tag; - /* - * From verbs perspective an empty VLAN is equivalent - * to a packet without VLAN layer. - */ - if (!eth->mask.vlan_tag) - goto error; - /* Outer TPID cannot be matched. */ - if (eth->mask.ether_type) { - msg = "VLAN TPID matching is not supported"; - goto error; - } - eth->val.ether_type = spec->inner_type; - eth->mask.ether_type = mask->inner_type; - eth->val.ether_type &= eth->mask.ether_type; - } - return 0; - } -error: - return rte_flow_error_set(data->error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, - item, msg); -} - -/** - * Convert IPv4 item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_ipv4(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - struct priv *priv = data->dev->data->dev_private; - const struct rte_flow_item_ipv4 *spec = item->spec; - const struct rte_flow_item_ipv4 *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int ipv4_size = sizeof(struct ibv_flow_spec_ipv4_ext); - struct ibv_flow_spec_ipv4_ext ipv4 = { - .type = parser->inner | IBV_FLOW_SPEC_IPV4_EXT, - .size = ipv4_size, - }; - - if (parser->layer == HASH_RXQ_TUNNEL && - parser->tunnel == ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN)] && - !priv->config.l3_vxlan_en) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 VXLAN not enabled by device" - " parameter and/or not configured" - " in firmware"); - parser->layer = HASH_RXQ_IPV4; - if (spec) { - if (!mask) - mask = default_mask; - ipv4.val = (struct ibv_flow_ipv4_ext_filter){ - .src_ip = spec->hdr.src_addr, - .dst_ip = spec->hdr.dst_addr, - .proto = spec->hdr.next_proto_id, - .tos = spec->hdr.type_of_service, - }; - ipv4.mask = (struct ibv_flow_ipv4_ext_filter){ - .src_ip = mask->hdr.src_addr, - .dst_ip = mask->hdr.dst_addr, - .proto = mask->hdr.next_proto_id, - .tos = mask->hdr.type_of_service, - }; - /* Remove unwanted bits from values. */ - ipv4.val.src_ip &= ipv4.mask.src_ip; - ipv4.val.dst_ip &= ipv4.mask.dst_ip; - ipv4.val.proto &= ipv4.mask.proto; - ipv4.val.tos &= ipv4.mask.tos; - } - mlx5_flow_create_copy(parser, &ipv4, ipv4_size); - return 0; -} - -/** - * Convert IPv6 item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_ipv6(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - struct priv *priv = data->dev->data->dev_private; - const struct rte_flow_item_ipv6 *spec = item->spec; - const struct rte_flow_item_ipv6 *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int ipv6_size = sizeof(struct ibv_flow_spec_ipv6); - struct ibv_flow_spec_ipv6 ipv6 = { - .type = parser->inner | IBV_FLOW_SPEC_IPV6, - .size = ipv6_size, - }; - - if (parser->layer == HASH_RXQ_TUNNEL && - parser->tunnel == ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN)] && - !priv->config.l3_vxlan_en) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 VXLAN not enabled by device" - " parameter and/or not configured" - " in firmware"); - parser->layer = HASH_RXQ_IPV6; - if (spec) { - unsigned int i; - uint32_t vtc_flow_val; - uint32_t vtc_flow_mask; - - if (!mask) - mask = default_mask; - memcpy(&ipv6.val.src_ip, spec->hdr.src_addr, - RTE_DIM(ipv6.val.src_ip)); - memcpy(&ipv6.val.dst_ip, spec->hdr.dst_addr, - RTE_DIM(ipv6.val.dst_ip)); - memcpy(&ipv6.mask.src_ip, mask->hdr.src_addr, - RTE_DIM(ipv6.mask.src_ip)); - memcpy(&ipv6.mask.dst_ip, mask->hdr.dst_addr, - RTE_DIM(ipv6.mask.dst_ip)); - vtc_flow_val = rte_be_to_cpu_32(spec->hdr.vtc_flow); - vtc_flow_mask = rte_be_to_cpu_32(mask->hdr.vtc_flow); - ipv6.val.flow_label = - rte_cpu_to_be_32((vtc_flow_val & IPV6_HDR_FL_MASK) >> - IPV6_HDR_FL_SHIFT); - ipv6.val.traffic_class = (vtc_flow_val & IPV6_HDR_TC_MASK) >> - IPV6_HDR_TC_SHIFT; - ipv6.val.next_hdr = spec->hdr.proto; - ipv6.val.hop_limit = spec->hdr.hop_limits; - ipv6.mask.flow_label = - rte_cpu_to_be_32((vtc_flow_mask & IPV6_HDR_FL_MASK) >> - IPV6_HDR_FL_SHIFT); - ipv6.mask.traffic_class = (vtc_flow_mask & IPV6_HDR_TC_MASK) >> - IPV6_HDR_TC_SHIFT; - ipv6.mask.next_hdr = mask->hdr.proto; - ipv6.mask.hop_limit = mask->hdr.hop_limits; - /* Remove unwanted bits from values. */ - for (i = 0; i < RTE_DIM(ipv6.val.src_ip); ++i) { - ipv6.val.src_ip[i] &= ipv6.mask.src_ip[i]; - ipv6.val.dst_ip[i] &= ipv6.mask.dst_ip[i]; - } - ipv6.val.flow_label &= ipv6.mask.flow_label; - ipv6.val.traffic_class &= ipv6.mask.traffic_class; - ipv6.val.next_hdr &= ipv6.mask.next_hdr; - ipv6.val.hop_limit &= ipv6.mask.hop_limit; - } - mlx5_flow_create_copy(parser, &ipv6, ipv6_size); - return 0; -} - -/** - * Convert UDP item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_udp(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_udp *spec = item->spec; - const struct rte_flow_item_udp *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int udp_size = sizeof(struct ibv_flow_spec_tcp_udp); - struct ibv_flow_spec_tcp_udp udp = { - .type = parser->inner | IBV_FLOW_SPEC_UDP, - .size = udp_size, - }; - - if (parser->layer == HASH_RXQ_IPV4) - parser->layer = HASH_RXQ_UDPV4; - else - parser->layer = HASH_RXQ_UDPV6; - if (spec) { - if (!mask) - mask = default_mask; - udp.val.dst_port = spec->hdr.dst_port; - udp.val.src_port = spec->hdr.src_port; - udp.mask.dst_port = mask->hdr.dst_port; - udp.mask.src_port = mask->hdr.src_port; - /* Remove unwanted bits from values. */ - udp.val.src_port &= udp.mask.src_port; - udp.val.dst_port &= udp.mask.dst_port; - } - mlx5_flow_create_copy(parser, &udp, udp_size); - return 0; -} - -/** - * Convert TCP item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_tcp(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_tcp *spec = item->spec; - const struct rte_flow_item_tcp *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int tcp_size = sizeof(struct ibv_flow_spec_tcp_udp); - struct ibv_flow_spec_tcp_udp tcp = { - .type = parser->inner | IBV_FLOW_SPEC_TCP, - .size = tcp_size, - }; - - if (parser->layer == HASH_RXQ_IPV4) - parser->layer = HASH_RXQ_TCPV4; - else - parser->layer = HASH_RXQ_TCPV6; - if (spec) { - if (!mask) - mask = default_mask; - tcp.val.dst_port = spec->hdr.dst_port; - tcp.val.src_port = spec->hdr.src_port; - tcp.mask.dst_port = mask->hdr.dst_port; - tcp.mask.src_port = mask->hdr.src_port; - /* Remove unwanted bits from values. */ - tcp.val.src_port &= tcp.mask.src_port; - tcp.val.dst_port &= tcp.mask.dst_port; - } - mlx5_flow_create_copy(parser, &tcp, tcp_size); - return 0; -} - -/** - * Convert VXLAN item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_vxlan(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_vxlan *spec = item->spec; - const struct rte_flow_item_vxlan *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int size = sizeof(struct ibv_flow_spec_tunnel); - struct ibv_flow_spec_tunnel vxlan = { - .type = parser->inner | IBV_FLOW_SPEC_VXLAN_TUNNEL, - .size = size, - }; - union vni { - uint32_t vlan_id; - uint8_t vni[4]; - } id; - - id.vni[0] = 0; - parser->inner = IBV_FLOW_SPEC_INNER; - parser->tunnel = ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN)]; - parser->out_layer = parser->layer; - parser->layer = HASH_RXQ_TUNNEL; - if (spec) { - if (!mask) - mask = default_mask; - memcpy(&id.vni[1], spec->vni, 3); - vxlan.val.tunnel_id = id.vlan_id; - memcpy(&id.vni[1], mask->vni, 3); - vxlan.mask.tunnel_id = id.vlan_id; - /* Remove unwanted bits from values. */ - vxlan.val.tunnel_id &= vxlan.mask.tunnel_id; - } - /* - * Tunnel id 0 is equivalent as not adding a VXLAN layer, if only this - * layer is defined in the Verbs specification it is interpreted as - * wildcard and all packets will match this rule, if it follows a full - * stack layer (ex: eth / ipv4 / udp), all packets matching the layers - * before will also match this rule. - * To avoid such situation, VNI 0 is currently refused. - */ - /* Only allow tunnel w/o tunnel id pattern after proper outer spec. */ - if (parser->out_layer == HASH_RXQ_ETH && !vxlan.val.tunnel_id) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "VxLAN vni cannot be 0"); - mlx5_flow_create_copy(parser, &vxlan, size); - return 0; -} - -/** - * Convert VXLAN-GPE item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_vxlan_gpe(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - struct priv *priv = data->dev->data->dev_private; - const struct rte_flow_item_vxlan_gpe *spec = item->spec; - const struct rte_flow_item_vxlan_gpe *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int size = sizeof(struct ibv_flow_spec_tunnel); - struct ibv_flow_spec_tunnel vxlan = { - .type = parser->inner | IBV_FLOW_SPEC_VXLAN_TUNNEL, - .size = size, - }; - union vni { - uint32_t vlan_id; - uint8_t vni[4]; - } id; - - if (!priv->config.l3_vxlan_en) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 VXLAN not enabled by device" - " parameter and/or not configured" - " in firmware"); - id.vni[0] = 0; - parser->inner = IBV_FLOW_SPEC_INNER; - parser->tunnel = ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN_GPE)]; - parser->out_layer = parser->layer; - parser->layer = HASH_RXQ_TUNNEL; - if (spec) { - if (!mask) - mask = default_mask; - memcpy(&id.vni[1], spec->vni, 3); - vxlan.val.tunnel_id = id.vlan_id; - memcpy(&id.vni[1], mask->vni, 3); - vxlan.mask.tunnel_id = id.vlan_id; - if (spec->protocol) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "VxLAN-GPE protocol not" - " supported"); - /* Remove unwanted bits from values. */ - vxlan.val.tunnel_id &= vxlan.mask.tunnel_id; - } - /* - * Tunnel id 0 is equivalent as not adding a VXLAN layer, if only this - * layer is defined in the Verbs specification it is interpreted as - * wildcard and all packets will match this rule, if it follows a full - * stack layer (ex: eth / ipv4 / udp), all packets matching the layers - * before will also match this rule. - * To avoid such situation, VNI 0 is currently refused. - */ - /* Only allow tunnel w/o tunnel id pattern after proper outer spec. */ - if (parser->out_layer == HASH_RXQ_ETH && !vxlan.val.tunnel_id) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "VxLAN-GPE vni cannot be 0"); - mlx5_flow_create_copy(parser, &vxlan, size); - return 0; -} - -/** - * Convert GRE item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_gre(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - struct mlx5_flow_parse *parser = data->parser; -#ifndef HAVE_IBV_DEVICE_MPLS_SUPPORT - (void)default_mask; - unsigned int size = sizeof(struct ibv_flow_spec_tunnel); - struct ibv_flow_spec_tunnel tunnel = { - .type = parser->inner | IBV_FLOW_SPEC_VXLAN_TUNNEL, - .size = size, - }; -#else - const struct rte_flow_item_gre *spec = item->spec; - const struct rte_flow_item_gre *mask = item->mask; - unsigned int size = sizeof(struct ibv_flow_spec_gre); - struct ibv_flow_spec_gre tunnel = { - .type = parser->inner | IBV_FLOW_SPEC_GRE, - .size = size, - }; -#endif - struct ibv_flow_spec_ipv4_ext *ipv4; - struct ibv_flow_spec_ipv6 *ipv6; - unsigned int i; - - parser->inner = IBV_FLOW_SPEC_INNER; - parser->tunnel = ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_GRE)]; - parser->out_layer = parser->layer; - parser->layer = HASH_RXQ_TUNNEL; -#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT - if (spec) { - if (!mask) - mask = default_mask; - tunnel.val.c_ks_res0_ver = spec->c_rsvd0_ver; - tunnel.val.protocol = spec->protocol; - tunnel.mask.c_ks_res0_ver = mask->c_rsvd0_ver; - tunnel.mask.protocol = mask->protocol; - /* Remove unwanted bits from values. */ - tunnel.val.c_ks_res0_ver &= tunnel.mask.c_ks_res0_ver; - tunnel.val.protocol &= tunnel.mask.protocol; - tunnel.val.key &= tunnel.mask.key; - } -#endif - /* Update encapsulation IP layer protocol. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - if (parser->out_layer == HASH_RXQ_IPV4) { - ipv4 = (void *)((uintptr_t)parser->queue[i].ibv_attr + - parser->queue[i].offset - - sizeof(struct ibv_flow_spec_ipv4_ext)); - if (ipv4->mask.proto && ipv4->val.proto != MLX5_GRE) - break; - ipv4->val.proto = MLX5_GRE; - ipv4->mask.proto = 0xff; - } else if (parser->out_layer == HASH_RXQ_IPV6) { - ipv6 = (void *)((uintptr_t)parser->queue[i].ibv_attr + - parser->queue[i].offset - - sizeof(struct ibv_flow_spec_ipv6)); - if (ipv6->mask.next_hdr && - ipv6->val.next_hdr != MLX5_GRE) - break; - ipv6->val.next_hdr = MLX5_GRE; - ipv6->mask.next_hdr = 0xff; - } - } - if (i != hash_rxq_init_n) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "IP protocol of GRE must be 47"); - mlx5_flow_create_copy(parser, &tunnel, size); - return 0; -} - -/** - * Convert MPLS item to Verbs specification. - * MPLS tunnel types currently supported are MPLS-in-GRE and MPLS-in-UDP. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_mpls(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ -#ifndef HAVE_IBV_DEVICE_MPLS_SUPPORT - (void)default_mask; - return rte_flow_error_set(data->error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "MPLS is not supported by driver"); -#else - const struct rte_flow_item_mpls *spec = item->spec; - const struct rte_flow_item_mpls *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int size = sizeof(struct ibv_flow_spec_mpls); - struct ibv_flow_spec_mpls mpls = { - .type = IBV_FLOW_SPEC_MPLS, - .size = size, - }; - - parser->inner = IBV_FLOW_SPEC_INNER; - if (parser->layer == HASH_RXQ_UDPV4 || - parser->layer == HASH_RXQ_UDPV6) { - parser->tunnel = - ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_MPLS_IN_UDP)]; - parser->out_layer = parser->layer; - } else { - parser->tunnel = - ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_MPLS_IN_GRE)]; - /* parser->out_layer stays as in GRE out_layer. */ - } - parser->layer = HASH_RXQ_TUNNEL; - if (spec) { - if (!mask) - mask = default_mask; - /* - * The verbs label field includes the entire MPLS header: - * bits 0:19 - label value field. - * bits 20:22 - traffic class field. - * bits 23 - bottom of stack bit. - * bits 24:31 - ttl field. - */ - mpls.val.label = *(const uint32_t *)spec; - mpls.mask.label = *(const uint32_t *)mask; - /* Remove unwanted bits from values. */ - mpls.val.label &= mpls.mask.label; - } - mlx5_flow_create_copy(parser, &mpls, size); - return 0; -#endif -} - -/** - * Convert mark/flag action to Verbs specification. - * - * @param parser - * Internal parser structure. - * @param mark_id - * Mark identifier. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_flag_mark(struct mlx5_flow_parse *parser, uint32_t mark_id) -{ - unsigned int size = sizeof(struct ibv_flow_spec_action_tag); - struct ibv_flow_spec_action_tag tag = { - .type = IBV_FLOW_SPEC_ACTION_TAG, - .size = size, - .tag_id = mlx5_flow_mark_set(mark_id), - }; - - assert(parser->mark); - mlx5_flow_create_copy(parser, &tag, size); - return 0; -} - -/** - * Convert count action to Verbs specification. - * - * @param dev - * Pointer to Ethernet device. - * @param parser - * Pointer to MLX5 flow parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_count(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flow_parse *parser __rte_unused) -{ -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT - struct priv *priv = dev->data->dev_private; - unsigned int size = sizeof(struct ibv_flow_spec_counter_action); - struct ibv_counter_set_init_attr init_attr = {0}; - struct ibv_flow_spec_counter_action counter = { - .type = IBV_FLOW_SPEC_ACTION_COUNT, - .size = size, - .counter_set_handle = 0, - }; - - init_attr.counter_set_id = 0; - parser->cs = mlx5_glue->create_counter_set(priv->ctx, &init_attr); - if (!parser->cs) { - rte_errno = EINVAL; - return -rte_errno; - } - counter.counter_set_handle = parser->cs->handle; - mlx5_flow_create_copy(parser, &counter, size); -#endif - return 0; -} - -/** - * Complete flow rule creation with a drop queue. - * - * @param dev - * Pointer to Ethernet device. - * @param parser - * Internal parser structure. - * @param flow - * Pointer to the rte_flow. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_action_queue_drop(struct rte_eth_dev *dev, - struct mlx5_flow_parse *parser, - struct rte_flow *flow, - struct rte_flow_error *error) -{ - struct priv *priv = dev->data->dev_private; - struct ibv_flow_spec_action_drop *drop; - unsigned int size = sizeof(struct ibv_flow_spec_action_drop); - - assert(priv->pd); - assert(priv->ctx); - flow->drop = 1; - drop = (void *)((uintptr_t)parser->queue[HASH_RXQ_ETH].ibv_attr + - parser->queue[HASH_RXQ_ETH].offset); - *drop = (struct ibv_flow_spec_action_drop){ - .type = IBV_FLOW_SPEC_ACTION_DROP, - .size = size, - }; - ++parser->queue[HASH_RXQ_ETH].ibv_attr->num_of_specs; - parser->queue[HASH_RXQ_ETH].offset += size; - flow->frxq[HASH_RXQ_ETH].ibv_attr = - parser->queue[HASH_RXQ_ETH].ibv_attr; - if (parser->count) - flow->cs = parser->cs; - if (!dev->data->dev_started) - return 0; - parser->queue[HASH_RXQ_ETH].ibv_attr = NULL; - flow->frxq[HASH_RXQ_ETH].ibv_flow = - mlx5_glue->create_flow(priv->flow_drop_queue->qp, - flow->frxq[HASH_RXQ_ETH].ibv_attr); - if (!flow->frxq[HASH_RXQ_ETH].ibv_flow) { - rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, "flow rule creation failure"); - goto error; - } - return 0; -error: - assert(flow); - if (flow->frxq[HASH_RXQ_ETH].ibv_flow) { - claim_zero(mlx5_glue->destroy_flow - (flow->frxq[HASH_RXQ_ETH].ibv_flow)); - flow->frxq[HASH_RXQ_ETH].ibv_flow = NULL; - } - if (flow->frxq[HASH_RXQ_ETH].ibv_attr) { - rte_free(flow->frxq[HASH_RXQ_ETH].ibv_attr); - flow->frxq[HASH_RXQ_ETH].ibv_attr = NULL; - } - if (flow->cs) { - claim_zero(mlx5_glue->destroy_counter_set(flow->cs)); - flow->cs = NULL; - parser->cs = NULL; - } - return -rte_errno; -} - -/** - * Create hash Rx queues when RSS is enabled. - * - * @param dev - * Pointer to Ethernet device. - * @param parser - * Internal parser structure. - * @param flow - * Pointer to the rte_flow. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_action_queue_rss(struct rte_eth_dev *dev, - struct mlx5_flow_parse *parser, - struct rte_flow *flow, - struct rte_flow_error *error) -{ - unsigned int i; - - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - flow->frxq[i].ibv_attr = parser->queue[i].ibv_attr; - parser->queue[i].ibv_attr = NULL; - flow->frxq[i].hash_fields = parser->queue[i].hash_fields; - if (!dev->data->dev_started) - continue; - flow->frxq[i].hrxq = - mlx5_hrxq_get(dev, - parser->rss_conf.key, - parser->rss_conf.key_len, - flow->frxq[i].hash_fields, - parser->rss_conf.queue, - parser->rss_conf.queue_num, - parser->tunnel, - parser->rss_conf.level); - if (flow->frxq[i].hrxq) - continue; - flow->frxq[i].hrxq = - mlx5_hrxq_new(dev, - parser->rss_conf.key, - parser->rss_conf.key_len, - flow->frxq[i].hash_fields, - parser->rss_conf.queue, - parser->rss_conf.queue_num, - parser->tunnel, - parser->rss_conf.level); - if (!flow->frxq[i].hrxq) { - return rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, - "cannot create hash rxq"); - } - } - return 0; -} - -/** - * RXQ update after flow rule creation. - * - * @param dev - * Pointer to Ethernet device. - * @param flow - * Pointer to the flow rule. - */ -static void -mlx5_flow_create_update_rxqs(struct rte_eth_dev *dev, struct rte_flow *flow) -{ - struct priv *priv = dev->data->dev_private; - unsigned int i; - unsigned int j; - - if (!dev->data->dev_started) - return; - for (i = 0; i != flow->rss_conf.queue_num; ++i) { - struct mlx5_rxq_data *rxq_data = (*priv->rxqs) - [(*flow->queues)[i]]; - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); - uint8_t tunnel = PTYPE_IDX(flow->tunnel); - - rxq_data->mark |= flow->mark; - if (!tunnel) - continue; - rxq_ctrl->tunnel_types[tunnel] += 1; - /* Clear tunnel type if more than one tunnel types set. */ - for (j = 0; j != RTE_DIM(rxq_ctrl->tunnel_types); ++j) { - if (j == tunnel) - continue; - if (rxq_ctrl->tunnel_types[j] > 0) { - rxq_data->tunnel = 0; - break; - } - } - if (j == RTE_DIM(rxq_ctrl->tunnel_types)) - rxq_data->tunnel = flow->tunnel; - } -} - -/** - * Dump flow hash RX queue detail. - * - * @param dev - * Pointer to Ethernet device. - * @param flow - * Pointer to the rte_flow. - * @param hrxq_idx - * Hash RX queue index. - */ -static void -mlx5_flow_dump(struct rte_eth_dev *dev __rte_unused, - struct rte_flow *flow __rte_unused, - unsigned int hrxq_idx __rte_unused) -{ -#ifndef NDEBUG - uintptr_t spec_ptr; - uint16_t j; - char buf[256]; - uint8_t off; - uint64_t extra_hash_fields = 0; +/* Dev ops structure defined in mlx5.c */ +extern const struct eth_dev_ops mlx5_dev_ops; +extern const struct eth_dev_ops mlx5_dev_ops_isolate; -#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - if (flow->tunnel && flow->rss_conf.level > 1) - extra_hash_fields = (uint32_t)IBV_RX_HASH_INNER; -#endif - spec_ptr = (uintptr_t)(flow->frxq[hrxq_idx].ibv_attr + 1); - for (j = 0, off = 0; j < flow->frxq[hrxq_idx].ibv_attr->num_of_specs; - j++) { - struct ibv_flow_spec *spec = (void *)spec_ptr; - off += sprintf(buf + off, " %x(%hu)", spec->hdr.type, - spec->hdr.size); - spec_ptr += spec->hdr.size; - } - DRV_LOG(DEBUG, - "port %u Verbs flow %p type %u: hrxq:%p qp:%p ind:%p," - " hash:%" PRIx64 "/%u specs:%hhu(%hu), priority:%hu, type:%d," - " flags:%x, comp_mask:%x specs:%s", - dev->data->port_id, (void *)flow, hrxq_idx, - (void *)flow->frxq[hrxq_idx].hrxq, - (void *)flow->frxq[hrxq_idx].hrxq->qp, - (void *)flow->frxq[hrxq_idx].hrxq->ind_table, - (flow->frxq[hrxq_idx].hash_fields | extra_hash_fields), - flow->rss_conf.queue_num, - flow->frxq[hrxq_idx].ibv_attr->num_of_specs, - flow->frxq[hrxq_idx].ibv_attr->size, - flow->frxq[hrxq_idx].ibv_attr->priority, - flow->frxq[hrxq_idx].ibv_attr->type, - flow->frxq[hrxq_idx].ibv_attr->flags, - flow->frxq[hrxq_idx].ibv_attr->comp_mask, - buf); -#endif -} +struct rte_flow { + TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ +}; -/** - * Complete flow rule creation. - * - * @param dev - * Pointer to Ethernet device. - * @param parser - * Internal parser structure. - * @param flow - * Pointer to the rte_flow. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_action_queue(struct rte_eth_dev *dev, - struct mlx5_flow_parse *parser, - struct rte_flow *flow, - struct rte_flow_error *error) -{ - struct priv *priv __rte_unused = dev->data->dev_private; - int ret; - unsigned int i; - unsigned int flows_n = 0; +static const struct rte_flow_ops mlx5_flow_ops = { + .isolate = mlx5_flow_isolate, +}; - assert(priv->pd); - assert(priv->ctx); - assert(!parser->drop); - ret = mlx5_flow_create_action_queue_rss(dev, parser, flow, error); - if (ret) - goto error; - if (parser->count) - flow->cs = parser->cs; - if (!dev->data->dev_started) - return 0; - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!flow->frxq[i].hrxq) - continue; - flow->frxq[i].ibv_flow = - mlx5_glue->create_flow(flow->frxq[i].hrxq->qp, - flow->frxq[i].ibv_attr); - mlx5_flow_dump(dev, flow, i); - if (!flow->frxq[i].ibv_flow) { - rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, "flow rule creation failure"); - goto error; - } - ++flows_n; - } - if (!flows_n) { - rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, "internal error in flow creation"); - goto error; - } - mlx5_flow_create_update_rxqs(dev, flow); - return 0; -error: - ret = rte_errno; /* Save rte_errno before cleanup. */ - assert(flow); - for (i = 0; i != hash_rxq_init_n; ++i) { - if (flow->frxq[i].ibv_flow) { - struct ibv_flow *ibv_flow = flow->frxq[i].ibv_flow; +/* Convert FDIR request to Generic flow. */ +struct mlx5_fdir { + struct rte_flow_attr attr; + struct rte_flow_action actions[2]; + struct rte_flow_item items[4]; + struct rte_flow_item_eth l2; + struct rte_flow_item_eth l2_mask; + union { + struct rte_flow_item_ipv4 ipv4; + struct rte_flow_item_ipv6 ipv6; + } l3; + union { + struct rte_flow_item_ipv4 ipv4; + struct rte_flow_item_ipv6 ipv6; + } l3_mask; + union { + struct rte_flow_item_udp udp; + struct rte_flow_item_tcp tcp; + } l4; + union { + struct rte_flow_item_udp udp; + struct rte_flow_item_tcp tcp; + } l4_mask; + struct rte_flow_action_queue queue; +}; - claim_zero(mlx5_glue->destroy_flow(ibv_flow)); - } - if (flow->frxq[i].hrxq) - mlx5_hrxq_release(dev, flow->frxq[i].hrxq); - if (flow->frxq[i].ibv_attr) - rte_free(flow->frxq[i].ibv_attr); - } - if (flow->cs) { - claim_zero(mlx5_glue->destroy_counter_set(flow->cs)); - flow->cs = NULL; - parser->cs = NULL; - } - rte_errno = ret; /* Restore rte_errno. */ - return -rte_errno; -} +/* Verbs specification header. */ +struct ibv_spec_header { + enum ibv_flow_spec_type type; + uint16_t size; +}; /** * Convert a flow. @@ -2452,69 +95,17 @@ mlx5_flow_create_action_queue(struct rte_eth_dev *dev, * A flow on success, NULL otherwise and rte_errno is set. */ static struct rte_flow * -mlx5_flow_list_create(struct rte_eth_dev *dev, - struct mlx5_flows *list, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], +mlx5_flow_list_create(struct rte_eth_dev *dev __rte_unused, + struct mlx5_flows *list __rte_unused, + const struct rte_flow_attr *attr __rte_unused, + const struct rte_flow_item items[] __rte_unused, + const struct rte_flow_action actions[] __rte_unused, struct rte_flow_error *error) { - struct mlx5_flow_parse parser = { .create = 1, }; - struct rte_flow *flow = NULL; - unsigned int i; - int ret; - - ret = mlx5_flow_convert(dev, attr, items, actions, error, &parser); - if (ret) - goto exit; - flow = rte_calloc(__func__, 1, - sizeof(*flow) + - parser.rss_conf.queue_num * sizeof(uint16_t), - 0); - if (!flow) { - rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot allocate flow memory"); - return NULL; - } - /* Copy configuration. */ - flow->queues = (uint16_t (*)[])(flow + 1); - flow->tunnel = parser.tunnel; - flow->rss_conf = (struct rte_flow_action_rss){ - .func = RTE_ETH_HASH_FUNCTION_DEFAULT, - .level = parser.rss_conf.level, - .types = parser.rss_conf.types, - .key_len = parser.rss_conf.key_len, - .queue_num = parser.rss_conf.queue_num, - .key = memcpy(flow->rss_key, parser.rss_conf.key, - sizeof(*parser.rss_conf.key) * - parser.rss_conf.key_len), - .queue = memcpy(flow->queues, parser.rss_conf.queue, - sizeof(*parser.rss_conf.queue) * - parser.rss_conf.queue_num), - }; - flow->mark = parser.mark; - /* finalise the flow. */ - if (parser.drop) - ret = mlx5_flow_create_action_queue_drop(dev, &parser, flow, - error); - else - ret = mlx5_flow_create_action_queue(dev, &parser, flow, error); - if (ret) - goto exit; - TAILQ_INSERT_TAIL(list, flow, next); - DRV_LOG(DEBUG, "port %u flow created %p", dev->data->port_id, - (void *)flow); - return flow; -exit: - DRV_LOG(ERR, "port %u flow creation error: %s", dev->data->port_id, - error->message); - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser.queue[i].ibv_attr) - rte_free(parser.queue[i].ibv_attr); - } - rte_free(flow); + rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "action not supported"); return NULL; } @@ -2525,15 +116,16 @@ mlx5_flow_list_create(struct rte_eth_dev *dev, * @see rte_flow_ops */ int -mlx5_flow_validate(struct rte_eth_dev *dev, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], +mlx5_flow_validate(struct rte_eth_dev *dev __rte_unused, + const struct rte_flow_attr *attr __rte_unused, + const struct rte_flow_item items[] __rte_unused, + const struct rte_flow_action actions[] __rte_unused, struct rte_flow_error *error) { - struct mlx5_flow_parse parser = { .create = 0, }; - - return mlx5_flow_convert(dev, attr, items, actions, error, &parser); + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "action not supported"); } /** @@ -2543,16 +135,17 @@ mlx5_flow_validate(struct rte_eth_dev *dev, * @see rte_flow_ops */ struct rte_flow * -mlx5_flow_create(struct rte_eth_dev *dev, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], +mlx5_flow_create(struct rte_eth_dev *dev __rte_unused, + const struct rte_flow_attr *attr __rte_unused, + const struct rte_flow_item items[] __rte_unused, + const struct rte_flow_action actions[] __rte_unused, struct rte_flow_error *error) { - struct priv *priv = dev->data->dev_private; - - return mlx5_flow_list_create(dev, &priv->flows, attr, items, actions, - error); + rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "action not supported"); + return NULL; } /** @@ -2566,99 +159,10 @@ mlx5_flow_create(struct rte_eth_dev *dev, * Flow to destroy. */ static void -mlx5_flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, - struct rte_flow *flow) +mlx5_flow_list_destroy(struct rte_eth_dev *dev __rte_unused, + struct mlx5_flows *list __rte_unused, + struct rte_flow *flow __rte_unused) { - struct priv *priv = dev->data->dev_private; - unsigned int i; - - if (flow->drop || !dev->data->dev_started) - goto free; - for (i = 0; flow->tunnel && i != flow->rss_conf.queue_num; ++i) { - /* Update queue tunnel type. */ - struct mlx5_rxq_data *rxq_data = (*priv->rxqs) - [(*flow->queues)[i]]; - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); - uint8_t tunnel = PTYPE_IDX(flow->tunnel); - - assert(rxq_ctrl->tunnel_types[tunnel] > 0); - rxq_ctrl->tunnel_types[tunnel] -= 1; - if (!rxq_ctrl->tunnel_types[tunnel]) { - /* Update tunnel type. */ - uint8_t j; - uint8_t types = 0; - uint8_t last; - - for (j = 0; j < RTE_DIM(rxq_ctrl->tunnel_types); j++) - if (rxq_ctrl->tunnel_types[j]) { - types += 1; - last = j; - } - /* Keep same if more than one tunnel types left. */ - if (types == 1) - rxq_data->tunnel = ptype_ext[last]; - else if (types == 0) - /* No tunnel type left. */ - rxq_data->tunnel = 0; - } - } - for (i = 0; flow->mark && i != flow->rss_conf.queue_num; ++i) { - struct rte_flow *tmp; - int mark = 0; - - /* - * To remove the mark from the queue, the queue must not be - * present in any other marked flow (RSS or not). - */ - TAILQ_FOREACH(tmp, list, next) { - unsigned int j; - uint16_t *tqs = NULL; - uint16_t tq_n = 0; - - if (!tmp->mark) - continue; - for (j = 0; j != hash_rxq_init_n; ++j) { - if (!tmp->frxq[j].hrxq) - continue; - tqs = tmp->frxq[j].hrxq->ind_table->queues; - tq_n = tmp->frxq[j].hrxq->ind_table->queues_n; - } - if (!tq_n) - continue; - for (j = 0; (j != tq_n) && !mark; j++) - if (tqs[j] == (*flow->queues)[i]) - mark = 1; - } - (*priv->rxqs)[(*flow->queues)[i]]->mark = mark; - } -free: - if (flow->drop) { - if (flow->frxq[HASH_RXQ_ETH].ibv_flow) - claim_zero(mlx5_glue->destroy_flow - (flow->frxq[HASH_RXQ_ETH].ibv_flow)); - rte_free(flow->frxq[HASH_RXQ_ETH].ibv_attr); - } else { - for (i = 0; i != hash_rxq_init_n; ++i) { - struct mlx5_flow *frxq = &flow->frxq[i]; - - if (frxq->ibv_flow) - claim_zero(mlx5_glue->destroy_flow - (frxq->ibv_flow)); - if (frxq->hrxq) - mlx5_hrxq_release(dev, frxq->hrxq); - if (frxq->ibv_attr) - rte_free(frxq->ibv_attr); - } - } - if (flow->cs) { - claim_zero(mlx5_glue->destroy_counter_set(flow->cs)); - flow->cs = NULL; - } - TAILQ_REMOVE(list, flow, next); - DRV_LOG(DEBUG, "port %u flow destroyed %p", dev->data->port_id, - (void *)flow); - rte_free(flow); } /** @@ -2690,97 +194,9 @@ mlx5_flow_list_flush(struct rte_eth_dev *dev, struct mlx5_flows *list) * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_create_drop_queue(struct rte_eth_dev *dev) +mlx5_flow_create_drop_queue(struct rte_eth_dev *dev __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct mlx5_hrxq_drop *fdq = NULL; - - assert(priv->pd); - assert(priv->ctx); - fdq = rte_calloc(__func__, 1, sizeof(*fdq), 0); - if (!fdq) { - DRV_LOG(WARNING, - "port %u cannot allocate memory for drop queue", - dev->data->port_id); - rte_errno = ENOMEM; - return -rte_errno; - } - fdq->cq = mlx5_glue->create_cq(priv->ctx, 1, NULL, NULL, 0); - if (!fdq->cq) { - DRV_LOG(WARNING, "port %u cannot allocate CQ for drop queue", - dev->data->port_id); - rte_errno = errno; - goto error; - } - fdq->wq = mlx5_glue->create_wq - (priv->ctx, - &(struct ibv_wq_init_attr){ - .wq_type = IBV_WQT_RQ, - .max_wr = 1, - .max_sge = 1, - .pd = priv->pd, - .cq = fdq->cq, - }); - if (!fdq->wq) { - DRV_LOG(WARNING, "port %u cannot allocate WQ for drop queue", - dev->data->port_id); - rte_errno = errno; - goto error; - } - fdq->ind_table = mlx5_glue->create_rwq_ind_table - (priv->ctx, - &(struct ibv_rwq_ind_table_init_attr){ - .log_ind_tbl_size = 0, - .ind_tbl = &fdq->wq, - .comp_mask = 0, - }); - if (!fdq->ind_table) { - DRV_LOG(WARNING, - "port %u cannot allocate indirection table for drop" - " queue", - dev->data->port_id); - rte_errno = errno; - goto error; - } - fdq->qp = mlx5_glue->create_qp_ex - (priv->ctx, - &(struct ibv_qp_init_attr_ex){ - .qp_type = IBV_QPT_RAW_PACKET, - .comp_mask = - IBV_QP_INIT_ATTR_PD | - IBV_QP_INIT_ATTR_IND_TABLE | - IBV_QP_INIT_ATTR_RX_HASH, - .rx_hash_conf = (struct ibv_rx_hash_conf){ - .rx_hash_function = - IBV_RX_HASH_FUNC_TOEPLITZ, - .rx_hash_key_len = rss_hash_default_key_len, - .rx_hash_key = rss_hash_default_key, - .rx_hash_fields_mask = 0, - }, - .rwq_ind_tbl = fdq->ind_table, - .pd = priv->pd - }); - if (!fdq->qp) { - DRV_LOG(WARNING, "port %u cannot allocate QP for drop queue", - dev->data->port_id); - rte_errno = errno; - goto error; - } - priv->flow_drop_queue = fdq; return 0; -error: - if (fdq->qp) - claim_zero(mlx5_glue->destroy_qp(fdq->qp)); - if (fdq->ind_table) - claim_zero(mlx5_glue->destroy_rwq_ind_table(fdq->ind_table)); - if (fdq->wq) - claim_zero(mlx5_glue->destroy_wq(fdq->wq)); - if (fdq->cq) - claim_zero(mlx5_glue->destroy_cq(fdq->cq)); - if (fdq) - rte_free(fdq); - priv->flow_drop_queue = NULL; - return -rte_errno; } /** @@ -2790,23 +206,8 @@ mlx5_flow_create_drop_queue(struct rte_eth_dev *dev) * Pointer to Ethernet device. */ void -mlx5_flow_delete_drop_queue(struct rte_eth_dev *dev) +mlx5_flow_delete_drop_queue(struct rte_eth_dev *dev __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct mlx5_hrxq_drop *fdq = priv->flow_drop_queue; - - if (!fdq) - return; - if (fdq->qp) - claim_zero(mlx5_glue->destroy_qp(fdq->qp)); - if (fdq->ind_table) - claim_zero(mlx5_glue->destroy_rwq_ind_table(fdq->ind_table)); - if (fdq->wq) - claim_zero(mlx5_glue->destroy_wq(fdq->wq)); - if (fdq->cq) - claim_zero(mlx5_glue->destroy_cq(fdq->cq)); - rte_free(fdq); - priv->flow_drop_queue = NULL; } /** @@ -2818,70 +219,9 @@ mlx5_flow_delete_drop_queue(struct rte_eth_dev *dev) * Pointer to a TAILQ flow list. */ void -mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) +mlx5_flow_stop(struct rte_eth_dev *dev __rte_unused, + struct mlx5_flows *list __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct rte_flow *flow; - unsigned int i; - - TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) { - struct mlx5_ind_table_ibv *ind_tbl = NULL; - - if (flow->drop) { - if (!flow->frxq[HASH_RXQ_ETH].ibv_flow) - continue; - claim_zero(mlx5_glue->destroy_flow - (flow->frxq[HASH_RXQ_ETH].ibv_flow)); - flow->frxq[HASH_RXQ_ETH].ibv_flow = NULL; - DRV_LOG(DEBUG, "port %u flow %p removed", - dev->data->port_id, (void *)flow); - /* Next flow. */ - continue; - } - /* Verify the flow has not already been cleaned. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!flow->frxq[i].ibv_flow) - continue; - /* - * Indirection table may be necessary to remove the - * flags in the Rx queues. - * This helps to speed-up the process by avoiding - * another loop. - */ - ind_tbl = flow->frxq[i].hrxq->ind_table; - break; - } - if (i == hash_rxq_init_n) - return; - if (flow->mark) { - assert(ind_tbl); - for (i = 0; i != ind_tbl->queues_n; ++i) - (*priv->rxqs)[ind_tbl->queues[i]]->mark = 0; - } - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!flow->frxq[i].ibv_flow) - continue; - claim_zero(mlx5_glue->destroy_flow - (flow->frxq[i].ibv_flow)); - flow->frxq[i].ibv_flow = NULL; - mlx5_hrxq_release(dev, flow->frxq[i].hrxq); - flow->frxq[i].hrxq = NULL; - } - DRV_LOG(DEBUG, "port %u flow %p removed", dev->data->port_id, - (void *)flow); - } - /* Cleanup Rx queue tunnel info. */ - for (i = 0; i != priv->rxqs_n; ++i) { - struct mlx5_rxq_data *q = (*priv->rxqs)[i]; - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of(q, struct mlx5_rxq_ctrl, rxq); - - if (!q) - continue; - memset((void *)rxq_ctrl->tunnel_types, 0, - sizeof(rxq_ctrl->tunnel_types)); - q->tunnel = 0; - } } /** @@ -2896,76 +236,9 @@ mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_start(struct rte_eth_dev *dev, struct mlx5_flows *list) +mlx5_flow_start(struct rte_eth_dev *dev __rte_unused, + struct mlx5_flows *list __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct rte_flow *flow; - - TAILQ_FOREACH(flow, list, next) { - unsigned int i; - - if (flow->drop) { - flow->frxq[HASH_RXQ_ETH].ibv_flow = - mlx5_glue->create_flow - (priv->flow_drop_queue->qp, - flow->frxq[HASH_RXQ_ETH].ibv_attr); - if (!flow->frxq[HASH_RXQ_ETH].ibv_flow) { - DRV_LOG(DEBUG, - "port %u flow %p cannot be applied", - dev->data->port_id, (void *)flow); - rte_errno = EINVAL; - return -rte_errno; - } - DRV_LOG(DEBUG, "port %u flow %p applied", - dev->data->port_id, (void *)flow); - /* Next flow. */ - continue; - } - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!flow->frxq[i].ibv_attr) - continue; - flow->frxq[i].hrxq = - mlx5_hrxq_get(dev, flow->rss_conf.key, - flow->rss_conf.key_len, - flow->frxq[i].hash_fields, - flow->rss_conf.queue, - flow->rss_conf.queue_num, - flow->tunnel, - flow->rss_conf.level); - if (flow->frxq[i].hrxq) - goto flow_create; - flow->frxq[i].hrxq = - mlx5_hrxq_new(dev, flow->rss_conf.key, - flow->rss_conf.key_len, - flow->frxq[i].hash_fields, - flow->rss_conf.queue, - flow->rss_conf.queue_num, - flow->tunnel, - flow->rss_conf.level); - if (!flow->frxq[i].hrxq) { - DRV_LOG(DEBUG, - "port %u flow %p cannot create hash" - " rxq", - dev->data->port_id, (void *)flow); - rte_errno = EINVAL; - return -rte_errno; - } -flow_create: - mlx5_flow_dump(dev, flow, i); - flow->frxq[i].ibv_flow = - mlx5_glue->create_flow(flow->frxq[i].hrxq->qp, - flow->frxq[i].ibv_attr); - if (!flow->frxq[i].ibv_flow) { - DRV_LOG(DEBUG, - "port %u flow %p type %u cannot be" - " applied", - dev->data->port_id, (void *)flow, i); - rte_errno = EINVAL; - return -rte_errno; - } - } - mlx5_flow_create_update_rxqs(dev, flow); - } return 0; } @@ -3019,7 +292,6 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_attr attr = { .ingress = 1, - .priority = MLX5_CTRL_FLOW_PRIORITY, }; struct rte_flow_item items[] = { { @@ -3129,83 +401,6 @@ mlx5_flow_flush(struct rte_eth_dev *dev, return 0; } -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT -/** - * Query flow counter. - * - * @param cs - * the counter set. - * @param counter_value - * returned data from the counter. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_query_count(struct ibv_counter_set *cs, - struct mlx5_flow_counter_stats *counter_stats, - struct rte_flow_query_count *query_count, - struct rte_flow_error *error) -{ - uint64_t counters[2]; - struct ibv_query_counter_set_attr query_cs_attr = { - .cs = cs, - .query_flags = IBV_COUNTER_SET_FORCE_UPDATE, - }; - struct ibv_counter_set_data query_out = { - .out = counters, - .outlen = 2 * sizeof(uint64_t), - }; - int err = mlx5_glue->query_counter_set(&query_cs_attr, &query_out); - - if (err) - return rte_flow_error_set(error, err, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot read counter"); - query_count->hits_set = 1; - query_count->bytes_set = 1; - query_count->hits = counters[0] - counter_stats->hits; - query_count->bytes = counters[1] - counter_stats->bytes; - if (query_count->reset) { - counter_stats->hits = counters[0]; - counter_stats->bytes = counters[1]; - } - return 0; -} - -/** - * Query a flows. - * - * @see rte_flow_query() - * @see rte_flow_ops - */ -int -mlx5_flow_query(struct rte_eth_dev *dev __rte_unused, - struct rte_flow *flow, - const struct rte_flow_action *action __rte_unused, - void *data, - struct rte_flow_error *error) -{ - if (flow->cs) { - int ret; - - ret = mlx5_flow_query_count(flow->cs, - &flow->counter_stats, - (struct rte_flow_query_count *)data, - error); - if (ret) - return ret; - } else { - return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "no counter found for flow"); - } - return 0; -} -#endif - /** * Isolated mode. * @@ -3445,18 +640,11 @@ mlx5_fdir_filter_add(struct rte_eth_dev *dev, .type = 0, }, }; - struct mlx5_flow_parse parser = { - .layer = HASH_RXQ_ETH, - }; struct rte_flow_error error; struct rte_flow *flow; int ret; ret = mlx5_fdir_filter_convert(dev, fdir_filter, &attributes); - if (ret) - return ret; - ret = mlx5_flow_convert(dev, &attributes.attr, attributes.items, - attributes.actions, &error, &parser); if (ret) return ret; flow = mlx5_flow_list_create(dev, &priv->flows, &attributes.attr, @@ -3482,96 +670,11 @@ mlx5_fdir_filter_add(struct rte_eth_dev *dev, * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -mlx5_fdir_filter_delete(struct rte_eth_dev *dev, - const struct rte_eth_fdir_filter *fdir_filter) +mlx5_fdir_filter_delete(struct rte_eth_dev *dev __rte_unused, + const struct rte_eth_fdir_filter *fdir_filter + __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct mlx5_fdir attributes = { - .attr.group = 0, - }; - struct mlx5_flow_parse parser = { - .create = 1, - .layer = HASH_RXQ_ETH, - }; - struct rte_flow_error error; - struct rte_flow *flow; - unsigned int i; - int ret; - - ret = mlx5_fdir_filter_convert(dev, fdir_filter, &attributes); - if (ret) - return ret; - ret = mlx5_flow_convert(dev, &attributes.attr, attributes.items, - attributes.actions, &error, &parser); - if (ret) - goto exit; - /* - * Special case for drop action which is only set in the - * specifications when the flow is created. In this situation the - * drop specification is missing. - */ - if (parser.drop) { - struct ibv_flow_spec_action_drop *drop; - - drop = (void *)((uintptr_t)parser.queue[HASH_RXQ_ETH].ibv_attr + - parser.queue[HASH_RXQ_ETH].offset); - *drop = (struct ibv_flow_spec_action_drop){ - .type = IBV_FLOW_SPEC_ACTION_DROP, - .size = sizeof(struct ibv_flow_spec_action_drop), - }; - parser.queue[HASH_RXQ_ETH].ibv_attr->num_of_specs++; - } - TAILQ_FOREACH(flow, &priv->flows, next) { - struct ibv_flow_attr *attr; - struct ibv_spec_header *attr_h; - void *spec; - struct ibv_flow_attr *flow_attr; - struct ibv_spec_header *flow_h; - void *flow_spec; - unsigned int specs_n; - unsigned int queue_id = parser.drop ? HASH_RXQ_ETH : - parser.layer; - - attr = parser.queue[queue_id].ibv_attr; - flow_attr = flow->frxq[queue_id].ibv_attr; - /* Compare first the attributes. */ - if (!flow_attr || - memcmp(attr, flow_attr, sizeof(struct ibv_flow_attr))) - continue; - if (attr->num_of_specs == 0) - continue; - spec = (void *)((uintptr_t)attr + - sizeof(struct ibv_flow_attr)); - flow_spec = (void *)((uintptr_t)flow_attr + - sizeof(struct ibv_flow_attr)); - specs_n = RTE_MIN(attr->num_of_specs, flow_attr->num_of_specs); - for (i = 0; i != specs_n; ++i) { - attr_h = spec; - flow_h = flow_spec; - if (memcmp(spec, flow_spec, - RTE_MIN(attr_h->size, flow_h->size))) - goto wrong_flow; - spec = (void *)((uintptr_t)spec + attr_h->size); - flow_spec = (void *)((uintptr_t)flow_spec + - flow_h->size); - } - /* At this point, the flow match. */ - break; -wrong_flow: - /* The flow does not match. */ - continue; - } - if (flow) - mlx5_flow_list_destroy(dev, &priv->flows, flow); -exit: - if (ret) - ret = rte_errno; /* Save rte_errno before cleanup. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser.queue[i].ibv_attr) - rte_free(parser.queue[i].ibv_attr); - } - if (ret) - rte_errno = ret; /* Restore rte_errno. */ + rte_errno = ENOTSUP; return -rte_errno; } @@ -3738,45 +841,7 @@ mlx5_dev_filter_ctrl(struct rte_eth_dev *dev, * number of supported Verbs flow priority. */ unsigned int -mlx5_get_max_verbs_prio(struct rte_eth_dev *dev) +mlx5_get_max_verbs_prio(struct rte_eth_dev *dev __rte_unused) { - struct priv *priv = dev->data->dev_private; - unsigned int verb_priorities = MLX5_VERBS_FLOW_PRIO_8; - struct { - struct ibv_flow_attr attr; - struct ibv_flow_spec_eth eth; - struct ibv_flow_spec_action_drop drop; - } flow_attr = { - .attr = { - .num_of_specs = 2, - }, - .eth = { - .type = IBV_FLOW_SPEC_ETH, - .size = sizeof(struct ibv_flow_spec_eth), - }, - .drop = { - .size = sizeof(struct ibv_flow_spec_action_drop), - .type = IBV_FLOW_SPEC_ACTION_DROP, - }, - }; - struct ibv_flow *flow; - - do { - flow_attr.attr.priority = verb_priorities - 1; - flow = mlx5_glue->create_flow(priv->flow_drop_queue->qp, - &flow_attr.attr); - if (flow) { - claim_zero(mlx5_glue->destroy_flow(flow)); - /* Try more priorities. */ - verb_priorities *= 2; - } else { - /* Failed, restore last right number. */ - verb_priorities /= 2; - break; - } - } while (1); - DRV_LOG(DEBUG, "port %u Verbs flow priorities: %d," - " user flow priorities: %d", - dev->data->port_id, verb_priorities, MLX5_CTRL_FLOW_PRIORITY); - return verb_priorities; + return 8; } diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 0007be08b..97b4d9eb6 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -136,7 +136,6 @@ struct mlx5_rxq_ctrl { struct priv *priv; /* Back pointer to private data. */ struct mlx5_rxq_data rxq; /* Data path structure. */ unsigned int socket; /* CPU socket ID for allocations. */ - uint32_t tunnel_types[16]; /* Tunnel type counter. */ unsigned int irq:1; /* Whether IRQ is enabled. */ uint16_t idx; /* Queue index. */ }; From patchwork Wed Jul 11 07:22:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42794 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A08EE1B471; Wed, 11 Jul 2018 09:23:20 +0200 (CEST) Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by dpdk.org (Postfix) with ESMTP id 58C251B459 for ; Wed, 11 Jul 2018 09:23:19 +0200 (CEST) Received: by mail-wr1-f67.google.com with SMTP id b15-v6so17029672wrv.10 for ; Wed, 11 Jul 2018 00:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9QrB3D6rQEj2UXsjVsZp8Rc98kaLh98v2f/ioKj6NtY=; b=mb7wxJU8/1puPgobJk3FIhXEV8wG8pxF+tugRq2aDBSZFqldqtSwgc2TyJg14Mw27g Y3nM06mhX4K9Pi2qw/eNZ/cixrCG2KOQ6+oeF4r/Th40I8uTomR8vNmiwCMf0sKey8Lt 1pxKQ/GExOJoldvD55QxZ6eiycGDY5yNzbsxXnV6Qs4MiU4/IWfpOeXowkL1sAkXD8MY TfZPTmIdMYrNhM/8yI2R9/cEvF0xs6zFjD2UYwuI31kD48v7+YJ8JWcPo3jKZ3v1X0M1 u0ciAuaRhVzL5UN7nLDhE6raGpNdUS1g2LwNSx7a+7WIqkspLzUh0xZUWlwRC4gsO4XP 8FOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9QrB3D6rQEj2UXsjVsZp8Rc98kaLh98v2f/ioKj6NtY=; b=fyvi1Zc7iZ7FUNr/ZXlgGX8rEPnaI/wndDxV0XKHJoHOLUwkbHnLb0hZEn11CoCFK4 rrl1X8nkgh6msDKpVZgVPDCKGYgZLtJ/Xbb0aU+dqWR6kQGx7sXRtIuP7U5xWVALFfgS PWPtCHOXbFGFuFcqqi0GBA4olGZlWFBEh6WJcv29lPksweUqVwiPqEHU8SMPyHMvtz+6 7iUjUOvIoIeiXrLUH04iwZe1f0k8bUXB2DACVHG+YDqRVbA1odPsZRUmjasF+3uPJGJQ WXuzOaRkgirD4bQnyBqucBaArC6Z2HPwAjs3uDwPGs1milVZg+RXd3QDBX/0jP9xcch0 /HUQ== X-Gm-Message-State: APt69E14/g21co9T1LP/Bxk2MlzqTW+nli4oVIzwN4ozUnVLjhyswsbs T63UyE8Z7wxYVAOoUioGfZFWPmA2bg== X-Google-Smtp-Source: AAOMgpcx5ofuyLczahRKwdVUPXK0kPRXGUubi7PxZ8nhoGqrnReJ+iQJaa6RUEA7nPymIidEy0P80A== X-Received: by 2002:adf:8e10:: with SMTP id n16-v6mr19946913wrb.100.1531293798644; Wed, 11 Jul 2018 00:23:18 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:18 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:35 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 02/21] net/mlx5: handle drop queues as regular queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Drop queues are essentially used in flows due to Verbs API, the information if the fate of the flow is a drop or not is already present in the flow. Due to this, drop queues can be fully mapped on regular queues. Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5.c | 24 ++-- drivers/net/mlx5/mlx5.h | 14 ++- drivers/net/mlx5/mlx5_flow.c | 94 +++++++------- drivers/net/mlx5/mlx5_rxq.c | 232 +++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 6 + 5 files changed, 308 insertions(+), 62 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index df7f39844..e9780ac8f 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -261,7 +261,6 @@ mlx5_dev_close(struct rte_eth_dev *dev) priv->txqs_n = 0; priv->txqs = NULL; } - mlx5_flow_delete_drop_queue(dev); mlx5_mprq_free_mp(dev); mlx5_mr_release(dev); if (priv->pd != NULL) { @@ -1139,22 +1138,15 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, mlx5_link_update(eth_dev, 0); /* Store device configuration on private structure. */ priv->config = config; - /* Create drop queue. */ - err = mlx5_flow_create_drop_queue(eth_dev); - if (err) { - DRV_LOG(ERR, "port %u drop queue allocation failed: %s", - eth_dev->data->port_id, strerror(rte_errno)); - err = rte_errno; - goto error; - } /* Supported Verbs flow priority number detection. */ - if (verb_priorities == 0) - verb_priorities = mlx5_get_max_verbs_prio(eth_dev); - if (verb_priorities < MLX5_VERBS_FLOW_PRIO_8) { - DRV_LOG(ERR, "port %u wrong Verbs flow priorities: %u", - eth_dev->data->port_id, verb_priorities); - err = ENOTSUP; - goto error; + if (verb_priorities == 0) { + err = mlx5_verbs_max_prio(eth_dev); + if (err < 0) { + DRV_LOG(ERR, "port %u wrong Verbs flow priorities", + eth_dev->data->port_id); + goto error; + } + verb_priorities = err; } priv->config.max_verbs_prio = verb_priorities; /* diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index cc01310e0..227429848 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -139,9 +139,6 @@ enum mlx5_verbs_alloc_type { MLX5_VERBS_ALLOC_TYPE_RX_QUEUE, }; -/* 8 Verbs priorities. */ -#define MLX5_VERBS_FLOW_PRIO_8 8 - /** * Verbs allocator needs a context to know in the callback which kind of * resources it is allocating. @@ -153,6 +150,12 @@ struct mlx5_verbs_alloc_ctx { LIST_HEAD(mlx5_mr_list, mlx5_mr); +/* Flow drop context necessary due to Verbs API. */ +struct mlx5_drop { + struct mlx5_hrxq *hrxq; /* Hash Rx queue queue. */ + struct mlx5_rxq_ibv *rxq; /* Verbs Rx queue. */ +}; + struct priv { LIST_ENTRY(priv) mem_event_cb; /* Called by memory event callback. */ struct rte_eth_dev_data *dev_data; /* Pointer to device data. */ @@ -182,7 +185,7 @@ struct priv { struct rte_intr_handle intr_handle; /* Interrupt handler. */ unsigned int (*reta_idx)[]; /* RETA index table. */ unsigned int reta_idx_n; /* RETA index size. */ - struct mlx5_hrxq_drop *flow_drop_queue; /* Flow drop queue. */ + struct mlx5_drop drop_queue; /* Flow drop queues. */ struct mlx5_flows flows; /* RTE Flow rules. */ struct mlx5_flows ctrl_flows; /* Control flow rules. */ struct { @@ -314,7 +317,8 @@ int mlx5_traffic_restart(struct rte_eth_dev *dev); /* mlx5_flow.c */ -unsigned int mlx5_get_max_verbs_prio(struct rte_eth_dev *dev); +int mlx5_verbs_max_prio(struct rte_eth_dev *dev); +void mlx5_flow_print(struct rte_flow *flow); int mlx5_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, const struct rte_flow_item items[], diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index a45cb06e1..5e325be37 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -75,6 +75,58 @@ struct ibv_spec_header { uint16_t size; }; + /** + * Get the maximum number of priority available. + * + * @param[in] dev + * Pointer to Ethernet device. + * + * @return + * number of supported Verbs flow priority on success, a negative errno + * value otherwise and rte_errno is set. + */ +int +mlx5_verbs_max_prio(struct rte_eth_dev *dev) +{ + struct { + struct ibv_flow_attr attr; + struct ibv_flow_spec_eth eth; + struct ibv_flow_spec_action_drop drop; + } flow_attr = { + .attr = { + .num_of_specs = 2, + }, + .eth = { + .type = IBV_FLOW_SPEC_ETH, + .size = sizeof(struct ibv_flow_spec_eth), + }, + .drop = { + .size = sizeof(struct ibv_flow_spec_action_drop), + .type = IBV_FLOW_SPEC_ACTION_DROP, + }, + }; + struct ibv_flow *flow; + uint32_t verb_priorities; + struct mlx5_hrxq *drop = mlx5_hrxq_drop_new(dev); + + if (!drop) { + rte_errno = ENOTSUP; + return -rte_errno; + } + for (verb_priorities = 0; 1; verb_priorities++) { + flow_attr.attr.priority = verb_priorities; + flow = mlx5_glue->create_flow(drop->qp, + &flow_attr.attr); + if (!flow) + break; + claim_zero(mlx5_glue->destroy_flow(flow)); + } + mlx5_hrxq_drop_release(dev); + DRV_LOG(INFO, "port %u flow maximum priority: %d", + dev->data->port_id, verb_priorities); + return verb_priorities; +} + /** * Convert a flow. * @@ -184,32 +236,6 @@ mlx5_flow_list_flush(struct rte_eth_dev *dev, struct mlx5_flows *list) } } -/** - * Create drop queue. - * - * @param dev - * Pointer to Ethernet device. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -int -mlx5_flow_create_drop_queue(struct rte_eth_dev *dev __rte_unused) -{ - return 0; -} - -/** - * Delete drop queue. - * - * @param dev - * Pointer to Ethernet device. - */ -void -mlx5_flow_delete_drop_queue(struct rte_eth_dev *dev __rte_unused) -{ -} - /** * Remove all flows. * @@ -292,6 +318,7 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_attr attr = { .ingress = 1, + .priority = priv->config.max_verbs_prio - 1, }; struct rte_flow_item items[] = { { @@ -830,18 +857,3 @@ mlx5_dev_filter_ctrl(struct rte_eth_dev *dev, } return 0; } - -/** - * Detect number of Verbs flow priorities supported. - * - * @param dev - * Pointer to Ethernet device. - * - * @return - * number of supported Verbs flow priority. - */ -unsigned int -mlx5_get_max_verbs_prio(struct rte_eth_dev *dev __rte_unused) -{ - return 8; -} diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index fd0df177e..d960daa43 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1957,3 +1957,235 @@ mlx5_hrxq_ibv_verify(struct rte_eth_dev *dev) } return ret; } + +/** + * Create a drop Rx queue Verbs object. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * The Verbs object initialised, NULL otherwise and rte_errno is set. + */ +struct mlx5_rxq_ibv * +mlx5_rxq_ibv_drop_new(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + struct ibv_cq *cq; + struct ibv_wq *wq = NULL; + struct mlx5_rxq_ibv *rxq; + + if (priv->drop_queue.rxq) + return priv->drop_queue.rxq; + cq = mlx5_glue->create_cq(priv->ctx, 1, NULL, NULL, 0); + if (!cq) { + DEBUG("port %u cannot allocate CQ for drop queue", + dev->data->port_id); + rte_errno = errno; + goto error; + } + wq = mlx5_glue->create_wq(priv->ctx, + &(struct ibv_wq_init_attr){ + .wq_type = IBV_WQT_RQ, + .max_wr = 1, + .max_sge = 1, + .pd = priv->pd, + .cq = cq, + }); + if (!wq) { + DEBUG("port %u cannot allocate WQ for drop queue", + dev->data->port_id); + rte_errno = errno; + goto error; + } + rxq = rte_calloc(__func__, 1, sizeof(*rxq), 0); + if (!rxq) { + DEBUG("port %u cannot allocate drop Rx queue memory", + dev->data->port_id); + rte_errno = ENOMEM; + goto error; + } + rxq->cq = cq; + rxq->wq = wq; + priv->drop_queue.rxq = rxq; + return rxq; +error: + if (wq) + claim_zero(mlx5_glue->destroy_wq(wq)); + if (cq) + claim_zero(mlx5_glue->destroy_cq(cq)); + return NULL; +} + +/** + * Release a drop Rx queue Verbs object. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * The Verbs object initialised, NULL otherwise and rte_errno is set. + */ +void +mlx5_rxq_ibv_drop_release(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + struct mlx5_rxq_ibv *rxq = priv->drop_queue.rxq; + + if (rxq->wq) + claim_zero(mlx5_glue->destroy_wq(rxq->wq)); + if (rxq->cq) + claim_zero(mlx5_glue->destroy_cq(rxq->cq)); + rte_free(rxq); + priv->drop_queue.rxq = NULL; +} + +/** + * Create a drop indirection table. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * The Verbs object initialised, NULL otherwise and rte_errno is set. + */ +struct mlx5_ind_table_ibv * +mlx5_ind_table_ibv_drop_new(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + struct mlx5_ind_table_ibv *ind_tbl; + struct mlx5_rxq_ibv *rxq; + struct mlx5_ind_table_ibv tmpl; + + rxq = mlx5_rxq_ibv_drop_new(dev); + if (!rxq) + return NULL; + tmpl.ind_table = mlx5_glue->create_rwq_ind_table + (priv->ctx, + &(struct ibv_rwq_ind_table_init_attr){ + .log_ind_tbl_size = 0, + .ind_tbl = &rxq->wq, + .comp_mask = 0, + }); + if (!tmpl.ind_table) { + DEBUG("port %u cannot allocate indirection table for drop" + " queue", + dev->data->port_id); + rte_errno = errno; + goto error; + } + ind_tbl = rte_calloc(__func__, 1, sizeof(*ind_tbl), 0); + if (!ind_tbl) { + rte_errno = ENOMEM; + goto error; + } + ind_tbl->ind_table = tmpl.ind_table; + return ind_tbl; +error: + mlx5_rxq_ibv_drop_release(dev); + return NULL; +} + +/** + * Release a drop indirection table. + * + * @param dev + * Pointer to Ethernet device. + */ +void +mlx5_ind_table_ibv_drop_release(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + struct mlx5_ind_table_ibv *ind_tbl = priv->drop_queue.hrxq->ind_table; + + claim_zero(mlx5_glue->destroy_rwq_ind_table(ind_tbl->ind_table)); + mlx5_rxq_ibv_drop_release(dev); + rte_free(ind_tbl); + priv->drop_queue.hrxq->ind_table = NULL; +} + +/** + * Create a drop Rx Hash queue. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * The Verbs object initialised, NULL otherwise and rte_errno is set. + */ +struct mlx5_hrxq * +mlx5_hrxq_drop_new(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + struct mlx5_ind_table_ibv *ind_tbl; + struct ibv_qp *qp; + struct mlx5_hrxq *hrxq; + + if (priv->drop_queue.hrxq) { + rte_atomic32_inc(&priv->drop_queue.hrxq->refcnt); + return priv->drop_queue.hrxq; + } + ind_tbl = mlx5_ind_table_ibv_drop_new(dev); + if (!ind_tbl) + return NULL; + qp = mlx5_glue->create_qp_ex(priv->ctx, + &(struct ibv_qp_init_attr_ex){ + .qp_type = IBV_QPT_RAW_PACKET, + .comp_mask = + IBV_QP_INIT_ATTR_PD | + IBV_QP_INIT_ATTR_IND_TABLE | + IBV_QP_INIT_ATTR_RX_HASH, + .rx_hash_conf = (struct ibv_rx_hash_conf){ + .rx_hash_function = + IBV_RX_HASH_FUNC_TOEPLITZ, + .rx_hash_key_len = rss_hash_default_key_len, + .rx_hash_key = rss_hash_default_key, + .rx_hash_fields_mask = 0, + }, + .rwq_ind_tbl = ind_tbl->ind_table, + .pd = priv->pd + }); + if (!qp) { + DEBUG("port %u cannot allocate QP for drop queue", + dev->data->port_id); + rte_errno = errno; + goto error; + } + hrxq = rte_calloc(__func__, 1, sizeof(*hrxq), 0); + if (!hrxq) { + DRV_LOG(WARNING, + "port %u cannot allocate memory for drop queue", + dev->data->port_id); + rte_errno = ENOMEM; + goto error; + } + hrxq->ind_table = ind_tbl; + hrxq->qp = qp; + priv->drop_queue.hrxq = hrxq; + rte_atomic32_set(&hrxq->refcnt, 1); + return hrxq; +error: + if (ind_tbl) + mlx5_ind_table_ibv_drop_release(dev); + return NULL; +} + +/** + * Release a drop hash Rx queue. + * + * @param dev + * Pointer to Ethernet device. + */ +void +mlx5_hrxq_drop_release(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + struct mlx5_hrxq *hrxq = priv->drop_queue.hrxq; + + if (rte_atomic32_dec_and_test(&hrxq->refcnt)) { + claim_zero(mlx5_glue->destroy_qp(hrxq->qp)); + mlx5_ind_table_ibv_drop_release(dev); + rte_free(hrxq); + priv->drop_queue.hrxq = NULL; + } +} diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 97b4d9eb6..708fdd4fa 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -244,6 +244,8 @@ struct mlx5_rxq_ibv *mlx5_rxq_ibv_new(struct rte_eth_dev *dev, uint16_t idx); struct mlx5_rxq_ibv *mlx5_rxq_ibv_get(struct rte_eth_dev *dev, uint16_t idx); int mlx5_rxq_ibv_release(struct mlx5_rxq_ibv *rxq_ibv); int mlx5_rxq_ibv_releasable(struct mlx5_rxq_ibv *rxq_ibv); +struct mlx5_rxq_ibv *mlx5_rxq_ibv_drop_new(struct rte_eth_dev *dev); +void mlx5_rxq_ibv_drop_release(struct rte_eth_dev *dev); int mlx5_rxq_ibv_verify(struct rte_eth_dev *dev); struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, unsigned int socket, @@ -264,6 +266,8 @@ struct mlx5_ind_table_ibv *mlx5_ind_table_ibv_get(struct rte_eth_dev *dev, int mlx5_ind_table_ibv_release(struct rte_eth_dev *dev, struct mlx5_ind_table_ibv *ind_tbl); int mlx5_ind_table_ibv_verify(struct rte_eth_dev *dev); +struct mlx5_ind_table_ibv *mlx5_ind_table_ibv_drop_new(struct rte_eth_dev *dev); +void mlx5_ind_table_ibv_drop_release(struct rte_eth_dev *dev); struct mlx5_hrxq *mlx5_hrxq_new(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, @@ -276,6 +280,8 @@ struct mlx5_hrxq *mlx5_hrxq_get(struct rte_eth_dev *dev, uint32_t tunnel, uint32_t rss_level); int mlx5_hrxq_release(struct rte_eth_dev *dev, struct mlx5_hrxq *hxrq); int mlx5_hrxq_ibv_verify(struct rte_eth_dev *dev); +struct mlx5_hrxq *mlx5_hrxq_drop_new(struct rte_eth_dev *dev); +void mlx5_hrxq_drop_release(struct rte_eth_dev *dev); uint64_t mlx5_get_rx_port_offloads(void); uint64_t mlx5_get_rx_queue_offloads(struct rte_eth_dev *dev); From patchwork Wed Jul 11 07:22:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42796 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A4E3F1B466; Wed, 11 Jul 2018 09:23:27 +0200 (CEST) Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by dpdk.org (Postfix) with ESMTP id D56C81B484 for ; Wed, 11 Jul 2018 09:23:20 +0200 (CEST) Received: by mail-wr1-f67.google.com with SMTP id c13-v6so16974327wrt.1 for ; Wed, 11 Jul 2018 00:23:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qjBo8v3AxKvoiPxec9Q3pQKBaqGYlE1KG6Z5NSIWHgI=; b=RJrvCjcmK6rz5w/Zp9w3O8OrkYS2FAr5GQUPmUclsndNy/jbOnwO11phfyaIryIdt1 Pm6hQtNwYm/PTfEkZ0OU0AxN3Q3gycDiMT0kIloWVZmRXqq/W+nU0dukrdt8xOYI9mCv U9nDEzB0Q7aazCN2iDKlg7DWHFamRc1D+5rss0oH8VmkylPwZK8avK+mkZsaNgX+C0TQ /uKSqbKwHhSIiM/i+KkyYzy1zx7PyBPxs0HKk4SRkAzGclObnBterhagWudkBBNED8yd cRzcemS3iZ5Pfs1XCNWlIT18v+ybLBj/sPsuFzaNJoBnLgY6pIM24CDTnFDFt7StV99O GKHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qjBo8v3AxKvoiPxec9Q3pQKBaqGYlE1KG6Z5NSIWHgI=; b=KvMy7CK3+i4FtZqJqdDLyP6qU1GLZZeUBJuZx8xeNGutnaKNZAWKJ6Ad0fnbXkL910 F6b+q4QYm2xdU9d0SNGBd3AdJCy4nnO9JNxTlMn53GHOO/Irs63sCPi436gMGT5/563j 5t8OChZ+M5+GM0Wxdb7jv+/msnqLaE5C8CLmi5XUYE1kVH2LKxH6gApvMQ5pH8fDHu7Y oaGRME78umLZyj7C+Ge4z3MIAbD+k/eGIomXN6ct6uNtquiRmxSTgVZJpYYvFCxeYQa2 nlQsfUA0JM5zECqSNnHxAwfg4HXWUX3vqUJGswhz/OV3vxbaudGk4n9j2KgrWeUu/eX0 ipCw== X-Gm-Message-State: APt69E0dzZEm8x9FH5hyslLG7eLvABWtgXrIgdvq3kMjVovCvsu7skrn ZASKlVgywuz/TRBKDnNE4UKWOLyfKA== X-Google-Smtp-Source: AAOMgpdfWmX2q7iahavop6rbdCKNHS/5HVCSlOAsZhfpIkkaE198ImLKFZzNmC68kdvxdN4Dozs2Iw== X-Received: by 2002:a5d:4e49:: with SMTP id r9-v6mr18529439wrt.27.1531293800446; Wed, 11 Jul 2018 00:23:20 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:18 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:36 +0200 Message-Id: <03092689aa33878a03193348a69bfbf0aeb091dd.1531293415.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 03/21] net/mlx5: replace verbs priorities by flow X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previous work introduce verbs priorities, whereas the PMD is making translation between Flow priority into Verbs. Rename this to make more sense on what the PMD has to translate. Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5.c | 15 ++++----------- drivers/net/mlx5/mlx5.h | 4 ++-- drivers/net/mlx5/mlx5_flow.c | 24 ++++++++++++------------ 3 files changed, 18 insertions(+), 25 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index e9780ac8f..74248f098 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -717,7 +717,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, unsigned int tunnel_en = 0; unsigned int mpls_en = 0; unsigned int swp = 0; - unsigned int verb_priorities = 0; unsigned int mprq = 0; unsigned int mprq_min_stride_size_n = 0; unsigned int mprq_max_stride_size_n = 0; @@ -1139,16 +1138,10 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, /* Store device configuration on private structure. */ priv->config = config; /* Supported Verbs flow priority number detection. */ - if (verb_priorities == 0) { - err = mlx5_verbs_max_prio(eth_dev); - if (err < 0) { - DRV_LOG(ERR, "port %u wrong Verbs flow priorities", - eth_dev->data->port_id); - goto error; - } - verb_priorities = err; - } - priv->config.max_verbs_prio = verb_priorities; + err = mlx5_flow_discover_priorities(eth_dev); + if (err < 0) + goto error; + priv->config.flow_prio = err; /* * Once the device is added to the list of memory event * callback, its global MR cache table cannot be expanded diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 227429848..9949cd3fa 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -122,7 +122,7 @@ struct mlx5_dev_config { unsigned int min_rxqs_num; /* Rx queue count threshold to enable MPRQ. */ } mprq; /* Configurations for Multi-Packet RQ. */ - unsigned int max_verbs_prio; /* Number of Verb flow priorities. */ + unsigned int flow_prio; /* Number of flow priorities. */ unsigned int tso_max_payload_sz; /* Maximum TCP payload for TSO. */ unsigned int ind_table_max_size; /* Maximum indirection table size. */ int txq_inline; /* Maximum packet size for inlining. */ @@ -317,7 +317,7 @@ int mlx5_traffic_restart(struct rte_eth_dev *dev); /* mlx5_flow.c */ -int mlx5_verbs_max_prio(struct rte_eth_dev *dev); +int mlx5_flow_discover_priorities(struct rte_eth_dev *dev); void mlx5_flow_print(struct rte_flow *flow); int mlx5_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 5e325be37..8fdc6d7bb 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -76,17 +76,17 @@ struct ibv_spec_header { }; /** - * Get the maximum number of priority available. + * Discover the maximum number of priority available. * * @param[in] dev * Pointer to Ethernet device. * * @return - * number of supported Verbs flow priority on success, a negative errno - * value otherwise and rte_errno is set. + * number of supported flow priority on success, a negative errno value + * otherwise and rte_errno is set. */ int -mlx5_verbs_max_prio(struct rte_eth_dev *dev) +mlx5_flow_discover_priorities(struct rte_eth_dev *dev) { struct { struct ibv_flow_attr attr; @@ -106,25 +106,25 @@ mlx5_verbs_max_prio(struct rte_eth_dev *dev) }, }; struct ibv_flow *flow; - uint32_t verb_priorities; struct mlx5_hrxq *drop = mlx5_hrxq_drop_new(dev); + uint16_t vprio[] = { 8, 16 }; + int i; if (!drop) { rte_errno = ENOTSUP; return -rte_errno; } - for (verb_priorities = 0; 1; verb_priorities++) { - flow_attr.attr.priority = verb_priorities; - flow = mlx5_glue->create_flow(drop->qp, - &flow_attr.attr); + for (i = 0; i != RTE_DIM(vprio); i++) { + flow_attr.attr.priority = vprio[i] - 1; + flow = mlx5_glue->create_flow(drop->qp, &flow_attr.attr); if (!flow) break; claim_zero(mlx5_glue->destroy_flow(flow)); } mlx5_hrxq_drop_release(dev); DRV_LOG(INFO, "port %u flow maximum priority: %d", - dev->data->port_id, verb_priorities); - return verb_priorities; + dev->data->port_id, vprio[i - 1]); + return vprio[i - 1]; } /** @@ -318,7 +318,7 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_attr attr = { .ingress = 1, - .priority = priv->config.max_verbs_prio - 1, + .priority = priv->config.flow_prio - 1, }; struct rte_flow_item items[] = { { From patchwork Wed Jul 11 07:22:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42797 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B8F2E1B4E3; Wed, 11 Jul 2018 09:23:30 +0200 (CEST) Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by dpdk.org (Postfix) with ESMTP id 148041B487 for ; Wed, 11 Jul 2018 09:23:22 +0200 (CEST) Received: by mail-wr1-f68.google.com with SMTP id a3-v6so7777292wrt.2 for ; Wed, 11 Jul 2018 00:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=zhq5AmuXd0NsEptDfY53xmuFUBtjTlIqpdcpXHTrPOo=; b=uCCZ2st/MOExzrz9pw4kob5xj4AmDT+fj4h8y7a6dbSv6qs2iv+GwoqoWS0IGVuzu7 /POwajE2Tm91uGDjEde6YGKwOu9s0kPX6kN9NylYnGCHYXe3A0+dAo3qoU67chb6J/Yo HThp0Tx9aNQRn3hvAIHV4w34IjuKDFdPKaVxBFyzN7j5edLxXiqQ+TLQhUzHAdhWDik3 8886V3NNYOIqeHZqR5oUKHhGy5i/F2zrovNkAlHTJ/o5WST/UALGk9NX5py49SdvvUI0 kR4XzV/QrzC0SU4BP1ZsjzRkhY/JmaFAWGMK093j33+iUejdC/MJkQtZa5Jpojylnyf6 0nKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=zhq5AmuXd0NsEptDfY53xmuFUBtjTlIqpdcpXHTrPOo=; b=laUNxmLlp4dmFOQMNYzOpNIZJLr+BekKdZMhN4DSFmabHcTD09PPWiAe43pycX9A4s kA11e4DdXb4PdMn+DtYshDAK6BAUtQAXQdqbkf7XpnwhjuhbCgFRbKjzFydh5ItmS5Ct KmdEsKUvPbg2OZleuny7k39bt1x0rCLIGQS8KPQUwkMqZpYukecAVn+lIiFh4Hxu0G00 4IYPuWtzVlQ9Ao5DSZ+tfAnLGZj9aMEBYnY5CFk0MWCOsnrft/KSH1w2ZGqiFqxFGklY QolyHx7xM4hvf7ppFBBVkg9YubfPmDduVEGI1B3AwiXB1Et+MKDxbrVq3ObqBtgQaAqK yi2Q== X-Gm-Message-State: AOUpUlG0KsFIIpSIK3pWBvPOw2fql2xaxR/MLoaouWdMGRZQx/CqX+Vk +R8Q8iDQaS6WMqxsFc5X9wweawhgKg== X-Google-Smtp-Source: AAOMgpcGTpajyP0hGjuFJa3oU9ZRtF+w56ZLcnDPqV3WiEQM/qJA/vFSqvPXnfwJJYK3wfbSRm5sYg== X-Received: by 2002:adf:ed41:: with SMTP id u1-v6mr8071304wro.262.1531293801242; Wed, 11 Jul 2018 00:23:21 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:20 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:37 +0200 Message-Id: <8065e341cc1fb95224302530e3ac08dfa6b8de2a.1531293415.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 04/21] net/mlx5: support flow Ethernet item along with drop action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5.c | 1 + drivers/net/mlx5/mlx5_flow.c | 664 +++++++++++++++++++++++++++++++++-- 2 files changed, 627 insertions(+), 38 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 74248f098..6d3421fae 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -242,6 +242,7 @@ mlx5_dev_close(struct rte_eth_dev *dev) /* In case mlx5_dev_stop() has not been called. */ mlx5_dev_interrupt_handler_uninstall(dev); mlx5_traffic_disable(dev); + mlx5_flow_flush(dev, NULL); /* Prevent crashes when queues are still in use. */ dev->rx_pkt_burst = removed_rx_burst; dev->tx_pkt_burst = removed_tx_burst; diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 8fdc6d7bb..036a8d440 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -35,11 +35,50 @@ extern const struct eth_dev_ops mlx5_dev_ops; extern const struct eth_dev_ops mlx5_dev_ops_isolate; +/* Pattern Layer bits. */ +#define MLX5_FLOW_LAYER_OUTER_L2 (1u << 0) +#define MLX5_FLOW_LAYER_OUTER_L3_IPV4 (1u << 1) +#define MLX5_FLOW_LAYER_OUTER_L3_IPV6 (1u << 2) +#define MLX5_FLOW_LAYER_OUTER_L4_UDP (1u << 3) +#define MLX5_FLOW_LAYER_OUTER_L4_TCP (1u << 4) +#define MLX5_FLOW_LAYER_OUTER_VLAN (1u << 5) +/* Masks. */ +#define MLX5_FLOW_LAYER_OUTER_L3 \ + (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6) +#define MLX5_FLOW_LAYER_OUTER_L4 \ + (MLX5_FLOW_LAYER_OUTER_L4_UDP | MLX5_FLOW_LAYER_OUTER_L4_TCP) + +/* Actions that modify the fate of matching traffic. */ +#define MLX5_FLOW_FATE_DROP (1u << 0) + +/** Handles information leading to a drop fate. */ +struct mlx5_flow_verbs { + unsigned int size; /**< Size of the attribute. */ + struct { + struct ibv_flow_attr *attr; + /**< Pointer to the Specification buffer. */ + uint8_t *specs; /**< Pointer to the specifications. */ + }; + struct ibv_flow *flow; /**< Verbs flow pointer. */ + struct mlx5_hrxq *hrxq; /**< Hash Rx queue object. */ +}; + +/* Flow structure. */ struct rte_flow { TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ + struct rte_flow_attr attributes; /**< User flow attribute. */ + uint32_t layers; + /**< Bit-fields of present layers see MLX5_FLOW_LAYER_*. */ + uint32_t fate; + /**< Bit-fields of present fate see MLX5_FLOW_FATE_*. */ + struct mlx5_flow_verbs verbs; /* Verbs drop flow. */ }; static const struct rte_flow_ops mlx5_flow_ops = { + .validate = mlx5_flow_validate, + .create = mlx5_flow_create, + .destroy = mlx5_flow_destroy, + .flush = mlx5_flow_flush, .isolate = mlx5_flow_isolate, }; @@ -128,13 +167,415 @@ mlx5_flow_discover_priorities(struct rte_eth_dev *dev) } /** - * Convert a flow. + * Verify the @p attributes will be correctly understood by the NIC and store + * them in the @p flow if everything is correct. * - * @param dev + * @param[in] dev * Pointer to Ethernet device. - * @param list - * Pointer to a TAILQ flow list. - * @param[in] attr + * @param[in] attributes + * Pointer to flow attributes + * @param[in, out] flow + * Pointer to the rte_flow structure. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_attributes(struct rte_eth_dev *dev, + const struct rte_flow_attr *attributes, + struct rte_flow *flow, + struct rte_flow_error *error) +{ + uint32_t priority_max = + ((struct priv *)dev->data->dev_private)->config.flow_prio; + + if (attributes->group) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_GROUP, + NULL, + "groups is not supported"); + if (attributes->priority >= priority_max) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, + NULL, + "priority out of range"); + if (attributes->egress) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, + NULL, + "egress is not supported"); + if (attributes->transfer) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER, + NULL, + "transfer is not supported"); + if (!attributes->ingress) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_INGRESS, + NULL, + "ingress attribute is mandatory"); + flow->attributes = *attributes; + return 0; +} + +/** + * Verify the @p item specifications (spec, last, mask) are compatible with the + * NIC capabilities. + * + * @param[in] item + * Item specification. + * @param[in] mask + * @p item->mask or flow default bit-masks. + * @param[in] nic_mask + * Bit-masks covering supported fields by the NIC to compare with user mask. + * @param[in] size + * Bit-masks size in bytes. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_acceptable(const struct rte_flow_item *item, + const uint8_t *mask, + const uint8_t *nic_mask, + unsigned int size, + struct rte_flow_error *error) +{ + unsigned int i; + + assert(nic_mask); + for (i = 0; i < size; ++i) + if ((nic_mask[i] | mask[i]) != nic_mask[i]) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "mask enables non supported" + " bits"); + if (!item->spec && (item->mask || item->last)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "mask/last without a spec is not" + " supported"); + if (item->spec && item->last) { + uint8_t spec[size]; + uint8_t last[size]; + unsigned int i; + int ret; + + for (i = 0; i < size; ++i) { + spec[i] = ((const uint8_t *)item->spec)[i] & mask[i]; + last[i] = ((const uint8_t *)item->last)[i] & mask[i]; + } + ret = memcmp(spec, last, size); + if (ret != 0) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "range is not supported"); + } + return 0; +} + +/** + * Add a verbs specification into @p flow. + * + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] src + * Create specification. + * @param[in] size + * Size in bytes of the specification to copy. + */ +static void +mlx5_flow_spec_verbs_add(struct rte_flow *flow, void *src, unsigned int size) +{ + if (flow->verbs.specs) { + void *dst; + + dst = (void *)(flow->verbs.specs + flow->verbs.size); + memcpy(dst, src, size); + ++flow->verbs.attr->num_of_specs; + } + flow->verbs.size += size; +} + +/** + * Convert the @p item into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_eth *spec = item->spec; + const struct rte_flow_item_eth *mask = item->mask; + const struct rte_flow_item_eth nic_mask = { + .dst.addr_bytes = "\xff\xff\xff\xff\xff\xff", + .src.addr_bytes = "\xff\xff\xff\xff\xff\xff", + .type = RTE_BE16(0xffff), + }; + const unsigned int size = sizeof(struct ibv_flow_spec_eth); + struct ibv_flow_spec_eth eth = { + .type = IBV_FLOW_SPEC_ETH, + .size = size, + }; + int ret; + + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L2) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L2 layers already configured"); + if (!mask) + mask = &rte_flow_item_eth_mask; + ret = mlx5_flow_item_acceptable(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_eth), + error); + if (ret) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L2; + if (size > flow_size) + return size; + if (spec) { + unsigned int i; + + memcpy(ð.val.dst_mac, spec->dst.addr_bytes, ETHER_ADDR_LEN); + memcpy(ð.val.src_mac, spec->src.addr_bytes, ETHER_ADDR_LEN); + eth.val.ether_type = spec->type; + memcpy(ð.mask.dst_mac, mask->dst.addr_bytes, ETHER_ADDR_LEN); + memcpy(ð.mask.src_mac, mask->src.addr_bytes, ETHER_ADDR_LEN); + eth.mask.ether_type = mask->type; + /* Remove unwanted bits from values. */ + for (i = 0; i < ETHER_ADDR_LEN; ++i) { + eth.val.dst_mac[i] &= eth.mask.dst_mac[i]; + eth.val.src_mac[i] &= eth.mask.src_mac[i]; + } + eth.val.ether_type &= eth.mask.ether_type; + } + mlx5_flow_spec_verbs_add(flow, ð, size); + return size; +} + +/** + * Convert the @p pattern into a Verbs specifications after ensuring the NIC + * will understand and process it correctly. + * The conversion is performed item per item, each of them is written into + * the @p flow if its size is lesser or equal to @p flow_size. + * Validation and memory consumption computation are still performed until the + * end of @p pattern, unless an error is encountered. + * + * @param[in] pattern + * Flow pattern. + * @param[in, out] flow + * Pointer to the rte_flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small some + * garbage may be present. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @pattern has fully been + * converted, otherwise another call with this returned memory size should + * be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_items(const struct rte_flow_item pattern[], + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + int remain = flow_size; + size_t size = 0; + + for (; pattern->type != RTE_FLOW_ITEM_TYPE_END; pattern++) { + int ret = 0; + + switch (pattern->type) { + case RTE_FLOW_ITEM_TYPE_VOID: + break; + case RTE_FLOW_ITEM_TYPE_ETH: + ret = mlx5_flow_item_eth(pattern, flow, remain, error); + break; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + pattern, + "item not supported"); + } + if (ret < 0) + return ret; + if (remain > ret) + remain -= ret; + else + remain = 0; + size += ret; + } + if (!flow->layers) { + const struct rte_flow_item item = { + .type = RTE_FLOW_ITEM_TYPE_ETH, + }; + + return mlx5_flow_item_eth(&item, flow, flow_size, error); + } + return size; +} + +/** + * Convert the @p action into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] action + * Action configuration. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p action has fully been + * converted, otherwise another call with this returned memory size should + * be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_action_drop(const struct rte_flow_action *action, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + unsigned int size = sizeof(struct ibv_flow_spec_action_drop); + struct ibv_flow_spec_action_drop drop = { + .type = IBV_FLOW_SPEC_ACTION_DROP, + .size = size, + }; + + if (flow->fate) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "multiple fate actions are not" + " supported"); + if (size < flow_size) + mlx5_flow_spec_verbs_add(flow, &drop, size); + flow->fate |= MLX5_FLOW_FATE_DROP; + return size; +} + +/** + * Convert the @p action into @p flow after ensuring the NIC will understand + * and process it correctly. + * The conversion is performed action per action, each of them is written into + * the @p flow if its size is lesser or equal to @p flow_size. + * Validation and memory consumption computation are still performed until the + * end of @p action, unless an error is encountered. + * + * @param[in] dev + * Pointer to Ethernet device structure. + * @param[in] actions + * Pointer to flow actions array. + * @param[in, out] flow + * Pointer to the rte_flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small some + * garbage may be present. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p actions has fully been + * converted, otherwise another call with this returned memory size should + * be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_actions(struct rte_eth_dev *dev __rte_unused, + const struct rte_flow_action actions[], + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + size_t size = 0; + int remain = flow_size; + int ret = 0; + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_VOID: + break; + case RTE_FLOW_ACTION_TYPE_DROP: + ret = mlx5_flow_action_drop(actions, flow, remain, + error); + break; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "action not supported"); + } + if (ret < 0) + return ret; + if (remain > ret) + remain -= ret; + else + remain = 0; + size += ret; + } + if (!flow->fate) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "no fate action found"); + return size; +} + +/** + * Convert the @p attributes, @p pattern, @p action, into an flow for the NIC + * after ensuring the NIC will understand and process it correctly. + * The conversion is only performed item/action per item/action, each of + * them is written into the @p flow if its size is lesser or equal to @p + * flow_size. + * Validation and memory consumption computation are still performed until the + * end, unless an error is encountered. + * + * @param[in] dev + * Pointer to Ethernet device. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small some + * garbage may be present. + * @param[in] attributes * Flow rule attributes. * @param[in] pattern * Pattern specification (list terminated by the END pattern item). @@ -144,21 +585,42 @@ mlx5_flow_discover_priorities(struct rte_eth_dev *dev) * Perform verbose error reporting if not NULL. * * @return - * A flow on success, NULL otherwise and rte_errno is set. + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the flow has fully been converted and + * can be applied, otherwise another call with this returned memory size + * should be done. + * On error, a negative errno value is returned and rte_errno is set. */ -static struct rte_flow * -mlx5_flow_list_create(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flows *list __rte_unused, - const struct rte_flow_attr *attr __rte_unused, - const struct rte_flow_item items[] __rte_unused, - const struct rte_flow_action actions[] __rte_unused, - struct rte_flow_error *error) +static int +mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, + const size_t flow_size, + const struct rte_flow_attr *attributes, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "action not supported"); - return NULL; + struct rte_flow local_flow = { .layers = 0, }; + size_t size = sizeof(*flow) + sizeof(struct ibv_flow_attr); + int remain = (flow_size > size) ? flow_size - size : 0; + int ret; + + if (!remain) + flow = &local_flow; + ret = mlx5_flow_attributes(dev, attributes, flow, error); + if (ret < 0) + return ret; + ret = mlx5_flow_items(pattern, flow, remain, error); + if (ret < 0) + return ret; + size += ret; + remain = (flow_size > size) ? flow_size - size : 0; + ret = mlx5_flow_actions(dev, actions, flow, remain, error); + if (ret < 0) + return ret; + size += ret; + if (size <= flow_size) + flow->verbs.attr->priority = flow->attributes.priority; + return size; } /** @@ -168,16 +630,142 @@ mlx5_flow_list_create(struct rte_eth_dev *dev __rte_unused, * @see rte_flow_ops */ int -mlx5_flow_validate(struct rte_eth_dev *dev __rte_unused, - const struct rte_flow_attr *attr __rte_unused, - const struct rte_flow_item items[] __rte_unused, - const struct rte_flow_action actions[] __rte_unused, +mlx5_flow_validate(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], struct rte_flow_error *error) { - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "action not supported"); + int ret = mlx5_flow_merge(dev, NULL, 0, attr, items, actions, error); + + if (ret < 0) + return ret; + return 0; +} + +/** + * Remove the flow. + * + * @param[in] dev + * Pointer to Ethernet device. + * @param[in, out] flow + * Pointer to flow structure. + */ +static void +mlx5_flow_remove(struct rte_eth_dev *dev, struct rte_flow *flow) +{ + if (flow->fate & MLX5_FLOW_FATE_DROP) { + if (flow->verbs.flow) { + claim_zero(mlx5_glue->destroy_flow(flow->verbs.flow)); + flow->verbs.flow = NULL; + } + } + if (flow->verbs.hrxq) { + mlx5_hrxq_drop_release(dev); + flow->verbs.hrxq = NULL; + } +} + +/** + * Apply the flow. + * + * @param[in] dev + * Pointer to Ethernet device structure. + * @param[in, out] flow + * Pointer to flow structure. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_apply(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error) +{ + flow->verbs.hrxq = mlx5_hrxq_drop_new(dev); + if (!flow->verbs.hrxq) + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot allocate Drop queue"); + flow->verbs.flow = + mlx5_glue->create_flow(flow->verbs.hrxq->qp, flow->verbs.attr); + if (!flow->verbs.flow) { + mlx5_hrxq_drop_release(dev); + flow->verbs.hrxq = NULL; + return rte_flow_error_set(error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "kernel module refuses to create" + " flow"); + } + return 0; +} + +/** + * Create a flow and add it to @p list. + * + * @param dev + * Pointer to Ethernet device. + * @param list + * Pointer to a TAILQ flow list. + * @param[in] attr + * Flow rule attributes. + * @param[in] items + * Pattern specification (list terminated by the END pattern item). + * @param[in] actions + * Associated actions (list terminated by the END action). + * @param[out] error + * Perform verbose error reporting if not NULL. + * + * @return + * A flow on success, NULL otherwise and rte_errno is set. + */ +static struct rte_flow * +mlx5_flow_list_create(struct rte_eth_dev *dev, + struct mlx5_flows *list, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct rte_flow *flow; + size_t size; + int ret; + + ret = mlx5_flow_merge(dev, NULL, 0, attr, items, actions, error); + if (ret < 0) + return NULL; + size = ret; + flow = rte_zmalloc(__func__, size, 0); + if (!flow) { + rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot allocate memory"); + return NULL; + } + flow->verbs.attr = (struct ibv_flow_attr *)(flow + 1); + flow->verbs.specs = (uint8_t *)(flow->verbs.attr + 1); + ret = mlx5_flow_merge(dev, flow, size, attr, items, actions, error); + if (ret < 0) + goto error; + assert((size_t)ret == size); + if (dev->data->dev_started) { + ret = mlx5_flow_apply(dev, flow, error); + if (ret < 0) + goto error; + } + TAILQ_INSERT_TAIL(list, flow, next); + return flow; +error: + ret = rte_errno; /* Save rte_errno before cleanup. */ + mlx5_flow_remove(dev, flow); + rte_free(flow); + rte_errno = ret; /* Restore rte_errno. */ + return NULL; } /** @@ -187,17 +775,15 @@ mlx5_flow_validate(struct rte_eth_dev *dev __rte_unused, * @see rte_flow_ops */ struct rte_flow * -mlx5_flow_create(struct rte_eth_dev *dev __rte_unused, - const struct rte_flow_attr *attr __rte_unused, - const struct rte_flow_item items[] __rte_unused, - const struct rte_flow_action actions[] __rte_unused, +mlx5_flow_create(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], struct rte_flow_error *error) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "action not supported"); - return NULL; + return mlx5_flow_list_create + (dev, &((struct priv *)dev->data->dev_private)->flows, + attr, items, actions, error); } /** @@ -211,10 +797,12 @@ mlx5_flow_create(struct rte_eth_dev *dev __rte_unused, * Flow to destroy. */ static void -mlx5_flow_list_destroy(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flows *list __rte_unused, - struct rte_flow *flow __rte_unused) +mlx5_flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, + struct rte_flow *flow) { + mlx5_flow_remove(dev, flow); + TAILQ_REMOVE(list, flow, next); + rte_free(flow); } /** From patchwork Wed Jul 11 07:22:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42798 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D57E31B4DB; Wed, 11 Jul 2018 09:23:32 +0200 (CEST) Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by dpdk.org (Postfix) with ESMTP id 84B5F1B4B8 for ; Wed, 11 Jul 2018 09:23:22 +0200 (CEST) Received: by mail-wr1-f65.google.com with SMTP id p1-v6so16999129wrs.9 for ; Wed, 11 Jul 2018 00:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ZEIFVBxPBpaxjyp1iYHHlFX3tzI/If+jotUy4ZQt4/4=; b=jCnEUaCfDLlsNYoM3t5RoiZ1owKxcamzQdgjZtrSGik++1FxsXdaMRfGeUGeD6OGNg GhWOuJ+dGCqL/3tP31BMzbMjlOjSMNt0iEBLImPLdWyKbTMtquJXRXyVGVtg6O/LFU91 R4e8MBMygkO5nqnVbs8Rwi2xNVwN/1seW01mKMCUzylLoEf6dBHImNxPULvtU2tm8f/o RwVHwPCSKZVvipgDFIqhtYdujnODLl5hLMghYcXMjNG/1GCOeExf9OPlia7/v+AwwReR /dgcwkAEBMUxiXU+8CvW4gzy4+SFFpzCmIaEomkuZV0fFi96ATkbx9xXlsvs4Nmc1FHd z/hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ZEIFVBxPBpaxjyp1iYHHlFX3tzI/If+jotUy4ZQt4/4=; b=J5UoWfo2JujnFH6Xz5GsoXSVyqK6fnpKrvra4AGslOk5dPlTeeplnVpGjDiYdBgYq7 VXlIaN4uxBYPSmUaxQDsM0S0wGte4by5693clX0otLj8Nuh0iUiSKqeSbe+s70XaLdAR SvvwyFFLkf75cMAYM6tinpR/+BCd6LuHkd9OI2Ibu5PKSMAj42ymVkOKxCg8zwq7EpsR fM1gQbNmbfbxmD704oFraowedP26exh1DdQGkWpbGP/TkTwJJC9jW3kV2Eu06ILFyBlM U/wfnC/XSucJ8yT8C6pjaJxGTrQBQEdZcm4Iyjy97xgeRXIDiw/1w0wLtFzHyvkOdsKc sDLg== X-Gm-Message-State: APt69E2useZMEO2i8wEaIqd9SSKugHXFT/B5PM72b3l2JpIZFIPrNvHk L6CK5qwbQHI+DsD6ghjPojvzyWklog== X-Google-Smtp-Source: AAOMgpf16ETr8nctIzuMytQb1QOA62dUiAIJVI2wMe16P8nSatQ2+FVoFEBuebn2h5hXNbvDIjq8fg== X-Received: by 2002:adf:fdcd:: with SMTP id i13-v6mr19058400wrs.276.1531293802077; Wed, 11 Jul 2018 00:23:22 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:21 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:38 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 05/21] net/mlx5: add flow queue action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 97 ++++++++++++++++++++++++++++++++---- 1 file changed, 86 insertions(+), 11 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 036a8d440..6041a4573 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -50,6 +50,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Actions that modify the fate of matching traffic. */ #define MLX5_FLOW_FATE_DROP (1u << 0) +#define MLX5_FLOW_FATE_QUEUE (1u << 1) /** Handles information leading to a drop fate. */ struct mlx5_flow_verbs { @@ -71,7 +72,8 @@ struct rte_flow { /**< Bit-fields of present layers see MLX5_FLOW_LAYER_*. */ uint32_t fate; /**< Bit-fields of present fate see MLX5_FLOW_FATE_*. */ - struct mlx5_flow_verbs verbs; /* Verbs drop flow. */ + struct mlx5_flow_verbs verbs; /* Verbs flow. */ + uint16_t queue; /**< Destination queue to redirect traffic to. */ }; static const struct rte_flow_ops mlx5_flow_ops = { @@ -492,6 +494,52 @@ mlx5_flow_action_drop(const struct rte_flow_action *action, return size; } +/** + * Convert the @p action into @p flow after ensuring the NIC will understand + * and process it correctly. + * + * @param[in] dev + * Pointer to Ethernet device structure. + * @param[in] action + * Action configuration. + * @param[in, out] flow + * Pointer to flow structure. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_action_queue(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow *flow, + struct rte_flow_error *error) +{ + struct priv *priv = dev->data->dev_private; + const struct rte_flow_action_queue *queue = action->conf; + + if (flow->fate) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "multiple fate actions are not" + " supported"); + if (queue->index >= priv->rxqs_n) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &queue->index, + "queue index out of range"); + if (!(*priv->rxqs)[queue->index]) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &queue->index, + "queue is not configured"); + flow->queue = queue->index; + flow->fate |= MLX5_FLOW_FATE_QUEUE; + return 0; +} + /** * Convert the @p action into @p flow after ensuring the NIC will understand * and process it correctly. @@ -520,7 +568,7 @@ mlx5_flow_action_drop(const struct rte_flow_action *action, * On error, a negative errno value is returned and rte_errno is set. */ static int -mlx5_flow_actions(struct rte_eth_dev *dev __rte_unused, +mlx5_flow_actions(struct rte_eth_dev *dev, const struct rte_flow_action actions[], struct rte_flow *flow, const size_t flow_size, struct rte_flow_error *error) @@ -537,6 +585,9 @@ mlx5_flow_actions(struct rte_eth_dev *dev __rte_unused, ret = mlx5_flow_action_drop(actions, flow, remain, error); break; + case RTE_FLOW_ACTION_TYPE_QUEUE: + ret = mlx5_flow_action_queue(dev, actions, flow, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -661,7 +712,10 @@ mlx5_flow_remove(struct rte_eth_dev *dev, struct rte_flow *flow) } } if (flow->verbs.hrxq) { - mlx5_hrxq_drop_release(dev); + if (flow->fate & MLX5_FLOW_FATE_DROP) + mlx5_hrxq_drop_release(dev); + else if (flow->fate & MLX5_FLOW_FATE_QUEUE) + mlx5_hrxq_release(dev, flow->verbs.hrxq); flow->verbs.hrxq = NULL; } } @@ -683,17 +737,38 @@ static int mlx5_flow_apply(struct rte_eth_dev *dev, struct rte_flow *flow, struct rte_flow_error *error) { - flow->verbs.hrxq = mlx5_hrxq_drop_new(dev); - if (!flow->verbs.hrxq) - return rte_flow_error_set - (error, errno, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot allocate Drop queue"); + if (flow->fate & MLX5_FLOW_FATE_DROP) { + flow->verbs.hrxq = mlx5_hrxq_drop_new(dev); + if (!flow->verbs.hrxq) + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot allocate Drop queue"); + } else if (flow->fate & MLX5_FLOW_FATE_QUEUE) { + struct mlx5_hrxq *hrxq; + + hrxq = mlx5_hrxq_get(dev, rss_hash_default_key, + rss_hash_default_key_len, 0, + &flow->queue, 1, 0, 0); + if (!hrxq) + hrxq = mlx5_hrxq_new(dev, rss_hash_default_key, + rss_hash_default_key_len, 0, + &flow->queue, 1, 0, 0); + if (!hrxq) + return rte_flow_error_set(error, rte_errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot create flow"); + flow->verbs.hrxq = hrxq; + } flow->verbs.flow = mlx5_glue->create_flow(flow->verbs.hrxq->qp, flow->verbs.attr); if (!flow->verbs.flow) { - mlx5_hrxq_drop_release(dev); + if (flow->fate & MLX5_FLOW_FATE_DROP) + mlx5_hrxq_drop_release(dev); + else + mlx5_hrxq_release(dev, flow->verbs.hrxq); flow->verbs.hrxq = NULL; return rte_flow_error_set(error, errno, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, From patchwork Wed Jul 11 07:22:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42799 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BBE2E1B4F4; Wed, 11 Jul 2018 09:23:35 +0200 (CEST) Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by dpdk.org (Postfix) with ESMTP id 541111B4CF for ; Wed, 11 Jul 2018 09:23:23 +0200 (CEST) Received: by mail-wm0-f66.google.com with SMTP id s14-v6so1409169wmc.1 for ; Wed, 11 Jul 2018 00:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Ccw4CjKG9Rqt+pwxyhFq3+hXv9opIbWyMqT/Y9RoQS8=; b=hY1nt15q1xPWUaX/BhHvgKA3qZ7oDtAXAukyO0MJNymrXgg63qSUoII74Pgb8Dg+WS /4DzF8RhCzeXu7qAp/4fUpahaTicBrmP53nxf88eRc+PYRps8IgkIPKVhmvIzmsPXkC6 JvV44SX1ZQ6rBcn+99YS9nowD0i5mOM5ZBsOaIkoXJOov/9ija9fVsN6jVNNllsYkzuS phuvHHFg5LS7dlAaaiJHO7GHd2FydBxTbW3KtFq/WuC9lKhorSYmGuqhqywEqBWA32vN iS6dXXWH7nZnik2LKapeKOv5cDjg61xsE7iODk2yWAZLZbdt5dVHEYF+/mK41M3xxKmP X49A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Ccw4CjKG9Rqt+pwxyhFq3+hXv9opIbWyMqT/Y9RoQS8=; b=ntWh6VG74UClznhji2H6Aba9uPCiknN407JWSeZ/zEBMG09EqYngx0E69ZVYXT6nam fwK7ruZHw13VCHXYjx9mEh5g7JgDlt+KslTmJZLqJirbYaP+7O/lGhZCuJK6ehhYEX2u QKp/OPn7nWEBrehe2NlRVsYP3JHvWPZdmyMgFchTpQLIsC8RQokK2Cn0cygH4W9rknD6 BOaYtuEuf5Q2s1DOO2IbG4UCAN0opa7/CmJlw3UFtOR56mzzKjutxtrcp7Ti3QaC12cD XLsWNfT71w23flv20KS9d9Q7Rbb/Snk5Cdj3tgJ5kptodQsp0/lKBafYg55qvvodHs1Y OVUQ== X-Gm-Message-State: APt69E3OqxwmPDlof+fEUWxmvGZW0eK77aXKAlvycOl4cyCisE74KXK5 jvWWrvBBViqUXlgopeLPrITYucAXyg== X-Google-Smtp-Source: AAOMgpeTw0TkO/mGB/9i8QQTuNq2LFGcmERYDyYlOu/wvCJftLghLDI3iC/nOovFaBfYy8dyjSUQoA== X-Received: by 2002:a1c:c44f:: with SMTP id u76-v6mr16186079wmf.43.1531293802915; Wed, 11 Jul 2018 00:23:22 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:22 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:39 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 06/21] net/mlx5: add flow stop/start X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 6041a4573..77f1bd5cc 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -908,9 +908,12 @@ mlx5_flow_list_flush(struct rte_eth_dev *dev, struct mlx5_flows *list) * Pointer to a TAILQ flow list. */ void -mlx5_flow_stop(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flows *list __rte_unused) +mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) { + struct rte_flow *flow; + + TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) + mlx5_flow_remove(dev, flow); } /** @@ -925,10 +928,23 @@ mlx5_flow_stop(struct rte_eth_dev *dev __rte_unused, * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_start(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flows *list __rte_unused) +mlx5_flow_start(struct rte_eth_dev *dev, struct mlx5_flows *list) { + struct rte_flow *flow; + struct rte_flow_error error; + int ret = 0; + + TAILQ_FOREACH(flow, list, next) { + ret = mlx5_flow_apply(dev, flow, &error); + if (ret < 0) + goto error; + } return 0; +error: + ret = rte_errno; /* Save rte_errno before cleanup. */ + mlx5_flow_stop(dev, list); + rte_errno = ret; /* Restore rte_errno. */ + return -rte_errno; } /** From patchwork Wed Jul 11 07:22:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42800 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A1C531B506; Wed, 11 Jul 2018 09:23:39 +0200 (CEST) Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by dpdk.org (Postfix) with ESMTP id 2753F1B4B8 for ; Wed, 11 Jul 2018 09:23:24 +0200 (CEST) Received: by mail-wm0-f66.google.com with SMTP id s14-v6so1409207wmc.1 for ; Wed, 11 Jul 2018 00:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=oN1ewf2kbSSXMWcTVsplBcZJodOEyXEvqN60n9wbWpU=; b=SvLsAhU6TWM0OVZem20lyMYMykD390HpVmNfNvUI0X9f2y7EI4U+zSCiqvzHqb0CLk MeO0iSfa2+p56aAp56IN7aJ3K0a+p2zdS3tn/QP9XzNCtyJOVCov+vO3LeSuR6ahp5yq ifQeRSzdv3pTh8dZui1F0fdu0JOhRtWA7weqRBtt4t0waT2oa79DXO+6kXH/z8QbGRBp QpR4lccYssomVIe8C/vLb63pJAfkC60ahcLy1N7F2iI/lPmlb1SWqoyrNg6HVtu5B9Rb bn+W7fNVF2MZDwJ9g6vmBZDA5Mb+j1+RxCr2COd/AecBw+50JJDo47h0rqPsqN1iC4rA p35Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oN1ewf2kbSSXMWcTVsplBcZJodOEyXEvqN60n9wbWpU=; b=J2BVlfIX6YiSZSbnrQLQvwg+1fhpbARqdU/qspSfL+l7nrdXyZRyWbyBGMAAGtg1v+ eojgbkRrfYgaDtQxNbuokj3Z4JOm9TFJscYhu+WO18S132UZSD1lYjUmeYMIeLf8qq4A BorRrMeP19yFtXlFHGsN8mZZGbMsy5N9YXFrsZEBq0BhkDa8hKVh2eVi71HBTsAfeuJy La1sGDnudci74kZ1fOvkGMw9u5mqvdLn0+eeLWwOWM/QOJ7YCq+Fjb5hGb6uj+ackMT0 g8ipyOp1WpAt/W9KwbCI6LEDSY2U1xM9mLBzP65I4cO9SiSV2SmjVbLCiaOhZrT4id9p qeYg== X-Gm-Message-State: APt69E1NvNhHFo4VAbYWgwlCnlZzW+xYGox2T7+nqbrs7brnxmNZyv7M CRzVmfhmUg4vgsMxUX9l8AbNeFJeGA== X-Google-Smtp-Source: AAOMgpcXsc99nD60Be15Gptneia6mb9fZqJYcoF36AA/VBsb1xRPqggyfdUNZGo/vmrZ0EUUEzxYCw== X-Received: by 2002:a1c:e708:: with SMTP id e8-v6mr10387896wmh.134.1531293803698; Wed, 11 Jul 2018 00:23:23 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:23 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:40 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 07/21] net/mlx5: add flow VLAN item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 127 +++++++++++++++++++++++++++++++++++ 1 file changed, 127 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 77f1bd5cc..659979283 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -382,6 +382,130 @@ mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Update the VLAN tag in the Verbs Ethernet specification. + * + * @param[in, out] attr + * Pointer to Verbs attributes structure. + * @param[in] eth + * Verbs structure containing the VLAN information to copy. + */ +static void +mlx5_flow_item_vlan_update(struct ibv_flow_attr *attr, + struct ibv_flow_spec_eth *eth) +{ + unsigned int i; + enum ibv_flow_spec_type search = IBV_FLOW_SPEC_ETH; + struct ibv_spec_header *hdr = (struct ibv_spec_header *) + ((uint8_t *)attr + sizeof(struct ibv_flow_attr)); + + for (i = 0; i != attr->num_of_specs; ++i) { + if (hdr->type == search) { + struct ibv_flow_spec_eth *e = + (struct ibv_flow_spec_eth *)hdr; + + e->val.vlan_tag = eth->val.vlan_tag; + e->mask.vlan_tag = eth->mask.vlan_tag; + e->val.ether_type = eth->val.ether_type; + e->mask.ether_type = eth->mask.ether_type; + break; + } + hdr = (struct ibv_spec_header *)((uint8_t *)hdr + hdr->size); + } +} + +/** + * Convert the @p item into @p flow (or by updating the already present + * Ethernet Verbs) specification after ensuring the NIC will understand and + * process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_vlan *spec = item->spec; + const struct rte_flow_item_vlan *mask = item->mask; + const struct rte_flow_item_vlan nic_mask = { + .tci = RTE_BE16(0x0fff), + .inner_type = RTE_BE16(0xffff), + }; + unsigned int size = sizeof(struct ibv_flow_spec_eth); + struct ibv_flow_spec_eth eth = { + .type = IBV_FLOW_SPEC_ETH, + .size = size, + }; + int ret; + const uint32_t l34m = MLX5_FLOW_LAYER_OUTER_L3 | + MLX5_FLOW_LAYER_OUTER_L4; + const uint32_t vlanm = MLX5_FLOW_LAYER_OUTER_VLAN; + const uint32_t l2m = MLX5_FLOW_LAYER_OUTER_L2; + + if (flow->layers & vlanm) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VLAN layer already configured"); + else if ((flow->layers & l34m) != 0) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L2 layer cannot follow L3/L4 layer"); + if (!mask) + mask = &rte_flow_item_vlan_mask; + ret = mlx5_flow_item_acceptable + (item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_vlan), error); + if (ret) + return ret; + if (spec) { + eth.val.vlan_tag = spec->tci; + eth.mask.vlan_tag = mask->tci; + eth.val.vlan_tag &= eth.mask.vlan_tag; + eth.val.ether_type = spec->inner_type; + eth.mask.ether_type = mask->inner_type; + eth.val.ether_type &= eth.mask.ether_type; + } + /* + * From verbs perspective an empty VLAN is equivalent + * to a packet without VLAN layer. + */ + if (!eth.mask.vlan_tag) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "VLAN cannot be empty"); + if (!(flow->layers & l2m)) { + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, ð, size); + } else { + if (flow->verbs.attr) + mlx5_flow_item_vlan_update(flow->verbs.attr, ð); + size = 0; /* Only an update is done in eth specification. */ + } + flow->layers |= MLX5_FLOW_LAYER_OUTER_L2 | + MLX5_FLOW_LAYER_OUTER_VLAN; + return size; +} + /** * Convert the @p pattern into a Verbs specifications after ensuring the NIC * will understand and process it correctly. @@ -424,6 +548,9 @@ mlx5_flow_items(const struct rte_flow_item pattern[], case RTE_FLOW_ITEM_TYPE_ETH: ret = mlx5_flow_item_eth(pattern, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_VLAN: + ret = mlx5_flow_item_vlan(pattern, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jul 11 07:22:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42801 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D2BE11B513; Wed, 11 Jul 2018 09:23:42 +0200 (CEST) Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com [74.125.82.65]) by dpdk.org (Postfix) with ESMTP id EDBEE1B466 for ; Wed, 11 Jul 2018 09:23:24 +0200 (CEST) Received: by mail-wm0-f65.google.com with SMTP id s12-v6so1361034wmc.1 for ; Wed, 11 Jul 2018 00:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mfbxs3WLJSYeG5D0sppAmHzD3jTOo1M2C+Io/1hQgSQ=; b=ynfroQIhMlemc//Puo1dXDt9YQOAd9xNE5EkHAzbstIL7gNKcOTmMQpNMBrdOIdxNs RH5pm/rlpgStoTapkAEO9FH9e7QFWCkMvelw9nKSkvHE1ufkNH2nIqrVh+QtYGjAHNDB B+xWlzyY87VHMyc13TaSsAKWQ7C1xyq6tRLYRdfB7i0DiiwF78jeFiwbVrAEYD/x0H+U yGlhW10lXouIrcIh6LeVwRR//b5aq90q2d/312zTHFkgqIAcv9ep61tCkSpPmpxWc2d+ MIQpZp+ydP9EPe96KaAy6EtXZ7HW3e2aE1VR/jIExnCRhVH7Pguf67DmGumvLKrAsQAD zc4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mfbxs3WLJSYeG5D0sppAmHzD3jTOo1M2C+Io/1hQgSQ=; b=pLlsLj8A7oO9iNja55x2LEWSsRkrvJj2gBp/OOSGmqUk2uCahQqUL5eTQaAB3NOtuF FsbSuJrulPidGVCtx1XarZcx9mv4G3eKm1rcWQX+8R57WG4HJGsMsK+YUUdTZ6+z+pPq 1YX4A6N/aylxE9Dl7JNbPICsW2lY8AG9Ln7Fqzo7v3ftL5u9UDHawIOgvEa2LDhszYSL LYl+C8+BdXwp2clAOAQg0jo98HgCLfjqnlfHe+0P8aK9ZVJmkdff/xJAj3dR8KBnzXnv fWZ5vvbsiQF3ha3hEU9O1eLcaLO87/fWDUElSNR7DTs1BK+X7CWZ1ZANhLWfjzr4imn0 z0Ig== X-Gm-Message-State: APt69E3RwMf34KdwZgeu268e1BXfxL86uWYNCOgVcnkb13Hjt8Oz/Qh3 6IoJVEeGyJye4Y2pvZY033JHrGDjMw== X-Google-Smtp-Source: AAOMgpeOlI3YLUZxn8vLAfl0oaVrzGz0AopIQSgltCiAjWW+ztMBleIIKjmWBM81CU2n27CG2R3GXA== X-Received: by 2002:a1c:b788:: with SMTP id h130-v6mr17892193wmf.27.1531293804488; Wed, 11 Jul 2018 00:23:24 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:23 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:41 +0200 Message-Id: <9d8fa5593b1874380094bfae87bd9aefd017b30c.1531293415.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 08/21] net/mlx5: add flow IPv4 item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 90 ++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 659979283..c05b8498d 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -506,6 +506,93 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Convert the @p item into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_ipv4 *spec = item->spec; + const struct rte_flow_item_ipv4 *mask = item->mask; + const struct rte_flow_item_ipv4 nic_mask = { + .hdr = { + .src_addr = RTE_BE32(0xffffffff), + .dst_addr = RTE_BE32(0xffffffff), + .type_of_service = 0xff, + .next_proto_id = 0xff, + }, + }; + unsigned int size = sizeof(struct ibv_flow_spec_ipv4_ext); + struct ibv_flow_spec_ipv4_ext ipv4 = { + .type = IBV_FLOW_SPEC_IPV4_EXT, + .size = size, + }; + int ret; + + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L3) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "multiple L3 layers not supported"); + else if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 cannot follow an L4 layer."); + if (!mask) + mask = &rte_flow_item_ipv4_mask; + ret = mlx5_flow_item_acceptable + (item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_ipv4), error); + if (ret < 0) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV4; + if (size > flow_size) + return size; + if (spec) { + ipv4.val = (struct ibv_flow_ipv4_ext_filter){ + .src_ip = spec->hdr.src_addr, + .dst_ip = spec->hdr.dst_addr, + .proto = spec->hdr.next_proto_id, + .tos = spec->hdr.type_of_service, + }; + ipv4.mask = (struct ibv_flow_ipv4_ext_filter){ + .src_ip = mask->hdr.src_addr, + .dst_ip = mask->hdr.dst_addr, + .proto = mask->hdr.next_proto_id, + .tos = mask->hdr.type_of_service, + }; + /* Remove unwanted bits from values. */ + ipv4.val.src_ip &= ipv4.mask.src_ip; + ipv4.val.dst_ip &= ipv4.mask.dst_ip; + ipv4.val.proto &= ipv4.mask.proto; + ipv4.val.tos &= ipv4.mask.tos; + } + mlx5_flow_spec_verbs_add(flow, &ipv4, size); + return size; +} + /** * Convert the @p pattern into a Verbs specifications after ensuring the NIC * will understand and process it correctly. @@ -551,6 +638,9 @@ mlx5_flow_items(const struct rte_flow_item pattern[], case RTE_FLOW_ITEM_TYPE_VLAN: ret = mlx5_flow_item_vlan(pattern, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_IPV4: + ret = mlx5_flow_item_ipv4(pattern, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jul 11 07:22:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42802 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7D77E1B50C; Wed, 11 Jul 2018 09:23:45 +0200 (CEST) Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id E18301B4D0 for ; Wed, 11 Jul 2018 09:23:25 +0200 (CEST) Received: by mail-wm0-f67.google.com with SMTP id v3-v6so1405890wmh.0 for ; Wed, 11 Jul 2018 00:23:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=H6r+7vlOXnaBtOi2cWpdGX7dqfwLOEk9Klglxa6xoJA=; b=jtiwbESYXMy53jfSQ1bmw1gALVgzWNGpHhvZkTk7g/MIJSVSiK02D+pclR15QxImAG gsK1BG+tBEcTJYNA8hwo9V7HmtUIg/aw4av1lT8BK5mT962Lh5KZw9KwGQ+xsohWY9J6 mJlkF4NBcUwnFAsrVeqsIrHfxvgxTKUgSQ/pvUlMg7b/4XpHdY0HlrMaKeTrnOxNiMWI 3gkShE05pdX1tcWq18xgHpjtkcQm5/dkkmhBYjWuSLXFAWfuG1ouJyU+TN2CJKC/SsiV o/7MAT4pO7b6MMdEIBUEUUjkX04NKzQSks9hGYhV6Ct3DhdC2CYmUWKwcV4MlSrBwm5d N4+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=H6r+7vlOXnaBtOi2cWpdGX7dqfwLOEk9Klglxa6xoJA=; b=Pev3RmpkfzEOs7lEdiqCBsgpLlk0djCjKFuXe4k2niDx2aB9CuTWQSWZi9sNR9SfzJ w+PF2tuqIyB3902USYDnI1pQYphqWR2M57+9RDJvz2aWjSCAR8mfMbGCtH8LVz/iCswp MyAQt4Htk96aOfnvWl5ZTiFB3cQAZDWXWrW0bCFaxAWv2k2bcfJZYFykNGEZAkPf8WDm y4L7GWc0ekhA0bXS7bEpf+xif9gKmmVgZr+DScM7aVSliEtrQ3y7T1mL0eLBhGUhy/1i 7HyJ9pqbxwQLE2F8Tyn1JUFhRy6ls6ZWLuEGnsE4/kZbwUzp+y/h0cW6aEYynSHlRTBJ ifxg== X-Gm-Message-State: APt69E3JQpT8aqIqqMyzfVIgpzgL5jgF/5sZlqn9YJOmsDNzciEWk/d3 bjplmX0lyN3eSvGguNy84snPQX/7zg== X-Google-Smtp-Source: AAOMgpeVw5D8PJIVj2eS66yzmcQDdhgGCifmkwGNr0LRNOQR0AqIBTHjUFJ4cvDOfk6UL12ROJDZ9g== X-Received: by 2002:a1c:ae8b:: with SMTP id x133-v6mr17950334wme.125.1531293805333; Wed, 11 Jul 2018 00:23:25 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:24 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:42 +0200 Message-Id: <961c18fbf00632b88d9be7ca87663dfaca4cfc0a.1531293415.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 09/21] net/mlx5: add flow IPv6 item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 115 +++++++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index c05b8498d..513f70d40 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -593,6 +593,118 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Convert the @p item into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_ipv6 *spec = item->spec; + const struct rte_flow_item_ipv6 *mask = item->mask; + const struct rte_flow_item_ipv6 nic_mask = { + .hdr = { + .src_addr = + "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff", + .dst_addr = + "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff", + .vtc_flow = RTE_BE32(0xffffffff), + .proto = 0xff, + .hop_limits = 0xff, + }, + }; + unsigned int size = sizeof(struct ibv_flow_spec_ipv6); + struct ibv_flow_spec_ipv6 ipv6 = { + .type = IBV_FLOW_SPEC_IPV6, + .size = size, + }; + int ret; + + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L3) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "multiple L3 layers not supported"); + else if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 cannot follow an L4 layer."); + if (!mask) + mask = &rte_flow_item_ipv6_mask; + ret = mlx5_flow_item_acceptable + (item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_ipv6), error); + if (ret < 0) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV6; + if (size > flow_size) + return size; + if (spec) { + unsigned int i; + uint32_t vtc_flow_val; + uint32_t vtc_flow_mask; + + memcpy(&ipv6.val.src_ip, spec->hdr.src_addr, + RTE_DIM(ipv6.val.src_ip)); + memcpy(&ipv6.val.dst_ip, spec->hdr.dst_addr, + RTE_DIM(ipv6.val.dst_ip)); + memcpy(&ipv6.mask.src_ip, mask->hdr.src_addr, + RTE_DIM(ipv6.mask.src_ip)); + memcpy(&ipv6.mask.dst_ip, mask->hdr.dst_addr, + RTE_DIM(ipv6.mask.dst_ip)); + vtc_flow_val = rte_be_to_cpu_32(spec->hdr.vtc_flow); + vtc_flow_mask = rte_be_to_cpu_32(mask->hdr.vtc_flow); + ipv6.val.flow_label = + rte_cpu_to_be_32((vtc_flow_val & IPV6_HDR_FL_MASK) >> + IPV6_HDR_FL_SHIFT); + ipv6.val.traffic_class = (vtc_flow_val & IPV6_HDR_TC_MASK) >> + IPV6_HDR_TC_SHIFT; + ipv6.val.next_hdr = spec->hdr.proto; + ipv6.val.hop_limit = spec->hdr.hop_limits; + ipv6.mask.flow_label = + rte_cpu_to_be_32((vtc_flow_mask & IPV6_HDR_FL_MASK) >> + IPV6_HDR_FL_SHIFT); + ipv6.mask.traffic_class = (vtc_flow_mask & IPV6_HDR_TC_MASK) >> + IPV6_HDR_TC_SHIFT; + ipv6.mask.next_hdr = mask->hdr.proto; + ipv6.mask.hop_limit = mask->hdr.hop_limits; + /* Remove unwanted bits from values. */ + for (i = 0; i < RTE_DIM(ipv6.val.src_ip); ++i) { + ipv6.val.src_ip[i] &= ipv6.mask.src_ip[i]; + ipv6.val.dst_ip[i] &= ipv6.mask.dst_ip[i]; + } + ipv6.val.flow_label &= ipv6.mask.flow_label; + ipv6.val.traffic_class &= ipv6.mask.traffic_class; + ipv6.val.next_hdr &= ipv6.mask.next_hdr; + ipv6.val.hop_limit &= ipv6.mask.hop_limit; + } + mlx5_flow_spec_verbs_add(flow, &ipv6, size); + return size; +} + /** * Convert the @p pattern into a Verbs specifications after ensuring the NIC * will understand and process it correctly. @@ -641,6 +753,9 @@ mlx5_flow_items(const struct rte_flow_item pattern[], case RTE_FLOW_ITEM_TYPE_IPV4: ret = mlx5_flow_item_ipv4(pattern, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_IPV6: + ret = mlx5_flow_item_ipv6(pattern, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jul 11 07:22:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42803 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 983CC1B51F; Wed, 11 Jul 2018 09:23:47 +0200 (CEST) Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id 988251B4DE for ; Wed, 11 Jul 2018 09:23:26 +0200 (CEST) Received: by mail-wm0-f67.google.com with SMTP id z13-v6so1385747wma.5 for ; Wed, 11 Jul 2018 00:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=j8VbX1jANxkfYceXfWDEHAqvwkXaiqVftC18KlRQkjc=; b=cL3nhL217aIdNiZhqkjPk0Zi1ZJbnYQ5QlJ/IUJF8ftrUFSQFHFLIuYnkreHoM54p1 o76z1WuIj0DJAxANOtXJMcf+IJjbGleyq0Sb2C6iMsrIcdp5GmSI7PotpDHQg876TrWr H2nuZ6cdHOb1yDBKl+61h9AsCFB811cdoQQT1VpnNWdCOyH3JD5Fhu5Q55Bny210M3EU gJsBVhkMRjY/yXhbsDXlKv5IjqeK+nQxarW2FxVUkmRdEay8NzhTbt7NQ4Sjsm50Jrhj bqWesqHUymlNz4IgfNQ+GURuJXfBTHJiagDTRo9+iNqJkIKJFb0whNFiriFxuVBOVVlg nntA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=j8VbX1jANxkfYceXfWDEHAqvwkXaiqVftC18KlRQkjc=; b=kcLZWPd7NniABK6Ped5qaw6ERyfwtxY66Zpn/1Eh7cIq3CpyBcvPmCTBLCEsoIgy2a 2BAd+RQ0P53gq4JEYsWNOkbHs7sg3Fr3r0XeFt84tREFzm8wgHMxZQCPEJTBD1ZFAnMo CIhQ7i+woQyGvQCTb6Jd6N1vpn6Bpwo3C5TJREoyo/DovV0MsMi18/NxcqElDp+JYoyl p1tLpGjXVkB0AhUMT4RiPQvW+XQLjsPM2rBdJ022uYo1HlKdxnNmcEbilzyGbQusxn2u d/rVNi2X2mRUcSVzaALcwhHv9ziIO8mUNRmeGoLAj+84EMWfPm1Ti8s4QgnCQyqFSz4Z HJyA== X-Gm-Message-State: APt69E1t5K7lwtqM1hQphivGfOAyXHzKaLa7hAgj0h4Vab3Yv1Tm+RE5 M9EfRnUGADEr4rFo1VM7WgK3nmIrCQ== X-Google-Smtp-Source: AAOMgpfol5i7OqDcVLQCZgKFAwEDSOwxjeFa7rSCKRqwZIjNIbnmtK+dtB0gdhBLZMPBf0x63nwnHg== X-Received: by 2002:a1c:3662:: with SMTP id d95-v6mr15766157wma.147.1531293806111; Wed, 11 Jul 2018 00:23:26 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:25 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:43 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 10/21] net/mlx5: add flow UDP item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 97 +++++++++++++++++++++++++++++++++--- 1 file changed, 91 insertions(+), 6 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 513f70d40..0096ed8a2 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -52,6 +52,9 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; #define MLX5_FLOW_FATE_DROP (1u << 0) #define MLX5_FLOW_FATE_QUEUE (1u << 1) +/* possible L3 layers protocols filtering. */ +#define MLX5_IP_PROTOCOL_UDP 17 + /** Handles information leading to a drop fate. */ struct mlx5_flow_verbs { unsigned int size; /**< Size of the attribute. */ @@ -68,10 +71,12 @@ struct mlx5_flow_verbs { struct rte_flow { TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ struct rte_flow_attr attributes; /**< User flow attribute. */ + uint32_t l3_protocol_en:1; /**< Protocol filtering requested. */ uint32_t layers; /**< Bit-fields of present layers see MLX5_FLOW_LAYER_*. */ uint32_t fate; /**< Bit-fields of present fate see MLX5_FLOW_FATE_*. */ + uint8_t l3_protocol; /**< valid when l3_protocol_en is set. */ struct mlx5_flow_verbs verbs; /* Verbs flow. */ uint16_t queue; /**< Destination queue to redirect traffic to. */ }; @@ -568,8 +573,6 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, if (ret < 0) return ret; flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV4; - if (size > flow_size) - return size; if (spec) { ipv4.val = (struct ibv_flow_ipv4_ext_filter){ .src_ip = spec->hdr.src_addr, @@ -589,7 +592,10 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, ipv4.val.proto &= ipv4.mask.proto; ipv4.val.tos &= ipv4.mask.tos; } - mlx5_flow_spec_verbs_add(flow, &ipv4, size); + flow->l3_protocol_en = !!ipv4.mask.proto; + flow->l3_protocol = ipv4.val.proto; + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &ipv4, size); return size; } @@ -660,8 +666,6 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, if (ret < 0) return ret; flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV6; - if (size > flow_size) - return size; if (spec) { unsigned int i; uint32_t vtc_flow_val; @@ -701,7 +705,85 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, ipv6.val.next_hdr &= ipv6.mask.next_hdr; ipv6.val.hop_limit &= ipv6.mask.hop_limit; } - mlx5_flow_spec_verbs_add(flow, &ipv6, size); + flow->l3_protocol_en = !!ipv6.mask.next_hdr; + flow->l3_protocol = ipv6.val.next_hdr; + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &ipv6, size); + return size; +} + +/** + * Convert the @p item into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_udp *spec = item->spec; + const struct rte_flow_item_udp *mask = item->mask; + unsigned int size = sizeof(struct ibv_flow_spec_tcp_udp); + struct ibv_flow_spec_tcp_udp udp = { + .type = IBV_FLOW_SPEC_UDP, + .size = size, + }; + int ret; + + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 is mandatory to filter on L4"); + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L4 layer is already present"); + if (flow->l3_protocol_en && flow->l3_protocol != MLX5_IP_PROTOCOL_UDP) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "protocol filtering not compatible" + " with UDP layer"); + if (!mask) + mask = &rte_flow_item_udp_mask; + ret = mlx5_flow_item_acceptable + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_udp_mask, + sizeof(struct rte_flow_item_udp), error); + if (ret < 0) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_UDP; + if (size > flow_size) + return size; + if (spec) { + udp.val.dst_port = spec->hdr.dst_port; + udp.val.src_port = spec->hdr.src_port; + udp.mask.dst_port = mask->hdr.dst_port; + udp.mask.src_port = mask->hdr.src_port; + /* Remove unwanted bits from values. */ + udp.val.src_port &= udp.mask.src_port; + udp.val.dst_port &= udp.mask.dst_port; + } + mlx5_flow_spec_verbs_add(flow, &udp, size); return size; } @@ -756,6 +838,9 @@ mlx5_flow_items(const struct rte_flow_item pattern[], case RTE_FLOW_ITEM_TYPE_IPV6: ret = mlx5_flow_item_ipv6(pattern, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_UDP: + ret = mlx5_flow_item_udp(pattern, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jul 11 07:22:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42804 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C394B1B538; Wed, 11 Jul 2018 09:23:50 +0200 (CEST) Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by dpdk.org (Postfix) with ESMTP id 7501E1B4D0 for ; Wed, 11 Jul 2018 09:23:27 +0200 (CEST) Received: by mail-wm0-f66.google.com with SMTP id 69-v6so1383368wmf.3 for ; Wed, 11 Jul 2018 00:23:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=+jIgZdPvzrOLEyYIES6ZvqCP8B4B6iee7ZK4xNloXdU=; b=SL+HfimsRvNPpPl1iZBUNjvdbXB2Qifr9eReS+v+etRVz+p9QXn4LOcvJQCkeFkyGg PxTNtXeV9iUHezwuqAhcKnqEdB6mBxrvr482nTdIBHVwhlgbJs1tktJgGJ7HnL3rkBZY Q3pniGYLUPwQWAgba3DEnZXReMOP90haP4VX4zIO+1DgxSBQlKP8qnmibbxA+Kv8DpFz YD+lOPy5LhH6VB98MzEfM16Ibi/d/RwfPpwemLmzMFuOHj+98SaCNrWMg83wjGAQiJZE IEj1Y6t8lLexapiQHah5uFV/3G4TfcTBVNo/cOZUpg+cdLkUVCCO0Uz8DUw3gUeu59ps T2Cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+jIgZdPvzrOLEyYIES6ZvqCP8B4B6iee7ZK4xNloXdU=; b=HWHBIABpNV2TBxEHPu91S3A4w2Nqv10F7U8SCsbvcxuNDksdzy9VE4x50+AJAUHxuP vlFXh5P14DhOodY/XP+294MnS3+YkM/qlNBFhpohyEuWtDk51p9tN7D2sc5hS27zXC/9 IW7WVko8U+lY00pfdhHU77HAEnVY4oCxA67y7EjNT9IfanL5Je3GGKxIWoMEzNxR5MCs OTffeNJV+GvhrnPefYFxG+mDWrzoLQ9MtcW/feCag/3wBXL3wXekS2BmKjq5cCosc68i tUf9HyeANc4BfCJ0HrdSISQEqaUvqhheHVpJtncdqwmSOOXBhnjqpvGtC5eskJWu0llh 0lAg== X-Gm-Message-State: APt69E3G97SGCZs5/xkLBY3hxwUZC5uVqdIzZPmxcNyGjTHNf4sSeXvS ANatW0MOeunMY6t/T1OG9tLLx3mtrA== X-Google-Smtp-Source: AAOMgpcLkxvy49Bzn3f0SiO8fydBJbheVux4n/B557dzLuDcFiU8uAG+vR9keCc7k8SWzyCUSFTEEQ== X-Received: by 2002:a1c:3351:: with SMTP id z78-v6mr16003435wmz.23.1531293807009; Wed, 11 Jul 2018 00:23:27 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:26 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:44 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 11/21] net/mlx5: add flow TCP item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 79 ++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 0096ed8a2..f646eee01 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -53,6 +53,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; #define MLX5_FLOW_FATE_QUEUE (1u << 1) /* possible L3 layers protocols filtering. */ +#define MLX5_IP_PROTOCOL_TCP 6 #define MLX5_IP_PROTOCOL_UDP 17 /** Handles information leading to a drop fate. */ @@ -787,6 +788,81 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Convert the @p item into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_tcp *spec = item->spec; + const struct rte_flow_item_tcp *mask = item->mask; + unsigned int size = sizeof(struct ibv_flow_spec_tcp_udp); + struct ibv_flow_spec_tcp_udp tcp = { + .type = IBV_FLOW_SPEC_TCP, + .size = size, + }; + int ret; + + if (flow->l3_protocol_en && flow->l3_protocol != MLX5_IP_PROTOCOL_TCP) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "protocol filtering not compatible" + " with TCP layer"); + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 is mandatory to filter on L4"); + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L4 layer is already present"); + if (!mask) + mask = &rte_flow_item_tcp_mask; + ret = mlx5_flow_item_acceptable + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_tcp_mask, + sizeof(struct rte_flow_item_tcp), error); + if (ret < 0) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_TCP; + if (size > flow_size) + return size; + if (spec) { + tcp.val.dst_port = spec->hdr.dst_port; + tcp.val.src_port = spec->hdr.src_port; + tcp.mask.dst_port = mask->hdr.dst_port; + tcp.mask.src_port = mask->hdr.src_port; + /* Remove unwanted bits from values. */ + tcp.val.src_port &= tcp.mask.src_port; + tcp.val.dst_port &= tcp.mask.dst_port; + } + mlx5_flow_spec_verbs_add(flow, &tcp, size); + return size; +} + /** * Convert the @p pattern into a Verbs specifications after ensuring the NIC * will understand and process it correctly. @@ -841,6 +917,9 @@ mlx5_flow_items(const struct rte_flow_item pattern[], case RTE_FLOW_ITEM_TYPE_UDP: ret = mlx5_flow_item_udp(pattern, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_TCP: + ret = mlx5_flow_item_tcp(pattern, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jul 11 07:22:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42805 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6B8D71B53E; Wed, 11 Jul 2018 09:23:53 +0200 (CEST) Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com [74.125.82.65]) by dpdk.org (Postfix) with ESMTP id 59CCD1B446 for ; Wed, 11 Jul 2018 09:23:28 +0200 (CEST) Received: by mail-wm0-f65.google.com with SMTP id n17-v6so1401999wmh.2 for ; Wed, 11 Jul 2018 00:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9S0/fMBnEvDf2xYlwsfWVDl31vuJUwAdbvVjG8cgNq4=; b=2F6nq+1bkXuWbHSzCmQIlmRjoWAxq2Wsfn2nx8qHSP/UwL0x0WcuLxNuC11xvhHPM3 juN/ZwaIo7q9A4Sev377LumE5bGkyD97SyiQeFLJAOexwd/I1MPWdAjjIdsxqW2y8K2I EfY0i9S2JLieY4N20HvGxJKaS3sKRHwnQiNyXBQgtr/TWBr/ceH9xFVI1cRMhDJQwPha Gi0UYiaMvP3ileGLlpdKgwkrViqY83X5B84id0iJQEUg1bIVpHQIQ98rqbiOhoZGVpvP nguTySLvvOubpdrhXTMTLitI6wklNGGxqTqGs2Mms3VIiRYtMUzJuZZ4NMlIYgR8pLym ApxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9S0/fMBnEvDf2xYlwsfWVDl31vuJUwAdbvVjG8cgNq4=; b=I0HXCyR2gJmsF6qAMwgDqU/xVs2CsHYER1OcBSswrMz/sg+ACMYIcGGKNRcnxZMIee QvVfqbbl7CFosmgid2ovB49Ok6qdz88AiCxeJ3xDqHpGMUcQ87Om9gEH6dEIQwG/YCnw CI+q3GNkDtCL/5m7HLABCOwpTyvc2XqoCuYsrHkgrIdZLtz71zHdfwbsNEgGymu3eGnd 93J6ehdpybPhv9KDEJQh1v67qP/7hHbJmZqBB0hMufb4t3BI1mrAVYF2grYxr1fazfCE g3AxCr8qocuUJbfzaaJGJVHocoNAlmP8sJF9A8UZTj9zzfxoidp++cRSdAYzT2Hye1D6 Ch2Q== X-Gm-Message-State: APt69E3f03a/bc463Y3jBb7Q4XgPI3RlvAubLCbmJDb4IKh/jkUUPQL7 PMGVVVkTz93/zPymrzfIUPlaV06IMQ== X-Google-Smtp-Source: AAOMgpeZozv9q4Dl0MFq1TlgGujirAmCvRPlsvs8jaQIz45gq5KLBwNpLb17lzWDHUQrbdzjZ7tdUA== X-Received: by 2002:a1c:ec86:: with SMTP id h6-v6mr12953200wmi.53.1531293807791; Wed, 11 Jul 2018 00:23:27 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:27 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:45 +0200 Message-Id: <7c8adad56872979f5e74c3d3aac3c9d848a97853.1531293415.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 12/21] net/mlx5: add mark/flag flow action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 252 +++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 1 + 2 files changed, 253 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index f646eee01..1280db486 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -52,6 +52,10 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; #define MLX5_FLOW_FATE_DROP (1u << 0) #define MLX5_FLOW_FATE_QUEUE (1u << 1) +/* Modify a packet. */ +#define MLX5_FLOW_MOD_FLAG (1u << 0) +#define MLX5_FLOW_MOD_MARK (1u << 1) + /* possible L3 layers protocols filtering. */ #define MLX5_IP_PROTOCOL_TCP 6 #define MLX5_IP_PROTOCOL_UDP 17 @@ -75,6 +79,8 @@ struct rte_flow { uint32_t l3_protocol_en:1; /**< Protocol filtering requested. */ uint32_t layers; /**< Bit-fields of present layers see MLX5_FLOW_LAYER_*. */ + uint32_t modifier; + /**< Bit-fields of present modifier see MLX5_FLOW_MOD_*. */ uint32_t fate; /**< Bit-fields of present fate see MLX5_FLOW_FATE_*. */ uint8_t l3_protocol; /**< valid when l3_protocol_en is set. */ @@ -984,6 +990,12 @@ mlx5_flow_action_drop(const struct rte_flow_action *action, action, "multiple fate actions are not" " supported"); + if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "drop is not compatible with" + " flag/mark action"); if (size < flow_size) mlx5_flow_spec_verbs_add(flow, &drop, size); flow->fate |= MLX5_FLOW_FATE_DROP; @@ -1036,6 +1048,161 @@ mlx5_flow_action_queue(struct rte_eth_dev *dev, return 0; } +/** + * Convert the @p action into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] action + * Action configuration. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p action has fully been + * converted, otherwise another call with this returned memory size should + * be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_action_flag(const struct rte_flow_action *action, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + unsigned int size = sizeof(struct ibv_flow_spec_action_tag); + struct ibv_flow_spec_action_tag tag = { + .type = IBV_FLOW_SPEC_ACTION_TAG, + .size = size, + .tag_id = mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT), + }; + + if (flow->modifier & MLX5_FLOW_MOD_FLAG) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "flag action already present"); + if (flow->fate & MLX5_FLOW_FATE_DROP) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "flag is not compatible with drop" + " action"); + if (flow->modifier & MLX5_FLOW_MOD_MARK) + return 0; + flow->modifier |= MLX5_FLOW_MOD_FLAG; + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &tag, size); + return size; +} + +/** + * Update verbs specification to modify the flag to mark. + * + * @param[in, out] flow + * Pointer to the rte_flow structure. + * @param[in] mark_id + * Mark identifier to replace the flag. + */ +static void +mlx5_flow_verbs_mark_update(struct rte_flow *flow, uint32_t mark_id) +{ + struct ibv_spec_header *hdr; + int i; + + /* Update Verbs specification. */ + hdr = (struct ibv_spec_header *)flow->verbs.specs; + if (!hdr) + return; + for (i = 0; i != flow->verbs.attr->num_of_specs; ++i) { + if (hdr->type == IBV_FLOW_SPEC_ACTION_TAG) { + struct ibv_flow_spec_action_tag *t = + (struct ibv_flow_spec_action_tag *)hdr; + + t->tag_id = mlx5_flow_mark_set(mark_id); + } + hdr = (struct ibv_spec_header *)((uintptr_t)hdr + hdr->size); + } +} + +/** + * Convert the @p action into @p flow (or by updating the already present + * Flag Verbs specification) after ensuring the NIC will understand and + * process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] action + * Action configuration. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p action has fully been + * converted, otherwise another call with this returned memory size should + * be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_action_mark(const struct rte_flow_action *action, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + const struct rte_flow_action_mark *mark = action->conf; + unsigned int size = sizeof(struct ibv_flow_spec_action_tag); + struct ibv_flow_spec_action_tag tag = { + .type = IBV_FLOW_SPEC_ACTION_TAG, + .size = size, + }; + + if (!mark) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "configuration cannot be null"); + if (mark->id >= MLX5_FLOW_MARK_MAX) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &mark->id, + "mark id must in 0 <= id < " + RTE_STR(MLX5_FLOW_MARK_MAX)); + if (flow->modifier & MLX5_FLOW_MOD_MARK) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "mark action already present"); + if (flow->fate & MLX5_FLOW_FATE_DROP) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "mark is not compatible with drop" + " action"); + if (flow->modifier & MLX5_FLOW_MOD_FLAG) { + mlx5_flow_verbs_mark_update(flow, mark->id); + size = 0; /**< Only an update is done in the specification. */ + } else { + tag.tag_id = mlx5_flow_mark_set(mark->id); + if (size <= flow_size) { + tag.tag_id = mlx5_flow_mark_set(mark->id); + mlx5_flow_spec_verbs_add(flow, &tag, size); + } + } + flow->modifier |= MLX5_FLOW_MOD_MARK; + return size; +} + /** * Convert the @p action into @p flow after ensuring the NIC will understand * and process it correctly. @@ -1077,6 +1244,14 @@ mlx5_flow_actions(struct rte_eth_dev *dev, switch (actions->type) { case RTE_FLOW_ACTION_TYPE_VOID: break; + case RTE_FLOW_ACTION_TYPE_FLAG: + ret = mlx5_flow_action_flag(actions, flow, remain, + error); + break; + case RTE_FLOW_ACTION_TYPE_MARK: + ret = mlx5_flow_action_mark(actions, flow, remain, + error); + break; case RTE_FLOW_ACTION_TYPE_DROP: ret = mlx5_flow_action_drop(actions, flow, remain, error); @@ -1170,6 +1345,79 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, return size; } +/** + * Mark the Rx queues mark flag if the flow has a mark or flag modifier. + * + * @param[in] dev + * Pointer to Ethernet device. + * @param[in] flow + * Pointer to flow structure. + */ +static void +mlx5_flow_rxq_mark_set(struct rte_eth_dev *dev, struct rte_flow *flow) +{ + struct priv *priv = dev->data->dev_private; + + if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) { + struct mlx5_rxq_ctrl *rxq_ctrl = + container_of((*priv->rxqs)[flow->queue], + struct mlx5_rxq_ctrl, rxq); + + rxq_ctrl->rxq.mark = 1; + rxq_ctrl->flow_mark_n++; + } +} + +/** + * Clear the Rx queue mark associated with the @p flow if no other flow uses + * it with a mark request. + * + * @param dev + * Pointer to Ethernet device. + * @param[in] flow + * Pointer to the flow. + */ +static void +mlx5_flow_rxq_mark_trim(struct rte_eth_dev *dev, struct rte_flow *flow) +{ + struct priv *priv = dev->data->dev_private; + + if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) { + struct mlx5_rxq_ctrl *rxq_ctrl = + container_of((*priv->rxqs)[flow->queue], + struct mlx5_rxq_ctrl, rxq); + + rxq_ctrl->flow_mark_n--; + rxq_ctrl->rxq.mark = !!rxq_ctrl->flow_mark_n; + } +} + +/** + * Clear the mark bit in all Rx queues. + * + * @param dev + * Pointer to Ethernet device. + */ +static void +mlx5_flow_rxq_mark_clear(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + unsigned int i; + unsigned int idx; + + for (idx = 0, i = 0; idx != priv->rxqs_n; ++i) { + struct mlx5_rxq_ctrl *rxq_ctrl; + + if (!(*priv->rxqs)[idx]) + continue; + rxq_ctrl = container_of((*priv->rxqs)[idx], + struct mlx5_rxq_ctrl, rxq); + rxq_ctrl->flow_mark_n = 0; + rxq_ctrl->rxq.mark = 0; + ++idx; + } +} + /** * Validate a flow supported by the NIC. * @@ -1329,6 +1577,7 @@ mlx5_flow_list_create(struct rte_eth_dev *dev, if (ret < 0) goto error; } + mlx5_flow_rxq_mark_set(dev, flow); TAILQ_INSERT_TAIL(list, flow, next); return flow; error: @@ -1373,6 +1622,7 @@ mlx5_flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, { mlx5_flow_remove(dev, flow); TAILQ_REMOVE(list, flow, next); + mlx5_flow_rxq_mark_trim(dev, flow); rte_free(flow); } @@ -1410,6 +1660,7 @@ mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) mlx5_flow_remove(dev, flow); + mlx5_flow_rxq_mark_clear(dev); } /** @@ -1434,6 +1685,7 @@ mlx5_flow_start(struct rte_eth_dev *dev, struct mlx5_flows *list) ret = mlx5_flow_apply(dev, flow, &error); if (ret < 0) goto error; + mlx5_flow_rxq_mark_set(dev, flow); } return 0; error: diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 708fdd4fa..99b769d8a 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -138,6 +138,7 @@ struct mlx5_rxq_ctrl { unsigned int socket; /* CPU socket ID for allocations. */ unsigned int irq:1; /* Whether IRQ is enabled. */ uint16_t idx; /* Queue index. */ + uint32_t flow_mark_n; /* Number of Mark/Flag flows using this Queue. */ }; /* Indirection table. */ From patchwork Wed Jul 11 07:22:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42806 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8A6021B54D; Wed, 11 Jul 2018 09:23:55 +0200 (CEST) Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by dpdk.org (Postfix) with ESMTP id 2BF571B4DA for ; Wed, 11 Jul 2018 09:23:30 +0200 (CEST) Received: by mail-wr1-f68.google.com with SMTP id b15-v6so17030118wrv.10 for ; Wed, 11 Jul 2018 00:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1c+WvFPh9dH6B8Z9Bv2qmfPMGLQA9MBNN50sBbthx0w=; b=yM/DBQWzH8cWwiJr3OQYHvF7CjEgOax8NMo/lfkbMWRyzeHxt3H4SPQrymqbcr3OL+ s2dZ/vz+gnc67ZVI6aBe7hZicdwjz8Ni4rmJbmcwM7qk16N/12jiI1skWFyFpARWQHVC AQomoQK5MV2x4U/nxP8C3TffNtgxz+c9Sxuj/SGtmngrgB3fcCPiaOAMuAaTEpA0OIxl nq9oYiKrGSKly9TugGTwt+B60fb7c13MrpB9/gqQbfE/8+Lb1Hf1LlWG9jiL5Az/c4o2 kWrGMlv1nmilgG9ZujjK8W1Djl4yYFiEKCawvJlJoDp1p3a+0iA+da1IyVWrdWV1554S 7gpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1c+WvFPh9dH6B8Z9Bv2qmfPMGLQA9MBNN50sBbthx0w=; b=jUcXG4PqbObIGiUbkxnHNOL+u35zskyVDW1N8YN8aV4fE2yAnNFdwVmgB4zaha0JOp fAA4eE5qIygt8I51wLN+1A6CYO7Wl7ggyVKr+3Gxu45AGzK+7FwuHf/hx4FCDfEhIESL crPpsnhD49L4QaIKZwIiJ9GnmuSq+8hHiYxFtWNwXEcshvtpIPB/77frnerb08vX6N8W eFU6ppcxAgs/vv3z6+7BKBgYaVPd86Nq15Mxw3vDwvl96kOloGqPL0W3GbmW+JftiEEH /zFmi6YbdcSpa8QU7X8kIGPsvk2w0onyrsEaDqfZtugP+MyjKRF2ctKaZ+6wAFiwyZAH 6o5Q== X-Gm-Message-State: AOUpUlGsFU2+OZ/6UgO1DgJzI9Gy89UQkBwj9VZrYqw9gILHRHDAvZIp 0Av9PrK/i3pN0tucIDOR7pK6aVtTsA== X-Google-Smtp-Source: AAOMgpcBdV6bw9SasmZcn4pdaOyQ5XqEmFepK9RQ74rMmVIwEtmCmI1VWdtuz6/Cj6Afbzser+4dhA== X-Received: by 2002:adf:f8c7:: with SMTP id f7-v6mr11833185wrq.237.1531293808653; Wed, 11 Jul 2018 00:23:28 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:27 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:46 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 13/21] net/mlx5: use a macro for the RSS key size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" ConnectX 4-5 support only 40 bytes of RSS key, using a compiled size hash key is not necessary. Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_ethdev.c | 14 +++++++------- drivers/net/mlx5/mlx5_flow.c | 4 ++-- drivers/net/mlx5/mlx5_prm.h | 3 +++ drivers/net/mlx5/mlx5_rss.c | 7 ++++--- drivers/net/mlx5/mlx5_rxq.c | 12 +++++++----- drivers/net/mlx5/mlx5_rxtx.h | 1 - 6 files changed, 23 insertions(+), 18 deletions(-) diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 05f66f7b6..6e44d5ff0 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -377,15 +377,15 @@ mlx5_dev_configure(struct rte_eth_dev *dev) if (use_app_rss_key && (dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len != - rss_hash_default_key_len)) { - DRV_LOG(ERR, "port %u RSS key len must be %zu Bytes long", - dev->data->port_id, rss_hash_default_key_len); + MLX5_RSS_HASH_KEY_LEN)) { + DRV_LOG(ERR, "port %u RSS key len must be %s Bytes long", + dev->data->port_id, RTE_STR(MLX5_RSS_HASH_KEY_LEN)); rte_errno = EINVAL; return -rte_errno; } priv->rss_conf.rss_key = rte_realloc(priv->rss_conf.rss_key, - rss_hash_default_key_len, 0); + MLX5_RSS_HASH_KEY_LEN, 0); if (!priv->rss_conf.rss_key) { DRV_LOG(ERR, "port %u cannot allocate RSS hash key memory (%u)", dev->data->port_id, rxqs_n); @@ -396,8 +396,8 @@ mlx5_dev_configure(struct rte_eth_dev *dev) use_app_rss_key ? dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key : rss_hash_default_key, - rss_hash_default_key_len); - priv->rss_conf.rss_key_len = rss_hash_default_key_len; + MLX5_RSS_HASH_KEY_LEN); + priv->rss_conf.rss_key_len = MLX5_RSS_HASH_KEY_LEN; priv->rss_conf.rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; priv->rxqs = (void *)dev->data->rx_queues; priv->txqs = (void *)dev->data->tx_queues; @@ -515,7 +515,7 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) info->if_index = if_nametoindex(ifname); info->reta_size = priv->reta_idx_n ? priv->reta_idx_n : config->ind_table_max_size; - info->hash_key_size = rss_hash_default_key_len; + info->hash_key_size = MLX5_RSS_HASH_KEY_LEN; info->speed_capa = priv->link_speed_capa; info->flow_type_rss_offloads = ~MLX5_RSS_HF_MASK; mlx5_set_default_params(dev, info); diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 1280db486..77483bd1f 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -1493,11 +1493,11 @@ mlx5_flow_apply(struct rte_eth_dev *dev, struct rte_flow *flow, struct mlx5_hrxq *hrxq; hrxq = mlx5_hrxq_get(dev, rss_hash_default_key, - rss_hash_default_key_len, 0, + MLX5_RSS_HASH_KEY_LEN, 0, &flow->queue, 1, 0, 0); if (!hrxq) hrxq = mlx5_hrxq_new(dev, rss_hash_default_key, - rss_hash_default_key_len, 0, + MLX5_RSS_HASH_KEY_LEN, 0, &flow->queue, 1, 0, 0); if (!hrxq) return rte_flow_error_set(error, rte_errno, diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index f9fae1e50..0870d32fd 100644 --- a/drivers/net/mlx5/mlx5_prm.h +++ b/drivers/net/mlx5/mlx5_prm.h @@ -21,6 +21,9 @@ #include #include "mlx5_autoconf.h" +/* RSS hash key size. */ +#define MLX5_RSS_HASH_KEY_LEN 40 + /* Get CQE owner bit. */ #define MLX5_CQE_OWNER(op_own) ((op_own) & MLX5_CQE_OWNER_MASK) diff --git a/drivers/net/mlx5/mlx5_rss.c b/drivers/net/mlx5/mlx5_rss.c index d69b4c09e..b95778a8c 100644 --- a/drivers/net/mlx5/mlx5_rss.c +++ b/drivers/net/mlx5/mlx5_rss.c @@ -50,10 +50,11 @@ mlx5_rss_hash_update(struct rte_eth_dev *dev, return -rte_errno; } if (rss_conf->rss_key && rss_conf->rss_key_len) { - if (rss_conf->rss_key_len != rss_hash_default_key_len) { + if (rss_conf->rss_key_len != MLX5_RSS_HASH_KEY_LEN) { DRV_LOG(ERR, - "port %u RSS key len must be %zu Bytes long", - dev->data->port_id, rss_hash_default_key_len); + "port %u RSS key len must be %s Bytes long", + dev->data->port_id, + RTE_STR(MLX5_RSS_HASH_KEY_LEN)); rte_errno = EINVAL; return -rte_errno; } diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index d960daa43..d50b82c69 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -52,7 +52,9 @@ uint8_t rss_hash_default_key[] = { }; /* Length of the default RSS hash key. */ -const size_t rss_hash_default_key_len = sizeof(rss_hash_default_key); +static_assert(MLX5_RSS_HASH_KEY_LEN == + (unsigned int)sizeof(rss_hash_default_key), + "wrong RSS default key size."); /** * Check whether Multi-Packet RQ can be enabled for the device. @@ -1771,7 +1773,7 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, return NULL; } if (!rss_key_len) { - rss_key_len = rss_hash_default_key_len; + rss_key_len = MLX5_RSS_HASH_KEY_LEN; rss_key = rss_hash_default_key; } #ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT @@ -1791,7 +1793,7 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, .rx_hash_conf = (struct ibv_rx_hash_conf){ .rx_hash_function = IBV_RX_HASH_FUNC_TOEPLITZ, .rx_hash_key_len = rss_key_len ? rss_key_len : - rss_hash_default_key_len, + MLX5_RSS_HASH_KEY_LEN, .rx_hash_key = rss_key ? (void *)(uintptr_t)rss_key : rss_hash_default_key, @@ -1815,7 +1817,7 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, .rx_hash_conf = (struct ibv_rx_hash_conf){ .rx_hash_function = IBV_RX_HASH_FUNC_TOEPLITZ, .rx_hash_key_len = rss_key_len ? rss_key_len : - rss_hash_default_key_len, + MLX5_RSS_HASH_KEY_LEN, .rx_hash_key = rss_key ? (void *)(uintptr_t)rss_key : rss_hash_default_key, @@ -2138,7 +2140,7 @@ mlx5_hrxq_drop_new(struct rte_eth_dev *dev) .rx_hash_conf = (struct ibv_rx_hash_conf){ .rx_hash_function = IBV_RX_HASH_FUNC_TOEPLITZ, - .rx_hash_key_len = rss_hash_default_key_len, + .rx_hash_key_len = MLX5_RSS_HASH_KEY_LEN, .rx_hash_key = rss_hash_default_key, .rx_hash_fields_mask = 0, }, diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 99b769d8a..59e374d8d 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -225,7 +225,6 @@ struct mlx5_txq_ctrl { /* mlx5_rxq.c */ extern uint8_t rss_hash_default_key[]; -extern const size_t rss_hash_default_key_len; int mlx5_check_mprq_support(struct rte_eth_dev *dev); int mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq); From patchwork Wed Jul 11 07:22:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42807 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 56AF41B551; Wed, 11 Jul 2018 09:23:58 +0200 (CEST) Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com [74.125.82.48]) by dpdk.org (Postfix) with ESMTP id CC4D11B4DA for ; Wed, 11 Jul 2018 09:23:30 +0200 (CEST) Received: by mail-wm0-f48.google.com with SMTP id z13-v6so1385926wma.5 for ; Wed, 11 Jul 2018 00:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=tTe4SWT7gjfInqPZWQo1QrZTRkJta3R7nGa7OQ4+Tlc=; b=ehP/tptJ+j0J+FnlOw79LFHqOmIF79siNcZw0PmmBe3Z+pjy2vuXcH1DNIFvn+Snrx ZRhm6K+0LCNzdyyWXPm4N2+xYJIVTKMLtPzJsS01cpf2Zzxiidq0CFjeKC9KZ2+6amZP 9ofC+EvXtyz+nRivdw6pCqWNqPtgBTTd1yXGX2axqAg1FPXtUYktHiSl1WJEtqGbo9xW m7tsatRYamP1rr2d28Zgf3Qh8ioRQjhIOywC4HRDgBPv6oU2OKBZeLHr04yWLosqWA76 LDAEdY3R061yWNHslQjs6Q16o658gGkCwkKHzRTg3gpO5E22bHy4zIJkYiloxon7kTP/ d/pA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=tTe4SWT7gjfInqPZWQo1QrZTRkJta3R7nGa7OQ4+Tlc=; b=trUf914+QuLCURzIwR3ZrUM/NovvW3c9T3HcRfB+C3cpOT0li9XV/RKZkgoY8pqelC GiEfOLVmNkvmIDRz0/Ok+Ndtpg3XYDPpUXBBLIEsssYiLR0N3SzBekQTg57I7HHH/WiB qYQJjFHc4Pr5odrsw6E85eMDkmLxhOE5BGedYbGz76G2/NzndtBtlRaqPAhKvaytBxh+ LSyZtMvprqc/SCJqx2csmLx1u99MwAQkl+nkJ0bR0H1AoKYN5uU0yWvULF12yo04fqC7 jP3UcabeZc9R5WZ6f1PFSVbV8aY199LeMldFAtxsdd6fMZyEGU26e2kjdHGVzibFgLWt XJWg== X-Gm-Message-State: APt69E0TUjalB8wMpJhZX5ctuPP7REh0NWRM47jmbrbWqcltsZdGQN5z gVMYd2Dk+h6KlB5qzclpPEjYHd4ryQ== X-Google-Smtp-Source: AAOMgpfKutqlxHCOmxyQCRJQtUX4RhmVT02xUeIqOhbDAwiwjiHtLkdosaBMlLkM7yJimj1uFT8rhw== X-Received: by 2002:a1c:98a:: with SMTP id 132-v6mr17116266wmj.86.1531293809822; Wed, 11 Jul 2018 00:23:29 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:28 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:47 +0200 Message-Id: <8b89f9d6ad4c0eeb9e1a050b8b0cc6562e16ca91.1531293415.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 14/21] net/mlx5: add RSS flow action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 682 +++++++++++++++++++++++++++-------- 1 file changed, 537 insertions(+), 145 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 77483bd1f..f098e727a 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -51,6 +51,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Actions that modify the fate of matching traffic. */ #define MLX5_FLOW_FATE_DROP (1u << 0) #define MLX5_FLOW_FATE_QUEUE (1u << 1) +#define MLX5_FLOW_FATE_RSS (1u << 2) /* Modify a packet. */ #define MLX5_FLOW_MOD_FLAG (1u << 0) @@ -60,8 +61,68 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; #define MLX5_IP_PROTOCOL_TCP 6 #define MLX5_IP_PROTOCOL_UDP 17 +/* Priority reserved for default flows. */ +#define MLX5_FLOW_PRIO_RSVD ((uint32_t)-1) + +enum mlx5_expansion { + MLX5_EXPANSION_ROOT, + MLX5_EXPANSION_ETH, + MLX5_EXPANSION_IPV4, + MLX5_EXPANSION_IPV4_UDP, + MLX5_EXPANSION_IPV4_TCP, + MLX5_EXPANSION_IPV6, + MLX5_EXPANSION_IPV6_UDP, + MLX5_EXPANSION_IPV6_TCP, +}; + +/** Supported expansion of items. */ +static const struct rte_flow_expand_node mlx5_support_expansion[] = { + [MLX5_EXPANSION_ROOT] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_ETH, + MLX5_EXPANSION_IPV4, + MLX5_EXPANSION_IPV6), + .type = RTE_FLOW_ITEM_TYPE_END, + }, + [MLX5_EXPANSION_ETH] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4, + MLX5_EXPANSION_IPV6), + .type = RTE_FLOW_ITEM_TYPE_ETH, + }, + [MLX5_EXPANSION_IPV4] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4_UDP, + MLX5_EXPANSION_IPV4_TCP), + .type = RTE_FLOW_ITEM_TYPE_IPV4, + .rss_types = ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | + ETH_RSS_NONFRAG_IPV4_OTHER, + }, + [MLX5_EXPANSION_IPV4_UDP] = { + .type = RTE_FLOW_ITEM_TYPE_UDP, + .rss_types = ETH_RSS_NONFRAG_IPV4_UDP, + }, + [MLX5_EXPANSION_IPV4_TCP] = { + .type = RTE_FLOW_ITEM_TYPE_TCP, + .rss_types = ETH_RSS_NONFRAG_IPV4_TCP, + }, + [MLX5_EXPANSION_IPV6] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV6_UDP, + MLX5_EXPANSION_IPV6_TCP), + .type = RTE_FLOW_ITEM_TYPE_IPV6, + .rss_types = ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 | + ETH_RSS_NONFRAG_IPV6_OTHER, + }, + [MLX5_EXPANSION_IPV6_UDP] = { + .type = RTE_FLOW_ITEM_TYPE_UDP, + .rss_types = ETH_RSS_NONFRAG_IPV6_UDP, + }, + [MLX5_EXPANSION_IPV6_TCP] = { + .type = RTE_FLOW_ITEM_TYPE_TCP, + .rss_types = ETH_RSS_NONFRAG_IPV6_TCP, + }, +}; + /** Handles information leading to a drop fate. */ struct mlx5_flow_verbs { + LIST_ENTRY(mlx5_flow_verbs) next; unsigned int size; /**< Size of the attribute. */ struct { struct ibv_flow_attr *attr; @@ -70,6 +131,7 @@ struct mlx5_flow_verbs { }; struct ibv_flow *flow; /**< Verbs flow pointer. */ struct mlx5_hrxq *hrxq; /**< Hash Rx queue object. */ + uint64_t hash_fields; /**< Verbs hash Rx queue hash fields. */ }; /* Flow structure. */ @@ -84,8 +146,12 @@ struct rte_flow { uint32_t fate; /**< Bit-fields of present fate see MLX5_FLOW_FATE_*. */ uint8_t l3_protocol; /**< valid when l3_protocol_en is set. */ - struct mlx5_flow_verbs verbs; /* Verbs flow. */ - uint16_t queue; /**< Destination queue to redirect traffic to. */ + LIST_HEAD(verbs, mlx5_flow_verbs) verbs; /**< Verbs flows list. */ + struct mlx5_flow_verbs *cur_verbs; + /**< Current Verbs flow structure being filled. */ + struct rte_flow_action_rss rss;/**< RSS context. */ + uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */ + uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */ }; static const struct rte_flow_ops mlx5_flow_ops = { @@ -128,16 +194,38 @@ struct ibv_spec_header { uint16_t size; }; - /** - * Discover the maximum number of priority available. - * - * @param[in] dev - * Pointer to Ethernet device. - * - * @return - * number of supported flow priority on success, a negative errno value - * otherwise and rte_errno is set. - */ +/* + * Number of sub priorities. + * For each kind of pattern matching i.e. L2, L3, L4 to have a correct + * matching on the NIC (firmware dependent) L4 most have the higher priority + * followed by L3 and ending with L2. + */ +#define MLX5_PRIORITY_MAP_L2 2 +#define MLX5_PRIORITY_MAP_L3 1 +#define MLX5_PRIORITY_MAP_L4 0 +#define MLX5_PRIORITY_MAP_MAX 3 + +/* Map of Verbs to Flow priority with 8 Verbs priorities. */ +static const uint32_t priority_map_3[][MLX5_PRIORITY_MAP_MAX] = { + { 0, 1, 2 }, { 2, 3, 4 }, { 5, 6, 7 }, +}; + +/* Map of Verbs to Flow priority with 16 Verbs priorities. */ +static const uint32_t priority_map_5[][MLX5_PRIORITY_MAP_MAX] = { + { 0, 1, 2 }, { 3, 4, 5 }, { 6, 7, 8 }, + { 9, 10, 11 }, { 12, 13, 14 }, +}; + +/** + * Discover the maximum number of priority available. + * + * @param[in] dev + * Pointer to Ethernet device. + * + * @return + * number of supported flow priority on success, a negative errno + * value otherwise and rte_errno is set. + */ int mlx5_flow_discover_priorities(struct rte_eth_dev *dev) { @@ -162,6 +250,7 @@ mlx5_flow_discover_priorities(struct rte_eth_dev *dev) struct mlx5_hrxq *drop = mlx5_hrxq_drop_new(dev); uint16_t vprio[] = { 8, 16 }; int i; + int priority = 0; if (!drop) { rte_errno = ENOTSUP; @@ -173,11 +262,52 @@ mlx5_flow_discover_priorities(struct rte_eth_dev *dev) if (!flow) break; claim_zero(mlx5_glue->destroy_flow(flow)); + priority = vprio[i]; + } + switch (priority) { + case 8: + priority = RTE_DIM(priority_map_3); + break; + case 16: + priority = RTE_DIM(priority_map_5); + break; + default: + rte_errno = ENOTSUP; + DRV_LOG(ERR, + "port %u verbs maximum priority: %d expected 8/16", + dev->data->port_id, vprio[i]); + return -rte_errno; } mlx5_hrxq_drop_release(dev); DRV_LOG(INFO, "port %u flow maximum priority: %d", - dev->data->port_id, vprio[i - 1]); - return vprio[i - 1]; + dev->data->port_id, priority); + return priority; +} + +/** + * Adjust flow priority. + * + * @param dev + * Pointer to Ethernet device. + * @param flow + * Pointer to an rte flow. + */ +static void +mlx5_flow_adjust_priority(struct rte_eth_dev *dev, struct rte_flow *flow) +{ + struct priv *priv = dev->data->dev_private; + uint32_t priority = flow->attributes.priority; + uint32_t subpriority = flow->cur_verbs->attr->priority; + + switch (priv->config.flow_prio) { + case RTE_DIM(priority_map_3): + priority = priority_map_3[priority][subpriority]; + break; + case RTE_DIM(priority_map_5): + priority = priority_map_5[priority][subpriority]; + break; + } + flow->cur_verbs->attr->priority = priority; } /** @@ -203,14 +333,15 @@ mlx5_flow_attributes(struct rte_eth_dev *dev, struct rte_flow_error *error) { uint32_t priority_max = - ((struct priv *)dev->data->dev_private)->config.flow_prio; + ((struct priv *)dev->data->dev_private)->config.flow_prio - 1; if (attributes->group) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_GROUP, NULL, "groups is not supported"); - if (attributes->priority >= priority_max) + if (attributes->priority != MLX5_FLOW_PRIO_RSVD && + attributes->priority >= priority_max) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, NULL, @@ -231,6 +362,8 @@ mlx5_flow_attributes(struct rte_eth_dev *dev, NULL, "ingress attribute is mandatory"); flow->attributes = *attributes; + if (attributes->priority == MLX5_FLOW_PRIO_RSVD) + flow->attributes.priority = priority_max; return 0; } @@ -296,7 +429,7 @@ mlx5_flow_item_acceptable(const struct rte_flow_item *item, } /** - * Add a verbs specification into @p flow. + * Add a verbs item specification into @p flow. * * @param[in, out] flow * Pointer to flow structure. @@ -308,14 +441,16 @@ mlx5_flow_item_acceptable(const struct rte_flow_item *item, static void mlx5_flow_spec_verbs_add(struct rte_flow *flow, void *src, unsigned int size) { - if (flow->verbs.specs) { + struct mlx5_flow_verbs *verbs = flow->cur_verbs; + + if (verbs->specs) { void *dst; - dst = (void *)(flow->verbs.specs + flow->verbs.size); + dst = (void *)(verbs->specs + verbs->size); memcpy(dst, src, size); - ++flow->verbs.attr->num_of_specs; + ++verbs->attr->num_of_specs; } - flow->verbs.size += size; + verbs->size += size; } /** @@ -390,6 +525,7 @@ mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, } eth.val.ether_type &= eth.mask.ether_type; } + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L2; mlx5_flow_spec_verbs_add(flow, ð, size); return size; } @@ -460,6 +596,7 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, .inner_type = RTE_BE16(0xffff), }; unsigned int size = sizeof(struct ibv_flow_spec_eth); + struct mlx5_flow_verbs *verbs = flow->cur_verbs; struct ibv_flow_spec_eth eth = { .type = IBV_FLOW_SPEC_ETH, .size = size, @@ -506,15 +643,16 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, item->spec, "VLAN cannot be empty"); if (!(flow->layers & l2m)) { - if (size <= flow_size) + if (size <= flow_size) { + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L2; mlx5_flow_spec_verbs_add(flow, ð, size); + } } else { - if (flow->verbs.attr) - mlx5_flow_item_vlan_update(flow->verbs.attr, ð); + if (verbs->attr) + mlx5_flow_item_vlan_update(verbs->attr, ð); size = 0; /* Only an update is done in eth specification. */ } - flow->layers |= MLX5_FLOW_LAYER_OUTER_L2 | - MLX5_FLOW_LAYER_OUTER_VLAN; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L2 | MLX5_FLOW_LAYER_OUTER_VLAN; return size; } @@ -601,8 +739,18 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, } flow->l3_protocol_en = !!ipv4.mask.proto; flow->l3_protocol = ipv4.val.proto; - if (size <= flow_size) + if (size <= flow_size) { + uint64_t hash_fields = IBV_RX_HASH_SRC_IPV4 | + IBV_RX_HASH_DST_IPV4; + + if (!(flow->rss.types & + (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | + ETH_RSS_NONFRAG_IPV4_OTHER))) + hash_fields = 0; + flow->cur_verbs->hash_fields |= hash_fields; + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L3; mlx5_flow_spec_verbs_add(flow, &ipv4, size); + } return size; } @@ -714,8 +862,17 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, } flow->l3_protocol_en = !!ipv6.mask.next_hdr; flow->l3_protocol = ipv6.val.next_hdr; - if (size <= flow_size) + if (size <= flow_size) { + uint64_t hash_fields = IBV_RX_HASH_SRC_IPV6 | + IBV_RX_HASH_DST_IPV6; + + if (!(flow->rss.types & + (ETH_RSS_IPV6 | ETH_RSS_NONFRAG_IPV6_OTHER))) + hash_fields = 0; + flow->cur_verbs->hash_fields |= hash_fields; + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L3; mlx5_flow_spec_verbs_add(flow, &ipv6, size); + } return size; } @@ -754,22 +911,24 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, }; int ret; - if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) + if (flow->l3_protocol_en && flow->l3_protocol != MLX5_IP_PROTOCOL_UDP) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, - "L3 is mandatory to filter on L4"); - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + "protocol filtering not compatible" + " with UDP layer"); + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, - "L4 layer is already present"); - if (flow->l3_protocol_en && flow->l3_protocol != MLX5_IP_PROTOCOL_UDP) + "L3 is mandatory to filter" + " on L4"); + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, - "protocol filtering not compatible" - " with UDP layer"); + "L4 layer is already" + " present"); if (!mask) mask = &rte_flow_item_udp_mask; ret = mlx5_flow_item_acceptable @@ -779,8 +938,6 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, if (ret < 0) return ret; flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_UDP; - if (size > flow_size) - return size; if (spec) { udp.val.dst_port = spec->hdr.dst_port; udp.val.src_port = spec->hdr.src_port; @@ -790,7 +947,16 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, udp.val.src_port &= udp.mask.src_port; udp.val.dst_port &= udp.mask.dst_port; } - mlx5_flow_spec_verbs_add(flow, &udp, size); + if (size <= flow_size) { + uint64_t hash_fields = IBV_RX_HASH_SRC_PORT_UDP | + IBV_RX_HASH_DST_PORT_UDP; + + if (!(flow->rss.types & ETH_RSS_UDP)) + hash_fields = 0; + flow->cur_verbs->hash_fields |= hash_fields; + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L4; + mlx5_flow_spec_verbs_add(flow, &udp, size); + } return size; } @@ -854,8 +1020,6 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, if (ret < 0) return ret; flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_TCP; - if (size > flow_size) - return size; if (spec) { tcp.val.dst_port = spec->hdr.dst_port; tcp.val.src_port = spec->hdr.src_port; @@ -865,7 +1029,16 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, tcp.val.src_port &= tcp.mask.src_port; tcp.val.dst_port &= tcp.mask.dst_port; } - mlx5_flow_spec_verbs_add(flow, &tcp, size); + if (size <= flow_size) { + uint64_t hash_fields = IBV_RX_HASH_SRC_PORT_TCP | + IBV_RX_HASH_DST_PORT_TCP; + + if (!(flow->rss.types & ETH_RSS_TCP)) + hash_fields = 0; + flow->cur_verbs->hash_fields |= hash_fields; + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L4; + mlx5_flow_spec_verbs_add(flow, &tcp, size); + } return size; } @@ -1043,11 +1216,95 @@ mlx5_flow_action_queue(struct rte_eth_dev *dev, RTE_FLOW_ERROR_TYPE_ACTION_CONF, &queue->index, "queue is not configured"); - flow->queue = queue->index; + if (flow->queue) + (*flow->queue)[0] = queue->index; + flow->rss.queue_num = 1; flow->fate |= MLX5_FLOW_FATE_QUEUE; return 0; } +/** + * Ensure the @p action will be understood and used correctly by the NIC. + * + * @param dev + * Pointer to Ethernet device structure. + * @param action[in] + * Pointer to flow actions array. + * @param flow[in, out] + * Pointer to the rte_flow structure. + * @param error[in, out] + * Pointer to error structure. + * + * @return + * On success @p flow->queue array and @p flow->rss are filled and valid. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_action_rss(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow *flow, + struct rte_flow_error *error) +{ + struct priv *priv = dev->data->dev_private; + const struct rte_flow_action_rss *rss = action->conf; + unsigned int i; + + if (flow->fate) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "multiple fate actions are not" + " supported"); + if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT && + rss->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->func, + "RSS hash function not supported"); + if (rss->level > 1) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->level, + "tunnel RSS is not supported"); + if (rss->key_len < MLX5_RSS_HASH_KEY_LEN) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->key_len, + "RSS hash key too small"); + if (rss->key_len > MLX5_RSS_HASH_KEY_LEN) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->key_len, + "RSS hash key too large"); + if (rss->queue_num > priv->config.ind_table_max_size) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->queue_num, + "number of queues too large"); + if (rss->types & MLX5_RSS_HF_MASK) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->types, + "some RSS protocols are not" + " supported"); + for (i = 0; i != rss->queue_num; ++i) { + if (!(*priv->rxqs)[rss->queue[i]]) + return rte_flow_error_set + (error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->queue[i], + "queue is not configured"); + } + if (flow->queue) + memcpy((*flow->queue), rss->queue, + rss->queue_num * sizeof(uint16_t)); + flow->rss.queue_num = rss->queue_num; + memcpy(flow->key, rss->key, MLX5_RSS_HASH_KEY_LEN); + flow->rss.types = rss->types; + flow->fate |= MLX5_FLOW_FATE_RSS; + return 0; +} + /** * Convert the @p action into a Verbs specification after ensuring the NIC * will understand and process it correctly. @@ -1082,6 +1339,7 @@ mlx5_flow_action_flag(const struct rte_flow_action *action, .size = size, .tag_id = mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT), }; + struct mlx5_flow_verbs *verbs = flow->cur_verbs; if (flow->modifier & MLX5_FLOW_MOD_FLAG) return rte_flow_error_set(error, ENOTSUP, @@ -1095,32 +1353,34 @@ mlx5_flow_action_flag(const struct rte_flow_action *action, "flag is not compatible with drop" " action"); if (flow->modifier & MLX5_FLOW_MOD_MARK) - return 0; - flow->modifier |= MLX5_FLOW_MOD_FLAG; - if (size <= flow_size) + size = 0; + else if (size <= flow_size && verbs) mlx5_flow_spec_verbs_add(flow, &tag, size); + flow->modifier |= MLX5_FLOW_MOD_FLAG; return size; } /** * Update verbs specification to modify the flag to mark. * - * @param[in, out] flow - * Pointer to the rte_flow structure. + * @param[in, out] verbs + * Pointer to the mlx5_flow_verbs structure. * @param[in] mark_id * Mark identifier to replace the flag. */ static void -mlx5_flow_verbs_mark_update(struct rte_flow *flow, uint32_t mark_id) +mlx5_flow_verbs_mark_update(struct mlx5_flow_verbs *verbs, uint32_t mark_id) { struct ibv_spec_header *hdr; int i; + if (!verbs) + return; /* Update Verbs specification. */ - hdr = (struct ibv_spec_header *)flow->verbs.specs; + hdr = (struct ibv_spec_header *)verbs->specs; if (!hdr) return; - for (i = 0; i != flow->verbs.attr->num_of_specs; ++i) { + for (i = 0; i != verbs->attr->num_of_specs; ++i) { if (hdr->type == IBV_FLOW_SPEC_ACTION_TAG) { struct ibv_flow_spec_action_tag *t = (struct ibv_flow_spec_action_tag *)hdr; @@ -1166,6 +1426,7 @@ mlx5_flow_action_mark(const struct rte_flow_action *action, .type = IBV_FLOW_SPEC_ACTION_TAG, .size = size, }; + struct mlx5_flow_verbs *verbs = flow->cur_verbs; if (!mark) return rte_flow_error_set(error, EINVAL, @@ -1190,14 +1451,11 @@ mlx5_flow_action_mark(const struct rte_flow_action *action, "mark is not compatible with drop" " action"); if (flow->modifier & MLX5_FLOW_MOD_FLAG) { - mlx5_flow_verbs_mark_update(flow, mark->id); - size = 0; /**< Only an update is done in the specification. */ - } else { + mlx5_flow_verbs_mark_update(verbs, mark->id); + size = 0; + } else if (size <= flow_size) { tag.tag_id = mlx5_flow_mark_set(mark->id); - if (size <= flow_size) { - tag.tag_id = mlx5_flow_mark_set(mark->id); - mlx5_flow_spec_verbs_add(flow, &tag, size); - } + mlx5_flow_spec_verbs_add(flow, &tag, size); } flow->modifier |= MLX5_FLOW_MOD_MARK; return size; @@ -1259,6 +1517,9 @@ mlx5_flow_actions(struct rte_eth_dev *dev, case RTE_FLOW_ACTION_TYPE_QUEUE: ret = mlx5_flow_action_queue(dev, actions, flow, error); break; + case RTE_FLOW_ACTION_TYPE_RSS: + ret = mlx5_flow_action_rss(dev, actions, flow, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -1322,26 +1583,122 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, struct rte_flow_error *error) { struct rte_flow local_flow = { .layers = 0, }; - size_t size = sizeof(*flow) + sizeof(struct ibv_flow_attr); - int remain = (flow_size > size) ? flow_size - size : 0; + size_t size = sizeof(*flow); + union { + struct rte_flow_expand_rss buf; + uint8_t buffer[2048]; + } expand_buffer; + struct rte_flow_expand_rss *buf = &expand_buffer.buf; + struct mlx5_flow_verbs *original_verbs = NULL; + size_t original_verbs_size = 0; + uint32_t original_layers = 0; int ret; + uint32_t i; - if (!remain) + if (size > flow_size) flow = &local_flow; ret = mlx5_flow_attributes(dev, attributes, flow, error); if (ret < 0) return ret; - ret = mlx5_flow_items(pattern, flow, remain, error); - if (ret < 0) - return ret; - size += ret; - remain = (flow_size > size) ? flow_size - size : 0; - ret = mlx5_flow_actions(dev, actions, flow, remain, error); + ret = mlx5_flow_actions(dev, actions, &local_flow, 0, error); if (ret < 0) return ret; - size += ret; + if (local_flow.rss.types) { + ret = rte_flow_expand_rss(buf, sizeof(expand_buffer.buffer), + pattern, local_flow.rss.types, + mlx5_support_expansion, + MLX5_EXPANSION_ROOT); + assert(ret > 0 && + (unsigned int)ret < sizeof(expand_buffer.buffer)); + } else { + buf->entries = 1; + buf->entry[0].pattern = (void *)(uintptr_t)pattern; + } + size += RTE_ALIGN_CEIL(local_flow.rss.queue_num * sizeof(uint16_t), + sizeof(void *)); if (size <= flow_size) - flow->verbs.attr->priority = flow->attributes.priority; + flow->queue = (void *)(flow + 1); + LIST_INIT(&flow->verbs); + flow->layers = 0; + flow->modifier = 0; + flow->fate = 0; + for (i = 0; i != buf->entries; ++i) { + size_t off = size; + size_t off2; + int pattern_start_idx = 0; + + flow->layers = original_layers; + size += sizeof(struct ibv_flow_attr) + + sizeof(struct mlx5_flow_verbs); + off2 = size; + if (size < flow_size) { + flow->cur_verbs = (void *)((uintptr_t)flow + off); + flow->cur_verbs->attr = (void *)(flow->cur_verbs + 1); + flow->cur_verbs->specs = + (void *)(flow->cur_verbs->attr + 1); + } + /* First iteration convert the pattern into Verbs. */ + if (i == 0) { + /* Actions don't need to be converted several time. */ + ret = mlx5_flow_actions(dev, actions, flow, + (size < flow_size) ? + flow_size - size : 0, + error); + if (ret < 0) + return ret; + size += ret; + } else { + /* + * Next iteration means the pattern has already been + * converted and an expansion is necessary to match + * the user RSS request. For that only the expanded + * items will be converted, the common part with the + * user pattern are just copied into the next buffer + * zone. + */ + const struct rte_flow_item *item = pattern; + + size += original_verbs_size; + if (size < flow_size) { + rte_memcpy(flow->cur_verbs->attr, + original_verbs->attr, + original_verbs_size + + sizeof(struct ibv_flow_attr)); + flow->cur_verbs->size = original_verbs_size; + } + if (pattern->type == RTE_FLOW_ITEM_TYPE_END) { + pattern_start_idx++; + } else { + for (item = pattern; + item->type != RTE_FLOW_ITEM_TYPE_END; + ++item) + pattern_start_idx++; + } + } + ret = mlx5_flow_items + ((const struct rte_flow_item *) + &buf->entry[i].pattern[pattern_start_idx], + flow, + (size < flow_size) ? flow_size - size : 0, error); + if (ret < 0) + return ret; + size += ret; + if (size <= flow_size) { + mlx5_flow_adjust_priority(dev, flow); + LIST_INSERT_HEAD(&flow->verbs, flow->cur_verbs, next); + } + /* + * Keep a pointer of the first verbs conversion and the layers + * it has encountered. + */ + if (i == 0) { + original_verbs = flow->cur_verbs; + original_verbs_size = size - off2; + original_layers = flow->layers; + } + } + /* Restore the origin layers in the flow. */ + flow->layers = original_layers; return size; } @@ -1359,12 +1716,17 @@ mlx5_flow_rxq_mark_set(struct rte_eth_dev *dev, struct rte_flow *flow) struct priv *priv = dev->data->dev_private; if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) { - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of((*priv->rxqs)[flow->queue], - struct mlx5_rxq_ctrl, rxq); + unsigned int i; + + for (i = 0; i != flow->rss.queue_num; ++i) { + int idx = (*flow->queue)[i]; + struct mlx5_rxq_ctrl *rxq_ctrl = + container_of((*priv->rxqs)[idx], + struct mlx5_rxq_ctrl, rxq); - rxq_ctrl->rxq.mark = 1; - rxq_ctrl->flow_mark_n++; + rxq_ctrl->rxq.mark = 1; + rxq_ctrl->flow_mark_n++; + } } } @@ -1383,12 +1745,17 @@ mlx5_flow_rxq_mark_trim(struct rte_eth_dev *dev, struct rte_flow *flow) struct priv *priv = dev->data->dev_private; if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) { - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of((*priv->rxqs)[flow->queue], - struct mlx5_rxq_ctrl, rxq); + unsigned int i; + + for (i = 0; i != flow->rss.queue_num; ++i) { + int idx = (*flow->queue)[i]; + struct mlx5_rxq_ctrl *rxq_ctrl = + container_of((*priv->rxqs)[idx], + struct mlx5_rxq_ctrl, rxq); - rxq_ctrl->flow_mark_n--; - rxq_ctrl->rxq.mark = !!rxq_ctrl->flow_mark_n; + rxq_ctrl->flow_mark_n--; + rxq_ctrl->rxq.mark = !!rxq_ctrl->flow_mark_n; + } } } @@ -1449,18 +1816,20 @@ mlx5_flow_validate(struct rte_eth_dev *dev, static void mlx5_flow_remove(struct rte_eth_dev *dev, struct rte_flow *flow) { - if (flow->fate & MLX5_FLOW_FATE_DROP) { - if (flow->verbs.flow) { - claim_zero(mlx5_glue->destroy_flow(flow->verbs.flow)); - flow->verbs.flow = NULL; + struct mlx5_flow_verbs *verbs; + + LIST_FOREACH(verbs, &flow->verbs, next) { + if (verbs->flow) { + claim_zero(mlx5_glue->destroy_flow(verbs->flow)); + verbs->flow = NULL; + } + if (verbs->hrxq) { + if (flow->fate & MLX5_FLOW_FATE_DROP) + mlx5_hrxq_drop_release(dev); + else + mlx5_hrxq_release(dev, verbs->hrxq); + verbs->hrxq = NULL; } - } - if (flow->verbs.hrxq) { - if (flow->fate & MLX5_FLOW_FATE_DROP) - mlx5_hrxq_drop_release(dev); - else if (flow->fate & MLX5_FLOW_FATE_QUEUE) - mlx5_hrxq_release(dev, flow->verbs.hrxq); - flow->verbs.hrxq = NULL; } } @@ -1481,46 +1850,68 @@ static int mlx5_flow_apply(struct rte_eth_dev *dev, struct rte_flow *flow, struct rte_flow_error *error) { - if (flow->fate & MLX5_FLOW_FATE_DROP) { - flow->verbs.hrxq = mlx5_hrxq_drop_new(dev); - if (!flow->verbs.hrxq) - return rte_flow_error_set - (error, errno, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot allocate Drop queue"); - } else if (flow->fate & MLX5_FLOW_FATE_QUEUE) { - struct mlx5_hrxq *hrxq; - - hrxq = mlx5_hrxq_get(dev, rss_hash_default_key, - MLX5_RSS_HASH_KEY_LEN, 0, - &flow->queue, 1, 0, 0); - if (!hrxq) - hrxq = mlx5_hrxq_new(dev, rss_hash_default_key, - MLX5_RSS_HASH_KEY_LEN, 0, - &flow->queue, 1, 0, 0); - if (!hrxq) - return rte_flow_error_set(error, rte_errno, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot create flow"); - flow->verbs.hrxq = hrxq; - } - flow->verbs.flow = - mlx5_glue->create_flow(flow->verbs.hrxq->qp, flow->verbs.attr); - if (!flow->verbs.flow) { - if (flow->fate & MLX5_FLOW_FATE_DROP) - mlx5_hrxq_drop_release(dev); - else - mlx5_hrxq_release(dev, flow->verbs.hrxq); - flow->verbs.hrxq = NULL; - return rte_flow_error_set(error, errno, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "kernel module refuses to create" - " flow"); + struct mlx5_flow_verbs *verbs; + int err; + + LIST_FOREACH(verbs, &flow->verbs, next) { + if (flow->fate & MLX5_FLOW_FATE_DROP) { + verbs->hrxq = mlx5_hrxq_drop_new(dev); + if (!verbs->hrxq) { + rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot get drop hash queue"); + goto error; + } + } else { + struct mlx5_hrxq *hrxq; + + hrxq = mlx5_hrxq_get(dev, flow->key, + MLX5_RSS_HASH_KEY_LEN, + verbs->hash_fields, + (*flow->queue), + flow->rss.queue_num, 0, 0); + if (!hrxq) + hrxq = mlx5_hrxq_new(dev, flow->key, + MLX5_RSS_HASH_KEY_LEN, + verbs->hash_fields, + (*flow->queue), + flow->rss.queue_num, 0, 0); + if (!hrxq) { + rte_flow_error_set + (error, rte_errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot get hash queue"); + goto error; + } + verbs->hrxq = hrxq; + } + verbs->flow = + mlx5_glue->create_flow(verbs->hrxq->qp, verbs->attr); + if (!verbs->flow) { + rte_flow_error_set(error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "hardware refuses to create flow"); + goto error; + } } return 0; +error: + err = rte_errno; /* Save rte_errno before cleanup. */ + LIST_FOREACH(verbs, &flow->verbs, next) { + if (verbs->hrxq) { + if (flow->fate & MLX5_FLOW_FATE_DROP) + mlx5_hrxq_drop_release(dev); + else + mlx5_hrxq_release(dev, verbs->hrxq); + verbs->hrxq = NULL; + } + } + rte_errno = err; /* Restore rte_errno. */ + return -rte_errno; } /** @@ -1550,42 +1941,43 @@ mlx5_flow_list_create(struct rte_eth_dev *dev, const struct rte_flow_action actions[], struct rte_flow_error *error) { - struct rte_flow *flow; - size_t size; + struct rte_flow *flow = NULL; + size_t size = 0; int ret; - ret = mlx5_flow_merge(dev, NULL, 0, attr, items, actions, error); + ret = mlx5_flow_merge(dev, flow, size, attr, items, actions, error); if (ret < 0) return NULL; size = ret; - flow = rte_zmalloc(__func__, size, 0); + flow = rte_calloc(__func__, 1, size, 0); if (!flow) { rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, - "cannot allocate memory"); + "not enough memory to create flow"); return NULL; } - flow->verbs.attr = (struct ibv_flow_attr *)(flow + 1); - flow->verbs.specs = (uint8_t *)(flow->verbs.attr + 1); ret = mlx5_flow_merge(dev, flow, size, attr, items, actions, error); - if (ret < 0) - goto error; + if (ret < 0) { + rte_free(flow); + return NULL; + } assert((size_t)ret == size); if (dev->data->dev_started) { ret = mlx5_flow_apply(dev, flow, error); - if (ret < 0) - goto error; + if (ret < 0) { + ret = rte_errno; /* Save rte_errno before cleanup. */ + if (flow) { + mlx5_flow_remove(dev, flow); + rte_free(flow); + } + rte_errno = ret; /* Restore rte_errno. */ + return NULL; + } } mlx5_flow_rxq_mark_set(dev, flow); TAILQ_INSERT_TAIL(list, flow, next); return flow; -error: - ret = rte_errno; /* Save rte_errno before cleanup. */ - mlx5_flow_remove(dev, flow); - rte_free(flow); - rte_errno = ret; /* Restore rte_errno. */ - return NULL; } /** @@ -1745,7 +2137,7 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_attr attr = { .ingress = 1, - .priority = priv->config.flow_prio - 1, + .priority = MLX5_FLOW_PRIO_RSVD, }; struct rte_flow_item items[] = { { From patchwork Wed Jul 11 07:22:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42808 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1B4C81B558; Wed, 11 Jul 2018 09:24:00 +0200 (CEST) Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by dpdk.org (Postfix) with ESMTP id F082F1B447 for ; Wed, 11 Jul 2018 09:23:30 +0200 (CEST) Received: by mail-wr1-f65.google.com with SMTP id q10-v6so17002359wrd.4 for ; Wed, 11 Jul 2018 00:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kJn5IYGw8svVfxeAoEPIjP9eACpfPLMlZjJgn8CcUcI=; b=XgmpX4nW9xDU3ibHCqpLjeyNBmplqUnqrH7Y0jswEBAJLvD2OHXyXjvQIHw6rgdYzU 4ZDf6yJlXDFnQOQds6tkzAc4ZWiabDYkUcYAD4CyrdZ4OmZcDjfDumyh/hSfoIuaVzVv /kHvRPin3PZXQh7zYl0KC02viuwl4w+FAX+Gal94dtnSKlBEHAF1tLCFRvRJpT5Gj506 39GfFzZSrgpstF7+GjTB3iZK2gtqn2cUzfsNOyOO2EJKpn8T45PBkrSmLxDgcGIhoMI3 BmWAJxfI/LsB1wJY4x3PHrGzOQwf9PfyHqd54IBlT256CuPXBSRpl4oxLoduxuw3I7kg 02WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kJn5IYGw8svVfxeAoEPIjP9eACpfPLMlZjJgn8CcUcI=; b=LkRH1+ED5/OoA/GRL49dSfxgW26SM5eJLZySr8gyW/EDwIrq9j18eWmbkvVctMQB07 CzZ9cPQATr3nQEkkrvwuQwcxw1fTohUw4Ux4E0VFEZLSr+qmqSHjM8UlkAq/bLVseB5I BQRw6JV7xWIbU0cn0rRfaZ0ynC1zgzuOlKROMoLO/YArUaL+fCEIruUk13aBcY9cKrGD 7vKstRWnfPxttg0wixY8uT1K57VZBMPuzRyuNeKi+3iE475GLSkI7gQ3246PUrZ2J0cb Ovb1W5A0OAL95JVWDYJOvELT9rNy8s5RPTqwLMFElGpS5fSgJSz6fTJG231Y6R56Sesn s/Pg== X-Gm-Message-State: APt69E1IFywrt5UVEXjIcsReZuXNvA50C7KV2FCsJ0i4kqXjGfbRHX8i ary8Qt+wjdq5NxzVPRe9ZtLe5B0m5Q== X-Google-Smtp-Source: AAOMgpdxHwMBWQT0Nj57FDQqpANXcvlaVYLzd4DimlcqvYt+gPASX/ymwYOQ/WR42KwRt6vW4C/ChA== X-Received: by 2002:adf:8ab0:: with SMTP id y45-v6mr18820518wry.98.1531293810555; Wed, 11 Jul 2018 00:23:30 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:30 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:48 +0200 Message-Id: <338ecaa26ab6ca75a554fa22d9d620e3feb29314.1531293415.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 15/21] net/mlx5: remove useless arguments in hrxq API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" RSS level is necessary to had a bit in the hash_fields which is already provided in this API, for the tunnel, it is necessary to request such queue to compute the checksum on the inner most, this last one should always be activated. Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 4 ++-- drivers/net/mlx5/mlx5_rxq.c | 39 +++++++++--------------------------- drivers/net/mlx5/mlx5_rxtx.h | 8 ++------ 3 files changed, 13 insertions(+), 38 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index f098e727a..f2acac1f5 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -1871,13 +1871,13 @@ mlx5_flow_apply(struct rte_eth_dev *dev, struct rte_flow *flow, MLX5_RSS_HASH_KEY_LEN, verbs->hash_fields, (*flow->queue), - flow->rss.queue_num, 0, 0); + flow->rss.queue_num); if (!hrxq) hrxq = mlx5_hrxq_new(dev, flow->key, MLX5_RSS_HASH_KEY_LEN, verbs->hash_fields, (*flow->queue), - flow->rss.queue_num, 0, 0); + flow->rss.queue_num); if (!hrxq) { rte_flow_error_set (error, rte_errno, diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index d50b82c69..071740b6d 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1740,10 +1740,6 @@ mlx5_ind_table_ibv_verify(struct rte_eth_dev *dev) * first queue index will be taken for the indirection table. * @param queues_n * Number of queues. - * @param tunnel - * Tunnel type, implies tunnel offloading like inner checksum if available. - * @param rss_level - * RSS hash on tunnel level. * * @return * The Verbs object initialised, NULL otherwise and rte_errno is set. @@ -1752,17 +1748,13 @@ struct mlx5_hrxq * mlx5_hrxq_new(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, - const uint16_t *queues, uint32_t queues_n, - uint32_t tunnel, uint32_t rss_level) + const uint16_t *queues, uint32_t queues_n) { struct priv *priv = dev->data->dev_private; struct mlx5_hrxq *hrxq; struct mlx5_ind_table_ibv *ind_tbl; struct ibv_qp *qp; int err; -#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - struct mlx5dv_qp_init_attr qp_init_attr = {0}; -#endif queues_n = hash_fields ? queues_n : 1; ind_tbl = mlx5_ind_table_ibv_get(dev, queues, queues_n); @@ -1777,11 +1769,6 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, rss_key = rss_hash_default_key; } #ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - if (tunnel) { - qp_init_attr.comp_mask = - MLX5DV_QP_INIT_ATTR_MASK_QP_CREATE_FLAGS; - qp_init_attr.create_flags = MLX5DV_QP_CREATE_TUNNEL_OFFLOADS; - } qp = mlx5_glue->dv_create_qp (priv->ctx, &(struct ibv_qp_init_attr_ex){ @@ -1797,14 +1784,17 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, .rx_hash_key = rss_key ? (void *)(uintptr_t)rss_key : rss_hash_default_key, - .rx_hash_fields_mask = hash_fields | - (tunnel && rss_level > 1 ? - (uint32_t)IBV_RX_HASH_INNER : 0), + .rx_hash_fields_mask = hash_fields, }, .rwq_ind_tbl = ind_tbl->ind_table, .pd = priv->pd, }, - &qp_init_attr); + &(struct mlx5dv_qp_init_attr){ + .comp_mask = (hash_fields & IBV_RX_HASH_INNER) ? + MLX5DV_QP_INIT_ATTR_MASK_QP_CREATE_FLAGS : + 0, + .create_flags = MLX5DV_QP_CREATE_TUNNEL_OFFLOADS, + }); #else qp = mlx5_glue->create_qp_ex (priv->ctx, @@ -1838,8 +1828,6 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, hrxq->qp = qp; hrxq->rss_key_len = rss_key_len; hrxq->hash_fields = hash_fields; - hrxq->tunnel = tunnel; - hrxq->rss_level = rss_level; memcpy(hrxq->rss_key, rss_key, rss_key_len); rte_atomic32_inc(&hrxq->refcnt); LIST_INSERT_HEAD(&priv->hrxqs, hrxq, next); @@ -1865,10 +1853,6 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, * first queue index will be taken for the indirection table. * @param queues_n * Number of queues. - * @param tunnel - * Tunnel type, implies tunnel offloading like inner checksum if available. - * @param rss_level - * RSS hash on tunnel level * * @return * An hash Rx queue on success. @@ -1877,8 +1861,7 @@ struct mlx5_hrxq * mlx5_hrxq_get(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, - const uint16_t *queues, uint32_t queues_n, - uint32_t tunnel, uint32_t rss_level) + const uint16_t *queues, uint32_t queues_n) { struct priv *priv = dev->data->dev_private; struct mlx5_hrxq *hrxq; @@ -1893,10 +1876,6 @@ mlx5_hrxq_get(struct rte_eth_dev *dev, continue; if (hrxq->hash_fields != hash_fields) continue; - if (hrxq->tunnel != tunnel) - continue; - if (hrxq->rss_level != rss_level) - continue; ind_tbl = mlx5_ind_table_ibv_get(dev, queues, queues_n); if (!ind_tbl) continue; diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 59e374d8d..808118e50 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -157,8 +157,6 @@ struct mlx5_hrxq { struct mlx5_ind_table_ibv *ind_table; /* Indirection table. */ struct ibv_qp *qp; /* Verbs queue pair. */ uint64_t hash_fields; /* Verbs Hash fields. */ - uint32_t tunnel; /* Tunnel type. */ - uint32_t rss_level; /* RSS on tunnel level. */ uint32_t rss_key_len; /* Hash key length in bytes. */ uint8_t rss_key[]; /* Hash key. */ }; @@ -271,13 +269,11 @@ void mlx5_ind_table_ibv_drop_release(struct rte_eth_dev *dev); struct mlx5_hrxq *mlx5_hrxq_new(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, - const uint16_t *queues, uint32_t queues_n, - uint32_t tunnel, uint32_t rss_level); + const uint16_t *queues, uint32_t queues_n); struct mlx5_hrxq *mlx5_hrxq_get(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, - const uint16_t *queues, uint32_t queues_n, - uint32_t tunnel, uint32_t rss_level); + const uint16_t *queues, uint32_t queues_n); int mlx5_hrxq_release(struct rte_eth_dev *dev, struct mlx5_hrxq *hxrq); int mlx5_hrxq_ibv_verify(struct rte_eth_dev *dev); struct mlx5_hrxq *mlx5_hrxq_drop_new(struct rte_eth_dev *dev); From patchwork Wed Jul 11 07:22:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42809 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2D7901B56B; Wed, 11 Jul 2018 09:24:02 +0200 (CEST) Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by dpdk.org (Postfix) with ESMTP id 6CE5B1B447 for ; Wed, 11 Jul 2018 09:23:32 +0200 (CEST) Received: by mail-wm0-f68.google.com with SMTP id s12-v6so1361407wmc.1 for ; Wed, 11 Jul 2018 00:23:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YVdXU5pci31tgKwmq3O3Yl17jgWI6rB1zRfabSgQRz0=; b=Bau1jGF8iD+2ZXeWNvmb1SM2ujh6j33t/rGtM4qkgJoK7l4jFkdjSUJCYukvBa0BXo RXs6ATnNxDVwqaZ0WXb/q5CLbuRwrOw08iIJCVmDRMjF03OnyJVbQW6qmys04H2dIC50 ln1p27wfAykBSdn+JOV76AFKRA1d1x102ibc/gyRyWTy74CjgGgyfn8PUrX8rgCm9BsF BWISeAMRcls+WuLG5KRAO4hdsIj1Y592kcmPNA0kQTZ06FUfmxZ7sGCc8AbQe5/aGbsR mp9ikXUuiVh96vYcxxoH5QNxrRzklkqaq1kthmWVRrgcy9tl7gnjJrWGhCecdU7uie+t EHOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YVdXU5pci31tgKwmq3O3Yl17jgWI6rB1zRfabSgQRz0=; b=gSaudH7ZAAm5Ee2MivTmX+kpW/87+Lh6XURAbk8q3x7CJzcKhS1dwT4hZ9K0z3cPF1 8h/CcqxLbXWk5j9MvejJO1pZOoH+VxquVlqwOpAvA1l8sDpgMZGeiriKnoiPhsP3ZKJX cu4mZHdZ9/4KD5NNpQoP84ycI4Sg1DhVfChXLdI6C/7TaFDOypm3VIu05hrW1RbXm+ui ojUc2w5Td0jzZe35I0Uzl6mnPO3xX2nnWcUDzYE4AGEGebDs42so6l819Ih/0fxHn0Sn mG58Fb4WxaQuizRUhk0DHymmRZzjZ4mCMMBrsZd5wM7RZUQ1ma/iU+lkF66s9hLYIV0a ph7w== X-Gm-Message-State: APt69E3qRkNCwU2vndrykUycgjodGuaRGzJ3QaTbgHhiV85zD+ZQI04w 0N2LknZif7LHr+gf4eGAL64l5ULn1w== X-Google-Smtp-Source: AAOMgpeGMJ9UmMNjQycx5iV111XuVGxh5KOSPspUawAIspgJVLhXcIm8RuNidn86ChmE3wJDdpXyvg== X-Received: by 2002:a1c:ec9d:: with SMTP id h29-v6mr17484818wmi.94.1531293811773; Wed, 11 Jul 2018 00:23:31 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:31 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:49 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 16/21] net/mlx5: support inner RSS computation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 241 ++++++++++++++++++++++++++--------- 1 file changed, 180 insertions(+), 61 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index f2acac1f5..edceb17ed 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -35,18 +35,42 @@ extern const struct eth_dev_ops mlx5_dev_ops; extern const struct eth_dev_ops mlx5_dev_ops_isolate; -/* Pattern Layer bits. */ +/* Pattern outer Layer bits. */ #define MLX5_FLOW_LAYER_OUTER_L2 (1u << 0) #define MLX5_FLOW_LAYER_OUTER_L3_IPV4 (1u << 1) #define MLX5_FLOW_LAYER_OUTER_L3_IPV6 (1u << 2) #define MLX5_FLOW_LAYER_OUTER_L4_UDP (1u << 3) #define MLX5_FLOW_LAYER_OUTER_L4_TCP (1u << 4) #define MLX5_FLOW_LAYER_OUTER_VLAN (1u << 5) -/* Masks. */ + +/* Pattern inner Layer bits. */ +#define MLX5_FLOW_LAYER_INNER_L2 (1u << 6) +#define MLX5_FLOW_LAYER_INNER_L3_IPV4 (1u << 7) +#define MLX5_FLOW_LAYER_INNER_L3_IPV6 (1u << 8) +#define MLX5_FLOW_LAYER_INNER_L4_UDP (1u << 9) +#define MLX5_FLOW_LAYER_INNER_L4_TCP (1u << 10) +#define MLX5_FLOW_LAYER_INNER_VLAN (1u << 11) + +/* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6) #define MLX5_FLOW_LAYER_OUTER_L4 \ (MLX5_FLOW_LAYER_OUTER_L4_UDP | MLX5_FLOW_LAYER_OUTER_L4_TCP) +#define MLX5_FLOW_LAYER_OUTER \ + (MLX5_FLOW_LAYER_OUTER_L2 | MLX5_FLOW_LAYER_OUTER_L3 | \ + MLX5_FLOW_LAYER_OUTER_L4) + +/* Tunnel Masks. */ +#define MLX5_FLOW_LAYER_TUNNEL 0 + +/* Inner Masks. */ +#define MLX5_FLOW_LAYER_INNER_L3 \ + (MLX5_FLOW_LAYER_INNER_L3_IPV4 | MLX5_FLOW_LAYER_INNER_L3_IPV6) +#define MLX5_FLOW_LAYER_INNER_L4 \ + (MLX5_FLOW_LAYER_INNER_L4_UDP | MLX5_FLOW_LAYER_INNER_L4_TCP) +#define MLX5_FLOW_LAYER_INNER \ + (MLX5_FLOW_LAYER_INNER_L2 | MLX5_FLOW_LAYER_INNER_L3 | \ + MLX5_FLOW_LAYER_INNER_L4) /* Actions that modify the fate of matching traffic. */ #define MLX5_FLOW_FATE_DROP (1u << 0) @@ -66,6 +90,14 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; enum mlx5_expansion { MLX5_EXPANSION_ROOT, + MLX5_EXPANSION_ROOT_OUTER, + MLX5_EXPANSION_OUTER_ETH, + MLX5_EXPANSION_OUTER_IPV4, + MLX5_EXPANSION_OUTER_IPV4_UDP, + MLX5_EXPANSION_OUTER_IPV4_TCP, + MLX5_EXPANSION_OUTER_IPV6, + MLX5_EXPANSION_OUTER_IPV6_UDP, + MLX5_EXPANSION_OUTER_IPV6_TCP, MLX5_EXPANSION_ETH, MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV4_UDP, @@ -83,6 +115,50 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { MLX5_EXPANSION_IPV6), .type = RTE_FLOW_ITEM_TYPE_END, }, + [MLX5_EXPANSION_ROOT_OUTER] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_OUTER_ETH, + MLX5_EXPANSION_OUTER_IPV4, + MLX5_EXPANSION_OUTER_IPV6), + .type = RTE_FLOW_ITEM_TYPE_END, + }, + [MLX5_EXPANSION_OUTER_ETH] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_OUTER_IPV4, + MLX5_EXPANSION_OUTER_IPV6), + .type = RTE_FLOW_ITEM_TYPE_ETH, + .rss_types = 0, + }, + [MLX5_EXPANSION_OUTER_IPV4] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT + (MLX5_EXPANSION_OUTER_IPV4_UDP, + MLX5_EXPANSION_OUTER_IPV4_TCP), + .type = RTE_FLOW_ITEM_TYPE_IPV4, + .rss_types = ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | + ETH_RSS_NONFRAG_IPV4_OTHER, + }, + [MLX5_EXPANSION_OUTER_IPV4_UDP] = { + .type = RTE_FLOW_ITEM_TYPE_UDP, + .rss_types = ETH_RSS_NONFRAG_IPV4_UDP, + }, + [MLX5_EXPANSION_OUTER_IPV4_TCP] = { + .type = RTE_FLOW_ITEM_TYPE_TCP, + .rss_types = ETH_RSS_NONFRAG_IPV4_TCP, + }, + [MLX5_EXPANSION_OUTER_IPV6] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT + (MLX5_EXPANSION_OUTER_IPV6_UDP, + MLX5_EXPANSION_OUTER_IPV6_TCP), + .type = RTE_FLOW_ITEM_TYPE_IPV6, + .rss_types = ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 | + ETH_RSS_NONFRAG_IPV6_OTHER, + }, + [MLX5_EXPANSION_OUTER_IPV6_UDP] = { + .type = RTE_FLOW_ITEM_TYPE_UDP, + .rss_types = ETH_RSS_NONFRAG_IPV6_UDP, + }, + [MLX5_EXPANSION_OUTER_IPV6_TCP] = { + .type = RTE_FLOW_ITEM_TYPE_TCP, + .rss_types = ETH_RSS_NONFRAG_IPV6_TCP, + }, [MLX5_EXPANSION_ETH] = { .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV6), @@ -453,6 +529,32 @@ mlx5_flow_spec_verbs_add(struct rte_flow *flow, void *src, unsigned int size) verbs->size += size; } +/** + * Adjust verbs hash fields according to the @p flow information. + * + * @param[in, out] flow. + * Pointer to flow structure. + * @param[in] tunnel + * 1 when the hash field is for a tunnel item. + * @param[in] layer_types + * ETH_RSS_* types. + * @param[in] hash_fields + * Item hash fields. + */ +static void +mlx5_flow_verbs_hashfields_adjust(struct rte_flow *flow, int tunnel, + uint32_t layer_types, uint64_t hash_fields) +{ + hash_fields |= (tunnel ? IBV_RX_HASH_INNER : 0); + if (!(flow->rss.types & layer_types)) + hash_fields = 0; + if (flow->rss.level == 2 && !tunnel) + hash_fields = 0; + else if (flow->rss.level < 2 && tunnel) + hash_fields = 0; + flow->cur_verbs->hash_fields |= hash_fields; +} + /** * Convert the @p item into a Verbs specification after ensuring the NIC * will understand and process it correctly. @@ -486,14 +588,16 @@ mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, .src.addr_bytes = "\xff\xff\xff\xff\xff\xff", .type = RTE_BE16(0xffff), }; + const int tunnel = !!(flow->layers & MLX5_FLOW_LAYER_TUNNEL); const unsigned int size = sizeof(struct ibv_flow_spec_eth); struct ibv_flow_spec_eth eth = { - .type = IBV_FLOW_SPEC_ETH, + .type = IBV_FLOW_SPEC_ETH | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L2) + if (flow->layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L2 : + MLX5_FLOW_LAYER_OUTER_L2)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -506,7 +610,8 @@ mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, error); if (ret) return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L2; + flow->layers |= tunnel ? MLX5_FLOW_LAYER_INNER_L2 : + MLX5_FLOW_LAYER_OUTER_L2; if (size > flow_size) return size; if (spec) { @@ -543,7 +648,7 @@ mlx5_flow_item_vlan_update(struct ibv_flow_attr *attr, struct ibv_flow_spec_eth *eth) { unsigned int i; - enum ibv_flow_spec_type search = IBV_FLOW_SPEC_ETH; + const enum ibv_flow_spec_type search = eth->type; struct ibv_spec_header *hdr = (struct ibv_spec_header *) ((uint8_t *)attr + sizeof(struct ibv_flow_attr)); @@ -596,16 +701,19 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, .inner_type = RTE_BE16(0xffff), }; unsigned int size = sizeof(struct ibv_flow_spec_eth); - struct mlx5_flow_verbs *verbs = flow->cur_verbs; + const int tunnel = !!(flow->layers & MLX5_FLOW_LAYER_TUNNEL); struct ibv_flow_spec_eth eth = { - .type = IBV_FLOW_SPEC_ETH, + .type = IBV_FLOW_SPEC_ETH | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; - const uint32_t l34m = MLX5_FLOW_LAYER_OUTER_L3 | - MLX5_FLOW_LAYER_OUTER_L4; - const uint32_t vlanm = MLX5_FLOW_LAYER_OUTER_VLAN; - const uint32_t l2m = MLX5_FLOW_LAYER_OUTER_L2; + const uint32_t l34m = tunnel ? (MLX5_FLOW_LAYER_INNER_L3 | + MLX5_FLOW_LAYER_INNER_L4) : + (MLX5_FLOW_LAYER_OUTER_L3 | MLX5_FLOW_LAYER_OUTER_L4); + const uint32_t vlanm = tunnel ? MLX5_FLOW_LAYER_INNER_VLAN : + MLX5_FLOW_LAYER_OUTER_VLAN; + const uint32_t l2m = tunnel ? MLX5_FLOW_LAYER_INNER_L2 : + MLX5_FLOW_LAYER_OUTER_L2; if (flow->layers & vlanm) return rte_flow_error_set(error, ENOTSUP, @@ -648,11 +756,14 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, mlx5_flow_spec_verbs_add(flow, ð, size); } } else { - if (verbs->attr) - mlx5_flow_item_vlan_update(verbs->attr, ð); + if (flow->cur_verbs) + mlx5_flow_item_vlan_update(flow->cur_verbs->attr, + ð); size = 0; /* Only an update is done in eth specification. */ } - flow->layers |= MLX5_FLOW_LAYER_OUTER_L2 | MLX5_FLOW_LAYER_OUTER_VLAN; + flow->layers |= tunnel ? + (MLX5_FLOW_LAYER_INNER_L2 | MLX5_FLOW_LAYER_INNER_VLAN) : + (MLX5_FLOW_LAYER_OUTER_L2 | MLX5_FLOW_LAYER_OUTER_VLAN); return size; } @@ -692,19 +803,23 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, .next_proto_id = 0xff, }, }; + const int tunnel = !!(flow->layers & MLX5_FLOW_LAYER_TUNNEL); unsigned int size = sizeof(struct ibv_flow_spec_ipv4_ext); struct ibv_flow_spec_ipv4_ext ipv4 = { - .type = IBV_FLOW_SPEC_IPV4_EXT, + .type = IBV_FLOW_SPEC_IPV4_EXT | + (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L3) + if (flow->layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L3 : + MLX5_FLOW_LAYER_OUTER_L3)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, "multiple L3 layers not supported"); - else if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + else if (flow->layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L4 : + MLX5_FLOW_LAYER_OUTER_L4)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -717,7 +832,8 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, sizeof(struct rte_flow_item_ipv4), error); if (ret < 0) return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV4; + flow->layers |= tunnel ? MLX5_FLOW_LAYER_INNER_L3_IPV4 : + MLX5_FLOW_LAYER_OUTER_L3_IPV4; if (spec) { ipv4.val = (struct ibv_flow_ipv4_ext_filter){ .src_ip = spec->hdr.src_addr, @@ -740,14 +856,11 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, flow->l3_protocol_en = !!ipv4.mask.proto; flow->l3_protocol = ipv4.val.proto; if (size <= flow_size) { - uint64_t hash_fields = IBV_RX_HASH_SRC_IPV4 | - IBV_RX_HASH_DST_IPV4; - - if (!(flow->rss.types & - (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | - ETH_RSS_NONFRAG_IPV4_OTHER))) - hash_fields = 0; - flow->cur_verbs->hash_fields |= hash_fields; + mlx5_flow_verbs_hashfields_adjust + (flow, tunnel, + (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | + ETH_RSS_NONFRAG_IPV4_OTHER), + (IBV_RX_HASH_SRC_IPV4 | IBV_RX_HASH_DST_IPV4)); flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L3; mlx5_flow_spec_verbs_add(flow, &ipv4, size); } @@ -795,19 +908,22 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, .hop_limits = 0xff, }, }; + const int tunnel = !!(flow->layers & MLX5_FLOW_LAYER_TUNNEL); unsigned int size = sizeof(struct ibv_flow_spec_ipv6); struct ibv_flow_spec_ipv6 ipv6 = { - .type = IBV_FLOW_SPEC_IPV6, + .type = IBV_FLOW_SPEC_IPV6 | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L3) + if (flow->layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L3 : + MLX5_FLOW_LAYER_OUTER_L3)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, "multiple L3 layers not supported"); - else if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + else if (flow->layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L4 : + MLX5_FLOW_LAYER_OUTER_L4)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -820,7 +936,8 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, sizeof(struct rte_flow_item_ipv6), error); if (ret < 0) return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV6; + flow->layers |= tunnel ? MLX5_FLOW_LAYER_INNER_L3_IPV6 : + MLX5_FLOW_LAYER_OUTER_L3_IPV6; if (spec) { unsigned int i; uint32_t vtc_flow_val; @@ -863,13 +980,10 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, flow->l3_protocol_en = !!ipv6.mask.next_hdr; flow->l3_protocol = ipv6.val.next_hdr; if (size <= flow_size) { - uint64_t hash_fields = IBV_RX_HASH_SRC_IPV6 | - IBV_RX_HASH_DST_IPV6; - - if (!(flow->rss.types & - (ETH_RSS_IPV6 | ETH_RSS_NONFRAG_IPV6_OTHER))) - hash_fields = 0; - flow->cur_verbs->hash_fields |= hash_fields; + mlx5_flow_verbs_hashfields_adjust + (flow, tunnel, + (ETH_RSS_IPV6 | ETH_RSS_NONFRAG_IPV6_OTHER), + (IBV_RX_HASH_SRC_IPV6 | IBV_RX_HASH_DST_IPV6)); flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L3; mlx5_flow_spec_verbs_add(flow, &ipv6, size); } @@ -904,9 +1018,10 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, { const struct rte_flow_item_udp *spec = item->spec; const struct rte_flow_item_udp *mask = item->mask; + const int tunnel = !!(flow->layers & MLX5_FLOW_LAYER_TUNNEL); unsigned int size = sizeof(struct ibv_flow_spec_tcp_udp); struct ibv_flow_spec_tcp_udp udp = { - .type = IBV_FLOW_SPEC_UDP, + .type = IBV_FLOW_SPEC_UDP | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; @@ -917,13 +1032,15 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, item, "protocol filtering not compatible" " with UDP layer"); - if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) + if (!(flow->layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L3 : + MLX5_FLOW_LAYER_OUTER_L3))) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, "L3 is mandatory to filter" " on L4"); - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + if (flow->layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L4 : + MLX5_FLOW_LAYER_OUTER_L4)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -937,7 +1054,8 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, sizeof(struct rte_flow_item_udp), error); if (ret < 0) return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_UDP; + flow->layers |= tunnel ? MLX5_FLOW_LAYER_INNER_L4_UDP : + MLX5_FLOW_LAYER_OUTER_L4_UDP; if (spec) { udp.val.dst_port = spec->hdr.dst_port; udp.val.src_port = spec->hdr.src_port; @@ -948,12 +1066,9 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, udp.val.dst_port &= udp.mask.dst_port; } if (size <= flow_size) { - uint64_t hash_fields = IBV_RX_HASH_SRC_PORT_UDP | - IBV_RX_HASH_DST_PORT_UDP; - - if (!(flow->rss.types & ETH_RSS_UDP)) - hash_fields = 0; - flow->cur_verbs->hash_fields |= hash_fields; + mlx5_flow_verbs_hashfields_adjust(flow, tunnel, ETH_RSS_UDP, + (IBV_RX_HASH_SRC_PORT_UDP | + IBV_RX_HASH_DST_PORT_UDP)); flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L4; mlx5_flow_spec_verbs_add(flow, &udp, size); } @@ -988,9 +1103,10 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, { const struct rte_flow_item_tcp *spec = item->spec; const struct rte_flow_item_tcp *mask = item->mask; + const int tunnel = !!(flow->layers & MLX5_FLOW_LAYER_TUNNEL); unsigned int size = sizeof(struct ibv_flow_spec_tcp_udp); struct ibv_flow_spec_tcp_udp tcp = { - .type = IBV_FLOW_SPEC_TCP, + .type = IBV_FLOW_SPEC_TCP | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; @@ -1001,12 +1117,14 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, item, "protocol filtering not compatible" " with TCP layer"); - if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) + if (!(flow->layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L3 : + MLX5_FLOW_LAYER_OUTER_L3))) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, "L3 is mandatory to filter on L4"); - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + if (flow->layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L4 : + MLX5_FLOW_LAYER_OUTER_L4)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -1019,7 +1137,8 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, sizeof(struct rte_flow_item_tcp), error); if (ret < 0) return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_TCP; + flow->layers |= tunnel ? MLX5_FLOW_LAYER_INNER_L4_TCP : + MLX5_FLOW_LAYER_OUTER_L4_TCP; if (spec) { tcp.val.dst_port = spec->hdr.dst_port; tcp.val.src_port = spec->hdr.src_port; @@ -1030,12 +1149,9 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, tcp.val.dst_port &= tcp.mask.dst_port; } if (size <= flow_size) { - uint64_t hash_fields = IBV_RX_HASH_SRC_PORT_TCP | - IBV_RX_HASH_DST_PORT_TCP; - - if (!(flow->rss.types & ETH_RSS_TCP)) - hash_fields = 0; - flow->cur_verbs->hash_fields |= hash_fields; + mlx5_flow_verbs_hashfields_adjust(flow, tunnel, ETH_RSS_TCP, + (IBV_RX_HASH_SRC_PORT_TCP | + IBV_RX_HASH_DST_PORT_TCP)); flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L4; mlx5_flow_spec_verbs_add(flow, &tcp, size); } @@ -1261,7 +1377,7 @@ mlx5_flow_action_rss(struct rte_eth_dev *dev, RTE_FLOW_ERROR_TYPE_ACTION_CONF, &rss->func, "RSS hash function not supported"); - if (rss->level > 1) + if (rss->level > 2) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION_CONF, &rss->level, @@ -1301,6 +1417,7 @@ mlx5_flow_action_rss(struct rte_eth_dev *dev, flow->rss.queue_num = rss->queue_num; memcpy(flow->key, rss->key, MLX5_RSS_HASH_KEY_LEN); flow->rss.types = rss->types; + flow->rss.level = rss->level; flow->fate |= MLX5_FLOW_FATE_RSS; return 0; } @@ -1607,7 +1724,9 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, ret = rte_flow_expand_rss(buf, sizeof(expand_buffer.buffer), pattern, local_flow.rss.types, mlx5_support_expansion, - MLX5_EXPANSION_ROOT); + local_flow.rss.level < 2 ? + MLX5_EXPANSION_ROOT : + MLX5_EXPANSION_ROOT_OUTER); assert(ret > 0 && (unsigned int)ret < sizeof(expand_buffer.buffer)); } else { @@ -1975,8 +2094,8 @@ mlx5_flow_list_create(struct rte_eth_dev *dev, return NULL; } } - mlx5_flow_rxq_mark_set(dev, flow); TAILQ_INSERT_TAIL(list, flow, next); + mlx5_flow_rxq_mark_set(dev, flow); return flow; } From patchwork Wed Jul 11 07:22:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42810 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 52C7B1B579; Wed, 11 Jul 2018 09:24:04 +0200 (CEST) Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by dpdk.org (Postfix) with ESMTP id 5C2E21B476 for ; Wed, 11 Jul 2018 09:23:33 +0200 (CEST) Received: by mail-wr1-f67.google.com with SMTP id g6-v6so8097416wrp.0 for ; Wed, 11 Jul 2018 00:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rKVgZXY+wLV2a/Ul/p7pYSoVg/kQPYZHpomUcDSr+/A=; b=NX1XDDHJyr5ftA7Uz0jeAUBTf5ZYdfqrE+4Vkn0qldP83fEkIODqAYFQtQwp/zX4dv im+4l5mGAiC0jaRzpxZwSkG68C85W7t6g7kIbmIC1ddbz67bursHUCUu2M13dtwdnod9 uyd50vPYml3hoDsMLYiDtq3QeF9PlTNj+FEyuoT5BXLDnrVwkxwYMOLkonMgw9FaTQWC IrYt3fu9xywerBbtPN5deYmXsOjIRGjFVwG4yq8lW8XITy3OHhO+ICIDn2eS++Nd6zMs fi6bLt2lKTu0Dj0PYznLo0KomihMk/sQpV/XgmKzcHFNJz84KsJ/YQdl2k22dnzHQAel tyUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rKVgZXY+wLV2a/Ul/p7pYSoVg/kQPYZHpomUcDSr+/A=; b=I837naN9mz8o8tJrsCs/6tFTdfaH9Z2YaInz8T4QOdAUAieQS/WkiQXBqX34MgSCof WCX5eP+ZbItzPlGKuDjm9U0u1DVE4CYNTb6o5jbvAY1H7k71bikRq6Y4r4bL2FlIIryN jiDoxbMyD8XHvbJxpWhRASQt2Y+EE45g77sCtvTnN3JTSZ8o8gB/5UVgo8R+DYGwkXM7 /AzbahATv+rL5n9rFaoJBVXWmZPRqXgANKrvbbg8XyIduc5F24/nLdHmoc9TmlVuFgJT 4gw6H/P7DyZislaqFyzIt592WxD3jWp4iCmJ3gt8Ipk0DD78NNNFE6ejESelNxtqgtCL Vf7w== X-Gm-Message-State: APt69E0aRH8hFrj/1ocbvOPgpi4cd1sp20AJAXv/Xqv02a7ryP1pH5Np /AunMcXebp7BscJpWGGEzHoHSy4Weg== X-Google-Smtp-Source: AAOMgpda8fk01iO1emEw2vwE8cf9I9cc835ZsNVcR90pqtBA7yh7BJdFuoTMuncX1UFOvVrPiwdYLw== X-Received: by 2002:adf:a14a:: with SMTP id r10-v6mr19336449wrr.104.1531293812448; Wed, 11 Jul 2018 00:23:32 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:32 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:50 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 17/21] net/mlx5: add flow VXLAN item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 190 +++++++++++++++++++++++++++++------ drivers/net/mlx5/mlx5_rxtx.h | 1 + 2 files changed, 163 insertions(+), 28 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index edceb17ed..95507059e 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -51,6 +51,9 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; #define MLX5_FLOW_LAYER_INNER_L4_TCP (1u << 10) #define MLX5_FLOW_LAYER_INNER_VLAN (1u << 11) +/* Pattern tunnel Layer bits. */ +#define MLX5_FLOW_LAYER_VXLAN (1u << 12) + /* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6) @@ -61,7 +64,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; MLX5_FLOW_LAYER_OUTER_L4) /* Tunnel Masks. */ -#define MLX5_FLOW_LAYER_TUNNEL 0 +#define MLX5_FLOW_LAYER_TUNNEL MLX5_FLOW_LAYER_VXLAN /* Inner Masks. */ #define MLX5_FLOW_LAYER_INNER_L3 \ @@ -98,6 +101,7 @@ enum mlx5_expansion { MLX5_EXPANSION_OUTER_IPV6, MLX5_EXPANSION_OUTER_IPV6_UDP, MLX5_EXPANSION_OUTER_IPV6_TCP, + MLX5_EXPANSION_VXLAN, MLX5_EXPANSION_ETH, MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV4_UDP, @@ -136,6 +140,7 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { ETH_RSS_NONFRAG_IPV4_OTHER, }, [MLX5_EXPANSION_OUTER_IPV4_UDP] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_VXLAN), .type = RTE_FLOW_ITEM_TYPE_UDP, .rss_types = ETH_RSS_NONFRAG_IPV4_UDP, }, @@ -152,6 +157,7 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { ETH_RSS_NONFRAG_IPV6_OTHER, }, [MLX5_EXPANSION_OUTER_IPV6_UDP] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_VXLAN), .type = RTE_FLOW_ITEM_TYPE_UDP, .rss_types = ETH_RSS_NONFRAG_IPV6_UDP, }, @@ -159,6 +165,10 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { .type = RTE_FLOW_ITEM_TYPE_TCP, .rss_types = ETH_RSS_NONFRAG_IPV6_TCP, }, + [MLX5_EXPANSION_VXLAN] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_ETH), + .type = RTE_FLOW_ITEM_TYPE_VXLAN, + }, [MLX5_EXPANSION_ETH] = { .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV6), @@ -226,6 +236,8 @@ struct rte_flow { struct mlx5_flow_verbs *cur_verbs; /**< Current Verbs flow structure being filled. */ struct rte_flow_action_rss rss;/**< RSS context. */ + uint32_t tunnel_ptype; + /**< Store tunnel packet type data to store in Rx queue. */ uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */ uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */ }; @@ -1158,6 +1170,103 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Convert the @p item into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_vxlan(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_vxlan *spec = item->spec; + const struct rte_flow_item_vxlan *mask = item->mask; + unsigned int size = sizeof(struct ibv_flow_spec_tunnel); + struct ibv_flow_spec_tunnel vxlan = { + .type = IBV_FLOW_SPEC_VXLAN_TUNNEL, + .size = size, + }; + int ret; + union vni { + uint32_t vlan_id; + uint8_t vni[4]; + } id = { .vlan_id = 0, }; + + if (flow->layers & MLX5_FLOW_LAYER_TUNNEL) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "a tunnel is already present"); + /* + * Verify only UDPv4 is present as defined in + * https://tools.ietf.org/html/rfc7348 + */ + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L4_UDP)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "no outer UDP layer found"); + if (!mask) + mask = &rte_flow_item_vxlan_mask; + ret = mlx5_flow_item_acceptable + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_vxlan_mask, + sizeof(struct rte_flow_item_vxlan), error); + if (ret < 0) + return ret; + if (spec) { + memcpy(&id.vni[1], spec->vni, 3); + vxlan.val.tunnel_id = id.vlan_id; + memcpy(&id.vni[1], mask->vni, 3); + vxlan.mask.tunnel_id = id.vlan_id; + /* Remove unwanted bits from values. */ + vxlan.val.tunnel_id &= vxlan.mask.tunnel_id; + } + /* + * Tunnel id 0 is equivalent as not adding a VXLAN layer, if + * only this layer is defined in the Verbs specification it is + * interpreted as wildcard and all packets will match this + * rule, if it follows a full stack layer (ex: eth / ipv4 / + * udp), all packets matching the layers before will also + * match this rule. To avoid such situation, VNI 0 is + * currently refused. + */ + if (!vxlan.val.tunnel_id) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VXLAN vni cannot be 0"); + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VXLAN tunnel must be fully defined"); + if (size <= flow_size) { + mlx5_flow_spec_verbs_add(flow, &vxlan, size); + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L2; + } + flow->layers |= MLX5_FLOW_LAYER_VXLAN; + flow->tunnel_ptype = RTE_PTYPE_TUNNEL_VXLAN | RTE_PTYPE_L4_UDP; + return size; +} + /** * Convert the @p pattern into a Verbs specifications after ensuring the NIC * will understand and process it correctly. @@ -1215,6 +1324,10 @@ mlx5_flow_items(const struct rte_flow_item pattern[], case RTE_FLOW_ITEM_TYPE_TCP: ret = mlx5_flow_item_tcp(pattern, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_VXLAN: + ret = mlx5_flow_item_vxlan(pattern, flow, remain, + error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, @@ -1822,7 +1935,7 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, } /** - * Mark the Rx queues mark flag if the flow has a mark or flag modifier. + * Set the Rx queue flags (Mark/Flag and Tunnel Ptypes) according to the flow. * * @param[in] dev * Pointer to Ethernet device. @@ -1830,28 +1943,34 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, * Pointer to flow structure. */ static void -mlx5_flow_rxq_mark_set(struct rte_eth_dev *dev, struct rte_flow *flow) +mlx5_flow_rxq_flags_set(struct rte_eth_dev *dev, struct rte_flow *flow) { struct priv *priv = dev->data->dev_private; + const int mark = !!(flow->modifier & + (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)); + const int tunnel = !!(flow->layers & MLX5_FLOW_LAYER_TUNNEL); + unsigned int i; - if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) { - unsigned int i; - - for (i = 0; i != flow->rss.queue_num; ++i) { - int idx = (*flow->queue)[i]; - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of((*priv->rxqs)[idx], - struct mlx5_rxq_ctrl, rxq); + for (i = 0; i != flow->rss.queue_num; ++i) { + int idx = (*flow->queue)[i]; + struct mlx5_rxq_ctrl *rxq_ctrl = + container_of((*priv->rxqs)[idx], + struct mlx5_rxq_ctrl, rxq); + if (mark) { rxq_ctrl->rxq.mark = 1; rxq_ctrl->flow_mark_n++; } + if (tunnel) { + rxq_ctrl->rxq.tunnel = flow->tunnel_ptype; + rxq_ctrl->flow_vxlan_n++; + } } } /** - * Clear the Rx queue mark associated with the @p flow if no other flow uses - * it with a mark request. + * Clear the Rx queue flags (Mark/Flag and Tunnel Ptype) associated with the + * @p flow if no other flow uses it with the same kind of request. * * @param dev * Pointer to Ethernet device. @@ -1859,33 +1978,41 @@ mlx5_flow_rxq_mark_set(struct rte_eth_dev *dev, struct rte_flow *flow) * Pointer to the flow. */ static void -mlx5_flow_rxq_mark_trim(struct rte_eth_dev *dev, struct rte_flow *flow) +mlx5_flow_rxq_flags_trim(struct rte_eth_dev *dev, struct rte_flow *flow) { struct priv *priv = dev->data->dev_private; + const int mark = !!(flow->modifier & + (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)); + const int tunnel = !!(flow->layers & MLX5_FLOW_LAYER_TUNNEL); + unsigned int i; - if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) { - unsigned int i; - - for (i = 0; i != flow->rss.queue_num; ++i) { - int idx = (*flow->queue)[i]; - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of((*priv->rxqs)[idx], - struct mlx5_rxq_ctrl, rxq); + assert(dev->data->dev_started); + for (i = 0; i != flow->rss.queue_num; ++i) { + int idx = (*flow->queue)[i]; + struct mlx5_rxq_ctrl *rxq_ctrl = + container_of((*priv->rxqs)[idx], + struct mlx5_rxq_ctrl, rxq); + if (mark) { rxq_ctrl->flow_mark_n--; rxq_ctrl->rxq.mark = !!rxq_ctrl->flow_mark_n; } + if (tunnel) { + rxq_ctrl->flow_vxlan_n++; + if (!rxq_ctrl->flow_vxlan_n) + rxq_ctrl->rxq.tunnel = 0; + } } } /** - * Clear the mark bit in all Rx queues. + * Clear the Mark/Flag and Tunnel ptype information in all Rx queues. * * @param dev * Pointer to Ethernet device. */ static void -mlx5_flow_rxq_mark_clear(struct rte_eth_dev *dev) +mlx5_flow_rxq_flags_clear(struct rte_eth_dev *dev) { struct priv *priv = dev->data->dev_private; unsigned int i; @@ -1900,6 +2027,8 @@ mlx5_flow_rxq_mark_clear(struct rte_eth_dev *dev) struct mlx5_rxq_ctrl, rxq); rxq_ctrl->flow_mark_n = 0; rxq_ctrl->rxq.mark = 0; + rxq_ctrl->flow_vxlan_n = 0; + rxq_ctrl->rxq.tunnel = 0; ++idx; } } @@ -2095,7 +2224,7 @@ mlx5_flow_list_create(struct rte_eth_dev *dev, } } TAILQ_INSERT_TAIL(list, flow, next); - mlx5_flow_rxq_mark_set(dev, flow); + mlx5_flow_rxq_flags_set(dev, flow); return flow; } @@ -2133,7 +2262,12 @@ mlx5_flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, { mlx5_flow_remove(dev, flow); TAILQ_REMOVE(list, flow, next); - mlx5_flow_rxq_mark_trim(dev, flow); + /* + * Update RX queue flags only if port is started, otherwise it is + * already clean. + */ + if (dev->data->dev_started) + mlx5_flow_rxq_flags_trim(dev, flow); rte_free(flow); } @@ -2171,7 +2305,7 @@ mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) mlx5_flow_remove(dev, flow); - mlx5_flow_rxq_mark_clear(dev); + mlx5_flow_rxq_flags_clear(dev); } /** @@ -2196,7 +2330,7 @@ mlx5_flow_start(struct rte_eth_dev *dev, struct mlx5_flows *list) ret = mlx5_flow_apply(dev, flow, &error); if (ret < 0) goto error; - mlx5_flow_rxq_mark_set(dev, flow); + mlx5_flow_rxq_flags_set(dev, flow); } return 0; error: diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 808118e50..ae9b564dd 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -139,6 +139,7 @@ struct mlx5_rxq_ctrl { unsigned int irq:1; /* Whether IRQ is enabled. */ uint16_t idx; /* Queue index. */ uint32_t flow_mark_n; /* Number of Mark/Flag flows using this Queue. */ + uint32_t flow_vxlan_n; /* Number of VXLAN flows using this queue. */ }; /* Indirection table. */ From patchwork Wed Jul 11 07:22:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42811 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 97F4C1B581; Wed, 11 Jul 2018 09:24:05 +0200 (CEST) Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by dpdk.org (Postfix) with ESMTP id C0AC61B476 for ; Wed, 11 Jul 2018 09:23:33 +0200 (CEST) Received: by mail-wm0-f66.google.com with SMTP id i139-v6so1339956wmf.4 for ; Wed, 11 Jul 2018 00:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=c0XnzNbmXJpgPiO4Wc/NZfu9/9xz31c4k+MQH2o/yp8=; b=zxiKQ3bkLbpTiLiRGiHYAJkITln4ij8jceTejB80nxKGzB6XCXWjHPrXkdAn2U45Ak RxDDTOyY0jD5nCraW65n7MqAUWb8j3TJdEXJcGhq6AZc/T7vL70KJg0mDlOny3q82BU3 J+6tw6i/tPAYb+7YEcmBIxrSVMfh13aQ4OIfnPaMKmuetdX3P/wwIvzCNEuETlif+dgt 6fZS//ye6E9KKR9AuJdyrW2z35Gj9qke+DrGGMZlDCDo0KqkZtB59f/RFEQ2DKbQV/g8 9RqsK3oaYu8ROjJ/RE69ZjyIdqFAQgHeIyV7FBe9Pan9YLpF4A/3z7ILJ91AM4GEm4yC aaKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=c0XnzNbmXJpgPiO4Wc/NZfu9/9xz31c4k+MQH2o/yp8=; b=Q7M0K+Rd3V/tJoHut1od7Mn6JapXG7iurP7m7CS6dt8IHfJyV+xk+SF5uZA0hDCFqk DlUFKjC3/oVJZZy0ivuDbAIlLPcWoK9v7CQIt7tnirSueZ8J2p+0XgDj9BYB1TBWprHa IyCvu1k4RkHnjuOCL9YmLYGElUFfzRAEifGC9Rle6jH4Zpm+621ugbeq3pj7Tm3/3w/C o6tnhkgAZkiHZs+78MZTED4gketYdBBaO/ki0mkyxSI1YHg4BzlodZlylljj9VkTVOd4 YwB/nitkYs/WXLFfN4YoabUNvEZSgu1VImW5ucRuzLuJ9jNS+YqHWaM0JogLNaFMcnLt 2FbA== X-Gm-Message-State: AOUpUlEyJVHoyriegRmBlCP++l27p/I7nKPDe8lqwHAunDH7mAlqojG8 Upt4K08g5ETHlT6MsGHy8yLvnhPhSA== X-Google-Smtp-Source: AAOMgpf53MhvG8nK/yojHT0lD9vu33TME8ILeNJ/3uNjpA52csW8Kp1BWYzmiWTtqt3DsinQVLiXMg== X-Received: by 2002:a1c:6c14:: with SMTP id h20-v6mr2960790wmc.138.1531293813149; Wed, 11 Jul 2018 00:23:33 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:32 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:51 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 18/21] net/mlx5: add flow VXLAN-GPE item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 219 ++++++++++++++++++++++++++++++++--- drivers/net/mlx5/mlx5_rxtx.h | 5 +- 2 files changed, 209 insertions(+), 15 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 95507059e..e06df0eb5 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -53,6 +53,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Pattern tunnel Layer bits. */ #define MLX5_FLOW_LAYER_VXLAN (1u << 12) +#define MLX5_FLOW_LAYER_VXLAN_GPE (1u << 13) /* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ @@ -64,7 +65,8 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; MLX5_FLOW_LAYER_OUTER_L4) /* Tunnel Masks. */ -#define MLX5_FLOW_LAYER_TUNNEL MLX5_FLOW_LAYER_VXLAN +#define MLX5_FLOW_LAYER_TUNNEL \ + (MLX5_FLOW_LAYER_VXLAN | MLX5_FLOW_LAYER_VXLAN_GPE) /* Inner Masks. */ #define MLX5_FLOW_LAYER_INNER_L3 \ @@ -102,6 +104,7 @@ enum mlx5_expansion { MLX5_EXPANSION_OUTER_IPV6_UDP, MLX5_EXPANSION_OUTER_IPV6_TCP, MLX5_EXPANSION_VXLAN, + MLX5_EXPANSION_VXLAN_GPE, MLX5_EXPANSION_ETH, MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV4_UDP, @@ -140,7 +143,8 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { ETH_RSS_NONFRAG_IPV4_OTHER, }, [MLX5_EXPANSION_OUTER_IPV4_UDP] = { - .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_VXLAN), + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_VXLAN, + MLX5_EXPANSION_VXLAN_GPE), .type = RTE_FLOW_ITEM_TYPE_UDP, .rss_types = ETH_RSS_NONFRAG_IPV4_UDP, }, @@ -157,7 +161,8 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { ETH_RSS_NONFRAG_IPV6_OTHER, }, [MLX5_EXPANSION_OUTER_IPV6_UDP] = { - .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_VXLAN), + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_VXLAN, + MLX5_EXPANSION_VXLAN_GPE), .type = RTE_FLOW_ITEM_TYPE_UDP, .rss_types = ETH_RSS_NONFRAG_IPV6_UDP, }, @@ -169,6 +174,12 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_ETH), .type = RTE_FLOW_ITEM_TYPE_VXLAN, }, + [MLX5_EXPANSION_VXLAN_GPE] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_ETH, + MLX5_EXPANSION_IPV4, + MLX5_EXPANSION_IPV6), + .type = RTE_FLOW_ITEM_TYPE_VXLAN_GPE, + }, [MLX5_EXPANSION_ETH] = { .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV6), @@ -236,8 +247,6 @@ struct rte_flow { struct mlx5_flow_verbs *cur_verbs; /**< Current Verbs flow structure being filled. */ struct rte_flow_action_rss rss;/**< RSS context. */ - uint32_t tunnel_ptype; - /**< Store tunnel packet type data to store in Rx queue. */ uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */ uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */ }; @@ -304,6 +313,23 @@ static const uint32_t priority_map_5[][MLX5_PRIORITY_MAP_MAX] = { { 9, 10, 11 }, { 12, 13, 14 }, }; +/* Tunnel information. */ +struct mlx5_flow_tunnel_info { + uint32_t tunnel; /**< Tunnel bit (see MLX5_FLOW_*). */ + uint32_t ptype; /**< Tunnel Ptype (see RTE_PTYPE_*). */ +}; + +static struct mlx5_flow_tunnel_info tunnels_info[] = { + { + .tunnel = MLX5_FLOW_LAYER_VXLAN, + .ptype = RTE_PTYPE_TUNNEL_VXLAN | RTE_PTYPE_L4_UDP, + }, + { + .tunnel = MLX5_FLOW_LAYER_VXLAN_GPE, + .ptype = RTE_PTYPE_TUNNEL_VXLAN_GPE | RTE_PTYPE_L4_UDP, + }, +}; + /** * Discover the maximum number of priority available. * @@ -1263,7 +1289,119 @@ mlx5_flow_item_vxlan(const struct rte_flow_item *item, struct rte_flow *flow, flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L2; } flow->layers |= MLX5_FLOW_LAYER_VXLAN; - flow->tunnel_ptype = RTE_PTYPE_TUNNEL_VXLAN | RTE_PTYPE_L4_UDP; + return size; +} + +/** + * Convert the @p item into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param dev + * Pointer to Ethernet device. + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_vxlan_gpe(struct rte_eth_dev *dev, + const struct rte_flow_item *item, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + const struct rte_flow_item_vxlan_gpe *spec = item->spec; + const struct rte_flow_item_vxlan_gpe *mask = item->mask; + unsigned int size = sizeof(struct ibv_flow_spec_tunnel); + struct ibv_flow_spec_tunnel vxlan_gpe = { + .type = IBV_FLOW_SPEC_VXLAN_TUNNEL, + .size = size, + }; + int ret; + union vni { + uint32_t vlan_id; + uint8_t vni[4]; + } id = { .vlan_id = 0, }; + + if (!((struct priv *)dev->data->dev_private)->config.l3_vxlan_en) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 VXLAN is not enabled by device" + " parameter and/or not configured in" + " firmware"); + if (flow->layers & MLX5_FLOW_LAYER_TUNNEL) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "a tunnel is already present"); + /* + * Verify only UDPv4 is present as defined in + * https://tools.ietf.org/html/rfc7348 + */ + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L4_UDP)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "no outer UDP layer found"); + if (!mask) + mask = &rte_flow_item_vxlan_gpe_mask; + ret = mlx5_flow_item_acceptable + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_vxlan_gpe_mask, + sizeof(struct rte_flow_item_vxlan_gpe), error); + if (ret < 0) + return ret; + if (spec) { + memcpy(&id.vni[1], spec->vni, 3); + vxlan_gpe.val.tunnel_id = id.vlan_id; + memcpy(&id.vni[1], mask->vni, 3); + vxlan_gpe.mask.tunnel_id = id.vlan_id; + if (spec->protocol) + return rte_flow_error_set + (error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VxLAN-GPE protocol not supported"); + /* Remove unwanted bits from values. */ + vxlan_gpe.val.tunnel_id &= vxlan_gpe.mask.tunnel_id; + } + /* + * Tunnel id 0 is equivalent as not adding a VXLAN layer, if only this + * layer is defined in the Verbs specification it is interpreted as + * wildcard and all packets will match this rule, if it follows a full + * stack layer (ex: eth / ipv4 / udp), all packets matching the layers + * before will also match this rule. To avoid such situation, VNI 0 + * is currently refused. + */ + if (!vxlan_gpe.val.tunnel_id) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VXLAN-GPE vni cannot be 0"); + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VXLAN-GPE tunnel must be fully" + " defined"); + if (size <= flow_size) { + mlx5_flow_spec_verbs_add(flow, &vxlan_gpe, size); + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L2; + } + flow->layers |= MLX5_FLOW_LAYER_VXLAN_GPE; return size; } @@ -1293,7 +1431,8 @@ mlx5_flow_item_vxlan(const struct rte_flow_item *item, struct rte_flow *flow, * On error, a negative errno value is returned and rte_errno is set. */ static int -mlx5_flow_items(const struct rte_flow_item pattern[], +mlx5_flow_items(struct rte_eth_dev *dev, + const struct rte_flow_item pattern[], struct rte_flow *flow, const size_t flow_size, struct rte_flow_error *error) { @@ -1328,6 +1467,10 @@ mlx5_flow_items(const struct rte_flow_item pattern[], ret = mlx5_flow_item_vxlan(pattern, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_VXLAN_GPE: + ret = mlx5_flow_item_vxlan_gpe(dev, pattern, flow, + remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, @@ -1908,7 +2051,8 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, } } ret = mlx5_flow_items - ((const struct rte_flow_item *) + (dev, + (const struct rte_flow_item *) &buf->entry[i].pattern[pattern_start_idx], flow, (size < flow_size) ? flow_size - size : 0, error); @@ -1934,6 +2078,34 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, return size; } +/** + * Lookup and set the ptype in the data Rx part. A single Ptype can be used, + * if several tunnel rules are used on this queue, the tunnel ptype will be + * cleared. + * + * @param rxq_ctrl + * Rx queue to update. + */ +static void +mlx5_flow_rxq_tunnel_ptype_update(struct mlx5_rxq_ctrl *rxq_ctrl) +{ + unsigned int i; + uint32_t tunnel_ptype = 0; + + /* Look up for the ptype to use. */ + for (i = 0; i != MLX5_FLOW_TUNNEL; ++i) { + if (!rxq_ctrl->flow_tunnels_n[i]) + continue; + if (!tunnel_ptype) { + tunnel_ptype = tunnels_info[i].ptype; + } else { + tunnel_ptype = 0; + break; + } + } + rxq_ctrl->rxq.tunnel = tunnel_ptype; +} + /** * Set the Rx queue flags (Mark/Flag and Tunnel Ptypes) according to the flow. * @@ -1962,8 +2134,17 @@ mlx5_flow_rxq_flags_set(struct rte_eth_dev *dev, struct rte_flow *flow) rxq_ctrl->flow_mark_n++; } if (tunnel) { - rxq_ctrl->rxq.tunnel = flow->tunnel_ptype; - rxq_ctrl->flow_vxlan_n++; + unsigned int j; + + /* Increase the counter matching the flow. */ + for (j = 0; j != MLX5_FLOW_TUNNEL; ++j) { + if ((tunnels_info[j].tunnel & flow->layers) == + tunnels_info[j].tunnel) { + rxq_ctrl->flow_tunnels_n[j]++; + break; + } + } + mlx5_flow_rxq_tunnel_ptype_update(rxq_ctrl); } } } @@ -1998,9 +2179,17 @@ mlx5_flow_rxq_flags_trim(struct rte_eth_dev *dev, struct rte_flow *flow) rxq_ctrl->rxq.mark = !!rxq_ctrl->flow_mark_n; } if (tunnel) { - rxq_ctrl->flow_vxlan_n++; - if (!rxq_ctrl->flow_vxlan_n) - rxq_ctrl->rxq.tunnel = 0; + unsigned int j; + + /* Decrease the counter matching the flow. */ + for (j = 0; j != MLX5_FLOW_TUNNEL; ++j) { + if ((tunnels_info[j].tunnel & flow->layers) == + tunnels_info[j].tunnel) { + rxq_ctrl->flow_tunnels_n[j]--; + break; + } + } + mlx5_flow_rxq_tunnel_ptype_update(rxq_ctrl); } } } @@ -2020,6 +2209,7 @@ mlx5_flow_rxq_flags_clear(struct rte_eth_dev *dev) for (idx = 0, i = 0; idx != priv->rxqs_n; ++i) { struct mlx5_rxq_ctrl *rxq_ctrl; + unsigned int j; if (!(*priv->rxqs)[idx]) continue; @@ -2027,7 +2217,8 @@ mlx5_flow_rxq_flags_clear(struct rte_eth_dev *dev) struct mlx5_rxq_ctrl, rxq); rxq_ctrl->flow_mark_n = 0; rxq_ctrl->rxq.mark = 0; - rxq_ctrl->flow_vxlan_n = 0; + for (j = 0; j != MLX5_FLOW_TUNNEL; ++j) + rxq_ctrl->flow_tunnels_n[j] = 0; rxq_ctrl->rxq.tunnel = 0; ++idx; } diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index ae9b564dd..e97f5766c 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -34,6 +34,9 @@ #include "mlx5_defs.h" #include "mlx5_prm.h" +/* Support tunnel matching. */ +#define MLX5_FLOW_TUNNEL 2 + struct mlx5_rxq_stats { unsigned int idx; /**< Mapping index. */ #ifdef MLX5_PMD_SOFT_COUNTERS @@ -139,7 +142,7 @@ struct mlx5_rxq_ctrl { unsigned int irq:1; /* Whether IRQ is enabled. */ uint16_t idx; /* Queue index. */ uint32_t flow_mark_n; /* Number of Mark/Flag flows using this Queue. */ - uint32_t flow_vxlan_n; /* Number of VXLAN flows using this queue. */ + uint32_t flow_tunnels_n[MLX5_FLOW_TUNNEL]; /* Tunnels counters. */ }; /* Indirection table. */ From patchwork Wed Jul 11 07:22:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42812 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 114841B58C; Wed, 11 Jul 2018 09:24:07 +0200 (CEST) Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by dpdk.org (Postfix) with ESMTP id 97F861B476 for ; Wed, 11 Jul 2018 09:23:34 +0200 (CEST) Received: by mail-wr1-f68.google.com with SMTP id p1-v6so16999737wrs.9 for ; Wed, 11 Jul 2018 00:23:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ZK6jeOWSx5vgwuwfJxA2sRxE+benqiDssi4mUszWmC4=; b=zM0vTLvdbnJy4WMeGwxc0JDVlxuiKo0Lrcb5+VE8eow/5Y3XzjlyDTiWcrAHLPRgm+ UNLElOE+rgx2n8K4hZsYNFntRKjsEVUwIl6NW0YNaw+wEj+k8bwDnu04Q0ZaL6zM2aIg ze/dBFdJMvk1Sp7ObMCB4A9yW+KUk0h/3qmDZfSoFwPainjMeXXJ4EwkrXg0wm3zpR8X sJY8BQmMhpqUCdU+f3w/BMf3PxvajlJYv3TU2po4+KU6zZM5snmr/c4F1xMA7UP1eftA HV6V+vhnK6HeDFDws+eoE6WbNjwixOdyo1bmcYYVaea4G4kTMeT17wNZjcipNa9AYnrb fGsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ZK6jeOWSx5vgwuwfJxA2sRxE+benqiDssi4mUszWmC4=; b=ghTKQUAReer+LKOOyfWygNa3XExr52nUm50Uy3TZoZZh67SsxCOoEOtKp4kQ2VQV5l iA+8dh5P/ON72R51uFj5BAdkU/I7a5CkjLOTfiVZqV5iUgu8CaO2q3VNrFqmpYx0KR5B SRJAP6Gn5gvP3o8B8uqCE+Yg+gfhMAjIsc+1SXuTb0+AxAhzINlZI3t8ihTICLBFgIpu 5yyCGbeGDSIPbWRUcdA1OcwEDi3vkp8yQqZQLLLFQypHw0hZGeKrTcauNcLB64F7fjqz 70a1rfV6R47BcHsrokc/cH4RsZZWtHnVyoH1cKSPaOyEQOQDvqZ8aQbaUOP82BIQWFwt k/NQ== X-Gm-Message-State: AOUpUlE4wEOhi8qqFUU/uObo2ucj52OxBdD9TEykDe17KvOWZLu8o8qu m749PDoeT3toSmC20qi3NCjvugD17A== X-Google-Smtp-Source: AAOMgpePZ+y24yjMkLMstQmcxqqMJZEwRwaWmworoZwVZPKJxARDsWVmVl2j52H3OssA2Kr3r11SxA== X-Received: by 2002:adf:bacf:: with SMTP id w15-v6mr13784710wrg.203.1531293814112; Wed, 11 Jul 2018 00:23:34 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:33 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:52 +0200 Message-Id: <671a715dcf5996c4b51581e5c24eb31d0caa3986.1531293415.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 19/21] net/mlx5: add flow GRE item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 193 ++++++++++++++++++++++++++++++++++- drivers/net/mlx5/mlx5_rxtx.h | 2 +- 2 files changed, 192 insertions(+), 3 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index e06df0eb5..3412c856a 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -54,6 +54,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Pattern tunnel Layer bits. */ #define MLX5_FLOW_LAYER_VXLAN (1u << 12) #define MLX5_FLOW_LAYER_VXLAN_GPE (1u << 13) +#define MLX5_FLOW_LAYER_GRE (1u << 14) /* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ @@ -66,7 +67,8 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Tunnel Masks. */ #define MLX5_FLOW_LAYER_TUNNEL \ - (MLX5_FLOW_LAYER_VXLAN | MLX5_FLOW_LAYER_VXLAN_GPE) + (MLX5_FLOW_LAYER_VXLAN | MLX5_FLOW_LAYER_VXLAN_GPE | \ + MLX5_FLOW_LAYER_GRE) /* Inner Masks. */ #define MLX5_FLOW_LAYER_INNER_L3 \ @@ -89,6 +91,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* possible L3 layers protocols filtering. */ #define MLX5_IP_PROTOCOL_TCP 6 #define MLX5_IP_PROTOCOL_UDP 17 +#define MLX5_IP_PROTOCOL_GRE 47 /* Priority reserved for default flows. */ #define MLX5_FLOW_PRIO_RSVD ((uint32_t)-1) @@ -105,6 +108,7 @@ enum mlx5_expansion { MLX5_EXPANSION_OUTER_IPV6_TCP, MLX5_EXPANSION_VXLAN, MLX5_EXPANSION_VXLAN_GPE, + MLX5_EXPANSION_GRE, MLX5_EXPANSION_ETH, MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV4_UDP, @@ -137,7 +141,8 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { [MLX5_EXPANSION_OUTER_IPV4] = { .next = RTE_FLOW_EXPAND_RSS_NEXT (MLX5_EXPANSION_OUTER_IPV4_UDP, - MLX5_EXPANSION_OUTER_IPV4_TCP), + MLX5_EXPANSION_OUTER_IPV4_TCP, + MLX5_EXPANSION_GRE), .type = RTE_FLOW_ITEM_TYPE_IPV4, .rss_types = ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_OTHER, @@ -180,6 +185,10 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { MLX5_EXPANSION_IPV6), .type = RTE_FLOW_ITEM_TYPE_VXLAN_GPE, }, + [MLX5_EXPANSION_GRE] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4), + .type = RTE_FLOW_ITEM_TYPE_GRE, + }, [MLX5_EXPANSION_ETH] = { .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV6), @@ -328,6 +337,10 @@ static struct mlx5_flow_tunnel_info tunnels_info[] = { .tunnel = MLX5_FLOW_LAYER_VXLAN_GPE, .ptype = RTE_PTYPE_TUNNEL_VXLAN_GPE | RTE_PTYPE_L4_UDP, }, + { + .tunnel = MLX5_FLOW_LAYER_GRE, + .ptype = RTE_PTYPE_TUNNEL_GRE, + }, }; /** @@ -966,6 +979,18 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, RTE_FLOW_ERROR_TYPE_ITEM, item, "L3 cannot follow an L4 layer."); + /* + * IPv6 is not recognised by the NIC inside a GRE tunnel. + * Such support has to be disabled as the rule will be + * accepted. Issue reproduced with Mellanox OFED 4.3-3.0.2.1 and + * Mellanox OFED 4.4-1.0.0.0. + */ + if (tunnel && flow->layers & MLX5_FLOW_LAYER_GRE) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "IPv6 inside a GRE tunnel is" + " not recognised."); if (!mask) mask = &rte_flow_item_ipv6_mask; ret = mlx5_flow_item_acceptable @@ -1405,6 +1430,167 @@ mlx5_flow_item_vxlan_gpe(struct rte_eth_dev *dev, return size; } +/** + * Update the protocol in Verbs IPv4/IPv6 spec. + * + * @param[in, out] attr + * Pointer to Verbs attributes structure. + * @param[in] search + * Specification type to search in order to update the IP protocol. + * @param[in] protocol + * Protocol value to set if none is present in the specification. + */ +static void +mlx5_flow_item_gre_ip_protocol_update(struct ibv_flow_attr *attr, + enum ibv_flow_spec_type search, + uint8_t protocol) +{ + unsigned int i; + struct ibv_spec_header *hdr = (struct ibv_spec_header *) + ((uint8_t *)attr + sizeof(struct ibv_flow_attr)); + + if (!attr) + return; + for (i = 0; i != attr->num_of_specs; ++i) { + if (hdr->type == search) { + union { + struct ibv_flow_spec_ipv4_ext *ipv4; + struct ibv_flow_spec_ipv6 *ipv6; + } ip; + + switch (search) { + case IBV_FLOW_SPEC_IPV4_EXT: + ip.ipv4 = (struct ibv_flow_spec_ipv4_ext *)hdr; + if (!ip.ipv4->val.proto) { + ip.ipv4->val.proto = protocol; + ip.ipv4->mask.proto = 0xff; + } + break; + case IBV_FLOW_SPEC_IPV6: + ip.ipv6 = (struct ibv_flow_spec_ipv6 *)hdr; + if (!ip.ipv6->val.next_hdr) { + ip.ipv6->val.next_hdr = protocol; + ip.ipv6->mask.next_hdr = 0xff; + } + break; + default: + break; + } + break; + } + hdr = (struct ibv_spec_header *)((uint8_t *)hdr + hdr->size); + } +} + +/** + * Convert the @p item into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * It will also update the previous L3 layer with the protocol value matching + * the GRE. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param dev + * Pointer to Ethernet device. + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_gre(const struct rte_flow_item *item, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + struct mlx5_flow_verbs *verbs = flow->cur_verbs; + const struct rte_flow_item_gre *spec = item->spec; + const struct rte_flow_item_gre *mask = item->mask; +#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT + unsigned int size = sizeof(struct ibv_flow_spec_gre); + struct ibv_flow_spec_gre tunnel = { + .type = IBV_FLOW_SPEC_GRE, + .size = size, + }; +#else + unsigned int size = sizeof(struct ibv_flow_spec_tunnel); + struct ibv_flow_spec_tunnel tunnel = { + .type = IBV_FLOW_SPEC_VXLAN_TUNNEL, + .size = size, + }; +#endif + int ret; + + if (flow->l3_protocol_en && flow->l3_protocol != MLX5_IP_PROTOCOL_GRE) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "protocol filtering not compatible" + " with this GRE layer"); + if (flow->layers & MLX5_FLOW_LAYER_TUNNEL) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "a tunnel is already present"); + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 Layer is missing"); + if (!mask) + mask = &rte_flow_item_gre_mask; + ret = mlx5_flow_item_acceptable + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_gre_mask, + sizeof(struct rte_flow_item_gre), error); + if (ret < 0) + return ret; +#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT + if (spec) { + tunnel.val.c_ks_res0_ver = spec->c_rsvd0_ver; + tunnel.val.protocol = spec->protocol; + tunnel.mask.c_ks_res0_ver = mask->c_rsvd0_ver; + tunnel.mask.protocol = mask->protocol; + /* Remove unwanted bits from values. */ + tunnel.val.c_ks_res0_ver &= tunnel.mask.c_ks_res0_ver; + tunnel.val.protocol &= tunnel.mask.protocol; + tunnel.val.key &= tunnel.mask.key; + } +#else + if (spec && (spec->protocol & mask->protocol)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "without MPLS support the" + " specification cannot be used for" + " filtering"); +#endif /* !HAVE_IBV_DEVICE_MPLS_SUPPORT */ + if (size <= flow_size) { + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L3_IPV4) + mlx5_flow_item_gre_ip_protocol_update + (verbs->attr, IBV_FLOW_SPEC_IPV4_EXT, + MLX5_IP_PROTOCOL_GRE); + else + mlx5_flow_item_gre_ip_protocol_update + (verbs->attr, IBV_FLOW_SPEC_IPV6, + MLX5_IP_PROTOCOL_GRE); + mlx5_flow_spec_verbs_add(flow, &tunnel, size); + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L2; + } + flow->layers |= MLX5_FLOW_LAYER_GRE; + return size; +} + /** * Convert the @p pattern into a Verbs specifications after ensuring the NIC * will understand and process it correctly. @@ -1471,6 +1657,9 @@ mlx5_flow_items(struct rte_eth_dev *dev, ret = mlx5_flow_item_vxlan_gpe(dev, pattern, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_GRE: + ret = mlx5_flow_item_gre(pattern, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index e97f5766c..53992e8db 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -35,7 +35,7 @@ #include "mlx5_prm.h" /* Support tunnel matching. */ -#define MLX5_FLOW_TUNNEL 2 +#define MLX5_FLOW_TUNNEL 3 struct mlx5_rxq_stats { unsigned int idx; /**< Mapping index. */ From patchwork Wed Jul 11 07:22:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42813 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3F7751B501; Wed, 11 Jul 2018 09:24:09 +0200 (CEST) Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by dpdk.org (Postfix) with ESMTP id 761CF1B4EE for ; Wed, 11 Jul 2018 09:23:35 +0200 (CEST) Received: by mail-wr1-f66.google.com with SMTP id p1-v6so16999764wrs.9 for ; Wed, 11 Jul 2018 00:23:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=La/1bqDoK5eNf7IegmAgKJlT68F29bkVG8vQbA3G0ow=; b=d9cgB9brbNfoc7a//cUUHbYCQAqWEUjppykVhwlVAdFyPTBj73oqzXFQ4w5n2Gy2kl P5qQro1U0elQwq7Z7fp1NatHEpqmCCVgxuFZWFBbu43+589f8RJza45bmuwdvsMKXdQq v/q0QGsnVsUKhjx9tvL0zkyC8hSlvCGZ9butu4YeUJQV/3y2V8csuwzmaSb5RPPBbie9 hBgqGgvNKkjJJBfmeL5cxKGnKmlX0Zww+6Yvmtu3W3FuoQKPl4eWVvdx7x8vJdGHYBve A6g8oPgHdIXBdy2b/arzGr6KyrJhZbcMC1YesYNXGet0cbVXkCSufK9QRejp5qSsfT+K dvSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=La/1bqDoK5eNf7IegmAgKJlT68F29bkVG8vQbA3G0ow=; b=QVQQrFd1Cox4dwKFrGA4v9g02jwrBvVATRvw9s4vZu7E5x3+60X5hwW/Q5z9yfqwFX gC8+gxeXzmDrVrLjjOxvsyv+HhQSQT0CkgH5OhLfnOSu5Bn3wMYoKwiD0+vimCsVF7Fo swgjkU2HXAjj6tiXVKW9Uvjm4G+zpPMGi3sXtFgp+jVCXBOJcjsNkqdNft/4FIWQs5Gh ztgLB3hG9xdd5FTKQDxt99NayI4y7qjOpjdBO8WEfXOukfw9dZDSaNm8i0Vbsq0vQ0BF YnZ8sRMKVKat44cVAcccb1soZBA50689jwxnoZNpKt3QUGMAFJBOrdrUl1Lv6/MGKwFL hTzg== X-Gm-Message-State: AOUpUlEUm5ZnzaRBsx9nQjPD+ac5iIrkwxmQx8BVeGUss9h/wwOW76iP QC/iz33Kl8saEf9qJATYrLsdyX08Kw== X-Google-Smtp-Source: AAOMgpdqV2f7clEKNDYEusZD79d/PFFDICUG21x5iYEPx/5WC4v+3D8IS7xX4gs7dbc2u/GWQhxaRw== X-Received: by 2002:adf:81c3:: with SMTP id 61-v6mr7425779wra.120.1531293815010; Wed, 11 Jul 2018 00:23:35 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:34 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:53 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 20/21] net/mlx5: add flow MPLS item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 102 ++++++++++++++++++++++++++++++++++- drivers/net/mlx5/mlx5_rxtx.h | 2 +- 2 files changed, 101 insertions(+), 3 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 3412c856a..4ad581ed9 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -55,6 +55,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; #define MLX5_FLOW_LAYER_VXLAN (1u << 12) #define MLX5_FLOW_LAYER_VXLAN_GPE (1u << 13) #define MLX5_FLOW_LAYER_GRE (1u << 14) +#define MLX5_FLOW_LAYER_MPLS (1u << 15) /* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ @@ -68,7 +69,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Tunnel Masks. */ #define MLX5_FLOW_LAYER_TUNNEL \ (MLX5_FLOW_LAYER_VXLAN | MLX5_FLOW_LAYER_VXLAN_GPE | \ - MLX5_FLOW_LAYER_GRE) + MLX5_FLOW_LAYER_GRE | MLX5_FLOW_LAYER_MPLS) /* Inner Masks. */ #define MLX5_FLOW_LAYER_INNER_L3 \ @@ -92,6 +93,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; #define MLX5_IP_PROTOCOL_TCP 6 #define MLX5_IP_PROTOCOL_UDP 17 #define MLX5_IP_PROTOCOL_GRE 47 +#define MLX5_IP_PROTOCOL_MPLS 147 /* Priority reserved for default flows. */ #define MLX5_FLOW_PRIO_RSVD ((uint32_t)-1) @@ -109,6 +111,7 @@ enum mlx5_expansion { MLX5_EXPANSION_VXLAN, MLX5_EXPANSION_VXLAN_GPE, MLX5_EXPANSION_GRE, + MLX5_EXPANSION_MPLS, MLX5_EXPANSION_ETH, MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV4_UDP, @@ -134,7 +137,8 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { }, [MLX5_EXPANSION_OUTER_ETH] = { .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_OUTER_IPV4, - MLX5_EXPANSION_OUTER_IPV6), + MLX5_EXPANSION_OUTER_IPV6, + MLX5_EXPANSION_MPLS), .type = RTE_FLOW_ITEM_TYPE_ETH, .rss_types = 0, }, @@ -189,6 +193,11 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4), .type = RTE_FLOW_ITEM_TYPE_GRE, }, + [MLX5_EXPANSION_MPLS] = { + .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4, + MLX5_EXPANSION_IPV6), + .type = RTE_FLOW_ITEM_TYPE_MPLS, + }, [MLX5_EXPANSION_ETH] = { .next = RTE_FLOW_EXPAND_RSS_NEXT(MLX5_EXPANSION_IPV4, MLX5_EXPANSION_IPV6), @@ -341,6 +350,14 @@ static struct mlx5_flow_tunnel_info tunnels_info[] = { .tunnel = MLX5_FLOW_LAYER_GRE, .ptype = RTE_PTYPE_TUNNEL_GRE, }, + { + .tunnel = MLX5_FLOW_LAYER_MPLS | MLX5_FLOW_LAYER_OUTER_L4_UDP, + .ptype = RTE_PTYPE_TUNNEL_MPLS_IN_GRE | RTE_PTYPE_L4_UDP, + }, + { + .tunnel = MLX5_FLOW_LAYER_MPLS, + .ptype = RTE_PTYPE_TUNNEL_MPLS_IN_GRE, + }, }; /** @@ -1591,6 +1608,84 @@ mlx5_flow_item_gre(const struct rte_flow_item *item, return size; } +/** + * Convert the @p item into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param[in] item + * Item specification. + * @param[in, out] flow + * Pointer to flow structure. + * @param[in] flow_size + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param[out] error + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p item has fully been converted, + * otherwise another call with this returned memory size should be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_item_mpls(const struct rte_flow_item *item __rte_unused, + struct rte_flow *flow __rte_unused, + const size_t flow_size __rte_unused, + struct rte_flow_error *error) +{ +#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT + const struct rte_flow_item_mpls *spec = item->spec; + const struct rte_flow_item_mpls *mask = item->mask; + unsigned int size = sizeof(struct ibv_flow_spec_mpls); + struct ibv_flow_spec_mpls mpls = { + .type = IBV_FLOW_SPEC_MPLS, + .size = size, + }; + int ret; + + if (flow->l3_protocol_en && flow->l3_protocol != MLX5_IP_PROTOCOL_MPLS) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "protocol filtering not compatible" + " with MPLS layer"); + if (flow->layers & MLX5_FLOW_LAYER_TUNNEL) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "a tunnel is already" + " present"); + if (!mask) + mask = &rte_flow_item_mpls_mask; + ret = mlx5_flow_item_acceptable + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_mpls_mask, + sizeof(struct rte_flow_item_mpls), error); + if (ret < 0) + return ret; + if (spec) { + memcpy(&mpls.val.label, spec, sizeof(mpls.val.label)); + memcpy(&mpls.mask.label, mask, sizeof(mpls.mask.label)); + /* Remove unwanted bits from values. */ + mpls.val.label &= mpls.mask.label; + } + if (size <= flow_size) { + mlx5_flow_spec_verbs_add(flow, &mpls, size); + flow->cur_verbs->attr->priority = MLX5_PRIORITY_MAP_L2; + } + flow->layers |= MLX5_FLOW_LAYER_MPLS; + return size; +#endif /* !HAVE_IBV_DEVICE_MPLS_SUPPORT */ + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "MPLS is not supported by Verbs, please" + " update."); +} + /** * Convert the @p pattern into a Verbs specifications after ensuring the NIC * will understand and process it correctly. @@ -1660,6 +1755,9 @@ mlx5_flow_items(struct rte_eth_dev *dev, case RTE_FLOW_ITEM_TYPE_GRE: ret = mlx5_flow_item_gre(pattern, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_MPLS: + ret = mlx5_flow_item_mpls(pattern, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 53992e8db..a04a84fec 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -35,7 +35,7 @@ #include "mlx5_prm.h" /* Support tunnel matching. */ -#define MLX5_FLOW_TUNNEL 3 +#define MLX5_FLOW_TUNNEL 5 struct mlx5_rxq_stats { unsigned int idx; /**< Mapping index. */ From patchwork Wed Jul 11 07:22:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 42814 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F3A4F1B599; Wed, 11 Jul 2018 09:24:10 +0200 (CEST) Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by dpdk.org (Postfix) with ESMTP id 5EA671B4F5 for ; Wed, 11 Jul 2018 09:23:36 +0200 (CEST) Received: by mail-wr1-f52.google.com with SMTP id r16-v6so17015587wrt.11 for ; Wed, 11 Jul 2018 00:23:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1gQw6f9oVc42vstXV4CQeEgttDFOGkY/ozLr86KpGLE=; b=honjJp4ehazIdpJIrbD3R3z/KMZboX+SnLHnn4YD9BZfM7ipBtxnGOnAeF8KDoSH1k /4ig0qJjIBgmkToCadxK13qJbvfBAxOLFE/1bcITBvJDAGPIvOE3HNOdZgfG+0hG/07S /Al48K2w5kOZ4EOYO+cmJ9f+uezXSUuJDulpHI4+7bF/V9scUK2bsa1AfWSW76LD76g2 ancyMG+Rmte9exKum8GZILO5256GCfcrL7d9G6OGZSGStQQ960mdZW3DiTCT3iWgUxgw cemwK7acJUwBbU/T2fo45UEYRh1M7F2lfPpK4hsLmqa5HUyS9Yi9ZeUz4e5ltdZx2Dr5 svvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1gQw6f9oVc42vstXV4CQeEgttDFOGkY/ozLr86KpGLE=; b=Ta8mX7bxhngwTt5kTu2p4BK9/HNpyvxl+VsQq8VF9hkO0QqT2ByGkaikUirL3spYEX iWVZIUFT2hYEpzLcft8ZrDcJW4ldc66MXH6OcPFyQ+O7ViWZNFtOBFbOcaEXcZo2kpeN 3+KTm8312J0Yj2rdnVxghKYx96zkY3Hv4bqkS5x8hbycH/62tFhz+HLB/S3OEe5qDyI0 2ROMk4zvMEahZKxaU8N6andeZoqDDEusr7JQVmsS/qmF2rE95h6uvzY7JXS7ohIT156z hWzJbjomcUpvWlFMZ+88FJ1orJ/wzt6znf6WYJzXGxs+ggO55PQcOeVA3E6wpEJL5XDl f+6w== X-Gm-Message-State: APt69E3qmtnzy+zU6ydQIbbIW+7oEBrDFlzwrIMjrhn8iG4cNo0A4Y0X nfqyFCjFGGiJdiri/rXos/91YcZTLw== X-Google-Smtp-Source: AAOMgpcfVjloFsVP2terSdHABg0i2AM6T6t6v0I7OlM5OsKeAbOeYoyOWWEme7JYNGZygGIu3Eg6VA== X-Received: by 2002:a5d:4e49:: with SMTP id r9-v6mr18530018wrt.27.1531293815877; Wed, 11 Jul 2018 00:23:35 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id t10-v6sm31314212wre.95.2018.07.11.00.23.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 00:23:35 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Yongseok Koh Cc: Adrien Mazarguil Date: Wed, 11 Jul 2018 09:22:54 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 21/21] net/mlx5: add count flow action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is only supported by Mellanox OFED. Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_flow.c | 242 +++++++++++++++++++++++++++++++++++ 2 files changed, 244 insertions(+) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 9949cd3fa..131be334c 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -188,6 +188,8 @@ struct priv { struct mlx5_drop drop_queue; /* Flow drop queues. */ struct mlx5_flows flows; /* RTE Flow rules. */ struct mlx5_flows ctrl_flows; /* Control flow rules. */ + LIST_HEAD(counters, mlx5_flow_counter) flow_counters; + /* Flow counters. */ struct { uint32_t dev_gen; /* Generation number to flush local caches. */ rte_rwlock_t rwlock; /* MR Lock. */ diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 4ad581ed9..9ce4dddd7 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -88,6 +88,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Modify a packet. */ #define MLX5_FLOW_MOD_FLAG (1u << 0) #define MLX5_FLOW_MOD_MARK (1u << 1) +#define MLX5_FLOW_MOD_COUNT (1u << 2) /* possible L3 layers protocols filtering. */ #define MLX5_IP_PROTOCOL_TCP 6 @@ -249,6 +250,17 @@ struct mlx5_flow_verbs { uint64_t hash_fields; /**< Verbs hash Rx queue hash fields. */ }; +/* Counters information. */ +struct mlx5_flow_counter { + LIST_ENTRY(mlx5_flow_counter) next; /**< Pointer to the next counter. */ + uint32_t shared:1; /**< Share counter ID with other flow rules. */ + uint32_t ref_cnt:31; /**< Reference counter. */ + uint32_t id; /**< Counter ID. */ + struct ibv_counter_set *cs; /**< Holds the counters for the rule. */ + uint64_t hits; /**< Number of packets matched by the rule. */ + uint64_t bytes; /**< Number of bytes matched by the rule. */ +}; + /* Flow structure. */ struct rte_flow { TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ @@ -264,6 +276,7 @@ struct rte_flow { LIST_HEAD(verbs, mlx5_flow_verbs) verbs; /**< Verbs flows list. */ struct mlx5_flow_verbs *cur_verbs; /**< Current Verbs flow structure being filled. */ + struct mlx5_flow_counter *counter; /**< Holds Verbs flow counter. */ struct rte_flow_action_rss rss;/**< RSS context. */ uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */ uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */ @@ -275,6 +288,7 @@ static const struct rte_flow_ops mlx5_flow_ops = { .destroy = mlx5_flow_destroy, .flush = mlx5_flow_flush, .isolate = mlx5_flow_isolate, + .query = mlx5_flow_query, }; /* Convert FDIR request to Generic flow. */ @@ -454,6 +468,80 @@ mlx5_flow_adjust_priority(struct rte_eth_dev *dev, struct rte_flow *flow) flow->cur_verbs->attr->priority = priority; } +/** + * Get a flow counter. + * + * @param[in] dev + * Pointer to Ethernet device. + * @param[in] shared + * Indicate if this counter is shared with other flows. + * @param[in] id + * Counter identifier. + * + * @return + * A pointer to the counter, NULL otherwise and rte_errno is set. + */ +static struct mlx5_flow_counter * +mlx5_flow_counter_new(struct rte_eth_dev *dev, uint32_t shared, uint32_t id) +{ + struct priv *priv = dev->data->dev_private; + struct mlx5_flow_counter *cnt; + + LIST_FOREACH(cnt, &priv->flow_counters, next) { + if (cnt->shared != shared) + continue; + if (cnt->id != id) + continue; + cnt->ref_cnt++; + return cnt; + } +#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT + + struct mlx5_flow_counter tmpl = { + .shared = shared, + .id = id, + .cs = mlx5_glue->create_counter_set + (priv->ctx, + &(struct ibv_counter_set_init_attr){ + .counter_set_id = id, + }), + .hits = 0, + .bytes = 0, + }; + + if (!tmpl.cs) { + rte_errno = errno; + return NULL; + } + cnt = rte_calloc(__func__, 1, sizeof(*cnt), 0); + if (!cnt) { + rte_errno = ENOMEM; + return NULL; + } + *cnt = tmpl; + LIST_INSERT_HEAD(&priv->flow_counters, cnt, next); + return cnt; +#endif + rte_errno = ENOTSUP; + return NULL; +} + +/** + * Release a flow counter. + * + * @param[in] counter + * Pointer to the counter handler. + */ +static void +mlx5_flow_counter_release(struct mlx5_flow_counter *counter) +{ + if (--counter->ref_cnt == 0) { + claim_zero(mlx5_glue->destroy_counter_set(counter->cs)); + LIST_REMOVE(counter, next); + rte_free(counter); + } +} + /** * Verify the @p attributes will be correctly understood by the NIC and store * them in the @p flow if everything is correct. @@ -2121,6 +2209,70 @@ mlx5_flow_action_mark(const struct rte_flow_action *action, return size; } +/** + * Convert the @p action into a Verbs specification after ensuring the NIC + * will understand and process it correctly. + * If the necessary size for the conversion is greater than the @p flow_size, + * nothing is written in @p flow, the validation is still performed. + * + * @param action[in] + * Action configuration. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size in bytes of the available space in @p flow, if too small, nothing is + * written. + * @param error[int, out] + * Pointer to error structure. + * + * @return + * On success the number of bytes consumed/necessary, if the returned value + * is lesser or equal to @p flow_size, the @p action has fully been + * converted, otherwise another call with this returned memory size should + * be done. + * On error, a negative errno value is returned and rte_errno is set. + */ +static int +mlx5_flow_action_count(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow *flow, + const size_t flow_size __rte_unused, + struct rte_flow_error *error) +{ + const struct rte_flow_action_count *count = action->conf; +#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT + unsigned int size = sizeof(struct ibv_flow_spec_counter_action); + struct ibv_flow_spec_counter_action counter = { + .type = IBV_FLOW_SPEC_ACTION_COUNT, + .size = size, + }; +#endif + + if (!flow->counter) { + flow->counter = mlx5_flow_counter_new(dev, count->shared, + count->id); + if (!flow->counter) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "cannot get counter" + " context."); + } + if (!((struct priv *)dev->data->dev_private)->config.flow_counter_en) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + action, + "flow counters are not supported."); + flow->modifier |= MLX5_FLOW_MOD_COUNT; +#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT + counter.counter_set_handle = flow->counter->cs->handle; + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &counter, size); + return size; +#endif + return 0; +} + /** * Convert the @p action into @p flow after ensuring the NIC will understand * and process it correctly. @@ -2180,6 +2332,10 @@ mlx5_flow_actions(struct rte_eth_dev *dev, case RTE_FLOW_ACTION_TYPE_RSS: ret = mlx5_flow_action_rss(dev, actions, flow, error); break; + case RTE_FLOW_ACTION_TYPE_COUNT: + ret = mlx5_flow_action_count(dev, actions, flow, remain, + error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -2557,6 +2713,10 @@ mlx5_flow_remove(struct rte_eth_dev *dev, struct rte_flow *flow) verbs->hrxq = NULL; } } + if (flow->counter) { + mlx5_flow_counter_release(flow->counter); + flow->counter = NULL; + } } /** @@ -3006,6 +3166,88 @@ mlx5_flow_isolate(struct rte_eth_dev *dev, return 0; } +/** + * Query flow counter. + * + * @param flow + * Pointer to the flow. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_query_count(struct rte_flow *flow __rte_unused, + void *data __rte_unused, + struct rte_flow_error *error) +{ +#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT + struct rte_flow_query_count *qc = data; + uint64_t counters[2] = {0, 0}; + struct ibv_query_counter_set_attr query_cs_attr = { + .cs = flow->counter->cs, + .query_flags = IBV_COUNTER_SET_FORCE_UPDATE, + }; + struct ibv_counter_set_data query_out = { + .out = counters, + .outlen = 2 * sizeof(uint64_t), + }; + int err = mlx5_glue->query_counter_set(&query_cs_attr, &query_out); + + if (err) + return rte_flow_error_set(error, err, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot read counter"); + qc->hits_set = 1; + qc->bytes_set = 1; + qc->hits = counters[0] - flow->counter->hits; + qc->bytes = counters[1] - flow->counter->bytes; + if (qc->reset) { + flow->counter->hits = counters[0]; + flow->counter->bytes = counters[1]; + } + return 0; +#endif + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "counters are not available"); +} + +/** + * Query a flows. + * + * @see rte_flow_query() + * @see rte_flow_ops + */ +int +mlx5_flow_query(struct rte_eth_dev *dev __rte_unused, + struct rte_flow *flow, + const struct rte_flow_action *actions, + void *data, + struct rte_flow_error *error) +{ + int ret = 0; + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_VOID: + break; + case RTE_FLOW_ACTION_TYPE_COUNT: + ret = mlx5_flow_query_count(flow, data, error); + break; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "action not supported"); + } + if (ret < 0) + return ret; + } + return 0; +} + /** * Convert a flow director filter to a generic flow. *