From patchwork Wed Jun 27 15:07:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41677 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5BDFC1BFAB; Wed, 27 Jun 2018 17:07:42 +0200 (CEST) Received: from mail-wr0-f173.google.com (mail-wr0-f173.google.com [209.85.128.173]) by dpdk.org (Postfix) with ESMTP id EEC991BF9B for ; Wed, 27 Jun 2018 17:07:36 +0200 (CEST) Received: by mail-wr0-f173.google.com with SMTP id c13-v6so2409364wrq.2 for ; Wed, 27 Jun 2018 08:07:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=6/Cwfvq9tKo+hZOYGChlbwgk8xVXePb/5UDVOJdrtBw=; b=bYFD0MJx/FIgyZVZ1Af6ovc7aeMOWDXx7z3TVQ8RS6jFRtY57z5k5TFU3tk8jFFs/W +Fchjlhyk5FqJR0abECR3ifK+26Wm+7YtpA5PQOztkS0nt1+xICNoKT1tdCZuoTmOyYr 9jLxdWLh9X+OZUC28DQRcVhP8yz805rdF5cT8XBuVjgwFbfIoJs/p0Q7uAPSyeTjuRnp KTEDxA8LGb1GB7td/f1DFLAAjrsFf8C0hmhJyrDdXQckaBAL3iH9DLtPItR13oKQXKI9 MSHtJ5YvtsHd4pFwsJm9rW3zu+tOqGfE9VxWXJY9zlMbDaqc331X+nT2DgTeb6k3DPaN r1sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=6/Cwfvq9tKo+hZOYGChlbwgk8xVXePb/5UDVOJdrtBw=; b=Xy86yYKgx1HBB++BaGDUoNrsQ+1RExJWjF4s1v2BvbVjE8X2+upspPELlsO0uALM4X cIiaq9H+LPNS+s+S/+0BTU35UgBLcfjgVkywhidnjhQ0LWRyy670NYVxr2+sXTYXnQyk 4I71ocBXXJ8/o6FyZre5rllzTt+r2FLSAH01IxT4j8CTfOCtEKaIYcgfkUuB/VODjwSv 5PmG4gv7gikaimSPSYK1i6pu38ifSsKWaUrOMR3AxjRvLkLO20W877Y45+K9zfeOlFbR BphJiYZM4FX2xeWCmlsUHOLQov/QyvGEruCXMjO4dTtfgphsTaYR0vR3T1byD1VC96nC EcBQ== X-Gm-Message-State: APt69E3xzmT9TqXkS479WnF2XG5WpJ+5os0hsWkwRTg57xl/6quh6e1x tq/WUFc/NSyn5Ztgk1/x5fGC5MTj1Q== X-Google-Smtp-Source: AAOMgpc8VceiPnvyv0YyxixRwWtWqxfZ3WipAHQpmhVSh6ZaEtTrQptxbUqYCP2BcDgMRBrdr4ipLA== X-Received: by 2002:adf:92a5:: with SMTP id 34-v6mr5862414wrn.83.1530112055001; Wed, 27 Jun 2018 08:07:35 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:34 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:33 +0200 Message-Id: <03bce7c3853bdcab01cccf7d0ee200697666f3bb.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 01/20] net/mlx5: remove flow support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This start a series to re-work the flow engine in mlx5 to easily support flow conversion to Verbs or TC. This is necessary to handle both regular flows and representors flows. As the full file needs to be clean-up to re-write all items/actions processing, this patch starts to disable the regular code and only let the PMD to start in isolated mode. After this patch flow API will not be usable. Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 3095 +--------------------------------- 1 file changed, 80 insertions(+), 3015 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 45207d70e..a45cb06e1 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -31,2406 +31,49 @@ #include "mlx5_prm.h" #include "mlx5_glue.h" -/* Flow priority for control plane flows. */ -#define MLX5_CTRL_FLOW_PRIORITY 1 - -/* Internet Protocol versions. */ -#define MLX5_IPV4 4 -#define MLX5_IPV6 6 -#define MLX5_GRE 47 - -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT -struct ibv_flow_spec_counter_action { - int dummy; -}; -#endif - -/* Dev ops structure defined in mlx5.c */ -extern const struct eth_dev_ops mlx5_dev_ops; -extern const struct eth_dev_ops mlx5_dev_ops_isolate; - -/** Structure give to the conversion functions. */ -struct mlx5_flow_data { - struct rte_eth_dev *dev; /** Ethernet device. */ - struct mlx5_flow_parse *parser; /** Parser context. */ - struct rte_flow_error *error; /** Error context. */ -}; - -static int -mlx5_flow_create_eth(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_vlan(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_ipv4(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_ipv6(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_udp(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_tcp(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_vxlan(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_vxlan_gpe(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_gre(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -static int -mlx5_flow_create_mpls(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - -struct mlx5_flow_parse; - -static void -mlx5_flow_create_copy(struct mlx5_flow_parse *parser, void *src, - unsigned int size); - -static int -mlx5_flow_create_flag_mark(struct mlx5_flow_parse *parser, uint32_t mark_id); - -static int -mlx5_flow_create_count(struct rte_eth_dev *dev, struct mlx5_flow_parse *parser); - -/* Hash RX queue types. */ -enum hash_rxq_type { - HASH_RXQ_TCPV4, - HASH_RXQ_UDPV4, - HASH_RXQ_IPV4, - HASH_RXQ_TCPV6, - HASH_RXQ_UDPV6, - HASH_RXQ_IPV6, - HASH_RXQ_ETH, - HASH_RXQ_TUNNEL, -}; - -/* Initialization data for hash RX queue. */ -struct hash_rxq_init { - uint64_t hash_fields; /* Fields that participate in the hash. */ - uint64_t dpdk_rss_hf; /* Matching DPDK RSS hash fields. */ - unsigned int flow_priority; /* Flow priority to use. */ - unsigned int ip_version; /* Internet protocol. */ -}; - -/* Initialization data for hash RX queues. */ -const struct hash_rxq_init hash_rxq_init[] = { - [HASH_RXQ_TCPV4] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV4 | - IBV_RX_HASH_DST_IPV4 | - IBV_RX_HASH_SRC_PORT_TCP | - IBV_RX_HASH_DST_PORT_TCP), - .dpdk_rss_hf = ETH_RSS_NONFRAG_IPV4_TCP, - .flow_priority = 0, - .ip_version = MLX5_IPV4, - }, - [HASH_RXQ_UDPV4] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV4 | - IBV_RX_HASH_DST_IPV4 | - IBV_RX_HASH_SRC_PORT_UDP | - IBV_RX_HASH_DST_PORT_UDP), - .dpdk_rss_hf = ETH_RSS_NONFRAG_IPV4_UDP, - .flow_priority = 0, - .ip_version = MLX5_IPV4, - }, - [HASH_RXQ_IPV4] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV4 | - IBV_RX_HASH_DST_IPV4), - .dpdk_rss_hf = (ETH_RSS_IPV4 | - ETH_RSS_FRAG_IPV4), - .flow_priority = 1, - .ip_version = MLX5_IPV4, - }, - [HASH_RXQ_TCPV6] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV6 | - IBV_RX_HASH_DST_IPV6 | - IBV_RX_HASH_SRC_PORT_TCP | - IBV_RX_HASH_DST_PORT_TCP), - .dpdk_rss_hf = ETH_RSS_NONFRAG_IPV6_TCP, - .flow_priority = 0, - .ip_version = MLX5_IPV6, - }, - [HASH_RXQ_UDPV6] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV6 | - IBV_RX_HASH_DST_IPV6 | - IBV_RX_HASH_SRC_PORT_UDP | - IBV_RX_HASH_DST_PORT_UDP), - .dpdk_rss_hf = ETH_RSS_NONFRAG_IPV6_UDP, - .flow_priority = 0, - .ip_version = MLX5_IPV6, - }, - [HASH_RXQ_IPV6] = { - .hash_fields = (IBV_RX_HASH_SRC_IPV6 | - IBV_RX_HASH_DST_IPV6), - .dpdk_rss_hf = (ETH_RSS_IPV6 | - ETH_RSS_FRAG_IPV6), - .flow_priority = 1, - .ip_version = MLX5_IPV6, - }, - [HASH_RXQ_ETH] = { - .hash_fields = 0, - .dpdk_rss_hf = 0, - .flow_priority = 2, - }, -}; - -/* Number of entries in hash_rxq_init[]. */ -const unsigned int hash_rxq_init_n = RTE_DIM(hash_rxq_init); - -/** Structure for holding counter stats. */ -struct mlx5_flow_counter_stats { - uint64_t hits; /**< Number of packets matched by the rule. */ - uint64_t bytes; /**< Number of bytes matched by the rule. */ -}; - -/** Structure for Drop queue. */ -struct mlx5_hrxq_drop { - struct ibv_rwq_ind_table *ind_table; /**< Indirection table. */ - struct ibv_qp *qp; /**< Verbs queue pair. */ - struct ibv_wq *wq; /**< Verbs work queue. */ - struct ibv_cq *cq; /**< Verbs completion queue. */ -}; - -/* Flows structures. */ -struct mlx5_flow { - uint64_t hash_fields; /**< Fields that participate in the hash. */ - struct ibv_flow_attr *ibv_attr; /**< Pointer to Verbs attributes. */ - struct ibv_flow *ibv_flow; /**< Verbs flow. */ - struct mlx5_hrxq *hrxq; /**< Hash Rx queues. */ -}; - -/* Drop flows structures. */ -struct mlx5_flow_drop { - struct ibv_flow_attr *ibv_attr; /**< Pointer to Verbs attributes. */ - struct ibv_flow *ibv_flow; /**< Verbs flow. */ -}; - -struct rte_flow { - TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ - uint32_t mark:1; /**< Set if the flow is marked. */ - uint32_t drop:1; /**< Drop queue. */ - struct rte_flow_action_rss rss_conf; /**< RSS configuration */ - uint16_t (*queues)[]; /**< Queues indexes to use. */ - uint8_t rss_key[40]; /**< copy of the RSS key. */ - uint32_t tunnel; /**< Tunnel type of RTE_PTYPE_TUNNEL_XXX. */ - struct ibv_counter_set *cs; /**< Holds the counters for the rule. */ - struct mlx5_flow_counter_stats counter_stats;/**> 12) - -const uint32_t ptype_ext[] = { - [PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN)] = RTE_PTYPE_TUNNEL_VXLAN | - RTE_PTYPE_L4_UDP, - [PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN_GPE)] = RTE_PTYPE_TUNNEL_VXLAN_GPE | - RTE_PTYPE_L4_UDP, - [PTYPE_IDX(RTE_PTYPE_TUNNEL_GRE)] = RTE_PTYPE_TUNNEL_GRE, - [PTYPE_IDX(RTE_PTYPE_TUNNEL_MPLS_IN_GRE)] = - RTE_PTYPE_TUNNEL_MPLS_IN_GRE, - [PTYPE_IDX(RTE_PTYPE_TUNNEL_MPLS_IN_UDP)] = - RTE_PTYPE_TUNNEL_MPLS_IN_GRE | RTE_PTYPE_L4_UDP, -}; - -/** Structure to generate a simple graph of layers supported by the NIC. */ -struct mlx5_flow_items { - /** List of possible actions for these items. */ - const enum rte_flow_action_type *const actions; - /** Bit-masks corresponding to the possibilities for the item. */ - const void *mask; - /** - * Default bit-masks to use when item->mask is not provided. When - * \default_mask is also NULL, the full supported bit-mask (\mask) is - * used instead. - */ - const void *default_mask; - /** Bit-masks size in bytes. */ - const unsigned int mask_sz; - /** - * Conversion function from rte_flow to NIC specific flow. - * - * @param item - * rte_flow item to convert. - * @param default_mask - * Default bit-masks to use when item->mask is not provided. - * @param data - * Internal structure to store the conversion. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is - * set. - */ - int (*convert)(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data); - /** Size in bytes of the destination structure. */ - const unsigned int dst_sz; - /** List of possible following items. */ - const enum rte_flow_item_type *const items; -}; - -/** Valid action for this PMD. */ -static const enum rte_flow_action_type valid_actions[] = { - RTE_FLOW_ACTION_TYPE_DROP, - RTE_FLOW_ACTION_TYPE_QUEUE, - RTE_FLOW_ACTION_TYPE_MARK, - RTE_FLOW_ACTION_TYPE_FLAG, -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT - RTE_FLOW_ACTION_TYPE_COUNT, -#endif - RTE_FLOW_ACTION_TYPE_END, -}; - -/** Graph of supported items and associated actions. */ -static const struct mlx5_flow_items mlx5_flow_items[] = { - [RTE_FLOW_ITEM_TYPE_END] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_VXLAN, - RTE_FLOW_ITEM_TYPE_VXLAN_GPE, - RTE_FLOW_ITEM_TYPE_GRE), - }, - [RTE_FLOW_ITEM_TYPE_ETH] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_VLAN, - RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_eth){ - .dst.addr_bytes = "\xff\xff\xff\xff\xff\xff", - .src.addr_bytes = "\xff\xff\xff\xff\xff\xff", - .type = -1, - }, - .default_mask = &rte_flow_item_eth_mask, - .mask_sz = sizeof(struct rte_flow_item_eth), - .convert = mlx5_flow_create_eth, - .dst_sz = sizeof(struct ibv_flow_spec_eth), - }, - [RTE_FLOW_ITEM_TYPE_VLAN] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_vlan){ - .tci = -1, - .inner_type = -1, - }, - .default_mask = &rte_flow_item_vlan_mask, - .mask_sz = sizeof(struct rte_flow_item_vlan), - .convert = mlx5_flow_create_vlan, - .dst_sz = 0, - }, - [RTE_FLOW_ITEM_TYPE_IPV4] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_UDP, - RTE_FLOW_ITEM_TYPE_TCP, - RTE_FLOW_ITEM_TYPE_GRE), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_ipv4){ - .hdr = { - .src_addr = -1, - .dst_addr = -1, - .type_of_service = -1, - .next_proto_id = -1, - }, - }, - .default_mask = &rte_flow_item_ipv4_mask, - .mask_sz = sizeof(struct rte_flow_item_ipv4), - .convert = mlx5_flow_create_ipv4, - .dst_sz = sizeof(struct ibv_flow_spec_ipv4_ext), - }, - [RTE_FLOW_ITEM_TYPE_IPV6] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_UDP, - RTE_FLOW_ITEM_TYPE_TCP, - RTE_FLOW_ITEM_TYPE_GRE), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_ipv6){ - .hdr = { - .src_addr = { - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - }, - .dst_addr = { - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, - }, - .vtc_flow = -1, - .proto = -1, - .hop_limits = -1, - }, - }, - .default_mask = &rte_flow_item_ipv6_mask, - .mask_sz = sizeof(struct rte_flow_item_ipv6), - .convert = mlx5_flow_create_ipv6, - .dst_sz = sizeof(struct ibv_flow_spec_ipv6), - }, - [RTE_FLOW_ITEM_TYPE_UDP] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_VXLAN, - RTE_FLOW_ITEM_TYPE_VXLAN_GPE, - RTE_FLOW_ITEM_TYPE_MPLS), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_udp){ - .hdr = { - .src_port = -1, - .dst_port = -1, - }, - }, - .default_mask = &rte_flow_item_udp_mask, - .mask_sz = sizeof(struct rte_flow_item_udp), - .convert = mlx5_flow_create_udp, - .dst_sz = sizeof(struct ibv_flow_spec_tcp_udp), - }, - [RTE_FLOW_ITEM_TYPE_TCP] = { - .actions = valid_actions, - .mask = &(const struct rte_flow_item_tcp){ - .hdr = { - .src_port = -1, - .dst_port = -1, - }, - }, - .default_mask = &rte_flow_item_tcp_mask, - .mask_sz = sizeof(struct rte_flow_item_tcp), - .convert = mlx5_flow_create_tcp, - .dst_sz = sizeof(struct ibv_flow_spec_tcp_udp), - }, - [RTE_FLOW_ITEM_TYPE_GRE] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6, - RTE_FLOW_ITEM_TYPE_MPLS), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_gre){ - .protocol = -1, - }, - .default_mask = &rte_flow_item_gre_mask, - .mask_sz = sizeof(struct rte_flow_item_gre), - .convert = mlx5_flow_create_gre, -#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT - .dst_sz = sizeof(struct ibv_flow_spec_gre), -#else - .dst_sz = sizeof(struct ibv_flow_spec_tunnel), -#endif - }, - [RTE_FLOW_ITEM_TYPE_MPLS] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_mpls){ - .label_tc_s = "\xff\xff\xf0", - }, - .default_mask = &rte_flow_item_mpls_mask, - .mask_sz = sizeof(struct rte_flow_item_mpls), - .convert = mlx5_flow_create_mpls, -#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT - .dst_sz = sizeof(struct ibv_flow_spec_mpls), -#endif - }, - [RTE_FLOW_ITEM_TYPE_VXLAN] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_IPV4, /* For L3 VXLAN. */ - RTE_FLOW_ITEM_TYPE_IPV6), /* For L3 VXLAN. */ - .actions = valid_actions, - .mask = &(const struct rte_flow_item_vxlan){ - .vni = "\xff\xff\xff", - }, - .default_mask = &rte_flow_item_vxlan_mask, - .mask_sz = sizeof(struct rte_flow_item_vxlan), - .convert = mlx5_flow_create_vxlan, - .dst_sz = sizeof(struct ibv_flow_spec_tunnel), - }, - [RTE_FLOW_ITEM_TYPE_VXLAN_GPE] = { - .items = ITEMS(RTE_FLOW_ITEM_TYPE_ETH, - RTE_FLOW_ITEM_TYPE_IPV4, - RTE_FLOW_ITEM_TYPE_IPV6), - .actions = valid_actions, - .mask = &(const struct rte_flow_item_vxlan_gpe){ - .vni = "\xff\xff\xff", - }, - .default_mask = &rte_flow_item_vxlan_gpe_mask, - .mask_sz = sizeof(struct rte_flow_item_vxlan_gpe), - .convert = mlx5_flow_create_vxlan_gpe, - .dst_sz = sizeof(struct ibv_flow_spec_tunnel), - }, -}; - -/** Structure to pass to the conversion function. */ -struct mlx5_flow_parse { - uint32_t inner; /**< Verbs value, set once tunnel is encountered. */ - uint32_t create:1; - /**< Whether resources should remain after a validate. */ - uint32_t drop:1; /**< Target is a drop queue. */ - uint32_t mark:1; /**< Mark is present in the flow. */ - uint32_t count:1; /**< Count is present in the flow. */ - uint32_t mark_id; /**< Mark identifier. */ - struct rte_flow_action_rss rss_conf; /**< RSS configuration */ - uint16_t queues[RTE_MAX_QUEUES_PER_PORT]; /**< Queues indexes to use. */ - uint8_t rss_key[40]; /**< copy of the RSS key. */ - enum hash_rxq_type layer; /**< Last pattern layer detected. */ - enum hash_rxq_type out_layer; /**< Last outer pattern layer detected. */ - uint32_t tunnel; /**< Tunnel type of RTE_PTYPE_TUNNEL_XXX. */ - struct ibv_counter_set *cs; /**< Holds the counter set for the rule */ - struct { - struct ibv_flow_attr *ibv_attr; - /**< Pointer to Verbs attributes. */ - unsigned int offset; - /**< Current position or total size of the attribute. */ - uint64_t hash_fields; /**< Verbs hash fields. */ - } queue[RTE_DIM(hash_rxq_init)]; -}; - -static const struct rte_flow_ops mlx5_flow_ops = { - .validate = mlx5_flow_validate, - .create = mlx5_flow_create, - .destroy = mlx5_flow_destroy, - .flush = mlx5_flow_flush, -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT - .query = mlx5_flow_query, -#else - .query = NULL, -#endif - .isolate = mlx5_flow_isolate, -}; - -/* Convert FDIR request to Generic flow. */ -struct mlx5_fdir { - struct rte_flow_attr attr; - struct rte_flow_action actions[2]; - struct rte_flow_item items[4]; - struct rte_flow_item_eth l2; - struct rte_flow_item_eth l2_mask; - union { - struct rte_flow_item_ipv4 ipv4; - struct rte_flow_item_ipv6 ipv6; - } l3; - union { - struct rte_flow_item_ipv4 ipv4; - struct rte_flow_item_ipv6 ipv6; - } l3_mask; - union { - struct rte_flow_item_udp udp; - struct rte_flow_item_tcp tcp; - } l4; - union { - struct rte_flow_item_udp udp; - struct rte_flow_item_tcp tcp; - } l4_mask; - struct rte_flow_action_queue queue; -}; - -/* Verbs specification header. */ -struct ibv_spec_header { - enum ibv_flow_spec_type type; - uint16_t size; -}; - -/** - * Check item is fully supported by the NIC matching capability. - * - * @param item[in] - * Item specification. - * @param mask[in] - * Bit-masks covering supported fields to compare with spec, last and mask in - * \item. - * @param size - * Bit-Mask size in bytes. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_item_validate(const struct rte_flow_item *item, - const uint8_t *mask, unsigned int size) -{ - unsigned int i; - const uint8_t *spec = item->spec; - const uint8_t *last = item->last; - const uint8_t *m = item->mask ? item->mask : mask; - - if (!spec && (item->mask || last)) - goto error; - if (!spec) - return 0; - /* - * Single-pass check to make sure that: - * - item->mask is supported, no bits are set outside mask. - * - Both masked item->spec and item->last are equal (no range - * supported). - */ - for (i = 0; i < size; i++) { - if (!m[i]) - continue; - if ((m[i] | mask[i]) != mask[i]) - goto error; - if (last && ((spec[i] & m[i]) != (last[i] & m[i]))) - goto error; - } - return 0; -error: - rte_errno = ENOTSUP; - return -rte_errno; -} - -/** - * Extract attribute to the parser. - * - * @param[in] attr - * Flow rule attributes. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert_attributes(const struct rte_flow_attr *attr, - struct rte_flow_error *error) -{ - if (attr->group) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_GROUP, - NULL, - "groups are not supported"); - return -rte_errno; - } - if (attr->priority && attr->priority != MLX5_CTRL_FLOW_PRIORITY) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, - NULL, - "priorities are not supported"); - return -rte_errno; - } - if (attr->egress) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, - NULL, - "egress is not supported"); - return -rte_errno; - } - if (attr->transfer) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER, - NULL, - "transfer is not supported"); - return -rte_errno; - } - if (!attr->ingress) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_INGRESS, - NULL, - "only ingress is supported"); - return -rte_errno; - } - return 0; -} - -/** - * Extract actions request to the parser. - * - * @param dev - * Pointer to Ethernet device. - * @param[in] actions - * Associated actions (list terminated by the END action). - * @param[out] error - * Perform verbose error reporting if not NULL. - * @param[in, out] parser - * Internal parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert_actions(struct rte_eth_dev *dev, - const struct rte_flow_action actions[], - struct rte_flow_error *error, - struct mlx5_flow_parse *parser) -{ - enum { FATE = 1, MARK = 2, COUNT = 4, }; - uint32_t overlap = 0; - struct priv *priv = dev->data->dev_private; - - for (; actions->type != RTE_FLOW_ACTION_TYPE_END; ++actions) { - if (actions->type == RTE_FLOW_ACTION_TYPE_VOID) { - continue; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_DROP) { - if (overlap & FATE) - goto exit_action_overlap; - overlap |= FATE; - parser->drop = 1; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_QUEUE) { - const struct rte_flow_action_queue *queue = - (const struct rte_flow_action_queue *) - actions->conf; - - if (overlap & FATE) - goto exit_action_overlap; - overlap |= FATE; - if (!queue || (queue->index > (priv->rxqs_n - 1))) - goto exit_action_not_supported; - parser->queues[0] = queue->index; - parser->rss_conf = (struct rte_flow_action_rss){ - .queue_num = 1, - .queue = parser->queues, - }; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_RSS) { - const struct rte_flow_action_rss *rss = - (const struct rte_flow_action_rss *) - actions->conf; - const uint8_t *rss_key; - uint32_t rss_key_len; - uint16_t n; - - if (overlap & FATE) - goto exit_action_overlap; - overlap |= FATE; - if (rss->func && - rss->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "the only supported RSS hash" - " function is Toeplitz"); - return -rte_errno; - } -#ifndef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - if (parser->rss_conf.level > 1) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "a nonzero RSS encapsulation" - " level is not supported"); - return -rte_errno; - } -#endif - if (parser->rss_conf.level > 2) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "RSS encapsulation level" - " > 1 is not supported"); - return -rte_errno; - } - if (rss->types & MLX5_RSS_HF_MASK) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "unsupported RSS type" - " requested"); - return -rte_errno; - } - if (rss->key_len) { - rss_key_len = rss->key_len; - rss_key = rss->key; - } else { - rss_key_len = rss_hash_default_key_len; - rss_key = rss_hash_default_key; - } - if (rss_key_len != RTE_DIM(parser->rss_key)) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "RSS hash key must be" - " exactly 40 bytes long"); - return -rte_errno; - } - if (!rss->queue_num) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "no valid queues"); - return -rte_errno; - } - if (rss->queue_num > RTE_DIM(parser->queues)) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "too many queues for RSS" - " context"); - return -rte_errno; - } - for (n = 0; n < rss->queue_num; ++n) { - if (rss->queue[n] >= priv->rxqs_n) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "queue id > number of" - " queues"); - return -rte_errno; - } - } - parser->rss_conf = (struct rte_flow_action_rss){ - .func = RTE_ETH_HASH_FUNCTION_DEFAULT, - .level = rss->level ? rss->level : 1, - .types = rss->types, - .key_len = rss_key_len, - .queue_num = rss->queue_num, - .key = memcpy(parser->rss_key, rss_key, - sizeof(*rss_key) * rss_key_len), - .queue = memcpy(parser->queues, rss->queue, - sizeof(*rss->queue) * - rss->queue_num), - }; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_MARK) { - const struct rte_flow_action_mark *mark = - (const struct rte_flow_action_mark *) - actions->conf; - - if (overlap & MARK) - goto exit_action_overlap; - overlap |= MARK; - if (!mark) { - rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "mark must be defined"); - return -rte_errno; - } else if (mark->id >= MLX5_FLOW_MARK_MAX) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "mark must be between 0" - " and 16777199"); - return -rte_errno; - } - parser->mark = 1; - parser->mark_id = mark->id; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_FLAG) { - if (overlap & MARK) - goto exit_action_overlap; - overlap |= MARK; - parser->mark = 1; - } else if (actions->type == RTE_FLOW_ACTION_TYPE_COUNT && - priv->config.flow_counter_en) { - if (overlap & COUNT) - goto exit_action_overlap; - overlap |= COUNT; - parser->count = 1; - } else { - goto exit_action_not_supported; - } - } - /* When fate is unknown, drop traffic. */ - if (!(overlap & FATE)) - parser->drop = 1; - if (parser->drop && parser->mark) - parser->mark = 0; - if (!parser->rss_conf.queue_num && !parser->drop) { - rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, "no valid action"); - return -rte_errno; - } - return 0; -exit_action_not_supported: - rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, - actions, "action not supported"); - return -rte_errno; -exit_action_overlap: - rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, - actions, "overlapping actions are not supported"); - return -rte_errno; -} - -/** - * Validate items. - * - * @param[in] items - * Pattern specification (list terminated by the END pattern item). - * @param[out] error - * Perform verbose error reporting if not NULL. - * @param[in, out] parser - * Internal parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert_items_validate(struct rte_eth_dev *dev, - const struct rte_flow_item items[], - struct rte_flow_error *error, - struct mlx5_flow_parse *parser) -{ - struct priv *priv = dev->data->dev_private; - const struct mlx5_flow_items *cur_item = mlx5_flow_items; - unsigned int i; - unsigned int last_voids = 0; - int ret = 0; - - /* Initialise the offsets to start after verbs attribute. */ - for (i = 0; i != hash_rxq_init_n; ++i) - parser->queue[i].offset = sizeof(struct ibv_flow_attr); - for (; items->type != RTE_FLOW_ITEM_TYPE_END; ++items) { - const struct mlx5_flow_items *token = NULL; - unsigned int n; - - if (items->type == RTE_FLOW_ITEM_TYPE_VOID) { - last_voids++; - continue; - } - for (i = 0; - cur_item->items && - cur_item->items[i] != RTE_FLOW_ITEM_TYPE_END; - ++i) { - if (cur_item->items[i] == items->type) { - token = &mlx5_flow_items[items->type]; - break; - } - } - if (!token) { - ret = -ENOTSUP; - goto exit_item_not_supported; - } - cur_item = token; - ret = mlx5_flow_item_validate(items, - (const uint8_t *)cur_item->mask, - cur_item->mask_sz); - if (ret) - goto exit_item_not_supported; - if (IS_TUNNEL(items->type)) { - if (parser->tunnel && - !((items - last_voids - 1)->type == - RTE_FLOW_ITEM_TYPE_GRE && items->type == - RTE_FLOW_ITEM_TYPE_MPLS)) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - items, - "Cannot recognize multiple" - " tunnel encapsulations."); - return -rte_errno; - } - if (items->type == RTE_FLOW_ITEM_TYPE_MPLS && - !priv->config.mpls_en) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - items, - "MPLS not supported or" - " disabled in firmware" - " configuration."); - return -rte_errno; - } - if (!priv->config.tunnel_en && - parser->rss_conf.level > 1) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - items, - "RSS on tunnel is not supported"); - return -rte_errno; - } - parser->inner = IBV_FLOW_SPEC_INNER; - parser->tunnel = flow_ptype[items->type]; - } - if (parser->drop) { - parser->queue[HASH_RXQ_ETH].offset += cur_item->dst_sz; - } else { - for (n = 0; n != hash_rxq_init_n; ++n) - parser->queue[n].offset += cur_item->dst_sz; - } - last_voids = 0; - } - if (parser->drop) { - parser->queue[HASH_RXQ_ETH].offset += - sizeof(struct ibv_flow_spec_action_drop); - } - if (parser->mark) { - for (i = 0; i != hash_rxq_init_n; ++i) - parser->queue[i].offset += - sizeof(struct ibv_flow_spec_action_tag); - } - if (parser->count) { - unsigned int size = sizeof(struct ibv_flow_spec_counter_action); - - for (i = 0; i != hash_rxq_init_n; ++i) - parser->queue[i].offset += size; - } - return 0; -exit_item_not_supported: - return rte_flow_error_set(error, -ret, RTE_FLOW_ERROR_TYPE_ITEM, - items, "item not supported"); -} - -/** - * Allocate memory space to store verbs flow attributes. - * - * @param[in] size - * Amount of byte to allocate. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * A verbs flow attribute on success, NULL otherwise and rte_errno is set. - */ -static struct ibv_flow_attr * -mlx5_flow_convert_allocate(unsigned int size, struct rte_flow_error *error) -{ - struct ibv_flow_attr *ibv_attr; - - ibv_attr = rte_calloc(__func__, 1, size, 0); - if (!ibv_attr) { - rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot allocate verbs spec attributes"); - return NULL; - } - return ibv_attr; -} - -/** - * Make inner packet matching with an higher priority from the non Inner - * matching. - * - * @param dev - * Pointer to Ethernet device. - * @param[in, out] parser - * Internal parser structure. - * @param attr - * User flow attribute. - */ -static void -mlx5_flow_update_priority(struct rte_eth_dev *dev, - struct mlx5_flow_parse *parser, - const struct rte_flow_attr *attr) -{ - struct priv *priv = dev->data->dev_private; - unsigned int i; - uint16_t priority; - - /* 8 priorities >= 16 priorities - * Control flow: 4-7 8-15 - * User normal flow: 1-3 4-7 - * User tunnel flow: 0-2 0-3 - */ - priority = attr->priority * MLX5_VERBS_FLOW_PRIO_8; - if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_PRIO_8) - priority /= 2; - /* - * Lower non-tunnel flow Verbs priority 1 if only support 8 Verbs - * priorities, lower 4 otherwise. - */ - if (!parser->inner) { - if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_PRIO_8) - priority += 1; - else - priority += MLX5_VERBS_FLOW_PRIO_8 / 2; - } - if (parser->drop) { - parser->queue[HASH_RXQ_ETH].ibv_attr->priority = priority + - hash_rxq_init[HASH_RXQ_ETH].flow_priority; - return; - } - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - parser->queue[i].ibv_attr->priority = priority + - hash_rxq_init[i].flow_priority; - } -} - -/** - * Finalise verbs flow attributes. - * - * @param[in, out] parser - * Internal parser structure. - */ -static void -mlx5_flow_convert_finalise(struct mlx5_flow_parse *parser) -{ - unsigned int i; - uint32_t inner = parser->inner; - - /* Don't create extra flows for outer RSS. */ - if (parser->tunnel && parser->rss_conf.level < 2) - return; - /* - * Fill missing layers in verbs specifications, or compute the correct - * offset to allocate the memory space for the attributes and - * specifications. - */ - for (i = 0; i != hash_rxq_init_n - 1; ++i) { - union { - struct ibv_flow_spec_ipv4_ext ipv4; - struct ibv_flow_spec_ipv6 ipv6; - struct ibv_flow_spec_tcp_udp udp_tcp; - struct ibv_flow_spec_eth eth; - } specs; - void *dst; - uint16_t size; - - if (i == parser->layer) - continue; - if (parser->layer == HASH_RXQ_ETH || - parser->layer == HASH_RXQ_TUNNEL) { - if (hash_rxq_init[i].ip_version == MLX5_IPV4) { - size = sizeof(struct ibv_flow_spec_ipv4_ext); - specs.ipv4 = (struct ibv_flow_spec_ipv4_ext){ - .type = inner | IBV_FLOW_SPEC_IPV4_EXT, - .size = size, - }; - } else { - size = sizeof(struct ibv_flow_spec_ipv6); - specs.ipv6 = (struct ibv_flow_spec_ipv6){ - .type = inner | IBV_FLOW_SPEC_IPV6, - .size = size, - }; - } - if (parser->queue[i].ibv_attr) { - dst = (void *)((uintptr_t) - parser->queue[i].ibv_attr + - parser->queue[i].offset); - memcpy(dst, &specs, size); - ++parser->queue[i].ibv_attr->num_of_specs; - } - parser->queue[i].offset += size; - } - if ((i == HASH_RXQ_UDPV4) || (i == HASH_RXQ_TCPV4) || - (i == HASH_RXQ_UDPV6) || (i == HASH_RXQ_TCPV6)) { - size = sizeof(struct ibv_flow_spec_tcp_udp); - specs.udp_tcp = (struct ibv_flow_spec_tcp_udp) { - .type = inner | ((i == HASH_RXQ_UDPV4 || - i == HASH_RXQ_UDPV6) ? - IBV_FLOW_SPEC_UDP : - IBV_FLOW_SPEC_TCP), - .size = size, - }; - if (parser->queue[i].ibv_attr) { - dst = (void *)((uintptr_t) - parser->queue[i].ibv_attr + - parser->queue[i].offset); - memcpy(dst, &specs, size); - ++parser->queue[i].ibv_attr->num_of_specs; - } - parser->queue[i].offset += size; - } - } -} - -/** - * Update flows according to pattern and RSS hash fields. - * - * @param[in, out] parser - * Internal parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert_rss(struct mlx5_flow_parse *parser) -{ - unsigned int i; - enum hash_rxq_type start; - enum hash_rxq_type layer; - int outer = parser->tunnel && parser->rss_conf.level < 2; - uint64_t rss = parser->rss_conf.types; - - layer = outer ? parser->out_layer : parser->layer; - if (layer == HASH_RXQ_TUNNEL) - layer = HASH_RXQ_ETH; - if (outer) { - /* Only one hash type for outer RSS. */ - if (rss && layer == HASH_RXQ_ETH) { - start = HASH_RXQ_TCPV4; - } else if (rss && layer != HASH_RXQ_ETH && - !(rss & hash_rxq_init[layer].dpdk_rss_hf)) { - /* If RSS not match L4 pattern, try L3 RSS. */ - if (layer < HASH_RXQ_IPV4) - layer = HASH_RXQ_IPV4; - else if (layer > HASH_RXQ_IPV4 && layer < HASH_RXQ_IPV6) - layer = HASH_RXQ_IPV6; - start = layer; - } else { - start = layer; - } - /* Scan first valid hash type. */ - for (i = start; rss && i <= layer; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - if (hash_rxq_init[i].dpdk_rss_hf & rss) - break; - } - if (rss && i <= layer) - parser->queue[layer].hash_fields = - hash_rxq_init[i].hash_fields; - /* Trim unused hash types. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser->queue[i].ibv_attr && i != layer) { - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - } - } - } else { - /* Expand for inner or normal RSS. */ - if (rss && (layer == HASH_RXQ_ETH || layer == HASH_RXQ_IPV4)) - start = HASH_RXQ_TCPV4; - else if (rss && layer == HASH_RXQ_IPV6) - start = HASH_RXQ_TCPV6; - else - start = layer; - /* For L4 pattern, try L3 RSS if no L4 RSS. */ - /* Trim unused hash types. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - if (i < start || i > layer) { - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - continue; - } - if (!rss) - continue; - if (hash_rxq_init[i].dpdk_rss_hf & rss) { - parser->queue[i].hash_fields = - hash_rxq_init[i].hash_fields; - } else if (i != layer) { - /* Remove unused RSS expansion. */ - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - } else if (layer < HASH_RXQ_IPV4 && - (hash_rxq_init[HASH_RXQ_IPV4].dpdk_rss_hf & - rss)) { - /* Allow IPv4 RSS on L4 pattern. */ - parser->queue[i].hash_fields = - hash_rxq_init[HASH_RXQ_IPV4] - .hash_fields; - } else if (i > HASH_RXQ_IPV4 && i < HASH_RXQ_IPV6 && - (hash_rxq_init[HASH_RXQ_IPV6].dpdk_rss_hf & - rss)) { - /* Allow IPv4 RSS on L4 pattern. */ - parser->queue[i].hash_fields = - hash_rxq_init[HASH_RXQ_IPV6] - .hash_fields; - } - } - } - return 0; -} - -/** - * Validate and convert a flow supported by the NIC. - * - * @param dev - * Pointer to Ethernet device. - * @param[in] attr - * Flow rule attributes. - * @param[in] pattern - * Pattern specification (list terminated by the END pattern item). - * @param[in] actions - * Associated actions (list terminated by the END action). - * @param[out] error - * Perform verbose error reporting if not NULL. - * @param[in, out] parser - * Internal parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_convert(struct rte_eth_dev *dev, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], - struct rte_flow_error *error, - struct mlx5_flow_parse *parser) -{ - const struct mlx5_flow_items *cur_item = mlx5_flow_items; - unsigned int i; - int ret; - - /* First step. Validate the attributes, items and actions. */ - *parser = (struct mlx5_flow_parse){ - .create = parser->create, - .layer = HASH_RXQ_ETH, - .mark_id = MLX5_FLOW_MARK_DEFAULT, - }; - ret = mlx5_flow_convert_attributes(attr, error); - if (ret) - return ret; - ret = mlx5_flow_convert_actions(dev, actions, error, parser); - if (ret) - return ret; - ret = mlx5_flow_convert_items_validate(dev, items, error, parser); - if (ret) - return ret; - mlx5_flow_convert_finalise(parser); - /* - * Second step. - * Allocate the memory space to store verbs specifications. - */ - if (parser->drop) { - unsigned int offset = parser->queue[HASH_RXQ_ETH].offset; - - parser->queue[HASH_RXQ_ETH].ibv_attr = - mlx5_flow_convert_allocate(offset, error); - if (!parser->queue[HASH_RXQ_ETH].ibv_attr) - goto exit_enomem; - parser->queue[HASH_RXQ_ETH].offset = - sizeof(struct ibv_flow_attr); - } else { - for (i = 0; i != hash_rxq_init_n; ++i) { - unsigned int offset; - - offset = parser->queue[i].offset; - parser->queue[i].ibv_attr = - mlx5_flow_convert_allocate(offset, error); - if (!parser->queue[i].ibv_attr) - goto exit_enomem; - parser->queue[i].offset = sizeof(struct ibv_flow_attr); - } - } - /* Third step. Conversion parse, fill the specifications. */ - parser->inner = 0; - parser->tunnel = 0; - parser->layer = HASH_RXQ_ETH; - for (; items->type != RTE_FLOW_ITEM_TYPE_END; ++items) { - struct mlx5_flow_data data = { - .dev = dev, - .parser = parser, - .error = error, - }; - - if (items->type == RTE_FLOW_ITEM_TYPE_VOID) - continue; - cur_item = &mlx5_flow_items[items->type]; - ret = cur_item->convert(items, - (cur_item->default_mask ? - cur_item->default_mask : - cur_item->mask), - &data); - if (ret) - goto exit_free; - } - if (!parser->drop) { - /* RSS check, remove unused hash types. */ - ret = mlx5_flow_convert_rss(parser); - if (ret) - goto exit_free; - /* Complete missing specification. */ - mlx5_flow_convert_finalise(parser); - } - mlx5_flow_update_priority(dev, parser, attr); - if (parser->mark) - mlx5_flow_create_flag_mark(parser, parser->mark_id); - if (parser->count && parser->create) { - mlx5_flow_create_count(dev, parser); - if (!parser->cs) - goto exit_count_error; - } -exit_free: - /* Only verification is expected, all resources should be released. */ - if (!parser->create) { - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser->queue[i].ibv_attr) { - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - } - } - } - return ret; -exit_enomem: - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser->queue[i].ibv_attr) { - rte_free(parser->queue[i].ibv_attr); - parser->queue[i].ibv_attr = NULL; - } - } - rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, "cannot allocate verbs spec attributes"); - return -rte_errno; -exit_count_error: - rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, "cannot create counter"); - return -rte_errno; -} - -/** - * Copy the specification created into the flow. - * - * @param parser - * Internal parser structure. - * @param src - * Create specification. - * @param size - * Size in bytes of the specification to copy. - */ -static void -mlx5_flow_create_copy(struct mlx5_flow_parse *parser, void *src, - unsigned int size) -{ - unsigned int i; - void *dst; - - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - dst = (void *)((uintptr_t)parser->queue[i].ibv_attr + - parser->queue[i].offset); - memcpy(dst, src, size); - ++parser->queue[i].ibv_attr->num_of_specs; - parser->queue[i].offset += size; - } -} - -/** - * Convert Ethernet item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_eth(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_eth *spec = item->spec; - const struct rte_flow_item_eth *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - const unsigned int eth_size = sizeof(struct ibv_flow_spec_eth); - struct ibv_flow_spec_eth eth = { - .type = parser->inner | IBV_FLOW_SPEC_ETH, - .size = eth_size, - }; - - parser->layer = HASH_RXQ_ETH; - if (spec) { - unsigned int i; - - if (!mask) - mask = default_mask; - memcpy(ð.val.dst_mac, spec->dst.addr_bytes, ETHER_ADDR_LEN); - memcpy(ð.val.src_mac, spec->src.addr_bytes, ETHER_ADDR_LEN); - eth.val.ether_type = spec->type; - memcpy(ð.mask.dst_mac, mask->dst.addr_bytes, ETHER_ADDR_LEN); - memcpy(ð.mask.src_mac, mask->src.addr_bytes, ETHER_ADDR_LEN); - eth.mask.ether_type = mask->type; - /* Remove unwanted bits from values. */ - for (i = 0; i < ETHER_ADDR_LEN; ++i) { - eth.val.dst_mac[i] &= eth.mask.dst_mac[i]; - eth.val.src_mac[i] &= eth.mask.src_mac[i]; - } - eth.val.ether_type &= eth.mask.ether_type; - } - mlx5_flow_create_copy(parser, ð, eth_size); - return 0; -} - -/** - * Convert VLAN item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_vlan(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_vlan *spec = item->spec; - const struct rte_flow_item_vlan *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - struct ibv_flow_spec_eth *eth; - const unsigned int eth_size = sizeof(struct ibv_flow_spec_eth); - const char *msg = "VLAN cannot be empty"; - - if (spec) { - unsigned int i; - if (!mask) - mask = default_mask; - - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - - eth = (void *)((uintptr_t)parser->queue[i].ibv_attr + - parser->queue[i].offset - eth_size); - eth->val.vlan_tag = spec->tci; - eth->mask.vlan_tag = mask->tci; - eth->val.vlan_tag &= eth->mask.vlan_tag; - /* - * From verbs perspective an empty VLAN is equivalent - * to a packet without VLAN layer. - */ - if (!eth->mask.vlan_tag) - goto error; - /* Outer TPID cannot be matched. */ - if (eth->mask.ether_type) { - msg = "VLAN TPID matching is not supported"; - goto error; - } - eth->val.ether_type = spec->inner_type; - eth->mask.ether_type = mask->inner_type; - eth->val.ether_type &= eth->mask.ether_type; - } - return 0; - } -error: - return rte_flow_error_set(data->error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, - item, msg); -} - -/** - * Convert IPv4 item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_ipv4(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - struct priv *priv = data->dev->data->dev_private; - const struct rte_flow_item_ipv4 *spec = item->spec; - const struct rte_flow_item_ipv4 *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int ipv4_size = sizeof(struct ibv_flow_spec_ipv4_ext); - struct ibv_flow_spec_ipv4_ext ipv4 = { - .type = parser->inner | IBV_FLOW_SPEC_IPV4_EXT, - .size = ipv4_size, - }; - - if (parser->layer == HASH_RXQ_TUNNEL && - parser->tunnel == ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN)] && - !priv->config.l3_vxlan_en) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 VXLAN not enabled by device" - " parameter and/or not configured" - " in firmware"); - parser->layer = HASH_RXQ_IPV4; - if (spec) { - if (!mask) - mask = default_mask; - ipv4.val = (struct ibv_flow_ipv4_ext_filter){ - .src_ip = spec->hdr.src_addr, - .dst_ip = spec->hdr.dst_addr, - .proto = spec->hdr.next_proto_id, - .tos = spec->hdr.type_of_service, - }; - ipv4.mask = (struct ibv_flow_ipv4_ext_filter){ - .src_ip = mask->hdr.src_addr, - .dst_ip = mask->hdr.dst_addr, - .proto = mask->hdr.next_proto_id, - .tos = mask->hdr.type_of_service, - }; - /* Remove unwanted bits from values. */ - ipv4.val.src_ip &= ipv4.mask.src_ip; - ipv4.val.dst_ip &= ipv4.mask.dst_ip; - ipv4.val.proto &= ipv4.mask.proto; - ipv4.val.tos &= ipv4.mask.tos; - } - mlx5_flow_create_copy(parser, &ipv4, ipv4_size); - return 0; -} - -/** - * Convert IPv6 item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_ipv6(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - struct priv *priv = data->dev->data->dev_private; - const struct rte_flow_item_ipv6 *spec = item->spec; - const struct rte_flow_item_ipv6 *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int ipv6_size = sizeof(struct ibv_flow_spec_ipv6); - struct ibv_flow_spec_ipv6 ipv6 = { - .type = parser->inner | IBV_FLOW_SPEC_IPV6, - .size = ipv6_size, - }; - - if (parser->layer == HASH_RXQ_TUNNEL && - parser->tunnel == ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN)] && - !priv->config.l3_vxlan_en) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 VXLAN not enabled by device" - " parameter and/or not configured" - " in firmware"); - parser->layer = HASH_RXQ_IPV6; - if (spec) { - unsigned int i; - uint32_t vtc_flow_val; - uint32_t vtc_flow_mask; - - if (!mask) - mask = default_mask; - memcpy(&ipv6.val.src_ip, spec->hdr.src_addr, - RTE_DIM(ipv6.val.src_ip)); - memcpy(&ipv6.val.dst_ip, spec->hdr.dst_addr, - RTE_DIM(ipv6.val.dst_ip)); - memcpy(&ipv6.mask.src_ip, mask->hdr.src_addr, - RTE_DIM(ipv6.mask.src_ip)); - memcpy(&ipv6.mask.dst_ip, mask->hdr.dst_addr, - RTE_DIM(ipv6.mask.dst_ip)); - vtc_flow_val = rte_be_to_cpu_32(spec->hdr.vtc_flow); - vtc_flow_mask = rte_be_to_cpu_32(mask->hdr.vtc_flow); - ipv6.val.flow_label = - rte_cpu_to_be_32((vtc_flow_val & IPV6_HDR_FL_MASK) >> - IPV6_HDR_FL_SHIFT); - ipv6.val.traffic_class = (vtc_flow_val & IPV6_HDR_TC_MASK) >> - IPV6_HDR_TC_SHIFT; - ipv6.val.next_hdr = spec->hdr.proto; - ipv6.val.hop_limit = spec->hdr.hop_limits; - ipv6.mask.flow_label = - rte_cpu_to_be_32((vtc_flow_mask & IPV6_HDR_FL_MASK) >> - IPV6_HDR_FL_SHIFT); - ipv6.mask.traffic_class = (vtc_flow_mask & IPV6_HDR_TC_MASK) >> - IPV6_HDR_TC_SHIFT; - ipv6.mask.next_hdr = mask->hdr.proto; - ipv6.mask.hop_limit = mask->hdr.hop_limits; - /* Remove unwanted bits from values. */ - for (i = 0; i < RTE_DIM(ipv6.val.src_ip); ++i) { - ipv6.val.src_ip[i] &= ipv6.mask.src_ip[i]; - ipv6.val.dst_ip[i] &= ipv6.mask.dst_ip[i]; - } - ipv6.val.flow_label &= ipv6.mask.flow_label; - ipv6.val.traffic_class &= ipv6.mask.traffic_class; - ipv6.val.next_hdr &= ipv6.mask.next_hdr; - ipv6.val.hop_limit &= ipv6.mask.hop_limit; - } - mlx5_flow_create_copy(parser, &ipv6, ipv6_size); - return 0; -} - -/** - * Convert UDP item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_udp(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_udp *spec = item->spec; - const struct rte_flow_item_udp *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int udp_size = sizeof(struct ibv_flow_spec_tcp_udp); - struct ibv_flow_spec_tcp_udp udp = { - .type = parser->inner | IBV_FLOW_SPEC_UDP, - .size = udp_size, - }; - - if (parser->layer == HASH_RXQ_IPV4) - parser->layer = HASH_RXQ_UDPV4; - else - parser->layer = HASH_RXQ_UDPV6; - if (spec) { - if (!mask) - mask = default_mask; - udp.val.dst_port = spec->hdr.dst_port; - udp.val.src_port = spec->hdr.src_port; - udp.mask.dst_port = mask->hdr.dst_port; - udp.mask.src_port = mask->hdr.src_port; - /* Remove unwanted bits from values. */ - udp.val.src_port &= udp.mask.src_port; - udp.val.dst_port &= udp.mask.dst_port; - } - mlx5_flow_create_copy(parser, &udp, udp_size); - return 0; -} - -/** - * Convert TCP item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_tcp(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_tcp *spec = item->spec; - const struct rte_flow_item_tcp *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int tcp_size = sizeof(struct ibv_flow_spec_tcp_udp); - struct ibv_flow_spec_tcp_udp tcp = { - .type = parser->inner | IBV_FLOW_SPEC_TCP, - .size = tcp_size, - }; - - if (parser->layer == HASH_RXQ_IPV4) - parser->layer = HASH_RXQ_TCPV4; - else - parser->layer = HASH_RXQ_TCPV6; - if (spec) { - if (!mask) - mask = default_mask; - tcp.val.dst_port = spec->hdr.dst_port; - tcp.val.src_port = spec->hdr.src_port; - tcp.mask.dst_port = mask->hdr.dst_port; - tcp.mask.src_port = mask->hdr.src_port; - /* Remove unwanted bits from values. */ - tcp.val.src_port &= tcp.mask.src_port; - tcp.val.dst_port &= tcp.mask.dst_port; - } - mlx5_flow_create_copy(parser, &tcp, tcp_size); - return 0; -} - -/** - * Convert VXLAN item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_vxlan(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - const struct rte_flow_item_vxlan *spec = item->spec; - const struct rte_flow_item_vxlan *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int size = sizeof(struct ibv_flow_spec_tunnel); - struct ibv_flow_spec_tunnel vxlan = { - .type = parser->inner | IBV_FLOW_SPEC_VXLAN_TUNNEL, - .size = size, - }; - union vni { - uint32_t vlan_id; - uint8_t vni[4]; - } id; - - id.vni[0] = 0; - parser->inner = IBV_FLOW_SPEC_INNER; - parser->tunnel = ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN)]; - parser->out_layer = parser->layer; - parser->layer = HASH_RXQ_TUNNEL; - if (spec) { - if (!mask) - mask = default_mask; - memcpy(&id.vni[1], spec->vni, 3); - vxlan.val.tunnel_id = id.vlan_id; - memcpy(&id.vni[1], mask->vni, 3); - vxlan.mask.tunnel_id = id.vlan_id; - /* Remove unwanted bits from values. */ - vxlan.val.tunnel_id &= vxlan.mask.tunnel_id; - } - /* - * Tunnel id 0 is equivalent as not adding a VXLAN layer, if only this - * layer is defined in the Verbs specification it is interpreted as - * wildcard and all packets will match this rule, if it follows a full - * stack layer (ex: eth / ipv4 / udp), all packets matching the layers - * before will also match this rule. - * To avoid such situation, VNI 0 is currently refused. - */ - /* Only allow tunnel w/o tunnel id pattern after proper outer spec. */ - if (parser->out_layer == HASH_RXQ_ETH && !vxlan.val.tunnel_id) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "VxLAN vni cannot be 0"); - mlx5_flow_create_copy(parser, &vxlan, size); - return 0; -} - -/** - * Convert VXLAN-GPE item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_vxlan_gpe(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - struct priv *priv = data->dev->data->dev_private; - const struct rte_flow_item_vxlan_gpe *spec = item->spec; - const struct rte_flow_item_vxlan_gpe *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int size = sizeof(struct ibv_flow_spec_tunnel); - struct ibv_flow_spec_tunnel vxlan = { - .type = parser->inner | IBV_FLOW_SPEC_VXLAN_TUNNEL, - .size = size, - }; - union vni { - uint32_t vlan_id; - uint8_t vni[4]; - } id; - - if (!priv->config.l3_vxlan_en) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 VXLAN not enabled by device" - " parameter and/or not configured" - " in firmware"); - id.vni[0] = 0; - parser->inner = IBV_FLOW_SPEC_INNER; - parser->tunnel = ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_VXLAN_GPE)]; - parser->out_layer = parser->layer; - parser->layer = HASH_RXQ_TUNNEL; - if (spec) { - if (!mask) - mask = default_mask; - memcpy(&id.vni[1], spec->vni, 3); - vxlan.val.tunnel_id = id.vlan_id; - memcpy(&id.vni[1], mask->vni, 3); - vxlan.mask.tunnel_id = id.vlan_id; - if (spec->protocol) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "VxLAN-GPE protocol not" - " supported"); - /* Remove unwanted bits from values. */ - vxlan.val.tunnel_id &= vxlan.mask.tunnel_id; - } - /* - * Tunnel id 0 is equivalent as not adding a VXLAN layer, if only this - * layer is defined in the Verbs specification it is interpreted as - * wildcard and all packets will match this rule, if it follows a full - * stack layer (ex: eth / ipv4 / udp), all packets matching the layers - * before will also match this rule. - * To avoid such situation, VNI 0 is currently refused. - */ - /* Only allow tunnel w/o tunnel id pattern after proper outer spec. */ - if (parser->out_layer == HASH_RXQ_ETH && !vxlan.val.tunnel_id) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "VxLAN-GPE vni cannot be 0"); - mlx5_flow_create_copy(parser, &vxlan, size); - return 0; -} - -/** - * Convert GRE item to Verbs specification. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_gre(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ - struct mlx5_flow_parse *parser = data->parser; -#ifndef HAVE_IBV_DEVICE_MPLS_SUPPORT - (void)default_mask; - unsigned int size = sizeof(struct ibv_flow_spec_tunnel); - struct ibv_flow_spec_tunnel tunnel = { - .type = parser->inner | IBV_FLOW_SPEC_VXLAN_TUNNEL, - .size = size, - }; -#else - const struct rte_flow_item_gre *spec = item->spec; - const struct rte_flow_item_gre *mask = item->mask; - unsigned int size = sizeof(struct ibv_flow_spec_gre); - struct ibv_flow_spec_gre tunnel = { - .type = parser->inner | IBV_FLOW_SPEC_GRE, - .size = size, - }; -#endif - struct ibv_flow_spec_ipv4_ext *ipv4; - struct ibv_flow_spec_ipv6 *ipv6; - unsigned int i; - - parser->inner = IBV_FLOW_SPEC_INNER; - parser->tunnel = ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_GRE)]; - parser->out_layer = parser->layer; - parser->layer = HASH_RXQ_TUNNEL; -#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT - if (spec) { - if (!mask) - mask = default_mask; - tunnel.val.c_ks_res0_ver = spec->c_rsvd0_ver; - tunnel.val.protocol = spec->protocol; - tunnel.mask.c_ks_res0_ver = mask->c_rsvd0_ver; - tunnel.mask.protocol = mask->protocol; - /* Remove unwanted bits from values. */ - tunnel.val.c_ks_res0_ver &= tunnel.mask.c_ks_res0_ver; - tunnel.val.protocol &= tunnel.mask.protocol; - tunnel.val.key &= tunnel.mask.key; - } -#endif - /* Update encapsulation IP layer protocol. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - if (parser->out_layer == HASH_RXQ_IPV4) { - ipv4 = (void *)((uintptr_t)parser->queue[i].ibv_attr + - parser->queue[i].offset - - sizeof(struct ibv_flow_spec_ipv4_ext)); - if (ipv4->mask.proto && ipv4->val.proto != MLX5_GRE) - break; - ipv4->val.proto = MLX5_GRE; - ipv4->mask.proto = 0xff; - } else if (parser->out_layer == HASH_RXQ_IPV6) { - ipv6 = (void *)((uintptr_t)parser->queue[i].ibv_attr + - parser->queue[i].offset - - sizeof(struct ibv_flow_spec_ipv6)); - if (ipv6->mask.next_hdr && - ipv6->val.next_hdr != MLX5_GRE) - break; - ipv6->val.next_hdr = MLX5_GRE; - ipv6->mask.next_hdr = 0xff; - } - } - if (i != hash_rxq_init_n) - return rte_flow_error_set(data->error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "IP protocol of GRE must be 47"); - mlx5_flow_create_copy(parser, &tunnel, size); - return 0; -} - -/** - * Convert MPLS item to Verbs specification. - * MPLS tunnel types currently supported are MPLS-in-GRE and MPLS-in-UDP. - * - * @param item[in] - * Item specification. - * @param default_mask[in] - * Default bit-masks to use when item->mask is not provided. - * @param data[in, out] - * User structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_mpls(const struct rte_flow_item *item, - const void *default_mask, - struct mlx5_flow_data *data) -{ -#ifndef HAVE_IBV_DEVICE_MPLS_SUPPORT - (void)default_mask; - return rte_flow_error_set(data->error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "MPLS is not supported by driver"); -#else - const struct rte_flow_item_mpls *spec = item->spec; - const struct rte_flow_item_mpls *mask = item->mask; - struct mlx5_flow_parse *parser = data->parser; - unsigned int size = sizeof(struct ibv_flow_spec_mpls); - struct ibv_flow_spec_mpls mpls = { - .type = IBV_FLOW_SPEC_MPLS, - .size = size, - }; - - parser->inner = IBV_FLOW_SPEC_INNER; - if (parser->layer == HASH_RXQ_UDPV4 || - parser->layer == HASH_RXQ_UDPV6) { - parser->tunnel = - ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_MPLS_IN_UDP)]; - parser->out_layer = parser->layer; - } else { - parser->tunnel = - ptype_ext[PTYPE_IDX(RTE_PTYPE_TUNNEL_MPLS_IN_GRE)]; - /* parser->out_layer stays as in GRE out_layer. */ - } - parser->layer = HASH_RXQ_TUNNEL; - if (spec) { - if (!mask) - mask = default_mask; - /* - * The verbs label field includes the entire MPLS header: - * bits 0:19 - label value field. - * bits 20:22 - traffic class field. - * bits 23 - bottom of stack bit. - * bits 24:31 - ttl field. - */ - mpls.val.label = *(const uint32_t *)spec; - mpls.mask.label = *(const uint32_t *)mask; - /* Remove unwanted bits from values. */ - mpls.val.label &= mpls.mask.label; - } - mlx5_flow_create_copy(parser, &mpls, size); - return 0; -#endif -} - -/** - * Convert mark/flag action to Verbs specification. - * - * @param parser - * Internal parser structure. - * @param mark_id - * Mark identifier. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_flag_mark(struct mlx5_flow_parse *parser, uint32_t mark_id) -{ - unsigned int size = sizeof(struct ibv_flow_spec_action_tag); - struct ibv_flow_spec_action_tag tag = { - .type = IBV_FLOW_SPEC_ACTION_TAG, - .size = size, - .tag_id = mlx5_flow_mark_set(mark_id), - }; - - assert(parser->mark); - mlx5_flow_create_copy(parser, &tag, size); - return 0; -} - -/** - * Convert count action to Verbs specification. - * - * @param dev - * Pointer to Ethernet device. - * @param parser - * Pointer to MLX5 flow parser structure. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_count(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flow_parse *parser __rte_unused) -{ -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT - struct priv *priv = dev->data->dev_private; - unsigned int size = sizeof(struct ibv_flow_spec_counter_action); - struct ibv_counter_set_init_attr init_attr = {0}; - struct ibv_flow_spec_counter_action counter = { - .type = IBV_FLOW_SPEC_ACTION_COUNT, - .size = size, - .counter_set_handle = 0, - }; - - init_attr.counter_set_id = 0; - parser->cs = mlx5_glue->create_counter_set(priv->ctx, &init_attr); - if (!parser->cs) { - rte_errno = EINVAL; - return -rte_errno; - } - counter.counter_set_handle = parser->cs->handle; - mlx5_flow_create_copy(parser, &counter, size); -#endif - return 0; -} - -/** - * Complete flow rule creation with a drop queue. - * - * @param dev - * Pointer to Ethernet device. - * @param parser - * Internal parser structure. - * @param flow - * Pointer to the rte_flow. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_action_queue_drop(struct rte_eth_dev *dev, - struct mlx5_flow_parse *parser, - struct rte_flow *flow, - struct rte_flow_error *error) -{ - struct priv *priv = dev->data->dev_private; - struct ibv_flow_spec_action_drop *drop; - unsigned int size = sizeof(struct ibv_flow_spec_action_drop); - - assert(priv->pd); - assert(priv->ctx); - flow->drop = 1; - drop = (void *)((uintptr_t)parser->queue[HASH_RXQ_ETH].ibv_attr + - parser->queue[HASH_RXQ_ETH].offset); - *drop = (struct ibv_flow_spec_action_drop){ - .type = IBV_FLOW_SPEC_ACTION_DROP, - .size = size, - }; - ++parser->queue[HASH_RXQ_ETH].ibv_attr->num_of_specs; - parser->queue[HASH_RXQ_ETH].offset += size; - flow->frxq[HASH_RXQ_ETH].ibv_attr = - parser->queue[HASH_RXQ_ETH].ibv_attr; - if (parser->count) - flow->cs = parser->cs; - if (!dev->data->dev_started) - return 0; - parser->queue[HASH_RXQ_ETH].ibv_attr = NULL; - flow->frxq[HASH_RXQ_ETH].ibv_flow = - mlx5_glue->create_flow(priv->flow_drop_queue->qp, - flow->frxq[HASH_RXQ_ETH].ibv_attr); - if (!flow->frxq[HASH_RXQ_ETH].ibv_flow) { - rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, "flow rule creation failure"); - goto error; - } - return 0; -error: - assert(flow); - if (flow->frxq[HASH_RXQ_ETH].ibv_flow) { - claim_zero(mlx5_glue->destroy_flow - (flow->frxq[HASH_RXQ_ETH].ibv_flow)); - flow->frxq[HASH_RXQ_ETH].ibv_flow = NULL; - } - if (flow->frxq[HASH_RXQ_ETH].ibv_attr) { - rte_free(flow->frxq[HASH_RXQ_ETH].ibv_attr); - flow->frxq[HASH_RXQ_ETH].ibv_attr = NULL; - } - if (flow->cs) { - claim_zero(mlx5_glue->destroy_counter_set(flow->cs)); - flow->cs = NULL; - parser->cs = NULL; - } - return -rte_errno; -} - -/** - * Create hash Rx queues when RSS is enabled. - * - * @param dev - * Pointer to Ethernet device. - * @param parser - * Internal parser structure. - * @param flow - * Pointer to the rte_flow. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_action_queue_rss(struct rte_eth_dev *dev, - struct mlx5_flow_parse *parser, - struct rte_flow *flow, - struct rte_flow_error *error) -{ - unsigned int i; - - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!parser->queue[i].ibv_attr) - continue; - flow->frxq[i].ibv_attr = parser->queue[i].ibv_attr; - parser->queue[i].ibv_attr = NULL; - flow->frxq[i].hash_fields = parser->queue[i].hash_fields; - if (!dev->data->dev_started) - continue; - flow->frxq[i].hrxq = - mlx5_hrxq_get(dev, - parser->rss_conf.key, - parser->rss_conf.key_len, - flow->frxq[i].hash_fields, - parser->rss_conf.queue, - parser->rss_conf.queue_num, - parser->tunnel, - parser->rss_conf.level); - if (flow->frxq[i].hrxq) - continue; - flow->frxq[i].hrxq = - mlx5_hrxq_new(dev, - parser->rss_conf.key, - parser->rss_conf.key_len, - flow->frxq[i].hash_fields, - parser->rss_conf.queue, - parser->rss_conf.queue_num, - parser->tunnel, - parser->rss_conf.level); - if (!flow->frxq[i].hrxq) { - return rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, - "cannot create hash rxq"); - } - } - return 0; -} - -/** - * RXQ update after flow rule creation. - * - * @param dev - * Pointer to Ethernet device. - * @param flow - * Pointer to the flow rule. - */ -static void -mlx5_flow_create_update_rxqs(struct rte_eth_dev *dev, struct rte_flow *flow) -{ - struct priv *priv = dev->data->dev_private; - unsigned int i; - unsigned int j; - - if (!dev->data->dev_started) - return; - for (i = 0; i != flow->rss_conf.queue_num; ++i) { - struct mlx5_rxq_data *rxq_data = (*priv->rxqs) - [(*flow->queues)[i]]; - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); - uint8_t tunnel = PTYPE_IDX(flow->tunnel); - - rxq_data->mark |= flow->mark; - if (!tunnel) - continue; - rxq_ctrl->tunnel_types[tunnel] += 1; - /* Clear tunnel type if more than one tunnel types set. */ - for (j = 0; j != RTE_DIM(rxq_ctrl->tunnel_types); ++j) { - if (j == tunnel) - continue; - if (rxq_ctrl->tunnel_types[j] > 0) { - rxq_data->tunnel = 0; - break; - } - } - if (j == RTE_DIM(rxq_ctrl->tunnel_types)) - rxq_data->tunnel = flow->tunnel; - } -} - -/** - * Dump flow hash RX queue detail. - * - * @param dev - * Pointer to Ethernet device. - * @param flow - * Pointer to the rte_flow. - * @param hrxq_idx - * Hash RX queue index. - */ -static void -mlx5_flow_dump(struct rte_eth_dev *dev __rte_unused, - struct rte_flow *flow __rte_unused, - unsigned int hrxq_idx __rte_unused) -{ -#ifndef NDEBUG - uintptr_t spec_ptr; - uint16_t j; - char buf[256]; - uint8_t off; - uint64_t extra_hash_fields = 0; +/* Dev ops structure defined in mlx5.c */ +extern const struct eth_dev_ops mlx5_dev_ops; +extern const struct eth_dev_ops mlx5_dev_ops_isolate; -#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - if (flow->tunnel && flow->rss_conf.level > 1) - extra_hash_fields = (uint32_t)IBV_RX_HASH_INNER; -#endif - spec_ptr = (uintptr_t)(flow->frxq[hrxq_idx].ibv_attr + 1); - for (j = 0, off = 0; j < flow->frxq[hrxq_idx].ibv_attr->num_of_specs; - j++) { - struct ibv_flow_spec *spec = (void *)spec_ptr; - off += sprintf(buf + off, " %x(%hu)", spec->hdr.type, - spec->hdr.size); - spec_ptr += spec->hdr.size; - } - DRV_LOG(DEBUG, - "port %u Verbs flow %p type %u: hrxq:%p qp:%p ind:%p," - " hash:%" PRIx64 "/%u specs:%hhu(%hu), priority:%hu, type:%d," - " flags:%x, comp_mask:%x specs:%s", - dev->data->port_id, (void *)flow, hrxq_idx, - (void *)flow->frxq[hrxq_idx].hrxq, - (void *)flow->frxq[hrxq_idx].hrxq->qp, - (void *)flow->frxq[hrxq_idx].hrxq->ind_table, - (flow->frxq[hrxq_idx].hash_fields | extra_hash_fields), - flow->rss_conf.queue_num, - flow->frxq[hrxq_idx].ibv_attr->num_of_specs, - flow->frxq[hrxq_idx].ibv_attr->size, - flow->frxq[hrxq_idx].ibv_attr->priority, - flow->frxq[hrxq_idx].ibv_attr->type, - flow->frxq[hrxq_idx].ibv_attr->flags, - flow->frxq[hrxq_idx].ibv_attr->comp_mask, - buf); -#endif -} +struct rte_flow { + TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ +}; -/** - * Complete flow rule creation. - * - * @param dev - * Pointer to Ethernet device. - * @param parser - * Internal parser structure. - * @param flow - * Pointer to the rte_flow. - * @param[out] error - * Perform verbose error reporting if not NULL. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_create_action_queue(struct rte_eth_dev *dev, - struct mlx5_flow_parse *parser, - struct rte_flow *flow, - struct rte_flow_error *error) -{ - struct priv *priv __rte_unused = dev->data->dev_private; - int ret; - unsigned int i; - unsigned int flows_n = 0; +static const struct rte_flow_ops mlx5_flow_ops = { + .isolate = mlx5_flow_isolate, +}; - assert(priv->pd); - assert(priv->ctx); - assert(!parser->drop); - ret = mlx5_flow_create_action_queue_rss(dev, parser, flow, error); - if (ret) - goto error; - if (parser->count) - flow->cs = parser->cs; - if (!dev->data->dev_started) - return 0; - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!flow->frxq[i].hrxq) - continue; - flow->frxq[i].ibv_flow = - mlx5_glue->create_flow(flow->frxq[i].hrxq->qp, - flow->frxq[i].ibv_attr); - mlx5_flow_dump(dev, flow, i); - if (!flow->frxq[i].ibv_flow) { - rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, "flow rule creation failure"); - goto error; - } - ++flows_n; - } - if (!flows_n) { - rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, "internal error in flow creation"); - goto error; - } - mlx5_flow_create_update_rxqs(dev, flow); - return 0; -error: - ret = rte_errno; /* Save rte_errno before cleanup. */ - assert(flow); - for (i = 0; i != hash_rxq_init_n; ++i) { - if (flow->frxq[i].ibv_flow) { - struct ibv_flow *ibv_flow = flow->frxq[i].ibv_flow; +/* Convert FDIR request to Generic flow. */ +struct mlx5_fdir { + struct rte_flow_attr attr; + struct rte_flow_action actions[2]; + struct rte_flow_item items[4]; + struct rte_flow_item_eth l2; + struct rte_flow_item_eth l2_mask; + union { + struct rte_flow_item_ipv4 ipv4; + struct rte_flow_item_ipv6 ipv6; + } l3; + union { + struct rte_flow_item_ipv4 ipv4; + struct rte_flow_item_ipv6 ipv6; + } l3_mask; + union { + struct rte_flow_item_udp udp; + struct rte_flow_item_tcp tcp; + } l4; + union { + struct rte_flow_item_udp udp; + struct rte_flow_item_tcp tcp; + } l4_mask; + struct rte_flow_action_queue queue; +}; - claim_zero(mlx5_glue->destroy_flow(ibv_flow)); - } - if (flow->frxq[i].hrxq) - mlx5_hrxq_release(dev, flow->frxq[i].hrxq); - if (flow->frxq[i].ibv_attr) - rte_free(flow->frxq[i].ibv_attr); - } - if (flow->cs) { - claim_zero(mlx5_glue->destroy_counter_set(flow->cs)); - flow->cs = NULL; - parser->cs = NULL; - } - rte_errno = ret; /* Restore rte_errno. */ - return -rte_errno; -} +/* Verbs specification header. */ +struct ibv_spec_header { + enum ibv_flow_spec_type type; + uint16_t size; +}; /** * Convert a flow. @@ -2452,69 +95,17 @@ mlx5_flow_create_action_queue(struct rte_eth_dev *dev, * A flow on success, NULL otherwise and rte_errno is set. */ static struct rte_flow * -mlx5_flow_list_create(struct rte_eth_dev *dev, - struct mlx5_flows *list, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], +mlx5_flow_list_create(struct rte_eth_dev *dev __rte_unused, + struct mlx5_flows *list __rte_unused, + const struct rte_flow_attr *attr __rte_unused, + const struct rte_flow_item items[] __rte_unused, + const struct rte_flow_action actions[] __rte_unused, struct rte_flow_error *error) { - struct mlx5_flow_parse parser = { .create = 1, }; - struct rte_flow *flow = NULL; - unsigned int i; - int ret; - - ret = mlx5_flow_convert(dev, attr, items, actions, error, &parser); - if (ret) - goto exit; - flow = rte_calloc(__func__, 1, - sizeof(*flow) + - parser.rss_conf.queue_num * sizeof(uint16_t), - 0); - if (!flow) { - rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot allocate flow memory"); - return NULL; - } - /* Copy configuration. */ - flow->queues = (uint16_t (*)[])(flow + 1); - flow->tunnel = parser.tunnel; - flow->rss_conf = (struct rte_flow_action_rss){ - .func = RTE_ETH_HASH_FUNCTION_DEFAULT, - .level = parser.rss_conf.level, - .types = parser.rss_conf.types, - .key_len = parser.rss_conf.key_len, - .queue_num = parser.rss_conf.queue_num, - .key = memcpy(flow->rss_key, parser.rss_conf.key, - sizeof(*parser.rss_conf.key) * - parser.rss_conf.key_len), - .queue = memcpy(flow->queues, parser.rss_conf.queue, - sizeof(*parser.rss_conf.queue) * - parser.rss_conf.queue_num), - }; - flow->mark = parser.mark; - /* finalise the flow. */ - if (parser.drop) - ret = mlx5_flow_create_action_queue_drop(dev, &parser, flow, - error); - else - ret = mlx5_flow_create_action_queue(dev, &parser, flow, error); - if (ret) - goto exit; - TAILQ_INSERT_TAIL(list, flow, next); - DRV_LOG(DEBUG, "port %u flow created %p", dev->data->port_id, - (void *)flow); - return flow; -exit: - DRV_LOG(ERR, "port %u flow creation error: %s", dev->data->port_id, - error->message); - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser.queue[i].ibv_attr) - rte_free(parser.queue[i].ibv_attr); - } - rte_free(flow); + rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "action not supported"); return NULL; } @@ -2525,15 +116,16 @@ mlx5_flow_list_create(struct rte_eth_dev *dev, * @see rte_flow_ops */ int -mlx5_flow_validate(struct rte_eth_dev *dev, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], +mlx5_flow_validate(struct rte_eth_dev *dev __rte_unused, + const struct rte_flow_attr *attr __rte_unused, + const struct rte_flow_item items[] __rte_unused, + const struct rte_flow_action actions[] __rte_unused, struct rte_flow_error *error) { - struct mlx5_flow_parse parser = { .create = 0, }; - - return mlx5_flow_convert(dev, attr, items, actions, error, &parser); + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "action not supported"); } /** @@ -2543,16 +135,17 @@ mlx5_flow_validate(struct rte_eth_dev *dev, * @see rte_flow_ops */ struct rte_flow * -mlx5_flow_create(struct rte_eth_dev *dev, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], +mlx5_flow_create(struct rte_eth_dev *dev __rte_unused, + const struct rte_flow_attr *attr __rte_unused, + const struct rte_flow_item items[] __rte_unused, + const struct rte_flow_action actions[] __rte_unused, struct rte_flow_error *error) { - struct priv *priv = dev->data->dev_private; - - return mlx5_flow_list_create(dev, &priv->flows, attr, items, actions, - error); + rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "action not supported"); + return NULL; } /** @@ -2566,99 +159,10 @@ mlx5_flow_create(struct rte_eth_dev *dev, * Flow to destroy. */ static void -mlx5_flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, - struct rte_flow *flow) +mlx5_flow_list_destroy(struct rte_eth_dev *dev __rte_unused, + struct mlx5_flows *list __rte_unused, + struct rte_flow *flow __rte_unused) { - struct priv *priv = dev->data->dev_private; - unsigned int i; - - if (flow->drop || !dev->data->dev_started) - goto free; - for (i = 0; flow->tunnel && i != flow->rss_conf.queue_num; ++i) { - /* Update queue tunnel type. */ - struct mlx5_rxq_data *rxq_data = (*priv->rxqs) - [(*flow->queues)[i]]; - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); - uint8_t tunnel = PTYPE_IDX(flow->tunnel); - - assert(rxq_ctrl->tunnel_types[tunnel] > 0); - rxq_ctrl->tunnel_types[tunnel] -= 1; - if (!rxq_ctrl->tunnel_types[tunnel]) { - /* Update tunnel type. */ - uint8_t j; - uint8_t types = 0; - uint8_t last; - - for (j = 0; j < RTE_DIM(rxq_ctrl->tunnel_types); j++) - if (rxq_ctrl->tunnel_types[j]) { - types += 1; - last = j; - } - /* Keep same if more than one tunnel types left. */ - if (types == 1) - rxq_data->tunnel = ptype_ext[last]; - else if (types == 0) - /* No tunnel type left. */ - rxq_data->tunnel = 0; - } - } - for (i = 0; flow->mark && i != flow->rss_conf.queue_num; ++i) { - struct rte_flow *tmp; - int mark = 0; - - /* - * To remove the mark from the queue, the queue must not be - * present in any other marked flow (RSS or not). - */ - TAILQ_FOREACH(tmp, list, next) { - unsigned int j; - uint16_t *tqs = NULL; - uint16_t tq_n = 0; - - if (!tmp->mark) - continue; - for (j = 0; j != hash_rxq_init_n; ++j) { - if (!tmp->frxq[j].hrxq) - continue; - tqs = tmp->frxq[j].hrxq->ind_table->queues; - tq_n = tmp->frxq[j].hrxq->ind_table->queues_n; - } - if (!tq_n) - continue; - for (j = 0; (j != tq_n) && !mark; j++) - if (tqs[j] == (*flow->queues)[i]) - mark = 1; - } - (*priv->rxqs)[(*flow->queues)[i]]->mark = mark; - } -free: - if (flow->drop) { - if (flow->frxq[HASH_RXQ_ETH].ibv_flow) - claim_zero(mlx5_glue->destroy_flow - (flow->frxq[HASH_RXQ_ETH].ibv_flow)); - rte_free(flow->frxq[HASH_RXQ_ETH].ibv_attr); - } else { - for (i = 0; i != hash_rxq_init_n; ++i) { - struct mlx5_flow *frxq = &flow->frxq[i]; - - if (frxq->ibv_flow) - claim_zero(mlx5_glue->destroy_flow - (frxq->ibv_flow)); - if (frxq->hrxq) - mlx5_hrxq_release(dev, frxq->hrxq); - if (frxq->ibv_attr) - rte_free(frxq->ibv_attr); - } - } - if (flow->cs) { - claim_zero(mlx5_glue->destroy_counter_set(flow->cs)); - flow->cs = NULL; - } - TAILQ_REMOVE(list, flow, next); - DRV_LOG(DEBUG, "port %u flow destroyed %p", dev->data->port_id, - (void *)flow); - rte_free(flow); } /** @@ -2690,97 +194,9 @@ mlx5_flow_list_flush(struct rte_eth_dev *dev, struct mlx5_flows *list) * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_create_drop_queue(struct rte_eth_dev *dev) +mlx5_flow_create_drop_queue(struct rte_eth_dev *dev __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct mlx5_hrxq_drop *fdq = NULL; - - assert(priv->pd); - assert(priv->ctx); - fdq = rte_calloc(__func__, 1, sizeof(*fdq), 0); - if (!fdq) { - DRV_LOG(WARNING, - "port %u cannot allocate memory for drop queue", - dev->data->port_id); - rte_errno = ENOMEM; - return -rte_errno; - } - fdq->cq = mlx5_glue->create_cq(priv->ctx, 1, NULL, NULL, 0); - if (!fdq->cq) { - DRV_LOG(WARNING, "port %u cannot allocate CQ for drop queue", - dev->data->port_id); - rte_errno = errno; - goto error; - } - fdq->wq = mlx5_glue->create_wq - (priv->ctx, - &(struct ibv_wq_init_attr){ - .wq_type = IBV_WQT_RQ, - .max_wr = 1, - .max_sge = 1, - .pd = priv->pd, - .cq = fdq->cq, - }); - if (!fdq->wq) { - DRV_LOG(WARNING, "port %u cannot allocate WQ for drop queue", - dev->data->port_id); - rte_errno = errno; - goto error; - } - fdq->ind_table = mlx5_glue->create_rwq_ind_table - (priv->ctx, - &(struct ibv_rwq_ind_table_init_attr){ - .log_ind_tbl_size = 0, - .ind_tbl = &fdq->wq, - .comp_mask = 0, - }); - if (!fdq->ind_table) { - DRV_LOG(WARNING, - "port %u cannot allocate indirection table for drop" - " queue", - dev->data->port_id); - rte_errno = errno; - goto error; - } - fdq->qp = mlx5_glue->create_qp_ex - (priv->ctx, - &(struct ibv_qp_init_attr_ex){ - .qp_type = IBV_QPT_RAW_PACKET, - .comp_mask = - IBV_QP_INIT_ATTR_PD | - IBV_QP_INIT_ATTR_IND_TABLE | - IBV_QP_INIT_ATTR_RX_HASH, - .rx_hash_conf = (struct ibv_rx_hash_conf){ - .rx_hash_function = - IBV_RX_HASH_FUNC_TOEPLITZ, - .rx_hash_key_len = rss_hash_default_key_len, - .rx_hash_key = rss_hash_default_key, - .rx_hash_fields_mask = 0, - }, - .rwq_ind_tbl = fdq->ind_table, - .pd = priv->pd - }); - if (!fdq->qp) { - DRV_LOG(WARNING, "port %u cannot allocate QP for drop queue", - dev->data->port_id); - rte_errno = errno; - goto error; - } - priv->flow_drop_queue = fdq; return 0; -error: - if (fdq->qp) - claim_zero(mlx5_glue->destroy_qp(fdq->qp)); - if (fdq->ind_table) - claim_zero(mlx5_glue->destroy_rwq_ind_table(fdq->ind_table)); - if (fdq->wq) - claim_zero(mlx5_glue->destroy_wq(fdq->wq)); - if (fdq->cq) - claim_zero(mlx5_glue->destroy_cq(fdq->cq)); - if (fdq) - rte_free(fdq); - priv->flow_drop_queue = NULL; - return -rte_errno; } /** @@ -2790,23 +206,8 @@ mlx5_flow_create_drop_queue(struct rte_eth_dev *dev) * Pointer to Ethernet device. */ void -mlx5_flow_delete_drop_queue(struct rte_eth_dev *dev) +mlx5_flow_delete_drop_queue(struct rte_eth_dev *dev __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct mlx5_hrxq_drop *fdq = priv->flow_drop_queue; - - if (!fdq) - return; - if (fdq->qp) - claim_zero(mlx5_glue->destroy_qp(fdq->qp)); - if (fdq->ind_table) - claim_zero(mlx5_glue->destroy_rwq_ind_table(fdq->ind_table)); - if (fdq->wq) - claim_zero(mlx5_glue->destroy_wq(fdq->wq)); - if (fdq->cq) - claim_zero(mlx5_glue->destroy_cq(fdq->cq)); - rte_free(fdq); - priv->flow_drop_queue = NULL; } /** @@ -2818,70 +219,9 @@ mlx5_flow_delete_drop_queue(struct rte_eth_dev *dev) * Pointer to a TAILQ flow list. */ void -mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) +mlx5_flow_stop(struct rte_eth_dev *dev __rte_unused, + struct mlx5_flows *list __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct rte_flow *flow; - unsigned int i; - - TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) { - struct mlx5_ind_table_ibv *ind_tbl = NULL; - - if (flow->drop) { - if (!flow->frxq[HASH_RXQ_ETH].ibv_flow) - continue; - claim_zero(mlx5_glue->destroy_flow - (flow->frxq[HASH_RXQ_ETH].ibv_flow)); - flow->frxq[HASH_RXQ_ETH].ibv_flow = NULL; - DRV_LOG(DEBUG, "port %u flow %p removed", - dev->data->port_id, (void *)flow); - /* Next flow. */ - continue; - } - /* Verify the flow has not already been cleaned. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!flow->frxq[i].ibv_flow) - continue; - /* - * Indirection table may be necessary to remove the - * flags in the Rx queues. - * This helps to speed-up the process by avoiding - * another loop. - */ - ind_tbl = flow->frxq[i].hrxq->ind_table; - break; - } - if (i == hash_rxq_init_n) - return; - if (flow->mark) { - assert(ind_tbl); - for (i = 0; i != ind_tbl->queues_n; ++i) - (*priv->rxqs)[ind_tbl->queues[i]]->mark = 0; - } - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!flow->frxq[i].ibv_flow) - continue; - claim_zero(mlx5_glue->destroy_flow - (flow->frxq[i].ibv_flow)); - flow->frxq[i].ibv_flow = NULL; - mlx5_hrxq_release(dev, flow->frxq[i].hrxq); - flow->frxq[i].hrxq = NULL; - } - DRV_LOG(DEBUG, "port %u flow %p removed", dev->data->port_id, - (void *)flow); - } - /* Cleanup Rx queue tunnel info. */ - for (i = 0; i != priv->rxqs_n; ++i) { - struct mlx5_rxq_data *q = (*priv->rxqs)[i]; - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of(q, struct mlx5_rxq_ctrl, rxq); - - if (!q) - continue; - memset((void *)rxq_ctrl->tunnel_types, 0, - sizeof(rxq_ctrl->tunnel_types)); - q->tunnel = 0; - } } /** @@ -2896,76 +236,9 @@ mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_start(struct rte_eth_dev *dev, struct mlx5_flows *list) +mlx5_flow_start(struct rte_eth_dev *dev __rte_unused, + struct mlx5_flows *list __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct rte_flow *flow; - - TAILQ_FOREACH(flow, list, next) { - unsigned int i; - - if (flow->drop) { - flow->frxq[HASH_RXQ_ETH].ibv_flow = - mlx5_glue->create_flow - (priv->flow_drop_queue->qp, - flow->frxq[HASH_RXQ_ETH].ibv_attr); - if (!flow->frxq[HASH_RXQ_ETH].ibv_flow) { - DRV_LOG(DEBUG, - "port %u flow %p cannot be applied", - dev->data->port_id, (void *)flow); - rte_errno = EINVAL; - return -rte_errno; - } - DRV_LOG(DEBUG, "port %u flow %p applied", - dev->data->port_id, (void *)flow); - /* Next flow. */ - continue; - } - for (i = 0; i != hash_rxq_init_n; ++i) { - if (!flow->frxq[i].ibv_attr) - continue; - flow->frxq[i].hrxq = - mlx5_hrxq_get(dev, flow->rss_conf.key, - flow->rss_conf.key_len, - flow->frxq[i].hash_fields, - flow->rss_conf.queue, - flow->rss_conf.queue_num, - flow->tunnel, - flow->rss_conf.level); - if (flow->frxq[i].hrxq) - goto flow_create; - flow->frxq[i].hrxq = - mlx5_hrxq_new(dev, flow->rss_conf.key, - flow->rss_conf.key_len, - flow->frxq[i].hash_fields, - flow->rss_conf.queue, - flow->rss_conf.queue_num, - flow->tunnel, - flow->rss_conf.level); - if (!flow->frxq[i].hrxq) { - DRV_LOG(DEBUG, - "port %u flow %p cannot create hash" - " rxq", - dev->data->port_id, (void *)flow); - rte_errno = EINVAL; - return -rte_errno; - } -flow_create: - mlx5_flow_dump(dev, flow, i); - flow->frxq[i].ibv_flow = - mlx5_glue->create_flow(flow->frxq[i].hrxq->qp, - flow->frxq[i].ibv_attr); - if (!flow->frxq[i].ibv_flow) { - DRV_LOG(DEBUG, - "port %u flow %p type %u cannot be" - " applied", - dev->data->port_id, (void *)flow, i); - rte_errno = EINVAL; - return -rte_errno; - } - } - mlx5_flow_create_update_rxqs(dev, flow); - } return 0; } @@ -3019,7 +292,6 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_attr attr = { .ingress = 1, - .priority = MLX5_CTRL_FLOW_PRIORITY, }; struct rte_flow_item items[] = { { @@ -3129,83 +401,6 @@ mlx5_flow_flush(struct rte_eth_dev *dev, return 0; } -#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT -/** - * Query flow counter. - * - * @param cs - * the counter set. - * @param counter_value - * returned data from the counter. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_flow_query_count(struct ibv_counter_set *cs, - struct mlx5_flow_counter_stats *counter_stats, - struct rte_flow_query_count *query_count, - struct rte_flow_error *error) -{ - uint64_t counters[2]; - struct ibv_query_counter_set_attr query_cs_attr = { - .cs = cs, - .query_flags = IBV_COUNTER_SET_FORCE_UPDATE, - }; - struct ibv_counter_set_data query_out = { - .out = counters, - .outlen = 2 * sizeof(uint64_t), - }; - int err = mlx5_glue->query_counter_set(&query_cs_attr, &query_out); - - if (err) - return rte_flow_error_set(error, err, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot read counter"); - query_count->hits_set = 1; - query_count->bytes_set = 1; - query_count->hits = counters[0] - counter_stats->hits; - query_count->bytes = counters[1] - counter_stats->bytes; - if (query_count->reset) { - counter_stats->hits = counters[0]; - counter_stats->bytes = counters[1]; - } - return 0; -} - -/** - * Query a flows. - * - * @see rte_flow_query() - * @see rte_flow_ops - */ -int -mlx5_flow_query(struct rte_eth_dev *dev __rte_unused, - struct rte_flow *flow, - const struct rte_flow_action *action __rte_unused, - void *data, - struct rte_flow_error *error) -{ - if (flow->cs) { - int ret; - - ret = mlx5_flow_query_count(flow->cs, - &flow->counter_stats, - (struct rte_flow_query_count *)data, - error); - if (ret) - return ret; - } else { - return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "no counter found for flow"); - } - return 0; -} -#endif - /** * Isolated mode. * @@ -3445,18 +640,11 @@ mlx5_fdir_filter_add(struct rte_eth_dev *dev, .type = 0, }, }; - struct mlx5_flow_parse parser = { - .layer = HASH_RXQ_ETH, - }; struct rte_flow_error error; struct rte_flow *flow; int ret; ret = mlx5_fdir_filter_convert(dev, fdir_filter, &attributes); - if (ret) - return ret; - ret = mlx5_flow_convert(dev, &attributes.attr, attributes.items, - attributes.actions, &error, &parser); if (ret) return ret; flow = mlx5_flow_list_create(dev, &priv->flows, &attributes.attr, @@ -3482,96 +670,11 @@ mlx5_fdir_filter_add(struct rte_eth_dev *dev, * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -mlx5_fdir_filter_delete(struct rte_eth_dev *dev, - const struct rte_eth_fdir_filter *fdir_filter) +mlx5_fdir_filter_delete(struct rte_eth_dev *dev __rte_unused, + const struct rte_eth_fdir_filter *fdir_filter + __rte_unused) { - struct priv *priv = dev->data->dev_private; - struct mlx5_fdir attributes = { - .attr.group = 0, - }; - struct mlx5_flow_parse parser = { - .create = 1, - .layer = HASH_RXQ_ETH, - }; - struct rte_flow_error error; - struct rte_flow *flow; - unsigned int i; - int ret; - - ret = mlx5_fdir_filter_convert(dev, fdir_filter, &attributes); - if (ret) - return ret; - ret = mlx5_flow_convert(dev, &attributes.attr, attributes.items, - attributes.actions, &error, &parser); - if (ret) - goto exit; - /* - * Special case for drop action which is only set in the - * specifications when the flow is created. In this situation the - * drop specification is missing. - */ - if (parser.drop) { - struct ibv_flow_spec_action_drop *drop; - - drop = (void *)((uintptr_t)parser.queue[HASH_RXQ_ETH].ibv_attr + - parser.queue[HASH_RXQ_ETH].offset); - *drop = (struct ibv_flow_spec_action_drop){ - .type = IBV_FLOW_SPEC_ACTION_DROP, - .size = sizeof(struct ibv_flow_spec_action_drop), - }; - parser.queue[HASH_RXQ_ETH].ibv_attr->num_of_specs++; - } - TAILQ_FOREACH(flow, &priv->flows, next) { - struct ibv_flow_attr *attr; - struct ibv_spec_header *attr_h; - void *spec; - struct ibv_flow_attr *flow_attr; - struct ibv_spec_header *flow_h; - void *flow_spec; - unsigned int specs_n; - unsigned int queue_id = parser.drop ? HASH_RXQ_ETH : - parser.layer; - - attr = parser.queue[queue_id].ibv_attr; - flow_attr = flow->frxq[queue_id].ibv_attr; - /* Compare first the attributes. */ - if (!flow_attr || - memcmp(attr, flow_attr, sizeof(struct ibv_flow_attr))) - continue; - if (attr->num_of_specs == 0) - continue; - spec = (void *)((uintptr_t)attr + - sizeof(struct ibv_flow_attr)); - flow_spec = (void *)((uintptr_t)flow_attr + - sizeof(struct ibv_flow_attr)); - specs_n = RTE_MIN(attr->num_of_specs, flow_attr->num_of_specs); - for (i = 0; i != specs_n; ++i) { - attr_h = spec; - flow_h = flow_spec; - if (memcmp(spec, flow_spec, - RTE_MIN(attr_h->size, flow_h->size))) - goto wrong_flow; - spec = (void *)((uintptr_t)spec + attr_h->size); - flow_spec = (void *)((uintptr_t)flow_spec + - flow_h->size); - } - /* At this point, the flow match. */ - break; -wrong_flow: - /* The flow does not match. */ - continue; - } - if (flow) - mlx5_flow_list_destroy(dev, &priv->flows, flow); -exit: - if (ret) - ret = rte_errno; /* Save rte_errno before cleanup. */ - for (i = 0; i != hash_rxq_init_n; ++i) { - if (parser.queue[i].ibv_attr) - rte_free(parser.queue[i].ibv_attr); - } - if (ret) - rte_errno = ret; /* Restore rte_errno. */ + rte_errno = ENOTSUP; return -rte_errno; } @@ -3738,45 +841,7 @@ mlx5_dev_filter_ctrl(struct rte_eth_dev *dev, * number of supported Verbs flow priority. */ unsigned int -mlx5_get_max_verbs_prio(struct rte_eth_dev *dev) +mlx5_get_max_verbs_prio(struct rte_eth_dev *dev __rte_unused) { - struct priv *priv = dev->data->dev_private; - unsigned int verb_priorities = MLX5_VERBS_FLOW_PRIO_8; - struct { - struct ibv_flow_attr attr; - struct ibv_flow_spec_eth eth; - struct ibv_flow_spec_action_drop drop; - } flow_attr = { - .attr = { - .num_of_specs = 2, - }, - .eth = { - .type = IBV_FLOW_SPEC_ETH, - .size = sizeof(struct ibv_flow_spec_eth), - }, - .drop = { - .size = sizeof(struct ibv_flow_spec_action_drop), - .type = IBV_FLOW_SPEC_ACTION_DROP, - }, - }; - struct ibv_flow *flow; - - do { - flow_attr.attr.priority = verb_priorities - 1; - flow = mlx5_glue->create_flow(priv->flow_drop_queue->qp, - &flow_attr.attr); - if (flow) { - claim_zero(mlx5_glue->destroy_flow(flow)); - /* Try more priorities. */ - verb_priorities *= 2; - } else { - /* Failed, restore last right number. */ - verb_priorities /= 2; - break; - } - } while (1); - DRV_LOG(DEBUG, "port %u Verbs flow priorities: %d," - " user flow priorities: %d", - dev->data->port_id, verb_priorities, MLX5_CTRL_FLOW_PRIORITY); - return verb_priorities; + return 8; } From patchwork Wed Jun 27 15:07:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41675 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CFAE21BF9C; Wed, 27 Jun 2018 17:07:37 +0200 (CEST) Received: from mail-wr0-f195.google.com (mail-wr0-f195.google.com [209.85.128.195]) by dpdk.org (Postfix) with ESMTP id 99E081BF92 for ; Wed, 27 Jun 2018 17:07:36 +0200 (CEST) Received: by mail-wr0-f195.google.com with SMTP id b8-v6so2405812wro.6 for ; Wed, 27 Jun 2018 08:07:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ToivEB5ymVB3JKbO2dxhXZxYCY3xI42unLIODHnd6iQ=; b=ZlUDkeqlk51Lv09BeO1DLrYrwvcwdHTCqvmQI0h2yTCAaU2wjqvLrP2C9wvHiVZlDR 4c4SO0ULynEPW2wfdmQlBKskEvwQiCKUSQrdlzGv8zsnSlNexhqhV7OEoFEuOaBgnkjK DwTWd5XJNUIKWkQ5MNhbw3xqEOJs6HZzemPbNfH5+w3YP2d96ZvQB0R93Vp/WONx0K+S VeqmSsYIYLYL1Wr+1UuQGIq/mTsGs/uwCsO3VBjJ4uzMWBqZOUhLKrD/zxOqiv++Ocz3 DSP0kjS49Aemx4Mz+8Q/VnAgTSGjngEZEWwY071oqdrEM6HBmlr4u1BSM/lSbVFfLxm0 w9ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ToivEB5ymVB3JKbO2dxhXZxYCY3xI42unLIODHnd6iQ=; b=pj7AE3eRjZ1PLO6RmjmOSFiY8PrtconK6dc54URui/Kh4tubAogJZcv0rjPJEHq12p C7CJsdk8DoWvQirgdES4vm0ioi4cZWAMZ+uLpua3Kpm2kft3OqvvOzzp1zoITOS18Va9 l03moUHuEtEqVtqXoMK8JE2qyRsiMwOt5Rhkdv+hoEK1pQm0ztIumKrcOtloXyVMASC1 0Nq/Uwmc6XfiytaqDtGQ94k6qYT+t5kmsX2PwgNggnXMjMmtcSKkNyUVb1DsXlwouZch lQCZFw0/AtG8F4l0aVPebLLC8sloUwULEMk66VgkZfo0BGfMLwmYeRFPF2313Cn8+wm7 bhtw== X-Gm-Message-State: APt69E2TxNvoXi4nv/7cedC2abI/V/Zmr05hf84JK8Ikq5I7r3sQlrxU 1Z1DEhCeeHKJS3+0CykR+4UwIzbV8w== X-Google-Smtp-Source: AAOMgpc2ewLDlrHHCecgcf77zwl21VY7c5si30FNIgmx44GhvUPDLbey9exkKiws+VlqGAUp/wuhjQ== X-Received: by 2002:adf:fc8c:: with SMTP id g12-v6mr5508760wrr.216.1530112055837; Wed, 27 Jun 2018 08:07:35 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:35 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:34 +0200 Message-Id: <1368a8720f3ec3c40e47a6b1d9ef1edf0f38a646.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 02/20] net/mlx5: handle drop queues are regular queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Drop queues are essentially used in flows due to Verbs API, the information if the fate of the flow is a drop or not is already present in the flow. Due to this, drop queues can be fully mapped on regular queues. Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5.c | 24 ++-- drivers/net/mlx5/mlx5.h | 14 ++- drivers/net/mlx5/mlx5_flow.c | 94 +++++++------- drivers/net/mlx5/mlx5_rxq.c | 232 +++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 8 ++ 5 files changed, 310 insertions(+), 62 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index d01c0f6cf..aba6c1f9f 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -260,7 +260,6 @@ mlx5_dev_close(struct rte_eth_dev *dev) priv->txqs_n = 0; priv->txqs = NULL; } - mlx5_flow_delete_drop_queue(dev); mlx5_mprq_free_mp(dev); mlx5_mr_release(dev); if (priv->pd != NULL) { @@ -1100,22 +1099,15 @@ mlx5_dev_spawn_one(struct rte_device *dpdk_dev, mlx5_link_update(eth_dev, 0); /* Store device configuration on private structure. */ priv->config = config; - /* Create drop queue. */ - err = mlx5_flow_create_drop_queue(eth_dev); - if (err) { - DRV_LOG(ERR, "port %u drop queue allocation failed: %s", - eth_dev->data->port_id, strerror(rte_errno)); - err = rte_errno; - goto error; - } /* Supported Verbs flow priority number detection. */ - if (verb_priorities == 0) - verb_priorities = mlx5_get_max_verbs_prio(eth_dev); - if (verb_priorities < MLX5_VERBS_FLOW_PRIO_8) { - DRV_LOG(ERR, "port %u wrong Verbs flow priorities: %u", - eth_dev->data->port_id, verb_priorities); - err = ENOTSUP; - goto error; + if (verb_priorities == 0) { + err = mlx5_verbs_max_prio(eth_dev); + if (err < 0) { + DRV_LOG(ERR, "port %u wrong Verbs flow priorities", + eth_dev->data->port_id); + goto error; + } + verb_priorities = err; } priv->config.max_verbs_prio = verb_priorities; /* diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 6674a77d4..11aa0932f 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -131,9 +131,6 @@ enum mlx5_verbs_alloc_type { MLX5_VERBS_ALLOC_TYPE_RX_QUEUE, }; -/* 8 Verbs priorities. */ -#define MLX5_VERBS_FLOW_PRIO_8 8 - /** * Verbs allocator needs a context to know in the callback which kind of * resources it is allocating. @@ -145,6 +142,12 @@ struct mlx5_verbs_alloc_ctx { LIST_HEAD(mlx5_mr_list, mlx5_mr); +/* Flow drop context necessary due to Verbs API. */ +struct mlx5_drop { + struct mlx5_hrxq *hrxq; /* Hash Rx queue queue. */ + struct mlx5_rxq_ibv *rxq; /* Verbs Rx queue. */ +}; + struct priv { LIST_ENTRY(priv) mem_event_cb; /* Called by memory event callback. */ struct rte_eth_dev_data *dev_data; /* Pointer to device data. */ @@ -175,7 +178,7 @@ struct priv { struct rte_intr_handle intr_handle; /* Interrupt handler. */ unsigned int (*reta_idx)[]; /* RETA index table. */ unsigned int reta_idx_n; /* RETA index size. */ - struct mlx5_hrxq_drop *flow_drop_queue; /* Flow drop queue. */ + struct mlx5_drop drop; /* Flow drop queues. */ struct mlx5_flows flows; /* RTE Flow rules. */ struct mlx5_flows ctrl_flows; /* Control flow rules. */ struct { @@ -306,7 +309,8 @@ int mlx5_traffic_restart(struct rte_eth_dev *dev); /* mlx5_flow.c */ -unsigned int mlx5_get_max_verbs_prio(struct rte_eth_dev *dev); +int mlx5_verbs_max_prio(struct rte_eth_dev *dev); +void mlx5_flow_print(struct rte_flow *flow); int mlx5_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, const struct rte_flow_item items[], diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index a45cb06e1..8b5b695d4 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -75,6 +75,58 @@ struct ibv_spec_header { uint16_t size; }; + /** + * Get the maximum number of priority available. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * number of supported Verbs flow priority on success, a negative errno + * value otherwise and rte_errno is set. + */ +int +mlx5_verbs_max_prio(struct rte_eth_dev *dev) +{ + struct { + struct ibv_flow_attr attr; + struct ibv_flow_spec_eth eth; + struct ibv_flow_spec_action_drop drop; + } flow_attr = { + .attr = { + .num_of_specs = 2, + }, + .eth = { + .type = IBV_FLOW_SPEC_ETH, + .size = sizeof(struct ibv_flow_spec_eth), + }, + .drop = { + .size = sizeof(struct ibv_flow_spec_action_drop), + .type = IBV_FLOW_SPEC_ACTION_DROP, + }, + }; + struct ibv_flow *flow; + uint32_t verb_priorities; + struct mlx5_hrxq *drop = mlx5_hrxq_drop_new(dev); + + if (!drop) { + rte_errno = ENOTSUP; + return -rte_errno; + } + for (verb_priorities = 0; 1; verb_priorities++) { + flow_attr.attr.priority = verb_priorities; + flow = mlx5_glue->create_flow(drop->qp, + &flow_attr.attr); + if (!flow) + break; + claim_zero(mlx5_glue->destroy_flow(flow)); + } + mlx5_hrxq_drop_release(dev, drop); + DRV_LOG(INFO, "port %u flow maximum priority: %d", + dev->data->port_id, verb_priorities); + return verb_priorities; +} + /** * Convert a flow. * @@ -184,32 +236,6 @@ mlx5_flow_list_flush(struct rte_eth_dev *dev, struct mlx5_flows *list) } } -/** - * Create drop queue. - * - * @param dev - * Pointer to Ethernet device. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -int -mlx5_flow_create_drop_queue(struct rte_eth_dev *dev __rte_unused) -{ - return 0; -} - -/** - * Delete drop queue. - * - * @param dev - * Pointer to Ethernet device. - */ -void -mlx5_flow_delete_drop_queue(struct rte_eth_dev *dev __rte_unused) -{ -} - /** * Remove all flows. * @@ -292,6 +318,7 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_attr attr = { .ingress = 1, + .priority = priv->config.max_verbs_prio - 1, }; struct rte_flow_item items[] = { { @@ -830,18 +857,3 @@ mlx5_dev_filter_ctrl(struct rte_eth_dev *dev, } return 0; } - -/** - * Detect number of Verbs flow priorities supported. - * - * @param dev - * Pointer to Ethernet device. - * - * @return - * number of supported Verbs flow priority. - */ -unsigned int -mlx5_get_max_verbs_prio(struct rte_eth_dev *dev __rte_unused) -{ - return 8; -} diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 17db7c160..5f17dce50 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1949,3 +1949,235 @@ mlx5_hrxq_ibv_verify(struct rte_eth_dev *dev) } return ret; } + +/** + * Create a drop Rx queue Verbs object. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * The Verbs object initialised, NULL otherwise and rte_errno is set. + */ +struct mlx5_rxq_ibv * +mlx5_rxq_ibv_drop_new(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + struct ibv_cq *cq; + struct ibv_wq *wq = NULL; + struct mlx5_rxq_ibv *rxq; + + if (priv->drop.rxq) + return priv->drop.rxq; + cq = mlx5_glue->create_cq(priv->ctx, 1, NULL, NULL, 0); + if (!cq) { + DEBUG("port %u cannot allocate CQ for drop queue", + dev->data->port_id); + rte_errno = errno; + goto error; + } + wq = mlx5_glue->create_wq(priv->ctx, + &(struct ibv_wq_init_attr){ + .wq_type = IBV_WQT_RQ, + .max_wr = 1, + .max_sge = 1, + .pd = priv->pd, + .cq = cq, + }); + if (!wq) { + DEBUG("port %u cannot allocate WQ for drop queue", + dev->data->port_id); + rte_errno = errno; + goto error; + } + rxq = rte_calloc(__func__, 1, sizeof(*rxq), 0); + if (!rxq) { + DEBUG("port %u cannot allocate drop Rx queue memory", + dev->data->port_id); + rte_errno = ENOMEM; + goto error; + } + rxq->cq = cq; + rxq->wq = wq; + priv->drop.rxq = rxq; + return rxq; +error: + if (wq) + claim_zero(mlx5_glue->destroy_wq(wq)); + if (cq) + claim_zero(mlx5_glue->destroy_cq(cq)); + return NULL; +} + +/** + * Release a drop Rx queue Verbs object. + * + * @param dev + * Pointer to Ethernet device. + * @param rxq + * Pointer to the drop Verbs Rx queue. + * + * @return + * The Verbs object initialised, NULL otherwise and rte_errno is set. + */ +void +mlx5_rxq_ibv_drop_release(struct rte_eth_dev *dev, struct mlx5_rxq_ibv *rxq) +{ + if (rxq->wq) + claim_zero(mlx5_glue->destroy_wq(rxq->wq)); + if (rxq->cq) + claim_zero(mlx5_glue->destroy_cq(rxq->cq)); + rte_free(rxq); + ((struct priv *)dev->data->dev_private)->drop.rxq = NULL; +} + +/** + * Create a drop indirection table. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * The Verbs object initialised, NULL otherwise and rte_errno is set. + */ +struct mlx5_ind_table_ibv * +mlx5_ind_table_ibv_drop_new(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + struct mlx5_ind_table_ibv *ind_tbl; + struct mlx5_rxq_ibv *rxq; + struct mlx5_ind_table_ibv tmpl; + + rxq = mlx5_rxq_ibv_drop_new(dev); + if (!rxq) + return NULL; + tmpl.ind_table = mlx5_glue->create_rwq_ind_table + (priv->ctx, + &(struct ibv_rwq_ind_table_init_attr){ + .log_ind_tbl_size = 0, + .ind_tbl = &rxq->wq, + .comp_mask = 0, + }); + if (!tmpl.ind_table) { + DEBUG("port %u cannot allocate indirection table for drop" + " queue", + dev->data->port_id); + rte_errno = errno; + goto error; + } + ind_tbl = rte_calloc(__func__, 1, sizeof(*ind_tbl), 0); + if (!ind_tbl) { + rte_errno = ENOMEM; + goto error; + } + ind_tbl->ind_table = tmpl.ind_table; + return ind_tbl; +error: + mlx5_rxq_ibv_drop_release(dev, rxq); + return NULL; +} + +/** + * Release a drop indirection table. + * + * @param dev + * Pointer to Ethernet device. + * @param ind_tbl + * Indirection table to release. + */ +void +mlx5_ind_table_ibv_drop_release(struct rte_eth_dev *dev, + struct mlx5_ind_table_ibv *ind_tbl) +{ + claim_zero(mlx5_glue->destroy_rwq_ind_table(ind_tbl->ind_table)); + mlx5_rxq_ibv_drop_release + (dev, ((struct priv *)dev->data->dev_private)->drop.rxq); + rte_free(ind_tbl); +} + +/** + * Create a drop Rx Hash queue. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * The Verbs object initialised, NULL otherwise and rte_errno is set. + */ +struct mlx5_hrxq * +mlx5_hrxq_drop_new(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + struct mlx5_ind_table_ibv *ind_tbl; + struct ibv_qp *qp; + struct mlx5_hrxq *hrxq; + + if (priv->drop.hrxq) { + rte_atomic32_inc(&priv->drop.hrxq->refcnt); + return priv->drop.hrxq; + } + ind_tbl = mlx5_ind_table_ibv_drop_new(dev); + if (!ind_tbl) + return NULL; + qp = mlx5_glue->create_qp_ex(priv->ctx, + &(struct ibv_qp_init_attr_ex){ + .qp_type = IBV_QPT_RAW_PACKET, + .comp_mask = + IBV_QP_INIT_ATTR_PD | + IBV_QP_INIT_ATTR_IND_TABLE | + IBV_QP_INIT_ATTR_RX_HASH, + .rx_hash_conf = (struct ibv_rx_hash_conf){ + .rx_hash_function = + IBV_RX_HASH_FUNC_TOEPLITZ, + .rx_hash_key_len = rss_hash_default_key_len, + .rx_hash_key = rss_hash_default_key, + .rx_hash_fields_mask = 0, + }, + .rwq_ind_tbl = ind_tbl->ind_table, + .pd = priv->pd + }); + if (!qp) { + DEBUG("port %u cannot allocate QP for drop queue", + dev->data->port_id); + rte_errno = errno; + goto error; + } + hrxq = rte_calloc(__func__, 1, sizeof(*hrxq), 0); + if (!hrxq) { + DRV_LOG(WARNING, + "port %u cannot allocate memory for drop queue", + dev->data->port_id); + rte_errno = ENOMEM; + goto error; + } + hrxq->ind_table = ind_tbl; + hrxq->qp = qp; + priv->drop.hrxq = hrxq; + rte_atomic32_set(&hrxq->refcnt, 1); + return hrxq; +error: + if (ind_tbl) + mlx5_ind_table_ibv_drop_release(dev, ind_tbl); + return NULL; +} + +/** + * Release a drop hash Rx queue. + * + * @param dev + * Pointer to Ethernet device. + * @param hrxq + * Pointer to Hash Rx queue to release. + */ +void +mlx5_hrxq_drop_release(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq) +{ + struct priv *priv = dev->data->dev_private; + + if (rte_atomic32_dec_and_test(&hrxq->refcnt)) { + claim_zero(mlx5_glue->destroy_qp(hrxq->qp)); + mlx5_ind_table_ibv_drop_release(dev, hrxq->ind_table); + rte_free(hrxq); + priv->drop.hrxq = NULL; + } +} diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index f53bb43c3..51f7f678b 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -245,6 +245,9 @@ struct mlx5_rxq_ibv *mlx5_rxq_ibv_new(struct rte_eth_dev *dev, uint16_t idx); struct mlx5_rxq_ibv *mlx5_rxq_ibv_get(struct rte_eth_dev *dev, uint16_t idx); int mlx5_rxq_ibv_release(struct mlx5_rxq_ibv *rxq_ibv); int mlx5_rxq_ibv_releasable(struct mlx5_rxq_ibv *rxq_ibv); +struct mlx5_rxq_ibv *mlx5_rxq_ibv_drop_new(struct rte_eth_dev *dev); +void mlx5_rxq_ibv_drop_release(struct rte_eth_dev *dev, + struct mlx5_rxq_ibv *rxq); int mlx5_rxq_ibv_verify(struct rte_eth_dev *dev); struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, unsigned int socket, @@ -265,6 +268,9 @@ struct mlx5_ind_table_ibv *mlx5_ind_table_ibv_get(struct rte_eth_dev *dev, int mlx5_ind_table_ibv_release(struct rte_eth_dev *dev, struct mlx5_ind_table_ibv *ind_tbl); int mlx5_ind_table_ibv_verify(struct rte_eth_dev *dev); +struct mlx5_ind_table_ibv *mlx5_ind_table_ibv_drop_new(struct rte_eth_dev *dev); +void mlx5_ind_table_ibv_drop_release(struct rte_eth_dev *dev, + struct mlx5_ind_table_ibv *ind_tbl); struct mlx5_hrxq *mlx5_hrxq_new(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, @@ -277,6 +283,8 @@ struct mlx5_hrxq *mlx5_hrxq_get(struct rte_eth_dev *dev, uint32_t tunnel, uint32_t rss_level); int mlx5_hrxq_release(struct rte_eth_dev *dev, struct mlx5_hrxq *hxrq); int mlx5_hrxq_ibv_verify(struct rte_eth_dev *dev); +struct mlx5_hrxq *mlx5_hrxq_drop_new(struct rte_eth_dev *dev); +void mlx5_hrxq_drop_release(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq); uint64_t mlx5_get_rx_port_offloads(void); uint64_t mlx5_get_rx_queue_offloads(struct rte_eth_dev *dev); From patchwork Wed Jun 27 15:07:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41676 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EB8741BFA6; Wed, 27 Jun 2018 17:07:39 +0200 (CEST) Received: from mail-wr0-f193.google.com (mail-wr0-f193.google.com [209.85.128.193]) by dpdk.org (Postfix) with ESMTP id 259EF1BF9C for ; Wed, 27 Jun 2018 17:07:37 +0200 (CEST) Received: by mail-wr0-f193.google.com with SMTP id f16-v6so2408810wrm.3 for ; Wed, 27 Jun 2018 08:07:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=60xI5M+vRxd4EpeGx71HPnAEEJfS/TUlZpd0pfrPlP0=; b=TZ5ix9TjogwO3JCYTO8/H5NFn4nrbgCV3Y/DBL5kHN9ZKhmU/UqGC3UJL5e05/NiRp K0fgIdjSzBjh+EZc/5jLGUuexyk/wUUuNJjx9gIr1grOnOy90H9ngzpRcNMgprLzEn6X lEFmeG+nOhGkXGnKdciU6ja8b7ItiT1hgAIg6DG9q0pXLk4j2WKMfUhjoY4QT6SKLbIb 9ezn8fklGI3ws33s/4NoaQxSJtUOYsIT0D5nbCnzZdiETORoLeGrn9h3q6t5y2gt9nh9 45DnwbwmNmFHoWJpEULfGe46mkD3cfC+4tdo6mG8CmXmjl0pn/RAYa/HBkSW6OsUZRCC cL4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=60xI5M+vRxd4EpeGx71HPnAEEJfS/TUlZpd0pfrPlP0=; b=Gp49/7BcRfv1h8BKDH69X3E6lZR/+DpDdH/syjovx1E/TnNwph9GO4XQ07ezOuzq9d DFyRB72P1SB/Ps7JsxcM8JbenhaUJJJDsHjMytx7zAeLPgx/XCORv0jAt2EIRZkenMty UkjG9Ay3960avRr4DyyhWrF5exwRL9nB+xCpdDFtftadwYaLIs0hyBezSK65Xozw91BU m7m7dfqnJS9jZrqvR0E95bxgfOkwRSY+T/7vtdQFed1j2wlpdOVx9ydnnWak2QgoVRho S+wfQTMqUpYpphEXsRdJvOxBIjoz4+VWYGrgbUTdZjOO7o8F4fvhYnwWubNF4aHFANba 83VQ== X-Gm-Message-State: APt69E1EKY8RRjfQRqtimQINtZ404c2iqhimuAK7UXSzCPqQekJaooqX jITahpIMNj6oucau4PB1EjrgnmiYlg== X-Google-Smtp-Source: AAOMgpfidZOhevN8HAVvdxMXSyjopUsLfi7C5hECsMqCe/4SRXjJUIfGOv6hvu/bfD/56Z81JTzYPQ== X-Received: by 2002:adf:9148:: with SMTP id j66-v6mr5739976wrj.118.1530112056547; Wed, 27 Jun 2018 08:07:36 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:36 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:35 +0200 Message-Id: <3c8d9a28a2627c3bde7eaa1a059704c9cca50f1e.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 03/20] net/mlx5: replace verbs priorities by flow X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previous work introduce verbs priorities, whereas the PMD is making translation between Flow priority into Verbs. Rename this to make more sense on what the PMD has to translate. Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5.c | 15 ++++----------- drivers/net/mlx5/mlx5.h | 4 ++-- drivers/net/mlx5/mlx5_flow.c | 22 +++++++++++----------- 3 files changed, 17 insertions(+), 24 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index aba6c1f9f..c3c8dffae 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -703,7 +703,6 @@ mlx5_dev_spawn_one(struct rte_device *dpdk_dev, unsigned int tunnel_en = 0; unsigned int mpls_en = 0; unsigned int swp = 0; - unsigned int verb_priorities = 0; unsigned int mprq = 0; unsigned int mprq_min_stride_size_n = 0; unsigned int mprq_max_stride_size_n = 0; @@ -1100,16 +1099,10 @@ mlx5_dev_spawn_one(struct rte_device *dpdk_dev, /* Store device configuration on private structure. */ priv->config = config; /* Supported Verbs flow priority number detection. */ - if (verb_priorities == 0) { - err = mlx5_verbs_max_prio(eth_dev); - if (err < 0) { - DRV_LOG(ERR, "port %u wrong Verbs flow priorities", - eth_dev->data->port_id); - goto error; - } - verb_priorities = err; - } - priv->config.max_verbs_prio = verb_priorities; + err = mlx5_flow_priorities(eth_dev); + if (err < 0) + goto error; + priv->config.flow_prio = err; /* * Once the device is added to the list of memory event * callback, its global MR cache table cannot be expanded diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 11aa0932f..ed8c1c9a2 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -114,7 +114,7 @@ struct mlx5_dev_config { unsigned int min_rxqs_num; /* Rx queue count threshold to enable MPRQ. */ } mprq; /* Configurations for Multi-Packet RQ. */ - unsigned int max_verbs_prio; /* Number of Verb flow priorities. */ + unsigned int flow_prio; /* Number of flow priorities. */ unsigned int tso_max_payload_sz; /* Maximum TCP payload for TSO. */ unsigned int ind_table_max_size; /* Maximum indirection table size. */ int txq_inline; /* Maximum packet size for inlining. */ @@ -309,7 +309,7 @@ int mlx5_traffic_restart(struct rte_eth_dev *dev); /* mlx5_flow.c */ -int mlx5_verbs_max_prio(struct rte_eth_dev *dev); +int mlx5_flow_priorities(struct rte_eth_dev *dev); void mlx5_flow_print(struct rte_flow *flow); int mlx5_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 8b5b695d4..5d3bc183d 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -82,11 +82,11 @@ struct ibv_spec_header { * Pointer to Ethernet device. * * @return - * number of supported Verbs flow priority on success, a negative errno - * value otherwise and rte_errno is set. + * number of supported flow priority on success, a negative errno value + * otherwise and rte_errno is set. */ int -mlx5_verbs_max_prio(struct rte_eth_dev *dev) +mlx5_flow_priorities(struct rte_eth_dev *dev) { struct { struct ibv_flow_attr attr; @@ -106,25 +106,25 @@ mlx5_verbs_max_prio(struct rte_eth_dev *dev) }, }; struct ibv_flow *flow; - uint32_t verb_priorities; struct mlx5_hrxq *drop = mlx5_hrxq_drop_new(dev); + uint16_t vprio[] = { 8, 16 }; + int i; if (!drop) { rte_errno = ENOTSUP; return -rte_errno; } - for (verb_priorities = 0; 1; verb_priorities++) { - flow_attr.attr.priority = verb_priorities; - flow = mlx5_glue->create_flow(drop->qp, - &flow_attr.attr); + for (i = 0; i != RTE_DIM(vprio); i++) { + flow_attr.attr.priority = vprio[i] - 1; + flow = mlx5_glue->create_flow(drop->qp, &flow_attr.attr); if (!flow) break; claim_zero(mlx5_glue->destroy_flow(flow)); } mlx5_hrxq_drop_release(dev, drop); DRV_LOG(INFO, "port %u flow maximum priority: %d", - dev->data->port_id, verb_priorities); - return verb_priorities; + dev->data->port_id, vprio[i]); + return vprio[i]; } /** @@ -318,7 +318,7 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_attr attr = { .ingress = 1, - .priority = priv->config.max_verbs_prio - 1, + .priority = priv->config.flow_prio - 1, }; struct rte_flow_item items[] = { { From patchwork Wed Jun 27 15:07:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41678 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 96F021BFBB; Wed, 27 Jun 2018 17:07:44 +0200 (CEST) Received: from mail-wr0-f196.google.com (mail-wr0-f196.google.com [209.85.128.196]) by dpdk.org (Postfix) with ESMTP id 8A7E51BFA1 for ; Wed, 27 Jun 2018 17:07:38 +0200 (CEST) Received: by mail-wr0-f196.google.com with SMTP id a12-v6so2419025wro.1 for ; Wed, 27 Jun 2018 08:07:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=PDKoLJj4Z7kbyTDSWa1bp0lfTAguLQDUgCg65X9euAM=; b=wModLgvgkmntMlpNfiSG4aniwLPY/5WKz2xvQtdCR2E1PTcw+RGG1DE9U7YN9gVKQy m3EGyu25QvaLEUaNyt8jJuX8+oKv3qXshJJs2U0B8Dq/6wXYiLhqLfNMlPqBURrPDoQV Vr5EJxZR/p/Yg+PUOnLnY0Xp4Q/1jA5Z5T5PdQbAhrIcyjZ2VEro4gp6MW4a6/e8mkC+ FinUofoQTb+F/7YNHuxJODeE3wIYnRgece2GTgsmBP5U1a2jgK5ym7JRlrzed1j3HWky vn9ZcGE5UPDACnLX0f7WbdV0G+ciFI2DRND6kmIBOsks7xQh+zmxA6aFYKUXEGAfJw/q zH3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=PDKoLJj4Z7kbyTDSWa1bp0lfTAguLQDUgCg65X9euAM=; b=c1de0u8Y7GDtzyIdZM0p+oLqC7ui0LsrE/IchDmtJdw49/L3rcmr2vLVSyZnUJdyGS NGJkf720b9Tgyf8VSZ7G/OtJX6LZpgkzBC3GL8JmukoeXr8U6smCht92318riKC0/99H KgrN80LlWA5SVQIiUXw6Akt4AXgxSAAH/rHaWjFA38DPYjAfbWBCLzWO9LapKYbQ5tL2 ydnzSy1A0qj93rpHjpjQ9LIqD7RRKm23AFMHfVJtu1wTJqNw9iwweiVvil/Yifm81n2+ Nfew7TR23neC5InoH6U7dQvxj1CUIMTPgJ8v4rq4gUWQsmk1njIhXwqmwIgewM12sSCK H5TA== X-Gm-Message-State: APt69E0c1Cj/cWfhAnf1xIUkwcZmFGOi/hbAsBIlQg5u5aUc4asNE7Q+ VS4509CKNCn6R7mM8N7/O5CaSA5sQQ== X-Google-Smtp-Source: AAOMgpffVha0c6hfa7AVju1Dp9ZryKtjsJhF9WSVoCDdOY1fSndU6mgVMXv5DMvYDj9Hwe70SFUncA== X-Received: by 2002:adf:9582:: with SMTP id p2-v6mr5463114wrp.252.1530112057718; Wed, 27 Jun 2018 08:07:37 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:37 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:36 +0200 Message-Id: <957989d98f2363d6e2e377bd54893cf7b29e4985.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 04/20] net/mlx5: support flow Ethernet item among with drop action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5.c | 1 + drivers/net/mlx5/mlx5_flow.c | 671 ++++++++++++++++++++++++++++++++--- 2 files changed, 630 insertions(+), 42 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index c3c8dffae..665a3c31f 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -241,6 +241,7 @@ mlx5_dev_close(struct rte_eth_dev *dev) /* In case mlx5_dev_stop() has not been called. */ mlx5_dev_interrupt_handler_uninstall(dev); mlx5_traffic_disable(dev); + mlx5_flow_flush(dev, NULL); /* Prevent crashes when queues are still in use. */ dev->rx_pkt_burst = removed_rx_burst; dev->tx_pkt_burst = removed_tx_burst; diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 5d3bc183d..bc8c5de33 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -35,11 +35,50 @@ extern const struct eth_dev_ops mlx5_dev_ops; extern const struct eth_dev_ops mlx5_dev_ops_isolate; +/* Pattern Layer bits. */ +#define MLX5_FLOW_LAYER_OUTER_L2 (1u << 0) +#define MLX5_FLOW_LAYER_OUTER_L3_IPV4 (1u << 1) +#define MLX5_FLOW_LAYER_OUTER_L3_IPV6 (1u << 2) +#define MLX5_FLOW_LAYER_OUTER_L4_UDP (1u << 3) +#define MLX5_FLOW_LAYER_OUTER_L4_TCP (1u << 4) +#define MLX5_FLOW_LAYER_OUTER_VLAN (1u << 5) +/* Masks. */ +#define MLX5_FLOW_LAYER_OUTER_L3 \ + (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6) +#define MLX5_FLOW_LAYER_OUTER_L4 \ + (MLX5_FLOW_LAYER_OUTER_L4_UDP | MLX5_FLOW_LAYER_OUTER_L4_TCP) + +/* Action fate on the packet. */ +#define MLX5_FLOW_FATE_DROP (1u << 0) + +/** Handles information leading to a drop fate. */ +struct mlx5_flow_verbs { + unsigned int size; /**< Size of the attribute. */ + struct { + struct ibv_flow_attr *attr; + /**< Pointer to the Specification buffer. */ + uint8_t *specs; /**< Pointer to the specifications. */ + }; + struct ibv_flow *flow; /**< Verbs flow pointer. */ + struct mlx5_hrxq *hrxq; /**< Hash Rx queue object. */ +}; + +/* Flow structure. */ struct rte_flow { TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ + struct rte_flow_attr attributes; /**< User flow attribute. */ + uint32_t layers; + /**< Bit-fields of present layers see MLX5_FLOW_ITEMS_*. */ + uint32_t fate; + /**< Bit-fields of present fate see MLX5_FLOW_FATE_*. */ + struct mlx5_flow_verbs verbs; /* Verbs drop flow. */ }; static const struct rte_flow_ops mlx5_flow_ops = { + .validate = mlx5_flow_validate, + .create = mlx5_flow_create, + .destroy = mlx5_flow_destroy, + .flush = mlx5_flow_flush, .isolate = mlx5_flow_isolate, }; @@ -128,12 +167,418 @@ mlx5_flow_priorities(struct rte_eth_dev *dev) } /** - * Convert a flow. + * Flow debug purpose function only available when + * CONFIG_RTE_LIBRTE_MLX5_DEBUG=y + * + * @param flow + * Pointer to the flow structure to display. + */ +void +mlx5_flow_print(struct rte_flow *flow __rte_unused) +{ +#ifndef NDEBUG + fprintf(stdout, "---------8<------------\n"); + fprintf(stdout, "%s: flow information\n", MLX5_DRIVER_NAME); + fprintf(stdout, " attributes: group %u priority %u ingress %d egress %d" + " transfer %d\n", flow->attributes.group, + flow->attributes.priority, + flow->attributes.ingress, + flow->attributes.egress, + flow->attributes.transfer); + fprintf(stdout, " layers: %s/%s/%s\n", + flow->layers & MLX5_FLOW_LAYER_OUTER_L2 ? "l2" : "-", + flow->layers & MLX5_FLOW_LAYER_OUTER_L3 ? "l3" : "-", + flow->layers & MLX5_FLOW_LAYER_OUTER_L4 ? "l4" : "-"); + if (flow->fate & MLX5_FLOW_FATE_DROP) + fprintf(stdout, " fate: drop queue\n"); + if (flow->verbs.attr) { + struct ibv_spec_header *hdr = + (struct ibv_spec_header *)flow->verbs.specs; + const int n = flow->verbs.attr->num_of_specs; + int i; + + fprintf(stdout, " Verbs attributes: specs_n %u\n", + flow->verbs.attr->num_of_specs); + for (i = 0; i != n; ++i) { + rte_hexdump(stdout, " ", hdr, hdr->size); + hdr = (struct ibv_spec_header *) + ((uint8_t *)hdr + hdr->size); + } + } + fprintf(stdout, "--------->8------------\n"); +#endif +} + +/** + * Validate Attributes provided by the user. * * @param dev * Pointer to Ethernet device. - * @param list - * Pointer to a TAILQ flow list. + * @param attr + * Pointer to flow attributes + * @param flow + * Pointer to the rte_flow structure. + * @param error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_attributes(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + struct rte_flow *flow, struct rte_flow_error *error) +{ + uint32_t priority_max = + ((struct priv *)dev->data->dev_private)->config.flow_prio; + + if (attr->group) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_GROUP, + NULL, + "groups are not supported"); + if (attr->priority >= priority_max) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, + NULL, + "priority value is not supported"); + if (attr->egress) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, + NULL, + "egress is not supported"); + if (attr->transfer) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER, + NULL, + "transfer is not supported"); + if (!attr->ingress) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ATTR_INGRESS, + NULL, + "only ingress is supported"); + flow->attributes = *attr; + return 0; +} + +/** + * Check support for a given item. + * + * @param item[in] + * Item specification. + * @param default_mask[in] + * Bit-masks covering supported fields to compare with spec, last and mask in + * \item. + * @param nic_mask[in] + * Bit-masks covering supported fields by the NIC to compare with user mask. + * @param size + * Bit-Mask size in bytes. + * @param error[out] + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_validate(const struct rte_flow_item *item, + const uint8_t *default_mask, + const uint8_t *nic_mask, + unsigned int size, + struct rte_flow_error *error) +{ + const uint8_t *mask = item->mask ? item->mask : default_mask; + unsigned int i; + + assert(nic_mask); + for (i = 0; i < size; ++i) + if ((nic_mask[i] | mask[i]) != nic_mask[i]) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "mask enables non supported" + " bits"); + if (!item->spec && (item->mask || item->last)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "mask/last without a spec is not" + " supported"); + if (item->spec && item->last) { + uint8_t spec[size]; + uint8_t last[size]; + unsigned int i; + int ret; + + for (i = 0; i < size; ++i) { + spec[i] = ((const uint8_t *)item->spec)[i] & mask[i]; + last[i] = ((const uint8_t *)item->last)[i] & mask[i]; + } + ret = memcmp(spec, last, size); + if (ret != 0) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "spec and last different" + " is not supported"); + } + return 0; +} + +/** + * Add a verbs specification. + * + * @param flow + * Pointer to flow structure. + * @param src + * Create specification. + * @param size + * Size in bytes of the specification to copy. + */ +static void +mlx5_flow_spec_verbs_add(struct rte_flow *flow, void *src, unsigned int size) +{ + if (flow->verbs.specs) { + void *dst; + + dst = (void *)(flow->verbs.specs + flow->verbs.size); + memcpy(dst, src, size); + ++flow->verbs.attr->num_of_specs; + } + flow->verbs.size += size; +} + +/** + * Validate Ethernet layer and possibly create the Verbs specification. + * + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size of the buffer to store the specification. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_eth *spec = item->spec; + const struct rte_flow_item_eth *mask = item->mask; + const struct rte_flow_item_eth nic_mask = { + .dst.addr_bytes = "\xff\xff\xff\xff\xff\xff", + .src.addr_bytes = "\xff\xff\xff\xff\xff\xff", + .type = RTE_BE16(0xffff), + }; + const unsigned int size = sizeof(struct ibv_flow_spec_eth); + struct ibv_flow_spec_eth eth = { + .type = IBV_FLOW_SPEC_ETH, + .size = size, + }; + int ret; + + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L2) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L2 layers already configured"); + if (!mask) + mask = &rte_flow_item_eth_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_eth), + error); + if (ret) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L2; + if (size > flow_size) + return size; + if (spec) { + unsigned int i; + + memcpy(ð.val.dst_mac, spec->dst.addr_bytes, ETHER_ADDR_LEN); + memcpy(ð.val.src_mac, spec->src.addr_bytes, ETHER_ADDR_LEN); + eth.val.ether_type = spec->type; + memcpy(ð.mask.dst_mac, mask->dst.addr_bytes, ETHER_ADDR_LEN); + memcpy(ð.mask.src_mac, mask->src.addr_bytes, ETHER_ADDR_LEN); + eth.mask.ether_type = mask->type; + /* Remove unwanted bits from values. */ + for (i = 0; i < ETHER_ADDR_LEN; ++i) { + eth.val.dst_mac[i] &= eth.mask.dst_mac[i]; + eth.val.src_mac[i] &= eth.mask.src_mac[i]; + } + eth.val.ether_type &= eth.mask.ether_type; + } + mlx5_flow_spec_verbs_add(flow, ð, size); + return size; +} + +/** + * Validate items provided by the user. + * + * @param items + * Pointer to flow items array. + * @param flow + * Pointer to the rte_flow structure. + * @param flow_size + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_items(const struct rte_flow_item items[], + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + int remain = flow_size; + size_t size = 0; + + for (; items->type != RTE_FLOW_ITEM_TYPE_END; items++) { + int ret = 0; + + switch (items->type) { + case RTE_FLOW_ITEM_TYPE_VOID: + break; + case RTE_FLOW_ITEM_TYPE_ETH: + ret = mlx5_flow_item_eth(items, flow, remain, error); + break; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + items, + "item not supported"); + } + if (ret < 0) + return ret; + if (remain > ret) + remain -= ret; + else + remain = 0; + size += ret; + } + if (!flow->layers) { + const struct rte_flow_item item = { + .type = RTE_FLOW_ITEM_TYPE_ETH, + }; + + return mlx5_flow_item_eth(&item, flow, flow_size, error); + } + return size; +} + +/** + * Validate action drop provided by the user. + * + * @param actions + * Pointer to flow actions array. + * @param flow + * Pointer to the rte_flow structure. + * @param flow_size + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_action_drop(const struct rte_flow_action *actions, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + unsigned int size = sizeof(struct ibv_flow_spec_action_drop); + struct ibv_flow_spec_action_drop drop = { + .type = IBV_FLOW_SPEC_ACTION_DROP, + .size = size, + }; + + if (flow->fate) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "multiple fate actions are not" + " supported"); + if (size < flow_size) + mlx5_flow_spec_verbs_add(flow, &drop, size); + flow->fate |= MLX5_FLOW_FATE_DROP; + return size; +} + +/** + * Validate actions provided by the user. + * + * @param dev + * Pointer to Ethernet device structure. + * @param actions + * Pointer to flow actions array. + * @param flow + * Pointer to the rte_flow structure. + * @param flow_size + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_actions(struct rte_eth_dev *dev __rte_unused, + const struct rte_flow_action actions[], + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + size_t size = 0; + int remain = flow_size; + int ret = 0; + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_VOID: + break; + case RTE_FLOW_ACTION_TYPE_DROP: + ret = mlx5_flow_action_drop(actions, flow, remain, + error); + break; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "action not supported"); + } + if (ret < 0) + return ret; + if (remain > ret) + remain -= ret; + else + remain = 0; + size += ret; + } + if (!flow->fate) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "no fate action found"); + return size; +} + +/** + * Validate the rule and return a flow structure filled accordingly. + * + * @param dev + * Pointer to Ethernet device. + * @param flow + * Pointer an flow structure. + * @param flow_size + * Size of the allocated space to store the flow information. * @param[in] attr * Flow rule attributes. * @param[in] pattern @@ -144,21 +589,39 @@ mlx5_flow_priorities(struct rte_eth_dev *dev) * Perform verbose error reporting if not NULL. * * @return - * A flow on success, NULL otherwise and rte_errno is set. + * The amount of bytes necessary to create the flow, a negative errno value + * otherwise and rte_errno is set. */ -static struct rte_flow * -mlx5_flow_list_create(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flows *list __rte_unused, - const struct rte_flow_attr *attr __rte_unused, - const struct rte_flow_item items[] __rte_unused, - const struct rte_flow_action actions[] __rte_unused, - struct rte_flow_error *error) +static int +mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, + const size_t flow_size, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "action not supported"); - return NULL; + struct rte_flow local_flow = { .layers = 0, }; + size_t size = sizeof(*flow) + sizeof(struct ibv_flow_attr); + int remain = (flow_size > size) ? flow_size - size : 0; + int ret; + + if (!remain) + flow = &local_flow; + ret = mlx5_flow_attributes(dev, attr, flow, error); + if (ret < 0) + return ret; + ret = mlx5_flow_items(items, flow, remain, error); + if (ret < 0) + return ret; + size += ret; + remain = (flow_size > size) ? flow_size - size : 0; + ret = mlx5_flow_actions(dev, actions, flow, remain, error); + if (ret < 0) + return ret; + size += ret; + if (size <= flow_size) + flow->verbs.attr->priority = flow->attributes.priority; + return size; } /** @@ -168,16 +631,142 @@ mlx5_flow_list_create(struct rte_eth_dev *dev __rte_unused, * @see rte_flow_ops */ int -mlx5_flow_validate(struct rte_eth_dev *dev __rte_unused, - const struct rte_flow_attr *attr __rte_unused, - const struct rte_flow_item items[] __rte_unused, - const struct rte_flow_action actions[] __rte_unused, +mlx5_flow_validate(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], struct rte_flow_error *error) { - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "action not supported"); + int ret = mlx5_flow_merge(dev, NULL, 0, attr, items, actions, error); + + if (ret < 0) + return ret; + return 0; +} + +/** + * Remove the flow. + * + * @param dev + * Pointer to Ethernet device. + * @param flow + * Pointer to flow structure. + */ +static void +mlx5_flow_fate_remove(struct rte_eth_dev *dev, struct rte_flow *flow) +{ + if (flow->fate & MLX5_FLOW_FATE_DROP) { + if (flow->verbs.flow) { + claim_zero(mlx5_glue->destroy_flow(flow->verbs.flow)); + flow->verbs.flow = NULL; + } + } + if (flow->verbs.hrxq) { + mlx5_hrxq_drop_release(dev, flow->verbs.hrxq); + flow->verbs.hrxq = NULL; + } +} + +/** + * Apply the flow. + * + * @param dev + * Pointer to Ethernet device structure. + * @param flow + * Pointer to flow structure. + * @param error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_fate_apply(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error) +{ + flow->verbs.hrxq = mlx5_hrxq_drop_new(dev); + if (!flow->verbs.hrxq) + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot allocate Drop queue"); + flow->verbs.flow = + mlx5_glue->create_flow(flow->verbs.hrxq->qp, flow->verbs.attr); + if (!flow->verbs.flow) { + mlx5_hrxq_drop_release(dev, flow->verbs.hrxq); + flow->verbs.hrxq = NULL; + return rte_flow_error_set(error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "kernel module refuses to create" + " flow"); + } + return 0; +} + +/** + * Create a flow. + * + * @param dev + * Pointer to Ethernet device. + * @param list + * Pointer to a TAILQ flow list. + * @param[in] attr + * Flow rule attributes. + * @param[in] items + * Pattern specification (list terminated by the END pattern item). + * @param[in] actions + * Associated actions (list terminated by the END action). + * @param[out] error + * Perform verbose error reporting if not NULL. + * + * @return + * A flow on success, NULL otherwise and rte_errno is set. + */ +static struct rte_flow * +mlx5_flow_list_create(struct rte_eth_dev *dev, + struct mlx5_flows *list, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct rte_flow *flow; + size_t size; + int ret; + + ret = mlx5_flow_merge(dev, NULL, 0, attr, items, actions, error); + if (ret < 0) + return NULL; + size = ret; + flow = rte_zmalloc(__func__, size, 0); + if (!flow) { + rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot allocate memory"); + return NULL; + } + flow->verbs.attr = (struct ibv_flow_attr *)(flow + 1); + flow->verbs.specs = (uint8_t *)(flow->verbs.attr + 1); + ret = mlx5_flow_merge(dev, flow, size, attr, items, actions, error); + if (ret < 0) + goto error; + assert((size_t)ret == size); + if (dev->data->dev_started) { + ret = mlx5_flow_fate_apply(dev, flow, error); + if (ret < 0) + goto error; + } + TAILQ_INSERT_TAIL(list, flow, next); + return flow; +error: + ret = rte_errno; /* Save rte_errno before cleanup. */ + mlx5_flow_fate_remove(dev, flow); + rte_free(flow); + rte_errno = ret; /* Restore rte_errno. */ + return NULL; } /** @@ -187,17 +776,15 @@ mlx5_flow_validate(struct rte_eth_dev *dev __rte_unused, * @see rte_flow_ops */ struct rte_flow * -mlx5_flow_create(struct rte_eth_dev *dev __rte_unused, - const struct rte_flow_attr *attr __rte_unused, - const struct rte_flow_item items[] __rte_unused, - const struct rte_flow_action actions[] __rte_unused, +mlx5_flow_create(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], struct rte_flow_error *error) { - rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "action not supported"); - return NULL; + return mlx5_flow_list_create + (dev, &((struct priv *)dev->data->dev_private)->flows, + attr, items, actions, error); } /** @@ -211,10 +798,12 @@ mlx5_flow_create(struct rte_eth_dev *dev __rte_unused, * Flow to destroy. */ static void -mlx5_flow_list_destroy(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flows *list __rte_unused, - struct rte_flow *flow __rte_unused) +mlx5_flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, + struct rte_flow *flow) { + mlx5_flow_fate_remove(dev, flow); + TAILQ_REMOVE(list, flow, next); + rte_free(flow); } /** @@ -422,9 +1011,8 @@ int mlx5_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error __rte_unused) { - struct priv *priv = dev->data->dev_private; - - mlx5_flow_list_flush(dev, &priv->flows); + mlx5_flow_list_flush(dev, + &((struct priv *)dev->data->dev_private)->flows); return 0; } @@ -737,9 +1325,8 @@ mlx5_fdir_filter_update(struct rte_eth_dev *dev, static void mlx5_fdir_filter_flush(struct rte_eth_dev *dev) { - struct priv *priv = dev->data->dev_private; - - mlx5_flow_list_flush(dev, &priv->flows); + mlx5_flow_list_flush(dev, + &((struct priv *)dev->data->dev_private)->flows); } /** From patchwork Wed Jun 27 15:07:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41679 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 48E421BFC2; Wed, 27 Jun 2018 17:07:46 +0200 (CEST) Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by dpdk.org (Postfix) with ESMTP id F2B7C1BFA3 for ; Wed, 27 Jun 2018 17:07:38 +0200 (CEST) Received: by mail-wm0-f68.google.com with SMTP id p11-v6so5849144wmc.4 for ; Wed, 27 Jun 2018 08:07:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=frA28TyoYiIm21r8XOA6BHu/xZcgzKPFfM+d6AlYP8I=; b=oO4ZXcYOGLIey7ifTKCH9frO4nG53IBcNxHn2eatbimh7lpBBI1r2271wjI3SnQiN5 xNvcwg/OlBFOs9kt5SyesdnLbl1c7uOFzB2/TOVZGLwRKN8vg0NZvPQTKmB4Y7oYmHXi jdHskBeMvOVRNy/mWaFHNBf3zNrcUa3fXPJHyoAnKhE8lU14lVqGYPSKoP5pPndQZgsM B5oHLXqZeBeFuTfsMuzQ+bd0WMdZGsTkPtvBsNPlznerVSiGEF5nCnY7XOh1bIK7OVj2 flLsrPN1i+9xhg4oxlxYsPoPq1vc+41h5963D1d7vp4A064M+AnnF7qRLPcqG8ubBwwq GxQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=frA28TyoYiIm21r8XOA6BHu/xZcgzKPFfM+d6AlYP8I=; b=AehmGynuli3yqIkQT68ZJYJphJVe7yEik3jbS6QBLJX0K7C1f4/Q12xlq0hHLgydWC J+jNvl9StHQO4BKO4BlJ8m8ccynKXGPQG0K+50I/h+hSWaV9yLPVRfJh1O4nowWQn8sH 96PcXd5MWjrKK/7WVJyE2GJWt/7ea9HsKLRvdNJuuYPoHyuTC/m+uRm5JUKHFwZvaAR7 hSbhltzcC6I/yYTeNovqBudZQeiqx2Rvi/zK1Mqf5YmZRGZB8LrGl7op3l7a4bemZ7UQ cRtHpLhiWle+oGMd2HJr3sVdSB1Ov1V8uyiNLLUtRcJKNQ0Rh9ucQPZgvOt8EQsOe5GO JPag== X-Gm-Message-State: APt69E3SUFBo2N45zsPa2IcKJR2rcjeStnWKf5BRXNdeT78nuMPaGqyw xqLQnBWzPpvGva3QJkwpY/APrYjPkg== X-Google-Smtp-Source: AAOMgped7+DDyI5f+yP1hD6q2iOPd2f0XMF4SwJBEfaurtmKhZvCNQm8puL+WTg2RglSr1pTly7XMw== X-Received: by 2002:a1c:ea06:: with SMTP id i6-v6mr4986682wmh.144.1530112058427; Wed, 27 Jun 2018 08:07:38 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:37 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:37 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 05/20] net/mlx5: add flow queue action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 100 +++++++++++++++++++++++++++++++---- 1 file changed, 89 insertions(+), 11 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index bc8c5de33..7ac1ba24c 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -50,6 +50,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Action fate on the packet. */ #define MLX5_FLOW_FATE_DROP (1u << 0) +#define MLX5_FLOW_FATE_QUEUE (1u << 1) /** Handles information leading to a drop fate. */ struct mlx5_flow_verbs { @@ -71,7 +72,8 @@ struct rte_flow { /**< Bit-fields of present layers see MLX5_FLOW_ITEMS_*. */ uint32_t fate; /**< Bit-fields of present fate see MLX5_FLOW_FATE_*. */ - struct mlx5_flow_verbs verbs; /* Verbs drop flow. */ + struct mlx5_flow_verbs verbs; /* Verbs flow. */ + uint16_t queue; /**< Destination queue to redirect traffic to. */ }; static const struct rte_flow_ops mlx5_flow_ops = { @@ -191,6 +193,8 @@ mlx5_flow_print(struct rte_flow *flow __rte_unused) flow->layers & MLX5_FLOW_LAYER_OUTER_L4 ? "l4" : "-"); if (flow->fate & MLX5_FLOW_FATE_DROP) fprintf(stdout, " fate: drop queue\n"); + else if (flow->fate & MLX5_FLOW_FATE_QUEUE) + fprintf(stdout, " fate: target queue %u\n", flow->queue); if (flow->verbs.attr) { struct ibv_spec_header *hdr = (struct ibv_spec_header *)flow->verbs.specs; @@ -512,6 +516,53 @@ mlx5_flow_action_drop(const struct rte_flow_action *actions, return size; } +/** + * Validate action queue provided by the user. + * + * @param dev + * Pointer to Ethernet device structure. + * @param actions + * Pointer to flow actions array. + * @param flow + * Pointer to the rte_flow structure. + * @param flow_size + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_action_queue(struct rte_eth_dev *dev, + const struct rte_flow_action *actions, + struct rte_flow *flow, + struct rte_flow_error *error) +{ + struct priv *priv = dev->data->dev_private; + const struct rte_flow_action_queue *queue = actions->conf; + + if (flow->fate) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "multiple fate actions are not" + " supported"); + if (queue->index >= priv->rxqs_n) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &queue->index, + "queue index out of range"); + if (!(*priv->rxqs)[queue->index]) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &queue->index, + "queue is not configured"); + flow->queue = queue->index; + flow->fate |= MLX5_FLOW_FATE_QUEUE; + return 0; +} + /** * Validate actions provided by the user. * @@ -531,7 +582,7 @@ mlx5_flow_action_drop(const struct rte_flow_action *actions, * otherwise and rte_errno is set. */ static int -mlx5_flow_actions(struct rte_eth_dev *dev __rte_unused, +mlx5_flow_actions(struct rte_eth_dev *dev, const struct rte_flow_action actions[], struct rte_flow *flow, const size_t flow_size, struct rte_flow_error *error) @@ -548,6 +599,9 @@ mlx5_flow_actions(struct rte_eth_dev *dev __rte_unused, ret = mlx5_flow_action_drop(actions, flow, remain, error); break; + case RTE_FLOW_ACTION_TYPE_QUEUE: + ret = mlx5_flow_action_queue(dev, actions, flow, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -662,7 +716,10 @@ mlx5_flow_fate_remove(struct rte_eth_dev *dev, struct rte_flow *flow) } } if (flow->verbs.hrxq) { - mlx5_hrxq_drop_release(dev, flow->verbs.hrxq); + if (flow->fate & MLX5_FLOW_FATE_DROP) + mlx5_hrxq_drop_release(dev, flow->verbs.hrxq); + else if (flow->fate & MLX5_FLOW_FATE_QUEUE) + mlx5_hrxq_release(dev, flow->verbs.hrxq); flow->verbs.hrxq = NULL; } } @@ -684,17 +741,38 @@ static int mlx5_flow_fate_apply(struct rte_eth_dev *dev, struct rte_flow *flow, struct rte_flow_error *error) { - flow->verbs.hrxq = mlx5_hrxq_drop_new(dev); - if (!flow->verbs.hrxq) - return rte_flow_error_set - (error, errno, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot allocate Drop queue"); + if (flow->fate & MLX5_FLOW_FATE_DROP) { + flow->verbs.hrxq = mlx5_hrxq_drop_new(dev); + if (!flow->verbs.hrxq) + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot allocate Drop queue"); + } else if (flow->fate & MLX5_FLOW_FATE_QUEUE) { + struct mlx5_hrxq *hrxq; + + hrxq = mlx5_hrxq_get(dev, rss_hash_default_key, + rss_hash_default_key_len, 0, + &flow->queue, 1, 0, 0); + if (!hrxq) + hrxq = mlx5_hrxq_new(dev, rss_hash_default_key, + rss_hash_default_key_len, 0, + &flow->queue, 1, 0, 0); + if (!hrxq) + return rte_flow_error_set(error, rte_errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot create flow"); + flow->verbs.hrxq = hrxq; + } flow->verbs.flow = mlx5_glue->create_flow(flow->verbs.hrxq->qp, flow->verbs.attr); if (!flow->verbs.flow) { - mlx5_hrxq_drop_release(dev, flow->verbs.hrxq); + if (flow->fate & MLX5_FLOW_FATE_DROP) + mlx5_hrxq_drop_release(dev, flow->verbs.hrxq); + else + mlx5_hrxq_release(dev, flow->verbs.hrxq); flow->verbs.hrxq = NULL; return rte_flow_error_set(error, errno, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, From patchwork Wed Jun 27 15:07:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41680 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 836141BFC7; Wed, 27 Jun 2018 17:07:48 +0200 (CEST) Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id B179D1BFA1 for ; Wed, 27 Jun 2018 17:07:39 +0200 (CEST) Received: by mail-wm0-f67.google.com with SMTP id w137-v6so6257526wmw.1 for ; Wed, 27 Jun 2018 08:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=QVugnlaUz5gNmRQ7wgMsd5fqAl07L5rQwDZ3gREIjCE=; b=Ej2shz/pplololityJYj1xLa1d8e5mVLdx2R1mKLbFYT59/G79Yx//XYlLjqYyLboC WwR/G2uqWGlR/HpO4Y1FOkitS/f3PAbTB4bcB69sqHnfJlO418WAwHc8mGINwb3qTRdR Jj0aGxHh3lTZPSUFQY2eA8Z+vkShLRo7o1xhplfYs5RQ89QFZR1x6w3lGbq7sbYw8cb9 Dmtf7HvIl+XqgAvovwzCaMRrMN3NuOZ2xOyc/66CIuoG34UqBiP6QQTs7sLSedeYZ+td lPAQXvHPSqXBrGj64MKktoVWBk/gMFOzkN4goS+pcsNOVMwImc9DmYTWUBuKDFOf4OPg rz7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=QVugnlaUz5gNmRQ7wgMsd5fqAl07L5rQwDZ3gREIjCE=; b=NMBTfuKKu9h0QEteiLkj15oAje89BUVjwfoU9y0BgV+DN9xv2NwKHgQ4LYF2pJFhuh 6iIfigd8Sax2TtnJns0EB5M5bsbVtlx+HoG9D+v9+zb97/eeKaIJebSs4b8GOETPgHFR qpLP7/hpFLy0mKdPmIzUky/FJuDQYgIMHUvOYrEWlIffE8PakrcfZawRAOj3J3/Drm2V 9QuQLrHw8iwF57a/5bqJkvUqT/K9pYKRbCQI4kLZpDkJJiVSSaJp2yZtdYS8KmEm+HR0 NSSL18urlQu4VYDzemDWblktNMHK1grazQjSu50Xj/M7h7r1D5/KMazLIpSwesK81euF vMcQ== X-Gm-Message-State: APt69E0HugLM/uS2g4yC8K4MFkO09PJBDAl0jS7elJSHs23koMmfQnNb uKdomUmfFAz/b4PLU557qyBQhLdK3A== X-Google-Smtp-Source: AAOMgpfkovwNvvKvgZLXl+u1bl2OBdr9uEEWzoepkVD1aqtaDSN3xDifSQ2S4oehnTBWuIt4nbCbnQ== X-Received: by 2002:a1c:ad09:: with SMTP id w9-v6mr5670507wme.95.1530112059261; Wed, 27 Jun 2018 08:07:39 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:38 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:38 +0200 Message-Id: <71da0cfabd6f56f7a5456cba33c67cbe2fbc46f4.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 06/20] net/mlx5: add flow stop/start X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 7ac1ba24c..6593eafa0 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -912,9 +912,12 @@ mlx5_flow_list_flush(struct rte_eth_dev *dev, struct mlx5_flows *list) * Pointer to a TAILQ flow list. */ void -mlx5_flow_stop(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flows *list __rte_unused) +mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) { + struct rte_flow *flow; + + TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) + mlx5_flow_fate_remove(dev, flow); } /** @@ -929,10 +932,23 @@ mlx5_flow_stop(struct rte_eth_dev *dev __rte_unused, * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_start(struct rte_eth_dev *dev __rte_unused, - struct mlx5_flows *list __rte_unused) +mlx5_flow_start(struct rte_eth_dev *dev, struct mlx5_flows *list) { + struct rte_flow *flow; + struct rte_flow_error error; + int ret = 0; + + TAILQ_FOREACH(flow, list, next) { + ret = mlx5_flow_fate_apply(dev, flow, &error); + if (ret < 0) + goto error; + } return 0; +error: + ret = rte_errno; /* Save rte_errno before cleanup. */ + mlx5_flow_stop(dev, list); + rte_errno = ret; /* Restore rte_errno. */ + return -rte_errno; } /** From patchwork Wed Jun 27 15:07:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41681 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A002B1BFCE; Wed, 27 Jun 2018 17:07:50 +0200 (CEST) Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id A11661BFA1 for ; Wed, 27 Jun 2018 17:07:40 +0200 (CEST) Received: by mail-wm0-f67.google.com with SMTP id w137-v6so6257578wmw.1 for ; Wed, 27 Jun 2018 08:07:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=+5aEi82fusxWPB2LYF+Ru3mmCLox0k2PwawO5KC544Q=; b=KlHpyzYHfd6pVGhlnRXTk8lLp0zEXRrWDJPuNHwrjNzMXOzIMEm5GE8l2iiEYYYyXk LMhxBF2tBVr69tGc0G5igsSJlR5RiMnKaQnJzS64GIPClASXSDXukALHQi+e9wY3idM4 TLrNPoXbTKfLNSwqkYo+P9gKyXAbvPEOO8kQr5MyDYu16blJxlQpKR7BgAtAeyblVIcw LshGHwriv9lzqiLilbK9VpHLt8HVY9HPZRNRlZiTE5siv8DawJjxpxc9jNJk3oBiVk4r B1Pq/8Mcq/lxkDmK0t2fL3hTPPSw9VzXpqE+3wYI4oeuolCgyHcKZo9sJ+bBwpUuGQtg kpMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=+5aEi82fusxWPB2LYF+Ru3mmCLox0k2PwawO5KC544Q=; b=lcgBQW7tONbB5NRQFYejbwc+oGlHuNUpKvGdEIWQvMNDIVKitsd01Xq38Vmvnl/9sf aTQ5U4Pr3nG+Oy5BWnIVF/9nOQXZGjIq5riMZzjpcRpsqxcyhe8xDYQUsGyVfDCc/2OH Gh4xzwDC+u09ivBpUOr+SlStzBTliBHPwfS7kJfGcDMaxd49DTCNyXD+5QABYt68dP0P m9Gif1PbNAqnwf/NIHsTFsKbP/QpR0UKyOjm9FN1Yqq0yizmwROSbcYolAC81KTjVTXL GWFZnvfplcyF2z+b3iXXOa8+7eyurEb8pkog3T09uKR6zz6cEdF3t/0xlfynzyHS6Wja GZyw== X-Gm-Message-State: APt69E1vxG73EebKi3oHyvMxmecZO2t8MXYUJ+NZtLc/MSNJeuPXuUXb gJ6YmuIBR5fZNk6QPQU30qy7D1L5Bw== X-Google-Smtp-Source: AAOMgpcnm0xUQ3UnmOCJDjblllTBrPFDxvUFD24rrHPHrXVaPVveu5ETLtSz9d53Di8Y0glchdCQyQ== X-Received: by 2002:a1c:a88a:: with SMTP id r132-v6mr3632782wme.31.1530112060143; Wed, 27 Jun 2018 08:07:40 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:39 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:39 +0200 Message-Id: <39544c46ba7237c339e89f10480c3e24a3810754.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 07/20] net/mlx5: add flow VLAN item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 123 +++++++++++++++++++++++++++++++++++ 1 file changed, 123 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 6593eafa0..6a576ddd9 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -420,6 +420,126 @@ mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Update the VLAN tag in the Ethernet spec. + * + * @param attr[in, out] + * Pointer to Verbs attributes structure. + * @param eth[in] + * Verbs structure containing the VLAN information to copy. + */ +static void +mlx5_flow_item_vlan_update(struct ibv_flow_attr *attr, + struct ibv_flow_spec_eth *eth) +{ + unsigned int i; + enum ibv_flow_spec_type search = IBV_FLOW_SPEC_ETH; + struct ibv_spec_header *hdr = (struct ibv_spec_header *) + ((uint8_t *)attr + sizeof(struct ibv_flow_attr)); + + for (i = 0; i != attr->num_of_specs; ++i) { + if (hdr->type == search) { + struct ibv_flow_spec_eth *e = + (struct ibv_flow_spec_eth *)hdr; + + e->val.vlan_tag = eth->val.vlan_tag; + e->mask.vlan_tag = eth->mask.vlan_tag; + break; + } + hdr = (struct ibv_spec_header *)((uint8_t *)hdr + hdr->size); + } +} + +/** + * Validate VLAN layer and possibly create/modify the Verbs specification. + * + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size of the buffer to store the specification. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_vlan *spec = item->spec; + const struct rte_flow_item_vlan *mask = item->mask; + const struct rte_flow_item_vlan nic_mask = { + .tci = RTE_BE16(0x0fff), + }; + unsigned int size = sizeof(struct ibv_flow_spec_eth); + struct ibv_flow_spec_eth eth = { + .type = IBV_FLOW_SPEC_ETH, + .size = size, + }; + int ret; + const uint32_t lm = MLX5_FLOW_LAYER_OUTER_L3 | + MLX5_FLOW_LAYER_OUTER_L4; + const uint32_t vlanm = MLX5_FLOW_LAYER_OUTER_VLAN; + const uint32_t l2m = MLX5_FLOW_LAYER_OUTER_L2; + + if (flow->layers & vlanm) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L2 layers already configured"); + else if ((flow->layers & lm) != 0) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L2 layer cannot follow L3/L4 layer"); + if (!mask) + mask = &rte_flow_item_vlan_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_vlan), error); + if (ret) + return ret; + if (spec) { + eth.val.vlan_tag = spec->tci; + eth.mask.vlan_tag = mask->tci; + eth.val.vlan_tag &= eth.mask.vlan_tag; + eth.val.ether_type = spec->inner_type; + eth.mask.ether_type = mask->inner_type; + eth.val.ether_type &= eth.mask.ether_type; + } + /* + * From verbs perspective an empty VLAN is equivalent + * to a packet without VLAN layer. + */ + if (!eth.mask.vlan_tag) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "VLAN cannot be empty"); + /* Outer TPID cannot be matched. */ + if (eth.mask.ether_type) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "VLAN TPID matching is not" + " supported"); + if (!(flow->layers & l2m)) { + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, ð, size); + } else { + if (flow->verbs.attr) + mlx5_flow_item_vlan_update(flow->verbs.attr, ð); + size = 0; /**< Only an update is done in eth specification. */ + } + flow->layers |= MLX5_FLOW_LAYER_OUTER_L2 | + MLX5_FLOW_LAYER_OUTER_VLAN; + return size; +} + /** * Validate items provided by the user. * @@ -453,6 +573,9 @@ mlx5_flow_items(const struct rte_flow_item items[], case RTE_FLOW_ITEM_TYPE_ETH: ret = mlx5_flow_item_eth(items, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_VLAN: + ret = mlx5_flow_item_vlan(items, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jun 27 15:07:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41682 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DFAF31BFCC; Wed, 27 Jun 2018 17:07:52 +0200 (CEST) Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by dpdk.org (Postfix) with ESMTP id F2B9F1BFA9 for ; Wed, 27 Jun 2018 17:07:41 +0200 (CEST) Received: by mail-wm0-f66.google.com with SMTP id p11-v6so5849319wmc.4 for ; Wed, 27 Jun 2018 08:07:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=+Ttfs57J/DFx6/1qOrmmY1w5o1u+vMJHbkJpO+YELK4=; b=ApGoYUihgBwZvmbN2nYRwaBwzGkz3EBxqoBMu4ONPNxbTImjjvf7+H7V1XNfThwUL+ ccvAYRK5msRjL7s7cx81Elj2t1NIAwdNYsW09tC3tc3P79PRfVXZ0Jlb0BQsN1dK9svB O6LTcCR6dpScHXhJpAv0qIyEJ/y+ZJM1UAaW1DJ/PQawvO5waHaKvFhKgBHkydL+KcJ9 WAFxFbS5iAUSBDX1Dl5eG5H88HtwMnfV0Cg8mMSxODHh54js8bVvQjuKzGRbDfe1Iypd z27Jpn7EMOIHks0BkZr28OXPCesspMrRgB3ULypfu5Wf5AwD+gLXss1IbNeKzC8b+rxs 4s+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=+Ttfs57J/DFx6/1qOrmmY1w5o1u+vMJHbkJpO+YELK4=; b=QPO3QTTygAcChSzWWjOiHval0ZIFFl4nE6SSb6POMCltm8n8nqClsnFYf3+bu6dqqh NOmR+r0NV85v9XDqoXLi3JE91Rx9k2vdoYhMXMmq6tZypD+0PaaPMQi85BMc4NFbA9R+ 2je2fcE+z4tTGEglcj7gOR4rnLdeShquMp1MzjyfIHxSZyHCzGXSHOWsERw8KSdA+/JX Wnsg90w7/ZZxX9XNhlXDDCjvVz1HPA/LQxsHq5fbvFoo7knG+O0TPeKne7xtj5IqnZYr 8qhfUTydSegtuppNyB3drVOriBM6ralqMc2WzWYeKRlQcfhUIsmzetkmGU17DCshpJmi UzwA== X-Gm-Message-State: APt69E057FXW+Rgg73a5xs6uNJeML5kbC/8H6poTnZ4sitGAef5bwftm Jax10bNKhC+ht08Z32VN9oCaXpk4uA== X-Google-Smtp-Source: AAOMgpedtk9euSnpS4mHWSItMYRQjRklWhnB9oFi3vWSDpJyFLg2dzbQ3zFCj+35+zORKGhulTkBlA== X-Received: by 2002:a1c:4291:: with SMTP id k17-v6mr5149609wmi.74.1530112061178; Wed, 27 Jun 2018 08:07:41 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:40 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:40 +0200 Message-Id: <7693b4973a8e683972895820d6db36aa7b12a404.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 08/20] net/mlx5: add flow IPv4 item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 83 ++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 6a576ddd9..8e7a0bb5a 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -540,6 +540,86 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Validate IPv4 layer and possibly create the Verbs specification. + * + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_ipv4 *spec = item->spec; + const struct rte_flow_item_ipv4 *mask = item->mask; + const struct rte_flow_item_ipv4 nic_mask = { + .hdr = { + .src_addr = RTE_BE32(0xffffffff), + .dst_addr = RTE_BE32(0xffffffff), + .type_of_service = 0xff, + .next_proto_id = 0xff, + }, + }; + unsigned int size = sizeof(struct ibv_flow_spec_ipv4_ext); + struct ibv_flow_spec_ipv4_ext ipv4 = { + .type = IBV_FLOW_SPEC_IPV4_EXT, + .size = size, + }; + int ret; + + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L3) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "multiple L3 layers not supported"); + else if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 cannot follow an L4 layer."); + if (!mask) + mask = &rte_flow_item_ipv4_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_ipv4), error); + if (ret < 0) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV4; + if (size > flow_size) + return size; + if (spec) { + ipv4.val = (struct ibv_flow_ipv4_ext_filter){ + .src_ip = spec->hdr.src_addr, + .dst_ip = spec->hdr.dst_addr, + .proto = spec->hdr.next_proto_id, + .tos = spec->hdr.type_of_service, + }; + ipv4.mask = (struct ibv_flow_ipv4_ext_filter){ + .src_ip = mask->hdr.src_addr, + .dst_ip = mask->hdr.dst_addr, + .proto = mask->hdr.next_proto_id, + .tos = mask->hdr.type_of_service, + }; + /* Remove unwanted bits from values. */ + ipv4.val.src_ip &= ipv4.mask.src_ip; + ipv4.val.dst_ip &= ipv4.mask.dst_ip; + ipv4.val.proto &= ipv4.mask.proto; + ipv4.val.tos &= ipv4.mask.tos; + } + mlx5_flow_spec_verbs_add(flow, &ipv4, size); + return size; +} + /** * Validate items provided by the user. * @@ -576,6 +656,9 @@ mlx5_flow_items(const struct rte_flow_item items[], case RTE_FLOW_ITEM_TYPE_VLAN: ret = mlx5_flow_item_vlan(items, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_IPV4: + ret = mlx5_flow_item_ipv4(items, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jun 27 15:07:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41683 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 344AA1BFD9; Wed, 27 Jun 2018 17:07:55 +0200 (CEST) Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by dpdk.org (Postfix) with ESMTP id 0DF321BFAF for ; Wed, 27 Jun 2018 17:07:43 +0200 (CEST) Received: by mail-wm0-f68.google.com with SMTP id e69-v6so3320740wme.2 for ; Wed, 27 Jun 2018 08:07:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=GFq7GfnL0me8dT1JBjk6Uu7HN9fJE+EdqLCBFZDmsak=; b=OaC8iOnri//XQ8pU2oFYs/7kz0cqyqirKpErs4lmV7of1IhQaIAE4IxdYKj2JM/6z1 gcyuWUN4u8aBs25hGZBdTqEnnW3UcFr7aGM77jb3gY6wNGSHnezDNsXNN5XZM4DI3Wyn B6xSwnp0cZbUVM/O+2vAamCPFYuSUb88BE5mx6i5nto0/dT7HaJn9EP4fv8m4C/8zpWy qeCCAeMUCKtZljhlD0Nmohw3FRthxgefIZh75HSfjkbFKt5G0wSJm84TGmzMmYSIhKPy XLhX5ZRPzv/JN9xVzXa/0l2elz7kj+Z4Nt/jTj98fEwfIhjB+QeEeVcicbbzxQQ2UxH3 D/zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=GFq7GfnL0me8dT1JBjk6Uu7HN9fJE+EdqLCBFZDmsak=; b=U/6jikUpiK+rLR9SPhyaUK2cdVyr0RA4+mVCG6/74MXUWn93IOpYIqgp28W3axmt8L v5IA+GhDNcBBhS4n35KeLaxYg/EZRfpFGuGbG6eWmFTToJGy3h4YzLNCIb6iK440w0Ug BgpmGgfvJMrZoWXFeSJ/ySrIIcJonMnbKXjF2Bi0E84fb5TBrBJl79xWYhtGEmzWxPyg QmUIHu47h5/0x7YBmRbeu4NuP9xAunVfKmuIVMOfhJvPVfAGca4Pv8Ed3ZQNwigAajrA 3ASBm3TAUaMjOYV76QDlYqmnsB19A6449Mz36NiS7nsFXrdoh+8YHYx4T9GyWHMmqeYn okPg== X-Gm-Message-State: APt69E2Ac584hVEZvhxY8RuUpU7CHRSUF6E4FbOOLew+Kg09Hr8yj6TV +1Kr7g6YyNMBR6zLLEHfal38hiHqvw== X-Google-Smtp-Source: AAOMgpf91yj0T/kcsbc2Um2DoB9BTtawMHAX9Xkb/1b1l3Vut/YlTsMrlVS06lXqJms4sSG4Nz/LMQ== X-Received: by 2002:a1c:944e:: with SMTP id w75-v6mr3048781wmd.105.1530112062574; Wed, 27 Jun 2018 08:07:42 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:41 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:41 +0200 Message-Id: <7bc5b0becafaeaf59c3e97a5ecdc955689a41bed.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 09/20] net/mlx5: add flow IPv6 item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 108 +++++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 8e7a0bb5a..4e018400a 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -620,6 +620,111 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Validate IPv6 layer and possibly create the Verbs specification. + * + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_ipv6 *spec = item->spec; + const struct rte_flow_item_ipv6 *mask = item->mask; + const struct rte_flow_item_ipv6 nic_mask = { + .hdr = { + .src_addr = + "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff", + .dst_addr = + "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff", + .vtc_flow = RTE_BE32(0xffffffff), + .proto = 0xff, + .hop_limits = 0xff, + }, + }; + unsigned int size = sizeof(struct ibv_flow_spec_ipv6); + struct ibv_flow_spec_ipv6 ipv6 = { + .type = IBV_FLOW_SPEC_IPV6, + .size = size, + }; + int ret; + + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L3) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "multiple L3 layers not supported"); + else if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 cannot follow an L4 layer."); + if (!mask) + mask = &rte_flow_item_ipv6_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_ipv6), error); + if (ret < 0) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV6; + if (size > flow_size) + return size; + if (spec) { + unsigned int i; + uint32_t vtc_flow_val; + uint32_t vtc_flow_mask; + + memcpy(&ipv6.val.src_ip, spec->hdr.src_addr, + RTE_DIM(ipv6.val.src_ip)); + memcpy(&ipv6.val.dst_ip, spec->hdr.dst_addr, + RTE_DIM(ipv6.val.dst_ip)); + memcpy(&ipv6.mask.src_ip, mask->hdr.src_addr, + RTE_DIM(ipv6.mask.src_ip)); + memcpy(&ipv6.mask.dst_ip, mask->hdr.dst_addr, + RTE_DIM(ipv6.mask.dst_ip)); + vtc_flow_val = rte_be_to_cpu_32(spec->hdr.vtc_flow); + vtc_flow_mask = rte_be_to_cpu_32(mask->hdr.vtc_flow); + ipv6.val.flow_label = + rte_cpu_to_be_32((vtc_flow_val & IPV6_HDR_FL_MASK) >> + IPV6_HDR_FL_SHIFT); + ipv6.val.traffic_class = (vtc_flow_val & IPV6_HDR_TC_MASK) >> + IPV6_HDR_TC_SHIFT; + ipv6.val.next_hdr = spec->hdr.proto; + ipv6.val.hop_limit = spec->hdr.hop_limits; + ipv6.mask.flow_label = + rte_cpu_to_be_32((vtc_flow_mask & IPV6_HDR_FL_MASK) >> + IPV6_HDR_FL_SHIFT); + ipv6.mask.traffic_class = (vtc_flow_mask & IPV6_HDR_TC_MASK) >> + IPV6_HDR_TC_SHIFT; + ipv6.mask.next_hdr = mask->hdr.proto; + ipv6.mask.hop_limit = mask->hdr.hop_limits; + /* Remove unwanted bits from values. */ + for (i = 0; i < RTE_DIM(ipv6.val.src_ip); ++i) { + ipv6.val.src_ip[i] &= ipv6.mask.src_ip[i]; + ipv6.val.dst_ip[i] &= ipv6.mask.dst_ip[i]; + } + ipv6.val.flow_label &= ipv6.mask.flow_label; + ipv6.val.traffic_class &= ipv6.mask.traffic_class; + ipv6.val.next_hdr &= ipv6.mask.next_hdr; + ipv6.val.hop_limit &= ipv6.mask.hop_limit; + } + mlx5_flow_spec_verbs_add(flow, &ipv6, size); + return size; +} + /** * Validate items provided by the user. * @@ -659,6 +764,9 @@ mlx5_flow_items(const struct rte_flow_item items[], case RTE_FLOW_ITEM_TYPE_IPV4: ret = mlx5_flow_item_ipv4(items, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_IPV6: + ret = mlx5_flow_item_ipv6(items, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jun 27 15:07:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41684 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BEFBF1BFDF; Wed, 27 Jun 2018 17:07:56 +0200 (CEST) Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by dpdk.org (Postfix) with ESMTP id CF1A51BFB0 for ; Wed, 27 Jun 2018 17:07:43 +0200 (CEST) Received: by mail-wm0-f66.google.com with SMTP id v16-v6so6258343wmv.5 for ; Wed, 27 Jun 2018 08:07:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=TrmUpaRcs/BmUCmQW9OU8vetXvfYA1JJPh7xJYIuHlI=; b=k5VhVdROHLF704R4o9HSK9vzSC9vWBzrp2eIFiCyPDwYttuEy2zbe2qg98Q4SQiEyA +dHtyAg/h9UWD5D4QHlVEzl91WjtioHWEinzjj+E9idPMcfL4x1kVkX17Hex/srnr/50 htCdaAeBeyqwmHsj0vw/6ig0mK7QvHBrLctLYv83VSnu0P9W97ZX89ZH7IqeupPM8hNu keVXJY2moT+dkmsn3YNPtMogOVGMWqH95oDsIeVF/FAH+X44xEWy28oo2K77eqVfQXEf 82rr/aROnwBS4SDOVhpLTrzIBbOCDeW/GHTn5od1wJepQ/JTPfPiqEXzF/IVKq0uIuhR 8baw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=TrmUpaRcs/BmUCmQW9OU8vetXvfYA1JJPh7xJYIuHlI=; b=EcsJEkxsgRGmTIaBCtSOI5psc6Z9sTuCLQDZHJ7awra691SF6jD04UX4x6hCNh78og Tlv2Rj+QnIzpZd//X4x4SxGJKguMolnNZdZMNhPIrjrIbRlsk7GTEV0+DpYZO4yIOoZb JI0uEg1yx5v22rjxd2K8JW4Lzur8lQtRpSJ1FkDCYcAjYz0NnNaEF5Czu8qJNPYL2MmF K1yBGirqPSvsVkE4j8/WV90axJzs5spGxtYxKkCMFVWYjWktN8wAVnLcXSwXv/N60lzc 2kofkAF9ptB0UgBefD6HMcXh2qMON35Stqxfb8170gIjoxL8ubMYNy05orI/LcL+Zr10 k8vw== X-Gm-Message-State: APt69E0pIgSZ3De02/3EsNuHdno7YSPawO3PUFRXe7OjTeeHPmPMADbw kzsUZkhB367w837HvxMqT/P/IJuJnA== X-Google-Smtp-Source: AAOMgpfmv5Xq75prgQDkyJIHbHC198uycVm1vFQIHPOuMOuinSuOcV7AHM/rOV07bnJ9XmK0Aes+1A== X-Received: by 2002:a1c:34c9:: with SMTP id b192-v6mr5478103wma.21.1530112063368; Wed, 27 Jun 2018 08:07:43 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:42 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:42 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 10/20] net/mlx5: add flow UDP item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 65 ++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 4e018400a..06b4b7f41 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -725,6 +725,68 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Validate UDP layer and possibly create the Verbs specification. + * + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_udp *spec = item->spec; + const struct rte_flow_item_udp *mask = item->mask; + unsigned int size = sizeof(struct ibv_flow_spec_tcp_udp); + struct ibv_flow_spec_tcp_udp udp = { + .type = IBV_FLOW_SPEC_UDP, + .size = size, + }; + int ret; + + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 is mandatory to filter on L4"); + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L4 layer is already present"); + if (!mask) + mask = &rte_flow_item_udp_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_udp_mask, + sizeof(struct rte_flow_item_udp), error); + if (ret < 0) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_UDP; + if (size > flow_size) + return size; + if (spec) { + udp.val.dst_port = spec->hdr.dst_port; + udp.val.src_port = spec->hdr.src_port; + udp.mask.dst_port = mask->hdr.dst_port; + udp.mask.src_port = mask->hdr.src_port; + /* Remove unwanted bits from values. */ + udp.val.src_port &= udp.mask.src_port; + udp.val.dst_port &= udp.mask.dst_port; + } + mlx5_flow_spec_verbs_add(flow, &udp, size); + return size; +} + /** * Validate items provided by the user. * @@ -767,6 +829,9 @@ mlx5_flow_items(const struct rte_flow_item items[], case RTE_FLOW_ITEM_TYPE_IPV6: ret = mlx5_flow_item_ipv6(items, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_UDP: + ret = mlx5_flow_item_udp(items, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jun 27 15:07:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41685 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5A18C1BFEB; Wed, 27 Jun 2018 17:07:59 +0200 (CEST) Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id A9F2B1BFBD for ; Wed, 27 Jun 2018 17:07:44 +0200 (CEST) Received: by mail-wm0-f67.google.com with SMTP id v16-v6so6258393wmv.5 for ; Wed, 27 Jun 2018 08:07:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=Frq/IvDC/kZVPBS4+K4m6e76igHSD27sGRakRKDd7dc=; b=beh8fQWkfTOjdgPy+J1QMkiZJweqzsez3+zYg3YRHjEqqrQFJfsEmTj3oKGq7MqxlD O62tynaK2BcSIyohvcICYLWg7p8IapHwhrHZyh65ZgpZbI8Dw/wwDtmuoio3wAy8Lqdo Lp6Z14X6lB4YWKOJnSnv6zIm3tky5ojFRx0Mafd7yI3J0uEdRTAQnJ3xvw7t4GoCQeck TdkUXVuJbdIJnFT/uBtCCMmhnYJOD5XDUtyQSfR5qOBxjhtlVVwcCo5TCE+bhkZTN8y5 RYhep1O7RiLdxdSLBomxZqf384muElGkdD7skC93hTR8J0EHxkXNJkhKvjr91j6h+FFq 1C5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Frq/IvDC/kZVPBS4+K4m6e76igHSD27sGRakRKDd7dc=; b=e7JVpPgAkiF3FjADxYVLc3QFng6an475CjKfMM//Kr35OZbrQyd3BlvJ+4wS0owwSv 32Qn2I7kLN/vUA6vdQtTV+e3/dNCSz8yaCVauTd8JJj7PoVvrL1q6XikybJ5ehnsAEja VX5tRb9+AfD7s2uk0rb03c9sPRmmEVWxQxW5/kgiOZpqqS3SC5x6hrESX+slDyMFBWov 6otSQUszsiOuJCeUPP4ZRGOim7drJhzLsITLT7nEK3O9Yzw/2d0PYnhlpWJDXiabUWiQ JIdu48zcnY63UOb9UM1ps4zA9K0xT4XaUDGpnmrLdPac6s+3Ay+ffUfTZ4TBNvlnwtwK paLA== X-Gm-Message-State: APt69E1xe40aqJc08ylb52RCPRzDOEj7n5PIM4adfIt7mzS/5A6MrSrw L4eQxZ8TPKPndBEV+V8TBZURAfzXAQ== X-Google-Smtp-Source: AAOMgpeg/p/WDC5nwrC8y6dJ5k0n+xLmhkUGeoEYLXipzQxNeeVFTLJXOtD3ssyhmzYr76fyyNWIjQ== X-Received: by 2002:a1c:4405:: with SMTP id r5-v6mr4984882wma.62.1530112064230; Wed, 27 Jun 2018 08:07:44 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:43 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:43 +0200 Message-Id: <2e0decb7243290b23f782e7e4d9a4f689d9afbd4.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 11/20] net/mlx5: add flow TCP item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 65 ++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 06b4b7f41..57f072c03 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -787,6 +787,68 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Validate TCP layer and possibly create the Verbs specification. + * + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_tcp *spec = item->spec; + const struct rte_flow_item_tcp *mask = item->mask; + unsigned int size = sizeof(struct ibv_flow_spec_tcp_udp); + struct ibv_flow_spec_tcp_udp tcp = { + .type = IBV_FLOW_SPEC_TCP, + .size = size, + }; + int ret; + + if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 is mandatory to filter on L4"); + if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L4 layer is already present"); + if (!mask) + mask = &rte_flow_item_tcp_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_tcp_mask, + sizeof(struct rte_flow_item_tcp), error); + if (ret < 0) + return ret; + flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_TCP; + if (size > flow_size) + return size; + if (spec) { + tcp.val.dst_port = spec->hdr.dst_port; + tcp.val.src_port = spec->hdr.src_port; + tcp.mask.dst_port = mask->hdr.dst_port; + tcp.mask.src_port = mask->hdr.src_port; + /* Remove unwanted bits from values. */ + tcp.val.src_port &= tcp.mask.src_port; + tcp.val.dst_port &= tcp.mask.dst_port; + } + mlx5_flow_spec_verbs_add(flow, &tcp, size); + return size; +} + /** * Validate items provided by the user. * @@ -832,6 +894,9 @@ mlx5_flow_items(const struct rte_flow_item items[], case RTE_FLOW_ITEM_TYPE_UDP: ret = mlx5_flow_item_udp(items, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_TCP: + ret = mlx5_flow_item_tcp(items, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jun 27 15:07:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41686 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 330531BFF1; Wed, 27 Jun 2018 17:08:01 +0200 (CEST) Received: from mail-wr0-f194.google.com (mail-wr0-f194.google.com [209.85.128.194]) by dpdk.org (Postfix) with ESMTP id 9DC6D1BFBF for ; Wed, 27 Jun 2018 17:07:45 +0200 (CEST) Received: by mail-wr0-f194.google.com with SMTP id k16-v6so2412324wro.0 for ; Wed, 27 Jun 2018 08:07:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=PqKDnXhwtZPMfywHazvZ9gKvn+hH1CQYQA5s61neoXs=; b=Zb9rcoZb+AqLkaGcgEcFK2Xg3WslnLZ+eSXDCk1APzOg+P1Rx6xwLRwyTBRZmQvjlx lkv55rSpmCbc3qMGSqbmQTNBYaMir1QH7IswtVDplqlqbTmw7EyQXIqN1/suQxrwnoTH 0wvp5Brr0LAbyucQb48ml44icMMV6ohBH74sOydli0u+uLdAim3f9WWZPCtmqNjXlLtG sCk6s2r7NSRvd0sVqqnSBxYY6V35OhPOoyFVhJchUxGCU3CwYzwpvhMxXSf8V7oXtvFc RgU/A0KNq7E+3JHHCjtovJhnnW7XOWs1DBEAEimQBC9VGCgWFZOxPjgp1EjHO/AKg7gd lxMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=PqKDnXhwtZPMfywHazvZ9gKvn+hH1CQYQA5s61neoXs=; b=AEF6eZUFbJePeBE0og/qyRuSFt96MGbXoIAk5eJH35Z0vYurMVkqKEy4nVKIqkqCx2 pABfCXEAmdAGd575MboZgkxA+cEkieNzg5QnTG4koxlWoSdXCDQ18jLPg0yew3oeEXgK TaIIxaHFi21xOuXCc2FKn8LvRm5KbETiTjBuD5+JEHhBZXZ/goguplTHnacGB90EML1g z/RYzUtHI7IwT8VSIZRIV0bWhS2y0SVs/Jwj4NrCJhzbAuQZPR9Lzwa2MrubPMnfL3d+ jJyq+dt7JDApjXLkzG358XV4oi+05L3RBikvK1PA4qxSeepsAEOfRcBZpAQKifI0YP1X V2DQ== X-Gm-Message-State: APt69E1HBnQvvVEfZZR3Ce9Ab1iwLm6KmbpJXDTcC7N3jNuntcFunyFj Xt9w9oU1MmmlL2zP4neBX9fxFTkRqg== X-Google-Smtp-Source: AAOMgpf1BiNv3ImEA1G2AGjC1muDBR5jzJHvp+3XrB7HEb3Oc9NB2jpTSxBhbsllNarVTCPdBwXqlA== X-Received: by 2002:adf:99ea:: with SMTP id y97-v6mr5414013wrb.215.1530112065047; Wed, 27 Jun 2018 08:07:45 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:44 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:44 +0200 Message-Id: <90a73b5f33e147ffa3a668f5d19410de17f96045.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 12/20] net/mlx5: add mark/flag flow action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 209 +++++++++++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 57f072c03..a39157533 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -52,6 +52,10 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; #define MLX5_FLOW_FATE_DROP (1u << 0) #define MLX5_FLOW_FATE_QUEUE (1u << 1) +/* Modify a packet. */ +#define MLX5_FLOW_MOD_FLAG (1u << 0) +#define MLX5_FLOW_MOD_MARK (1u << 1) + /** Handles information leading to a drop fate. */ struct mlx5_flow_verbs { unsigned int size; /**< Size of the attribute. */ @@ -70,6 +74,8 @@ struct rte_flow { struct rte_flow_attr attributes; /**< User flow attribute. */ uint32_t layers; /**< Bit-fields of present layers see MLX5_FLOW_ITEMS_*. */ + uint32_t modifier; + /**< Bit-fields of present modifier see MLX5_FLOW_MOD_*. */ uint32_t fate; /**< Bit-fields of present fate see MLX5_FLOW_FATE_*. */ struct mlx5_flow_verbs verbs; /* Verbs flow. */ @@ -954,6 +960,12 @@ mlx5_flow_action_drop(const struct rte_flow_action *actions, actions, "multiple fate actions are not" " supported"); + if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "drop is not compatible with" + " flag/mark action"); if (size < flow_size) mlx5_flow_spec_verbs_add(flow, &drop, size); flow->fate |= MLX5_FLOW_FATE_DROP; @@ -1007,6 +1019,144 @@ mlx5_flow_action_queue(struct rte_eth_dev *dev, return 0; } +/** + * Validate action flag provided by the user. + * + * @param actions + * Pointer to flow actions array. + * @param flow + * Pointer to the rte_flow structure. + * @param flow_size + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_action_flag(const struct rte_flow_action *actions, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + unsigned int size = sizeof(struct ibv_flow_spec_action_tag); + struct ibv_flow_spec_action_tag tag = { + .type = IBV_FLOW_SPEC_ACTION_TAG, + .size = size, + .tag_id = mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT), + }; + + if (flow->modifier & MLX5_FLOW_MOD_FLAG) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "flag action already present"); + if (flow->fate & MLX5_FLOW_FATE_DROP) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "flag is not compatible with drop" + " action"); + if (flow->modifier & MLX5_FLOW_MOD_MARK) + return 0; + flow->modifier |= MLX5_FLOW_MOD_FLAG; + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &tag, size); + return size; +} + +/** + * Update verbs specification to modify the flag to mark. + * + * @param flow + * Pointer to the rte_flow structure. + * @param mark_id + * Mark identifier to replace the flag. + */ +static void +mlx5_flow_verbs_mark_update(struct rte_flow *flow, uint32_t mark_id) +{ + struct ibv_spec_header *hdr; + int i; + + /* Update Verbs specification. */ + hdr = (struct ibv_spec_header *)flow->verbs.specs; + for (i = 0; i != flow->verbs.attr->num_of_specs; ++i) { + if (hdr->type == IBV_FLOW_SPEC_ACTION_TAG) { + struct ibv_flow_spec_action_tag *t = + (struct ibv_flow_spec_action_tag *)hdr; + + t->tag_id = mlx5_flow_mark_set(mark_id); + } + hdr = (struct ibv_spec_header *)((uintptr_t)hdr + hdr->size); + } +} + +/** + * Validate action mark provided by the user. + * + * @param actions + * Pointer to flow actions array. + * @param flow + * Pointer to the rte_flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_action_mark(const struct rte_flow_action *actions, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + const struct rte_flow_action_mark *mark = actions->conf; + unsigned int size = sizeof(struct ibv_flow_spec_action_tag); + struct ibv_flow_spec_action_tag tag = { + .type = IBV_FLOW_SPEC_ACTION_TAG, + .size = size, + }; + + if (!mark) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "configuration cannot be null"); + if (mark->id >= MLX5_FLOW_MARK_MAX) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &mark->id, + "mark must be between 0 and" + " 16777199"); + if (flow->modifier & MLX5_FLOW_MOD_MARK) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "mark action already present"); + if (flow->fate & MLX5_FLOW_FATE_DROP) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "mark is not compatible with drop" + " action"); + if (flow->modifier & MLX5_FLOW_MOD_FLAG) { + mlx5_flow_verbs_mark_update(flow, mark->id); + size = 0; /**< Only an update is done in the specification. */ + } else { + tag.tag_id = mlx5_flow_mark_set(mark->id); + if (size <= flow_size) { + tag.tag_id = mlx5_flow_mark_set(mark->id); + mlx5_flow_spec_verbs_add(flow, &tag, size); + } + } + flow->modifier |= MLX5_FLOW_MOD_MARK; + return size; +} + /** * Validate actions provided by the user. * @@ -1039,6 +1189,14 @@ mlx5_flow_actions(struct rte_eth_dev *dev, switch (actions->type) { case RTE_FLOW_ACTION_TYPE_VOID: break; + case RTE_FLOW_ACTION_TYPE_FLAG: + ret = mlx5_flow_action_flag(actions, flow, remain, + error); + break; + case RTE_FLOW_ACTION_TYPE_MARK: + ret = mlx5_flow_action_mark(actions, flow, remain, + error); + break; case RTE_FLOW_ACTION_TYPE_DROP: ret = mlx5_flow_action_drop(actions, flow, remain, error); @@ -1122,6 +1280,23 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, return size; } +/** + * Mark the Rx queues mark flag if the flow has a mark or flag modifier. + * + * @param dev + * Pointer to Ethernet device. + * @param flow + * Pointer to flow structure. + */ +static void +mlx5_flow_rxq_mark(struct rte_eth_dev *dev, struct rte_flow *flow) +{ + struct priv *priv = dev->data->dev_private; + + (*priv->rxqs)[flow->queue]->mark |= + flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK); +} + /** * Validate a flow supported by the NIC. * @@ -1281,6 +1456,7 @@ mlx5_flow_list_create(struct rte_eth_dev *dev, if (ret < 0) goto error; } + mlx5_flow_rxq_mark(dev, flow); TAILQ_INSERT_TAIL(list, flow, next); return flow; error: @@ -1323,8 +1499,31 @@ static void mlx5_flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, struct rte_flow *flow) { + struct priv *priv = dev->data->dev_private; + struct rte_flow *rflow; + const uint32_t mask = MLX5_FLOW_MOD_FLAG & MLX5_FLOW_MOD_MARK; + int mark = 0; + mlx5_flow_fate_remove(dev, flow); TAILQ_REMOVE(list, flow, next); + if (!(flow->modifier & mask)) { + rte_free(flow); + return; + } + /* + * When a flow is removed and this flow has a flag/mark modifier, all + * flows needs to be parse to verify if the Rx queue use by the flow + * still need to track the flag/mark request. + */ + TAILQ_FOREACH(rflow, &priv->flows, next) { + if (!(rflow->modifier & mask)) + continue; + if (flow->queue == rflow->queue) { + mark = 1; + break; + } + } + (*priv->rxqs)[flow->queue]->mark = !!mark; rte_free(flow); } @@ -1358,10 +1557,19 @@ mlx5_flow_list_flush(struct rte_eth_dev *dev, struct mlx5_flows *list) void mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) { + struct priv *priv = dev->data->dev_private; struct rte_flow *flow; + unsigned int i; + unsigned int idx; TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) mlx5_flow_fate_remove(dev, flow); + for (idx = 0, i = 0; idx != priv->rxqs_n; ++i) { + if (!(*priv->rxqs)[idx]) + continue; + (*priv->rxqs)[idx]->mark = 0; + ++idx; + } } /** @@ -1386,6 +1594,7 @@ mlx5_flow_start(struct rte_eth_dev *dev, struct mlx5_flows *list) ret = mlx5_flow_fate_apply(dev, flow, &error); if (ret < 0) goto error; + mlx5_flow_rxq_mark(dev, flow); } return 0; error: From patchwork Wed Jun 27 15:07:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41687 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4E8AF1BFF6; Wed, 27 Jun 2018 17:08:03 +0200 (CEST) Received: from mail-wr0-f196.google.com (mail-wr0-f196.google.com [209.85.128.196]) by dpdk.org (Postfix) with ESMTP id AF6551BFC7 for ; Wed, 27 Jun 2018 17:07:47 +0200 (CEST) Received: by mail-wr0-f196.google.com with SMTP id l2-v6so1823334wro.7 for ; Wed, 27 Jun 2018 08:07:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=2qr9b5Z3XYB3gsS8Slff33S6PWoHfnkYKrv8VzPcw/U=; b=bjqFXDTG8ga8sjo6bQWBTUS+bjp+EqICOViLfo+cXT11HETPloOCNuGezirpQM+O8z s/QyrSf7VZdhvPjLn+Igl0Q42pIXDMeqVvCmqT9GNYm9CkIiCJpFcvlhfXLUlZ8A0SoZ O229OaUt5fskABGkwptagpcS7QfgZi4xksIf2o81bQ987JjbyQl3bWnG8r8EW1tYmnP7 DWUheiHKnhOZfMKoX+nEH1SfrrE9pFCBjRu7KoBMF/ZBzQN12DrQ4V7rTHlkFesjHfPT 8X4dvBBZaKmv3hW3I1hj5HbYpgWQ/BEJbgV+1zuEocdtg1PgyYCgI5NUzU9oqO1wbzB8 zc4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=2qr9b5Z3XYB3gsS8Slff33S6PWoHfnkYKrv8VzPcw/U=; b=dhxkUtGClvQn39vZl3/zJ+gvcUevqlLGiIpmihv1j536mrnlfmibfJkD67BvVscnbE nf1NY9tqxbxpOf0HL7DV5Z2xRiVm4aslZPfXTFzJAGREYDfl8tm2PcwdGgtmCNkuN9/d GWY33G+QsNacMUom4Z1L1ZuLAdEFMkwexkcN+KLtUhDGkWAPewiTTohwWbu/KeuJOswR q+WDZOPsmptnXZJ35EfIq4dCwnbJvHb3zHwFJuOIXMrEomHoaKCHDH98phFZmXTpdaXI juZYYoIcT8xo11QRaNTHKK3aavCIuPxfqLTYf2n5qdC1qaB5HlU6mlAyOoeYztsyWPOc 1TmQ== X-Gm-Message-State: APt69E2z+piOF82Xswc8+jsFT0OA5uASSMDTrHv8FIz39owR/gFw0jz9 LhchoYYbKkUhKJhLb7mMrzvU8Ooo0Q== X-Google-Smtp-Source: AAOMgpe5AvsVP9Ek/+nb2tH5P082PtoXD4IoiD3YDIGcXjMhY0VQE6ia5KI8Xzsp9qtZdRexsXqbTw== X-Received: by 2002:adf:ff0b:: with SMTP id k11-v6mr5917544wrr.15.1530112066036; Wed, 27 Jun 2018 08:07:46 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:45 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:45 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 13/20] net/mlx5: add RSS flow action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 1211 +++++++++++++++++++++++++--------- 1 file changed, 899 insertions(+), 312 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index a39157533..08e0a6556 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -51,13 +51,148 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Action fate on the packet. */ #define MLX5_FLOW_FATE_DROP (1u << 0) #define MLX5_FLOW_FATE_QUEUE (1u << 1) +#define MLX5_FLOW_FATE_RSS (1u << 2) /* Modify a packet. */ #define MLX5_FLOW_MOD_FLAG (1u << 0) #define MLX5_FLOW_MOD_MARK (1u << 1) +/* Priority reserved for default flows. */ +#define MLX5_FLOW_PRIO_RSVD ((uint32_t)-1) + +enum mlx5_expansion { + MLX5_EXPANSION_ROOT, + MLX5_EXPANSION_ROOT2, + MLX5_EXPANSION_OUTER_ETH, + MLX5_EXPANSION_OUTER_IPV4, + MLX5_EXPANSION_OUTER_IPV4_UDP, + MLX5_EXPANSION_OUTER_IPV4_TCP, + MLX5_EXPANSION_OUTER_IPV6, + MLX5_EXPANSION_OUTER_IPV6_UDP, + MLX5_EXPANSION_OUTER_IPV6_TCP, + MLX5_EXPANSION_VXLAN, + MLX5_EXPANSION_VXLAN_GPE, + MLX5_EXPANSION_GRE, + MLX5_EXPANSION_MPLS, + MLX5_EXPANSION_ETH, + MLX5_EXPANSION_IPV4, + MLX5_EXPANSION_IPV4_UDP, + MLX5_EXPANSION_IPV4_TCP, + MLX5_EXPANSION_IPV6, + MLX5_EXPANSION_IPV6_UDP, + MLX5_EXPANSION_IPV6_TCP, +}; + +/** Supported expansion of items. */ +static const struct rte_flow_expand_node mlx5_support_expansion[] = { + [MLX5_EXPANSION_ROOT] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_ETH, + MLX5_EXPANSION_IPV4, + MLX5_EXPANSION_IPV6), + .type = RTE_FLOW_ITEM_TYPE_END, + }, + [MLX5_EXPANSION_ROOT2] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_OUTER_ETH, + MLX5_EXPANSION_OUTER_IPV4, + MLX5_EXPANSION_OUTER_IPV6), + .type = RTE_FLOW_ITEM_TYPE_END, + }, + [MLX5_EXPANSION_OUTER_ETH] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_OUTER_IPV4, + MLX5_EXPANSION_OUTER_IPV6), + .type = RTE_FLOW_ITEM_TYPE_ETH, + .rss_types = 0, + }, + [MLX5_EXPANSION_OUTER_IPV4] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_OUTER_IPV4_UDP, + MLX5_EXPANSION_OUTER_IPV4_TCP), + .type = RTE_FLOW_ITEM_TYPE_IPV4, + .rss_types = ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | + ETH_RSS_NONFRAG_IPV4_OTHER, + }, + [MLX5_EXPANSION_OUTER_IPV4_UDP] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_VXLAN), + .type = RTE_FLOW_ITEM_TYPE_UDP, + .rss_types = ETH_RSS_NONFRAG_IPV4_UDP, + }, + [MLX5_EXPANSION_OUTER_IPV4_TCP] = { + .type = RTE_FLOW_ITEM_TYPE_TCP, + .rss_types = ETH_RSS_NONFRAG_IPV4_TCP, + }, + [MLX5_EXPANSION_OUTER_IPV6] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_OUTER_IPV6_UDP, + MLX5_EXPANSION_OUTER_IPV6_TCP), + .type = RTE_FLOW_ITEM_TYPE_IPV6, + .rss_types = ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 | + ETH_RSS_NONFRAG_IPV6_OTHER, + }, + [MLX5_EXPANSION_OUTER_IPV6_UDP] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_VXLAN), + .type = RTE_FLOW_ITEM_TYPE_UDP, + .rss_types = ETH_RSS_NONFRAG_IPV6_UDP, + }, + [MLX5_EXPANSION_OUTER_IPV6_TCP] = { + .type = RTE_FLOW_ITEM_TYPE_TCP, + .rss_types = ETH_RSS_NONFRAG_IPV6_TCP, + }, + [MLX5_EXPANSION_VXLAN] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_ETH), + .type = RTE_FLOW_ITEM_TYPE_VXLAN, + }, + [MLX5_EXPANSION_VXLAN_GPE] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_ETH, + MLX5_EXPANSION_IPV4, + MLX5_EXPANSION_IPV6), + .type = RTE_FLOW_ITEM_TYPE_VXLAN_GPE, + }, + [MLX5_EXPANSION_GRE] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_IPV4), + .type = RTE_FLOW_ITEM_TYPE_GRE, + }, + [MLX5_EXPANSION_ETH] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_IPV4, + MLX5_EXPANSION_IPV6), + .type = RTE_FLOW_ITEM_TYPE_ETH, + }, + [MLX5_EXPANSION_IPV4] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_IPV4_UDP, + MLX5_EXPANSION_IPV4_TCP), + .type = RTE_FLOW_ITEM_TYPE_IPV4, + .rss_types = ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | + ETH_RSS_NONFRAG_IPV4_OTHER, + }, + [MLX5_EXPANSION_IPV4_UDP] = { + .type = RTE_FLOW_ITEM_TYPE_UDP, + .rss_types = ETH_RSS_NONFRAG_IPV4_UDP, + }, + [MLX5_EXPANSION_IPV4_TCP] = { + .type = RTE_FLOW_ITEM_TYPE_TCP, + .rss_types = ETH_RSS_NONFRAG_IPV4_TCP, + }, + [MLX5_EXPANSION_IPV6] = { + .next = RTE_FLOW_EXPAND_ITEMS(MLX5_EXPANSION_IPV6_UDP, + MLX5_EXPANSION_IPV6_TCP), + .type = RTE_FLOW_ITEM_TYPE_IPV6, + .rss_types = ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 | + ETH_RSS_NONFRAG_IPV6_OTHER, + }, + [MLX5_EXPANSION_IPV6_UDP] = { + .type = RTE_FLOW_ITEM_TYPE_UDP, + .rss_types = ETH_RSS_NONFRAG_IPV6_UDP, + }, + [MLX5_EXPANSION_IPV6_TCP] = { + .type = RTE_FLOW_ITEM_TYPE_TCP, + .rss_types = ETH_RSS_NONFRAG_IPV6_TCP, + }, +}; + /** Handles information leading to a drop fate. */ struct mlx5_flow_verbs { + LIST_ENTRY(mlx5_flow_verbs) next; + uint32_t layers; + /**< Bit-fields of expanded layers see MLX5_FLOW_ITEMS_*. */ + uint32_t modifier; + /**< Bit-fields of expanded modifier see MLX5_FLOW_MOD_*. */ unsigned int size; /**< Size of the attribute. */ struct { struct ibv_flow_attr *attr; @@ -66,20 +201,26 @@ struct mlx5_flow_verbs { }; struct ibv_flow *flow; /**< Verbs flow pointer. */ struct mlx5_hrxq *hrxq; /**< Hash Rx queue object. */ + uint64_t hash_fields; /**< Verbs hash Rx queue hash fields. */ }; /* Flow structure. */ struct rte_flow { TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ struct rte_flow_attr attributes; /**< User flow attribute. */ + uint32_t expand:1; /**< Flow is expanded due to RSS configuration. */ uint32_t layers; /**< Bit-fields of present layers see MLX5_FLOW_ITEMS_*. */ uint32_t modifier; /**< Bit-fields of present modifier see MLX5_FLOW_MOD_*. */ uint32_t fate; /**< Bit-fields of present fate see MLX5_FLOW_FATE_*. */ - struct mlx5_flow_verbs verbs; /* Verbs flow. */ - uint16_t queue; /**< Destination queue to redirect traffic to. */ + LIST_HEAD(verbs, mlx5_flow_verbs) verbs; /**< Verbs flows list. */ + struct mlx5_flow_verbs *cur_verbs; + /**< Current Verbs flow structure being filled. */ + struct rte_flow_action_rss rss;/**< RSS context. */ + uint8_t key[40]; /**< RSS hash key. */ + uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */ }; static const struct rte_flow_ops mlx5_flow_ops = { @@ -122,16 +263,27 @@ struct ibv_spec_header { uint16_t size; }; - /** - * Get the maximum number of priority available. - * - * @param dev - * Pointer to Ethernet device. - * - * @return - * number of supported flow priority on success, a negative errno value - * otherwise and rte_errno is set. - */ +/* Map of Verbs to Flow priority with 8 Verbs priorities. */ +static const uint32_t priority_map_3[][3] = { + { 0, 1, 2 }, { 2, 3, 4 }, { 5, 6, 7 }, +}; + +/* Map of Verbs to Flow priority with 16 Verbs priorities. */ +static const uint32_t priority_map_5[][3] = { + { 0, 1, 2 }, { 3, 4, 5 }, { 6, 7, 8 }, + { 9, 10, 11 }, { 12, 13, 14 }, +}; + +/** + * Get the maximum number of priority available. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * number of supported flow priority on success, a negative errno + * value otherwise and rte_errno is set. + */ int mlx5_flow_priorities(struct rte_eth_dev *dev) { @@ -156,6 +308,7 @@ mlx5_flow_priorities(struct rte_eth_dev *dev) struct mlx5_hrxq *drop = mlx5_hrxq_drop_new(dev); uint16_t vprio[] = { 8, 16 }; int i; + int priority = 0; if (!drop) { rte_errno = ENOTSUP; @@ -167,11 +320,54 @@ mlx5_flow_priorities(struct rte_eth_dev *dev) if (!flow) break; claim_zero(mlx5_glue->destroy_flow(flow)); + priority = vprio[i]; + } + switch (priority) { + case 8: + priority = 3; + break; + case 16: + priority = 5; + break; + default: + rte_errno = ENOTSUP; + DRV_LOG(ERR, + "port %u verbs maximum priority: %d expected 8/16", + dev->data->port_id, vprio[i]); + return -rte_errno; } mlx5_hrxq_drop_release(dev, drop); DRV_LOG(INFO, "port %u flow maximum priority: %d", - dev->data->port_id, vprio[i]); - return vprio[i]; + dev->data->port_id, priority); + return priority; +} + +/** + * Adjust flow priority. + * + * @param dev + * Pointer to Ethernet device. + * @param flow + * Pointer to an rte flow. + * + * @return + * The priority adjusted. + */ +static int +mlx5_flow_priority(struct rte_eth_dev *dev, uint32_t priority, + uint32_t subpriority) +{ + struct priv *priv = dev->data->dev_private; + + switch (priv->config.flow_prio) { + case 3: + priority = priority_map_3[priority][subpriority]; + break; + case 5: + priority = priority_map_5[priority][subpriority]; + break; + } + return priority; } /** @@ -185,6 +381,8 @@ void mlx5_flow_print(struct rte_flow *flow __rte_unused) { #ifndef NDEBUG + struct mlx5_flow_verbs *verbs = LIST_FIRST(&flow->verbs); + fprintf(stdout, "---------8<------------\n"); fprintf(stdout, "%s: flow information\n", MLX5_DRIVER_NAME); fprintf(stdout, " attributes: group %u priority %u ingress %d egress %d" @@ -193,26 +391,36 @@ mlx5_flow_print(struct rte_flow *flow __rte_unused) flow->attributes.ingress, flow->attributes.egress, flow->attributes.transfer); - fprintf(stdout, " layers: %s/%s/%s\n", - flow->layers & MLX5_FLOW_LAYER_OUTER_L2 ? "l2" : "-", - flow->layers & MLX5_FLOW_LAYER_OUTER_L3 ? "l3" : "-", - flow->layers & MLX5_FLOW_LAYER_OUTER_L4 ? "l4" : "-"); - if (flow->fate & MLX5_FLOW_FATE_DROP) + if (flow->fate & MLX5_FLOW_FATE_DROP) { fprintf(stdout, " fate: drop queue\n"); - else if (flow->fate & MLX5_FLOW_FATE_QUEUE) - fprintf(stdout, " fate: target queue %u\n", flow->queue); - if (flow->verbs.attr) { - struct ibv_spec_header *hdr = - (struct ibv_spec_header *)flow->verbs.specs; - const int n = flow->verbs.attr->num_of_specs; - int i; - - fprintf(stdout, " Verbs attributes: specs_n %u\n", - flow->verbs.attr->num_of_specs); - for (i = 0; i != n; ++i) { - rte_hexdump(stdout, " ", hdr, hdr->size); - hdr = (struct ibv_spec_header *) - ((uint8_t *)hdr + hdr->size); + } else { + uint16_t i; + + fprintf(stdout, " fate: target queues"); + for (i = 0; i != flow->rss.queue_num; ++i) + fprintf(stdout, " %u", (*flow->queue)[i]); + fprintf(stdout, "\n"); + } + LIST_FOREACH(verbs, &flow->verbs, next) { + uint32_t layers = flow->layers | verbs->layers; + + fprintf(stdout, " layers: %s/%s/%s\n", + layers & MLX5_FLOW_LAYER_OUTER_L2 ? "l2" : "-", + layers & MLX5_FLOW_LAYER_OUTER_L3 ? "l3" : "-", + layers & MLX5_FLOW_LAYER_OUTER_L4 ? "l4" : "-"); + if (verbs->attr) { + struct ibv_spec_header *hdr = + (struct ibv_spec_header *)verbs->specs; + const int n = verbs->attr->num_of_specs; + int i; + + fprintf(stdout, " Verbs attributes: specs_n %u\n", + verbs->attr->num_of_specs); + for (i = 0; i != n; ++i) { + rte_hexdump(stdout, " ", hdr, hdr->size); + hdr = (struct ibv_spec_header *) + ((uint8_t *)hdr + hdr->size); + } } } fprintf(stdout, "--------->8------------\n"); @@ -239,18 +447,20 @@ mlx5_flow_attributes(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, struct rte_flow *flow, struct rte_flow_error *error) { uint32_t priority_max = - ((struct priv *)dev->data->dev_private)->config.flow_prio; + ((struct priv *)dev->data->dev_private)->config.flow_prio - 1; if (attr->group) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_GROUP, NULL, "groups are not supported"); - if (attr->priority >= priority_max) + if (attr->priority != MLX5_FLOW_PRIO_RSVD && + attr->priority >= priority_max) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, NULL, - "priority value is not supported"); + "requested priority value is not" + " supported"); if (attr->egress) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, @@ -267,6 +477,8 @@ mlx5_flow_attributes(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, NULL, "only ingress is supported"); flow->attributes = *attr; + if (attr->priority == MLX5_FLOW_PRIO_RSVD) + flow->attributes.priority = priority_max; return 0; } @@ -346,14 +558,51 @@ mlx5_flow_item_validate(const struct rte_flow_item *item, static void mlx5_flow_spec_verbs_add(struct rte_flow *flow, void *src, unsigned int size) { - if (flow->verbs.specs) { + struct mlx5_flow_verbs *verbs = flow->cur_verbs; + + if (verbs->specs) { void *dst; - dst = (void *)(flow->verbs.specs + flow->verbs.size); + dst = (void *)(verbs->specs + verbs->size); memcpy(dst, src, size); - ++flow->verbs.attr->num_of_specs; + ++verbs->attr->num_of_specs; } - flow->verbs.size += size; + verbs->size += size; +} + +/** + * Update layer bit-field. + * + * @param flow[in, out] + * Pointer to flow structure. + * @param layers + * Bit-fields of layers to add see MLX5_FLOW_ITEMS_*. + */ +static void +mlx5_flow_layers_update(struct rte_flow *flow, uint32_t layers) +{ + if (flow->expand) { + if (flow->cur_verbs) + flow->cur_verbs->layers |= layers; + } else { + flow->layers |= layers; + } +} + +/** + * Get layers bit-field. + * + * @param flow[in, out] + * Pointer to flow structure. + */ +static uint32_t +mlx5_flow_layers(struct rte_flow *flow) +{ + uint32_t layers = flow->layers; + + if (flow->expand && flow->cur_verbs) + layers |= flow->cur_verbs->layers; + return layers; } /** @@ -388,22 +637,26 @@ mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, .type = IBV_FLOW_SPEC_ETH, .size = size, }; + const uint32_t layers = mlx5_flow_layers(flow); int ret; - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L2) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L2 layers already configured"); - if (!mask) - mask = &rte_flow_item_eth_mask; - ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, - (const uint8_t *)&nic_mask, - sizeof(struct rte_flow_item_eth), - error); - if (ret) - return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L2; + if (!flow->expand) { + if (layers & MLX5_FLOW_LAYER_OUTER_L2) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L2 layers already" + " configured"); + if (!mask) + mask = &rte_flow_item_eth_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_eth), + error); + if (ret) + return ret; + } + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L2); if (size > flow_size) return size; if (spec) { @@ -482,6 +735,7 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, .tci = RTE_BE16(0x0fff), }; unsigned int size = sizeof(struct ibv_flow_spec_eth); + struct mlx5_flow_verbs *verbs = flow->cur_verbs; struct ibv_flow_spec_eth eth = { .type = IBV_FLOW_SPEC_ETH, .size = size, @@ -491,24 +745,30 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, MLX5_FLOW_LAYER_OUTER_L4; const uint32_t vlanm = MLX5_FLOW_LAYER_OUTER_VLAN; const uint32_t l2m = MLX5_FLOW_LAYER_OUTER_L2; + const uint32_t layers = mlx5_flow_layers(flow); - if (flow->layers & vlanm) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L2 layers already configured"); - else if ((flow->layers & lm) != 0) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L2 layer cannot follow L3/L4 layer"); - if (!mask) - mask = &rte_flow_item_vlan_mask; - ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, - (const uint8_t *)&nic_mask, - sizeof(struct rte_flow_item_vlan), error); - if (ret) - return ret; + if (!flow->expand) { + if (layers & vlanm) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L2 layers already" + " configured"); + else if ((layers & lm) != 0) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L2 layer cannot follow" + " L3/L4 layer"); + if (!mask) + mask = &rte_flow_item_vlan_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_vlan), + error); + if (ret) + return ret; + } if (spec) { eth.val.vlan_tag = spec->tci; eth.mask.vlan_tag = mask->tci; @@ -517,32 +777,34 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, eth.mask.ether_type = mask->inner_type; eth.val.ether_type &= eth.mask.ether_type; } - /* - * From verbs perspective an empty VLAN is equivalent - * to a packet without VLAN layer. - */ - if (!eth.mask.vlan_tag) - return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM_SPEC, - item->spec, - "VLAN cannot be empty"); - /* Outer TPID cannot be matched. */ - if (eth.mask.ether_type) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM_SPEC, - item->spec, - "VLAN TPID matching is not" - " supported"); - if (!(flow->layers & l2m)) { + if (!flow->expand) { + /* + * From verbs perspective an empty VLAN is equivalent + * to a packet without VLAN layer. + */ + if (!eth.mask.vlan_tag) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "VLAN cannot be empty"); + /* Outer TPID cannot be matched. */ + if (eth.mask.ether_type) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "VLAN TPID matching is not" + " supported"); + } + if (!(layers & l2m)) { if (size <= flow_size) mlx5_flow_spec_verbs_add(flow, ð, size); } else { - if (flow->verbs.attr) - mlx5_flow_item_vlan_update(flow->verbs.attr, ð); + if (verbs->attr) + mlx5_flow_item_vlan_update(verbs->attr, ð); size = 0; /**< Only an update is done in eth specification. */ } - flow->layers |= MLX5_FLOW_LAYER_OUTER_L2 | - MLX5_FLOW_LAYER_OUTER_VLAN; + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L2 | + MLX5_FLOW_LAYER_OUTER_VLAN); return size; } @@ -582,25 +844,31 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, .size = size, }; int ret; + const uint32_t layers = mlx5_flow_layers(flow); - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L3) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "multiple L3 layers not supported"); - else if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 cannot follow an L4 layer."); - if (!mask) - mask = &rte_flow_item_ipv4_mask; - ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, - (const uint8_t *)&nic_mask, - sizeof(struct rte_flow_item_ipv4), error); - if (ret < 0) - return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV4; + if (!flow->expand) { + if (layers & MLX5_FLOW_LAYER_OUTER_L3) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "multiple L3 layers not" + " supported"); + else if (layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 cannot follow an L4" + " layer"); + if (!mask) + mask = &rte_flow_item_ipv4_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_ipv4), + error); + if (ret < 0) + return ret; + } + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L3_IPV4); if (size > flow_size) return size; if (spec) { @@ -667,25 +935,31 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, .size = size, }; int ret; + const uint32_t layers = mlx5_flow_layers(flow); - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L3) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "multiple L3 layers not supported"); - else if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 cannot follow an L4 layer."); - if (!mask) - mask = &rte_flow_item_ipv6_mask; - ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, - (const uint8_t *)&nic_mask, - sizeof(struct rte_flow_item_ipv6), error); - if (ret < 0) - return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L3_IPV6; + if (!flow->expand) { + if (layers & MLX5_FLOW_LAYER_OUTER_L3) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "multiple L3 layers not" + " supported"); + else if (layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 cannot follow an L4" + " layer"); + if (!mask) + mask = &rte_flow_item_ipv6_mask; + ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_ipv6), + error); + if (ret < 0) + return ret; + } + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L3_IPV6); if (size > flow_size) return size; if (spec) { @@ -759,25 +1033,31 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, .size = size, }; int ret; + const uint32_t layers = mlx5_flow_layers(flow); - if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 is mandatory to filter on L4"); - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L4 layer is already present"); - if (!mask) - mask = &rte_flow_item_udp_mask; - ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, - (const uint8_t *)&rte_flow_item_udp_mask, - sizeof(struct rte_flow_item_udp), error); - if (ret < 0) - return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_UDP; + if (!flow->expand) { + if (!(layers & MLX5_FLOW_LAYER_OUTER_L3)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 is mandatory to filter" + " on L4"); + if (layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L4 layer is already" + " present"); + if (!mask) + mask = &rte_flow_item_udp_mask; + ret = mlx5_flow_item_validate + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_udp_mask, + sizeof(struct rte_flow_item_udp), error); + if (ret < 0) + return ret; + } + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L4_UDP); if (size > flow_size) return size; if (spec) { @@ -821,25 +1101,31 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, .size = size, }; int ret; + const uint32_t layers = mlx5_flow_layers(flow); - if (!(flow->layers & MLX5_FLOW_LAYER_OUTER_L3)) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L3 is mandatory to filter on L4"); - if (flow->layers & MLX5_FLOW_LAYER_OUTER_L4) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ITEM, - item, - "L4 layer is already present"); - if (!mask) - mask = &rte_flow_item_tcp_mask; - ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, - (const uint8_t *)&rte_flow_item_tcp_mask, - sizeof(struct rte_flow_item_tcp), error); - if (ret < 0) - return ret; - flow->layers |= MLX5_FLOW_LAYER_OUTER_L4_TCP; + if (!flow->expand) { + if (!(layers & MLX5_FLOW_LAYER_OUTER_L3)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 is mandatory to filter" + " on L4"); + if (layers & MLX5_FLOW_LAYER_OUTER_L4) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L4 layer is already" + " present"); + if (!mask) + mask = &rte_flow_item_tcp_mask; + ret = mlx5_flow_item_validate + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_tcp_mask, + sizeof(struct rte_flow_item_tcp), error); + if (ret < 0) + return ret; + } + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L4_TCP); if (size > flow_size) return size; if (spec) { @@ -954,18 +1240,20 @@ mlx5_flow_action_drop(const struct rte_flow_action *actions, .size = size, }; - if (flow->fate) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "multiple fate actions are not" - " supported"); - if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "drop is not compatible with" - " flag/mark action"); + if (!flow->expand) { + if (flow->fate) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "multiple fate actions are" + " not supported"); + if (flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "drop is not compatible with" + " flag/mark action"); + } if (size < flow_size) mlx5_flow_spec_verbs_add(flow, &drop, size); flow->fate |= MLX5_FLOW_FATE_DROP; @@ -998,6 +1286,8 @@ mlx5_flow_action_queue(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_action_queue *queue = actions->conf; + if (flow->expand) + return 0; if (flow->fate) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -1014,11 +1304,162 @@ mlx5_flow_action_queue(struct rte_eth_dev *dev, RTE_FLOW_ERROR_TYPE_ACTION_CONF, &queue->index, "queue is not configured"); - flow->queue = queue->index; + if (flow->queue) + (*flow->queue)[0] = queue->index; + flow->rss.queue_num = 1; flow->fate |= MLX5_FLOW_FATE_QUEUE; return 0; } +/** + * Store the Verbs hash fields and priority according to the layer and types. + * + * @param dev + * Pointer to Ethernet device. + * @param flow + * Pointer to flow structure. + * @param types + * RSS types for this flow (see ETH_RSS_*). + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_action_rss_verbs_attr(struct rte_eth_dev *dev, struct rte_flow *flow, + uint32_t types) +{ + const uint32_t layers = mlx5_flow_layers(flow); + uint64_t hash_fields; + uint32_t priority; + + if ((types & ETH_RSS_NONFRAG_IPV4_TCP) && + (layers & MLX5_FLOW_LAYER_OUTER_L4_TCP)) { + hash_fields = IBV_RX_HASH_SRC_IPV4 | + IBV_RX_HASH_DST_IPV4 | + IBV_RX_HASH_SRC_PORT_TCP | + IBV_RX_HASH_DST_PORT_TCP; + priority = 0; + } else if ((types & ETH_RSS_NONFRAG_IPV4_UDP) && + (layers & MLX5_FLOW_LAYER_OUTER_L4_UDP)) { + hash_fields = IBV_RX_HASH_SRC_IPV4 | + IBV_RX_HASH_DST_IPV4 | + IBV_RX_HASH_SRC_PORT_UDP | + IBV_RX_HASH_DST_PORT_UDP; + priority = 0; + } else if ((types & (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4)) && + (layers & MLX5_FLOW_LAYER_OUTER_L3_IPV4)) { + hash_fields = IBV_RX_HASH_SRC_IPV4 | + IBV_RX_HASH_DST_IPV4; + priority = 1; + } else if ((types & ETH_RSS_NONFRAG_IPV6_TCP) && + (layers & MLX5_FLOW_LAYER_OUTER_L4_TCP)) { + hash_fields = IBV_RX_HASH_SRC_IPV6 | + IBV_RX_HASH_DST_IPV6 | + IBV_RX_HASH_SRC_PORT_TCP | + IBV_RX_HASH_DST_PORT_TCP; + priority = 0; + } else if ((types & ETH_RSS_NONFRAG_IPV6_UDP) && + (layers & MLX5_FLOW_LAYER_OUTER_L3_IPV6)) { + hash_fields = IBV_RX_HASH_SRC_IPV6 | + IBV_RX_HASH_DST_IPV6 | + IBV_RX_HASH_SRC_PORT_UDP | + IBV_RX_HASH_DST_PORT_UDP; + priority = 0; + } else if ((types & (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6)) && + (layers & MLX5_FLOW_LAYER_OUTER_L3_IPV6)) { + hash_fields = IBV_RX_HASH_SRC_IPV6 | + IBV_RX_HASH_DST_IPV6; + priority = 1; + } else { + hash_fields = 0; + priority = 2; + } + flow->cur_verbs->hash_fields = hash_fields; + flow->cur_verbs->attr->priority = + mlx5_flow_priority(dev, flow->attributes.priority, priority); + return 0; +} + +/** + * Validate action queue provided by the user. + * + * @param dev + * Pointer to Ethernet device structure. + * @param actions + * Pointer to flow actions array. + * @param flow + * Pointer to the rte_flow structure. + * @param error + * Pointer to error structure. + */ +static int +mlx5_flow_action_rss(struct rte_eth_dev *dev, + const struct rte_flow_action *actions, + struct rte_flow *flow, + struct rte_flow_error *error) +{ + struct priv *priv = dev->data->dev_private; + const struct rte_flow_action_rss *rss = actions->conf; + unsigned int i; + + if (flow->expand) + return 0; + if (flow->fate) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "multiple fate actions are not" + " supported"); + if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT && + rss->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->func, + "RSS hash function not supported"); + if (rss->level > 1) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->level, + "tunnel RSS is not supported"); + if (rss->key_len < rss_hash_default_key_len) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->key_len, + "RSS hash key too small"); + if (rss->key_len > rss_hash_default_key_len) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->key_len, + "RSS hash key too large"); + if (rss->queue_num > priv->config.ind_table_max_size) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->queue_num, + "number of queues too large"); + if (rss->types & MLX5_RSS_HF_MASK) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->types, + "some RSS protocols are not" + " supported"); + for (i = 0; i != rss->queue_num; ++i) { + if (!(*priv->rxqs)[rss->queue[i]]) + return rte_flow_error_set + (error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->queue[i], + "queue is not configured"); + } + if (flow->queue) + memcpy((*flow->queue), rss->queue, + rss->queue_num * sizeof(uint16_t)); + flow->rss.queue_num = rss->queue_num; + memcpy(flow->key, rss->key, rss_hash_default_key_len); + flow->rss.types = rss->types; + flow->fate |= MLX5_FLOW_FATE_RSS; + return 0; +} + /** * Validate action flag provided by the user. * @@ -1046,43 +1487,59 @@ mlx5_flow_action_flag(const struct rte_flow_action *actions, .size = size, .tag_id = mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT), }; + struct mlx5_flow_verbs *verbs = flow->cur_verbs; - if (flow->modifier & MLX5_FLOW_MOD_FLAG) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "flag action already present"); - if (flow->fate & MLX5_FLOW_FATE_DROP) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "flag is not compatible with drop" - " action"); - if (flow->modifier & MLX5_FLOW_MOD_MARK) - return 0; + if (!flow->expand) { + if (flow->modifier & MLX5_FLOW_MOD_FLAG) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "flag action already present"); + if (flow->fate & MLX5_FLOW_FATE_DROP) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "flag is not compatible with" + " drop action"); + } + /* + * The two only possible cases, a mark has already been added in the + * specification, in such case, the flag is already present in + * addition of the mark. + * Second case, has it is not possible to have two flags, it just + * needs to add it. + */ + if (verbs) { + verbs->modifier |= MLX5_FLOW_MOD_FLAG; + if (verbs->modifier & MLX5_FLOW_MOD_MARK) + size = 0; + else if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &tag, size); + } else { + if (flow->modifier & MLX5_FLOW_MOD_MARK) + size = 0; + } flow->modifier |= MLX5_FLOW_MOD_FLAG; - if (size <= flow_size) - mlx5_flow_spec_verbs_add(flow, &tag, size); return size; } /** * Update verbs specification to modify the flag to mark. * - * @param flow - * Pointer to the rte_flow structure. + * @param verbs + * Pointer to the mlx5_flow_verbs structure. * @param mark_id * Mark identifier to replace the flag. */ static void -mlx5_flow_verbs_mark_update(struct rte_flow *flow, uint32_t mark_id) +mlx5_flow_verbs_mark_update(struct mlx5_flow_verbs *verbs, uint32_t mark_id) { struct ibv_spec_header *hdr; int i; /* Update Verbs specification. */ - hdr = (struct ibv_spec_header *)flow->verbs.specs; - for (i = 0; i != flow->verbs.attr->num_of_specs; ++i) { + hdr = (struct ibv_spec_header *)verbs->specs; + for (i = 0; i != verbs->attr->num_of_specs; ++i) { if (hdr->type == IBV_FLOW_SPEC_ACTION_TAG) { struct ibv_flow_spec_action_tag *t = (struct ibv_flow_spec_action_tag *)hdr; @@ -1120,38 +1577,52 @@ mlx5_flow_action_mark(const struct rte_flow_action *actions, .type = IBV_FLOW_SPEC_ACTION_TAG, .size = size, }; + struct mlx5_flow_verbs *verbs = flow->cur_verbs; - if (!mark) - return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "configuration cannot be null"); - if (mark->id >= MLX5_FLOW_MARK_MAX) - return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION_CONF, - &mark->id, - "mark must be between 0 and" - " 16777199"); - if (flow->modifier & MLX5_FLOW_MOD_MARK) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "mark action already present"); - if (flow->fate & MLX5_FLOW_FATE_DROP) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ACTION, - actions, - "mark is not compatible with drop" - " action"); - if (flow->modifier & MLX5_FLOW_MOD_FLAG) { - mlx5_flow_verbs_mark_update(flow, mark->id); - size = 0; /**< Only an update is done in the specification. */ - } else { - tag.tag_id = mlx5_flow_mark_set(mark->id); - if (size <= flow_size) { + if (!flow->expand) { + if (!mark) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "configuration cannot be" + " null"); + if (mark->id >= MLX5_FLOW_MARK_MAX) + return rte_flow_error_set + (error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &mark->id, + "mark must be between 0 and 16777199"); + if (flow->modifier & MLX5_FLOW_MOD_MARK) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "mark action already" + " present"); + if (flow->fate & MLX5_FLOW_FATE_DROP) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "mark is not compatible with" + " drop action"); + } + /* + * The two only possible cases, a flag has already been added in the + * specification, in such case, it needs to be update to add the id. + * Second case, has it is not possible to have two mark, it just + * needs to add it. + */ + if (verbs) { + verbs->modifier |= MLX5_FLOW_MOD_MARK; + if (verbs->modifier & MLX5_FLOW_MOD_FLAG) { + mlx5_flow_verbs_mark_update(verbs, mark->id); + size = 0; + } else if (size <= flow_size) { tag.tag_id = mlx5_flow_mark_set(mark->id); mlx5_flow_spec_verbs_add(flow, &tag, size); } + } else { + if (flow->modifier & MLX5_FLOW_MOD_FLAG) + size = 0; } flow->modifier |= MLX5_FLOW_MOD_MARK; return size; @@ -1185,6 +1656,15 @@ mlx5_flow_actions(struct rte_eth_dev *dev, int remain = flow_size; int ret = 0; + /* + * FLAG/MARK are the only actions having a specification in Verbs and + * not making part of the packet fate. Due to this specificity and to + * avoid extra variable, their bit in the flow->modifier bit-field are + * disabled here to compute the exact necessary memory those action + * needs. + */ + flow->modifier &= ~(MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK); + /* Process the actions. */ for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { switch (actions->type) { case RTE_FLOW_ACTION_TYPE_VOID: @@ -1204,6 +1684,9 @@ mlx5_flow_actions(struct rte_eth_dev *dev, case RTE_FLOW_ACTION_TYPE_QUEUE: ret = mlx5_flow_action_queue(dev, actions, flow, error); break; + case RTE_FLOW_ACTION_TYPE_RSS: + ret = mlx5_flow_action_rss(dev, actions, flow, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -1257,27 +1740,92 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, struct rte_flow_error *error) { struct rte_flow local_flow = { .layers = 0, }; - size_t size = sizeof(*flow) + sizeof(struct ibv_flow_attr); + size_t size = sizeof(*flow); int remain = (flow_size > size) ? flow_size - size : 0; + struct rte_flow_expand_rss *buf; int ret; + uint32_t i; if (!remain) flow = &local_flow; ret = mlx5_flow_attributes(dev, attr, flow, error); if (ret < 0) return ret; - ret = mlx5_flow_items(items, flow, remain, error); - if (ret < 0) - return ret; - size += ret; - remain = (flow_size > size) ? flow_size - size : 0; - ret = mlx5_flow_actions(dev, actions, flow, remain, error); + ret = mlx5_flow_actions(dev, actions, &local_flow, 0, error); if (ret < 0) return ret; - size += ret; + ret = rte_flow_expand_rss(NULL, 0, items, local_flow.rss.types, + mlx5_support_expansion, + local_flow.rss.level < 2 ? + MLX5_EXPANSION_ROOT : MLX5_EXPANSION_ROOT2); + assert(ret > 0); + buf = rte_calloc(__func__, 1, ret, 0); + if (!buf) { + rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "not enough memory to expand the RSS flow"); + goto error; + } + ret = rte_flow_expand_rss(buf, ret, items, local_flow.rss.types, + mlx5_support_expansion, + local_flow.rss.level < 2 ? + MLX5_EXPANSION_ROOT : MLX5_EXPANSION_ROOT2); + assert(ret > 0); + size += RTE_ALIGN_CEIL(local_flow.rss.queue_num * sizeof(uint16_t), + sizeof(void *)); if (size <= flow_size) - flow->verbs.attr->priority = flow->attributes.priority; + flow->queue = (void *)(flow + 1); + LIST_INIT(&flow->verbs); + flow->layers = 0; + flow->modifier = 0; + flow->fate = 0; + for (i = 0; i != buf->entries; ++i) { + size_t off = size; + + size += sizeof(struct ibv_flow_attr) + + sizeof(struct mlx5_flow_verbs); + remain = (flow_size > size) ? flow_size - size : 0; + if (remain) { + flow->cur_verbs = (void *)((uintptr_t)flow + off); + flow->cur_verbs->attr = (void *)(flow->cur_verbs + 1); + flow->cur_verbs->specs = + (void *)(flow->cur_verbs->attr + 1); + } + ret = mlx5_flow_items + ((const struct rte_flow_item *)buf->patterns[i], + flow, remain, error); + if (ret < 0) + goto error; + size += ret; + if (remain > ret) + remain -= ret; + else + remain = 0; + ret = mlx5_flow_actions(dev, actions, flow, remain, error); + if (ret < 0) + goto error; + size += ret; + if (remain > ret) + remain -= ret; + else + remain = 0; + if (size <= flow_size) { + flow->cur_verbs->attr->priority = + flow->attributes.priority; + ret = mlx5_flow_action_rss_verbs_attr(dev, flow, + flow->rss.types); + if (ret < 0) + goto error; + LIST_INSERT_HEAD(&flow->verbs, flow->cur_verbs, next); + } + flow->expand = !!(buf->entries > 1); + } + rte_free(buf); return size; +error: + rte_free(buf); + return ret; } /** @@ -1292,9 +1840,13 @@ static void mlx5_flow_rxq_mark(struct rte_eth_dev *dev, struct rte_flow *flow) { struct priv *priv = dev->data->dev_private; + const uint32_t mask = MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK; + uint32_t i; - (*priv->rxqs)[flow->queue]->mark |= - flow->modifier & (MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK); + if (!(flow->modifier & mask)) + return; + for (i = 0; i != flow->rss.queue_num; ++i) + (*priv->rxqs)[(*flow->queue)[i]]->mark = 1; } /** @@ -1328,18 +1880,20 @@ mlx5_flow_validate(struct rte_eth_dev *dev, static void mlx5_flow_fate_remove(struct rte_eth_dev *dev, struct rte_flow *flow) { - if (flow->fate & MLX5_FLOW_FATE_DROP) { - if (flow->verbs.flow) { - claim_zero(mlx5_glue->destroy_flow(flow->verbs.flow)); - flow->verbs.flow = NULL; + struct mlx5_flow_verbs *verbs; + + LIST_FOREACH(verbs, &flow->verbs, next) { + if (verbs->flow) { + claim_zero(mlx5_glue->destroy_flow(verbs->flow)); + verbs->flow = NULL; + } + if (verbs->hrxq) { + if (flow->fate & MLX5_FLOW_FATE_DROP) + mlx5_hrxq_drop_release(dev, verbs->hrxq); + else + mlx5_hrxq_release(dev, verbs->hrxq); + verbs->hrxq = NULL; } - } - if (flow->verbs.hrxq) { - if (flow->fate & MLX5_FLOW_FATE_DROP) - mlx5_hrxq_drop_release(dev, flow->verbs.hrxq); - else if (flow->fate & MLX5_FLOW_FATE_QUEUE) - mlx5_hrxq_release(dev, flow->verbs.hrxq); - flow->verbs.hrxq = NULL; } } @@ -1360,46 +1914,68 @@ static int mlx5_flow_fate_apply(struct rte_eth_dev *dev, struct rte_flow *flow, struct rte_flow_error *error) { - if (flow->fate & MLX5_FLOW_FATE_DROP) { - flow->verbs.hrxq = mlx5_hrxq_drop_new(dev); - if (!flow->verbs.hrxq) - return rte_flow_error_set - (error, errno, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot allocate Drop queue"); - } else if (flow->fate & MLX5_FLOW_FATE_QUEUE) { - struct mlx5_hrxq *hrxq; - - hrxq = mlx5_hrxq_get(dev, rss_hash_default_key, - rss_hash_default_key_len, 0, - &flow->queue, 1, 0, 0); - if (!hrxq) - hrxq = mlx5_hrxq_new(dev, rss_hash_default_key, - rss_hash_default_key_len, 0, - &flow->queue, 1, 0, 0); - if (!hrxq) - return rte_flow_error_set(error, rte_errno, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "cannot create flow"); - flow->verbs.hrxq = hrxq; - } - flow->verbs.flow = - mlx5_glue->create_flow(flow->verbs.hrxq->qp, flow->verbs.attr); - if (!flow->verbs.flow) { - if (flow->fate & MLX5_FLOW_FATE_DROP) - mlx5_hrxq_drop_release(dev, flow->verbs.hrxq); - else - mlx5_hrxq_release(dev, flow->verbs.hrxq); - flow->verbs.hrxq = NULL; - return rte_flow_error_set(error, errno, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, - "kernel module refuses to create" - " flow"); + struct mlx5_flow_verbs *verbs; + int err; + + LIST_FOREACH(verbs, &flow->verbs, next) { + if (flow->fate & MLX5_FLOW_FATE_DROP) { + verbs->hrxq = mlx5_hrxq_drop_new(dev); + if (!verbs->hrxq) { + rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot get drop hash queue"); + goto error; + } + } else { + struct mlx5_hrxq *hrxq; + + hrxq = mlx5_hrxq_get(dev, flow->key, + rss_hash_default_key_len, + verbs->hash_fields, + (*flow->queue), + flow->rss.queue_num, 0, 0); + if (!hrxq) + hrxq = mlx5_hrxq_new(dev, flow->key, + rss_hash_default_key_len, + verbs->hash_fields, + (*flow->queue), + flow->rss.queue_num, 0, 0); + if (!hrxq) { + rte_flow_error_set + (error, rte_errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot get hash queue"); + goto error; + } + verbs->hrxq = hrxq; + } + verbs->flow = + mlx5_glue->create_flow(verbs->hrxq->qp, verbs->attr); + if (!verbs->flow) { + rte_flow_error_set(error, errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "hardware refuses to create flow"); + goto error; + } } return 0; +error: + err = rte_errno; /* Save rte_errno before cleanup. */ + LIST_FOREACH(verbs, &flow->verbs, next) { + if (verbs->hrxq) { + if (flow->fate & MLX5_FLOW_FATE_DROP) + mlx5_hrxq_drop_release(dev, verbs->hrxq); + else + mlx5_hrxq_release(dev, verbs->hrxq); + verbs->hrxq = NULL; + } + } + rte_errno = err; /* Restore rte_errno. */ + return -rte_errno; } /** @@ -1429,42 +2005,43 @@ mlx5_flow_list_create(struct rte_eth_dev *dev, const struct rte_flow_action actions[], struct rte_flow_error *error) { - struct rte_flow *flow; - size_t size; + struct rte_flow *flow = NULL; + size_t size = 0; int ret; - ret = mlx5_flow_merge(dev, NULL, 0, attr, items, actions, error); + ret = mlx5_flow_merge(dev, flow, size, attr, items, actions, error); if (ret < 0) return NULL; size = ret; - flow = rte_zmalloc(__func__, size, 0); + flow = rte_calloc(__func__, 1, size, 0); if (!flow) { rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, - "cannot allocate memory"); + "not enough memory to create flow"); return NULL; } - flow->verbs.attr = (struct ibv_flow_attr *)(flow + 1); - flow->verbs.specs = (uint8_t *)(flow->verbs.attr + 1); ret = mlx5_flow_merge(dev, flow, size, attr, items, actions, error); - if (ret < 0) - goto error; + if (ret < 0) { + rte_free(flow); + return NULL; + } assert((size_t)ret == size); if (dev->data->dev_started) { ret = mlx5_flow_fate_apply(dev, flow, error); - if (ret < 0) - goto error; + if (ret < 0) { + ret = rte_errno; /* Save rte_errno before cleanup. */ + if (flow) { + mlx5_flow_fate_remove(dev, flow); + rte_free(flow); + } + rte_errno = ret; /* Restore rte_errno. */ + return NULL; + } } mlx5_flow_rxq_mark(dev, flow); TAILQ_INSERT_TAIL(list, flow, next); return flow; -error: - ret = rte_errno; /* Save rte_errno before cleanup. */ - mlx5_flow_fate_remove(dev, flow); - rte_free(flow); - rte_errno = ret; /* Restore rte_errno. */ - return NULL; } /** @@ -1502,7 +2079,7 @@ mlx5_flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, struct priv *priv = dev->data->dev_private; struct rte_flow *rflow; const uint32_t mask = MLX5_FLOW_MOD_FLAG & MLX5_FLOW_MOD_MARK; - int mark = 0; + unsigned int i; mlx5_flow_fate_remove(dev, flow); TAILQ_REMOVE(list, flow, next); @@ -1512,18 +2089,28 @@ mlx5_flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, } /* * When a flow is removed and this flow has a flag/mark modifier, all - * flows needs to be parse to verify if the Rx queue use by the flow + * flows needs to be parse to verify if the Rx queues use by the flow * still need to track the flag/mark request. */ - TAILQ_FOREACH(rflow, &priv->flows, next) { - if (!(rflow->modifier & mask)) - continue; - if (flow->queue == rflow->queue) { - mark = 1; - break; + for (i = 0; i != flow->rss.queue_num; ++i) { + int mark = 0; + + TAILQ_FOREACH(rflow, &priv->flows, next) { + unsigned int j; + + if (!(rflow->modifier & mask)) + continue; + for (j = 0; j != rflow->rss.queue_num; ++j) { + if ((*flow->queue)[i] == (*rflow->queue)[j]) { + mark = 1; + break; + } + } + if (mark) + break; } + (*priv->rxqs)[i]->mark = !!mark; } - (*priv->rxqs)[flow->queue]->mark = !!mark; rte_free(flow); } @@ -1654,7 +2241,7 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, struct priv *priv = dev->data->dev_private; const struct rte_flow_attr attr = { .ingress = 1, - .priority = priv->config.flow_prio - 1, + .priority = MLX5_FLOW_PRIO_RSVD, }; struct rte_flow_item items[] = { { From patchwork Wed Jun 27 15:07:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41688 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EF97F1BFBC; Wed, 27 Jun 2018 17:08:05 +0200 (CEST) Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id A2C801BFC6 for ; Wed, 27 Jun 2018 17:07:47 +0200 (CEST) Received: by mail-wm0-f67.google.com with SMTP id z137-v6so5914256wmc.0 for ; Wed, 27 Jun 2018 08:07:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=rJmJKM5KNHd1jjnS4mDj5P6LiXHzuP1xkVsjfeTRzwI=; b=YvJCv5/5ybbRQKRZPKQ2ctR+T3yoOCtzrimgRw+VWT1t61fLu08jNwLn2cjKMsAIC+ JJL/UxdkMGyU4Md6ysmOWt6n539iZ1Z9STufGgjlwx+fO6V/zhe7FYtED2u/3MCfFwJA FAhmm6vSjZQTJf4D1dQs1VAQLPL4DKzekqr4B5Bin3iq1pqWfOoq4364TKO6taJHDGnr d+toPyTJbdlJOehUceOUuilWh3SVj40dHVv5PadU8gMowDnPfwaEfxlmtbV6phwqTd+u 8WvGgWUsfvl+dm60BmHZkjV0LixIxm/QdShxup+BHO3SbnBbOivwMbUsSrcQKnetPzYE fRhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=rJmJKM5KNHd1jjnS4mDj5P6LiXHzuP1xkVsjfeTRzwI=; b=Inxzm/LuiRNJkYjsF87YZw/p7LS+ZpYj9mhhyK7bix7bO6LEwICxIq3XmuXQ1virUv Tevn9zMTXziv0KZF3sbWbjXMoOxAxy0EMdgyNRFhSnTCmoGRcteVH3QhPAf7bK71ATj0 nhG0QMpPc8JwL688gpZLxMjko+5DqikD5NKsiDV+6g/vUXbNS9FZM7DtInvBXbVR6LaW Q+i3PRulfcr9DUJ3U3w87wyhmuWo8sCOyMMOgRb3XBocFvdCBhu//q7jXGIH65j+aGEP oy1hzkxV0alz1Kbqo8YKkN8c46ghsyLNhiyCpz3i6lCfXDxr4T+CyZ04eYLnO5XaHdI9 v8RA== X-Gm-Message-State: APt69E1Eb8YJeRFHcMDPt6SMathn+4w2WOhxUmer2KomsqcrMsfNFuFo fCtwmEawZAWH7dm6QbhEx9RQagI+jQ== X-Google-Smtp-Source: AAOMgpehaum7kOtp0s0PE05SsbVQCZa9h2EufuYkDQ6rK2sPda8dpbfneALQi+zMYM87Vuoe+nUlPA== X-Received: by 2002:a1c:c05:: with SMTP id 5-v6mr5587224wmm.117.1530112066768; Wed, 27 Jun 2018 08:07:46 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:46 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:46 +0200 Message-Id: <2cb1bee89e0537dd4a8165f5cbce2bb284490b9b.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 14/20] net/mlx5: remove useless arguments in hrxq API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" RSS level is necessary to had a bit in the hash_fields which is already provided in this API, for the tunnel, it is necessary to request such queue to compute the checksum on the inner most, this last one should always be activated. Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 4 ++-- drivers/net/mlx5/mlx5_rxq.c | 39 +++++++++--------------------------- drivers/net/mlx5/mlx5_rxtx.h | 8 ++------ 3 files changed, 13 insertions(+), 38 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 08e0a6556..7dda88641 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -1935,13 +1935,13 @@ mlx5_flow_fate_apply(struct rte_eth_dev *dev, struct rte_flow *flow, rss_hash_default_key_len, verbs->hash_fields, (*flow->queue), - flow->rss.queue_num, 0, 0); + flow->rss.queue_num); if (!hrxq) hrxq = mlx5_hrxq_new(dev, flow->key, rss_hash_default_key_len, verbs->hash_fields, (*flow->queue), - flow->rss.queue_num, 0, 0); + flow->rss.queue_num); if (!hrxq) { rte_flow_error_set (error, rte_errno, diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 5f17dce50..8f65e4299 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1730,10 +1730,6 @@ mlx5_ind_table_ibv_verify(struct rte_eth_dev *dev) * first queue index will be taken for the indirection table. * @param queues_n * Number of queues. - * @param tunnel - * Tunnel type, implies tunnel offloading like inner checksum if available. - * @param rss_level - * RSS hash on tunnel level. * * @return * The Verbs object initialised, NULL otherwise and rte_errno is set. @@ -1742,17 +1738,13 @@ struct mlx5_hrxq * mlx5_hrxq_new(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, - const uint16_t *queues, uint32_t queues_n, - uint32_t tunnel, uint32_t rss_level) + const uint16_t *queues, uint32_t queues_n) { struct priv *priv = dev->data->dev_private; struct mlx5_hrxq *hrxq; struct mlx5_ind_table_ibv *ind_tbl; struct ibv_qp *qp; int err; -#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - struct mlx5dv_qp_init_attr qp_init_attr = {0}; -#endif queues_n = hash_fields ? queues_n : 1; ind_tbl = mlx5_ind_table_ibv_get(dev, queues, queues_n); @@ -1767,11 +1759,6 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, rss_key = rss_hash_default_key; } #ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - if (tunnel) { - qp_init_attr.comp_mask = - MLX5DV_QP_INIT_ATTR_MASK_QP_CREATE_FLAGS; - qp_init_attr.create_flags = MLX5DV_QP_CREATE_TUNNEL_OFFLOADS; - } qp = mlx5_glue->dv_create_qp (priv->ctx, &(struct ibv_qp_init_attr_ex){ @@ -1787,14 +1774,17 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, .rx_hash_key = rss_key ? (void *)(uintptr_t)rss_key : rss_hash_default_key, - .rx_hash_fields_mask = hash_fields | - (tunnel && rss_level > 1 ? - (uint32_t)IBV_RX_HASH_INNER : 0), + .rx_hash_fields_mask = hash_fields, }, .rwq_ind_tbl = ind_tbl->ind_table, .pd = priv->pd, }, - &qp_init_attr); + &(struct mlx5dv_qp_init_attr){ + .comp_mask = (hash_fields & IBV_RX_HASH_INNER) ? + MLX5DV_QP_INIT_ATTR_MASK_QP_CREATE_FLAGS : + 0, + .create_flags = MLX5DV_QP_CREATE_TUNNEL_OFFLOADS, + }); #else qp = mlx5_glue->create_qp_ex (priv->ctx, @@ -1828,8 +1818,6 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, hrxq->qp = qp; hrxq->rss_key_len = rss_key_len; hrxq->hash_fields = hash_fields; - hrxq->tunnel = tunnel; - hrxq->rss_level = rss_level; memcpy(hrxq->rss_key, rss_key, rss_key_len); rte_atomic32_inc(&hrxq->refcnt); LIST_INSERT_HEAD(&priv->hrxqs, hrxq, next); @@ -1855,10 +1843,6 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, * first queue index will be taken for the indirection table. * @param queues_n * Number of queues. - * @param tunnel - * Tunnel type, implies tunnel offloading like inner checksum if available. - * @param rss_level - * RSS hash on tunnel level * * @return * An hash Rx queue on success. @@ -1867,8 +1851,7 @@ struct mlx5_hrxq * mlx5_hrxq_get(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, - const uint16_t *queues, uint32_t queues_n, - uint32_t tunnel, uint32_t rss_level) + const uint16_t *queues, uint32_t queues_n) { struct priv *priv = dev->data->dev_private; struct mlx5_hrxq *hrxq; @@ -1883,10 +1866,6 @@ mlx5_hrxq_get(struct rte_eth_dev *dev, continue; if (hrxq->hash_fields != hash_fields) continue; - if (hrxq->tunnel != tunnel) - continue; - if (hrxq->rss_level != rss_level) - continue; ind_tbl = mlx5_ind_table_ibv_get(dev, queues, queues_n); if (!ind_tbl) continue; diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 51f7f678b..bb67c32a6 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -157,8 +157,6 @@ struct mlx5_hrxq { struct mlx5_ind_table_ibv *ind_table; /* Indirection table. */ struct ibv_qp *qp; /* Verbs queue pair. */ uint64_t hash_fields; /* Verbs Hash fields. */ - uint32_t tunnel; /* Tunnel type. */ - uint32_t rss_level; /* RSS on tunnel level. */ uint32_t rss_key_len; /* Hash key length in bytes. */ uint8_t rss_key[]; /* Hash key. */ }; @@ -274,13 +272,11 @@ void mlx5_ind_table_ibv_drop_release(struct rte_eth_dev *dev, struct mlx5_hrxq *mlx5_hrxq_new(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, - const uint16_t *queues, uint32_t queues_n, - uint32_t tunnel, uint32_t rss_level); + const uint16_t *queues, uint32_t queues_n); struct mlx5_hrxq *mlx5_hrxq_get(struct rte_eth_dev *dev, const uint8_t *rss_key, uint32_t rss_key_len, uint64_t hash_fields, - const uint16_t *queues, uint32_t queues_n, - uint32_t tunnel, uint32_t rss_level); + const uint16_t *queues, uint32_t queues_n); int mlx5_hrxq_release(struct rte_eth_dev *dev, struct mlx5_hrxq *hxrq); int mlx5_hrxq_ibv_verify(struct rte_eth_dev *dev); struct mlx5_hrxq *mlx5_hrxq_drop_new(struct rte_eth_dev *dev); From patchwork Wed Jun 27 15:07:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41689 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E832C1BFFF; Wed, 27 Jun 2018 17:08:07 +0200 (CEST) Received: from mail-wr0-f194.google.com (mail-wr0-f194.google.com [209.85.128.194]) by dpdk.org (Postfix) with ESMTP id 105701BFC9 for ; Wed, 27 Jun 2018 17:07:48 +0200 (CEST) Received: by mail-wr0-f194.google.com with SMTP id p1-v6so2404710wrs.9 for ; Wed, 27 Jun 2018 08:07:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=jPfyD6CAN2aKYBZB50wtlhj7p+BVpkVkul6J4E3lU9k=; b=S88vsgJP5/cNV6tdekHlBCk1hEuaJBb0PdlIAF60l0nIEDOa8lNllEigenOwsGNP6U 683HQ3y4kyq7fq/rj8fu939Dh4FdKyByX4E4Nt84wvtYyyJ8KtJiGjBU+ejb2dLuwxgP XYqEUcD4YvW3rF7quseeC5vjyfvVjMZHK8S3KpFniqNvuB4Qby4e+4FXOn+/C2ig7TXk BmOY6AFpOUqlKYPt0PgdOz9T0yOKIZDRODYgTCEmSbxu0NqnXOOChMygOSfeugxUcaIY EUReuV3l6bpptnL6xARWSEZaUwYUAO4bRW64mZpPGdCWje9uYPZkrGvmatT3XdpLfa84 CxPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=jPfyD6CAN2aKYBZB50wtlhj7p+BVpkVkul6J4E3lU9k=; b=QlIgUEvvRmjWp0QmFPR/M7na6TN9dQhKuBNYZCwutxhHlvjAg3SASXdM/tSXYTLPQw HRZ0zStXY7gbpMrwmEtbMko4AT/PwwWdOlAwXHElf9lPBgLscd9UrtyxEwk0Dm3/g6lR hKnZXzldkDtF8qEElVw7GrP6MlFziiuXn1zu+ACjZ60yS19TNNlqcmQORdTTdBF1/9ex 1U5hwV+xLltXR2hB3nPoMdT0nPLkf73FcXWrb9fza0MoDEDh8+nfO4svipcyQCggNg3+ U2YcZocSwtHauozgs++OyEPcAAAFV1KIDfGglqMqGryLpcrcVARC/LfBKt9/VkG8wHCh h42Q== X-Gm-Message-State: APt69E1BfMEpcKQxCANheB0W0UezXiAg4QQVkgOoc/6oJSnXFdM3CRoU +myYV1/Yr+ue1NljWOyDbRn1yPjssw== X-Google-Smtp-Source: AAOMgpeEkjVTtUygdG3BGTK8M7MKQq14j9vG06nRVybNNVq4kCe3vzq8/0wEvHP9oRxdVfPMhKBPhQ== X-Received: by 2002:adf:8227:: with SMTP id 36-v6mr5429344wrb.144.1530112067482; Wed, 27 Jun 2018 08:07:47 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:47 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:47 +0200 Message-Id: <3d98905c5684a15c9cf96be6b65fc6d83d5443d8.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 15/20] net/mlx5: support inner RSS computation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 131 +++++++++++++++++++++++++---------- drivers/net/mlx5/mlx5_rxtx.h | 1 - 2 files changed, 96 insertions(+), 36 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 7dda88641..eedf0c461 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -219,6 +219,8 @@ struct rte_flow { struct mlx5_flow_verbs *cur_verbs; /**< Current Verbs flow structure being filled. */ struct rte_flow_action_rss rss;/**< RSS context. */ + uint32_t ptype; + /**< Store tunnel packet type data to store in Rx queue. */ uint8_t key[40]; /**< RSS hash key. */ uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */ }; @@ -1320,13 +1322,15 @@ mlx5_flow_action_queue(struct rte_eth_dev *dev, * Pointer to flow structure. * @param types * RSS types for this flow (see ETH_RSS_*). + * @param level + * RSS level. * * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int mlx5_flow_action_rss_verbs_attr(struct rte_eth_dev *dev, struct rte_flow *flow, - uint32_t types) + uint32_t types, uint32_t level) { const uint32_t layers = mlx5_flow_layers(flow); uint64_t hash_fields; @@ -1374,6 +1378,8 @@ mlx5_flow_action_rss_verbs_attr(struct rte_eth_dev *dev, struct rte_flow *flow, hash_fields = 0; priority = 2; } + if (hash_fields && level == 2) + hash_fields |= IBV_RX_HASH_INNER; flow->cur_verbs->hash_fields = hash_fields; flow->cur_verbs->attr->priority = mlx5_flow_priority(dev, flow->attributes.priority, priority); @@ -1416,7 +1422,7 @@ mlx5_flow_action_rss(struct rte_eth_dev *dev, RTE_FLOW_ERROR_TYPE_ACTION_CONF, &rss->func, "RSS hash function not supported"); - if (rss->level > 1) + if (rss->level > 2) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION_CONF, &rss->level, @@ -1456,6 +1462,7 @@ mlx5_flow_action_rss(struct rte_eth_dev *dev, flow->rss.queue_num = rss->queue_num; memcpy(flow->key, rss->key, rss_hash_default_key_len); flow->rss.types = rss->types; + flow->rss.level = rss->level; flow->fate |= MLX5_FLOW_FATE_RSS; return 0; } @@ -1814,7 +1821,8 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, flow->cur_verbs->attr->priority = flow->attributes.priority; ret = mlx5_flow_action_rss_verbs_attr(dev, flow, - flow->rss.types); + flow->rss.types, + flow->rss.level); if (ret < 0) goto error; LIST_INSERT_HEAD(&flow->verbs, flow->cur_verbs, next); @@ -1828,27 +1836,6 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, return ret; } -/** - * Mark the Rx queues mark flag if the flow has a mark or flag modifier. - * - * @param dev - * Pointer to Ethernet device. - * @param flow - * Pointer to flow structure. - */ -static void -mlx5_flow_rxq_mark(struct rte_eth_dev *dev, struct rte_flow *flow) -{ - struct priv *priv = dev->data->dev_private; - const uint32_t mask = MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK; - uint32_t i; - - if (!(flow->modifier & mask)) - return; - for (i = 0; i != flow->rss.queue_num; ++i) - (*priv->rxqs)[(*flow->queue)[i]]->mark = 1; -} - /** * Validate a flow supported by the NIC. * @@ -1978,6 +1965,88 @@ mlx5_flow_fate_apply(struct rte_eth_dev *dev, struct rte_flow *flow, return -rte_errno; } +/** + * Set the Tunnel packet type and the Mark in the Rx queue. + * + * @param dev + * Pointer to Ethernet device. + * @param idx + * Rx queue index. + */ +static void +mlx5_flow_rxq(struct rte_eth_dev *dev, uint16_t idx) +{ + struct priv *priv = dev->data->dev_private; + struct rte_flow *flow; + const uint32_t mark_m = MLX5_FLOW_MOD_FLAG | MLX5_FLOW_MOD_MARK; + uint32_t ptype = 0; + uint32_t mark = 0; + + TAILQ_FOREACH(flow, &priv->flows, next) { + unsigned int i; + + for (i = 0; i != flow->rss.queue_num; ++i) { + if ((*flow->queue)[i] == idx) { + mark |= !!(flow->modifier & mark_m); + if (ptype == 0) + ptype = flow->ptype; + else if (ptype != flow->ptype) + ptype = (uint32_t)-1; + break; + } + } + } + if (ptype == (uint32_t)-1) + ptype = 0; + (*priv->rxqs)[idx]->tunnel = ptype; + (*priv->rxqs)[idx]->mark = mark; +} + +/** + * Set the tunnel packet type and Mark in the Rx queues, if several packet + * types are possible, the information in the Rx queue is just cleared. + * Mark is not impacted by this case. + * + * @param dev + * Pointer to Ethernet device. + */ +static void +mlx5_flow_rxqs(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + unsigned int idx; + unsigned int n; + + for (idx = 0, n = 0; n != priv->rxqs_n; ++idx) { + if (!(*priv->rxqs)[idx]) + continue; + mlx5_flow_rxq(dev, idx); + n++; + } +} + +/** + * Clear tunnel ptypes and Mark in Rx queues. + * + * @param dev + * Pointer to Ethernet device. + */ +static void +mlx5_flow_rxqs_clear(struct rte_eth_dev *dev) +{ + struct priv *priv = dev->data->dev_private; + unsigned int idx; + unsigned int n; + + for (idx = 0, n = 0; n != priv->rxqs_n; ++idx) { + if (!(*priv->rxqs)[idx]) + continue; + (*priv->rxqs)[idx]->tunnel = 0; + (*priv->rxqs)[idx]->mark = 0; + n++; + } +} + /** * Create a flow. * @@ -2039,8 +2108,8 @@ mlx5_flow_list_create(struct rte_eth_dev *dev, return NULL; } } - mlx5_flow_rxq_mark(dev, flow); TAILQ_INSERT_TAIL(list, flow, next); + mlx5_flow_rxqs(dev); return flow; } @@ -2144,19 +2213,11 @@ mlx5_flow_list_flush(struct rte_eth_dev *dev, struct mlx5_flows *list) void mlx5_flow_stop(struct rte_eth_dev *dev, struct mlx5_flows *list) { - struct priv *priv = dev->data->dev_private; struct rte_flow *flow; - unsigned int i; - unsigned int idx; TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) mlx5_flow_fate_remove(dev, flow); - for (idx = 0, i = 0; idx != priv->rxqs_n; ++i) { - if (!(*priv->rxqs)[idx]) - continue; - (*priv->rxqs)[idx]->mark = 0; - ++idx; - } + mlx5_flow_rxqs_clear(dev); } /** @@ -2181,8 +2242,8 @@ mlx5_flow_start(struct rte_eth_dev *dev, struct mlx5_flows *list) ret = mlx5_flow_fate_apply(dev, flow, &error); if (ret < 0) goto error; - mlx5_flow_rxq_mark(dev, flow); } + mlx5_flow_rxqs(dev); return 0; error: ret = rte_errno; /* Save rte_errno before cleanup. */ diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index bb67c32a6..57504ceb2 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -136,7 +136,6 @@ struct mlx5_rxq_ctrl { struct priv *priv; /* Back pointer to private data. */ struct mlx5_rxq_data rxq; /* Data path structure. */ unsigned int socket; /* CPU socket ID for allocations. */ - uint32_t tunnel_types[16]; /* Tunnel type counter. */ unsigned int irq:1; /* Whether IRQ is enabled. */ uint16_t idx; /* Queue index. */ }; From patchwork Wed Jun 27 15:07:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41690 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 204D71BFFC; Wed, 27 Jun 2018 17:08:10 +0200 (CEST) Received: from mail-wr0-f196.google.com (mail-wr0-f196.google.com [209.85.128.196]) by dpdk.org (Postfix) with ESMTP id 30C701BFB5 for ; Wed, 27 Jun 2018 17:07:49 +0200 (CEST) Received: by mail-wr0-f196.google.com with SMTP id u7-v6so2396265wrn.12 for ; Wed, 27 Jun 2018 08:07:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=W4hG2Adl28FBtFPZh1X+CWOcxoVqTL3DmEOsmwObpv8=; b=gLYldRR4A9RkD0qdOi9HulDMejlfwONvntzOmkmkcBd8xBsnmhv5amZMsGLLs6IuXy nq+BPW3aXhAgGSMRObKvXk/rwCn/WvBMiUeZJgMGRK7HMtpLyRs1xLGiGs+94dE5sST6 zPN71LMDCqYvr7NOP9PAdR+stA/zdXwcqHfNeJ8nKgJ/yCga1zHzo63TadAoWZRH0lhH GJxGAyjyKH22F1zuOMN1/tSoG9ZWetm0UksBVDvkuOIfcktkvNHJ1V08algVuY3EKXZr aUVnPklMawDKAfVOo0W1aEBKgqAxm9WNCJY5/8embgPi1X8ykBPN9yyEbGaUgsEUL7bm NInQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=W4hG2Adl28FBtFPZh1X+CWOcxoVqTL3DmEOsmwObpv8=; b=tdU+jfQzziE37EpUNbDizB+BbwqpsPPHhabizYAE9PL29ShesH0lXy3oO+vnMoxgIL /kTq99itNwc5MHwBEGmHks6A/AV8MC9RK34/dZs/O8tL8Q6glZRnP3qpZxTCuPZnqHz9 BGdZ+b61QphFAjq4H4GYlGuvQ47mcmyrXOsYzBY/d+kI56OfZAs/GiBktNXxAHSGqnWa TC3Z5QXv2ppqZiiGfc4R5ydHonpUsp7WppNpKypMAznieli8UaupSiQBEZUzc4iFgD3Q xr+cLLDtfgbmdACJQ4UZWdFjfZyrAfmnhs4YqXyS5f9gCBp+C6quPO9tE2/JXhWcIeEV ZbKQ== X-Gm-Message-State: APt69E0crUgsXeLOAMNyovjRjoyBCWGmq2yxiZQop2OW/J4aLA2ffNDi 88LzhUBiPA7McfWeNSQyPN6vJfBeaA== X-Google-Smtp-Source: AAOMgpcnRh8awJ+TqEwwlP39p2IQhO/6Sn40YMMcelURnA1ohj4EvtmdVKPLATOKkotXdtl976soSQ== X-Received: by 2002:adf:fc8c:: with SMTP id g12-v6mr5509411wrr.216.1530112068368; Wed, 27 Jun 2018 08:07:48 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:47 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:48 +0200 Message-Id: <723a4eb5db839302778ca3fb0894dbe24f5acd20.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 16/20] net/mlx5: add flow VXLAN item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5_flow.c | 242 +++++++++++++++++++++++++++++------ 1 file changed, 202 insertions(+), 40 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index eedf0c461..daf5b9b5a 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -35,18 +35,45 @@ extern const struct eth_dev_ops mlx5_dev_ops; extern const struct eth_dev_ops mlx5_dev_ops_isolate; -/* Pattern Layer bits. */ +/* Pattern outer Layer bits. */ #define MLX5_FLOW_LAYER_OUTER_L2 (1u << 0) #define MLX5_FLOW_LAYER_OUTER_L3_IPV4 (1u << 1) #define MLX5_FLOW_LAYER_OUTER_L3_IPV6 (1u << 2) #define MLX5_FLOW_LAYER_OUTER_L4_UDP (1u << 3) #define MLX5_FLOW_LAYER_OUTER_L4_TCP (1u << 4) #define MLX5_FLOW_LAYER_OUTER_VLAN (1u << 5) -/* Masks. */ + +/* Pattern inner Layer bits. */ +#define MLX5_FLOW_LAYER_INNER_L2 (1u << 6) +#define MLX5_FLOW_LAYER_INNER_L3_IPV4 (1u << 7) +#define MLX5_FLOW_LAYER_INNER_L3_IPV6 (1u << 8) +#define MLX5_FLOW_LAYER_INNER_L4_UDP (1u << 9) +#define MLX5_FLOW_LAYER_INNER_L4_TCP (1u << 10) +#define MLX5_FLOW_LAYER_INNER_VLAN (1u << 11) + +/* Pattern tunnel Layer bits. */ +#define MLX5_FLOW_LAYER_VXLAN (1u << 12) + +/* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6) #define MLX5_FLOW_LAYER_OUTER_L4 \ (MLX5_FLOW_LAYER_OUTER_L4_UDP | MLX5_FLOW_LAYER_OUTER_L4_TCP) +#define MLX5_FLOW_LAYER_OUTER \ + (MLX5_FLOW_LAYER_OUTER_L2 | MLX5_FLOW_LAYER_OUTER_L3 | \ + MLX5_FLOW_LAYER_OUTER_L4) + +/* Tunnel masks. */ +#define MLX5_FLOW_LAYER_TUNNEL MLX5_FLOW_LAYER_VXLAN + +/* Inner Masks. */ +#define MLX5_FLOW_LAYER_INNER_L3 \ + (MLX5_FLOW_LAYER_INNER_L3_IPV4 | MLX5_FLOW_LAYER_INNER_L3_IPV6) +#define MLX5_FLOW_LAYER_INNER_L4 \ + (MLX5_FLOW_LAYER_INNER_L4_UDP | MLX5_FLOW_LAYER_INNER_L4_TCP) +#define MLX5_FLOW_LAYER_INNER \ + (MLX5_FLOW_LAYER_INNER_L2 | MLX5_FLOW_LAYER_INNER_L3 | \ + MLX5_FLOW_LAYER_INNER_L4) /* Action fate on the packet. */ #define MLX5_FLOW_FATE_DROP (1u << 0) @@ -406,10 +433,14 @@ mlx5_flow_print(struct rte_flow *flow __rte_unused) LIST_FOREACH(verbs, &flow->verbs, next) { uint32_t layers = flow->layers | verbs->layers; - fprintf(stdout, " layers: %s/%s/%s\n", + fprintf(stdout, " layers: %s/%s/%s/%s/%s/%s/%s\n", layers & MLX5_FLOW_LAYER_OUTER_L2 ? "l2" : "-", layers & MLX5_FLOW_LAYER_OUTER_L3 ? "l3" : "-", - layers & MLX5_FLOW_LAYER_OUTER_L4 ? "l4" : "-"); + layers & MLX5_FLOW_LAYER_OUTER_L4 ? "l4" : "-", + layers & MLX5_FLOW_LAYER_TUNNEL ? "T" : "-", + layers & MLX5_FLOW_LAYER_INNER_L2 ? "l2" : "-", + layers & MLX5_FLOW_LAYER_INNER_L3 ? "l3" : "-", + layers & MLX5_FLOW_LAYER_INNER_L4 ? "l4" : "-"); if (verbs->attr) { struct ibv_spec_header *hdr = (struct ibv_spec_header *)verbs->specs; @@ -634,16 +665,18 @@ mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, .src.addr_bytes = "\xff\xff\xff\xff\xff\xff", .type = RTE_BE16(0xffff), }; + const uint32_t layers = mlx5_flow_layers(flow); + const int tunnel = !!(layers & MLX5_FLOW_LAYER_TUNNEL); const unsigned int size = sizeof(struct ibv_flow_spec_eth); struct ibv_flow_spec_eth eth = { - .type = IBV_FLOW_SPEC_ETH, + .type = IBV_FLOW_SPEC_ETH | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; - const uint32_t layers = mlx5_flow_layers(flow); int ret; if (!flow->expand) { - if (layers & MLX5_FLOW_LAYER_OUTER_L2) + if (layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L2 : + MLX5_FLOW_LAYER_OUTER_L2)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -658,7 +691,8 @@ mlx5_flow_item_eth(const struct rte_flow_item *item, struct rte_flow *flow, if (ret) return ret; } - mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L2); + mlx5_flow_layers_update(flow, tunnel ? MLX5_FLOW_LAYER_INNER_L2 : + MLX5_FLOW_LAYER_OUTER_L2); if (size > flow_size) return size; if (spec) { @@ -694,7 +728,7 @@ mlx5_flow_item_vlan_update(struct ibv_flow_attr *attr, struct ibv_flow_spec_eth *eth) { unsigned int i; - enum ibv_flow_spec_type search = IBV_FLOW_SPEC_ETH; + const enum ibv_flow_spec_type search = eth->type; struct ibv_spec_header *hdr = (struct ibv_spec_header *) ((uint8_t *)attr + sizeof(struct ibv_flow_attr)); @@ -737,17 +771,20 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, .tci = RTE_BE16(0x0fff), }; unsigned int size = sizeof(struct ibv_flow_spec_eth); - struct mlx5_flow_verbs *verbs = flow->cur_verbs; + const uint32_t layers = mlx5_flow_layers(flow); + const int tunnel = !!(layers & MLX5_FLOW_LAYER_TUNNEL); struct ibv_flow_spec_eth eth = { - .type = IBV_FLOW_SPEC_ETH, + .type = IBV_FLOW_SPEC_ETH | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; - const uint32_t lm = MLX5_FLOW_LAYER_OUTER_L3 | - MLX5_FLOW_LAYER_OUTER_L4; - const uint32_t vlanm = MLX5_FLOW_LAYER_OUTER_VLAN; - const uint32_t l2m = MLX5_FLOW_LAYER_OUTER_L2; - const uint32_t layers = mlx5_flow_layers(flow); + const uint32_t lm = tunnel ? (MLX5_FLOW_LAYER_INNER_L3 | + MLX5_FLOW_LAYER_INNER_L4) : + (MLX5_FLOW_LAYER_OUTER_L3 | MLX5_FLOW_LAYER_OUTER_L4); + const uint32_t vlanm = tunnel ? MLX5_FLOW_LAYER_INNER_VLAN : + MLX5_FLOW_LAYER_OUTER_VLAN; + const uint32_t l2m = tunnel ? MLX5_FLOW_LAYER_INNER_L2 : + MLX5_FLOW_LAYER_OUTER_L2; if (!flow->expand) { if (layers & vlanm) @@ -801,12 +838,17 @@ mlx5_flow_item_vlan(const struct rte_flow_item *item, struct rte_flow *flow, if (size <= flow_size) mlx5_flow_spec_verbs_add(flow, ð, size); } else { - if (verbs->attr) - mlx5_flow_item_vlan_update(verbs->attr, ð); + if (flow->cur_verbs) + mlx5_flow_item_vlan_update(flow->cur_verbs->attr, + ð); size = 0; /**< Only an update is done in eth specification. */ } - mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L2 | - MLX5_FLOW_LAYER_OUTER_VLAN); + mlx5_flow_layers_update(flow, + tunnel ? + (MLX5_FLOW_LAYER_INNER_L2 | + MLX5_FLOW_LAYER_INNER_VLAN) : + (MLX5_FLOW_LAYER_OUTER_L2 | + MLX5_FLOW_LAYER_OUTER_VLAN)); return size; } @@ -840,22 +882,26 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, .next_proto_id = 0xff, }, }; + const uint32_t layers = mlx5_flow_layers(flow); + const int tunnel = !!(layers & MLX5_FLOW_LAYER_TUNNEL); unsigned int size = sizeof(struct ibv_flow_spec_ipv4_ext); struct ibv_flow_spec_ipv4_ext ipv4 = { - .type = IBV_FLOW_SPEC_IPV4_EXT, + .type = IBV_FLOW_SPEC_IPV4_EXT | + (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; - const uint32_t layers = mlx5_flow_layers(flow); if (!flow->expand) { - if (layers & MLX5_FLOW_LAYER_OUTER_L3) + if (layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L3 : + MLX5_FLOW_LAYER_OUTER_L3)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, "multiple L3 layers not" " supported"); - else if (layers & MLX5_FLOW_LAYER_OUTER_L4) + else if (layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L4 : + MLX5_FLOW_LAYER_OUTER_L4)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -870,7 +916,9 @@ mlx5_flow_item_ipv4(const struct rte_flow_item *item, struct rte_flow *flow, if (ret < 0) return ret; } - mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L3_IPV4); + mlx5_flow_layers_update(flow, + tunnel ? MLX5_FLOW_LAYER_INNER_L3_IPV4 : + MLX5_FLOW_LAYER_OUTER_L3_IPV4); if (size > flow_size) return size; if (spec) { @@ -931,22 +979,25 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, .hop_limits = 0xff, }, }; + const uint32_t layers = mlx5_flow_layers(flow); + const int tunnel = !!(layers & MLX5_FLOW_LAYER_TUNNEL); unsigned int size = sizeof(struct ibv_flow_spec_ipv6); struct ibv_flow_spec_ipv6 ipv6 = { - .type = IBV_FLOW_SPEC_IPV6, + .type = IBV_FLOW_SPEC_IPV6 | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; - const uint32_t layers = mlx5_flow_layers(flow); if (!flow->expand) { - if (layers & MLX5_FLOW_LAYER_OUTER_L3) + if (layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L3 : + MLX5_FLOW_LAYER_OUTER_L3)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, "multiple L3 layers not" " supported"); - else if (layers & MLX5_FLOW_LAYER_OUTER_L4) + else if (layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L4 : + MLX5_FLOW_LAYER_OUTER_L4)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -961,7 +1012,9 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, if (ret < 0) return ret; } - mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L3_IPV6); + mlx5_flow_layers_update(flow, + tunnel ? MLX5_FLOW_LAYER_INNER_L3_IPV6 : + MLX5_FLOW_LAYER_OUTER_L3_IPV6); if (size > flow_size) return size; if (spec) { @@ -1029,22 +1082,25 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, { const struct rte_flow_item_udp *spec = item->spec; const struct rte_flow_item_udp *mask = item->mask; + const uint32_t layers = mlx5_flow_layers(flow); + const int tunnel = !!(layers & MLX5_FLOW_LAYER_TUNNEL); unsigned int size = sizeof(struct ibv_flow_spec_tcp_udp); struct ibv_flow_spec_tcp_udp udp = { - .type = IBV_FLOW_SPEC_UDP, + .type = IBV_FLOW_SPEC_UDP | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; - const uint32_t layers = mlx5_flow_layers(flow); if (!flow->expand) { - if (!(layers & MLX5_FLOW_LAYER_OUTER_L3)) + if (!(layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L3 : + MLX5_FLOW_LAYER_OUTER_L3))) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, "L3 is mandatory to filter" " on L4"); - if (layers & MLX5_FLOW_LAYER_OUTER_L4) + if (layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L4 : + MLX5_FLOW_LAYER_OUTER_L4)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -1059,7 +1115,9 @@ mlx5_flow_item_udp(const struct rte_flow_item *item, struct rte_flow *flow, if (ret < 0) return ret; } - mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L4_UDP); + mlx5_flow_layers_update(flow, + tunnel ? MLX5_FLOW_LAYER_INNER_L4_UDP : + MLX5_FLOW_LAYER_OUTER_L4_UDP); if (size > flow_size) return size; if (spec) { @@ -1097,22 +1155,25 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, { const struct rte_flow_item_tcp *spec = item->spec; const struct rte_flow_item_tcp *mask = item->mask; + const uint32_t layers = mlx5_flow_layers(flow); + const int tunnel = !!(layers & MLX5_FLOW_LAYER_TUNNEL); unsigned int size = sizeof(struct ibv_flow_spec_tcp_udp); struct ibv_flow_spec_tcp_udp tcp = { - .type = IBV_FLOW_SPEC_TCP, + .type = IBV_FLOW_SPEC_TCP | (tunnel ? IBV_FLOW_SPEC_INNER : 0), .size = size, }; int ret; - const uint32_t layers = mlx5_flow_layers(flow); if (!flow->expand) { - if (!(layers & MLX5_FLOW_LAYER_OUTER_L3)) + if (!(layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L3 : + MLX5_FLOW_LAYER_OUTER_L3))) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, "L3 is mandatory to filter" " on L4"); - if (layers & MLX5_FLOW_LAYER_OUTER_L4) + if (layers & (tunnel ? MLX5_FLOW_LAYER_INNER_L4 : + MLX5_FLOW_LAYER_OUTER_L4)) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, @@ -1127,7 +1188,9 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, if (ret < 0) return ret; } - mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_OUTER_L4_TCP); + mlx5_flow_layers_update(flow, + tunnel ? MLX5_FLOW_LAYER_INNER_L4_TCP : + MLX5_FLOW_LAYER_OUTER_L4_TCP); if (size > flow_size) return size; if (spec) { @@ -1143,6 +1206,102 @@ mlx5_flow_item_tcp(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Validate VXLAN layer and possibly create the Verbs specification. + * + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_vxlan(const struct rte_flow_item *item, struct rte_flow *flow, + const size_t flow_size, struct rte_flow_error *error) +{ + const struct rte_flow_item_vxlan *spec = item->spec; + const struct rte_flow_item_vxlan *mask = item->mask; + const uint32_t layers = mlx5_flow_layers(flow); + unsigned int size = sizeof(struct ibv_flow_spec_tunnel); + struct ibv_flow_spec_tunnel vxlan = { + .type = IBV_FLOW_SPEC_VXLAN_TUNNEL, + .size = size, + }; + int ret; + union vni { + uint32_t vlan_id; + uint8_t vni[4]; + } id = { .vlan_id = 0, }; + + if (!flow->expand) { + if (layers & MLX5_FLOW_LAYER_TUNNEL) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "a tunnel is already" + " present"); + /* + * Verify only UDPv4 is present as defined in + * https://tools.ietf.org/html/rfc7348 + */ + if (!(layers & MLX5_FLOW_LAYER_OUTER_L4_UDP)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "no outer UDP layer found"); + if (!mask) + mask = &rte_flow_item_vxlan_mask; + ret = mlx5_flow_item_validate + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_vxlan_mask, + sizeof(struct rte_flow_item_vxlan), error); + if (ret < 0) + return ret; + } + if (spec) { + memcpy(&id.vni[1], spec->vni, 3); + vxlan.val.tunnel_id = id.vlan_id; + memcpy(&id.vni[1], mask->vni, 3); + vxlan.mask.tunnel_id = id.vlan_id; + /* Remove unwanted bits from values. */ + vxlan.val.tunnel_id &= vxlan.mask.tunnel_id; + } + if (!flow->expand) { + /* + * Tunnel id 0 is equivalent as not adding a VXLAN layer, if + * only this layer is defined in the Verbs specification it is + * interpreted as wildcard and all packets will match this + * rule, if it follows a full stack layer (ex: eth / ipv4 / + * udp), all packets matching the layers before will also + * match this rule. To avoid such situation, VNI 0 is + * currently refused. + */ + if (!vxlan.val.tunnel_id) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VXLAN vni cannot be 0"); + if (!(layers & MLX5_FLOW_LAYER_OUTER)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VXLAN tunnel must be fully" + " defined"); + } + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &vxlan, size); + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_VXLAN); + flow->ptype = RTE_PTYPE_TUNNEL_VXLAN | RTE_PTYPE_L4_UDP; + return size; +} + /** * Validate items provided by the user. * @@ -1191,6 +1350,9 @@ mlx5_flow_items(const struct rte_flow_item items[], case RTE_FLOW_ITEM_TYPE_TCP: ret = mlx5_flow_item_tcp(items, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_VXLAN: + ret = mlx5_flow_item_vxlan(items, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jun 27 15:07:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41691 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C8AE01C012; Wed, 27 Jun 2018 17:08:11 +0200 (CEST) Received: from mail-wr0-f195.google.com (mail-wr0-f195.google.com [209.85.128.195]) by dpdk.org (Postfix) with ESMTP id A72A61BFCC for ; Wed, 27 Jun 2018 17:07:49 +0200 (CEST) Received: by mail-wr0-f195.google.com with SMTP id c13-v6so2410130wrq.2 for ; Wed, 27 Jun 2018 08:07:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=Gd7i2AsWX41QgQk8Ar7YsqHX/20TGthlM9YUUXpSoxo=; b=L1bAmyNDhpvWls+SjkmsH634Lmi9nifbfFPIUqk50cUUbXnuy9dHvat8dumBEycorZ teXCJAgkUO7YTtuX7+g2bcFBwU8K9ebfXTRH7zdGxgITvA2ZtiuRW0gCPbafLLciioH3 qgqWEXAoBvhW3llmd5Ah4yVNrGPEsqRuBTYwo8LcnME2BJ5BCpemgubn2ntOoyMjTWLo kAQ+1R9eEHoMLWuxGLgXlytBKuNOFRPZRVL/dH1fMGabBDS4m4B5AzAFXQ8DbVUS1TvV a1LbJ4LspF6ZsZc9NoFWwdPUrzdnRaUaX0p47p/ImW07gabXuksSDEj+Ze2DR4FmfPq9 +DOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Gd7i2AsWX41QgQk8Ar7YsqHX/20TGthlM9YUUXpSoxo=; b=gSPoBQTCeYaOTh6co1D3PLbC0MBvktpCp1by1P6Ysbd5DgX4En1+mwly6KXrTzZC0S fT07iEiY4wqs3buPXArFJ7a0zde2jNfwGVexfGzALBuYsqc6tRui6dR6aeOrT8cn5ys4 SY4e/QRZ51TQf1YkKK68CZ8vGHI6kQh8g2DKqN5eYUZowD0uJOY5ZAJoD5e62o4Iri4X 11ce60fZlY1KmcwqhsTptR9wTNoSCCD/xOuHFbA2XNj+e0+Oqck/czEe71Mp1cAKmP5e lsbHfKpnGDeiLDn+JS/f9qONfNRZQh+XoHbjmw4egzSKOXIHTGW0hTHoROGGui2vYOWQ ifbA== X-Gm-Message-State: APt69E0VILXfSbwWFbBLogNSq/PgkRqdHXMbuR8uidNXcn8we0AomkZ8 UEW0ljQX95HSZ8lQb34+hYdKvFIVSA== X-Google-Smtp-Source: AAOMgpesZ/MW35jhUfyT/RUEqNA3MyI4LuEbRPiJp9iag25rPhW/MlDYYBjM2IEyMYLF8EuQ/YuK5w== X-Received: by 2002:adf:8895:: with SMTP id f21-v6mr5608758wrf.250.1530112069072; Wed, 27 Jun 2018 08:07:49 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:48 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:49 +0200 Message-Id: <5e9bb490cd3cccb660e11ea0c7ca256e0fd73cc7.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 17/20] net/mlx5: add flow VXLAN-GPE item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 123 ++++++++++++++++++++++++++++++++++- 1 file changed, 120 insertions(+), 3 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index daf5b9b5a..47c55b426 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -53,6 +53,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Pattern tunnel Layer bits. */ #define MLX5_FLOW_LAYER_VXLAN (1u << 12) +#define MLX5_FLOW_LAYER_VXLAN_GPE (1u << 13) /* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ @@ -64,7 +65,8 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; MLX5_FLOW_LAYER_OUTER_L4) /* Tunnel masks. */ -#define MLX5_FLOW_LAYER_TUNNEL MLX5_FLOW_LAYER_VXLAN +#define MLX5_FLOW_LAYER_TUNNEL \ + (MLX5_FLOW_LAYER_VXLAN | MLX5_FLOW_LAYER_VXLAN_GPE) /* Inner Masks. */ #define MLX5_FLOW_LAYER_INNER_L3 \ @@ -1302,9 +1304,118 @@ mlx5_flow_item_vxlan(const struct rte_flow_item *item, struct rte_flow *flow, return size; } +/** + * Validate VXLAN-GPE layer and possibly create the Verbs specification. + * + * @param dev + * Pointer to Ethernet device. + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_vxlan_gpe(struct rte_eth_dev *dev, + const struct rte_flow_item *item, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + const struct rte_flow_item_vxlan_gpe *spec = item->spec; + const struct rte_flow_item_vxlan_gpe *mask = item->mask; + const uint32_t layers = mlx5_flow_layers(flow); + unsigned int size = sizeof(struct ibv_flow_spec_tunnel); + struct ibv_flow_spec_tunnel vxlan_gpe = { + .type = IBV_FLOW_SPEC_VXLAN_TUNNEL, + .size = size, + }; + int ret; + union vni { + uint32_t vlan_id; + uint8_t vni[4]; + } id = { .vlan_id = 0, }; + + if (!((struct priv *)dev->data->dev_private)->config.l3_vxlan_en) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 VXLAN is not enabled by device" + " parameter and/or not configured in" + " firmware"); + if (layers & MLX5_FLOW_LAYER_TUNNEL) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "a tunnel is already present"); + /* + * Verify only UDPv4 is present as defined in + * https://tools.ietf.org/html/rfc7348 + */ + if (!(layers & MLX5_FLOW_LAYER_OUTER_L4_UDP)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "no outer UDP layer found"); + if (!mask) + mask = &rte_flow_item_vxlan_gpe_mask; + ret = mlx5_flow_item_validate + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_vxlan_gpe_mask, + sizeof(struct rte_flow_item_vxlan_gpe), error); + if (ret < 0) + return ret; + if (spec) { + memcpy(&id.vni[1], spec->vni, 3); + vxlan_gpe.val.tunnel_id = id.vlan_id; + memcpy(&id.vni[1], mask->vni, 3); + vxlan_gpe.mask.tunnel_id = id.vlan_id; + if (spec->protocol) + return rte_flow_error_set + (error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VxLAN-GPE protocol not supported"); + /* Remove unwanted bits from values. */ + vxlan_gpe.val.tunnel_id &= vxlan_gpe.mask.tunnel_id; + } + /* + * Tunnel id 0 is equivalent as not adding a VXLAN layer, if only this + * layer is defined in the Verbs specification it is interpreted as + * wildcard and all packets will match this rule, if it follows a full + * stack layer (ex: eth / ipv4 / udp), all packets matching the layers + * before will also match this rule. To avoid such situation, VNI 0 + * is currently refused. + */ + if (!vxlan_gpe.val.tunnel_id) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VXLAN-GPE vni cannot be 0"); + if (!(layers & MLX5_FLOW_LAYER_OUTER)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "VXLAN-GPE tunnel must be fully" + " defined"); + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &vxlan_gpe, size); + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_VXLAN_GPE); + flow->ptype = RTE_PTYPE_TUNNEL_VXLAN_GPE | RTE_PTYPE_L4_UDP; + return size; +} + /** * Validate items provided by the user. * + * @param dev + * Pointer to Ethernet device. * @param items * Pointer to flow items array. * @param flow @@ -1319,7 +1430,8 @@ mlx5_flow_item_vxlan(const struct rte_flow_item *item, struct rte_flow *flow, * otherwise and rte_errno is set. */ static int -mlx5_flow_items(const struct rte_flow_item items[], +mlx5_flow_items(struct rte_eth_dev *dev, + const struct rte_flow_item items[], struct rte_flow *flow, const size_t flow_size, struct rte_flow_error *error) { @@ -1353,6 +1465,10 @@ mlx5_flow_items(const struct rte_flow_item items[], case RTE_FLOW_ITEM_TYPE_VXLAN: ret = mlx5_flow_item_vxlan(items, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_VXLAN_GPE: + ret = mlx5_flow_item_vxlan_gpe(dev, items, flow, + remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, @@ -1962,7 +2078,8 @@ mlx5_flow_merge(struct rte_eth_dev *dev, struct rte_flow *flow, (void *)(flow->cur_verbs->attr + 1); } ret = mlx5_flow_items - ((const struct rte_flow_item *)buf->patterns[i], + (dev, + (const struct rte_flow_item *)buf->patterns[i], flow, remain, error); if (ret < 0) goto error; From patchwork Wed Jun 27 15:07:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41692 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2E5EC1C021; Wed, 27 Jun 2018 17:08:13 +0200 (CEST) Received: from mail-wr0-f194.google.com (mail-wr0-f194.google.com [209.85.128.194]) by dpdk.org (Postfix) with ESMTP id 99F2A1BFCC for ; Wed, 27 Jun 2018 17:07:50 +0200 (CEST) Received: by mail-wr0-f194.google.com with SMTP id h10-v6so2402009wrq.8 for ; Wed, 27 Jun 2018 08:07:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=MmXU0N2J9XQdaqzbqt7S9MQnihgDYkriac/MPePP/0o=; b=dsa9NbYwVgFzHwtQT0ZMJHwRmdkzsgKjW99rC3gg1itbzHLJk3+9ohlMkbRQhsLhAt lfsiJ2yBFDDYeqHobtOb6/OEUr0PwXLJhUxy4y9gJRfy566K8YQjG4FiuTjMQ5gC13Ru eBCP3j3Kz7UCKiKqp5F3uS/21VpaBxLepSjcgmedKwZOp0wNpEzI92buWf7gen4Ozpb1 rsK21A0QL/s9wjpPIGijJ/USMXV8tdW8GY2tUAsgMpwOTB8RBerNFDqzpjzuVBNtvhgS +PiDBDU0MAjSEX4y+1SJkNDT7NQm3zeqtwpVIraMftvEDyZ8H1iCVDDQCuHDpJ4BtRwT NBaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=MmXU0N2J9XQdaqzbqt7S9MQnihgDYkriac/MPePP/0o=; b=sB8yym7ovyWim4zb9EMAPiQA31LOCGb3rTCDC9jCUMsWSxi6uXD8gMaPXnJaq/EWUw iWXva0+lZK/kxT92DVPkI/v0G+VPaKT0nQzDJ91jcYVQS8DZP27IuE+4B77IebZCfXHk 5AJPJgTe/a/1wjiwbFimHNSYBtybsFnrAgVrf14Gsd4tXKMW21aOKTdB/5kFlr50vVsG smS4Ly2PZjLcskDax6hy+NgP0BpwDme+QJCj8uax39ukHZgLwuTz3RoR56SbyuztEmNT xHIBeRTreRWxCGSKoKYSvSy4aU22VgAs4oTvD+B/VfMEvb54D02rZeoGXxtcvOQOZWW+ wF9g== X-Gm-Message-State: APt69E176J3/1CgXmOromR65dIqOkVudjOUE/CmSOIWnQfN5Nu7JYGXJ d9vECOsy3A7qbDzWT0yMg5huTANo0Q== X-Google-Smtp-Source: AAOMgpep4v16lVLopZPF+qgEgUUUEEAV2lUe99vlR6epMIxmItnHquVlBchlSDmGb+LUikD3quT9nQ== X-Received: by 2002:adf:c892:: with SMTP id k18-v6mr5831789wrh.6.1530112070029; Wed, 27 Jun 2018 08:07:50 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:49 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:50 +0200 Message-Id: <6190b2d787d3a9d2095e2eed17db1c20b2e3c4e1.1530111623.git.nelio.laranjeiro@6wind.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 18/20] net/mlx5: add flow GRE item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 191 ++++++++++++++++++++++++++++++++++- 1 file changed, 186 insertions(+), 5 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 47c55b426..636aaabe8 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -54,6 +54,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Pattern tunnel Layer bits. */ #define MLX5_FLOW_LAYER_VXLAN (1u << 12) #define MLX5_FLOW_LAYER_VXLAN_GPE (1u << 13) +#define MLX5_FLOW_LAYER_GRE (1u << 14) /* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ @@ -66,7 +67,8 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Tunnel masks. */ #define MLX5_FLOW_LAYER_TUNNEL \ - (MLX5_FLOW_LAYER_VXLAN | MLX5_FLOW_LAYER_VXLAN_GPE) + (MLX5_FLOW_LAYER_VXLAN | MLX5_FLOW_LAYER_VXLAN_GPE | \ + MLX5_FLOW_LAYER_GRE) /* Inner Masks. */ #define MLX5_FLOW_LAYER_INNER_L3 \ @@ -215,6 +217,9 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { }, }; +/* Tunnel protocol values. */ +#define MLX5_PROTOCOL_GRE 47 + /** Handles information leading to a drop fate. */ struct mlx5_flow_verbs { LIST_ENTRY(mlx5_flow_verbs) next; @@ -1005,12 +1010,23 @@ mlx5_flow_item_ipv6(const struct rte_flow_item *item, struct rte_flow *flow, item, "L3 cannot follow an L4" " layer"); + /* + * IPv6 is not recognised by the NIC inside a GRE tunnel. + * Such support has to be disabled as the rule will be + * accepted. Tested with Mellanox OFED 4.3-3.0.2.1 + */ + if (tunnel && layers & MLX5_FLOW_LAYER_GRE) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "IPv6 inside a GRE tunnel is" + " not recognised."); if (!mask) mask = &rte_flow_item_ipv6_mask; - ret = mlx5_flow_item_validate(item, (const uint8_t *)mask, - (const uint8_t *)&nic_mask, - sizeof(struct rte_flow_item_ipv6), - error); + ret = mlx5_flow_item_validate + (item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_ipv6), error); if (ret < 0) return ret; } @@ -1411,6 +1427,168 @@ mlx5_flow_item_vxlan_gpe(struct rte_eth_dev *dev, return size; } +/** + * Update the protocol in Verbs IPv4 spec. + * + * @param attr[in, out] + * Pointer to Verbs attributes structure. + * @param protocol[in] + * Protocol value to set if none is present in the specification. + */ +static void +mlx5_flow_item_gre_ipv4_protocol_update(struct ibv_flow_attr *attr, + uint8_t protocol) +{ + unsigned int i; + const enum ibv_flow_spec_type search = IBV_FLOW_SPEC_IPV4_EXT; + struct ibv_spec_header *hdr = (struct ibv_spec_header *) + ((uint8_t *)attr + sizeof(struct ibv_flow_attr)); + + if (!attr) + return; + for (i = 0; i != attr->num_of_specs; ++i) { + if (hdr->type == search) { + struct ibv_flow_spec_ipv4_ext *ip = + (struct ibv_flow_spec_ipv4_ext *)hdr; + + if (!ip->val.proto) { + ip->val.proto = protocol; + ip->mask.proto = 0xff; + } + break; + } + hdr = (struct ibv_spec_header *)((uint8_t *)hdr + hdr->size); + } +} + +/** + * Update the protocol in Verbs IPv6 spec. + * + * @param attr[in, out] + * Pointer to Verbs attributes structure. + * @param protocol[in] + * Protocol value to set if none is present in the specification. + */ +static void +mlx5_flow_item_gre_ipv6_protocol_update(struct ibv_flow_attr *attr, + uint8_t protocol) +{ + unsigned int i; + const enum ibv_flow_spec_type search = IBV_FLOW_SPEC_IPV6; + struct ibv_spec_header *hdr = (struct ibv_spec_header *) + ((uint8_t *)attr + sizeof(struct ibv_flow_attr)); + + if (!attr) + return; + for (i = 0; i != attr->num_of_specs; ++i) { + if (hdr->type == search) { + struct ibv_flow_spec_ipv6 *ip = + (struct ibv_flow_spec_ipv6 *)hdr; + + if (!ip->val.next_hdr) { + ip->val.next_hdr = protocol; + ip->mask.next_hdr = 0xff; + } + break; + } + hdr = (struct ibv_spec_header *)((uint8_t *)hdr + hdr->size); + } +} + +/** + * Validate GRE layer and possibly create the Verbs specification. + * + * @param dev + * Pointer to Ethernet device. + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_gre(const struct rte_flow_item *item, + struct rte_flow *flow, const size_t flow_size, + struct rte_flow_error *error) +{ + struct mlx5_flow_verbs *verbs = flow->cur_verbs; + const struct rte_flow_item_gre *spec = item->spec; + const struct rte_flow_item_gre *mask = item->mask; + const uint32_t layers = mlx5_flow_layers(flow); +#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT + unsigned int size = sizeof(struct ibv_flow_spec_gre); + struct ibv_flow_spec_gre tunnel = { + .type = IBV_FLOW_SPEC_GRE, + .size = size, + }; +#else + unsigned int size = sizeof(struct ibv_flow_spec_tunnel); + struct ibv_flow_spec_tunnel tunnel = { + .type = IBV_FLOW_SPEC_VXLAN_TUNNEL, + .size = size, + }; +#endif + int ret; + + if (layers & MLX5_FLOW_LAYER_TUNNEL) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "a tunnel is already present"); + if (!(layers & MLX5_FLOW_LAYER_OUTER_L3)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "L3 Layer is missing"); + if (!mask) + mask = &rte_flow_item_gre_mask; + ret = mlx5_flow_item_validate + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_gre_mask, + sizeof(struct rte_flow_item_gre), error); + if (ret < 0) + return ret; +#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT + if (spec) { + tunnel.val.c_ks_res0_ver = spec->c_rsvd0_ver; + tunnel.val.protocol = spec->protocol; + tunnel.mask.c_ks_res0_ver = mask->c_rsvd0_ver; + tunnel.mask.protocol = mask->protocol; + /* Remove unwanted bits from values. */ + tunnel.val.c_ks_res0_ver &= tunnel.mask.c_ks_res0_ver; + tunnel.val.protocol &= tunnel.mask.protocol; + tunnel.val.key &= tunnel.mask.key; + } +#else + if (spec && (spec->protocol & mask->protocol)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "without MPLS support the" + " specification cannot be used for" + " filtering"); +#endif /* !HAVE_IBV_DEVICE_MPLS_SUPPORT */ + if (size <= flow_size) { + if (layers & MLX5_FLOW_LAYER_OUTER_L3_IPV4) + mlx5_flow_item_gre_ipv4_protocol_update + (verbs->attr, MLX5_PROTOCOL_GRE); + else + mlx5_flow_item_gre_ipv6_protocol_update + (verbs->attr, MLX5_PROTOCOL_GRE); + mlx5_flow_spec_verbs_add(flow, &tunnel, size); + } + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_GRE); + flow->ptype = RTE_PTYPE_TUNNEL_GRE; + return size; +} + /** * Validate items provided by the user. * @@ -1469,6 +1647,9 @@ mlx5_flow_items(struct rte_eth_dev *dev, ret = mlx5_flow_item_vxlan_gpe(dev, items, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_GRE: + ret = mlx5_flow_item_gre(items, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jun 27 15:07:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41693 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 717AA1C027; Wed, 27 Jun 2018 17:08:14 +0200 (CEST) Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by dpdk.org (Postfix) with ESMTP id 74C721BFB3 for ; Wed, 27 Jun 2018 17:07:51 +0200 (CEST) Received: by mail-wm0-f68.google.com with SMTP id z13-v6so5887360wma.5 for ; Wed, 27 Jun 2018 08:07:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=PRNbGMzLnw+u8zm2HpUIy0dpaKmm+KntfFVkJ/PR6wU=; b=r3P78um5b/UV6E3h4D2+qEP0txvIRT9d3vh70Xw/U+bV3W4EsHn62OsoraSaP2usw0 JPPAF8iTaSsD3dux3V09mesnKSfVjJQpJppnDpm9zIvcVioq5EHm3AW9Lb5pCp291k1+ /njHinUn885pTscNWtu7J8qGZBbSQESbJanKjGgh8HEHAPNQyCBmkFWG+Vm2tafVdx7S ZbzTZpnUqH+qAdC7nxO7ZxGrF0EIdr9CMpss2QDs3emk75s4/zg+rzmF+6kyy5xrwiop CrIjG0x3kAKUGZbUEWZbnG/m05u2xIutFT2t5iaiFOu/jRi9drtwK+oEc1farjz74AaM n/2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=PRNbGMzLnw+u8zm2HpUIy0dpaKmm+KntfFVkJ/PR6wU=; b=Hc6A76vZ7jVXKUjpdAy9cerGwoYBYDTsCZ1JApJTpMfRlS2ehDTo3ox0t/y562yAaD sLxI2wh44M4xWVK6iSXbkG8d5CLT87nKlVqal2GS2Jxz7wobCT2UnuEOe1LFcVq3mkMk CThg0gspJ25im+Fb/2v29100tvXdwgi09jGiGYYTe+FMRXzidWbGYSLj0RUacDdrFbi4 b+g4x3oCZL4TRUMnVpva7Ddg1aIfKW/KOQoIO6cpZRLOyR6fgoP908Fh+2JCn36XHPYV FkkVFOYAZDvYfKJN0wMHD058I9JArKYCffJkkELbEUu9kqiuKrURZ4dEUI6D3HDOr1Ec BugQ== X-Gm-Message-State: APt69E2xb47QFGtzb6s8ojVg5/7RT8ulTPo84s/EjRFylg46F2kF/PEb DqNws3dSPJPugmYsBPBLvsBPmjpGAg== X-Google-Smtp-Source: AAOMgpdnrt2fZYdy/HDyZe0y4L4U3QlhNfr3tXzMFa/HhykwcruZ/5qsS2erekbJH8kVP5x/EECBVw== X-Received: by 2002:a1c:7153:: with SMTP id m80-v6mr5088411wmc.7.1530112070935; Wed, 27 Jun 2018 08:07:50 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:50 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:51 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 19/20] net/mlx5: add flow MPLS item X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_flow.c | 77 +++++++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 636aaabe8..7aa4e6ed5 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -55,6 +55,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; #define MLX5_FLOW_LAYER_VXLAN (1u << 12) #define MLX5_FLOW_LAYER_VXLAN_GPE (1u << 13) #define MLX5_FLOW_LAYER_GRE (1u << 14) +#define MLX5_FLOW_LAYER_MPLS (1u << 15) /* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ @@ -68,7 +69,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Tunnel masks. */ #define MLX5_FLOW_LAYER_TUNNEL \ (MLX5_FLOW_LAYER_VXLAN | MLX5_FLOW_LAYER_VXLAN_GPE | \ - MLX5_FLOW_LAYER_GRE) + MLX5_FLOW_LAYER_GRE | MLX5_FLOW_LAYER_MPLS) /* Inner Masks. */ #define MLX5_FLOW_LAYER_INNER_L3 \ @@ -1589,6 +1590,77 @@ mlx5_flow_item_gre(const struct rte_flow_item *item, return size; } +/** + * Validate MPLS layer and possibly create the Verbs specification. + * + * @param dev + * Pointer to Ethernet device. + * @param item[in] + * Item specification. + * @param flow[in, out] + * Pointer to flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_item_mpls(const struct rte_flow_item *item __rte_unused, + struct rte_flow *flow __rte_unused, + const size_t flow_size __rte_unused, + struct rte_flow_error *error) +{ +#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT + const struct rte_flow_item_mpls *spec = item->spec; + const struct rte_flow_item_mpls *mask = item->mask; + unsigned int size = sizeof(struct ibv_flow_spec_mpls); + struct ibv_flow_spec_mpls mpls = { + .type = IBV_FLOW_SPEC_MPLS, + .size = size, + }; + const uint32_t layers = mlx5_flow_layers(flow); + int ret; + + if (layers & MLX5_FLOW_LAYER_TUNNEL) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "a tunnel is already" + " present"); + if (!mask) + mask = &rte_flow_item_mpls_mask; + ret = mlx5_flow_item_validate + (item, (const uint8_t *)mask, + (const uint8_t *)&rte_flow_item_mpls_mask, + sizeof(struct rte_flow_item_mpls), error); + if (ret < 0) + return ret; + if (spec) { + memcpy(&mpls.val.label, spec, sizeof(mpls.val.label)); + memcpy(&mpls.mask.label, mask, sizeof(mpls.mask.label)); + /* Remove unwanted bits from values. */ + mpls.val.label &= mpls.mask.label; + } + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &mpls, size); + mlx5_flow_layers_update(flow, MLX5_FLOW_LAYER_MPLS); + if (layers & MLX5_FLOW_LAYER_OUTER_L4_UDP) + flow->ptype = RTE_PTYPE_TUNNEL_MPLS_IN_GRE | RTE_PTYPE_L4_UDP; + else + flow->ptype = RTE_PTYPE_TUNNEL_MPLS_IN_GRE; + return size; +#endif /* !HAVE_IBV_DEVICE_MPLS_SUPPORT */ + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + item, + "MPLS is not supported by Verbs, please" + " update."); +} + /** * Validate items provided by the user. * @@ -1650,6 +1722,9 @@ mlx5_flow_items(struct rte_eth_dev *dev, case RTE_FLOW_ITEM_TYPE_GRE: ret = mlx5_flow_item_gre(items, flow, remain, error); break; + case RTE_FLOW_ITEM_TYPE_MPLS: + ret = mlx5_flow_item_mpls(items, flow, remain, error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, From patchwork Wed Jun 27 15:07:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?N=C3=A9lio_Laranjeiro?= X-Patchwork-Id: 41694 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 940761C02D; Wed, 27 Jun 2018 17:08:15 +0200 (CEST) Received: from mail-wr0-f171.google.com (mail-wr0-f171.google.com [209.85.128.171]) by dpdk.org (Postfix) with ESMTP id 8936F1BFB3 for ; Wed, 27 Jun 2018 17:07:52 +0200 (CEST) Received: by mail-wr0-f171.google.com with SMTP id h10-v6so2402099wrq.8 for ; Wed, 27 Jun 2018 08:07:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=i+7dXZTu+kAwCGAzc5tAnD6uziD6l5YID1kqtZ1r89s=; b=cR+rV83T2efHvxu7joBazNoouPQYf8O9LjRoJqYUIfFS0y4WFKQS4PCOfOP64IMf8N 2ZBiNNkLlthojuuACFUsArz/6S4sPTG33fsKQWuqpGG4pJm1OrIKFrRi5uyMOBr0vvSw ORSD1TMlyq0cmaEjdAJepgOz8pB5wyft2ljTgMpkGYw1PbNAzB7lEgn7WEo9EgEYlOEE tLhD1xMCgF1JGifg4dkRki8YRZ7fRQ5qiQq20VtY4kKvSlt1Oolg7ly03fA/pSFSJfsk gSrlCuRN+A3naO59QVC9njES566hFswHAiIdVktOTehjB0QSucDQYJscRcJgeczA0TCX lOTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=i+7dXZTu+kAwCGAzc5tAnD6uziD6l5YID1kqtZ1r89s=; b=RpGwqB4GbuTbBZUplShtMjjkO06tpveLHLz0g6AdrQOb68PRa7Wztq6NeDY3ORP/jd 1FgPGOuHlQMPSUGRrcg4BpoLrZrZNZB+nQlmAdDLuQfWiPOHr8MDBpTQ3Ypcia6gDHp1 oYr4aOX/yjEqvIx+JgmfS0d2uWy1ae24vATWhBCXAZcFgAdNZyzEu1UfROIa5gxmYTb7 DUmcK6Uc0OdA3xqe/rCeaa/gxmkuLT83VGlPsSaH+kqjTDRHWswHrl/ESOe+ydryr4pX cNjjyOirxIpCdCmYY5J34VM3XMksg+aQNhVIl8s3fv7GOmNVcbGYOgL3bCqPuytKL+Fk z56g== X-Gm-Message-State: APt69E2Sok4Wg3wAV8sU3AyWpM3KYYWEiQtXnOg/heaS+3Mhd3eeqtp/ h6Jsg/naBUfhYzYM9cuk7WEh32KOmw== X-Google-Smtp-Source: AAOMgpfqCB4z9bAwlaRPHa/CUOqE7TZxRMcKZYXRtdsZggF9FRqjL73CsKWtx+23LplwDFNIsVzqQg== X-Received: by 2002:a5d:4c4c:: with SMTP id n12-v6mr5405724wrt.71.1530112071985; Wed, 27 Jun 2018 08:07:51 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id k17-v6sm4872513wrp.19.2018.06.27.08.07.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 08:07:51 -0700 (PDT) From: Nelio Laranjeiro To: dev@dpdk.org, Adrien Mazarguil , Yongseok Koh Date: Wed, 27 Jun 2018 17:07:52 +0200 Message-Id: X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 20/20] net/mlx5: add count flow action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is only supported by Mellanox OFED. Signed-off-by: Nelio Laranjeiro Acked-by: Yongseok Koh --- drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_flow.c | 238 +++++++++++++++++++++++++++++++++++ 2 files changed, 240 insertions(+) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index ed8c1c9a2..1d8e156c8 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -181,6 +181,8 @@ struct priv { struct mlx5_drop drop; /* Flow drop queues. */ struct mlx5_flows flows; /* RTE Flow rules. */ struct mlx5_flows ctrl_flows; /* Control flow rules. */ + LIST_HEAD(counters, mlx5_flow_counter) flow_counters; + /* Flow counters. */ struct { uint32_t dev_gen; /* Generation number to flush local caches. */ rte_rwlock_t rwlock; /* MR Lock. */ diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 7aa4e6ed5..9241855be 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -88,6 +88,7 @@ extern const struct eth_dev_ops mlx5_dev_ops_isolate; /* Modify a packet. */ #define MLX5_FLOW_MOD_FLAG (1u << 0) #define MLX5_FLOW_MOD_MARK (1u << 1) +#define MLX5_FLOW_MOD_COUNT (1u << 2) /* Priority reserved for default flows. */ #define MLX5_FLOW_PRIO_RSVD ((uint32_t)-1) @@ -239,6 +240,17 @@ struct mlx5_flow_verbs { uint64_t hash_fields; /**< Verbs hash Rx queue hash fields. */ }; +/* Counters information. */ +struct mlx5_flow_counter { + LIST_ENTRY(mlx5_flow_counter) next; /**< Pointer to the next counter. */ + uint32_t shared:1; /**< Share counter ID with other flow rules. */ + uint32_t ref_cnt:31; /**< Reference counter. */ + uint32_t id; /**< Counter ID. */ + struct ibv_counter_set *cs; /**< Holds the counters for the rule. */ + uint64_t hits; /**< Number of packets matched by the rule. */ + uint64_t bytes; /**< Number of bytes matched by the rule. */ +}; + /* Flow structure. */ struct rte_flow { TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ @@ -253,6 +265,7 @@ struct rte_flow { LIST_HEAD(verbs, mlx5_flow_verbs) verbs; /**< Verbs flows list. */ struct mlx5_flow_verbs *cur_verbs; /**< Current Verbs flow structure being filled. */ + struct mlx5_flow_counter *counter; /**< Holds Verbs flow counter. */ struct rte_flow_action_rss rss;/**< RSS context. */ uint32_t ptype; /**< Store tunnel packet type data to store in Rx queue. */ @@ -266,6 +279,7 @@ static const struct rte_flow_ops mlx5_flow_ops = { .destroy = mlx5_flow_destroy, .flush = mlx5_flow_flush, .isolate = mlx5_flow_isolate, + .query = mlx5_flow_query, }; /* Convert FDIR request to Generic flow. */ @@ -407,6 +421,81 @@ mlx5_flow_priority(struct rte_eth_dev *dev, uint32_t priority, return priority; } +/** + * Get a flow counter. + * + * @param dev + * Pointer to Ethernet device. + * @param id + * Counter identifier. + * + * @return + * A pointer to the counter, NULL otherwise and rte_errno is set. + */ +static struct mlx5_flow_counter * +mlx5_flow_counter_new(struct rte_eth_dev *dev, uint32_t shared, uint32_t id) +{ + struct priv *priv = dev->data->dev_private; + struct mlx5_flow_counter *cnt; + + LIST_FOREACH(cnt, &priv->flow_counters, next) { + if (cnt->shared != shared) + continue; + if (cnt->id != id) + continue; + cnt->ref_cnt++; + return cnt; + } +#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT + + struct mlx5_flow_counter tmpl = { + .shared = shared, + .id = id, + .cs = mlx5_glue->create_counter_set + (priv->ctx, + &(struct ibv_counter_set_init_attr){ + .counter_set_id = id, + }), + .hits = 0, + .bytes = 0, + }; + + if (!tmpl.cs) { + rte_errno = errno; + return NULL; + } + cnt = rte_calloc(__func__, 1, sizeof(*cnt), 0); + if (!cnt) { + rte_errno = ENOMEM; + return NULL; + } + *cnt = tmpl; + LIST_INSERT_HEAD(&priv->flow_counters, cnt, next); + return cnt; +#endif + rte_errno = ENOTSUP; + return NULL; +} + +/** + * Release a flow counter. + * + * @param id + * Counter identifier. + * + * @return + * A pointer to the counter, NULL otherwise and rte_errno is set. + */ +static void +mlx5_flow_counter_release(struct mlx5_flow_counter *counter) +{ + if (--counter->ref_cnt == 0) { + claim_zero(mlx5_glue->destroy_counter_set(counter->cs)); + LIST_REMOVE(counter, next); + rte_free(counter); + } +} + /** * Flow debug purpose function only available when * CONFIG_RTE_LIBRTE_MLX5_DEBUG=y @@ -2169,6 +2258,65 @@ mlx5_flow_action_mark(const struct rte_flow_action *actions, return size; } +/** + * Validate action count provided by the user. + * + * @param dev + * Pointer to Ethernet device. + * @param actions + * Pointer to flow actions array. + * @param flow + * Pointer to the rte_flow structure. + * @param flow_size[in] + * Size in bytes of the available space for to store the flow information. + * @param error + * Pointer to error structure. + * + * @return + * size in bytes necessary for the conversion, a negative errno value + * otherwise and rte_errno is set. + */ +static int +mlx5_flow_action_count(struct rte_eth_dev *dev, + const struct rte_flow_action *actions, + struct rte_flow *flow, + const size_t flow_size __rte_unused, + struct rte_flow_error *error) +{ + const struct rte_flow_action_count *count = actions->conf; +#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT + unsigned int size = sizeof(struct ibv_flow_spec_counter_action); + struct ibv_flow_spec_counter_action counter = { + .type = IBV_FLOW_SPEC_ACTION_COUNT, + .size = size, + }; +#endif + + if (!flow->counter) { + flow->counter = mlx5_flow_counter_new(dev, count->shared, + count->id); + if (!flow->counter) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "cannot get counter" + " context."); + } + if (!((struct priv *)dev->data->dev_private)->config.flow_counter_en) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "flow counters are not supported."); + flow->modifier |= MLX5_FLOW_MOD_COUNT; +#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT + counter.counter_set_handle = flow->counter->cs->handle; + if (size <= flow_size) + mlx5_flow_spec_verbs_add(flow, &counter, size); + return size; +#endif + return 0; +} + /** * Validate actions provided by the user. * @@ -2228,6 +2376,10 @@ mlx5_flow_actions(struct rte_eth_dev *dev, case RTE_FLOW_ACTION_TYPE_RSS: ret = mlx5_flow_action_rss(dev, actions, flow, error); break; + case RTE_FLOW_ACTION_TYPE_COUNT: + ret = mlx5_flow_action_count(dev, actions, flow, remain, + error); + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -2417,6 +2569,10 @@ mlx5_flow_fate_remove(struct rte_eth_dev *dev, struct rte_flow *flow) verbs->hrxq = NULL; } } + if (flow->counter) { + mlx5_flow_counter_release(flow->counter); + flow->counter = NULL; + } } /** @@ -2974,6 +3130,88 @@ mlx5_flow_isolate(struct rte_eth_dev *dev, return 0; } +/** + * Query flow counter. + * + * @param flow + * Pointer to the flow. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_flow_query_count(struct rte_flow *flow __rte_unused, + void *data __rte_unused, + struct rte_flow_error *error) +{ +#ifdef HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT + struct rte_flow_query_count *qc = data; + uint64_t counters[2] = {0, 0}; + struct ibv_query_counter_set_attr query_cs_attr = { + .cs = flow->counter->cs, + .query_flags = IBV_COUNTER_SET_FORCE_UPDATE, + }; + struct ibv_counter_set_data query_out = { + .out = counters, + .outlen = 2 * sizeof(uint64_t), + }; + int err = mlx5_glue->query_counter_set(&query_cs_attr, &query_out); + + if (err) + return rte_flow_error_set(error, err, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot read counter"); + qc->hits_set = 1; + qc->bytes_set = 1; + qc->hits = counters[0] - flow->counter->hits; + qc->bytes = counters[1] - flow->counter->bytes; + if (qc->reset) { + flow->counter->hits = counters[0]; + flow->counter->bytes = counters[1]; + } + return 0; +#endif + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "counters are not available"); +} + +/** + * Query a flows. + * + * @see rte_flow_query() + * @see rte_flow_ops + */ +int +mlx5_flow_query(struct rte_eth_dev *dev __rte_unused, + struct rte_flow *flow, + const struct rte_flow_action *actions, + void *data, + struct rte_flow_error *error) +{ + int ret = 0; + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_VOID: + break; + case RTE_FLOW_ACTION_TYPE_COUNT: + ret = mlx5_flow_query_count(flow, data, error); + break; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + actions, + "action not supported"); + } + if (ret < 0) + return ret; + } + return 0; +} + /** * Convert a flow director filter to a generic flow. *