From patchwork Tue Nov 5 08:01:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62420 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 510C3A0352; Tue, 5 Nov 2019 09:02:09 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2C82A4C8B; Tue, 5 Nov 2019 09:02:05 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 5B9AB4C8B for ; Tue, 5 Nov 2019 09:02:04 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:00 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA5820Si026355; Tue, 5 Nov 2019 10:02:00 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA5820bn030753; Tue, 5 Nov 2019 08:02:00 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA5820V6030752; Tue, 5 Nov 2019 08:02:00 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com Date: Tue, 5 Nov 2019 08:01:36 +0000 Message-Id: <1572940915-29416-2-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 01/20] net/mlx5: convert internal tag endianness X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Public API RTE_FLOW_ACTION_TYPE_TAG and RTE_FLOW_ITEM_TYPE_TAG present data in host-endian format, as all metadata related entities. The internal mlx5 tag related action and item should use the same endianness to be conformed. Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow.c | 6 +++--- drivers/net/mlx5/mlx5_flow.h | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index b4b08f4..5408797 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -2707,7 +2707,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, actions_rx++; set_tag = (void *)actions_rx; set_tag->id = flow_get_reg_id(dev, MLX5_HAIRPIN_RX, 0, &error); - set_tag->data = rte_cpu_to_be_32(*flow_id); + set_tag->data = *flow_id; tag_action->conf = set_tag; /* Create Tx item list. */ rte_memcpy(actions_tx, actions, sizeof(struct rte_flow_action)); @@ -2715,8 +2715,8 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, item = pattern_tx; item->type = MLX5_RTE_FLOW_ITEM_TYPE_TAG; tag_item = (void *)addr; - tag_item->data = rte_cpu_to_be_32(*flow_id); - tag_item->id = flow_get_reg_id(dev, MLX5_HAIRPIN_TX, 0, &error); + tag_item->data = *flow_id; + tag_item->id = flow_get_reg_id(dev, MLX5_HAIRPIN_TX, 0, NULL); item->spec = tag_item; addr += sizeof(struct mlx5_rte_flow_item_tag); tag_item = (void *)addr; diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 7559810..8cc6c47 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -56,13 +56,13 @@ enum mlx5_rte_flow_action_type { /* Matches on selected register. */ struct mlx5_rte_flow_item_tag { uint16_t id; - rte_be32_t data; + uint32_t data; }; /* Modify selected register. */ struct mlx5_rte_flow_action_set_tag { uint16_t id; - rte_be32_t data; + uint32_t data; }; /* Matches on source queue. */ From patchwork Tue Nov 5 08:01:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62421 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 83C7FA0352; Tue, 5 Nov 2019 09:02:18 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D39354CBD; Tue, 5 Nov 2019 09:02:06 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 5D8374C8E for ; Tue, 5 Nov 2019 09:02:04 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:02 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA5822Q5026438; Tue, 5 Nov 2019 10:02:02 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA5822gi030755; Tue, 5 Nov 2019 08:02:02 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA5822Zm030754; Tue, 5 Nov 2019 08:02:02 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:37 +0000 Message-Id: <1572940915-29416-3-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 02/20] net/mlx5: update modify header action translator X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When composing device command for modify header action, provided mask should be taken more accurate into account thus length and offset in action should be set accordingly at precise bit-wise boundaries. For the future use, metadata register copy action is also added. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow_dv.c | 148 ++++++++++++++++++++++++++++++---------- drivers/net/mlx5/mlx5_prm.h | 18 +++-- 2 files changed, 126 insertions(+), 40 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 42c265f..3df2609 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -240,12 +240,62 @@ struct field_modify_info modify_tcp[] = { } /** + * Fetch 1, 2, 3 or 4 byte field from the byte array + * and return as unsigned integer in host-endian format. + * + * @param[in] data + * Pointer to data array. + * @param[in] size + * Size of field to extract. + * + * @return + * converted field in host endian format. + */ +static inline uint32_t +flow_dv_fetch_field(const uint8_t *data, uint32_t size) +{ + uint32_t ret; + + switch (size) { + case 1: + ret = *data; + break; + case 2: + ret = rte_be_to_cpu_16(*(const unaligned_uint16_t *)data); + break; + case 3: + ret = rte_be_to_cpu_16(*(const unaligned_uint16_t *)data); + ret = (ret << 8) | *(data + sizeof(uint16_t)); + break; + case 4: + ret = rte_be_to_cpu_32(*(const unaligned_uint32_t *)data); + break; + default: + assert(false); + ret = 0; + break; + } + return ret; +} + +/** * Convert modify-header action to DV specification. * + * Data length of each action is determined by provided field description + * and the item mask. Data bit offset and width of each action is determined + * by provided item mask. + * * @param[in] item * Pointer to item specification. * @param[in] field * Pointer to field modification information. + * For MLX5_MODIFICATION_TYPE_SET specifies destination field. + * For MLX5_MODIFICATION_TYPE_ADD specifies destination field. + * For MLX5_MODIFICATION_TYPE_COPY specifies source field. + * @param[in] dcopy + * Destination field info for MLX5_MODIFICATION_TYPE_COPY in @type. + * Negative offset value sets the same offset as source offset. + * size field is ignored, value is taken from source field. * @param[in,out] resource * Pointer to the modify-header resource. * @param[in] type @@ -259,38 +309,66 @@ struct field_modify_info modify_tcp[] = { static int flow_dv_convert_modify_action(struct rte_flow_item *item, struct field_modify_info *field, + struct field_modify_info *dcopy, struct mlx5_flow_dv_modify_hdr_resource *resource, - uint32_t type, - struct rte_flow_error *error) + uint32_t type, struct rte_flow_error *error) { uint32_t i = resource->actions_num; struct mlx5_modification_cmd *actions = resource->actions; - const uint8_t *spec = item->spec; - const uint8_t *mask = item->mask; - uint32_t set; - - while (field->size) { - set = 0; - /* Generate modify command for each mask segment. */ - memcpy(&set, &mask[field->offset], field->size); - if (set) { - if (i >= MLX5_MODIFY_NUM) - return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, NULL, - "too many items to modify"); - actions[i].action_type = type; - actions[i].field = field->id; - actions[i].length = field->size == - 4 ? 0 : field->size * 8; - rte_memcpy(&actions[i].data[4 - field->size], - &spec[field->offset], field->size); - actions[i].data0 = rte_cpu_to_be_32(actions[i].data0); - ++i; + + /* + * The item and mask are provided in big-endian format. + * The fields should be presented as in big-endian format either. + * Mask must be always present, it defines the actual field width. + */ + assert(item->mask); + assert(field->size); + do { + unsigned int size_b; + unsigned int off_b; + uint32_t mask; + uint32_t data; + + if (i >= MLX5_MODIFY_NUM) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "too many items to modify"); + /* Fetch variable byte size mask from the array. */ + mask = flow_dv_fetch_field((const uint8_t *)item->mask + + field->offset, field->size); + if (!mask) + continue; + /* Deduce actual data width in bits from mask value. */ + off_b = rte_bsf32(mask); + size_b = sizeof(uint32_t) * CHAR_BIT - + off_b - __builtin_clz(mask); + assert(size_b); + size_b = size_b == sizeof(uint32_t) * CHAR_BIT ? 0 : size_b; + actions[i].action_type = type; + actions[i].field = field->id; + actions[i].offset = off_b; + actions[i].length = size_b; + /* Convert entire record to expected big-endian format. */ + actions[i].data0 = rte_cpu_to_be_32(actions[i].data0); + if (type == MLX5_MODIFICATION_TYPE_COPY) { + assert(dcopy); + actions[i].dst_field = dcopy->id; + actions[i].dst_offset = + (int)dcopy->offset < 0 ? off_b : dcopy->offset; + /* Convert entire record to big-endian format. */ + actions[i].data1 = rte_cpu_to_be_32(actions[i].data1); + } else { + assert(item->spec); + data = flow_dv_fetch_field((const uint8_t *)item->spec + + field->offset, field->size); + /* Shift out the trailing masked bits from data. */ + data = (data & mask) >> off_b; + actions[i].data1 = rte_cpu_to_be_32(data); } - if (resource->actions_num != i) - resource->actions_num = i; - field++; - } + ++i; + ++field; + } while (field->size); + resource->actions_num = i; if (!resource->actions_num) return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION, NULL, @@ -334,7 +412,7 @@ struct field_modify_info modify_tcp[] = { } item.spec = &ipv4; item.mask = &ipv4_mask; - return flow_dv_convert_modify_action(&item, modify_ipv4, resource, + return flow_dv_convert_modify_action(&item, modify_ipv4, NULL, resource, MLX5_MODIFICATION_TYPE_SET, error); } @@ -380,7 +458,7 @@ struct field_modify_info modify_tcp[] = { } item.spec = &ipv6; item.mask = &ipv6_mask; - return flow_dv_convert_modify_action(&item, modify_ipv6, resource, + return flow_dv_convert_modify_action(&item, modify_ipv6, NULL, resource, MLX5_MODIFICATION_TYPE_SET, error); } @@ -426,7 +504,7 @@ struct field_modify_info modify_tcp[] = { } item.spec = ð item.mask = ð_mask; - return flow_dv_convert_modify_action(&item, modify_eth, resource, + return flow_dv_convert_modify_action(&item, modify_eth, NULL, resource, MLX5_MODIFICATION_TYPE_SET, error); } @@ -540,7 +618,7 @@ struct field_modify_info modify_tcp[] = { item.mask = &tcp_mask; field = modify_tcp; } - return flow_dv_convert_modify_action(&item, field, resource, + return flow_dv_convert_modify_action(&item, field, NULL, resource, MLX5_MODIFICATION_TYPE_SET, error); } @@ -600,7 +678,7 @@ struct field_modify_info modify_tcp[] = { item.mask = &ipv6_mask; field = modify_ipv6; } - return flow_dv_convert_modify_action(&item, field, resource, + return flow_dv_convert_modify_action(&item, field, NULL, resource, MLX5_MODIFICATION_TYPE_SET, error); } @@ -657,7 +735,7 @@ struct field_modify_info modify_tcp[] = { item.mask = &ipv6_mask; field = modify_ipv6; } - return flow_dv_convert_modify_action(&item, field, resource, + return flow_dv_convert_modify_action(&item, field, NULL, resource, MLX5_MODIFICATION_TYPE_ADD, error); } @@ -702,7 +780,7 @@ struct field_modify_info modify_tcp[] = { item.type = RTE_FLOW_ITEM_TYPE_TCP; item.spec = &tcp; item.mask = &tcp_mask; - return flow_dv_convert_modify_action(&item, modify_tcp, resource, + return flow_dv_convert_modify_action(&item, modify_tcp, NULL, resource, MLX5_MODIFICATION_TYPE_ADD, error); } @@ -747,7 +825,7 @@ struct field_modify_info modify_tcp[] = { item.type = RTE_FLOW_ITEM_TYPE_TCP; item.spec = &tcp; item.mask = &tcp_mask; - return flow_dv_convert_modify_action(&item, modify_tcp, resource, + return flow_dv_convert_modify_action(&item, modify_tcp, NULL, resource, MLX5_MODIFICATION_TYPE_ADD, error); } diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index 96b9166..b9e53f5 100644 --- a/drivers/net/mlx5/mlx5_prm.h +++ b/drivers/net/mlx5/mlx5_prm.h @@ -383,11 +383,12 @@ struct mlx5_cqe { /* CQE format value. */ #define MLX5_COMPRESSED 0x3 -/* Write a specific data value to a field. */ -#define MLX5_MODIFICATION_TYPE_SET 1 - -/* Add a specific data value to a field. */ -#define MLX5_MODIFICATION_TYPE_ADD 2 +/* Action type of header modification. */ +enum { + MLX5_MODIFICATION_TYPE_SET = 0x1, + MLX5_MODIFICATION_TYPE_ADD = 0x2, + MLX5_MODIFICATION_TYPE_COPY = 0x3, +}; /* The field of packet to be modified. */ enum mlx5_modification_field { @@ -470,6 +471,13 @@ struct mlx5_modification_cmd { union { uint32_t data1; uint8_t data[4]; + struct { + unsigned int rsvd2:8; + unsigned int dst_offset:5; + unsigned int rsvd3:3; + unsigned int dst_field:12; + unsigned int rsvd4:4; + }; }; }; From patchwork Tue Nov 5 08:01:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62422 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5B715A0352; Tue, 5 Nov 2019 09:02:29 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 55A6C44C7; Tue, 5 Nov 2019 09:02:10 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 4CD2C37B0 for ; Tue, 5 Nov 2019 09:02:09 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:03 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA5823oj026456; Tue, 5 Nov 2019 10:02:03 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA5823ON030757; Tue, 5 Nov 2019 08:02:03 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA5823VK030756; Tue, 5 Nov 2019 08:02:03 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:38 +0000 Message-Id: <1572940915-29416-4-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 03/20] net/mlx5: add metadata register copy X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add flow metadata register copy action which is supported through modify header command. As it is an internal action, not exposed to users, item type (MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG) is negative value. This can be used when creating PMD internal subflows. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow.h | 13 +++++++---- drivers/net/mlx5/mlx5_flow_dv.c | 50 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 58 insertions(+), 5 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 8cc6c47..170192d 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -47,24 +47,30 @@ enum mlx5_rte_flow_item_type { MLX5_RTE_FLOW_ITEM_TYPE_TX_QUEUE, }; -/* Private rte flow actions. */ +/* Private (internal) rte flow actions. */ enum mlx5_rte_flow_action_type { MLX5_RTE_FLOW_ACTION_TYPE_END = INT_MIN, MLX5_RTE_FLOW_ACTION_TYPE_TAG, + MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG, }; /* Matches on selected register. */ struct mlx5_rte_flow_item_tag { - uint16_t id; + enum modify_reg id; uint32_t data; }; /* Modify selected register. */ struct mlx5_rte_flow_action_set_tag { - uint16_t id; + enum modify_reg id; uint32_t data; }; +struct mlx5_flow_action_copy_mreg { + enum modify_reg dst; + enum modify_reg src; +}; + /* Matches on source queue. */ struct mlx5_rte_flow_item_tx_queue { uint32_t queue; @@ -227,7 +233,6 @@ struct mlx5_rte_flow_item_tx_queue { #define MLX5_FLOW_VLAN_ACTIONS (MLX5_FLOW_ACTION_OF_POP_VLAN | \ MLX5_FLOW_ACTION_OF_PUSH_VLAN) - #ifndef IPPROTO_MPLS #define IPPROTO_MPLS 137 #endif diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 3df2609..930f088 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -861,7 +861,7 @@ struct field_modify_info modify_tcp[] = { const struct rte_flow_action *action, struct rte_flow_error *error) { - const struct mlx5_rte_flow_action_set_tag *conf = (action->conf); + const struct mlx5_rte_flow_action_set_tag *conf = action->conf; struct mlx5_modification_cmd *actions = resource->actions; uint32_t i = resource->actions_num; @@ -883,6 +883,47 @@ struct field_modify_info modify_tcp[] = { } /** + * Convert internal COPY_REG action to DV specification. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in,out] res + * Pointer to the modify-header resource. + * @param[in] action + * Pointer to action specification. + * @param[out] error + * Pointer to the error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_convert_action_copy_mreg(struct rte_eth_dev *dev __rte_unused, + struct mlx5_flow_dv_modify_hdr_resource *res, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + const struct mlx5_flow_action_copy_mreg *conf = action->conf; + uint32_t mask = RTE_BE32(UINT32_MAX); + struct rte_flow_item item = { + .spec = NULL, + .mask = &mask, + }; + struct field_modify_info reg_src[] = { + {4, 0, reg_to_field[conf->src]}, + {0, 0, 0}, + }; + struct field_modify_info reg_dst = { + .offset = (uint32_t)-1, /* Same as src. */ + .id = reg_to_field[conf->dst], + }; + return flow_dv_convert_modify_action(&item, + reg_src, ®_dst, res, + MLX5_MODIFICATION_TYPE_COPY, + error); +} + +/** * Validate META item. * * @param[in] dev @@ -3949,6 +3990,7 @@ struct field_modify_info modify_tcp[] = { MLX5_FLOW_ACTION_DEC_TCP_ACK; break; case MLX5_RTE_FLOW_ACTION_TYPE_TAG: + case MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG: break; default: return rte_flow_error_set(error, ENOTSUP, @@ -5945,6 +5987,12 @@ struct field_modify_info modify_tcp[] = { return -rte_errno; action_flags |= MLX5_FLOW_ACTION_SET_TAG; break; + case MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG: + if (flow_dv_convert_action_copy_mreg(dev, &res, + actions, error)) + return -rte_errno; + action_flags |= MLX5_FLOW_ACTION_SET_TAG; + break; case RTE_FLOW_ACTION_TYPE_END: actions_end = true; if (action_flags & MLX5_FLOW_MODIFY_HDR_ACTIONS) { From patchwork Tue Nov 5 08:01:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62426 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B1D10A0352; Tue, 5 Nov 2019 09:03:17 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5F1FA1BE95; Tue, 5 Nov 2019 09:02:17 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 9F61B5B3A for ; Tue, 5 Nov 2019 09:02:09 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:05 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582566026462; Tue, 5 Nov 2019 10:02:05 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA5825ZH030759; Tue, 5 Nov 2019 08:02:05 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA5825IM030758; Tue, 5 Nov 2019 08:02:05 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:39 +0000 Message-Id: <1572940915-29416-5-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 04/20] net/mlx5: refactor flow structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Some rte_flow fields which are local to subflows have been moved to mlx5_flow structure. RSS attributes are grouped by mlx5_flow_rss structure. tag_resource is moved to mlx5_flow_dv structure. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow.c | 18 +++++--- drivers/net/mlx5/mlx5_flow.h | 25 ++++++----- drivers/net/mlx5/mlx5_flow_dv.c | 89 ++++++++++++++++++++------------------ drivers/net/mlx5/mlx5_flow_verbs.c | 55 ++++++++++++----------- 4 files changed, 105 insertions(+), 82 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 5408797..d1661f2 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -612,7 +612,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, unsigned int i; for (i = 0; i != flow->rss.queue_num; ++i) { - int idx = (*flow->queue)[i]; + int idx = (*flow->rss.queue)[i]; struct mlx5_rxq_ctrl *rxq_ctrl = container_of((*priv->rxqs)[idx], struct mlx5_rxq_ctrl, rxq); @@ -676,7 +676,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, assert(dev->data->dev_started); for (i = 0; i != flow->rss.queue_num; ++i) { - int idx = (*flow->queue)[i]; + int idx = (*flow->rss.queue)[i]; struct mlx5_rxq_ctrl *rxq_ctrl = container_of((*priv->rxqs)[idx], struct mlx5_rxq_ctrl, rxq); @@ -2815,13 +2815,20 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, goto error_before_flow; } flow->drv_type = flow_get_drv_type(dev, attr); - flow->ingress = attr->ingress; - flow->transfer = attr->transfer; if (hairpin_id != 0) flow->hairpin_flow_id = hairpin_id; assert(flow->drv_type > MLX5_FLOW_TYPE_MIN && flow->drv_type < MLX5_FLOW_TYPE_MAX); - flow->queue = (void *)(flow + 1); + flow->rss.queue = (void *)(flow + 1); + if (rss) { + /* + * The following information is required by + * mlx5_flow_hashfields_adjust() in advance. + */ + flow->rss.level = rss->level; + /* RSS type 0 indicates default RSS type (ETH_RSS_IP). */ + flow->rss.types = !rss->types ? ETH_RSS_IP : rss->types; + } LIST_INIT(&flow->dev_flows); if (rss && rss->types) { unsigned int graph_root; @@ -2861,6 +2868,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, if (!dev_flow) goto error; dev_flow->flow = flow; + dev_flow->external = 0; LIST_INSERT_HEAD(&flow->dev_flows, dev_flow, next); ret = flow_drv_translate(dev, dev_flow, &attr_tx, items_tx.items, diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 170192d..b9a9507 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -417,7 +417,6 @@ struct mlx5_flow_dv_push_vlan_action_resource { /* DV flows structure. */ struct mlx5_flow_dv { - uint64_t hash_fields; /**< Fields that participate in the hash. */ struct mlx5_hrxq *hrxq; /**< Hash Rx queues. */ /* Flow DV api: */ struct mlx5_flow_dv_matcher *matcher; /**< Cache to matcher. */ @@ -436,6 +435,8 @@ struct mlx5_flow_dv { /**< Structure for VF VLAN workaround. */ struct mlx5_flow_dv_push_vlan_action_resource *push_vlan_res; /**< Pointer to push VLAN action resource in cache. */ + struct mlx5_flow_dv_tag_resource *tag_resource; + /**< pointer to the tag action. */ #ifdef HAVE_IBV_FLOW_DV_SUPPORT void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS]; /**< Action list. */ @@ -460,11 +461,18 @@ struct mlx5_flow_verbs { }; struct ibv_flow *flow; /**< Verbs flow pointer. */ struct mlx5_hrxq *hrxq; /**< Hash Rx queue object. */ - uint64_t hash_fields; /**< Verbs hash Rx queue hash fields. */ struct mlx5_vf_vlan vf_vlan; /**< Structure for VF VLAN workaround. */ }; +struct mlx5_flow_rss { + uint32_t level; + uint32_t queue_num; /**< Number of entries in @p queue. */ + uint64_t types; /**< Specific RSS hash types (see ETH_RSS_*). */ + uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */ + uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */ +}; + /** Device flow structure. */ struct mlx5_flow { LIST_ENTRY(mlx5_flow) next; @@ -473,6 +481,10 @@ struct mlx5_flow { /**< Bit-fields of present layers, see MLX5_FLOW_LAYER_*. */ uint64_t actions; /**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */ + uint64_t hash_fields; /**< Verbs hash Rx queue hash fields. */ + uint8_t ingress; /**< 1 if the flow is ingress. */ + uint32_t group; /**< The group index. */ + uint8_t transfer; /**< 1 if the flow is E-Switch flow. */ union { #ifdef HAVE_IBV_FLOW_DV_SUPPORT struct mlx5_flow_dv dv; @@ -486,18 +498,11 @@ struct mlx5_flow { struct rte_flow { TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */ enum mlx5_flow_drv_type drv_type; /**< Driver type. */ + struct mlx5_flow_rss rss; /**< RSS context. */ struct mlx5_flow_counter *counter; /**< Holds flow counter. */ - struct mlx5_flow_dv_tag_resource *tag_resource; - /**< pointer to the tag action. */ - struct rte_flow_action_rss rss;/**< RSS context. */ - uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */ - uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */ LIST_HEAD(dev_flows, mlx5_flow) dev_flows; /**< Device flows that are part of the flow. */ struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */ - uint8_t ingress; /**< 1 if the flow is ingress. */ - uint32_t group; /**< The group index. */ - uint8_t transfer; /**< 1 if the flow is E-Switch flow. */ uint32_t hairpin_flow_id; /**< The flow id used for hairpin. */ }; diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 930f088..019c9b3 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -1583,10 +1583,9 @@ struct field_modify_info modify_tcp[] = { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_ibv_shared *sh = priv->sh; struct mlx5_flow_dv_encap_decap_resource *cache_resource; - struct rte_flow *flow = dev_flow->flow; struct mlx5dv_dr_domain *domain; - resource->flags = flow->group ? 0 : 1; + resource->flags = dev_flow->group ? 0 : 1; if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB) domain = sh->fdb_domain; else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_RX) @@ -2745,7 +2744,7 @@ struct field_modify_info modify_tcp[] = { else ns = sh->rx_domain; resource->flags = - dev_flow->flow->group ? 0 : MLX5DV_DR_ACTION_FLAGS_ROOT_LEVEL; + dev_flow->group ? 0 : MLX5DV_DR_ACTION_FLAGS_ROOT_LEVEL; /* Lookup a matching resource from cache. */ LIST_FOREACH(cache_resource, &sh->modify_cmds, next) { if (resource->ft_type == cache_resource->ft_type && @@ -4066,18 +4065,20 @@ struct field_modify_info modify_tcp[] = { const struct rte_flow_action actions[] __rte_unused, struct rte_flow_error *error) { - uint32_t size = sizeof(struct mlx5_flow); - struct mlx5_flow *flow; + size_t size = sizeof(struct mlx5_flow); + struct mlx5_flow *dev_flow; - flow = rte_calloc(__func__, 1, size, 0); - if (!flow) { + dev_flow = rte_calloc(__func__, 1, size, 0); + if (!dev_flow) { rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, "not enough memory to create flow"); return NULL; } - flow->dv.value.size = MLX5_ST_SZ_BYTES(fte_match_param); - return flow; + dev_flow->dv.value.size = MLX5_ST_SZ_BYTES(fte_match_param); + dev_flow->ingress = attr->ingress; + dev_flow->transfer = attr->transfer; + return dev_flow; } #ifndef NDEBUG @@ -5458,7 +5459,7 @@ struct field_modify_info modify_tcp[] = { (void *)cache_resource, rte_atomic32_read(&cache_resource->refcnt)); rte_atomic32_inc(&cache_resource->refcnt); - dev_flow->flow->tag_resource = cache_resource; + dev_flow->dv.tag_resource = cache_resource; return 0; } } @@ -5480,7 +5481,7 @@ struct field_modify_info modify_tcp[] = { rte_atomic32_init(&cache_resource->refcnt); rte_atomic32_inc(&cache_resource->refcnt); LIST_INSERT_HEAD(&sh->tags, cache_resource, next); - dev_flow->flow->tag_resource = cache_resource; + dev_flow->dv.tag_resource = cache_resource; DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++", (void *)cache_resource, rte_atomic32_read(&cache_resource->refcnt)); @@ -5660,7 +5661,7 @@ struct field_modify_info modify_tcp[] = { &table, error); if (ret) return ret; - flow->group = table; + dev_flow->group = table; if (attr->transfer) res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB; if (priority == MLX5_FLOW_PRIO_RSVD) @@ -5697,47 +5698,50 @@ struct field_modify_info modify_tcp[] = { case RTE_FLOW_ACTION_TYPE_FLAG: tag_resource.tag = mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT); - if (!flow->tag_resource) + if (!dev_flow->dv.tag_resource) if (flow_dv_tag_resource_register (dev, &tag_resource, dev_flow, error)) return errno; dev_flow->dv.actions[actions_n++] = - flow->tag_resource->action; + dev_flow->dv.tag_resource->action; action_flags |= MLX5_FLOW_ACTION_FLAG; break; case RTE_FLOW_ACTION_TYPE_MARK: tag_resource.tag = mlx5_flow_mark_set (((const struct rte_flow_action_mark *) (actions->conf))->id); - if (!flow->tag_resource) + if (!dev_flow->dv.tag_resource) if (flow_dv_tag_resource_register (dev, &tag_resource, dev_flow, error)) return errno; dev_flow->dv.actions[actions_n++] = - flow->tag_resource->action; + dev_flow->dv.tag_resource->action; action_flags |= MLX5_FLOW_ACTION_MARK; break; case RTE_FLOW_ACTION_TYPE_DROP: action_flags |= MLX5_FLOW_ACTION_DROP; break; case RTE_FLOW_ACTION_TYPE_QUEUE: + assert(flow->rss.queue); queue = actions->conf; flow->rss.queue_num = 1; - (*flow->queue)[0] = queue->index; + (*flow->rss.queue)[0] = queue->index; action_flags |= MLX5_FLOW_ACTION_QUEUE; break; case RTE_FLOW_ACTION_TYPE_RSS: + assert(flow->rss.queue); rss = actions->conf; - if (flow->queue) - memcpy((*flow->queue), rss->queue, + if (flow->rss.queue) + memcpy((*flow->rss.queue), rss->queue, rss->queue_num * sizeof(uint16_t)); flow->rss.queue_num = rss->queue_num; /* NULL RSS key indicates default RSS key. */ rss_key = !rss->key ? rss_hash_default_key : rss->key; - memcpy(flow->key, rss_key, MLX5_RSS_HASH_KEY_LEN); - /* RSS type 0 indicates default RSS type ETH_RSS_IP. */ - flow->rss.types = !rss->types ? ETH_RSS_IP : rss->types; - flow->rss.level = rss->level; + memcpy(flow->rss.key, rss_key, MLX5_RSS_HASH_KEY_LEN); + /* + * rss->level and rss.types should be set in advance + * when expanding items for RSS. + */ action_flags |= MLX5_FLOW_ACTION_RSS; break; case RTE_FLOW_ACTION_TYPE_COUNT: @@ -5748,7 +5752,7 @@ struct field_modify_info modify_tcp[] = { flow->counter = flow_dv_counter_alloc(dev, count->shared, count->id, - flow->group); + dev_flow->group); if (flow->counter == NULL) goto cnt_err; dev_flow->dv.actions[actions_n++] = @@ -6046,9 +6050,10 @@ struct field_modify_info modify_tcp[] = { mlx5_flow_tunnel_ip_check(items, next_protocol, &item_flags, &tunnel); flow_dv_translate_item_ipv4(match_mask, match_value, - items, tunnel, flow->group); + items, tunnel, + dev_flow->group); matcher.priority = MLX5_PRIORITY_MAP_L3; - dev_flow->dv.hash_fields |= + dev_flow->hash_fields |= mlx5_flow_hashfields_adjust (dev_flow, tunnel, MLX5_IPV4_LAYER_TYPES, @@ -6073,9 +6078,10 @@ struct field_modify_info modify_tcp[] = { mlx5_flow_tunnel_ip_check(items, next_protocol, &item_flags, &tunnel); flow_dv_translate_item_ipv6(match_mask, match_value, - items, tunnel, flow->group); + items, tunnel, + dev_flow->group); matcher.priority = MLX5_PRIORITY_MAP_L3; - dev_flow->dv.hash_fields |= + dev_flow->hash_fields |= mlx5_flow_hashfields_adjust (dev_flow, tunnel, MLX5_IPV6_LAYER_TYPES, @@ -6100,7 +6106,7 @@ struct field_modify_info modify_tcp[] = { flow_dv_translate_item_tcp(match_mask, match_value, items, tunnel); matcher.priority = MLX5_PRIORITY_MAP_L4; - dev_flow->dv.hash_fields |= + dev_flow->hash_fields |= mlx5_flow_hashfields_adjust (dev_flow, tunnel, ETH_RSS_TCP, IBV_RX_HASH_SRC_PORT_TCP | @@ -6112,7 +6118,7 @@ struct field_modify_info modify_tcp[] = { flow_dv_translate_item_udp(match_mask, match_value, items, tunnel); matcher.priority = MLX5_PRIORITY_MAP_L4; - dev_flow->dv.hash_fields |= + dev_flow->hash_fields |= mlx5_flow_hashfields_adjust (dev_flow, tunnel, ETH_RSS_UDP, IBV_RX_HASH_SRC_PORT_UDP | @@ -6208,7 +6214,7 @@ struct field_modify_info modify_tcp[] = { matcher.priority = mlx5_flow_adjust_priority(dev, priority, matcher.priority); matcher.egress = attr->egress; - matcher.group = flow->group; + matcher.group = dev_flow->group; matcher.transfer = attr->transfer; if (flow_dv_matcher_register(dev, &matcher, dev_flow, error)) return -rte_errno; @@ -6242,7 +6248,7 @@ struct field_modify_info modify_tcp[] = { dv = &dev_flow->dv; n = dv->actions_n; if (dev_flow->actions & MLX5_FLOW_ACTION_DROP) { - if (flow->transfer) { + if (dev_flow->transfer) { dv->actions[n++] = priv->sh->esw_drop_action; } else { dv->hrxq = mlx5_hrxq_drop_new(dev); @@ -6260,15 +6266,18 @@ struct field_modify_info modify_tcp[] = { (MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) { struct mlx5_hrxq *hrxq; - hrxq = mlx5_hrxq_get(dev, flow->key, + assert(flow->rss.queue); + hrxq = mlx5_hrxq_get(dev, flow->rss.key, MLX5_RSS_HASH_KEY_LEN, - dv->hash_fields, - (*flow->queue), + dev_flow->hash_fields, + (*flow->rss.queue), flow->rss.queue_num); if (!hrxq) { hrxq = mlx5_hrxq_new - (dev, flow->key, MLX5_RSS_HASH_KEY_LEN, - dv->hash_fields, (*flow->queue), + (dev, flow->rss.key, + MLX5_RSS_HASH_KEY_LEN, + dev_flow->hash_fields, + (*flow->rss.queue), flow->rss.queue_num, !!(dev_flow->layers & MLX5_FLOW_LAYER_TUNNEL)); @@ -6578,10 +6587,6 @@ struct field_modify_info modify_tcp[] = { flow_dv_counter_release(dev, flow->counter); flow->counter = NULL; } - if (flow->tag_resource) { - flow_dv_tag_release(dev, flow->tag_resource); - flow->tag_resource = NULL; - } while (!LIST_EMPTY(&flow->dev_flows)) { dev_flow = LIST_FIRST(&flow->dev_flows); LIST_REMOVE(dev_flow, next); @@ -6597,6 +6602,8 @@ struct field_modify_info modify_tcp[] = { flow_dv_port_id_action_resource_release(dev_flow); if (dev_flow->dv.push_vlan_res) flow_dv_push_vlan_action_resource_release(dev_flow); + if (dev_flow->dv.tag_resource) + flow_dv_tag_release(dev, dev_flow->dv.tag_resource); rte_free(dev_flow); } } diff --git a/drivers/net/mlx5/mlx5_flow_verbs.c b/drivers/net/mlx5/mlx5_flow_verbs.c index fd27f6c..3ab73c2 100644 --- a/drivers/net/mlx5/mlx5_flow_verbs.c +++ b/drivers/net/mlx5/mlx5_flow_verbs.c @@ -864,8 +864,8 @@ const struct rte_flow_action_queue *queue = action->conf; struct rte_flow *flow = dev_flow->flow; - if (flow->queue) - (*flow->queue)[0] = queue->index; + if (flow->rss.queue) + (*flow->rss.queue)[0] = queue->index; flow->rss.queue_num = 1; } @@ -889,16 +889,17 @@ const uint8_t *rss_key; struct rte_flow *flow = dev_flow->flow; - if (flow->queue) - memcpy((*flow->queue), rss->queue, + if (flow->rss.queue) + memcpy((*flow->rss.queue), rss->queue, rss->queue_num * sizeof(uint16_t)); flow->rss.queue_num = rss->queue_num; /* NULL RSS key indicates default RSS key. */ rss_key = !rss->key ? rss_hash_default_key : rss->key; - memcpy(flow->key, rss_key, MLX5_RSS_HASH_KEY_LEN); - /* RSS type 0 indicates default RSS type (ETH_RSS_IP). */ - flow->rss.types = !rss->types ? ETH_RSS_IP : rss->types; - flow->rss.level = rss->level; + memcpy(flow->rss.key, rss_key, MLX5_RSS_HASH_KEY_LEN); + /* + * rss->level and rss.types should be set in advance when expanding + * items for RSS. + */ } /** @@ -1365,22 +1366,23 @@ const struct rte_flow_action actions[], struct rte_flow_error *error) { - uint32_t size = sizeof(struct mlx5_flow) + sizeof(struct ibv_flow_attr); - struct mlx5_flow *flow; + size_t size = sizeof(struct mlx5_flow) + sizeof(struct ibv_flow_attr); + struct mlx5_flow *dev_flow; size += flow_verbs_get_actions_size(actions); size += flow_verbs_get_items_size(items); - flow = rte_calloc(__func__, 1, size, 0); - if (!flow) { + dev_flow = rte_calloc(__func__, 1, size, 0); + if (!dev_flow) { rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, "not enough memory to create flow"); return NULL; } - flow->verbs.attr = (void *)(flow + 1); - flow->verbs.specs = - (uint8_t *)(flow + 1) + sizeof(struct ibv_flow_attr); - return flow; + dev_flow->verbs.attr = (void *)(dev_flow + 1); + dev_flow->verbs.specs = (void *)(dev_flow->verbs.attr + 1); + dev_flow->ingress = attr->ingress; + dev_flow->transfer = attr->transfer; + return dev_flow; } /** @@ -1486,7 +1488,7 @@ flow_verbs_translate_item_ipv4(dev_flow, items, item_flags); subpriority = MLX5_PRIORITY_MAP_L3; - dev_flow->verbs.hash_fields |= + dev_flow->hash_fields |= mlx5_flow_hashfields_adjust (dev_flow, tunnel, MLX5_IPV4_LAYER_TYPES, @@ -1498,7 +1500,7 @@ flow_verbs_translate_item_ipv6(dev_flow, items, item_flags); subpriority = MLX5_PRIORITY_MAP_L3; - dev_flow->verbs.hash_fields |= + dev_flow->hash_fields |= mlx5_flow_hashfields_adjust (dev_flow, tunnel, MLX5_IPV6_LAYER_TYPES, @@ -1510,7 +1512,7 @@ flow_verbs_translate_item_tcp(dev_flow, items, item_flags); subpriority = MLX5_PRIORITY_MAP_L4; - dev_flow->verbs.hash_fields |= + dev_flow->hash_fields |= mlx5_flow_hashfields_adjust (dev_flow, tunnel, ETH_RSS_TCP, (IBV_RX_HASH_SRC_PORT_TCP | @@ -1522,7 +1524,7 @@ flow_verbs_translate_item_udp(dev_flow, items, item_flags); subpriority = MLX5_PRIORITY_MAP_L4; - dev_flow->verbs.hash_fields |= + dev_flow->hash_fields |= mlx5_flow_hashfields_adjust (dev_flow, tunnel, ETH_RSS_UDP, (IBV_RX_HASH_SRC_PORT_UDP | @@ -1667,16 +1669,17 @@ } else { struct mlx5_hrxq *hrxq; - hrxq = mlx5_hrxq_get(dev, flow->key, + assert(flow->rss.queue); + hrxq = mlx5_hrxq_get(dev, flow->rss.key, MLX5_RSS_HASH_KEY_LEN, - verbs->hash_fields, - (*flow->queue), + dev_flow->hash_fields, + (*flow->rss.queue), flow->rss.queue_num); if (!hrxq) - hrxq = mlx5_hrxq_new(dev, flow->key, + hrxq = mlx5_hrxq_new(dev, flow->rss.key, MLX5_RSS_HASH_KEY_LEN, - verbs->hash_fields, - (*flow->queue), + dev_flow->hash_fields, + (*flow->rss.queue), flow->rss.queue_num, !!(dev_flow->layers & MLX5_FLOW_LAYER_TUNNEL)); From patchwork Tue Nov 5 08:01:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62423 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6A7F3A0352; Tue, 5 Nov 2019 09:02:40 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C5D598F96; Tue, 5 Nov 2019 09:02:11 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 5E5C444C7 for ; Tue, 5 Nov 2019 09:02:09 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:06 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA5826CA026469; Tue, 5 Nov 2019 10:02:06 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA58262A030761; Tue, 5 Nov 2019 08:02:06 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA5826e0030760; Tue, 5 Nov 2019 08:02:06 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:40 +0000 Message-Id: <1572940915-29416-6-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 05/20] net/mlx5: update flow functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Update flow creation/destroy functions for future reuse. List operations can be skipped inside functions and done separately out of flow creation. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index d1661f2..6e6c845 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -2736,7 +2736,10 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, * @param dev * Pointer to Ethernet device. * @param list - * Pointer to a TAILQ flow list. + * Pointer to a TAILQ flow list. If this parameter NULL, + * no list insertion occurred, flow is just created, + * this is caller's responsibility to track the + * created flow. * @param[in] attr * Flow rule attributes. * @param[in] items @@ -2881,7 +2884,8 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, if (ret < 0) goto error; } - TAILQ_INSERT_TAIL(list, flow, next); + if (list) + TAILQ_INSERT_TAIL(list, flow, next); flow_rxq_flags_set(dev, flow); return flow; error_before_flow: @@ -2975,7 +2979,8 @@ struct rte_flow * * @param dev * Pointer to Ethernet device. * @param list - * Pointer to a TAILQ flow list. + * Pointer to a TAILQ flow list. If this parameter NULL, + * there is no flow removal from the list. * @param[in] flow * Flow to destroy. */ @@ -2995,7 +3000,8 @@ struct rte_flow * mlx5_flow_id_release(priv->sh->flow_id_pool, flow->hairpin_flow_id); flow_drv_destroy(dev, flow); - TAILQ_REMOVE(list, flow, next); + if (list) + TAILQ_REMOVE(list, flow, next); rte_free(flow->fdir); rte_free(flow); } From patchwork Tue Nov 5 08:01:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62424 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 62C66A0352; Tue, 5 Nov 2019 09:02:52 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 484B51BDAC; Tue, 5 Nov 2019 09:02:13 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 63180493D for ; Tue, 5 Nov 2019 09:02:09 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:07 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA5827LI026479; Tue, 5 Nov 2019 10:02:07 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA5827OW030766; Tue, 5 Nov 2019 08:02:07 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA5827Qo030765; Tue, 5 Nov 2019 08:02:07 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:41 +0000 Message-Id: <1572940915-29416-7-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 06/20] net/mlx5: update meta register matcher set X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Introduce the dedicated matcher register field setup routine. Update the code to use this unified one. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow_dv.c | 171 +++++++++++++++++++--------------------- 1 file changed, 82 insertions(+), 89 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 019c9b3..eb7e481 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -4903,6 +4903,78 @@ struct field_modify_info modify_tcp[] = { } /** + * Add metadata register item to matcher + * + * @param[in, out] matcher + * Flow matcher. + * @param[in, out] key + * Flow matcher value. + * @param[in] reg_type + * Type of device metadata register + * @param[in] value + * Register value + * @param[in] mask + * Register mask + */ +static void +flow_dv_match_meta_reg(void *matcher, void *key, + enum modify_reg reg_type, + uint32_t data, uint32_t mask) +{ + void *misc2_m = + MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters_2); + void *misc2_v = + MLX5_ADDR_OF(fte_match_param, key, misc_parameters_2); + + data &= mask; + switch (reg_type) { + case REG_A: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_a, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_a, data); + break; + case REG_B: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_b, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_b, data); + break; + case REG_C_0: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_0, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_0, data); + break; + case REG_C_1: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_1, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_1, data); + break; + case REG_C_2: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_2, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_2, data); + break; + case REG_C_3: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_3, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_3, data); + break; + case REG_C_4: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_4, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_4, data); + break; + case REG_C_5: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_5, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_5, data); + break; + case REG_C_6: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_6, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_6, data); + break; + case REG_C_7: + MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_7, mask); + MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_7, data); + break; + default: + assert(false); + break; + } +} + +/** * Add META item to matcher * * @param[in, out] matcher @@ -4920,21 +4992,15 @@ struct field_modify_info modify_tcp[] = { { const struct rte_flow_item_meta *meta_m; const struct rte_flow_item_meta *meta_v; - void *misc2_m = - MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters_2); - void *misc2_v = - MLX5_ADDR_OF(fte_match_param, key, misc_parameters_2); meta_m = (const void *)item->mask; if (!meta_m) meta_m = &rte_flow_item_meta_mask; meta_v = (const void *)item->spec; - if (meta_v) { - MLX5_SET(fte_match_set_misc2, misc2_m, - metadata_reg_a, meta_m->data); - MLX5_SET(fte_match_set_misc2, misc2_v, - metadata_reg_a, meta_v->data & meta_m->data); - } + if (meta_v) + flow_dv_match_meta_reg(matcher, key, REG_A, + rte_cpu_to_be_32(meta_v->data), + rte_cpu_to_be_32(meta_m->data)); } /** @@ -4951,13 +5017,7 @@ struct field_modify_info modify_tcp[] = { flow_dv_translate_item_meta_vport(void *matcher, void *key, uint32_t value, uint32_t mask) { - void *misc2_m = - MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters_2); - void *misc2_v = - MLX5_ADDR_OF(fte_match_param, key, misc_parameters_2); - - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_0, mask); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_0, value); + flow_dv_match_meta_reg(matcher, key, REG_C_0, value, mask); } /** @@ -4971,81 +5031,14 @@ struct field_modify_info modify_tcp[] = { * Flow pattern to translate. */ static void -flow_dv_translate_item_tag(void *matcher, void *key, - const struct rte_flow_item *item) +flow_dv_translate_mlx5_item_tag(void *matcher, void *key, + const struct rte_flow_item *item) { - void *misc2_m = - MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters_2); - void *misc2_v = - MLX5_ADDR_OF(fte_match_param, key, misc_parameters_2); const struct mlx5_rte_flow_item_tag *tag_v = item->spec; const struct mlx5_rte_flow_item_tag *tag_m = item->mask; enum modify_reg reg = tag_v->id; - rte_be32_t value = tag_v->data; - rte_be32_t mask = tag_m->data; - switch (reg) { - case REG_A: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_a, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_a, - rte_be_to_cpu_32(value)); - break; - case REG_B: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_b, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_b, - rte_be_to_cpu_32(value)); - break; - case REG_C_0: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_0, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_0, - rte_be_to_cpu_32(value)); - break; - case REG_C_1: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_1, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_1, - rte_be_to_cpu_32(value)); - break; - case REG_C_2: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_2, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_2, - rte_be_to_cpu_32(value)); - break; - case REG_C_3: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_3, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_3, - rte_be_to_cpu_32(value)); - break; - case REG_C_4: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_4, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_4, - rte_be_to_cpu_32(value)); - break; - case REG_C_5: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_5, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_5, - rte_be_to_cpu_32(value)); - break; - case REG_C_6: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_6, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_6, - rte_be_to_cpu_32(value)); - break; - case REG_C_7: - MLX5_SET(fte_match_set_misc2, misc2_m, metadata_reg_c_7, - rte_be_to_cpu_32(mask)); - MLX5_SET(fte_match_set_misc2, misc2_v, metadata_reg_c_7, - rte_be_to_cpu_32(value)); - break; - } + flow_dv_match_meta_reg(matcher, key, reg, tag_v->data, tag_m->data); } /** @@ -6177,8 +6170,8 @@ struct field_modify_info modify_tcp[] = { last_item = MLX5_FLOW_LAYER_ICMP6; break; case MLX5_RTE_FLOW_ITEM_TYPE_TAG: - flow_dv_translate_item_tag(match_mask, match_value, - items); + flow_dv_translate_mlx5_item_tag(match_mask, + match_value, items); last_item = MLX5_FLOW_ITEM_TAG; break; case MLX5_RTE_FLOW_ITEM_TYPE_TX_QUEUE: From patchwork Tue Nov 5 08:01:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62425 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C541BA0352; Tue, 5 Nov 2019 09:03:03 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B92B85B3E; Tue, 5 Nov 2019 09:02:15 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 71B665B32 for ; Tue, 5 Nov 2019 09:02:09 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:08 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA5828VS026482; Tue, 5 Nov 2019 10:02:08 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA5828iW030768; Tue, 5 Nov 2019 08:02:08 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA5827Li030767; Tue, 5 Nov 2019 08:02:07 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:42 +0000 Message-Id: <1572940915-29416-8-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 07/20] net/mlx5: rename structure and function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" There are some renaming: - in the DV flow engine overall: flow_d_* -> flow_dv_* - in flow_dv_translate(): res -> mhdr_res Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow_dv.c | 151 ++++++++++++++++++++-------------------- 1 file changed, 76 insertions(+), 75 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index eb7e481..853e879 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -183,7 +183,7 @@ struct field_modify_info modify_tcp[] = { * Pointer to the rte_eth_dev structure. */ static void -flow_d_shared_lock(struct rte_eth_dev *dev) +flow_dv_shared_lock(struct rte_eth_dev *dev) { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_ibv_shared *sh = priv->sh; @@ -198,7 +198,7 @@ struct field_modify_info modify_tcp[] = { } static void -flow_d_shared_unlock(struct rte_eth_dev *dev) +flow_dv_shared_unlock(struct rte_eth_dev *dev) { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_ibv_shared *sh = priv->sh; @@ -5597,7 +5597,8 @@ struct field_modify_info modify_tcp[] = { } /** - * Fill the flow with DV spec. + * Fill the flow with DV spec, lock free + * (mutex should be acquired by caller). * * @param[in] dev * Pointer to rte_eth_dev structure. @@ -5616,12 +5617,12 @@ struct field_modify_info modify_tcp[] = { * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -flow_dv_translate(struct rte_eth_dev *dev, - struct mlx5_flow *dev_flow, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], - struct rte_flow_error *error) +__flow_dv_translate(struct rte_eth_dev *dev, + struct mlx5_flow *dev_flow, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) { struct mlx5_priv *priv = dev->data->dev_private; struct rte_flow *flow = dev_flow->flow; @@ -5636,7 +5637,7 @@ struct field_modify_info modify_tcp[] = { }; int actions_n = 0; bool actions_end = false; - struct mlx5_flow_dv_modify_hdr_resource res = { + struct mlx5_flow_dv_modify_hdr_resource mhdr_res = { .ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX : MLX5DV_FLOW_TABLE_TYPE_NIC_RX }; @@ -5656,7 +5657,7 @@ struct field_modify_info modify_tcp[] = { return ret; dev_flow->group = table; if (attr->transfer) - res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB; + mhdr_res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB; if (priority == MLX5_FLOW_PRIO_RSVD) priority = priv->config.flow_prio - 1; for (; !actions_end ; actions++) { @@ -5805,7 +5806,7 @@ struct field_modify_info modify_tcp[] = { mlx5_update_vlan_vid_pcp(actions, &vlan); /* If no VLAN push - this is a modify header action */ if (flow_dv_convert_action_modify_vlan_vid - (&res, actions, error)) + (&mhdr_res, actions, error)) return -rte_errno; action_flags |= MLX5_FLOW_ACTION_OF_SET_VLAN_VID; break; @@ -5904,8 +5905,8 @@ struct field_modify_info modify_tcp[] = { break; case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC: case RTE_FLOW_ACTION_TYPE_SET_MAC_DST: - if (flow_dv_convert_action_modify_mac(&res, actions, - error)) + if (flow_dv_convert_action_modify_mac + (&mhdr_res, actions, error)) return -rte_errno; action_flags |= actions->type == RTE_FLOW_ACTION_TYPE_SET_MAC_SRC ? @@ -5914,8 +5915,8 @@ struct field_modify_info modify_tcp[] = { break; case RTE_FLOW_ACTION_TYPE_SET_IPV4_SRC: case RTE_FLOW_ACTION_TYPE_SET_IPV4_DST: - if (flow_dv_convert_action_modify_ipv4(&res, actions, - error)) + if (flow_dv_convert_action_modify_ipv4 + (&mhdr_res, actions, error)) return -rte_errno; action_flags |= actions->type == RTE_FLOW_ACTION_TYPE_SET_IPV4_SRC ? @@ -5924,8 +5925,8 @@ struct field_modify_info modify_tcp[] = { break; case RTE_FLOW_ACTION_TYPE_SET_IPV6_SRC: case RTE_FLOW_ACTION_TYPE_SET_IPV6_DST: - if (flow_dv_convert_action_modify_ipv6(&res, actions, - error)) + if (flow_dv_convert_action_modify_ipv6 + (&mhdr_res, actions, error)) return -rte_errno; action_flags |= actions->type == RTE_FLOW_ACTION_TYPE_SET_IPV6_SRC ? @@ -5934,9 +5935,9 @@ struct field_modify_info modify_tcp[] = { break; case RTE_FLOW_ACTION_TYPE_SET_TP_SRC: case RTE_FLOW_ACTION_TYPE_SET_TP_DST: - if (flow_dv_convert_action_modify_tp(&res, actions, - items, &flow_attr, - error)) + if (flow_dv_convert_action_modify_tp + (&mhdr_res, actions, items, + &flow_attr, error)) return -rte_errno; action_flags |= actions->type == RTE_FLOW_ACTION_TYPE_SET_TP_SRC ? @@ -5944,23 +5945,22 @@ struct field_modify_info modify_tcp[] = { MLX5_FLOW_ACTION_SET_TP_DST; break; case RTE_FLOW_ACTION_TYPE_DEC_TTL: - if (flow_dv_convert_action_modify_dec_ttl(&res, items, - &flow_attr, - error)) + if (flow_dv_convert_action_modify_dec_ttl + (&mhdr_res, items, &flow_attr, error)) return -rte_errno; action_flags |= MLX5_FLOW_ACTION_DEC_TTL; break; case RTE_FLOW_ACTION_TYPE_SET_TTL: - if (flow_dv_convert_action_modify_ttl(&res, actions, - items, &flow_attr, - error)) + if (flow_dv_convert_action_modify_ttl + (&mhdr_res, actions, items, + &flow_attr, error)) return -rte_errno; action_flags |= MLX5_FLOW_ACTION_SET_TTL; break; case RTE_FLOW_ACTION_TYPE_INC_TCP_SEQ: case RTE_FLOW_ACTION_TYPE_DEC_TCP_SEQ: - if (flow_dv_convert_action_modify_tcp_seq(&res, actions, - error)) + if (flow_dv_convert_action_modify_tcp_seq + (&mhdr_res, actions, error)) return -rte_errno; action_flags |= actions->type == RTE_FLOW_ACTION_TYPE_INC_TCP_SEQ ? @@ -5970,8 +5970,8 @@ struct field_modify_info modify_tcp[] = { case RTE_FLOW_ACTION_TYPE_INC_TCP_ACK: case RTE_FLOW_ACTION_TYPE_DEC_TCP_ACK: - if (flow_dv_convert_action_modify_tcp_ack(&res, actions, - error)) + if (flow_dv_convert_action_modify_tcp_ack + (&mhdr_res, actions, error)) return -rte_errno; action_flags |= actions->type == RTE_FLOW_ACTION_TYPE_INC_TCP_ACK ? @@ -5979,14 +5979,14 @@ struct field_modify_info modify_tcp[] = { MLX5_FLOW_ACTION_DEC_TCP_ACK; break; case MLX5_RTE_FLOW_ACTION_TYPE_TAG: - if (flow_dv_convert_action_set_reg(&res, actions, - error)) + if (flow_dv_convert_action_set_reg + (&mhdr_res, actions, error)) return -rte_errno; action_flags |= MLX5_FLOW_ACTION_SET_TAG; break; case MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG: - if (flow_dv_convert_action_copy_mreg(dev, &res, - actions, error)) + if (flow_dv_convert_action_copy_mreg + (dev, &mhdr_res, actions, error)) return -rte_errno; action_flags |= MLX5_FLOW_ACTION_SET_TAG; break; @@ -5995,9 +5995,7 @@ struct field_modify_info modify_tcp[] = { if (action_flags & MLX5_FLOW_MODIFY_HDR_ACTIONS) { /* create modify action if needed. */ if (flow_dv_modify_hdr_resource_register - (dev, &res, - dev_flow, - error)) + (dev, &mhdr_res, dev_flow, error)) return -rte_errno; dev_flow->dv.actions[modify_action_position] = dev_flow->dv.modify_hdr->verbs_action; @@ -6215,7 +6213,8 @@ struct field_modify_info modify_tcp[] = { } /** - * Apply the flow to the NIC. + * Apply the flow to the NIC, lock free, + * (mutex should be acquired by caller). * * @param[in] dev * Pointer to the Ethernet device structure. @@ -6228,8 +6227,8 @@ struct field_modify_info modify_tcp[] = { * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -flow_dv_apply(struct rte_eth_dev *dev, struct rte_flow *flow, - struct rte_flow_error *error) +__flow_dv_apply(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error) { struct mlx5_flow_dv *dv; struct mlx5_flow *dev_flow; @@ -6527,6 +6526,7 @@ struct field_modify_info modify_tcp[] = { /** * Remove the flow from the NIC but keeps it in memory. + * Lock free, (mutex should be acquired by caller). * * @param[in] dev * Pointer to Ethernet device. @@ -6534,7 +6534,7 @@ struct field_modify_info modify_tcp[] = { * Pointer to flow structure. */ static void -flow_dv_remove(struct rte_eth_dev *dev, struct rte_flow *flow) +__flow_dv_remove(struct rte_eth_dev *dev, struct rte_flow *flow) { struct mlx5_flow_dv *dv; struct mlx5_flow *dev_flow; @@ -6562,6 +6562,7 @@ struct field_modify_info modify_tcp[] = { /** * Remove the flow from the NIC and the memory. + * Lock free, (mutex should be acquired by caller). * * @param[in] dev * Pointer to the Ethernet device structure. @@ -6569,13 +6570,13 @@ struct field_modify_info modify_tcp[] = { * Pointer to flow structure. */ static void -flow_dv_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) +__flow_dv_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) { struct mlx5_flow *dev_flow; if (!flow) return; - flow_dv_remove(dev, flow); + __flow_dv_remove(dev, flow); if (flow->counter) { flow_dv_counter_release(dev, flow->counter); flow->counter = NULL; @@ -6686,69 +6687,69 @@ struct field_modify_info modify_tcp[] = { } /* - * Mutex-protected thunk to flow_dv_translate(). + * Mutex-protected thunk to lock-free __flow_dv_translate(). */ static int -flow_d_translate(struct rte_eth_dev *dev, - struct mlx5_flow *dev_flow, - const struct rte_flow_attr *attr, - const struct rte_flow_item items[], - const struct rte_flow_action actions[], - struct rte_flow_error *error) +flow_dv_translate(struct rte_eth_dev *dev, + struct mlx5_flow *dev_flow, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) { int ret; - flow_d_shared_lock(dev); - ret = flow_dv_translate(dev, dev_flow, attr, items, actions, error); - flow_d_shared_unlock(dev); + flow_dv_shared_lock(dev); + ret = __flow_dv_translate(dev, dev_flow, attr, items, actions, error); + flow_dv_shared_unlock(dev); return ret; } /* - * Mutex-protected thunk to flow_dv_apply(). + * Mutex-protected thunk to lock-free __flow_dv_apply(). */ static int -flow_d_apply(struct rte_eth_dev *dev, - struct rte_flow *flow, - struct rte_flow_error *error) +flow_dv_apply(struct rte_eth_dev *dev, + struct rte_flow *flow, + struct rte_flow_error *error) { int ret; - flow_d_shared_lock(dev); - ret = flow_dv_apply(dev, flow, error); - flow_d_shared_unlock(dev); + flow_dv_shared_lock(dev); + ret = __flow_dv_apply(dev, flow, error); + flow_dv_shared_unlock(dev); return ret; } /* - * Mutex-protected thunk to flow_dv_remove(). + * Mutex-protected thunk to lock-free __flow_dv_remove(). */ static void -flow_d_remove(struct rte_eth_dev *dev, struct rte_flow *flow) +flow_dv_remove(struct rte_eth_dev *dev, struct rte_flow *flow) { - flow_d_shared_lock(dev); - flow_dv_remove(dev, flow); - flow_d_shared_unlock(dev); + flow_dv_shared_lock(dev); + __flow_dv_remove(dev, flow); + flow_dv_shared_unlock(dev); } /* - * Mutex-protected thunk to flow_dv_destroy(). + * Mutex-protected thunk to lock-free __flow_dv_destroy(). */ static void -flow_d_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) +flow_dv_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) { - flow_d_shared_lock(dev); - flow_dv_destroy(dev, flow); - flow_d_shared_unlock(dev); + flow_dv_shared_lock(dev); + __flow_dv_destroy(dev, flow); + flow_dv_shared_unlock(dev); } const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops = { .validate = flow_dv_validate, .prepare = flow_dv_prepare, - .translate = flow_d_translate, - .apply = flow_d_apply, - .remove = flow_d_remove, - .destroy = flow_d_destroy, + .translate = flow_dv_translate, + .apply = flow_dv_apply, + .remove = flow_dv_remove, + .destroy = flow_dv_destroy, .query = flow_dv_query, }; From patchwork Tue Nov 5 08:01:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62432 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E66D2A0352; Tue, 5 Nov 2019 09:04:49 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 455161BECD; Tue, 5 Nov 2019 09:02:28 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id BC1D01BE93 for ; Tue, 5 Nov 2019 09:02:13 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:09 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA5828Ou026488; Tue, 5 Nov 2019 10:02:08 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA5828Ym030770; Tue, 5 Nov 2019 08:02:08 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA5828Bn030769; Tue, 5 Nov 2019 08:02:08 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:43 +0000 Message-Id: <1572940915-29416-9-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 08/20] net/mlx5: check metadata registers availability X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The metadata registers reg_c provide support for TAG and SET_TAG features. Although there are 8 registers are available on the current mlx5 devices, some of them can be reserved. The availability should be queried by iterative trial-and-error implemented by mlx5_flow_discover_mreg_c() routine. If reg_c is available, it can be regarded inclusively that the extensive metadata support is possible. E.g. metadata register copy action, supporting 16 modify header actions (instead of 8 by default) preserving register across different domains (FDB and NIC) and so on. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.c | 11 +++++ drivers/net/mlx5/mlx5.h | 11 ++++- drivers/net/mlx5/mlx5_ethdev.c | 8 +++- drivers/net/mlx5/mlx5_flow.c | 98 +++++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow.h | 13 ------ drivers/net/mlx5/mlx5_flow_dv.c | 9 ++-- drivers/net/mlx5/mlx5_prm.h | 18 ++++++++ 7 files changed, 148 insertions(+), 20 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 72c30bf..1b86b7b 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -2341,6 +2341,17 @@ struct mlx5_flow_id_pool * goto error; } priv->config.flow_prio = err; + /* Query availibility of metadata reg_c's. */ + err = mlx5_flow_discover_mreg_c(eth_dev); + if (err < 0) { + err = -err; + goto error; + } + if (!mlx5_flow_ext_mreg_supported(eth_dev)) { + DRV_LOG(DEBUG, + "port %u extensive metadata register is not supported", + eth_dev->data->port_id); + } return eth_dev; error: if (priv) { diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index f644998..6b82c6d 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -37,6 +37,7 @@ #include "mlx5_autoconf.h" #include "mlx5_defs.h" #include "mlx5_glue.h" +#include "mlx5_prm.h" enum { PCI_VENDOR_ID_MELLANOX = 0x15b3, @@ -252,6 +253,8 @@ struct mlx5_dev_config { } mprq; /* Configurations for Multi-Packet RQ. */ int mps; /* Multi-packet send supported mode. */ unsigned int flow_prio; /* Number of flow priorities. */ + enum modify_reg flow_mreg_c[MLX5_MREG_C_NUM]; + /* Availibility of mreg_c's. */ unsigned int tso_max_payload_sz; /* Maximum TCP payload for TSO. */ unsigned int ind_table_max_size; /* Maximum indirection table size. */ unsigned int max_dump_files_num; /* Maximum dump files per queue. */ @@ -561,6 +564,10 @@ struct mlx5_flow_tbl_resource { #define MLX5_MAX_TABLES UINT16_MAX #define MLX5_HAIRPIN_TX_TABLE (UINT16_MAX - 1) +/* Reserve the last two tables for metadata register copy. */ +#define MLX5_FLOW_MREG_ACT_TABLE_GROUP (MLX5_MAX_TABLES - 1) +#define MLX5_FLOW_MREG_CP_TABLE_GROUP \ + (MLX5_FLOW_MREG_ACT_TABLE_GROUP - 1) #define MLX5_MAX_TABLES_FDB UINT16_MAX #define MLX5_DBR_PAGE_SIZE 4096 /* Must be >= 512. */ @@ -786,7 +793,7 @@ int mlx5_dev_to_pci_addr(const char *dev_path, int mlx5_is_removed(struct rte_eth_dev *dev); eth_tx_burst_t mlx5_select_tx_function(struct rte_eth_dev *dev); eth_rx_burst_t mlx5_select_rx_function(struct rte_eth_dev *dev); -struct mlx5_priv *mlx5_port_to_eswitch_info(uint16_t port); +struct mlx5_priv *mlx5_port_to_eswitch_info(uint16_t port, bool valid); struct mlx5_priv *mlx5_dev_to_eswitch_info(struct rte_eth_dev *dev); int mlx5_sysfs_switch_info(unsigned int ifindex, struct mlx5_switch_info *info); @@ -866,6 +873,8 @@ int mlx5_xstats_get_names(struct rte_eth_dev *dev __rte_unused, /* mlx5_flow.c */ +int mlx5_flow_discover_mreg_c(struct rte_eth_dev *eth_dev); +bool mlx5_flow_ext_mreg_supported(struct rte_eth_dev *dev); int mlx5_flow_discover_priorities(struct rte_eth_dev *dev); void mlx5_flow_print(struct rte_flow *flow); int mlx5_flow_validate(struct rte_eth_dev *dev, diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index c2bed2f..2b7c867 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -1793,6 +1793,10 @@ int mlx5_fw_version_get(struct rte_eth_dev *dev, char *fw_ver, size_t fw_size) * * @param[in] port * Device port id. + * @param[in] valid + * Device port id is valid, skip check. This flag is useful + * when trials are performed from probing and device is not + * flagged as valid yet (in attaching process). * @param[out] es_domain_id * E-Switch domain id. * @param[out] es_port_id @@ -1803,7 +1807,7 @@ int mlx5_fw_version_get(struct rte_eth_dev *dev, char *fw_ver, size_t fw_size) * on success, NULL otherwise and rte_errno is set. */ struct mlx5_priv * -mlx5_port_to_eswitch_info(uint16_t port) +mlx5_port_to_eswitch_info(uint16_t port, bool valid) { struct rte_eth_dev *dev; struct mlx5_priv *priv; @@ -1812,7 +1816,7 @@ struct mlx5_priv * rte_errno = EINVAL; return NULL; } - if (!rte_eth_dev_is_valid_port(port)) { + if (!valid && !rte_eth_dev_is_valid_port(port)) { rte_errno = ENODEV; return NULL; } diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 6e6c845..f32ea8d 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -368,6 +368,33 @@ static enum modify_reg flow_get_reg_id(struct rte_eth_dev *dev, NULL, "invalid feature name"); } + +/** + * Check extensive flow metadata register support. + * + * @param dev + * Pointer to rte_eth_dev structure. + * + * @return + * True if device supports extensive flow metadata register, otherwise false. + */ +bool +mlx5_flow_ext_mreg_supported(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *config = &priv->config; + + /* + * Having available reg_c can be regarded inclusively as supporting + * extensive flow metadata register, which could mean, + * - metadata register copy action by modify header. + * - 16 modify header actions is supported. + * - reg_c's are preserved across different domain (FDB and NIC) on + * packet loopback by flow lookup miss. + */ + return config->flow_mreg_c[2] != REG_NONE; +} + /** * Discover the maximum number of priority available. * @@ -4033,3 +4060,74 @@ struct rte_flow * } return 0; } + +/** + * Discover availability of metadata reg_c's. + * + * Iteratively use test flows to check availability. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_flow_discover_mreg_c(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *config = &priv->config; + enum modify_reg idx; + int n = 0; + + /* reg_c[0] and reg_c[1] are reserved. */ + config->flow_mreg_c[n++] = REG_C_0; + config->flow_mreg_c[n++] = REG_C_1; + /* Discover availability of other reg_c's. */ + for (idx = REG_C_2; idx <= REG_C_7; ++idx) { + struct rte_flow_attr attr = { + .group = MLX5_FLOW_MREG_CP_TABLE_GROUP, + .priority = MLX5_FLOW_PRIO_RSVD, + .ingress = 1, + }; + struct rte_flow_item items[] = { + [0] = { + .type = RTE_FLOW_ITEM_TYPE_END, + }, + }; + struct rte_flow_action actions[] = { + [0] = { + .type = MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG, + .conf = &(struct mlx5_flow_action_copy_mreg){ + .src = REG_C_1, + .dst = idx, + }, + }, + [1] = { + .type = RTE_FLOW_ACTION_TYPE_JUMP, + .conf = &(struct rte_flow_action_jump){ + .group = MLX5_FLOW_MREG_ACT_TABLE_GROUP, + }, + }, + [2] = { + .type = RTE_FLOW_ACTION_TYPE_END, + }, + }; + struct rte_flow *flow; + struct rte_flow_error error; + + if (!config->dv_flow_en) + break; + /* Create internal flow, validation skips copy action. */ + flow = flow_list_create(dev, NULL, &attr, items, + actions, false, &error); + if (!flow) + continue; + if (dev->data->dev_started || !flow_drv_apply(dev, flow, NULL)) + config->flow_mreg_c[n++] = idx; + flow_list_destroy(dev, NULL, flow); + } + for (; n < MLX5_MREG_C_NUM; ++n) + config->flow_mreg_c[n] = REG_NONE; + return 0; +} diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index b9a9507..f2b6726 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -27,19 +27,6 @@ #include "mlx5.h" #include "mlx5_prm.h" -enum modify_reg { - REG_A, - REG_B, - REG_C_0, - REG_C_1, - REG_C_2, - REG_C_3, - REG_C_4, - REG_C_5, - REG_C_6, - REG_C_7, -}; - /* Private rte flow items. */ enum mlx5_rte_flow_item_type { MLX5_RTE_FLOW_ITEM_TYPE_END = INT_MIN, diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 853e879..8b93022 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -830,6 +830,7 @@ struct field_modify_info modify_tcp[] = { } static enum mlx5_modification_field reg_to_field[] = { + [REG_NONE] = MLX5_MODI_OUT_NONE, [REG_A] = MLX5_MODI_META_DATA_REG_A, [REG_B] = MLX5_MODI_META_DATA_REG_B, [REG_C_0] = MLX5_MODI_META_REG_C_0, @@ -1038,7 +1039,7 @@ struct field_modify_info modify_tcp[] = { return ret; if (!spec) return 0; - esw_priv = mlx5_port_to_eswitch_info(spec->id); + esw_priv = mlx5_port_to_eswitch_info(spec->id, false); if (!esw_priv) return rte_flow_error_set(error, rte_errno, RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec, @@ -2695,7 +2696,7 @@ struct field_modify_info modify_tcp[] = { "failed to obtain E-Switch info"); port_id = action->conf; port = port_id->original ? dev->data->port_id : port_id->id; - act_priv = mlx5_port_to_eswitch_info(port); + act_priv = mlx5_port_to_eswitch_info(port, false); if (!act_priv) return rte_flow_error_set (error, rte_errno, @@ -5090,7 +5091,7 @@ struct field_modify_info modify_tcp[] = { mask = pid_m ? pid_m->id : 0xffff; id = pid_v ? pid_v->id : dev->data->port_id; - priv = mlx5_port_to_eswitch_info(id); + priv = mlx5_port_to_eswitch_info(id, item == NULL); if (!priv) return -rte_errno; /* Translate to vport field or to metadata, depending on mode. */ @@ -5538,7 +5539,7 @@ struct field_modify_info modify_tcp[] = { (const struct rte_flow_action_port_id *)action->conf; port = conf->original ? dev->data->port_id : conf->id; - priv = mlx5_port_to_eswitch_info(port); + priv = mlx5_port_to_eswitch_info(port, false); if (!priv) return rte_flow_error_set(error, -rte_errno, RTE_FLOW_ERROR_TYPE_ACTION, diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index b9e53f5..c17ba66 100644 --- a/drivers/net/mlx5/mlx5_prm.h +++ b/drivers/net/mlx5/mlx5_prm.h @@ -392,6 +392,7 @@ enum { /* The field of packet to be modified. */ enum mlx5_modification_field { + MLX5_MODI_OUT_NONE = -1, MLX5_MODI_OUT_SMAC_47_16 = 1, MLX5_MODI_OUT_SMAC_15_0, MLX5_MODI_OUT_ETHERTYPE, @@ -455,6 +456,23 @@ enum mlx5_modification_field { MLX5_MODI_IN_TCP_ACK_NUM = 0x5C, }; +/* Total number of metadata reg_c's. */ +#define MLX5_MREG_C_NUM (MLX5_MODI_META_REG_C_7 - MLX5_MODI_META_REG_C_0 + 1) + +enum modify_reg { + REG_NONE = 0, + REG_A, + REG_B, + REG_C_0, + REG_C_1, + REG_C_2, + REG_C_3, + REG_C_4, + REG_C_5, + REG_C_6, + REG_C_7, +}; + /* Modification sub command. */ struct mlx5_modification_cmd { union { From patchwork Tue Nov 5 08:01:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62428 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 71914A0352; Tue, 5 Nov 2019 09:03:51 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D8FD41BEB4; Tue, 5 Nov 2019 09:02:21 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 7C9BD1BE80 for ; Tue, 5 Nov 2019 09:02:13 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:09 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA5829cF026492; Tue, 5 Nov 2019 10:02:09 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA5829BY030772; Tue, 5 Nov 2019 08:02:09 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA5829Mc030771; Tue, 5 Nov 2019 08:02:09 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:44 +0000 Message-Id: <1572940915-29416-10-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 09/20] net/mlx5: add devarg for extensive metadata support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The PMD parameter dv_xmeta_en is added to control extensive metadata support. A nonzero value enables extensive flow metadata support if device is capable and driver supports it. This can enable extensive support of MARK and META item of rte_flow. The newly introduced SET_TAG and SET_META actions do not depend on dv_xmeta_en parameter, because there is no compatibility issue for new entities. The dv_xmeta_en is disabled by default. There are some possible configurations, depending on parameter value: - 0, this is default value, defines the legacy mode, the MARK and META related actions and items operate only within NIC Tx and NIC Rx steering domains, no MARK and META information crosses the domain boundaries. The MARK item is 24 bits wide, the META item is 32 bits wide. - 1, this engages extensive metadata mode, the MARK and META related actions and items operate within all supported steering domains, including FDB, MARK and META information may cross the domain boundaries. The ``MARK`` item is 24 bits wide, the META item width depends on kernel and firmware configurations and might be 0, 16 or 32 bits. Within NIC Tx domain META data width is 32 bits for compatibility, the actual width of data transferred to the FDB domain depends on kernel configuration and may be vary. The actual supported width can be retrieved in runtime by series of rte_flow_validate() trials. - 2, this engages extensive metadata mode, the MARK and META related actions and items operate within all supported steering domains, including FDB, MARK and META information may cross the domain boundaries. The META item is 32 bits wide, the MARK item width depends on kernel and firmware configurations and might be 0, 16 or 24 bits. The actual supported width can be retrieved in runtime by series of rte_flow_validate() trials. If there is no E-Switch configuration the ``dv_xmeta_en`` parameter is ignored and the device is configured to operate in legacy mode (0). Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- doc/guides/nics/mlx5.rst | 49 ++++++++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5.c | 33 +++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_defs.h | 4 ++++ drivers/net/mlx5/mlx5_prm.h | 3 +++ 5 files changed, 90 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 4f1093f..8870969 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -578,6 +578,55 @@ Run-time configuration Disabled by default. +- ``dv_xmeta_en`` parameter [int] + + A nonzero value enables extensive flow metadata support if device is + capable and driver supports it. This can enable extensive support of + ``MARK`` and ``META`` item of ``rte_flow``. The newly introduced + ``SET_TAG`` and ``SET_META`` actions do not depend on ``dv_xmeta_en``. + + There are some possible configurations, depending on parameter value: + + - 0, this is default value, defines the legacy mode, the ``MARK`` and + ``META`` related actions and items operate only within NIC Tx and + NIC Rx steering domains, no ``MARK`` and ``META`` information crosses + the domain boundaries. The ``MARK`` item is 24 bits wide, the ``META`` + item is 32 bits wide. + + - 1, this engages extensive metadata mode, the ``MARK`` and ``META`` + related actions and items operate within all supported steering domains, + including FDB, ``MARK`` and ``META`` information may cross the domain + boundaries. The ``MARK`` item is 24 bits wide, the ``META`` item width + depends on kernel and firmware configurations and might be 0, 16 or + 32 bits. Within NIC Tx domain ``META`` data width is 32 bits for + compatibility, the actual width of data transferred to the FDB domain + depends on kernel configuration and may be vary. The actual supported + width can be retrieved in runtime by series of rte_flow_validate() + trials. + + - 2, this engages extensive metadata mode, the ``MARK`` and ``META`` + related actions and items operate within all supported steering domains, + including FDB, ``MARK`` and ``META`` information may cross the domain + boundaries. The ``META`` item is 32 bits wide, the ``MARK`` item width + depends on kernel and firmware configurations and might be 0, 16 or + 24 bits. The actual supported width can be retrieved in runtime by + series of rte_flow_validate() trials. + + +------+-----------+-----------+-------------+-------------+ + | Mode | ``MARK`` | ``META`` | ``META`` Tx | FDB/Through | + +======+===========+===========+=============+=============+ + | 0 | 24 bits | 32 bits | 32 bits | no | + +------+-----------+-----------+-------------+-------------+ + | 1 | 24 bits | vary 0-32 | 32 bits | yes | + +------+-----------+-----------+-------------+-------------+ + | 2 | vary 0-32 | 32 bits | 32 bits | yes | + +------+-----------+-----------+-------------+-------------+ + + If there is no E-Switch configuration the ``dv_xmeta_en`` parameter is + ignored and the device is configured to operate in legacy mode (0). + + Disabled by default (set to 0). + - ``dv_flow_en`` parameter [int] A nonzero value enables the DV flow steering assuming it is supported diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 1b86b7b..943d0e8 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -125,6 +125,9 @@ /* Activate DV flow steering. */ #define MLX5_DV_FLOW_EN "dv_flow_en" +/* Enable extensive flow metadata support. */ +#define MLX5_DV_XMETA_EN "dv_xmeta_en" + /* Activate Netlink support in VF mode. */ #define MLX5_VF_NL_EN "vf_nl_en" @@ -1310,6 +1313,16 @@ struct mlx5_flow_id_pool * config->dv_esw_en = !!tmp; } else if (strcmp(MLX5_DV_FLOW_EN, key) == 0) { config->dv_flow_en = !!tmp; + } else if (strcmp(MLX5_DV_XMETA_EN, key) == 0) { + if (tmp != MLX5_XMETA_MODE_LEGACY && + tmp != MLX5_XMETA_MODE_META16 && + tmp != MLX5_XMETA_MODE_META32) { + DRV_LOG(WARNING, "invalid extensive " + "metadata parameter"); + rte_errno = EINVAL; + return -rte_errno; + } + config->dv_xmeta_en = tmp; } else if (strcmp(MLX5_MR_EXT_MEMSEG_EN, key) == 0) { config->mr_ext_memseg_en = !!tmp; } else if (strcmp(MLX5_MAX_DUMP_FILES_NUM, key) == 0) { @@ -1361,6 +1374,7 @@ struct mlx5_flow_id_pool * MLX5_VF_NL_EN, MLX5_DV_ESW_EN, MLX5_DV_FLOW_EN, + MLX5_DV_XMETA_EN, MLX5_MR_EXT_MEMSEG_EN, MLX5_REPRESENTOR, MLX5_MAX_DUMP_FILES_NUM, @@ -1734,6 +1748,12 @@ struct mlx5_flow_id_pool * rte_errno = EINVAL; return rte_errno; } + if (sh_conf->dv_xmeta_en ^ config->dv_xmeta_en) { + DRV_LOG(ERR, "\"dv_xmeta_en\" configuration mismatch" + " for shared %s context", sh->ibdev_name); + rte_errno = EINVAL; + return rte_errno; + } return 0; } /** @@ -2347,10 +2367,23 @@ struct mlx5_flow_id_pool * err = -err; goto error; } + if (!priv->config.dv_esw_en && + priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) { + DRV_LOG(WARNING, "metadata mode %u is not supported " + "(no E-Switch)", priv->config.dv_xmeta_en); + priv->config.dv_xmeta_en = MLX5_XMETA_MODE_LEGACY; + } if (!mlx5_flow_ext_mreg_supported(eth_dev)) { DRV_LOG(DEBUG, "port %u extensive metadata register is not supported", eth_dev->data->port_id); + if (priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) { + DRV_LOG(ERR, "metadata mode %u is not supported " + "(no metadata registers available)", + priv->config.dv_xmeta_en); + err = ENOTSUP; + goto error; + } } return eth_dev; error: diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 6b82c6d..e59f8f6 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -238,6 +238,7 @@ struct mlx5_dev_config { unsigned int vf_nl_en:1; /* Enable Netlink requests in VF mode. */ unsigned int dv_esw_en:1; /* Enable E-Switch DV flow. */ unsigned int dv_flow_en:1; /* Enable DV flow. */ + unsigned int dv_xmeta_en:2; /* Enable extensive flow metadata. */ unsigned int swp:1; /* Tx generic tunnel checksum and TSO offload. */ unsigned int devx:1; /* Whether devx interface is available or not. */ unsigned int dest_tir:1; /* Whether advanced DR API is available. */ diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index e36ab55..a77c430 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -141,6 +141,10 @@ /* Cache size of mempool for Multi-Packet RQ. */ #define MLX5_MPRQ_MP_CACHE_SZ 32U +#define MLX5_XMETA_MODE_LEGACY 0 +#define MLX5_XMETA_MODE_META16 1 +#define MLX5_XMETA_MODE_META32 2 + /* Definition of static_assert found in /usr/include/assert.h */ #ifndef HAVE_STATIC_ASSERT #define static_assert _Static_assert diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index c17ba66..b405cb6 100644 --- a/drivers/net/mlx5/mlx5_prm.h +++ b/drivers/net/mlx5/mlx5_prm.h @@ -226,6 +226,9 @@ /* Default mark value used when none is provided. */ #define MLX5_FLOW_MARK_DEFAULT 0xffffff +/* Default mark mask for metadata legacy mode. */ +#define MLX5_FLOW_MARK_MASK 0xffffff + /* Maximum number of DS in WQE. Limited by 6-bit field. */ #define MLX5_DSEG_MAX 63 From patchwork Tue Nov 5 08:01:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62427 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id ACD98A0352; Tue, 5 Nov 2019 09:03:40 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2E4361BEAA; Tue, 5 Nov 2019 09:02:20 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 852D41BE86 for ; Tue, 5 Nov 2019 09:02:13 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:10 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582Aai026497; Tue, 5 Nov 2019 10:02:10 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582AEa030774; Tue, 5 Nov 2019 08:02:10 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582AFt030773; Tue, 5 Nov 2019 08:02:10 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com Date: Tue, 5 Nov 2019 08:01:45 +0000 Message-Id: <1572940915-29416-11-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 10/20] net/mlx5: adjust shared register according to mask X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The metadata register reg_c[0] might be used by kernel or firmware for their internal purposes. The actual used mask can be queried from the kernel. The remaining bits can be used by PMD to provide META or MARK feature. The code queries the mask of reg_c[0] and adjust the resource usage dynamically. Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.c | 95 +++++++++++++++++++++++++++++++++++------ drivers/net/mlx5/mlx5.h | 3 ++ drivers/net/mlx5/mlx5_flow_dv.c | 26 +++++++++-- 3 files changed, 107 insertions(+), 17 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 943d0e8..fb7b94b 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1584,6 +1584,60 @@ struct mlx5_flow_id_pool * } /** + * Configures the metadata mask fields in the shared context. + * + * @param [in] dev + * Pointer to Ethernet device. + */ +static void +mlx5_set_metadata_mask(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_ibv_shared *sh = priv->sh; + uint32_t meta, mark, reg_c0; + + reg_c0 = ~priv->vport_meta_mask; + switch (priv->config.dv_xmeta_en) { + case MLX5_XMETA_MODE_LEGACY: + meta = UINT32_MAX; + mark = MLX5_FLOW_MARK_MASK; + break; + case MLX5_XMETA_MODE_META16: + meta = reg_c0 >> rte_bsf32(reg_c0); + mark = MLX5_FLOW_MARK_MASK; + break; + case MLX5_XMETA_MODE_META32: + meta = UINT32_MAX; + mark = (reg_c0 >> rte_bsf32(reg_c0)) & MLX5_FLOW_MARK_MASK; + break; + default: + meta = 0; + mark = 0; + assert(false); + break; + } + if (sh->dv_mark_mask && sh->dv_mark_mask != mark) + DRV_LOG(WARNING, "metadata MARK mask mismatche %08X:%08X", + sh->dv_mark_mask, mark); + else + sh->dv_mark_mask = mark; + if (sh->dv_meta_mask && sh->dv_meta_mask != meta) + DRV_LOG(WARNING, "metadata META mask mismatche %08X:%08X", + sh->dv_meta_mask, meta); + else + sh->dv_meta_mask = meta; + if (sh->dv_regc0_mask && sh->dv_regc0_mask != reg_c0) + DRV_LOG(WARNING, "metadata reg_c0 mask mismatche %08X:%08X", + sh->dv_meta_mask, reg_c0); + else + sh->dv_regc0_mask = reg_c0; + DRV_LOG(DEBUG, "metadata mode %u", priv->config.dv_xmeta_en); + DRV_LOG(DEBUG, "metadata MARK mask %08X", sh->dv_mark_mask); + DRV_LOG(DEBUG, "metadata META mask %08X", sh->dv_meta_mask); + DRV_LOG(DEBUG, "metadata reg_c0 mask %08X", sh->dv_regc0_mask); +} + +/** * Allocate page of door-bells and register it using DevX API. * * @param [in] dev @@ -1803,7 +1857,7 @@ struct mlx5_flow_id_pool * uint16_t port_id; unsigned int i; #ifdef HAVE_MLX5DV_DR_DEVX_PORT - struct mlx5dv_devx_port devx_port; + struct mlx5dv_devx_port devx_port = { .comp_mask = 0 }; #endif /* Determine if this port representor is supposed to be spawned. */ @@ -2035,13 +2089,17 @@ struct mlx5_flow_id_pool * * vport index. The engaged part of metadata register is * defined by mask. */ - devx_port.comp_mask = MLX5DV_DEVX_PORT_VPORT | - MLX5DV_DEVX_PORT_MATCH_REG_C_0; - err = mlx5_glue->devx_port_query(sh->ctx, spawn->ibv_port, &devx_port); - if (err) { - DRV_LOG(WARNING, "can't query devx port %d on device %s", - spawn->ibv_port, spawn->ibv_dev->name); - devx_port.comp_mask = 0; + if (switch_info->representor || switch_info->master) { + devx_port.comp_mask = MLX5DV_DEVX_PORT_VPORT | + MLX5DV_DEVX_PORT_MATCH_REG_C_0; + err = mlx5_glue->devx_port_query(sh->ctx, spawn->ibv_port, + &devx_port); + if (err) { + DRV_LOG(WARNING, + "can't query devx port %d on device %s", + spawn->ibv_port, spawn->ibv_dev->name); + devx_port.comp_mask = 0; + } } if (devx_port.comp_mask & MLX5DV_DEVX_PORT_MATCH_REG_C_0) { priv->vport_meta_tag = devx_port.reg_c_0.value; @@ -2361,18 +2419,27 @@ struct mlx5_flow_id_pool * goto error; } priv->config.flow_prio = err; - /* Query availibility of metadata reg_c's. */ - err = mlx5_flow_discover_mreg_c(eth_dev); - if (err < 0) { - err = -err; - goto error; - } if (!priv->config.dv_esw_en && priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) { DRV_LOG(WARNING, "metadata mode %u is not supported " "(no E-Switch)", priv->config.dv_xmeta_en); priv->config.dv_xmeta_en = MLX5_XMETA_MODE_LEGACY; } + mlx5_set_metadata_mask(eth_dev); + if (priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY && + !priv->sh->dv_regc0_mask) { + DRV_LOG(ERR, "metadata mode %u is not supported " + "(no metadata reg_c[0] is available)", + priv->config.dv_xmeta_en); + err = ENOTSUP; + goto error; + } + /* Query availibility of metadata reg_c's. */ + err = mlx5_flow_discover_mreg_c(eth_dev); + if (err < 0) { + err = -err; + goto error; + } if (!mlx5_flow_ext_mreg_supported(eth_dev)) { DRV_LOG(DEBUG, "port %u extensive metadata register is not supported", diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index e59f8f6..92d445a 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -622,6 +622,9 @@ struct mlx5_ibv_shared { } mr; /* Shared DV/DR flow data section. */ pthread_mutex_t dv_mutex; /* DV context mutex. */ + uint32_t dv_meta_mask; /* flow META metadata supported mask. */ + uint32_t dv_mark_mask; /* flow MARK metadata supported mask. */ + uint32_t dv_regc0_mask; /* available bits of metatada reg_c[0]. */ uint32_t dv_refcnt; /* DV/DR data reference counter. */ void *fdb_domain; /* FDB Direct Rules name space handle. */ struct mlx5_flow_tbl_resource fdb_tbl[MLX5_MAX_TABLES_FDB]; diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 8b93022..049d6ae 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -899,13 +899,13 @@ struct field_modify_info modify_tcp[] = { * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -flow_dv_convert_action_copy_mreg(struct rte_eth_dev *dev __rte_unused, +flow_dv_convert_action_copy_mreg(struct rte_eth_dev *dev, struct mlx5_flow_dv_modify_hdr_resource *res, const struct rte_flow_action *action, struct rte_flow_error *error) { const struct mlx5_flow_action_copy_mreg *conf = action->conf; - uint32_t mask = RTE_BE32(UINT32_MAX); + rte_be32_t mask = RTE_BE32(UINT32_MAX); struct rte_flow_item item = { .spec = NULL, .mask = &mask, @@ -915,9 +915,29 @@ struct field_modify_info modify_tcp[] = { {0, 0, 0}, }; struct field_modify_info reg_dst = { - .offset = (uint32_t)-1, /* Same as src. */ + .offset = 0, .id = reg_to_field[conf->dst], }; + /* Adjust reg_c[0] usage according to reported mask. */ + if (conf->dst == REG_C_0 || conf->src == REG_C_0) { + struct mlx5_priv *priv = dev->data->dev_private; + uint32_t reg_c0 = priv->sh->dv_regc0_mask; + + assert(reg_c0); + assert(priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY); + if (conf->dst == REG_C_0) { + /* Copy to reg_c[0], within mask only. */ + reg_dst.offset = rte_bsf32(reg_c0); + /* + * Mask is ignoring the enianness, because + * there is no conversion in datapath. + */ + mask = reg_c0 >> reg_dst.offset; + } else { + /* Copy from reg_c[0] to destination lower bits. */ + mask = rte_cpu_to_be_32(reg_c0); + } + } return flow_dv_convert_modify_action(&item, reg_src, ®_dst, res, MLX5_MODIFICATION_TYPE_COPY, From patchwork Tue Nov 5 08:01:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62429 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5161AA0352; Tue, 5 Nov 2019 09:04:07 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 72B481BEB7; Tue, 5 Nov 2019 09:02:23 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 921584C8E for ; Tue, 5 Nov 2019 09:02:13 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:11 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582BDa026503; Tue, 5 Nov 2019 10:02:11 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582A3a030776; Tue, 5 Nov 2019 08:02:10 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582ANV030775; Tue, 5 Nov 2019 08:02:10 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:46 +0000 Message-Id: <1572940915-29416-12-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 11/20] net/mlx5: check the maximal modify actions number X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" If the extensive metadata registers are supported, it can be regarded inclusively that the extensive metadata support is possible. E.g. metadata register copy action, supporting 16 modify header actions, reserving register across different steering domain (FDB and NIC) and so on. This patch handles the maximal amount of header modify actions depending on discovered metadata registers support. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow.h | 9 +++++++-- drivers/net/mlx5/mlx5_flow_dv.c | 25 +++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index f2b6726..c1d0a65 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -348,8 +348,13 @@ struct mlx5_flow_dv_tag_resource { uint32_t tag; /**< the tag value. */ }; -/* Number of modification commands. */ -#define MLX5_MODIFY_NUM 8 +/* + * Number of modification commands. + * If extensive metadata registers are supported + * the maximal actions amount is 16 and 8 otherwise. + */ +#define MLX5_MODIFY_NUM 16 +#define MLX5_MODIFY_NUM_NO_MREG 8 /* Modify resource structure */ struct mlx5_flow_dv_modify_hdr_resource { diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 049d6ae..f83c6ff 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -2732,6 +2732,27 @@ struct field_modify_info modify_tcp[] = { } /** + * Get the maximum number of modify header actions. + * + * @param dev + * Pointer to rte_eth_dev structure. + * + * @return + * Max number of modify header actions device can support. + */ +static unsigned int +flow_dv_modify_hdr_action_max(struct rte_eth_dev *dev) +{ + /* + * There's no way to directly query the max cap. Although it has to be + * acquried by iterative trial, it is a safe assumption that more + * actions are supported by FW if extensive metadata register is + * supported. + */ + return mlx5_flow_ext_mreg_supported(dev) ? MLX5_MODIFY_NUM : + MLX5_MODIFY_NUM_NO_MREG; +} +/** * Find existing modify-header resource or create and register a new one. * * @param dev[in, out] @@ -2758,6 +2779,10 @@ struct field_modify_info modify_tcp[] = { struct mlx5_flow_dv_modify_hdr_resource *cache_resource; struct mlx5dv_dr_domain *ns; + if (resource->actions_num > flow_dv_modify_hdr_action_max(dev)) + return rte_flow_error_set(error, EOVERFLOW, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "too many modify header items"); if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB) ns = sh->fdb_domain; else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX) From patchwork Tue Nov 5 08:01:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62431 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0CE9DA0352; Tue, 5 Nov 2019 09:04:35 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D18841BEC9; Tue, 5 Nov 2019 09:02:26 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id ACD845B3E for ; Tue, 5 Nov 2019 09:02:13 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:12 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582Bwr026506; Tue, 5 Nov 2019 10:02:11 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582BSm030779; Tue, 5 Nov 2019 08:02:11 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582B0r030778; Tue, 5 Nov 2019 08:02:11 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com Date: Tue, 5 Nov 2019 08:01:47 +0000 Message-Id: <1572940915-29416-13-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 12/20] net/mlx5: update metadata register id query X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The NIC might support up to 8 extensive metadata registers. These registers are supposed to be used by multiple features. There is register id query routine to allow determine which register is actually used by specified feature. Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow.c | 76 +++++++++++++++++++++++++++++--------------- drivers/net/mlx5/mlx5_flow.h | 17 ++++++++++ 2 files changed, 67 insertions(+), 26 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index f32ea8d..c38208c 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -316,12 +316,6 @@ struct mlx5_flow_tunnel_info { }, }; -enum mlx5_feature_name { - MLX5_HAIRPIN_RX, - MLX5_HAIRPIN_TX, - MLX5_APPLICATION, -}; - /** * Translate tag ID to register. * @@ -338,37 +332,66 @@ enum mlx5_feature_name { * The request register on success, a negative errno * value otherwise and rte_errno is set. */ -__rte_unused -static enum modify_reg flow_get_reg_id(struct rte_eth_dev *dev, - enum mlx5_feature_name feature, - uint32_t id, - struct rte_flow_error *error) +enum modify_reg +mlx5_flow_get_reg_id(struct rte_eth_dev *dev, + enum mlx5_feature_name feature, + uint32_t id, + struct rte_flow_error *error) { - static enum modify_reg id2reg[] = { - [0] = REG_A, - [1] = REG_C_2, - [2] = REG_C_3, - [3] = REG_C_4, - [4] = REG_B,}; - - dev = (void *)dev; + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *config = &priv->config; + switch (feature) { case MLX5_HAIRPIN_RX: return REG_B; case MLX5_HAIRPIN_TX: return REG_A; - case MLX5_APPLICATION: - if (id > 4) + case MLX5_METADATA_RX: + switch (config->dv_xmeta_en) { + case MLX5_XMETA_MODE_LEGACY: + return REG_B; + case MLX5_XMETA_MODE_META16: + return REG_C_0; + case MLX5_XMETA_MODE_META32: + return REG_C_1; + } + break; + case MLX5_METADATA_TX: + return REG_A; + case MLX5_METADATA_FDB: + return REG_C_0; + case MLX5_FLOW_MARK: + switch (config->dv_xmeta_en) { + case MLX5_XMETA_MODE_LEGACY: + return REG_NONE; + case MLX5_XMETA_MODE_META16: + return REG_C_1; + case MLX5_XMETA_MODE_META32: + return REG_C_0; + } + break; + case MLX5_COPY_MARK: + return REG_C_2; + case MLX5_APP_TAG: + /* + * Suppose engaging reg_c_2 .. reg_c_7 registers. + * reg_c_2 is reserved for coloring by meters. + */ + if (id > (REG_C_7 - REG_C_3)) return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, NULL, "invalid tag id"); - return id2reg[id]; + if (config->flow_mreg_c[id + REG_C_3 - REG_C_0] == REG_NONE) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, + NULL, "unsupported tag id"); + return config->flow_mreg_c[id + REG_C_3 - REG_C_0]; } + assert(false); return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, NULL, "invalid feature name"); } - /** * Check extensive flow metadata register support. * @@ -2667,7 +2690,6 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, struct mlx5_rte_flow_item_tag *tag_item; struct rte_flow_item *item; char *addr; - struct rte_flow_error error; int encap = 0; mlx5_flow_id_get(priv->sh->flow_id_pool, flow_id); @@ -2733,7 +2755,8 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, rte_memcpy(actions_rx, actions, sizeof(struct rte_flow_action)); actions_rx++; set_tag = (void *)actions_rx; - set_tag->id = flow_get_reg_id(dev, MLX5_HAIRPIN_RX, 0, &error); + set_tag->id = mlx5_flow_get_reg_id(dev, MLX5_HAIRPIN_RX, 0, NULL); + assert(set_tag->id > REG_NONE); set_tag->data = *flow_id; tag_action->conf = set_tag; /* Create Tx item list. */ @@ -2743,7 +2766,8 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, item->type = MLX5_RTE_FLOW_ITEM_TYPE_TAG; tag_item = (void *)addr; tag_item->data = *flow_id; - tag_item->id = flow_get_reg_id(dev, MLX5_HAIRPIN_TX, 0, NULL); + tag_item->id = mlx5_flow_get_reg_id(dev, MLX5_HAIRPIN_TX, 0, NULL); + assert(set_tag->id > REG_NONE); item->spec = tag_item; addr += sizeof(struct mlx5_rte_flow_item_tag); tag_item = (void *)addr; diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index c1d0a65..9371e11 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -63,6 +63,18 @@ struct mlx5_rte_flow_item_tx_queue { uint32_t queue; }; +/* Feature name to allocate metadata register. */ +enum mlx5_feature_name { + MLX5_HAIRPIN_RX, + MLX5_HAIRPIN_TX, + MLX5_METADATA_RX, + MLX5_METADATA_TX, + MLX5_METADATA_FDB, + MLX5_FLOW_MARK, + MLX5_APP_TAG, + MLX5_COPY_MARK, +}; + /* Pattern outer Layer bits. */ #define MLX5_FLOW_LAYER_OUTER_L2 (1u << 0) #define MLX5_FLOW_LAYER_OUTER_L3_IPV4 (1u << 1) @@ -534,6 +546,7 @@ struct mlx5_flow_driver_ops { mlx5_flow_query_t query; }; + #define MLX5_CNT_CONTAINER(sh, batch, thread) (&(sh)->cmng.ccont \ [(((sh)->cmng.mhi[batch] >> (thread)) & 0x1) * 2 + (batch)]) #define MLX5_CNT_CONTAINER_UNUSED(sh, batch, thread) (&(sh)->cmng.ccont \ @@ -554,6 +567,10 @@ uint64_t mlx5_flow_hashfields_adjust(struct mlx5_flow *dev_flow, int tunnel, uint64_t hash_fields); uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, uint32_t subpriority); +enum modify_reg mlx5_flow_get_reg_id(struct rte_eth_dev *dev, + enum mlx5_feature_name feature, + uint32_t id, + struct rte_flow_error *error); const struct rte_flow_action *mlx5_flow_find_action (const struct rte_flow_action *actions, enum rte_flow_action_type action); From patchwork Tue Nov 5 08:01:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62430 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B323AA0352; Tue, 5 Nov 2019 09:04:20 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2D21D1BEC2; Tue, 5 Nov 2019 09:02:25 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id B2AF21BE91 for ; Tue, 5 Nov 2019 09:02:13 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:12 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582Cjr026515; Tue, 5 Nov 2019 10:02:12 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582CSd030781; Tue, 5 Nov 2019 08:02:12 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582CiT030780; Tue, 5 Nov 2019 08:02:12 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:48 +0000 Message-Id: <1572940915-29416-14-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 13/20] net/mlx5: add flow tag support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support of new rte_flow item and action - TAG and SET_TAG. TAG is a transient value which can be kept during flow matching. This is supported through device metadata register reg_c[]. Although there are 8 registers are available on the current mlx5 device, some of them can be reserved for firmware or kernel purposes. The availability should be queried by iterative trial-and-error mlx5_flow_discover_mreg_c() routine. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow_dv.c | 232 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 228 insertions(+), 4 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index f83c6ff..08e78b0 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -870,10 +870,12 @@ struct field_modify_info modify_tcp[] = { return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION, NULL, "too many items to modify"); + assert(conf->id != REG_NONE); + assert(conf->id < RTE_DIM(reg_to_field)); actions[i].action_type = MLX5_MODIFICATION_TYPE_SET; actions[i].field = reg_to_field[conf->id]; actions[i].data0 = rte_cpu_to_be_32(actions[i].data0); - actions[i].data1 = conf->data; + actions[i].data1 = rte_cpu_to_be_32(conf->data); ++i; resource->actions_num = i; if (!resource->actions_num) @@ -884,6 +886,52 @@ struct field_modify_info modify_tcp[] = { } /** + * Convert SET_TAG action to DV specification. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in,out] resource + * Pointer to the modify-header resource. + * @param[in] conf + * Pointer to action specification. + * @param[out] error + * Pointer to the error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_convert_action_set_tag + (struct rte_eth_dev *dev, + struct mlx5_flow_dv_modify_hdr_resource *resource, + const struct rte_flow_action_set_tag *conf, + struct rte_flow_error *error) +{ + rte_be32_t data = rte_cpu_to_be_32(conf->data); + rte_be32_t mask = rte_cpu_to_be_32(conf->mask); + struct rte_flow_item item = { + .spec = &data, + .mask = &mask, + }; + struct field_modify_info reg_c_x[] = { + [1] = {0, 0, 0}, + }; + enum mlx5_modification_field reg_type; + int ret; + + ret = mlx5_flow_get_reg_id(dev, MLX5_APP_TAG, conf->index, error); + if (ret < 0) + return ret; + assert(ret != REG_NONE); + assert((unsigned int)ret < RTE_DIM(reg_to_field)); + reg_type = reg_to_field[ret]; + assert(reg_type > 0); + reg_c_x[0] = (struct field_modify_info){4, 0, reg_type}; + return flow_dv_convert_modify_action(&item, reg_c_x, NULL, resource, + MLX5_MODIFICATION_TYPE_SET, error); +} + +/** * Convert internal COPY_REG action to DV specification. * * @param[in] dev @@ -999,6 +1047,65 @@ struct field_modify_info modify_tcp[] = { } /** + * Validate TAG item. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] item + * Item specification. + * @param[in] attr + * Attributes of flow that includes this item. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_item_tag(struct rte_eth_dev *dev, + const struct rte_flow_item *item, + const struct rte_flow_attr *attr __rte_unused, + struct rte_flow_error *error) +{ + const struct rte_flow_item_tag *spec = item->spec; + const struct rte_flow_item_tag *mask = item->mask; + const struct rte_flow_item_tag nic_mask = { + .data = RTE_BE32(UINT32_MAX), + .index = 0xff, + }; + int ret; + + if (!mlx5_flow_ext_mreg_supported(dev)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, item, + "extensive metadata register" + " isn't supported"); + if (!spec) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "data cannot be empty"); + if (!mask) + mask = &rte_flow_item_tag_mask; + ret = mlx5_flow_item_acceptable(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_tag), + error); + if (ret < 0) + return ret; + if (mask->index != 0xff) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, NULL, + "partial mask for tag index" + " is not supported"); + ret = mlx5_flow_get_reg_id(dev, MLX5_APP_TAG, spec->index, error); + if (ret < 0) + return ret; + assert(ret != REG_NONE); + return 0; +} + +/** * Validate vport item. * * @param[in] dev @@ -1359,6 +1466,62 @@ struct field_modify_info modify_tcp[] = { } /** + * Validate SET_TAG action. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] action + * Pointer to the encap action. + * @param[in] action_flags + * Holds the actions detected until now. + * @param[in] attr + * Pointer to flow attributes + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_action_set_tag(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + uint64_t action_flags, + const struct rte_flow_attr *attr, + struct rte_flow_error *error) +{ + const struct rte_flow_action_set_tag *conf; + const uint64_t terminal_action_flags = + MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | + MLX5_FLOW_ACTION_RSS; + int ret; + + if (!mlx5_flow_ext_mreg_supported(dev)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "extensive metadata register" + " isn't supported"); + if (!(action->conf)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "configuration cannot be null"); + conf = (const struct rte_flow_action_set_tag *)action->conf; + if (!conf->mask) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "zero mask doesn't have any effect"); + ret = mlx5_flow_get_reg_id(dev, MLX5_APP_TAG, conf->index, error); + if (ret < 0) + return ret; + if (!attr->transfer && attr->ingress && + (action_flags & terminal_action_flags)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "set_tag has no effect" + " with terminal actions"); + return 0; +} + +/** * Validate count action. * * @param[in] dev @@ -3748,6 +3911,13 @@ struct field_modify_info modify_tcp[] = { return ret; last_item = MLX5_FLOW_LAYER_ICMP6; break; + case RTE_FLOW_ITEM_TYPE_TAG: + ret = flow_dv_validate_item_tag(dev, items, + attr, error); + if (ret < 0) + return ret; + last_item = MLX5_FLOW_ITEM_TAG; + break; case MLX5_RTE_FLOW_ITEM_TYPE_TAG: case MLX5_RTE_FLOW_ITEM_TYPE_TX_QUEUE: break; @@ -3795,6 +3965,17 @@ struct field_modify_info modify_tcp[] = { action_flags |= MLX5_FLOW_ACTION_MARK; ++actions_n; break; + case RTE_FLOW_ACTION_TYPE_SET_TAG: + ret = flow_dv_validate_action_set_tag(dev, actions, + action_flags, + attr, error); + if (ret < 0) + return ret; + /* Count all modify-header actions as one action. */ + if (!(action_flags & MLX5_FLOW_MODIFY_HDR_ACTIONS)) + ++actions_n; + action_flags |= MLX5_FLOW_ACTION_SET_TAG; + break; case RTE_FLOW_ACTION_TYPE_DROP: ret = mlx5_flow_validate_action_drop(action_flags, attr, error); @@ -5082,8 +5263,38 @@ struct field_modify_info modify_tcp[] = { { const struct mlx5_rte_flow_item_tag *tag_v = item->spec; const struct mlx5_rte_flow_item_tag *tag_m = item->mask; - enum modify_reg reg = tag_v->id; + assert(tag_v); + flow_dv_match_meta_reg(matcher, key, tag_v->id, tag_v->data, + tag_m ? tag_m->data : UINT32_MAX); +} + +/** + * Add TAG item to matcher + * + * @param[in] dev + * The devich to configure through. + * @param[in, out] matcher + * Flow matcher. + * @param[in, out] key + * Flow matcher value. + * @param[in] item + * Flow pattern to translate. + */ +static void +flow_dv_translate_item_tag(struct rte_eth_dev *dev, + void *matcher, void *key, + const struct rte_flow_item *item) +{ + const struct rte_flow_item_tag *tag_v = item->spec; + const struct rte_flow_item_tag *tag_m = item->mask; + enum modify_reg reg; + + assert(tag_v); + tag_m = tag_m ? tag_m : &rte_flow_item_tag_mask; + /* Get the metadata register index for the tag. */ + reg = mlx5_flow_get_reg_id(dev, MLX5_APP_TAG, tag_v->index, NULL); + assert(reg > 0); flow_dv_match_meta_reg(matcher, key, reg, tag_v->data, tag_m->data); } @@ -5758,6 +5969,14 @@ struct field_modify_info modify_tcp[] = { dev_flow->dv.tag_resource->action; action_flags |= MLX5_FLOW_ACTION_MARK; break; + case RTE_FLOW_ACTION_TYPE_SET_TAG: + if (flow_dv_convert_action_set_tag + (dev, &mhdr_res, + (const struct rte_flow_action_set_tag *) + actions->conf, error)) + return -rte_errno; + action_flags |= MLX5_FLOW_ACTION_SET_TAG; + break; case RTE_FLOW_ACTION_TYPE_DROP: action_flags |= MLX5_FLOW_ACTION_DROP; break; @@ -6038,7 +6257,7 @@ struct field_modify_info modify_tcp[] = { break; case RTE_FLOW_ACTION_TYPE_END: actions_end = true; - if (action_flags & MLX5_FLOW_MODIFY_HDR_ACTIONS) { + if (mhdr_res.actions_num) { /* create modify action if needed. */ if (flow_dv_modify_hdr_resource_register (dev, &mhdr_res, dev_flow, error)) @@ -6050,7 +6269,7 @@ struct field_modify_info modify_tcp[] = { default: break; } - if ((action_flags & MLX5_FLOW_MODIFY_HDR_ACTIONS) && + if (mhdr_res.actions_num && modify_action_position == UINT32_MAX) modify_action_position = actions_n++; } @@ -6213,6 +6432,11 @@ struct field_modify_info modify_tcp[] = { items, tunnel); last_item = MLX5_FLOW_LAYER_ICMP6; break; + case RTE_FLOW_ITEM_TYPE_TAG: + flow_dv_translate_item_tag(dev, match_mask, + match_value, items); + last_item = MLX5_FLOW_ITEM_TAG; + break; case MLX5_RTE_FLOW_ITEM_TYPE_TAG: flow_dv_translate_mlx5_item_tag(match_mask, match_value, items); From patchwork Tue Nov 5 08:01:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62438 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4DA72A0352; Tue, 5 Nov 2019 09:07:08 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D5F971BEDA; Tue, 5 Nov 2019 09:02:37 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id AF0791BE99 for ; Tue, 5 Nov 2019 09:02:18 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:13 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582DgV026518; Tue, 5 Nov 2019 10:02:13 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582DBC030783; Tue, 5 Nov 2019 08:02:13 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582DXv030782; Tue, 5 Nov 2019 08:02:13 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:49 +0000 Message-Id: <1572940915-29416-15-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 14/20] net/mlx5: extend flow mark support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Flow MARK item is newly supported along with MARK action. MARK action and item are supported on both Rx and Tx. It works on the metadata reg_c[] only if extensive flow metadata register is supported. Without the support, MARK action behaves same as before - valid only on Rx and no MARK item is valid. FLAG action is also modified accordingly. FLAG action is supported on both Rx and Tx via reg_c[] if extensive flow metadata register is supported. However, the new MARK/FLAG item and action are currently disabled until register copy on loopback is supported by forthcoming patches. The actual index of engaged metadata reg_c[] register to support FLAG/MARK actions depends on dv_xmeta_en devarg value. For extensive metadata mode 1 the reg_c[1] is used and transitive MARK data width is 24. For extensive metadata mode 2 the reg_c[0] is used and transitive MARK data width might be restricted to 0 or 16 bits, depending on kernel usage of reg_c[0]. The actual supported width can be discovered by series of trials with rte_flow_validate(). Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow.h | 5 +- drivers/net/mlx5/mlx5_flow_dv.c | 383 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 370 insertions(+), 18 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 9371e11..d6209ff 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -102,6 +102,7 @@ enum mlx5_feature_name { #define MLX5_FLOW_ITEM_METADATA (1u << 16) #define MLX5_FLOW_ITEM_PORT_ID (1u << 17) #define MLX5_FLOW_ITEM_TAG (1u << 18) +#define MLX5_FLOW_ITEM_MARK (1u << 19) /* Pattern MISC bits. */ #define MLX5_FLOW_LAYER_ICMP (1u << 19) @@ -194,6 +195,7 @@ enum mlx5_feature_name { #define MLX5_FLOW_ACTION_INC_TCP_ACK (1u << 30) #define MLX5_FLOW_ACTION_DEC_TCP_ACK (1u << 31) #define MLX5_FLOW_ACTION_SET_TAG (1ull << 32) +#define MLX5_FLOW_ACTION_MARK_EXT (1ull << 33) #define MLX5_FLOW_FATE_ACTIONS \ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \ @@ -228,7 +230,8 @@ enum mlx5_feature_name { MLX5_FLOW_ACTION_INC_TCP_ACK | \ MLX5_FLOW_ACTION_DEC_TCP_ACK | \ MLX5_FLOW_ACTION_OF_SET_VLAN_VID | \ - MLX5_FLOW_ACTION_SET_TAG) + MLX5_FLOW_ACTION_SET_TAG | \ + MLX5_FLOW_ACTION_MARK_EXT) #define MLX5_FLOW_VLAN_ACTIONS (MLX5_FLOW_ACTION_OF_POP_VLAN | \ MLX5_FLOW_ACTION_OF_PUSH_VLAN) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 08e78b0..5714a6d 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -993,6 +993,125 @@ struct field_modify_info modify_tcp[] = { } /** + * Convert MARK action to DV specification. This routine is used + * in extensive metadata only and requires metadata register to be + * handled. In legacy mode hardware tag resource is engaged. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] conf + * Pointer to MARK action specification. + * @param[in,out] resource + * Pointer to the modify-header resource. + * @param[out] error + * Pointer to the error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_convert_action_mark(struct rte_eth_dev *dev, + const struct rte_flow_action_mark *conf, + struct mlx5_flow_dv_modify_hdr_resource *resource, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + rte_be32_t mask = rte_cpu_to_be_32(MLX5_FLOW_MARK_MASK & + priv->sh->dv_mark_mask); + rte_be32_t data = rte_cpu_to_be_32(conf->id) & mask; + struct rte_flow_item item = { + .spec = &data, + .mask = &mask, + }; + struct field_modify_info reg_c_x[] = { + {4, 0, 0}, /* dynamic instead of MLX5_MODI_META_REG_C_1. */ + {0, 0, 0}, + }; + enum modify_reg reg; + + if (!mask) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, "zero mark action mask"); + reg = mlx5_flow_get_reg_id(dev, MLX5_FLOW_MARK, 0, error); + if (reg < 0) + return reg; + assert(reg > 0); + reg_c_x[0].id = reg_to_field[reg]; + return flow_dv_convert_modify_action(&item, reg_c_x, NULL, resource, + MLX5_MODIFICATION_TYPE_SET, error); +} + +/** + * Validate MARK item. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] item + * Item specification. + * @param[in] attr + * Attributes of flow that includes this item. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_item_mark(struct rte_eth_dev *dev, + const struct rte_flow_item *item, + const struct rte_flow_attr *attr __rte_unused, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *config = &priv->config; + const struct rte_flow_item_mark *spec = item->spec; + const struct rte_flow_item_mark *mask = item->mask; + const struct rte_flow_item_mark nic_mask = { + .id = priv->sh->dv_mark_mask, + }; + int ret; + + if (config->dv_xmeta_en == MLX5_XMETA_MODE_LEGACY) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, item, + "extended metadata feature" + " isn't enabled"); + if (!mlx5_flow_ext_mreg_supported(dev)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, item, + "extended metadata register" + " isn't supported"); + if (!nic_mask.id) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, item, + "extended metadata register" + " isn't available"); + ret = mlx5_flow_get_reg_id(dev, MLX5_FLOW_MARK, 0, error); + if (ret < 0) + return ret; + if (!spec) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "data cannot be empty"); + if (spec->id >= (MLX5_FLOW_MARK_MAX & nic_mask.id)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &spec->id, + "mark id exceeds the limit"); + if (!mask) + mask = &nic_mask; + ret = mlx5_flow_item_acceptable(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_mark), + error); + if (ret < 0) + return ret; + return 0; +} + +/** * Validate META item. * * @param[in] dev @@ -1465,6 +1584,139 @@ struct field_modify_info modify_tcp[] = { return 0; } +/* + * Validate the FLAG action. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] action_flags + * Holds the actions detected until now. + * @param[in] attr + * Pointer to flow attributes + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_action_flag(struct rte_eth_dev *dev, + uint64_t action_flags, + const struct rte_flow_attr *attr, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *config = &priv->config; + int ret; + + /* Fall back if no extended metadata register support. */ + if (config->dv_xmeta_en == MLX5_XMETA_MODE_LEGACY) + return mlx5_flow_validate_action_flag(action_flags, attr, + error); + /* Extensive metadata mode requires registers. */ + if (!mlx5_flow_ext_mreg_supported(dev)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "no metadata registers " + "to support flag action"); + if (!(priv->sh->dv_mark_mask & MLX5_FLOW_MARK_DEFAULT)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "extended metadata register" + " isn't available"); + ret = mlx5_flow_get_reg_id(dev, MLX5_FLOW_MARK, 0, error); + if (ret < 0) + return ret; + assert(ret > 0); + if (action_flags & MLX5_FLOW_ACTION_DROP) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "can't drop and flag in same flow"); + if (action_flags & MLX5_FLOW_ACTION_MARK) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "can't mark and flag in same flow"); + if (action_flags & MLX5_FLOW_ACTION_FLAG) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "can't have 2 flag" + " actions in same flow"); + return 0; +} + +/** + * Validate MARK action. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] action + * Pointer to action. + * @param[in] action_flags + * Holds the actions detected until now. + * @param[in] attr + * Pointer to flow attributes + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_action_mark(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + uint64_t action_flags, + const struct rte_flow_attr *attr, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *config = &priv->config; + const struct rte_flow_action_mark *mark = action->conf; + int ret; + + /* Fall back if no extended metadata register support. */ + if (config->dv_xmeta_en == MLX5_XMETA_MODE_LEGACY) + return mlx5_flow_validate_action_mark(action, action_flags, + attr, error); + /* Extensive metadata mode requires registers. */ + if (!mlx5_flow_ext_mreg_supported(dev)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "no metadata registers " + "to support mark action"); + if (!priv->sh->dv_mark_mask) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "extended metadata register" + " isn't available"); + ret = mlx5_flow_get_reg_id(dev, MLX5_FLOW_MARK, 0, error); + if (ret < 0) + return ret; + assert(ret > 0); + if (!mark) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "configuration cannot be null"); + if (mark->id >= (MLX5_FLOW_MARK_MAX & priv->sh->dv_mark_mask)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &mark->id, + "mark id exceeds the limit"); + if (action_flags & MLX5_FLOW_ACTION_DROP) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "can't drop and mark in same flow"); + if (action_flags & MLX5_FLOW_ACTION_FLAG) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "can't flag and mark in same flow"); + if (action_flags & MLX5_FLOW_ACTION_MARK) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "can't have 2 mark actions in same" + " flow"); + return 0; +} + /** * Validate SET_TAG action. * @@ -3732,6 +3984,8 @@ struct field_modify_info modify_tcp[] = { .dst_port = RTE_BE16(UINT16_MAX), } }; + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *dev_conf = &priv->config; if (items == NULL) return -1; @@ -3888,6 +4142,14 @@ struct field_modify_info modify_tcp[] = { return ret; last_item = MLX5_FLOW_LAYER_MPLS; break; + + case RTE_FLOW_ITEM_TYPE_MARK: + ret = flow_dv_validate_item_mark(dev, items, attr, + error); + if (ret < 0) + return ret; + last_item = MLX5_FLOW_ITEM_MARK; + break; case RTE_FLOW_ITEM_TYPE_META: ret = flow_dv_validate_item_meta(dev, items, attr, error); @@ -3949,21 +4211,39 @@ struct field_modify_info modify_tcp[] = { ++actions_n; break; case RTE_FLOW_ACTION_TYPE_FLAG: - ret = mlx5_flow_validate_action_flag(action_flags, - attr, error); + ret = flow_dv_validate_action_flag(dev, action_flags, + attr, error); if (ret < 0) return ret; - action_flags |= MLX5_FLOW_ACTION_FLAG; - ++actions_n; + if (dev_conf->dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) { + /* Count all modify-header actions as one. */ + if (!(action_flags & + MLX5_FLOW_MODIFY_HDR_ACTIONS)) + ++actions_n; + action_flags |= MLX5_FLOW_ACTION_FLAG | + MLX5_FLOW_ACTION_MARK_EXT; + } else { + action_flags |= MLX5_FLOW_ACTION_FLAG; + ++actions_n; + } break; case RTE_FLOW_ACTION_TYPE_MARK: - ret = mlx5_flow_validate_action_mark(actions, - action_flags, - attr, error); + ret = flow_dv_validate_action_mark(dev, actions, + action_flags, + attr, error); if (ret < 0) return ret; - action_flags |= MLX5_FLOW_ACTION_MARK; - ++actions_n; + if (dev_conf->dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) { + /* Count all modify-header actions as one. */ + if (!(action_flags & + MLX5_FLOW_MODIFY_HDR_ACTIONS)) + ++actions_n; + action_flags |= MLX5_FLOW_ACTION_MARK | + MLX5_FLOW_ACTION_MARK_EXT; + } else { + action_flags |= MLX5_FLOW_ACTION_MARK; + ++actions_n; + } break; case RTE_FLOW_ACTION_TYPE_SET_TAG: ret = flow_dv_validate_action_set_tag(dev, actions, @@ -4234,12 +4514,14 @@ struct field_modify_info modify_tcp[] = { " actions in the same rule"); /* Eswitch has few restrictions on using items and actions */ if (attr->transfer) { - if (action_flags & MLX5_FLOW_ACTION_FLAG) + if (!mlx5_flow_ext_mreg_supported(dev) && + action_flags & MLX5_FLOW_ACTION_FLAG) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, NULL, "unsupported action FLAG"); - if (action_flags & MLX5_FLOW_ACTION_MARK) + if (!mlx5_flow_ext_mreg_supported(dev) && + action_flags & MLX5_FLOW_ACTION_MARK) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, NULL, @@ -5202,6 +5484,44 @@ struct field_modify_info modify_tcp[] = { } /** + * Add MARK item to matcher + * + * @param[in] dev + * The device to configure through. + * @param[in, out] matcher + * Flow matcher. + * @param[in, out] key + * Flow matcher value. + * @param[in] item + * Flow pattern to translate. + */ +static void +flow_dv_translate_item_mark(struct rte_eth_dev *dev, + void *matcher, void *key, + const struct rte_flow_item *item) +{ + struct mlx5_priv *priv = dev->data->dev_private; + const struct rte_flow_item_mark *mark; + uint32_t value; + uint32_t mask; + + mark = item->mask ? (const void *)item->mask : + &rte_flow_item_mark_mask; + mask = mark->id & priv->sh->dv_mark_mask; + mark = (const void *)item->spec; + assert(mark); + value = mark->id & priv->sh->dv_mark_mask & mask; + if (mask) { + enum modify_reg reg; + + /* Get the metadata register index for the mark. */ + reg = mlx5_flow_get_reg_id(dev, MLX5_FLOW_MARK, 0, NULL); + assert(reg > 0); + flow_dv_match_meta_reg(matcher, key, reg, value, mask); + } +} + +/** * Add META item to matcher * * @param[in, out] matcher @@ -5210,8 +5530,6 @@ struct field_modify_info modify_tcp[] = { * Flow matcher value. * @param[in] item * Flow pattern to translate. - * @param[in] inner - * Item is inner pattern. */ static void flow_dv_translate_item_meta(void *matcher, void *key, @@ -5882,6 +6200,7 @@ struct field_modify_info modify_tcp[] = { struct rte_flow_error *error) { struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *dev_conf = &priv->config; struct rte_flow *flow = dev_flow->flow; uint64_t item_flags = 0; uint64_t last_item = 0; @@ -5916,7 +6235,7 @@ struct field_modify_info modify_tcp[] = { if (attr->transfer) mhdr_res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB; if (priority == MLX5_FLOW_PRIO_RSVD) - priority = priv->config.flow_prio - 1; + priority = dev_conf->flow_prio - 1; for (; !actions_end ; actions++) { const struct rte_flow_action_queue *queue; const struct rte_flow_action_rss *rss; @@ -5947,6 +6266,19 @@ struct field_modify_info modify_tcp[] = { action_flags |= MLX5_FLOW_ACTION_PORT_ID; break; case RTE_FLOW_ACTION_TYPE_FLAG: + action_flags |= MLX5_FLOW_ACTION_FLAG; + if (dev_conf->dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) { + struct rte_flow_action_mark mark = { + .id = MLX5_FLOW_MARK_DEFAULT, + }; + + if (flow_dv_convert_action_mark(dev, &mark, + &mhdr_res, + error)) + return -rte_errno; + action_flags |= MLX5_FLOW_ACTION_MARK_EXT; + break; + } tag_resource.tag = mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT); if (!dev_flow->dv.tag_resource) @@ -5955,9 +6287,22 @@ struct field_modify_info modify_tcp[] = { return errno; dev_flow->dv.actions[actions_n++] = dev_flow->dv.tag_resource->action; - action_flags |= MLX5_FLOW_ACTION_FLAG; break; case RTE_FLOW_ACTION_TYPE_MARK: + action_flags |= MLX5_FLOW_ACTION_MARK; + if (dev_conf->dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) { + const struct rte_flow_action_mark *mark = + (const struct rte_flow_action_mark *) + actions->conf; + + if (flow_dv_convert_action_mark(dev, mark, + &mhdr_res, + error)) + return -rte_errno; + action_flags |= MLX5_FLOW_ACTION_MARK_EXT; + break; + } + /* Legacy (non-extensive) MARK action. */ tag_resource.tag = mlx5_flow_mark_set (((const struct rte_flow_action_mark *) (actions->conf))->id); @@ -5967,7 +6312,6 @@ struct field_modify_info modify_tcp[] = { return errno; dev_flow->dv.actions[actions_n++] = dev_flow->dv.tag_resource->action; - action_flags |= MLX5_FLOW_ACTION_MARK; break; case RTE_FLOW_ACTION_TYPE_SET_TAG: if (flow_dv_convert_action_set_tag @@ -6004,7 +6348,7 @@ struct field_modify_info modify_tcp[] = { action_flags |= MLX5_FLOW_ACTION_RSS; break; case RTE_FLOW_ACTION_TYPE_COUNT: - if (!priv->config.devx) { + if (!dev_conf->devx) { rte_errno = ENOTSUP; goto cnt_err; } @@ -6417,6 +6761,11 @@ struct field_modify_info modify_tcp[] = { items, last_item, tunnel); last_item = MLX5_FLOW_LAYER_MPLS; break; + case RTE_FLOW_ITEM_TYPE_MARK: + flow_dv_translate_item_mark(dev, match_mask, + match_value, items); + last_item = MLX5_FLOW_ITEM_MARK; + break; case RTE_FLOW_ITEM_TYPE_META: flow_dv_translate_item_meta(match_mask, match_value, items); From patchwork Tue Nov 5 08:01:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62436 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DAC88A0352; Tue, 5 Nov 2019 09:06:16 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D12AD1BEF1; Tue, 5 Nov 2019 09:02:34 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id AFC621BE9F for ; Tue, 5 Nov 2019 09:02:18 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:14 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582E2j026530; Tue, 5 Nov 2019 10:02:14 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582Ean030786; Tue, 5 Nov 2019 08:02:14 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582E3h030785; Tue, 5 Nov 2019 08:02:14 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:50 +0000 Message-Id: <1572940915-29416-16-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 15/20] net/mlx5: extend flow meta data support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" META item is supported on both Rx and Tx. 'transfer' attribute is also supported. SET_META action is also added. Due to restriction on reg_c[meta], various bit width might be available. If devarg parameter dv_xmeta_en=1, the META uses metadata register reg_c[0], which may be required for internal kernel or firmware needs. In this case PMD queries kernel about available fields in reg_c[0] and restricts the register usage accordingly. If devarg parameter dv_xmeta_en=2, the META feature uses reg_c[1], there should be no limitations on the data width. However, extensive MEAT feature is currently disabled until register copy on loopback is supported by forthcoming patches. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow.h | 4 +- drivers/net/mlx5/mlx5_flow_dv.c | 250 +++++++++++++++++++++++++++++++++++++--- 2 files changed, 235 insertions(+), 19 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index d6209ff..ef16aef 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -196,6 +196,7 @@ enum mlx5_feature_name { #define MLX5_FLOW_ACTION_DEC_TCP_ACK (1u << 31) #define MLX5_FLOW_ACTION_SET_TAG (1ull << 32) #define MLX5_FLOW_ACTION_MARK_EXT (1ull << 33) +#define MLX5_FLOW_ACTION_SET_META (1ull << 34) #define MLX5_FLOW_FATE_ACTIONS \ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \ @@ -231,7 +232,8 @@ enum mlx5_feature_name { MLX5_FLOW_ACTION_DEC_TCP_ACK | \ MLX5_FLOW_ACTION_OF_SET_VLAN_VID | \ MLX5_FLOW_ACTION_SET_TAG | \ - MLX5_FLOW_ACTION_MARK_EXT) + MLX5_FLOW_ACTION_MARK_EXT | \ + MLX5_FLOW_ACTION_SET_META) #define MLX5_FLOW_VLAN_ACTIONS (MLX5_FLOW_ACTION_OF_POP_VLAN | \ MLX5_FLOW_ACTION_OF_PUSH_VLAN) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 5714a6d..19f58cb 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -1043,6 +1043,103 @@ struct field_modify_info modify_tcp[] = { } /** + * Get metadata register index for specified steering domain. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] attr + * Attributes of flow to determine steering domain. + * @param[out] error + * Pointer to the error structure. + * + * @return + * positive index on success, a negative errno value otherwise + * and rte_errno is set. + */ +static enum modify_reg +flow_dv_get_metadata_reg(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + struct rte_flow_error *error) +{ + enum modify_reg reg = + mlx5_flow_get_reg_id(dev, attr->transfer ? + MLX5_METADATA_FDB : + attr->egress ? + MLX5_METADATA_TX : + MLX5_METADATA_RX, 0, error); + if (reg < 0) + return rte_flow_error_set(error, + ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, + NULL, "unavailable " + "metadata register"); + return reg; +} + +/** + * Convert SET_META action to DV specification. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in,out] resource + * Pointer to the modify-header resource. + * @param[in] attr + * Attributes of flow that includes this item. + * @param[in] conf + * Pointer to action specification. + * @param[out] error + * Pointer to the error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_convert_action_set_meta + (struct rte_eth_dev *dev, + struct mlx5_flow_dv_modify_hdr_resource *resource, + const struct rte_flow_attr *attr, + const struct rte_flow_action_set_meta *conf, + struct rte_flow_error *error) +{ + uint32_t data = conf->data; + uint32_t mask = conf->mask; + struct rte_flow_item item = { + .spec = &data, + .mask = &mask, + }; + struct field_modify_info reg_c_x[] = { + [1] = {0, 0, 0}, + }; + enum modify_reg reg = flow_dv_get_metadata_reg(dev, attr, error); + + if (reg < 0) + return reg; + /* + * In datapath code there is no endianness + * coversions for perfromance reasons, all + * pattern conversions are done in rte_flow. + */ + if (reg == REG_C_0) { + struct mlx5_priv *priv = dev->data->dev_private; + uint32_t msk_c0 = priv->sh->dv_regc0_mask; + uint32_t shl_c0 = rte_bsf32(msk_c0); + + data = rte_cpu_to_be_32(data); + mask = rte_cpu_to_be_32(mask); + msk_c0 = rte_cpu_to_be_32(msk_c0); + mask <<= shl_c0; + data <<= shl_c0; + assert(msk_c0); + assert(!(~msk_c0 & mask)); + data = rte_be_to_cpu_32(data); + mask = rte_be_to_cpu_32(mask); + } + reg_c_x[0] = (struct field_modify_info){4, 0, reg_to_field[reg]}; + /* The routine expects parameters in memory as big-endian ones. */ + return flow_dv_convert_modify_action(&item, reg_c_x, NULL, resource, + MLX5_MODIFICATION_TYPE_SET, error); +} + +/** * Validate MARK item. * * @param[in] dev @@ -1132,11 +1229,14 @@ struct field_modify_info modify_tcp[] = { const struct rte_flow_attr *attr, struct rte_flow_error *error) { + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *config = &priv->config; const struct rte_flow_item_meta *spec = item->spec; const struct rte_flow_item_meta *mask = item->mask; - const struct rte_flow_item_meta nic_mask = { + struct rte_flow_item_meta nic_mask = { .data = UINT32_MAX }; + enum modify_reg reg; int ret; if (!spec) @@ -1146,23 +1246,27 @@ struct field_modify_info modify_tcp[] = { "data cannot be empty"); if (!spec->data) return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ITEM_SPEC, - NULL, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, NULL, "data cannot be zero"); + if (config->dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) { + if (!mlx5_flow_ext_mreg_supported(dev)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM, item, + "extended metadata register" + " isn't supported"); + reg = flow_dv_get_metadata_reg(dev, attr, error); + if (reg < 0) + return reg; + if (reg != REG_A && reg != REG_B) + nic_mask.data = priv->sh->dv_meta_mask; + } if (!mask) mask = &rte_flow_item_meta_mask; ret = mlx5_flow_item_acceptable(item, (const uint8_t *)mask, (const uint8_t *)&nic_mask, sizeof(struct rte_flow_item_meta), error); - if (ret < 0) - return ret; - if (attr->ingress) - return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_INGRESS, - NULL, - "pattern not supported for ingress"); - return 0; + return ret; } /** @@ -1718,6 +1822,67 @@ struct field_modify_info modify_tcp[] = { } /** + * Validate SET_META action. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] action + * Pointer to the encap action. + * @param[in] action_flags + * Holds the actions detected until now. + * @param[in] attr + * Pointer to flow attributes + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_action_set_meta(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + uint64_t action_flags __rte_unused, + const struct rte_flow_attr *attr, + struct rte_flow_error *error) +{ + const struct rte_flow_action_set_meta *conf; + uint32_t nic_mask = UINT32_MAX; + enum modify_reg reg; + + if (!mlx5_flow_ext_mreg_supported(dev)) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "extended metadata register" + " isn't supported"); + reg = flow_dv_get_metadata_reg(dev, attr, error); + if (reg < 0) + return reg; + if (reg != REG_A && reg != REG_B) { + struct mlx5_priv *priv = dev->data->dev_private; + + nic_mask = priv->sh->dv_meta_mask; + } + if (!(action->conf)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "configuration cannot be null"); + conf = (const struct rte_flow_action_set_meta *)action->conf; + if (!conf->mask) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "zero mask doesn't have any effect"); + if (conf->mask & ~nic_mask) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "meta data must be within reg C0"); + if (!(conf->data & conf->mask)) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, action, + "zero value has no effect"); + return 0; +} + +/** * Validate SET_TAG action. * * @param[in] dev @@ -4245,6 +4410,17 @@ struct field_modify_info modify_tcp[] = { ++actions_n; } break; + case RTE_FLOW_ACTION_TYPE_SET_META: + ret = flow_dv_validate_action_set_meta(dev, actions, + action_flags, + attr, error); + if (ret < 0) + return ret; + /* Count all modify-header actions as one action. */ + if (!(action_flags & MLX5_FLOW_MODIFY_HDR_ACTIONS)) + ++actions_n; + action_flags |= MLX5_FLOW_ACTION_SET_META; + break; case RTE_FLOW_ACTION_TYPE_SET_TAG: ret = flow_dv_validate_action_set_tag(dev, actions, action_flags, @@ -5524,15 +5700,21 @@ struct field_modify_info modify_tcp[] = { /** * Add META item to matcher * + * @param[in] dev + * The devich to configure through. * @param[in, out] matcher * Flow matcher. * @param[in, out] key * Flow matcher value. + * @param[in] attr + * Attributes of flow that includes this item. * @param[in] item * Flow pattern to translate. */ static void -flow_dv_translate_item_meta(void *matcher, void *key, +flow_dv_translate_item_meta(struct rte_eth_dev *dev, + void *matcher, void *key, + const struct rte_flow_attr *attr, const struct rte_flow_item *item) { const struct rte_flow_item_meta *meta_m; @@ -5542,10 +5724,34 @@ struct field_modify_info modify_tcp[] = { if (!meta_m) meta_m = &rte_flow_item_meta_mask; meta_v = (const void *)item->spec; - if (meta_v) - flow_dv_match_meta_reg(matcher, key, REG_A, - rte_cpu_to_be_32(meta_v->data), - rte_cpu_to_be_32(meta_m->data)); + if (meta_v) { + enum modify_reg reg; + uint32_t value = meta_v->data; + uint32_t mask = meta_m->data; + + reg = flow_dv_get_metadata_reg(dev, attr, NULL); + if (reg < 0) + return; + /* + * In datapath code there is no endianness + * coversions for perfromance reasons, all + * pattern conversions are done in rte_flow. + */ + value = rte_cpu_to_be_32(value); + mask = rte_cpu_to_be_32(mask); + if (reg == REG_C_0) { + struct mlx5_priv *priv = dev->data->dev_private; + uint32_t msk_c0 = priv->sh->dv_regc0_mask; + uint32_t shl_c0 = rte_bsf32(msk_c0); + + msk_c0 = rte_cpu_to_be_32(msk_c0); + value <<= shl_c0; + mask <<= shl_c0; + assert(msk_c0); + assert(!(~msk_c0 & mask)); + } + flow_dv_match_meta_reg(matcher, key, reg, value, mask); + } } /** @@ -6313,6 +6519,14 @@ struct field_modify_info modify_tcp[] = { dev_flow->dv.actions[actions_n++] = dev_flow->dv.tag_resource->action; break; + case RTE_FLOW_ACTION_TYPE_SET_META: + if (flow_dv_convert_action_set_meta + (dev, &mhdr_res, attr, + (const struct rte_flow_action_set_meta *) + actions->conf, error)) + return -rte_errno; + action_flags |= MLX5_FLOW_ACTION_SET_META; + break; case RTE_FLOW_ACTION_TYPE_SET_TAG: if (flow_dv_convert_action_set_tag (dev, &mhdr_res, @@ -6767,8 +6981,8 @@ struct field_modify_info modify_tcp[] = { last_item = MLX5_FLOW_ITEM_MARK; break; case RTE_FLOW_ITEM_TYPE_META: - flow_dv_translate_item_meta(match_mask, match_value, - items); + flow_dv_translate_item_meta(dev, match_mask, + match_value, attr, items); last_item = MLX5_FLOW_ITEM_METADATA; break; case RTE_FLOW_ITEM_TYPE_ICMP: From patchwork Tue Nov 5 08:01:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62435 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6DA16A0352; Tue, 5 Nov 2019 09:05:48 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 61C551BEE1; Tue, 5 Nov 2019 09:02:33 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id B15181BEA0 for ; Tue, 5 Nov 2019 09:02:18 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:15 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582FkM026533; Tue, 5 Nov 2019 10:02:15 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582Fi1030788; Tue, 5 Nov 2019 08:02:15 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582FDq030787; Tue, 5 Nov 2019 08:02:15 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:51 +0000 Message-Id: <1572940915-29416-17-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 16/20] net/mlx5: add meta data support to Rx datapath X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch moves metadata from completion descriptor to appropriate dynamic mbuf field. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_prm.h | 6 ++++-- drivers/net/mlx5/mlx5_rxtx.c | 6 ++++++ drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 25 +++++++++++++++++++++++-- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 23 +++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 27 +++++++++++++++++++++++---- 5 files changed, 79 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index b405cb6..a0c37c8 100644 --- a/drivers/net/mlx5/mlx5_prm.h +++ b/drivers/net/mlx5/mlx5_prm.h @@ -357,12 +357,14 @@ struct mlx5_cqe { uint16_t hdr_type_etc; uint16_t vlan_info; uint8_t lro_num_seg; - uint8_t rsvd3[11]; + uint8_t rsvd3[3]; + uint32_t flow_table_metadata; + uint8_t rsvd4[4]; uint32_t byte_cnt; uint64_t timestamp; uint32_t sop_drop_qpn; uint16_t wqe_counter; - uint8_t rsvd4; + uint8_t rsvd5; uint8_t op_own; }; diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 887e283..b9a9bab 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -26,6 +26,7 @@ #include #include #include +#include #include "mlx5.h" #include "mlx5_utils.h" @@ -1251,6 +1252,11 @@ enum mlx5_txcmp_code { pkt->hash.fdir.hi = mlx5_flow_mark_get(mark); } } + if (rte_flow_dynf_metadata_avail() && cqe->flow_table_metadata) { + pkt->ol_flags |= PKT_RX_DYNF_METADATA; + *RTE_FLOW_DYNF_METADATA(pkt) = + rte_be_to_cpu_32(cqe->flow_table_metadata); + } if (rxq->csum) pkt->ol_flags |= rxq_cq_to_ol_flags(cqe); if (rxq->vlan_strip && diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h index 3be3a6d..8e79883 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h @@ -416,7 +416,6 @@ vec_cmpeq((vector unsigned int)flow_tag, (vector unsigned int)pinfo_ft_mask))); } - /* * Merge the two fields to generate the following: * bit[1] = l3_ok @@ -1011,7 +1010,29 @@ pkts[pos + 3]->timestamp = rte_be_to_cpu_64(cq[pos + p3].timestamp); } - + if (rte_flow_dynf_metadata_avail()) { + uint64_t flag = rte_flow_dynf_metadata_mask; + int offs = rte_flow_dynf_metadata_offs; + uint32_t metadata; + + /* This code is subject for futher optimization. */ + metadata = cq[pos].flow_table_metadata; + *RTE_MBUF_DYNFIELD(pkts[pos], offs, uint32_t *) = + metadata; + pkts[pos]->ol_flags |= metadata ? flag : 0ULL; + metadata = cq[pos + 1].flow_table_metadata; + *RTE_MBUF_DYNFIELD(pkts[pos + 1], offs, uint32_t *) = + metadata; + pkts[pos + 1]->ol_flags |= metadata ? flag : 0ULL; + metadata = cq[pos + 2].flow_table_metadata; + *RTE_MBUF_DYNFIELD(pkts[pos + 2], offs, uint32_t *) = + metadata; + pkts[pos + 2]->ol_flags |= metadata ? flag : 0ULL; + metadata = cq[pos + 3].flow_table_metadata; + *RTE_MBUF_DYNFIELD(pkts[pos + 3], offs, uint32_t *) = + metadata; + pkts[pos + 3]->ol_flags |= metadata ? flag : 0ULL; + } #ifdef MLX5_PMD_SOFT_COUNTERS /* Add up received bytes count. */ byte_cnt = vec_perm(op_own, zero, len_shuf_mask); diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index e914d01..86785c7 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -687,6 +687,29 @@ container_of(p3, struct mlx5_cqe, pkt_info)->timestamp); } + if (rte_flow_dynf_metadata_avail()) { + /* This code is subject for futher optimization. */ + *RTE_FLOW_DYNF_METADATA(elts[pos]) = + container_of(p0, struct mlx5_cqe, + pkt_info)->flow_table_metadata; + *RTE_FLOW_DYNF_METADATA(elts[pos + 1]) = + container_of(p1, struct mlx5_cqe, + pkt_info)->flow_table_metadata; + *RTE_FLOW_DYNF_METADATA(elts[pos + 2]) = + container_of(p2, struct mlx5_cqe, + pkt_info)->flow_table_metadata; + *RTE_FLOW_DYNF_METADATA(elts[pos + 3]) = + container_of(p3, struct mlx5_cqe, + pkt_info)->flow_table_metadata; + if (*RTE_FLOW_DYNF_METADATA(elts[pos])) + elts[pos]->ol_flags |= PKT_RX_DYNF_METADATA; + if (*RTE_FLOW_DYNF_METADATA(elts[pos + 1])) + elts[pos + 1]->ol_flags |= PKT_RX_DYNF_METADATA; + if (*RTE_FLOW_DYNF_METADATA(elts[pos + 2])) + elts[pos + 2]->ol_flags |= PKT_RX_DYNF_METADATA; + if (*RTE_FLOW_DYNF_METADATA(elts[pos + 3])) + elts[pos + 3]->ol_flags |= PKT_RX_DYNF_METADATA; + } #ifdef MLX5_PMD_SOFT_COUNTERS /* Add up received bytes count. */ byte_cnt = vbic_u16(byte_cnt, invalid_mask); diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index ca8ed41..35b7761 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -537,8 +537,8 @@ cqe_tmp1 = _mm_loadu_si128((__m128i *)&cq[pos + p2].csum); cqes[3] = _mm_blend_epi16(cqes[3], cqe_tmp2, 0x30); cqes[2] = _mm_blend_epi16(cqes[2], cqe_tmp1, 0x30); - cqe_tmp2 = _mm_loadl_epi64((__m128i *)&cq[pos + p3].rsvd3[9]); - cqe_tmp1 = _mm_loadl_epi64((__m128i *)&cq[pos + p2].rsvd3[9]); + cqe_tmp2 = _mm_loadl_epi64((__m128i *)&cq[pos + p3].rsvd4[2]); + cqe_tmp1 = _mm_loadl_epi64((__m128i *)&cq[pos + p2].rsvd4[2]); cqes[3] = _mm_blend_epi16(cqes[3], cqe_tmp2, 0x04); cqes[2] = _mm_blend_epi16(cqes[2], cqe_tmp1, 0x04); /* C.2 generate final structure for mbuf with swapping bytes. */ @@ -564,8 +564,8 @@ cqe_tmp1 = _mm_loadu_si128((__m128i *)&cq[pos].csum); cqes[1] = _mm_blend_epi16(cqes[1], cqe_tmp2, 0x30); cqes[0] = _mm_blend_epi16(cqes[0], cqe_tmp1, 0x30); - cqe_tmp2 = _mm_loadl_epi64((__m128i *)&cq[pos + p1].rsvd3[9]); - cqe_tmp1 = _mm_loadl_epi64((__m128i *)&cq[pos].rsvd3[9]); + cqe_tmp2 = _mm_loadl_epi64((__m128i *)&cq[pos + p1].rsvd4[2]); + cqe_tmp1 = _mm_loadl_epi64((__m128i *)&cq[pos].rsvd4[2]); cqes[1] = _mm_blend_epi16(cqes[1], cqe_tmp2, 0x04); cqes[0] = _mm_blend_epi16(cqes[0], cqe_tmp1, 0x04); /* C.2 generate final structure for mbuf with swapping bytes. */ @@ -640,6 +640,25 @@ pkts[pos + 3]->timestamp = rte_be_to_cpu_64(cq[pos + p3].timestamp); } + if (rte_flow_dynf_metadata_avail()) { + /* This code is subject for futher optimization. */ + *RTE_FLOW_DYNF_METADATA(pkts[pos]) = + cq[pos].flow_table_metadata; + *RTE_FLOW_DYNF_METADATA(pkts[pos + 1]) = + cq[pos + p1].flow_table_metadata; + *RTE_FLOW_DYNF_METADATA(pkts[pos + 2]) = + cq[pos + p2].flow_table_metadata; + *RTE_FLOW_DYNF_METADATA(pkts[pos + 3]) = + cq[pos + p3].flow_table_metadata; + if (*RTE_FLOW_DYNF_METADATA(pkts[pos])) + pkts[pos]->ol_flags |= PKT_RX_DYNF_METADATA; + if (*RTE_FLOW_DYNF_METADATA(pkts[pos + 1])) + pkts[pos + 1]->ol_flags |= PKT_RX_DYNF_METADATA; + if (*RTE_FLOW_DYNF_METADATA(pkts[pos + 2])) + pkts[pos + 2]->ol_flags |= PKT_RX_DYNF_METADATA; + if (*RTE_FLOW_DYNF_METADATA(pkts[pos + 3])) + pkts[pos + 3]->ol_flags |= PKT_RX_DYNF_METADATA; + } #ifdef MLX5_PMD_SOFT_COUNTERS /* Add up received bytes count. */ byte_cnt = _mm_shuffle_epi8(op_own, len_shuf_mask); From patchwork Tue Nov 5 08:01:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62439 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B85A8A0352; Tue, 5 Nov 2019 09:07:38 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 73ED61BEFD; Tue, 5 Nov 2019 09:02:39 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 054651BEA8 for ; Tue, 5 Nov 2019 09:02:18 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:16 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582Ggr026536; Tue, 5 Nov 2019 10:02:16 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582GnZ030791; Tue, 5 Nov 2019 08:02:16 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582G40030790; Tue, 5 Nov 2019 08:02:16 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:52 +0000 Message-Id: <1572940915-29416-18-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 17/20] net/mlx5: add simple hash table X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Simple hash table is added. Hash function is modulo operator and no conflict is allowed. Key must be unique. It would be useful in managing a resource pool having finite unique keys, e.g. flow table entry with unique flow ID. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_utils.h | 115 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 112 insertions(+), 3 deletions(-) diff --git a/drivers/net/mlx5/mlx5_utils.h b/drivers/net/mlx5/mlx5_utils.h index 97092c7..5a7de62 100644 --- a/drivers/net/mlx5/mlx5_utils.h +++ b/drivers/net/mlx5/mlx5_utils.h @@ -6,12 +6,13 @@ #ifndef RTE_PMD_MLX5_UTILS_H_ #define RTE_PMD_MLX5_UTILS_H_ +#include +#include +#include #include #include #include -#include -#include -#include +#include #include "mlx5_defs.h" @@ -150,6 +151,114 @@ \ snprintf(name, sizeof(name), __VA_ARGS__) +/* + * Simple Hash Table for Key-Data pair. + * + * Key must be unique. No conflict is allowed. + * + * mlx5_shtable_entry could be a part of the data structure to store, e.g., + * + * struct DATA { + * struct mlx5_shtable_entry entry; + * custom_data_t custom_data; + * }; + * + * And the actual hash table should be defined as, + * + * struct mlx5_shtable_bucket TABLE[TABLE_SZ]; + * + * Hash function is simply modulo (%) operator for now. + */ + +/* Data entry for hash table. */ +struct mlx5_shtable_entry { + LIST_ENTRY(mlx5_shtable_entry) next; + uint32_t key; +}; + +/* Structure for hash bucket. */ +LIST_HEAD(mlx5_shtable_bucket, mlx5_shtable_entry); + +/** + * Search an entry matching the key. + * + * @param htable + * Pointer to the table. + * @param tbl_sz + * Size of the table. + * @param key + * Key for the searching entry. + * + * @return + * Pointer of the table entry if found, NULL otherwise. + */ +static inline struct mlx5_shtable_entry * +mlx5_shtable_search(struct mlx5_shtable_bucket *htable, int tbl_sz, + uint32_t key) +{ + struct mlx5_shtable_bucket *bucket; + struct mlx5_shtable_entry *node; + uint32_t idx; + + idx = key % tbl_sz; + bucket = &htable[idx]; + LIST_FOREACH(node, bucket, next) { + if (node->key == key) + return node; + } + return NULL; +} + +/** + * Insert an entry. + * + * @param htable + * Pointer to the table. + * @param tbl_sz + * Size of the table. + * @param key + * Key for the searching entry. + * + * @return + * 0 if succeed, -EEXIST if conflict. + */ +static inline int +mlx5_shtable_insert(struct mlx5_shtable_bucket *htable, int tbl_sz, + struct mlx5_shtable_entry *ent) +{ + struct mlx5_shtable_bucket *bucket; + struct mlx5_shtable_entry *node; + uint32_t idx; + + idx = ent->key % tbl_sz; + bucket = &htable[idx]; + LIST_FOREACH(node, bucket, next) { + if (node->key == ent->key) + return -EEXIST; + } + LIST_INSERT_HEAD(bucket, ent, next); + return 0; +} + +/** + * Revmoe an entry from its table. + * + * @param htable + * Pointer to the table. + * @param tbl_sz + * Size of the table. + * @param key + * Key for the searching entry. + */ +static inline void +mlx5_shtable_remove(struct mlx5_shtable_entry *ent) +{ + /* Check if entry is not attached. */ + if (!ent->next.le_prev) + return; + LIST_REMOVE(ent, next); +} + /** * Return nearest power of two above input value. * From patchwork Tue Nov 5 08:01:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62437 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6BD0FA0352; Tue, 5 Nov 2019 09:06:43 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3B3DE1BEF8; Tue, 5 Nov 2019 09:02:36 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id C67121BEA2 for ; Tue, 5 Nov 2019 09:02:18 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:17 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582HBi026539; Tue, 5 Nov 2019 10:02:17 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582Gj4030793; Tue, 5 Nov 2019 08:02:16 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582GK3030792; Tue, 5 Nov 2019 08:02:16 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com Date: Tue, 5 Nov 2019 08:01:53 +0000 Message-Id: <1572940915-29416-19-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 18/20] net/mlx5: introduce flow splitters chain X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The mlx5 hardware has some limitations and flow might require to be split into multiple internal subflows. For example this is needed to provide the meter object sharing between multiple flows or to provide metadata register copying before final queue/rss action. The multiple features might require several level of splitting. For example, hairpin feature splits the original flow into two ones - rx and tx parts. Then RSS feature should split rx part into multiple subflows with extended item sets. Then, metering feature might require splitting each RSS subflow into meter jump chain, and then metadata extensive support might require the final subflows splitting. So, we have to organize the chain of splitting subroutines to abstract each level of splitting. Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow.c | 116 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 106 insertions(+), 10 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index c38208c..a310a88 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -2782,6 +2782,103 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, } /** + * The last stage of splitting chain, just creates the subflow + * without any modification. + * + * @param dev + * Pointer to Ethernet device. + * @param[in] flow + * Parent flow structure pointer. + * @param[in, out] sub_flow + * Pointer to return the created subflow, may be NULL. + * @param[in] attr + * Flow rule attributes. + * @param[in] items + * Pattern specification (list terminated by the END pattern item). + * @param[in] actions + * Associated actions (list terminated by the END action). + * @param[in] external + * This flow rule is created by request external to PMD. + * @param[out] error + * Perform verbose error reporting if not NULL. + * @return + * 0 on success, negative value otherwise + */ +static int +flow_create_split_inner(struct rte_eth_dev *dev, + struct rte_flow *flow, + struct mlx5_flow **sub_flow, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], + bool external, struct rte_flow_error *error) +{ + struct mlx5_flow *dev_flow; + + dev_flow = flow_drv_prepare(flow, attr, items, actions, error); + if (!dev_flow) + return -rte_errno; + dev_flow->flow = flow; + dev_flow->external = external; + /* Subflow object was created, we must include one in the list. */ + LIST_INSERT_HEAD(&flow->dev_flows, dev_flow, next); + if (sub_flow) + *sub_flow = dev_flow; + return flow_drv_translate(dev, dev_flow, attr, items, actions, error); +} + +/** + * Split the flow to subflow set. The splitters might be linked + * in the chain, like this: + * flow_create_split_outer() calls: + * flow_create_split_meter() calls: + * flow_create_split_metadata(meter_subflow_0) calls: + * flow_create_split_inner(metadata_subflow_0) + * flow_create_split_inner(metadata_subflow_1) + * flow_create_split_inner(metadata_subflow_2) + * flow_create_split_metadata(meter_subflow_1) calls: + * flow_create_split_inner(metadata_subflow_0) + * flow_create_split_inner(metadata_subflow_1) + * flow_create_split_inner(metadata_subflow_2) + * + * This provide flexible way to add new levels of flow splitting. + * The all of successfully created subflows are included to the + * parent flow dev_flow list. + * + * @param dev + * Pointer to Ethernet device. + * @param[in] flow + * Parent flow structure pointer. + * @param[in] attr + * Flow rule attributes. + * @param[in] items + * Pattern specification (list terminated by the END pattern item). + * @param[in] actions + * Associated actions (list terminated by the END action). + * @param[in] external + * This flow rule is created by request external to PMD. + * @param[out] error + * Perform verbose error reporting if not NULL. + * @return + * 0 on success, negative value otherwise + */ +static int +flow_create_split_outer(struct rte_eth_dev *dev, + struct rte_flow *flow, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], + bool external, struct rte_flow_error *error) +{ + int ret; + + ret = flow_create_split_inner(dev, flow, NULL, attr, items, + actions, external, error); + assert(ret <= 0); + return ret; +} + +/** * Create a flow and add it to @p list. * * @param dev @@ -2899,16 +2996,15 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, buf->entry[0].pattern = (void *)(uintptr_t)items; } for (i = 0; i < buf->entries; ++i) { - dev_flow = flow_drv_prepare(flow, attr, buf->entry[i].pattern, - p_actions_rx, error); - if (!dev_flow) - goto error; - dev_flow->flow = flow; - dev_flow->external = external; - LIST_INSERT_HEAD(&flow->dev_flows, dev_flow, next); - ret = flow_drv_translate(dev, dev_flow, attr, - buf->entry[i].pattern, - p_actions_rx, error); + /* + * The splitter may create multiple dev_flows, + * depending on configuration. In the simplest + * case it just creates unmodified original flow. + */ + ret = flow_create_split_outer(dev, flow, attr, + buf->entry[i].pattern, + p_actions_rx, external, + error); if (ret < 0) goto error; } From patchwork Tue Nov 5 08:01:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62441 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DCCD2A0352; Tue, 5 Nov 2019 09:08:43 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D2B9C1BF16; Tue, 5 Nov 2019 09:02:42 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id BEEE71BEBF for ; Tue, 5 Nov 2019 09:02:23 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:18 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582I96026543; Tue, 5 Nov 2019 10:02:18 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582I8Y030795; Tue, 5 Nov 2019 08:02:18 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582Ims030794; Tue, 5 Nov 2019 08:02:18 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:54 +0000 Message-Id: <1572940915-29416-20-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 19/20] net/mlx5: split Rx flows to provide metadata copy X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Values set by MARK and SET_META actions should be carried over to the VF representor in case of flow miss on Tx path. However, as not all metadata registers are preserved across the different domains (NIC Rx/Tx and E-Switch FDB), as a workaround, those values should be carried by reg_c's which are preserved across domains and copied to STE flow_tag (MARK) and reg_b (META) fields in the last stage of flow steering, in order to scatter those values to flow_tag and flow_table_metadata of CQE. While reg_c[meta] can be copied to reg_b simply by modify-header action (it is supported by hardware), it is not possible to copy reg_c[mark] to the STE flow_tag as flow_tag is not a metadata register and this is not supported by hardware. Instead, it should be manually set by a flow per MARK ID. For this purpose, there should be a dedicated flow table - RX_CP_TBL and all the Rx flow should pass by the table to properly copy values. As the last action of Rx flow steering must be a terminal action such as QUEUE, RSS or DROP, if a user flow has Q/RSS action, the flow must be split in order to pass by the RX_CP_TBL. And the remained Q/RSS action will be performed by another dedicated action table - RX_ACT_TBL. For example, for an ingress flow: pattern, actions_having_QRSS it must be split into two flows. The first one is, pattern, actions_except_QRSS / copy (reg_c[2] := flow_id) / jump to RX_CP_TBL and the second one in RX_ACT_TBL. (if reg_c[2] == flow_id), action_QRSS where flow_id is uniquely allocated and managed identifier. This patch implements the Rx flow splitting and build the RX_ACT_TBL. Also, per each egress flow on NIC Tx, a copy action (reg_c[]= reg_a) should be added in order to transfer metadata from WQE. Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5.c | 8 + drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_flow.c | 428 ++++++++++++++++++++++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 1 + 4 files changed, 436 insertions(+), 2 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index fb7b94b..6359bc9 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -2411,6 +2411,12 @@ struct mlx5_flow_id_pool * err = mlx5_alloc_shared_dr(priv); if (err) goto error; + priv->qrss_id_pool = mlx5_flow_id_pool_alloc(); + if (!priv->qrss_id_pool) { + DRV_LOG(ERR, "can't create flow id pool"); + err = ENOMEM; + goto error; + } } /* Supported Verbs flow priority number detection. */ err = mlx5_flow_discover_priorities(eth_dev); @@ -2463,6 +2469,8 @@ struct mlx5_flow_id_pool * close(priv->nl_socket_rdma); if (priv->vmwa_context) mlx5_vlan_vmwa_exit(priv->vmwa_context); + if (priv->qrss_id_pool) + mlx5_flow_id_pool_release(priv->qrss_id_pool); if (own_domain_id) claim_zero(rte_eth_switch_domain_free(priv->domain_id)); rte_free(priv); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 92d445a..9c1a88a 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -733,6 +733,7 @@ struct mlx5_priv { uint32_t nl_sn; /* Netlink message sequence number. */ LIST_HEAD(dbrpage, mlx5_devx_dbr_page) dbrpgs; /* Door-bell pages. */ struct mlx5_vlan_vmwa_context *vmwa_context; /* VLAN WA context. */ + struct mlx5_flow_id_pool *qrss_id_pool; #ifndef RTE_ARCH_64 rte_spinlock_t uar_lock_cq; /* CQs share a common distinct UAR */ rte_spinlock_t uar_lock[MLX5_UAR_PAGE_NUM_MAX]; diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index a310a88..0ed8308 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -2218,6 +2218,49 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, return 0; } +/* Allocate unique ID for the split Q/RSS subflows. */ +static uint32_t +flow_qrss_get_id(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + uint32_t qrss_id, ret; + + ret = mlx5_flow_id_get(priv->qrss_id_pool, &qrss_id); + if (ret) + return 0; + assert(qrss_id); + return qrss_id; +} + +/* Free unique ID for the split Q/RSS subflows. */ +static void +flow_qrss_free_id(struct rte_eth_dev *dev, uint32_t qrss_id) +{ + struct mlx5_priv *priv = dev->data->dev_private; + + if (qrss_id) + mlx5_flow_id_release(priv->qrss_id_pool, qrss_id); +} + +/** + * Release resource related QUEUE/RSS action split. + * + * @param dev + * Pointer to Ethernet device. + * @param flow + * Flow to release id's from. + */ +static void +flow_mreg_split_qrss_release(struct rte_eth_dev *dev, + struct rte_flow *flow) +{ + struct mlx5_flow *dev_flow; + + LIST_FOREACH(dev_flow, &flow->dev_flows, next) + if (dev_flow->qrss_id) + flow_qrss_free_id(dev, dev_flow->qrss_id); +} + static int flow_null_validate(struct rte_eth_dev *dev __rte_unused, const struct rte_flow_attr *attr __rte_unused, @@ -2507,6 +2550,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, const struct mlx5_flow_driver_ops *fops; enum mlx5_flow_drv_type type = flow->drv_type; + flow_mreg_split_qrss_release(dev, flow); assert(type > MLX5_FLOW_TYPE_MIN && type < MLX5_FLOW_TYPE_MAX); fops = flow_get_drv_ops(type); fops->destroy(dev, flow); @@ -2577,6 +2621,41 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, } /** + * Get QUEUE/RSS action from the action list. + * + * @param[in] actions + * Pointer to the list of actions. + * @param[out] qrss + * Pointer to the return pointer. + * @param[out] qrss_type + * Pointer to the action type to return. RTE_FLOW_ACTION_TYPE_END is returned + * if no QUEUE/RSS is found. + * + * @return + * Total number of actions. + */ +static int +flow_parse_qrss_action(const struct rte_flow_action actions[], + const struct rte_flow_action **qrss) +{ + int actions_n = 0; + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_QUEUE: + case RTE_FLOW_ACTION_TYPE_RSS: + *qrss = actions; + break; + default: + break; + } + actions_n++; + } + /* Count RTE_FLOW_ACTION_TYPE_END. */ + return actions_n + 1; +} + +/** * Check if the flow should be splited due to hairpin. * The reason for the split is that in current HW we can't * support encap on Rx, so if a flow have encap we move it @@ -2828,6 +2907,351 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, } /** + * Split action list having QUEUE/RSS for metadata register copy. + * + * Once Q/RSS action is detected in user's action list, the flow action + * should be split in order to copy metadata registers, which will happen in + * RX_CP_TBL like, + * - CQE->flow_tag := reg_c[1] (MARK) + * - CQE->flow_table_metadata (reg_b) := reg_c[0] (META) + * The Q/RSS action will be performed on RX_ACT_TBL after passing by RX_CP_TBL. + * This is because the last action of each flow must be a terminal action + * (QUEUE, RSS or DROP). + * + * Flow ID must be allocated to identify actions in the RX_ACT_TBL and it is + * stored and kept in the mlx5_flow structure per each sub_flow. + * + * The Q/RSS action is replaced with, + * - SET_TAG, setting the allocated flow ID to reg_c[2]. + * And the following JUMP action is added at the end, + * - JUMP, to RX_CP_TBL. + * + * A flow to perform remained Q/RSS action will be created in RX_ACT_TBL by + * flow_create_split_metadata() routine. The flow will look like, + * - If flow ID matches (reg_c[2]), perform Q/RSS. + * + * @param dev + * Pointer to Ethernet device. + * @param[out] split_actions + * Pointer to store split actions to jump to CP_TBL. + * @param[in] actions + * Pointer to the list of original flow actions. + * @param[in] qrss + * Pointer to the Q/RSS action. + * @param[in] actions_n + * Number of original actions. + * @param[out] error + * Perform verbose error reporting if not NULL. + * + * @return + * non-zero unique flow_id on success, otherwise 0 and + * error/rte_error are set. + */ +static uint32_t +flow_mreg_split_qrss_prep(struct rte_eth_dev *dev, + struct rte_flow_action *split_actions, + const struct rte_flow_action *actions, + const struct rte_flow_action *qrss, + int actions_n, struct rte_flow_error *error) +{ + struct mlx5_rte_flow_action_set_tag *set_tag; + struct rte_flow_action_jump *jump; + const int qrss_idx = qrss - actions; + uint32_t flow_id; + int ret = 0; + + /* + * Given actions will be split + * - Replace QUEUE/RSS action with SET_TAG to set flow ID. + * - Add jump to mreg CP_TBL. + * As a result, there will be one more action. + */ + ++actions_n; + /* + * Allocate the new subflow ID. This one is unique within + * device and not shared with representors. Otherwise, + * we would have to resolve multi-thread access synch + * issue. Each flow on the shared device is appended + * with source vport identifier, so the resulting + * flows will be unique in the shared (by master and + * representors) domain even if they have coinciding + * IDs. + */ + flow_id = flow_qrss_get_id(dev); + if (!flow_id) + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, "can't allocate id " + "for split Q/RSS subflow"); + /* Internal SET_TAG action to set flow ID. */ + set_tag = (void *)(split_actions + actions_n); + *set_tag = (struct mlx5_rte_flow_action_set_tag){ + .data = flow_id, + }; + ret = mlx5_flow_get_reg_id(dev, MLX5_COPY_MARK, 0, error); + if (ret < 0) + return ret; + set_tag->id = ret; + /* JUMP action to jump to mreg copy table (CP_TBL). */ + jump = (void *)(set_tag + 1); + *jump = (struct rte_flow_action_jump){ + .group = MLX5_FLOW_MREG_CP_TABLE_GROUP, + }; + /* Construct new actions array. */ + memcpy(split_actions, actions, sizeof(*split_actions) * actions_n); + /* Replace QUEUE/RSS action. */ + split_actions[qrss_idx] = (struct rte_flow_action){ + .type = MLX5_RTE_FLOW_ACTION_TYPE_TAG, + .conf = set_tag, + }; + split_actions[actions_n - 2] = (struct rte_flow_action){ + .type = RTE_FLOW_ACTION_TYPE_JUMP, + .conf = jump, + }; + split_actions[actions_n - 1] = (struct rte_flow_action){ + .type = RTE_FLOW_ACTION_TYPE_END, + }; + return flow_id; +} + +/** + * Extend the given action list for Tx metadata copy. + * + * Copy the given action list to the ext_actions and add flow metadata register + * copy action in order to copy reg_a set by WQE to reg_c[0]. + * + * @param[out] ext_actions + * Pointer to the extended action list. + * @param[in] actions + * Pointer to the list of actions. + * @param[in] actions_n + * Number of actions in the list. + * @param[out] error + * Perform verbose error reporting if not NULL. + * + * @return + * 0 on success, negative value otherwise + */ +static int +flow_mreg_tx_copy_prep(struct rte_eth_dev *dev, + struct rte_flow_action *ext_actions, + const struct rte_flow_action *actions, + int actions_n, struct rte_flow_error *error) +{ + struct mlx5_flow_action_copy_mreg *cp_mreg = + (struct mlx5_flow_action_copy_mreg *) + (ext_actions + actions_n + 1); + int ret; + + ret = mlx5_flow_get_reg_id(dev, MLX5_METADATA_RX, 0, error); + if (ret < 0) + return ret; + cp_mreg->dst = ret; + ret = mlx5_flow_get_reg_id(dev, MLX5_METADATA_TX, 0, error); + if (ret < 0) + return ret; + cp_mreg->src = ret; + memcpy(ext_actions, actions, + sizeof(*ext_actions) * actions_n); + ext_actions[actions_n - 1] = (struct rte_flow_action){ + .type = MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG, + .conf = cp_mreg, + }; + ext_actions[actions_n] = (struct rte_flow_action){ + .type = RTE_FLOW_ACTION_TYPE_END, + }; + return 0; +} + +/** + * The splitting for metadata feature. + * + * - Q/RSS action on NIC Rx should be split in order to pass by + * the mreg copy table (RX_CP_TBL) and then it jumps to the + * action table (RX_ACT_TBL) which has the split Q/RSS action. + * + * - All the actions on NIC Tx should have a mreg copy action to + * copy reg_a from WQE to reg_c[0]. + * + * @param dev + * Pointer to Ethernet device. + * @param[in] flow + * Parent flow structure pointer. + * @param[in] attr + * Flow rule attributes. + * @param[in] items + * Pattern specification (list terminated by the END pattern item). + * @param[in] actions + * Associated actions (list terminated by the END action). + * @param[in] external + * This flow rule is created by request external to PMD. + * @param[out] error + * Perform verbose error reporting if not NULL. + * @return + * 0 on success, negative value otherwise + */ +static int +flow_create_split_metadata(struct rte_eth_dev *dev, + struct rte_flow *flow, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], + bool external, struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *config = &priv->config; + const struct rte_flow_action *qrss = NULL; + struct rte_flow_action *ext_actions = NULL; + struct mlx5_flow *dev_flow = NULL; + uint32_t qrss_id = 0; + size_t act_size; + int actions_n; + int ret; + + /* Check whether extensive metadata feature is engaged. */ + if (!config->dv_flow_en || + config->dv_xmeta_en == MLX5_XMETA_MODE_LEGACY || + !mlx5_flow_ext_mreg_supported(dev)) + return flow_create_split_inner(dev, flow, NULL, attr, items, + actions, external, error); + actions_n = flow_parse_qrss_action(actions, &qrss); + if (qrss) { + /* Exclude hairpin flows from splitting. */ + if (qrss->type == RTE_FLOW_ACTION_TYPE_QUEUE) { + const struct rte_flow_action_queue *queue; + + queue = qrss->conf; + if (mlx5_rxq_get_type(dev, queue->index) == + MLX5_RXQ_TYPE_HAIRPIN) + qrss = NULL; + } else if (qrss->type == RTE_FLOW_ACTION_TYPE_RSS) { + const struct rte_flow_action_rss *rss; + + rss = qrss->conf; + if (mlx5_rxq_get_type(dev, rss->queue[0]) == + MLX5_RXQ_TYPE_HAIRPIN) + qrss = NULL; + } + } + if (qrss) { + /* + * Q/RSS action on NIC Rx should be split in order to pass by + * the mreg copy table (RX_CP_TBL) and then it jumps to the + * action table (RX_ACT_TBL) which has the split Q/RSS action. + */ + act_size = sizeof(struct rte_flow_action) * (actions_n + 1) + + sizeof(struct rte_flow_action_set_tag) + + sizeof(struct rte_flow_action_jump); + ext_actions = rte_zmalloc(__func__, act_size, 0); + if (!ext_actions) + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, "no memory to split " + "metadata flow"); + /* + * Create the new actions list with removed Q/RSS action + * and appended set tag and jump to register copy table + * (RX_CP_TBL). We should preallocate unique tag ID here + * in advance, because it is needed for set tag action. + */ + qrss_id = flow_mreg_split_qrss_prep(dev, ext_actions, actions, + qrss, actions_n, error); + if (!qrss_id) { + ret = -rte_errno; + goto exit; + } + } else if (attr->egress && !attr->transfer) { + /* + * All the actions on NIC Tx should have a metadata register + * copy action to copy reg_a from WQE to reg_c[meta] + */ + act_size = sizeof(struct rte_flow_action) * (actions_n + 1) + + sizeof(struct mlx5_flow_action_copy_mreg); + ext_actions = rte_zmalloc(__func__, act_size, 0); + if (!ext_actions) + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, "no memory to split " + "metadata flow"); + /* Create the action list appended with copy register. */ + ret = flow_mreg_tx_copy_prep(dev, ext_actions, actions, + actions_n, error); + if (ret < 0) + goto exit; + } + /* Add the unmodified original or prefix subflow. */ + ret = flow_create_split_inner(dev, flow, &dev_flow, attr, items, + ext_actions ? ext_actions : actions, + external, error); + if (ret < 0) + goto exit; + assert(dev_flow); + if (qrss_id) { + const struct rte_flow_attr q_attr = { + .group = MLX5_FLOW_MREG_ACT_TABLE_GROUP, + .ingress = 1, + }; + /* Internal PMD action to set register. */ + struct mlx5_rte_flow_item_tag q_tag_spec = { + .data = qrss_id, + .id = 0, + }; + struct rte_flow_item q_items[] = { + { + .type = MLX5_RTE_FLOW_ITEM_TYPE_TAG, + .spec = &q_tag_spec, + .last = NULL, + .mask = NULL, + }, + { + .type = RTE_FLOW_ITEM_TYPE_END, + }, + }; + struct rte_flow_action q_actions[] = { + { + .type = qrss->type, + .conf = qrss->conf, + }, + { + .type = RTE_FLOW_ACTION_TYPE_END, + }, + }; + uint64_t hash_fields = dev_flow->hash_fields; + /* + * Put unique id in prefix flow due to it is destroyed after + * prefix flow and id will be freed after there is no actual + * flows with this id and identifier reallocation becomes + * possible (for example, for other flows in other threads). + */ + dev_flow->qrss_id = qrss_id; + qrss_id = 0; + dev_flow = NULL; + ret = mlx5_flow_get_reg_id(dev, MLX5_COPY_MARK, 0, error); + if (ret < 0) + goto exit; + q_tag_spec.id = ret; + /* Add suffix subflow to execute Q/RSS. */ + ret = flow_create_split_inner(dev, flow, &dev_flow, + &q_attr, q_items, q_actions, + external, error); + if (ret < 0) + goto exit; + assert(dev_flow); + dev_flow->hash_fields = hash_fields; + } + +exit: + /* + * We do not destroy the partially created sub_flows in case of error. + * These ones are included into parent flow list and will be destroyed + * by flow_drv_destroy. + */ + flow_qrss_free_id(dev, qrss_id); + rte_free(ext_actions); + return ret; +} + +/** * Split the flow to subflow set. The splitters might be linked * in the chain, like this: * flow_create_split_outer() calls: @@ -2872,8 +3296,8 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, { int ret; - ret = flow_create_split_inner(dev, flow, NULL, attr, items, - actions, external, error); + ret = flow_create_split_metadata(dev, flow, attr, items, + actions, external, error); assert(ret <= 0); return ret; } diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index ef16aef..c71938b 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -500,6 +500,7 @@ struct mlx5_flow { #endif struct mlx5_flow_verbs verbs; }; + uint32_t qrss_id; /**< Uniqie Q/RSS suffix subflow tag. */ bool external; /**< true if the flow is created external to PMD. */ }; From patchwork Tue Nov 5 08:01:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62440 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 53070A0352; Tue, 5 Nov 2019 09:08:12 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 626F51BF0D; Tue, 5 Nov 2019 09:02:41 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id BDE1F1BEBB for ; Tue, 5 Nov 2019 09:02:23 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:19 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582JPT026547; Tue, 5 Nov 2019 10:02:19 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582JMQ030797; Tue, 5 Nov 2019 08:02:19 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582J0w030796; Tue, 5 Nov 2019 08:02:19 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com, Yongseok Koh Date: Tue, 5 Nov 2019 08:01:55 +0000 Message-Id: <1572940915-29416-21-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 20/20] net/mlx5: add metadata register copy table X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" While reg_c[meta] can be copied to reg_b simply by modify-header action (it is supported by hardware), it is not possible to copy reg_c[mark] to the STE flow_tag as flow_tag is not a metadata register and this is not supported by hardware. Instead, it should be manually set by a flow per each unique MARK ID. For this purpose, there should be a dedicated flow table - RX_CP_TBL and all the Rx flow should pass by the table to properly copy values from the register to flow tag field. And for each MARK action, a copy flow should be added to RX_CP_TBL according to the MARK ID like: (if reg_c[mark] == mark_id), flow_tag := mark_id / reg_b := reg_c[meta] / jump to RX_ACT_TBL For SET_META action, there can be only one default flow like: reg_b := reg_c[meta] / jump to RX_ACT_TBL Signed-off-by: Yongseok Koh Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5.h | 7 +- drivers/net/mlx5/mlx5_defs.h | 3 + drivers/net/mlx5/mlx5_flow.c | 432 +++++++++++++++++++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 19 ++ drivers/net/mlx5/mlx5_flow_dv.c | 10 +- 5 files changed, 464 insertions(+), 7 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 9c1a88a..470778b 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -567,8 +567,9 @@ struct mlx5_flow_tbl_resource { #define MLX5_HAIRPIN_TX_TABLE (UINT16_MAX - 1) /* Reserve the last two tables for metadata register copy. */ #define MLX5_FLOW_MREG_ACT_TABLE_GROUP (MLX5_MAX_TABLES - 1) -#define MLX5_FLOW_MREG_CP_TABLE_GROUP \ - (MLX5_FLOW_MREG_ACT_TABLE_GROUP - 1) +#define MLX5_FLOW_MREG_CP_TABLE_GROUP (MLX5_MAX_TABLES - 2) +/* Tables for metering splits should be added here. */ +#define MLX5_MAX_TABLES_EXTERNAL (MLX5_MAX_TABLES - 3) #define MLX5_MAX_TABLES_FDB UINT16_MAX #define MLX5_DBR_PAGE_SIZE 4096 /* Must be >= 512. */ @@ -734,6 +735,8 @@ struct mlx5_priv { LIST_HEAD(dbrpage, mlx5_devx_dbr_page) dbrpgs; /* Door-bell pages. */ struct mlx5_vlan_vmwa_context *vmwa_context; /* VLAN WA context. */ struct mlx5_flow_id_pool *qrss_id_pool; + struct mlx5_shtable_bucket mreg_cp_tbl[MLX5_FLOW_MREG_HTABLE_SZ]; + /* Hash table of Rx metadata register copy table. */ #ifndef RTE_ARCH_64 rte_spinlock_t uar_lock_cq; /* CQs share a common distinct UAR */ rte_spinlock_t uar_lock[MLX5_UAR_PAGE_NUM_MAX]; diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index a77c430..fa2009c 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -145,6 +145,9 @@ #define MLX5_XMETA_MODE_META16 1 #define MLX5_XMETA_MODE_META32 2 +/* Size of the simple hash table for metadata register table. */ +#define MLX5_FLOW_MREG_HTABLE_SZ 1024 + /* Definition of static_assert found in /usr/include/assert.h */ #ifndef HAVE_STATIC_ASSERT #define static_assert _Static_assert diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 0ed8308..2f6cd92 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -667,7 +667,17 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, container_of((*priv->rxqs)[idx], struct mlx5_rxq_ctrl, rxq); - if (mark) { + /* + * To support metadata register copy on Tx loopback, + * this must be always enabled (metadata may arive + * from other port - not from local flows only. + */ + if (priv->config.dv_flow_en && + priv->config.dv_xmeta_en && + mlx5_flow_ext_mreg_supported(dev)) { + rxq_ctrl->rxq.mark = 1; + rxq_ctrl->flow_mark_n = 1; + } else if (mark) { rxq_ctrl->rxq.mark = 1; rxq_ctrl->flow_mark_n++; } @@ -731,7 +741,12 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, container_of((*priv->rxqs)[idx], struct mlx5_rxq_ctrl, rxq); - if (mark) { + if (priv->config.dv_flow_en && + priv->config.dv_xmeta_en && + mlx5_flow_ext_mreg_supported(dev)) { + rxq_ctrl->rxq.mark = 1; + rxq_ctrl->flow_mark_n = 1; + } else if (mark) { rxq_ctrl->flow_mark_n--; rxq_ctrl->rxq.mark = !!rxq_ctrl->flow_mark_n; } @@ -2727,6 +2742,392 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, return 0; } +/* Declare flow create/destroy prototype in advance. */ +static struct rte_flow * +flow_list_create(struct rte_eth_dev *dev, struct mlx5_flows *list, + const struct rte_flow_attr *attr, + const struct rte_flow_item items[], + const struct rte_flow_action actions[], + bool external, struct rte_flow_error *error); + +static void +flow_list_destroy(struct rte_eth_dev *dev, struct mlx5_flows *list, + struct rte_flow *flow); + +/** + * Add a flow of copying flow metadata registers in RX_CP_TBL. + * + * As mark_id is unique, if there's already a registered flow for the mark_id, + * return by increasing the reference counter of the resource. Otherwise, create + * the resource (mcp_res) and flow. + * + * Flow looks like, + * - If ingress port is ANY and reg_c[1] is mark_id, + * flow_tag := mark_id, reg_b := reg_c[0] and jump to RX_ACT_TBL. + * + * For default flow (zero mark_id), flow is like, + * - If ingress port is ANY, + * reg_b := reg_c[0] and jump to RX_ACT_TBL. + * + * @param dev + * Pointer to Ethernet device. + * @param mark_id + * ID of MARK action, zero means default flow for META. + * @param[out] error + * Perform verbose error reporting if not NULL. + * + * @return + * Associated resource on success, NULL otherwise and rte_errno is set. + */ +static struct mlx5_flow_mreg_copy_resource * +flow_mreg_add_copy_action(struct rte_eth_dev *dev, uint32_t mark_id, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct rte_flow_attr attr = { + .group = MLX5_FLOW_MREG_CP_TABLE_GROUP, + .ingress = 1, + }; + struct mlx5_rte_flow_item_tag tag_spec = { + .data = mark_id, + }; + struct rte_flow_item items[] = { + [1] = { .type = RTE_FLOW_ITEM_TYPE_END, }, + }; + struct rte_flow_action_mark ftag = { + .id = mark_id, + }; + struct mlx5_flow_action_copy_mreg cp_mreg = { + .dst = REG_B, + .src = 0, + }; + struct rte_flow_action_jump jump = { + .group = MLX5_FLOW_MREG_ACT_TABLE_GROUP, + }; + struct rte_flow_action actions[] = { + [3] = { .type = RTE_FLOW_ACTION_TYPE_END, }, + }; + struct mlx5_flow_mreg_copy_resource *mcp_res; + int ret; + + /* Fill the register fileds in the flow. */ + ret = mlx5_flow_get_reg_id(dev, MLX5_FLOW_MARK, 0, error); + if (ret < 0) + return NULL; + tag_spec.id = ret; + ret = mlx5_flow_get_reg_id(dev, MLX5_METADATA_RX, 0, error); + if (ret < 0) + return NULL; + cp_mreg.src = ret; + /* Check if already registered. */ + mcp_res = (void *)mlx5_shtable_search(priv->mreg_cp_tbl, + RTE_DIM(priv->mreg_cp_tbl), + mark_id); + if (mcp_res) { + /* For non-default rule. */ + if (mark_id) + mcp_res->refcnt++; + assert(mark_id || mcp_res->refcnt == 1); + return mcp_res; + } + /* Provide the full width of FLAG specific value. */ + if (mark_id == (priv->sh->dv_regc0_mask & MLX5_FLOW_MARK_DEFAULT)) + tag_spec.data = MLX5_FLOW_MARK_DEFAULT; + /* Build a new flow. */ + if (mark_id) { + items[0] = (struct rte_flow_item){ + .type = MLX5_RTE_FLOW_ITEM_TYPE_TAG, + .spec = &tag_spec, + }; + items[1] = (struct rte_flow_item){ + .type = RTE_FLOW_ITEM_TYPE_END, + }; + actions[0] = (struct rte_flow_action){ + .type = MLX5_RTE_FLOW_ACTION_TYPE_MARK, + .conf = &ftag, + }; + actions[1] = (struct rte_flow_action){ + .type = MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG, + .conf = &cp_mreg, + }; + actions[2] = (struct rte_flow_action){ + .type = RTE_FLOW_ACTION_TYPE_JUMP, + .conf = &jump, + }; + actions[3] = (struct rte_flow_action){ + .type = RTE_FLOW_ACTION_TYPE_END, + }; + } else { + /* Default rule, wildcard match. */ + attr.priority = MLX5_FLOW_PRIO_RSVD; + items[0] = (struct rte_flow_item){ + .type = RTE_FLOW_ITEM_TYPE_END, + }; + actions[0] = (struct rte_flow_action){ + .type = MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG, + .conf = &cp_mreg, + }; + actions[1] = (struct rte_flow_action){ + .type = RTE_FLOW_ACTION_TYPE_JUMP, + .conf = &jump, + }; + actions[2] = (struct rte_flow_action){ + .type = RTE_FLOW_ACTION_TYPE_END, + }; + } + /* Build a new entry. */ + mcp_res = rte_zmalloc(__func__, sizeof(*mcp_res), 0); + if (!mcp_res) { + rte_errno = ENOMEM; + return NULL; + } + /* + * The copy Flows are not included in any list. There + * ones are referenced from other Flows and can not + * be applied, removed, deleted in ardbitrary order + * by list traversing. + */ + mcp_res->flow = flow_list_create(dev, NULL, &attr, items, + actions, false, error); + if (!mcp_res->flow) + goto error; + mcp_res->refcnt++; + mcp_res->htbl_ent.key = mark_id; + ret = mlx5_shtable_insert(priv->mreg_cp_tbl, + RTE_DIM(priv->mreg_cp_tbl), + &mcp_res->htbl_ent); + assert(!ret); + return mcp_res; +error: + rte_free(mcp_res); + return NULL; +} + +/** + * Release flow in RX_CP_TBL. + * + * @param dev + * Pointer to Ethernet device. + * @flow + * Parent flow for wich copying is provided. + */ +static void +flow_mreg_del_copy_action(struct rte_eth_dev *dev, + struct rte_flow *flow) +{ + struct mlx5_flow_mreg_copy_resource *mcp_res = flow->mreg_copy; + + if (!mcp_res) + return; + if (flow->copy_applied) { + assert(mcp_res->appcnt); + flow->copy_applied = 0; + --mcp_res->appcnt; + if (!mcp_res->appcnt) + flow_drv_remove(dev, mcp_res->flow); + } + /* + * We do not check availability of metadata registers here, + * because copy resources are allocated in this case. + */ + if (--mcp_res->refcnt) + return; + flow_list_destroy(dev, NULL, mcp_res->flow); + mlx5_shtable_remove(&mcp_res->htbl_ent); + rte_free(mcp_res); + flow->mreg_copy = NULL; +} + +/** + * Start flow in RX_CP_TBL. + * + * @param dev + * Pointer to Ethernet device. + * @flow + * Parent flow for wich copying is provided. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_mreg_start_copy_action(struct rte_eth_dev *dev, + struct rte_flow *flow) +{ + struct mlx5_flow_mreg_copy_resource *mcp_res = flow->mreg_copy; + int ret; + + if (!mcp_res || flow->copy_applied) + return 0; + if (!mcp_res->appcnt) { + ret = flow_drv_apply(dev, mcp_res->flow, NULL); + if (ret) + return ret; + } + ++mcp_res->appcnt; + flow->copy_applied = 1; + return 0; +} + +/** + * Stop flow in RX_CP_TBL. + * + * @param dev + * Pointer to Ethernet device. + * @flow + * Parent flow for wich copying is provided. + */ +static void +flow_mreg_stop_copy_action(struct rte_eth_dev *dev, + struct rte_flow *flow) +{ + struct mlx5_flow_mreg_copy_resource *mcp_res = flow->mreg_copy; + + if (!mcp_res || !flow->copy_applied) + return; + assert(mcp_res->appcnt); + --mcp_res->appcnt; + flow->copy_applied = 0; + if (!mcp_res->appcnt) + flow_drv_remove(dev, mcp_res->flow); +} + +/** + * Remove the default copy action from RX_CP_TBL. + * + * @param dev + * Pointer to Ethernet device. + */ +static void +flow_mreg_del_default_copy_action(struct rte_eth_dev *dev) +{ + struct mlx5_flow_mreg_copy_resource *mcp_res; + struct mlx5_priv *priv = dev->data->dev_private; + + /* Check if default flow is registered. */ + mcp_res = (void *)mlx5_shtable_search(priv->mreg_cp_tbl, + RTE_DIM(priv->mreg_cp_tbl), 0); + if (!mcp_res) + return; + flow_list_destroy(dev, NULL, mcp_res->flow); + mlx5_shtable_remove(&mcp_res->htbl_ent); + rte_free(mcp_res); +} + +/** + * Add the default copy action in in RX_CP_TBL. + * + * @param dev + * Pointer to Ethernet device. + * @param[out] error + * Perform verbose error reporting if not NULL. + * + * @return + * 0 for success, negative value otherwise and rte_errno is set. + */ +static int +flow_mreg_add_default_copy_action(struct rte_eth_dev *dev, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_flow_mreg_copy_resource *mcp_res; + + /* Check whether extensive metadata feature is engaged. */ + if (!priv->config.dv_flow_en || + !priv->config.dv_xmeta_en || + !mlx5_flow_ext_mreg_supported(dev) || + !priv->sh->dv_regc0_mask) + return 0; + mcp_res = flow_mreg_add_copy_action(dev, 0, error); + if (!mcp_res) + return -rte_errno; + return 0; +} + +/** + * Add a flow of copying flow metadata registers in RX_CP_TBL. + * + * All the flow having Q/RSS action should be split by + * flow_mreg_split_qrss_prep() to pass by RX_CP_TBL. A flow in the RX_CP_TBL + * performs the following, + * - CQE->flow_tag := reg_c[1] (MARK) + * - CQE->flow_table_metadata (reg_b) := reg_c[0] (META) + * As CQE's flow_tag is not a register, it can't be simply copied from reg_c[1] + * but there should be a flow per each MARK ID set by MARK action. + * + * For the aforementioned reason, if there's a MARK action in flow's action + * list, a corresponding flow should be added to the RX_CP_TBL in order to copy + * the MARK ID to CQE's flow_tag like, + * - If reg_c[1] is mark_id, + * flow_tag := mark_id, reg_b := reg_c[0] and jump to RX_ACT_TBL. + * + * For SET_META action which stores value in reg_c[0], as the destination is + * also a flow metadata register (reg_b), adding a default flow is enough. Zero + * MARK ID means the default flow. The default flow looks like, + * - For all flow, reg_b := reg_c[0] and jump to RX_ACT_TBL. + * + * @param dev + * Pointer to Ethernet device. + * @param flow + * Pointer to flow structure. + * @param[in] actions + * Pointer to the list of actions. + * @param[out] error + * Perform verbose error reporting if not NULL. + * + * @return + * 0 on success, negative value otherwise and rte_errno is set. + */ +static int +flow_mreg_update_copy_table(struct rte_eth_dev *dev, + struct rte_flow *flow, + const struct rte_flow_action *actions, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_config *config = &priv->config; + struct mlx5_flow_mreg_copy_resource *mcp_res; + const struct rte_flow_action_mark *mark; + + /* Check whether extensive metadata feature is engaged. */ + if (!config->dv_flow_en || + !config->dv_xmeta_en || + !mlx5_flow_ext_mreg_supported(dev) || + !priv->sh->dv_regc0_mask) + return 0; + /* Find MARK action. */ + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_FLAG: + mcp_res = flow_mreg_add_copy_action + (dev, MLX5_FLOW_MARK_DEFAULT, error); + if (!mcp_res) + return -rte_errno; + flow->mreg_copy = mcp_res; + if (dev->data->dev_started) { + mcp_res->appcnt++; + flow->copy_applied = 1; + } + return 0; + case RTE_FLOW_ACTION_TYPE_MARK: + mark = (const struct rte_flow_action_mark *) + actions->conf; + mcp_res = + flow_mreg_add_copy_action(dev, mark->id, error); + if (!mcp_res) + return -rte_errno; + flow->mreg_copy = mcp_res; + if (dev->data->dev_started) { + mcp_res->appcnt++; + flow->copy_applied = 1; + } + return 0; + default: + break; + } + } + return 0; +} + #define MLX5_MAX_SPLIT_ACTIONS 24 #define MLX5_MAX_SPLIT_ITEMS 24 @@ -3450,6 +3851,17 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, if (ret < 0) goto error; } + /* + * Update the metadata register copy table. If extensive + * metadata feature is enabled and registers are supported + * we might create the extra rte_flow for each unique + * MARK/FLAG action ID. + */ + if (external || attr->group != MLX5_FLOW_MREG_CP_TABLE_GROUP) { + ret = flow_mreg_update_copy_table(dev, flow, actions, error); + if (ret) + goto error; + } if (dev->data->dev_started) { ret = flow_drv_apply(dev, flow, error); if (ret < 0) @@ -3465,6 +3877,8 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority, hairpin_id); return NULL; error: + assert(flow); + flow_mreg_del_copy_action(dev, flow); ret = rte_errno; /* Save rte_errno before cleanup. */ if (flow->hairpin_flow_id) mlx5_flow_id_release(priv->sh->flow_id_pool, @@ -3573,6 +3987,7 @@ struct rte_flow * flow_drv_destroy(dev, flow); if (list) TAILQ_REMOVE(list, flow, next); + flow_mreg_del_copy_action(dev, flow); rte_free(flow->fdir); rte_free(flow); } @@ -3609,8 +4024,11 @@ struct rte_flow * { struct rte_flow *flow; - TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) + TAILQ_FOREACH_REVERSE(flow, list, mlx5_flows, next) { flow_drv_remove(dev, flow); + flow_mreg_stop_copy_action(dev, flow); + } + flow_mreg_del_default_copy_action(dev); flow_rxq_flags_clear(dev); } @@ -3632,7 +4050,15 @@ struct rte_flow * struct rte_flow_error error; int ret = 0; + /* Make sure default copy action (reg_c[0] -> reg_b) is created. */ + ret = flow_mreg_add_default_copy_action(dev, &error); + if (ret < 0) + return -rte_errno; + /* Apply Flows created by application. */ TAILQ_FOREACH(flow, list, next) { + ret = flow_mreg_start_copy_action(dev, flow); + if (ret < 0) + goto error; ret = flow_drv_apply(dev, flow, &error); if (ret < 0) goto error; diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index c71938b..1fa5fb9 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -38,6 +38,7 @@ enum mlx5_rte_flow_item_type { enum mlx5_rte_flow_action_type { MLX5_RTE_FLOW_ACTION_TYPE_END = INT_MIN, MLX5_RTE_FLOW_ACTION_TYPE_TAG, + MLX5_RTE_FLOW_ACTION_TYPE_MARK, MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG, }; @@ -417,6 +418,21 @@ struct mlx5_flow_dv_push_vlan_action_resource { rte_be32_t vlan_tag; /**< VLAN tag value. */ }; +/* Metadata register copy table entry. */ +struct mlx5_flow_mreg_copy_resource { + /* + * Hash table for copy table. + * - Key is 32-bit MARK action ID. + * - MUST be the first entry. + */ + struct mlx5_shtable_entry htbl_ent; + LIST_ENTRY(mlx5_flow_mreg_copy_resource) next; + /* List entry for device flows. */ + uint32_t refcnt; /* Reference counter. */ + uint32_t appcnt; /* Apply/Remove counter. */ + struct rte_flow *flow; /* Built flow for copy. */ +}; + /* * Max number of actions per DV flow. * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED @@ -510,10 +526,13 @@ struct rte_flow { enum mlx5_flow_drv_type drv_type; /**< Driver type. */ struct mlx5_flow_rss rss; /**< RSS context. */ struct mlx5_flow_counter *counter; /**< Holds flow counter. */ + struct mlx5_flow_mreg_copy_resource *mreg_copy; + /**< pointer to metadata register copy table resource. */ LIST_HEAD(dev_flows, mlx5_flow) dev_flows; /**< Device flows that are part of the flow. */ struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */ uint32_t hairpin_flow_id; /**< The flow id used for hairpin. */ + uint32_t copy_applied:1; /**< The MARK copy Flow os applied. */ }; typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev, diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 19f58cb..6a7e612 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -4064,8 +4064,11 @@ struct field_modify_info modify_tcp[] = { NULL, "groups are not supported"); #else - uint32_t max_group = attributes->transfer ? MLX5_MAX_TABLES_FDB : - MLX5_MAX_TABLES; + uint32_t max_group = attributes->transfer ? + MLX5_MAX_TABLES_FDB : + external ? + MLX5_MAX_TABLES_EXTERNAL : + MLX5_MAX_TABLES; uint32_t table; int ret; @@ -4672,6 +4675,7 @@ struct field_modify_info modify_tcp[] = { MLX5_FLOW_ACTION_DEC_TCP_ACK; break; case MLX5_RTE_FLOW_ACTION_TYPE_TAG: + case MLX5_RTE_FLOW_ACTION_TYPE_MARK: case MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG: break; default: @@ -6508,6 +6512,8 @@ struct field_modify_info modify_tcp[] = { action_flags |= MLX5_FLOW_ACTION_MARK_EXT; break; } + /* Fall-through */ + case MLX5_RTE_FLOW_ACTION_TYPE_MARK: /* Legacy (non-extensive) MARK action. */ tag_resource.tag = mlx5_flow_mark_set (((const struct rte_flow_action_mark *)