From patchwork Wed Jul 7 08:09:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Liu X-Patchwork-Id: 95461 X-Patchwork-Delegate: andrew.rybchenko@oktetlabs.ru Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 08F42A0C48; Wed, 7 Jul 2021 09:24:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EB7604135F; Wed, 7 Jul 2021 09:24:02 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2047.outbound.protection.outlook.com [40.107.243.47]) by mails.dpdk.org (Postfix) with ESMTP id 0F3F8406FF for ; Wed, 7 Jul 2021 09:24:01 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aYIwKHkx1PJYCDSCPqZfNSi8Ib+YP6qi+/MiIjjPpDvCcLtLRwcnu7l8CjCDLlY/piMjfnjpXAJkzzWju0xSPyQXKzErgxBugOgnjpPxAq5Lzq7OccqeMwd9Fkbxz+eGfKJtvyt9yBKYe+v2CFdEwBNXTjh7C0aRZeHQKGNgpvAWeFxv+nmYstqQhGmC90RNRS/Afxoj/DaXq4UiggBSC9h12PO1Z9zzHluPU6kPRHVk1VvuPgOFhP2Vcw8y9l1ELP6ok0HmTx5vLwL7V4D13CMvpaNXalQo7dF7FhL2yHUnG0DV13/N1XYWCSC92/UORYsYnDpzkX1DA6qsay50MQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KlzmshYDjk8cz4bu9jo+66ipEmORGOIdo7uY0tguXu0=; b=NbGNHHJmTQEp0q/vV7GuOHy4QyyCQ7GKerOHebDKoFQqGKyny/uGM2VDIkzalFlXOvM1xLQ4FIN1mMXqDYLURH3pqsvh83MLM4oRytRskO/gnhbKUiKZbtAFc8vJo6Q8VrnQt0n4qt6YABI1zbpG9h4m7MGFevu1sZpVhgHAvdwgE1FdLRnzUG83BKVF4i4lHk4e9wA1r2GLSh7VKFxJm3/Bi0H1MOT4XATtW6/w7aLvII1COS9k9Lcx7z6JBbrtAMXWkgIJ6Av85WQ8gTzGGxB2kGnTwRmOKk52r09GFkNRCeT8UOlcwSdyN4sjTlEIL8xMHb0m4wcmvOW9Y1kp9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KlzmshYDjk8cz4bu9jo+66ipEmORGOIdo7uY0tguXu0=; b=atDWL6lqEjdmhXbrWpCeBa9PPajRjgxa49XOcuVl7t+J6e6PzmJ77BYTWPOp0SxeOzu2UpBUI8Cdf6iQLkluYSwoFQdWSsNTQ6kB5Nq3Yw9hR2wQwyUgJbU9AW/8/rZJaedLdpkl+Spwpqm2uNiI4ueMKYizoFeq1eDddDyrUyEMlG5LDkwhDOuu9EQlPpiPsR3K/fh4N+2v4uz8d+9xM3mqRSArsoVDPN3/rcKW3nWwa0PEi6x2WUDUa0WOmB8ub7KqUZmc8vY+iuwPljyW+r/+EFJr3k6O6br6RgIRHbVEr528+9ggAu9GqqyumUGo7Yw7PGYB5GcCpBnNIv6Ycw== Received: from MWHPR1701CA0014.namprd17.prod.outlook.com (2603:10b6:301:14::24) by BYAPR12MB2695.namprd12.prod.outlook.com (2603:10b6:a03:71::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.33; Wed, 7 Jul 2021 07:23:48 +0000 Received: from CO1NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:301:14:cafe::76) by MWHPR1701CA0014.outlook.office365.com (2603:10b6:301:14::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.19 via Frontend Transport; Wed, 7 Jul 2021 07:23:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT019.mail.protection.outlook.com (10.13.175.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4308.20 via Frontend Transport; Wed, 7 Jul 2021 07:23:47 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 7 Jul 2021 07:23:43 +0000 From: Rongwei Liu To: , , , , Shahaf Shuler CC: , Date: Wed, 7 Jul 2021 11:09:48 +0300 Message-ID: <20210707080949.1433899-2-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210707080949.1433899-1-rongweil@nvidia.com> References: <5746076.uJIXkIHoMQ@thomas> <20210707080949.1433899-1-rongweil@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 50b32053-6e5a-49ed-6fd2-08d941182967 X-MS-TrafficTypeDiagnostic: BYAPR12MB2695: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: A+wZdT5Ey/GWNTOlqrUPHYs0z9mH2tCdUORC9kRIrqTfApKxAklDRqqQpVeklIbqQbQ6V9hmP7fwTh/OC4Tpzszxy8NaTKuYEUd5Q3GiJ1HVNV2/uOgfX21queDJTh1xxK9d2ajueHBmPSycw0rjNtKFCMGgRGhyKvi0aynt6FmHcKpdTknfuc/346JWepXYpSDJTW/NhtcSZoCgnb9QuLtnFHmf8EHKIsG1QxUZDCFdea72NieGGbmxKqozfZUFv3/T5WE4yMYw7+Z6jjjrT+d543ccwJQhF7z8UAhXRfygG5RytUNqs11h6DPsRsMx1qJEAAJ1lCu+B36YLjYPGt29LY9KhGWybkuYLaVTX09Ww/olbxMBchyhBt6owZGETPb9zoX81kIuu6F1Jm9V928Hl1lpdbMN7vPta9n9oF8zgCOPF9V/FRWcHbwGZPdYlLGRpf4C9RRnncpwOGZBsemyZc+BjmOU0HZTNoLCz0BmA5lZzhA8mS9LAlHSmPCDUIDSXLBo+dgDmM9+eKOfvgahiD4+D2+AJmwg3FFfDH9sFSV7ZWhtKg6QJHrrhO+26DLn/MXJHH5owew0Ex6ph994aP7eePkkt2V+8Sz1GEnzB12zNPzKzVASckBiRe17AqfyO2sm/KJduje9o1iGc7TLUmyAvkF3nlZ8AhsQUmBsPNphrvZUjbQQTLVMB94EVHQDzfH1bQ2FHhncppSRm9LNxDIG/hMPYrYHORtB2hJy4DPI+KEO4oWMZ4EP1TrdtLHIwhlFCcPNuxe2NyGUDw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(136003)(346002)(39860400002)(376002)(46966006)(36840700001)(6286002)(82310400003)(86362001)(26005)(16526019)(36906005)(356005)(82740400003)(6636002)(70586007)(55016002)(186003)(8936002)(8676002)(110136005)(316002)(1076003)(47076005)(30864003)(4326008)(336012)(7696005)(54906003)(36860700001)(70206006)(7636003)(478600001)(107886003)(966005)(2906002)(5660300002)(83380400001)(2616005)(36756003)(426003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jul 2021 07:23:47.8531 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 50b32053-6e5a-49ed-6fd2-08d941182967 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2695 Subject: [dpdk-dev] [PATCH v4 1/2] net/mlx5: add VXLAN header the last 8-bits matching support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This update adds support for the VXLAN header last 8-bits matching when creating steering rules. At the PCIe probe stage, we create a dummy VXLAN matcher using misc5 to check rdma-core library's capability. The logic is, group 0 depends on HCA_CAP to enable misc or misc5 for VXLAN matching while group non zero depends on the rdma-core capability. Signed-off-by: Rongwei Liu Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 11 +- drivers/common/mlx5/mlx5_devx_cmds.c | 3 + drivers/common/mlx5/mlx5_devx_cmds.h | 6 + drivers/common/mlx5/mlx5_prm.h | 41 +++++-- drivers/net/mlx5/linux/mlx5_os.c | 77 +++++++++++++ drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_flow.c | 26 ++++- drivers/net/mlx5/mlx5_flow.h | 4 +- drivers/net/mlx5/mlx5_flow_dv.c | 160 +++++++++++++++++++-------- drivers/net/mlx5/mlx5_flow_verbs.c | 3 +- drivers/vdpa/mlx5/mlx5_vdpa_steer.c | 6 +- 11 files changed, 274 insertions(+), 65 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index a16af32e67..2ae7157617 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -189,8 +189,15 @@ Limitations size and ``txq_inline_min`` settings and may be from 2 (worst case forced by maximal inline settings) to 58. -- Flows with a VXLAN Network Identifier equal (or ends to be equal) - to 0 are not supported. +- Match on VXLAN supports the following fields only: + + - VNI + - Last reserved 8-bits + + Last reserved 8-bits matching is only supported When using DV flow + engine (``dv_flow_en`` = 1). + Group zero's behavior may differ which depends on FW. + Matching value equals 0 (value & mask) is not supported. - L3 VXLAN and VXLAN-GPE tunnels cannot be supported together with MPLSoGRE and MPLSoUDP. diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index f5914bce32..63ae95832d 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -947,6 +947,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, attr->log_max_ft_sampler_num = MLX5_GET (flow_table_nic_cap, hcattr, flow_table_properties_nic_receive.log_max_ft_sampler_num); + attr->flow.tunnel_header_0_1 = MLX5_GET + (flow_table_nic_cap, hcattr, + ft_field_support_2_nic_receive.tunnel_header_0_1); attr->pkt_integrity_match = mlx5_devx_query_pkt_integrity_match(hcattr); /* Query HCA offloads for Ethernet protocol. */ memset(in, 0, sizeof(in)); diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index f8a17b886b..124f43e852 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -89,6 +89,11 @@ struct mlx5_hca_vdpa_attr { uint64_t doorbell_bar_offset; }; +struct mlx5_hca_flow_attr { + uint32_t tunnel_header_0_1; + uint32_t tunnel_header_2_3; +}; + /* HCA supports this number of time periods for LRO. */ #define MLX5_LRO_NUM_SUPP_PERIODS 4 @@ -155,6 +160,7 @@ struct mlx5_hca_attr { uint32_t pkt_integrity_match:1; /* 1 if HW supports integrity item */ struct mlx5_hca_qos_attr qos; struct mlx5_hca_vdpa_attr vdpa; + struct mlx5_hca_flow_attr flow; int log_max_qp_sz; int log_max_cq_sz; int log_max_qp; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 26761f5bd3..7950070976 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -977,6 +977,18 @@ struct mlx5_ifc_fte_match_set_misc4_bits { u8 reserved_at_100[0x100]; }; +struct mlx5_ifc_fte_match_set_misc5_bits { + u8 macsec_tag_0[0x20]; + u8 macsec_tag_1[0x20]; + u8 macsec_tag_2[0x20]; + u8 macsec_tag_3[0x20]; + u8 tunnel_header_0[0x20]; + u8 tunnel_header_1[0x20]; + u8 tunnel_header_2[0x20]; + u8 tunnel_header_3[0x20]; + u8 reserved[0x100]; +}; + /* Flow matcher. */ struct mlx5_ifc_fte_match_param_bits { struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers; @@ -985,12 +997,13 @@ struct mlx5_ifc_fte_match_param_bits { struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2; struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3; struct mlx5_ifc_fte_match_set_misc4_bits misc_parameters_4; + struct mlx5_ifc_fte_match_set_misc5_bits misc_parameters_5; /* * Add reserved bit to match the struct size with the size defined in PRM. * This extension is not required in Linux. */ #ifndef HAVE_INFINIBAND_VERBS_H - u8 reserved_0[0x400]; + u8 reserved_0[0x200]; #endif }; @@ -1007,6 +1020,7 @@ enum { MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT, MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT, MLX5_MATCH_CRITERIA_ENABLE_MISC4_BIT, + MLX5_MATCH_CRITERIA_ENABLE_MISC5_BIT, }; enum { @@ -1784,7 +1798,12 @@ struct mlx5_ifc_roce_caps_bits { * Table 1872 - Flow Table Fields Supported 2 Format */ struct mlx5_ifc_ft_fields_support_2_bits { - u8 reserved_at_0[0x14]; + u8 reserved_at_0[0xf]; + u8 tunnel_header_2_3[0x1]; + u8 tunnel_header_0_1[0x1]; + u8 macsec_syndrome[0x1]; + u8 macsec_tag[0x1]; + u8 outer_lrh_sl[0x1]; u8 inner_ipv4_ihl[0x1]; u8 outer_ipv4_ihl[0x1]; u8 psp_syndrome[0x1]; @@ -1797,18 +1816,26 @@ struct mlx5_ifc_ft_fields_support_2_bits { u8 inner_l4_checksum_ok[0x1]; u8 outer_ipv4_checksum_ok[0x1]; u8 outer_l4_checksum_ok[0x1]; + u8 reserved_at_20[0x60]; }; struct mlx5_ifc_flow_table_nic_cap_bits { u8 reserved_at_0[0x200]; struct mlx5_ifc_flow_table_prop_layout_bits - flow_table_properties_nic_receive; + flow_table_properties_nic_receive; + struct mlx5_ifc_flow_table_prop_layout_bits + flow_table_properties_nic_receive_rdma; + struct mlx5_ifc_flow_table_prop_layout_bits + flow_table_properties_nic_receive_sniffer; + struct mlx5_ifc_flow_table_prop_layout_bits + flow_table_properties_nic_transmit; + struct mlx5_ifc_flow_table_prop_layout_bits + flow_table_properties_nic_transmit_rdma; struct mlx5_ifc_flow_table_prop_layout_bits - flow_table_properties_unused[5]; - u8 reserved_at_1C0[0x200]; - u8 header_modify_nic_receive[0x400]; + flow_table_properties_nic_transmit_sniffer; + u8 reserved_at_e00[0x600]; struct mlx5_ifc_ft_fields_support_2_bits - ft_field_support_2_nic_receive; + ft_field_support_2_nic_receive; }; struct mlx5_ifc_cmd_hca_cap_2_bits { diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index b3f9e392ab..4fc6969a30 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -193,6 +193,79 @@ mlx5_alloc_verbs_buf(size_t size, void *data) return ret; } +/** + * Detect misc5 support or not + * + * @param[in] priv + * Device private data pointer + */ +#ifdef HAVE_MLX5DV_DR +static void +__mlx5_discovery_misc5_cap(struct mlx5_priv *priv) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT + /* Dummy VxLAN matcher to detect rdma-core misc5 cap + * Case: IPv4--->UDP--->VxLAN--->vni + */ + void *tbl; + struct mlx5_flow_dv_match_params matcher_mask; + void *match_m; + void *matcher; + void *headers_m; + void *misc5_m; + uint32_t *tunnel_header_m; + struct mlx5dv_flow_matcher_attr dv_attr; + + memset(&matcher_mask, 0, sizeof(matcher_mask)); + matcher_mask.size = sizeof(matcher_mask.buf); + match_m = matcher_mask.buf; + headers_m = MLX5_ADDR_OF(fte_match_param, match_m, outer_headers); + misc5_m = MLX5_ADDR_OF(fte_match_param, + match_m, misc_parameters_5); + tunnel_header_m = (uint32_t *) + MLX5_ADDR_OF(fte_match_set_misc5, + misc5_m, tunnel_header_1); + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_protocol, 0xff); + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 4); + MLX5_SET(fte_match_set_lyr_2_4, headers_m, udp_dport, 0xffff); + *tunnel_header_m = 0xffffff; + + tbl = mlx5_glue->dr_create_flow_tbl(priv->sh->rx_domain, 1); + if (!tbl) { + DRV_LOG(INFO, "No SW steering support"); + return; + } + dv_attr.type = IBV_FLOW_ATTR_NORMAL, + dv_attr.match_mask = (void *)&matcher_mask, + dv_attr.match_criteria_enable = + (1 << MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT) | + (1 << MLX5_MATCH_CRITERIA_ENABLE_MISC5_BIT); + dv_attr.priority = 3; +#ifdef HAVE_MLX5DV_DR_ESWITCH + void *misc2_m; + if (priv->config.dv_esw_en) { + /* FDB enabled reg_c_0 */ + dv_attr.match_criteria_enable |= + (1 << MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT); + misc2_m = MLX5_ADDR_OF(fte_match_param, + match_m, misc_parameters_2); + MLX5_SET(fte_match_set_misc2, misc2_m, + metadata_reg_c_0, 0xffff); + } +#endif + matcher = mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, + &dv_attr, tbl); + if (matcher) { + priv->sh->misc5_cap = 1; + mlx5_glue->dv_destroy_flow_matcher(matcher); + } + mlx5_glue->dr_destroy_flow_tbl(tbl); +#else + RTE_SET_USED(priv); +#endif +} +#endif + /** * Verbs callback to free a memory. * @@ -355,6 +428,8 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv) mlx5_glue->dr_reclaim_domain_memory(sh->fdb_domain, 1); } sh->pop_vlan_action = mlx5_glue->dr_create_flow_action_pop_vlan(); + + __mlx5_discovery_misc5_cap(priv); #endif /* HAVE_MLX5DV_DR */ sh->default_miss_action = mlx5_glue->dr_create_flow_action_default_miss(); @@ -1304,6 +1379,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, goto error; } } + if (config->hca_attr.flow.tunnel_header_0_1) + sh->tunnel_header_0_1 = 1; #endif #ifdef HAVE_MLX5_DR_CREATE_ACTION_ASO if (config->hca_attr.flow_hit_aso && diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 36d7c9ce77..dd9cdff4e0 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1071,6 +1071,8 @@ struct mlx5_dev_ctx_shared { uint32_t qp_ts_format:2; /* QP timestamp formats supported. */ uint32_t meter_aso_en:1; /* Flow Meter ASO is supported. */ uint32_t ct_aso_en:1; /* Connection Tracking ASO is supported. */ + uint32_t tunnel_header_0_1:1; /* tunnel_header_0_1 is supported. */ + uint32_t misc5_cap:1; /* misc5 matcher parameter is supported. */ uint32_t max_port; /* Maximal IB device port index. */ struct mlx5_bond_info bond; /* Bonding information. */ void *ctx; /* Verbs/DV/DevX context. */ diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 459f03ff40..ce1d649347 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -2398,12 +2398,14 @@ mlx5_flow_validate_item_tcp(const struct rte_flow_item *item, /** * Validate VXLAN item. * + * @param[in] dev + * Pointer to the Ethernet device structure. * @param[in] item * Item specification. * @param[in] item_flags * Bit-fields that holds the items detected until now. - * @param[in] target_protocol - * The next protocol in the previous item. + * @param[in] attr + * Flow rule attributes. * @param[out] error * Pointer to error structure. * @@ -2411,24 +2413,32 @@ mlx5_flow_validate_item_tcp(const struct rte_flow_item *item, * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_validate_item_vxlan(const struct rte_flow_item *item, +mlx5_flow_validate_item_vxlan(struct rte_eth_dev *dev, + const struct rte_flow_item *item, uint64_t item_flags, + const struct rte_flow_attr *attr, struct rte_flow_error *error) { const struct rte_flow_item_vxlan *spec = item->spec; const struct rte_flow_item_vxlan *mask = item->mask; int ret; + struct mlx5_priv *priv = dev->data->dev_private; union vni { uint32_t vlan_id; uint8_t vni[4]; } id = { .vlan_id = 0, }; - + const struct rte_flow_item_vxlan nic_mask = { + .vni = "\xff\xff\xff", + .rsvd1 = 0xff, + }; + const struct rte_flow_item_vxlan *valid_mask; if (item_flags & MLX5_FLOW_LAYER_TUNNEL) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, item, "multiple tunnel layers not" " supported"); + valid_mask = &rte_flow_item_vxlan_mask; /* * Verify only UDPv4 is present as defined in * https://tools.ietf.org/html/rfc7348 @@ -2439,9 +2449,15 @@ mlx5_flow_validate_item_vxlan(const struct rte_flow_item *item, "no outer UDP layer found"); if (!mask) mask = &rte_flow_item_vxlan_mask; + /* FDB domain & NIC domain non-zero group */ + if ((attr->transfer || attr->group) && priv->sh->misc5_cap) + valid_mask = &nic_mask; + /* Group zero in NIC domain */ + if (!attr->group && !attr->transfer && priv->sh->tunnel_header_0_1) + valid_mask = &nic_mask; ret = mlx5_flow_item_acceptable (item, (const uint8_t *)mask, - (const uint8_t *)&rte_flow_item_vxlan_mask, + (const uint8_t *)valid_mask, sizeof(struct rte_flow_item_vxlan), MLX5_ITEM_RANGE_NOT_ACCEPTED, error); if (ret < 0) diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 2f2aa962f9..3739dcc319 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -1521,8 +1521,10 @@ int mlx5_flow_validate_item_vlan(const struct rte_flow_item *item, uint64_t item_flags, struct rte_eth_dev *dev, struct rte_flow_error *error); -int mlx5_flow_validate_item_vxlan(const struct rte_flow_item *item, +int mlx5_flow_validate_item_vxlan(struct rte_eth_dev *dev, + const struct rte_flow_item *item, uint64_t item_flags, + const struct rte_flow_attr *attr, struct rte_flow_error *error); int mlx5_flow_validate_item_vxlan_gpe(const struct rte_flow_item *item, uint64_t item_flags, diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index a5a7990d53..87d5d5b90a 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -6924,7 +6924,8 @@ flow_dv_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, last_item = MLX5_FLOW_LAYER_GRE_KEY; break; case RTE_FLOW_ITEM_TYPE_VXLAN: - ret = mlx5_flow_validate_item_vxlan(items, item_flags, + ret = mlx5_flow_validate_item_vxlan(dev, items, + item_flags, attr, error); if (ret < 0) return ret; @@ -7884,15 +7885,7 @@ flow_dv_prepare(struct rte_eth_dev *dev, memset(dev_flow, 0, sizeof(*dev_flow)); dev_flow->handle = dev_handle; dev_flow->handle_idx = handle_idx; - /* - * In some old rdma-core releases, before continuing, a check of the - * length of matching parameter will be done at first. It needs to use - * the length without misc4 param. If the flow has misc4 support, then - * the length needs to be adjusted accordingly. Each param member is - * aligned with a 64B boundary naturally. - */ - dev_flow->dv.value.size = MLX5_ST_SZ_BYTES(fte_match_param) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4); + dev_flow->dv.value.size = MLX5_ST_SZ_BYTES(fte_match_param); dev_flow->ingress = attr->ingress; dev_flow->dv.transfer = attr->transfer; return dev_flow; @@ -8673,6 +8666,10 @@ flow_dv_translate_item_nvgre(void *matcher, void *key, /** * Add VXLAN item to matcher and to the value. * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] attr + * Flow rule attributes. * @param[in, out] matcher * Flow matcher. * @param[in, out] key @@ -8683,7 +8680,9 @@ flow_dv_translate_item_nvgre(void *matcher, void *key, * Item is inner pattern. */ static void -flow_dv_translate_item_vxlan(void *matcher, void *key, +flow_dv_translate_item_vxlan(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + void *matcher, void *key, const struct rte_flow_item *item, int inner) { @@ -8691,13 +8690,16 @@ flow_dv_translate_item_vxlan(void *matcher, void *key, const struct rte_flow_item_vxlan *vxlan_v = item->spec; void *headers_m; void *headers_v; - void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters); - void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters); - char *vni_m; - char *vni_v; + void *misc5_m; + void *misc5_v; + uint32_t *tunnel_header_v; + uint32_t *tunnel_header_m; uint16_t dport; - int size; - int i; + struct mlx5_priv *priv = dev->data->dev_private; + const struct rte_flow_item_vxlan nic_mask = { + .vni = "\xff\xff\xff", + .rsvd1 = 0xff, + }; if (inner) { headers_m = MLX5_ADDR_OF(fte_match_param, matcher, @@ -8716,14 +8718,52 @@ flow_dv_translate_item_vxlan(void *matcher, void *key, } if (!vxlan_v) return; - if (!vxlan_m) - vxlan_m = &rte_flow_item_vxlan_mask; - size = sizeof(vxlan_m->vni); - vni_m = MLX5_ADDR_OF(fte_match_set_misc, misc_m, vxlan_vni); - vni_v = MLX5_ADDR_OF(fte_match_set_misc, misc_v, vxlan_vni); - memcpy(vni_m, vxlan_m->vni, size); - for (i = 0; i < size; ++i) - vni_v[i] = vni_m[i] & vxlan_v->vni[i]; + if (!vxlan_m) { + if ((!attr->group && !priv->sh->tunnel_header_0_1) || + (attr->group && !priv->sh->misc5_cap)) + vxlan_m = &rte_flow_item_vxlan_mask; + else + vxlan_m = &nic_mask; + } + if ((!attr->group && !attr->transfer && !priv->sh->tunnel_header_0_1) || + ((attr->group || attr->transfer) && !priv->sh->misc5_cap)) { + void *misc_m; + void *misc_v; + char *vni_m; + char *vni_v; + int size; + int i; + misc_m = MLX5_ADDR_OF(fte_match_param, + matcher, misc_parameters); + misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters); + size = sizeof(vxlan_m->vni); + vni_m = MLX5_ADDR_OF(fte_match_set_misc, misc_m, vxlan_vni); + vni_v = MLX5_ADDR_OF(fte_match_set_misc, misc_v, vxlan_vni); + memcpy(vni_m, vxlan_m->vni, size); + for (i = 0; i < size; ++i) + vni_v[i] = vni_m[i] & vxlan_v->vni[i]; + return; + } + misc5_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters_5); + misc5_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters_5); + tunnel_header_v = (uint32_t *)MLX5_ADDR_OF(fte_match_set_misc5, + misc5_v, + tunnel_header_1); + tunnel_header_m = (uint32_t *)MLX5_ADDR_OF(fte_match_set_misc5, + misc5_m, + tunnel_header_1); + *tunnel_header_v = (vxlan_v->vni[0] & vxlan_m->vni[0]) | + (vxlan_v->vni[1] & vxlan_m->vni[1]) << 8 | + (vxlan_v->vni[2] & vxlan_m->vni[2]) << 16; + if (*tunnel_header_v) + *tunnel_header_m = vxlan_m->vni[0] | + vxlan_m->vni[1] << 8 | + vxlan_m->vni[2] << 16; + else + *tunnel_header_m = 0x0; + *tunnel_header_v |= (vxlan_v->rsvd1 & vxlan_m->rsvd1) << 24; + if (vxlan_v->rsvd1 & vxlan_m->rsvd1) + *tunnel_header_m |= vxlan_m->rsvd1 << 24; } /** @@ -9887,9 +9927,32 @@ flow_dv_matcher_enable(uint32_t *match_criteria) match_criteria_enable |= (!HEADER_IS_ZERO(match_criteria, misc_parameters_4)) << MLX5_MATCH_CRITERIA_ENABLE_MISC4_BIT; + match_criteria_enable |= + (!HEADER_IS_ZERO(match_criteria, misc_parameters_5)) << + MLX5_MATCH_CRITERIA_ENABLE_MISC5_BIT; return match_criteria_enable; } +static void +__flow_dv_adjust_buf_size(size_t *size, uint8_t match_criteria) +{ + /* + * Check flow matching criteria first, subtract misc5/4 length if flow + * doesn't own misc5/4 parameters. In some old rdma-core releases, + * misc5/4 are not supported, and matcher creation failure is expected + * w/o subtration. If misc5 is provided, misc4 must be counted in since + * misc5 is right after misc4. + */ + if (!(match_criteria & (1 << MLX5_MATCH_CRITERIA_ENABLE_MISC5_BIT))) { + *size = MLX5_ST_SZ_BYTES(fte_match_param) - + MLX5_ST_SZ_BYTES(fte_match_set_misc5); + if (!(match_criteria & (1 << + MLX5_MATCH_CRITERIA_ENABLE_MISC4_BIT))) { + *size -= MLX5_ST_SZ_BYTES(fte_match_set_misc4); + } + } +} + struct mlx5_hlist_entry * flow_dv_tbl_create_cb(struct mlx5_hlist *list, uint64_t key64, void *cb_ctx) { @@ -10156,6 +10219,8 @@ flow_dv_matcher_create_cb(struct mlx5_cache_list *list, *cache = *ref; dv_attr.match_criteria_enable = flow_dv_matcher_enable(cache->mask.buf); + __flow_dv_adjust_buf_size(&ref->mask.size, + dv_attr.match_criteria_enable); dv_attr.priority = ref->priority; if (tbl->is_egress) dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS; @@ -10205,7 +10270,6 @@ flow_dv_matcher_register(struct rte_eth_dev *dev, .error = error, .data = ref, }; - /** * tunnel offload API requires this registration for cases when * tunnel match rule was inserted before tunnel set rule. @@ -12064,8 +12128,7 @@ flow_dv_translate(struct rte_eth_dev *dev, uint64_t action_flags = 0; struct mlx5_flow_dv_matcher matcher = { .mask = { - .size = sizeof(matcher.mask.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + .size = sizeof(matcher.mask.buf), }, }; int actions_n = 0; @@ -12872,7 +12935,8 @@ flow_dv_translate(struct rte_eth_dev *dev, last_item = MLX5_FLOW_LAYER_GRE; break; case RTE_FLOW_ITEM_TYPE_VXLAN: - flow_dv_translate_item_vxlan(match_mask, match_value, + flow_dv_translate_item_vxlan(dev, attr, + match_mask, match_value, items, tunnel); matcher.priority = MLX5_TUNNEL_PRIO_GET(rss_desc); last_item = MLX5_FLOW_LAYER_VXLAN; @@ -12970,10 +13034,6 @@ flow_dv_translate(struct rte_eth_dev *dev, NULL, "cannot create eCPRI parser"); } - /* Adjust the length matcher and device flow value. */ - matcher.mask.size = MLX5_ST_SZ_BYTES(fte_match_param); - dev_flow->dv.value.size = - MLX5_ST_SZ_BYTES(fte_match_param); flow_dv_translate_item_ecpri(dev, match_mask, match_value, items); /* No other protocol should follow eCPRI layer. */ @@ -13274,6 +13334,7 @@ flow_dv_apply(struct rte_eth_dev *dev, struct rte_flow *flow, int idx; struct mlx5_flow_workspace *wks = mlx5_flow_get_thread_workspace(); struct mlx5_flow_rss_desc *rss_desc = &wks->rss_desc; + uint8_t misc_mask; MLX5_ASSERT(wks); for (idx = wks->flow_idx - 1; idx >= 0; idx--) { @@ -13344,6 +13405,8 @@ flow_dv_apply(struct rte_eth_dev *dev, struct rte_flow *flow, } dv->actions[n++] = priv->sh->default_miss_action; } + misc_mask = flow_dv_matcher_enable(dv->value.buf); + __flow_dv_adjust_buf_size(&dv->value.size, misc_mask); err = mlx5_flow_os_create_flow(dv_h->matcher->matcher_object, (void *)&dv->value, n, dv->actions, &dh->drv_flow); @@ -15392,14 +15455,13 @@ __flow_dv_create_policy_flow(struct rte_eth_dev *dev, { int ret; struct mlx5_flow_dv_match_params value = { - .size = sizeof(value.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + .size = sizeof(value.buf), }; struct mlx5_flow_dv_match_params matcher = { - .size = sizeof(matcher.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + .size = sizeof(matcher.buf), }; struct mlx5_priv *priv = dev->data->dev_private; + uint8_t misc_mask; if (match_src_port && (priv->representor || priv->master)) { if (flow_dv_translate_item_port_id(dev, matcher.buf, @@ -15413,6 +15475,8 @@ __flow_dv_create_policy_flow(struct rte_eth_dev *dev, (enum modify_reg)color_reg_c_idx, rte_col_2_mlx5_col(color), UINT32_MAX); + misc_mask = flow_dv_matcher_enable(value.buf); + __flow_dv_adjust_buf_size(&value.size, misc_mask); ret = mlx5_flow_os_create_flow(matcher_object, (void *)&value, actions_n, actions, rule); if (ret) { @@ -15435,14 +15499,12 @@ __flow_dv_create_policy_matcher(struct rte_eth_dev *dev, struct mlx5_flow_tbl_resource *tbl_rsc = sub_policy->tbl_rsc; struct mlx5_flow_dv_matcher matcher = { .mask = { - .size = sizeof(matcher.mask.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + .size = sizeof(matcher.mask.buf), }, .tbl = tbl_rsc, }; struct mlx5_flow_dv_match_params value = { - .size = sizeof(value.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + .size = sizeof(value.buf), }; struct mlx5_flow_cb_ctx ctx = { .error = error, @@ -15822,12 +15884,10 @@ flow_dv_create_mtr_tbls(struct rte_eth_dev *dev, int domain, ret, i; struct mlx5_flow_counter *cnt; struct mlx5_flow_dv_match_params value = { - .size = sizeof(value.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + .size = sizeof(value.buf), }; struct mlx5_flow_dv_match_params matcher_para = { - .size = sizeof(matcher_para.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + .size = sizeof(matcher_para.buf), }; int mtr_id_reg_c = mlx5_flow_get_reg_id(dev, MLX5_MTR_ID, 0, &error); @@ -15836,8 +15896,7 @@ flow_dv_create_mtr_tbls(struct rte_eth_dev *dev, struct mlx5_cache_entry *entry; struct mlx5_flow_dv_matcher matcher = { .mask = { - .size = sizeof(matcher.mask.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + .size = sizeof(matcher.mask.buf), }, }; struct mlx5_flow_dv_matcher *drop_matcher; @@ -15845,6 +15904,7 @@ flow_dv_create_mtr_tbls(struct rte_eth_dev *dev, .error = &error, .data = &matcher, }; + uint8_t misc_mask; if (!priv->mtr_en || mtr_id_reg_c < 0) { rte_errno = ENOTSUP; @@ -15894,6 +15954,8 @@ flow_dv_create_mtr_tbls(struct rte_eth_dev *dev, actions[i++] = priv->sh->dr_drop_action; flow_dv_match_meta_reg(matcher_para.buf, value.buf, (enum modify_reg)mtr_id_reg_c, 0, 0); + misc_mask = flow_dv_matcher_enable(value.buf); + __flow_dv_adjust_buf_size(&value.size, misc_mask); ret = mlx5_flow_os_create_flow (mtrmng->def_matcher[domain]->matcher_object, (void *)&value, i, actions, @@ -15937,6 +15999,8 @@ flow_dv_create_mtr_tbls(struct rte_eth_dev *dev, fm->drop_cnt, NULL); actions[i++] = cnt->action; actions[i++] = priv->sh->dr_drop_action; + misc_mask = flow_dv_matcher_enable(value.buf); + __flow_dv_adjust_buf_size(&value.size, misc_mask); ret = mlx5_flow_os_create_flow(drop_matcher->matcher_object, (void *)&value, i, actions, &fm->drop_rule[domain]); @@ -16217,10 +16281,12 @@ mlx5_flow_dv_discover_counter_offset_support(struct rte_eth_dev *dev) if (ret) goto err; dv_attr.match_criteria_enable = flow_dv_matcher_enable(mask.buf); + __flow_dv_adjust_buf_size(&mask.size, dv_attr.match_criteria_enable); ret = mlx5_flow_os_create_flow_matcher(sh->ctx, &dv_attr, tbl->obj, &matcher); if (ret) goto err; + __flow_dv_adjust_buf_size(&value.size, dv_attr.match_criteria_enable); ret = mlx5_flow_os_create_flow(matcher, (void *)&value, 1, actions, &flow); err: diff --git a/drivers/net/mlx5/mlx5_flow_verbs.c b/drivers/net/mlx5/mlx5_flow_verbs.c index fe9673310a..7b3d0b320d 100644 --- a/drivers/net/mlx5/mlx5_flow_verbs.c +++ b/drivers/net/mlx5/mlx5_flow_verbs.c @@ -1381,7 +1381,8 @@ flow_verbs_validate(struct rte_eth_dev *dev, MLX5_FLOW_LAYER_OUTER_L4_TCP; break; case RTE_FLOW_ITEM_TYPE_VXLAN: - ret = mlx5_flow_validate_item_vxlan(items, item_flags, + ret = mlx5_flow_validate_item_vxlan(dev, items, + item_flags, attr, error); if (ret < 0) return ret; diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_steer.c b/drivers/vdpa/mlx5/mlx5_vdpa_steer.c index 1fcd24c002..383f003966 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_steer.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_steer.c @@ -140,11 +140,13 @@ mlx5_vdpa_rss_flows_create(struct mlx5_vdpa_priv *priv) /**< Matcher value. This value is used as the mask or a key. */ } matcher_mask = { .size = sizeof(matcher_mask.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + MLX5_ST_SZ_BYTES(fte_match_set_misc4) - + MLX5_ST_SZ_BYTES(fte_match_set_misc5), }, matcher_value = { .size = sizeof(matcher_value.buf) - - MLX5_ST_SZ_BYTES(fte_match_set_misc4), + MLX5_ST_SZ_BYTES(fte_match_set_misc4) - + MLX5_ST_SZ_BYTES(fte_match_set_misc5), }; struct mlx5dv_flow_matcher_attr dv_attr = { .type = IBV_FLOW_ATTR_NORMAL,