From patchwork Thu Feb 24 03:10:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suanming Mou X-Patchwork-Id: 108202 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 281F9A0353; Thu, 24 Feb 2022 04:10:53 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F3FBE40DF6; Thu, 24 Feb 2022 04:10:52 +0100 (CET) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2058.outbound.protection.outlook.com [40.107.92.58]) by mails.dpdk.org (Postfix) with ESMTP id 2C44240696 for ; Thu, 24 Feb 2022 04:10:51 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iIzaRLlobqJY0lZByhKkBDY48QsDZ/4fiM1wnIozxpWqk3fqd3CL1+qkhCvjSjfCQys7Lphj6wCCX6T1n1zPTJ/994yIBvB4RW6M/I7nsEJPmFGa0iugG0pwmCNC0Ct2jxKYwFiMjZVhSZrjUtBU8YF7qQROnEmAdHF8rMRklDx85/7BElzNjJLYf8JePGj7hIfQqHGv8I2qGduOBGxtQhy5nq/SCTRsSl4vgNY9grhLTAsp/agcqOcogua3vfJ97KJlmPqGj3MWTcafoM1G9sGG/1d9j1ZlQCqIGdr3bn0kkCuenA3LlmtWM3BErAuLAo3mi7Od4UJHerqQ1VAsCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=x2LgJYy6Qm1uMaHizEKq/qsBbQj7h/5wMe+wVS+v0vI=; b=GvHioD9yKK7PNqlNFkvH1S5DBxAqEfWDvTabmnKMHUxQ3xLf5WMpB3ryroQr+6usIokiFuVm0ssebRRNP5sJSSUtw9Ft4Nx5JTlkNq/EphqtMfALqBevOXbYTdLmwrAeY3dDvpC15qhx5JPxPPa1VHy7KLcw47SSnX1Fww/t8Sq23GuVixo9KzoEwqDTHi/LhoEcPFmbxGI2sAD4wJvllDS/+8dEpytzgOOSnD2iYPDzKP5O0j3V3kdm22c7lW4YpOMIw+z+qPmQeS2RJJja0rwc1na1hcynESZfO6BPEJIes74P7cVl9UelyBLMSMsuXq8M1dtOIjEZ+i8Qr7w/iA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x2LgJYy6Qm1uMaHizEKq/qsBbQj7h/5wMe+wVS+v0vI=; b=TUq1PTLiX+2djCutzc0kMLBxzL5+DcpYQq7okteegMJcyXgkhH/csX70Xcqa8Js6LQ/mgwYMevqC4hUdG4WOUGKFSR2GZDrjDGi7tvmkfI3GMhzxeKOj4RSDH0S2jD9Za96xZV8SBqK7ljo9FMZ+RqijyySM9ARTQ3q+Kue073hKHY1svUZ+jWtWBIwmaDIvZiwY7lGHkL/I0HY+5TySjbhnJobf1cjkxzuD9aHPDE73N2u7Pg9cCZqNcxPjX7zrs0pMpqDYKlN7BdwQJFqO8Fxpaa+7qcpNNzuGq9ZfLVd4XXv0J6vQx+4+wakdebbRYIU8KT2N3iVkYv+GdAFlIg== Received: from DM5PR04CA0033.namprd04.prod.outlook.com (2603:10b6:3:12b::19) by BY5PR12MB4145.namprd12.prod.outlook.com (2603:10b6:a03:212::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22; Thu, 24 Feb 2022 03:10:49 +0000 Received: from DM6NAM11FT004.eop-nam11.prod.protection.outlook.com (2603:10b6:3:12b:cafe::a9) by DM5PR04CA0033.outlook.office365.com (2603:10b6:3:12b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.24 via Frontend Transport; Thu, 24 Feb 2022 03:10:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by DM6NAM11FT004.mail.protection.outlook.com (10.13.172.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Thu, 24 Feb 2022 03:10:48 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 24 Feb 2022 03:10:47 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Wed, 23 Feb 2022 19:10:45 -0800 From: Suanming Mou To: , CC: , , Subject: [PATCH v3 01/14] net/mlx5: introduce hardware steering operation Date: Thu, 24 Feb 2022 05:10:16 +0200 Message-ID: <20220224031029.14049-2-suanmingm@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20220224031029.14049-1-suanmingm@nvidia.com> References: <20220210162926.20436-1-suanmingm@nvidia.com> <20220224031029.14049-1-suanmingm@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: af03077b-d766-4ab9-0347-08d9f7434195 X-MS-TrafficTypeDiagnostic: BY5PR12MB4145:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Y7RxgUF5WKUPicZ6zRy4PmNATQMB6mDpen6GXIAnOc7JoBPmKYKBx8oVu5a8apzytXubPTF450YcBYkNhUJJXXFUGuBXu4E5ggquQyH1NDu8n9pJT38Kakv8C8N5/bKQ4F8BtDUju9j8NwHBcTTsAZxrzE8LQN3Si/S9aElpZM4ouKp3ELOV2iT7wmWiD+6y8MughnbKdrIN6UsZR5+wMZSvh13VfYLewJlWK+mExnVzxTQuCqxLZpsSIKaVDTHN3GNSvgVkZjdJZXQGjPsxf/WDCk/mdAN69kw4iAlEg+IuqLxGn6FnIhGI7A/btWj3FXwsaE/oVHNXMf0h1QyoiFzw/pfihPMI+5e7b6s9ltLLpGEeXAuXWQ3QDiSPU5FD0Q/kF1Q+8uA3plToMI/Y8zqplD1QgnaN3t4zZeHZ0bbGlZb75nmr3WsvmJHCOeUwXjwj85o5nrAanoe6kdTzBz2UntK41QjcWrnCOMgIS0H7k0dPCbtQpjKpm6OHhNqzq9crYrjE2Ud7l7LSHP1buUqb/G+Cl3CNoa29LYVlEBVy7nyl6M+fGult5dpXgCYwJOO2NX2UGhpoXzphKSSTmhBM1e3s8glzCGZcHZKVEw/JJvCn38F+p+fyF8HERPn1/6un+m5rj3u3rQf4yEy2sLDLhgt8L9yA4cJLDMHRt+hYxfMwyGWZuqvCbAQn5j05JfQRFxl78PJE6EA6ofErjA== X-Forefront-Antispam-Report: CIP:12.22.5.238; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(82310400004)(70586007)(8936002)(70206006)(8676002)(5660300002)(55016003)(4326008)(2906002)(40460700003)(36756003)(47076005)(36860700001)(6286002)(2616005)(26005)(508600001)(336012)(356005)(1076003)(7696005)(186003)(316002)(110136005)(6636002)(54906003)(83380400001)(81166007)(6666004)(16526019)(86362001)(426003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Feb 2022 03:10:48.3383 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: af03077b-d766-4ab9-0347-08d9f7434195 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.238]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT004.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4145 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The Connect-X steering is a lookup hardware mechanism that accesses flow tables, matches packets to the rules, and performs specified actions. Historically, mlx5 PMD implements several software engines to manage steering hardware facility: - FW Steering - Verbs/Direct Verbs, uses FW calls to manage flows - SW Steering - DevX/mlx5dv, uses WQEs to access table memory directly However, there are still some disadvantages: - performance is limited, we should invoke firmware either to manage the entire flow, or to handle some internal steering objects - organizing and preparing flow infrastructure (actions, matchers, groups, etc.) on the flow inserting is sure to cause slow flow insertion - security, exposing the low-level steering entries directly to the userspace may cause security risks A new hardware WQE based steering operation with codename "HW Steering" is going to be introduced to get rid of the security risks. And it will take advantage of the recently new introduced async queue-based rte_flow APIs to prepare everything in advance to achieve high insertion rate. In this new HW steering engine, the original SW steering rte_flow API will not be supported in the first implementation, only the new async queue-based flow operations is going to be supported. A new steering mode parameter for dv_flow_en will be introduced and user will be able to engage the new steering engine. This commit adds the basic driver operation. Signed-off-by: Suanming Mou Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_flow_os.h | 1 + drivers/net/mlx5/meson.build | 1 + drivers/net/mlx5/mlx5_flow.c | 1 + drivers/net/mlx5/mlx5_flow.h | 1 + drivers/net/mlx5/mlx5_flow_hw.c | 13 +++++++++++++ drivers/net/mlx5/windows/mlx5_flow_os.h | 1 + 6 files changed, 18 insertions(+) create mode 100644 drivers/net/mlx5/mlx5_flow_hw.c diff --git a/drivers/net/mlx5/linux/mlx5_flow_os.h b/drivers/net/mlx5/linux/mlx5_flow_os.h index 1926d26410..e28a9e0436 100644 --- a/drivers/net/mlx5/linux/mlx5_flow_os.h +++ b/drivers/net/mlx5/linux/mlx5_flow_os.h @@ -9,6 +9,7 @@ #ifdef HAVE_IBV_FLOW_DV_SUPPORT extern const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops; +extern const struct mlx5_flow_driver_ops mlx5_flow_hw_drv_ops; #endif /** diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build index 2f6d8cbb3d..39a2b8c523 100644 --- a/drivers/net/mlx5/meson.build +++ b/drivers/net/mlx5/meson.build @@ -16,6 +16,7 @@ sources = files( 'mlx5_flow.c', 'mlx5_flow_meter.c', 'mlx5_flow_dv.c', + 'mlx5_flow_hw.c', 'mlx5_flow_aso.c', 'mlx5_flow_flex.c', 'mlx5_mac.c', diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index a87ac8e6d7..b289f13fc0 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -76,6 +76,7 @@ const struct mlx5_flow_driver_ops *flow_drv_ops[] = { [MLX5_FLOW_TYPE_MIN] = &mlx5_flow_null_drv_ops, #if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H) [MLX5_FLOW_TYPE_DV] = &mlx5_flow_dv_drv_ops, + [MLX5_FLOW_TYPE_HW] = &mlx5_flow_hw_drv_ops, #endif [MLX5_FLOW_TYPE_VERBS] = &mlx5_flow_verbs_drv_ops, [MLX5_FLOW_TYPE_MAX] = &mlx5_flow_null_drv_ops diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index a20773eeb2..b70ef0c1b8 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -452,6 +452,7 @@ enum mlx5_flow_drv_type { MLX5_FLOW_TYPE_MIN, MLX5_FLOW_TYPE_DV, MLX5_FLOW_TYPE_VERBS, + MLX5_FLOW_TYPE_HW, MLX5_FLOW_TYPE_MAX, }; diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c new file mode 100644 index 0000000000..729d5914a8 --- /dev/null +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 NVIDIA Corporation & Affiliates + */ + +#include + +#include "mlx5_flow.h" + +#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H) + +const struct mlx5_flow_driver_ops mlx5_flow_hw_drv_ops; + +#endif diff --git a/drivers/net/mlx5/windows/mlx5_flow_os.h b/drivers/net/mlx5/windows/mlx5_flow_os.h index dfcb012334..52013b06a0 100644 --- a/drivers/net/mlx5/windows/mlx5_flow_os.h +++ b/drivers/net/mlx5/windows/mlx5_flow_os.h @@ -10,6 +10,7 @@ #if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H) extern const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops; +extern const struct mlx5_flow_driver_ops mlx5_flow_hw_drv_ops; #endif /**