From patchwork Tue Feb 22 08:51:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suanming Mou X-Patchwork-Id: 107938 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9F263A0350; Tue, 22 Feb 2022 09:52:43 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F0DA04115B; Tue, 22 Feb 2022 09:52:27 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2057.outbound.protection.outlook.com [40.107.236.57]) by mails.dpdk.org (Postfix) with ESMTP id B800A41151 for ; Tue, 22 Feb 2022 09:52:26 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MUHoNJYGpUSusLfgKkzDmJDRcyqNAgEUutLuu5wyqzWu0bPv0hmJwE6int58DmujRYru8QcFrM9yQt5qTsg38QMp8HA9xnJ3+rKfQ3BOsiZbzawj45w0/sf91yp59/SH0NlJWMazqt2soaq7RSDmGBwG9iHYjSWat9ttHVUUCwqIbNL9MX+uZYqL27l/Swa0aI7czVqvsBKZ6zun4Hno6x8sm7XA36EgpxV7AVOBXDYFGwp/A4xQgr3xh8nMOjdCF6JPQxJXhAHIvxoXarj0Q1uEJOgH7xg9AqHtYnMMrwSgYUsVGW6DBwopo80XDcEExH1ZtrNV6xy5hjTdvYQ3oA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ctim6DmBhi9P/tbme6wFpI7d1TBNVk5ELJ749b9HJ1M=; b=BU2K2pP7TmH1Gu8qoPsm11Jvwk8DdyPuwWn7tZWL47mMNrpMkqmFuqXLZ9cGgHjpAASNWuZLad5/qQPlyhTkXhPwy7Ao9+8ZyYAAQ+4WI85Skp8ki8waBpwIsH3mcOwKrZJFNV3eO0HnPwVtesv/XcRfY41hzYIHYcVgKnrAVOuXuY682Oi2L6XTIG6Q+NKemStjXeVmLXiHlvce3CbBh/9JHz8y0Me+idlCSBDWOCeLLJ0YqZvU3ElAsVY1QZp2TnZbkTo3/+TUeev70IWRmXUwuurztBgcEkOu2IP/5Ay4b3yL0RHRCuu1qB2FfQCgfqq8SHE9RdHc4Z5ynPGqCg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ctim6DmBhi9P/tbme6wFpI7d1TBNVk5ELJ749b9HJ1M=; b=E1G4EKaHepyOTTM3bCz45nxHsdPOln6DgDJsvrqvIth5DdPfCLAt/cphGw62iBTK+Jz/SxfpHnn1sUcAgJYh0qCuWid5gPJHlHOdsFb6LmyUTZmc16pwXwGGF+WInOGbmID0acg+k7NVPgU7sgoYDuj+plBIAorwcuvmdL9P7err7u6IoorXg9LGPODQz+xFLd5uFXg+UEhL40upDHbnrfSVhmD74pvRnUITu3qx8mCa0y5FIHa1TqE5eWsKWskqEzhcERj0c9aExcXAFRzMcvZzMwh/EmT2a4TXPzO/noTRuMNDKD0yHh7sgnXf6qei/QBvWSYnPjoV0LALpIb+hg== Received: from BN6PR18CA0019.namprd18.prod.outlook.com (2603:10b6:404:121::29) by MN2PR12MB3103.namprd12.prod.outlook.com (2603:10b6:208:c5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.16; Tue, 22 Feb 2022 08:52:23 +0000 Received: from BN8NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:404:121:cafe::29) by BN6PR18CA0019.outlook.office365.com (2603:10b6:404:121::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.19 via Frontend Transport; Tue, 22 Feb 2022 08:52:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by BN8NAM11FT064.mail.protection.outlook.com (10.13.176.160) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4995.15 via Frontend Transport; Tue, 22 Feb 2022 08:52:23 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 22 Feb 2022 08:52:22 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Tue, 22 Feb 2022 00:52:19 -0800 From: Suanming Mou To: , CC: , , Subject: [PATCH v2 03/14] net/mlx5: introduce hardware steering enable routine Date: Tue, 22 Feb 2022 10:51:45 +0200 Message-ID: <20220222085156.27137-4-suanmingm@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20220222085156.27137-1-suanmingm@nvidia.com> References: <20220210162926.20436-1-suanmingm@nvidia.com> <20220222085156.27137-1-suanmingm@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 400a94d2-c336-4e17-15ed-08d9f5e0a4db X-MS-TrafficTypeDiagnostic: MN2PR12MB3103:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JQKbs5+i8KLusxDcG2aRW+fRvM2ho5C2TmW63gAVLIfaIOmef1gcaT5jvBMkis0Kw2LDuXZZ90cX8VkqdNLK31c66T1LhcsskhmfZjk9ZfGGMFH2h4vouLnjdvIN/YI5HZSX0bv92qVG3wDXjdumCj1zLtZpF9lNcmv+63dMlvBn1uhElDkIqmSM3/i85fzlkX/EVrWJj47kZD0lfH+ryr3o3UUyPBgEMLoudOk8lvRpVJOvOfdAG5J7UBtPqxWq9vo6ab5UyY6H3u5YG+TyDmXRqR4GEz5+Y/sLkSsZo71Wzsn5NJs24BIyyDzUzkYZog1h8UvN5IwENRMvquexcWdXw2/KINXBZom56/YW+8gjGav0YqjfiifPBWI/WoCyVY7NgdDIaIR+PPvy1q4AnEnVmkbAe6SJ0NscgBhZvjmLDvEsNXTgjYWb6BGSfeCQ3iO3pwIRUgMRPFkFvUWLToR05bnL6V5ucdXLN+3gRWGZsCRn2k4MWKZ9zQMdG1Jw5kEzcNOqpK5MawRbcjgp05+gPoy8+B2AwkDrJy/1pqJJVlU5HFse782+uHORtDOqoQj1mNrs1c9oKuOoGrRlvMeZwBfsqhjQ2XkZwjnYNNu7PrsF4Cq7iLLWrKnjRl7RBMjDkcM1xIYFCDn5ds/ZGaOR8Yw+SnDGm9SC9r0KmpoXgvG/Z9BeJehK1m4rSbPYT3Sn8RYUBk+EnI8MXTHI1Er9oiS0q14EH6LbHfsCClE= X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(40460700003)(36860700001)(2906002)(55016003)(86362001)(82310400004)(356005)(54906003)(81166007)(426003)(2616005)(110136005)(316002)(70206006)(336012)(36756003)(6666004)(7696005)(508600001)(5660300002)(70586007)(4326008)(1076003)(8936002)(16526019)(8676002)(6286002)(47076005)(83380400001)(6636002)(26005)(186003)(36900700001)(309714004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Feb 2022 08:52:23.5061 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 400a94d2-c336-4e17-15ed-08d9f5e0a4db X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3103 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The new hardware steering engine relies on using dedicated steering WQEs instead of direct writing to the low-level steering table entries directly. In the first introduce implementation the hardware steering engine supports the new queue based Flow API, the existing synchronous non-queue based Flow API is not supported. A new dv_flow_en value 2 is added to manage mlx5 PMD steering engine: dv_flow_en rte_flow API rte_flow_async API ------------------------------------------------ 0 support not support 1 support not support 2 not support support This commit introduces the extra dv_flow_en = 2 to specify the new flow initialize and manage operation routine. Signed-off-by: Suanming Mou Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 13 ++++++++++--- drivers/net/mlx5/linux/mlx5_os.c | 4 ++++ drivers/net/mlx5/mlx5.c | 7 ++++++- drivers/net/mlx5/mlx5.h | 3 ++- drivers/net/mlx5/mlx5_flow.c | 22 ++++++++++++++++++++++ 5 files changed, 44 insertions(+), 5 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index c3cc0c0f41..c3f7d61e7d 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1042,10 +1042,17 @@ Driver options - ``dv_flow_en`` parameter [int] - A nonzero value enables the DV flow steering assuming it is supported - by the driver (RDMA Core library version is rdma-core-24.0 or higher). + Value 0 means legacy Verbs flow offloading. - Enabled by default if supported. + Value 1 enables the DV flow steering assuming it is supported by the + driver (RDMA Core library version is rdma-core-24.0 or higher). + + Value 2 enables the WQE based hardware steering. In this mode only + the queue-based rte_flow_q flow management is supported. + + Configured by default to 1 DV flow steering if the driver(RDMA CORE library) + supported. Otherwise, the value will be 0 which indicates legacy Verbs flow + offloading. - ``dv_esw_en`` parameter [int] diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index ecf823da56..0faf26f5b8 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -482,6 +482,8 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv) err = mlx5_alloc_table_hash_list(priv); if (err) goto error; + if (priv->sh->config.dv_flow_en == 2) + return 0; /* The resources below are only valid with DV support. */ #ifdef HAVE_IBV_FLOW_DV_SUPPORT /* Init port id action list. */ @@ -1519,6 +1521,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, priv->drop_queue.hrxq = mlx5_drop_action_create(eth_dev); if (!priv->drop_queue.hrxq) goto error; + if (priv->sh->config.dv_flow_en == 2) + return eth_dev; /* Port representor shares the same max priority with pf port. */ if (!priv->sh->flow_priority_check_flag) { /* Supported Verbs flow priority number detection. */ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 9f65a8f901..f49d30c05c 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1199,7 +1199,12 @@ mlx5_dev_args_check_handler(const char *key, const char *val, void *opaque) } else if (strcmp(MLX5_DV_ESW_EN, key) == 0) { config->dv_esw_en = !!tmp; } else if (strcmp(MLX5_DV_FLOW_EN, key) == 0) { - config->dv_flow_en = !!tmp; + if (tmp > 2) { + DRV_LOG(ERR, "Invalid %s parameter.", key); + rte_errno = EINVAL; + return -rte_errno; + } + config->dv_flow_en = tmp; } else if (strcmp(MLX5_DV_XMETA_EN, key) == 0) { if (tmp != MLX5_XMETA_MODE_LEGACY && tmp != MLX5_XMETA_MODE_META16 && diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 35ea3fb47c..42e725be63 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -287,7 +287,8 @@ struct mlx5_sh_config { int tx_skew; /* Tx scheduling skew between WQE and data on wire. */ uint32_t reclaim_mode:2; /* Memory reclaim mode. */ uint32_t dv_esw_en:1; /* Enable E-Switch DV flow. */ - uint32_t dv_flow_en:1; /* Enable DV flow. */ + /* Enable DV flow. 1 means SW steering, 2 means HW steering. */ + unsigned int dv_flow_en:2; uint32_t dv_xmeta_en:2; /* Enable extensive flow metadata. */ uint32_t dv_miss_info:1; /* Restore packet after partial hw miss. */ uint32_t l3_vxlan_en:1; /* Enable L3 VXLAN flow creation. */ diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index de203c7b33..0f384cc13f 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -6838,6 +6838,15 @@ mlx5_flow_create(struct rte_eth_dev *dev, const struct rte_flow_action actions[], struct rte_flow_error *error) { + struct mlx5_priv *priv = dev->data->dev_private; + + if (priv->sh->config.dv_flow_en == 2) { + rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "Flow non-Q creation not supported"); + return NULL; + } /* * If the device is not started yet, it is not allowed to created a * flow from application. PMD default flows and traffic control flows @@ -7334,6 +7343,13 @@ mlx5_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow, struct rte_flow_error *error __rte_unused) { + struct mlx5_priv *priv = dev->data->dev_private; + + if (priv->sh->config.dv_flow_en == 2) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "Flow non-Q destruction not supported"); flow_list_destroy(dev, MLX5_FLOW_TYPE_GEN, (uintptr_t)(void *)flow); return 0; @@ -7431,7 +7447,13 @@ mlx5_flow_query(struct rte_eth_dev *dev, struct rte_flow_error *error) { int ret; + struct mlx5_priv *priv = dev->data->dev_private; + if (priv->sh->config.dv_flow_en == 2) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "Flow non-Q query not supported"); ret = flow_drv_query(dev, (uintptr_t)(void *)flow, actions, data, error); if (ret < 0)