From patchwork Tue Oct 31 14:27:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 133668 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B22A843252; Tue, 31 Oct 2023 15:28:09 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5FB354029E; Tue, 31 Oct 2023 15:28:09 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2069.outbound.protection.outlook.com [40.107.93.69]) by mails.dpdk.org (Postfix) with ESMTP id 42A8B40284 for ; Tue, 31 Oct 2023 15:28:08 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jLWUHEaa6weL9IXkfJiUGR/y6G0qbJMCEL4MHR9r0JpwdQizXVWpykFExcqhen75+Ch2teDTeRdWevdzUuGRAKneS5xfxHV7ApN9Eu/vMxr3wi4jW0OPpQV0w1Y+LLytspgnQouV/tekk5P91h0V8GlPEEhh2wdFzeyIM9D3cwpw2893EXbcc71hWp7DjA0wXDHO0ltsB8JxXxWHCt+ncbsB6K4wSXW+2pAeiRof2XQ3egaMgs06XHpvJC3OgQQgB5sfgv7+TtN8d6+QSt162i0J/eVxmByfeP8+piRmWsd0/mgOYvEj8aO72VTtV8ctx8hbWmfjkBgX80svrPGFvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=q3LBPYbgN5D47JeYiqDWQ+4OZLLrLB09GKRt2cNZh78=; b=LKkP9wWhM4PDRtfjbc5S84CC6ljc+2Jeku40/dlUPal/TE7k2vfOUlb0iozrzQRNnU1qp45ENbwoR+brjHY0OqaaEvY+4WagztGMiM0K7hFsJyNbJU1B0eu0iNzDz8lEZ7v67CSfRKqmYnR+vv3G816JVjqUTiOcM9fhYSzEewB8ftrT+0u5qYbAoprU2wT4Fz5PreFSslNPTSZruLCink3J9uFHvBwvQAvcovLtBov1uhBA2eA2qwZzulYHoGZhFvOp5oVWZc72TAo6Hzn8Hix9SoFnhnpHSnXxUke+A/GI8/v8pVMsio9OFSCZniOCunZusOb0OzrM0IpqvV2viw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q3LBPYbgN5D47JeYiqDWQ+4OZLLrLB09GKRt2cNZh78=; b=gMQm7PAYlNNt9EOP5lPlw3te3IWSgH+g99S2vrWOdRAXft1sld1s889OMxJuhpRytyCGl0mjBr9GvcVmRIttho9Ui1c7nYoaCuD7A09HBiG/Rw5z3cM/Pe0umwxbDOrPkZxwt/9V7Ak5lJO+EyIsAzXQdyAUJdooxWzCekV92CjjCn+rF+167wseK4AxtfpbUn54g6RHsm6izrgE9gTswK4lIwVw57xEViHoQ0ZhqOMeMHJtgI/7/XiJaBGDgGEERt/V5fzxfFWI052AuqzyL26EyLVLu/0s4mY+8JhCS0aEnYw5HaWBVL/CRC6rRUtxVO2Oh0yZDa5QEE6G+OuTEw== Received: from CYXPR02CA0006.namprd02.prod.outlook.com (2603:10b6:930:cf::10) by DM4PR12MB6302.namprd12.prod.outlook.com (2603:10b6:8:a4::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.28; Tue, 31 Oct 2023 14:28:06 +0000 Received: from CY4PEPF0000EDD5.namprd03.prod.outlook.com (2603:10b6:930:cf:cafe::22) by CYXPR02CA0006.outlook.office365.com (2603:10b6:930:cf::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.30 via Frontend Transport; Tue, 31 Oct 2023 14:28:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CY4PEPF0000EDD5.mail.protection.outlook.com (10.167.241.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Tue, 31 Oct 2023 14:28:05 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:27:54 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:27:51 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou , Itamar Gozlan CC: , Raslan Darawsheh Subject: [PATCH 1/8] net/mlx5/hws: fix leak in FT management Date: Tue, 31 Oct 2023 16:27:26 +0200 Message-ID: <20231031142733.2009166-2-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231031142733.2009166-1-dsosnowski@nvidia.com> References: <20231031142733.2009166-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD5:EE_|DM4PR12MB6302:EE_ X-MS-Office365-Filtering-Correlation-Id: 7cc43abb-dba6-47e9-c616-08dbda1d98ee X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2qprI5+T7Os/uAtvnaPyACA7LuttjqIRMvGLItlLfK3DWtPOOA4ZuASm8EJGXEcBWtu3kyAMJZ8CY7Sytx5PwrQyzd/W6u4VGqI170Drcj0eaqtq0b9AHBHlQy4N//GmSRJm0cmDJ03XBfHDG9fGzgvv/HzZcUgvYhLJC1fpaZ5Wpt3rcEgh5yyhpaA15DOoCwoGFzKDjXf5nCJCJlOn1kgcQFsyYXqrSch50PkxQYxs1EKTia9CJERMmjEhlgkVwDwgkta/q5MDO+ysKsOtTzOZJ4f45Ng3DapWH++jV+clDOdTO2iHFNVVSdRVAiprEoEIlHRdH3RKr9NUf8/A9PG0e5TKISMl925NE6mvZNDuMMe9hHemnC09P2p2mFLhFjS/oNi9UHkrge9tq9GZ9Umu6yLDN7tGs5o6FOuzhbODaBhDlohiL5ly4cVyfYO0Zt6A97P6iBL4gdqt0nnmcWIHl34ybZKGDfL+/2wgJ3GtNg5RiaZlfAKFFeDuEAUI7w9OIIHiWtYkKdRjJKibrclR5lfnZpJbQMJ1hn1Hp4fLgNLmovi/Sl1ug2lm6qE1hbQ2pWzK+H2doVs2qbReapImqTgNd1ADxUwUiNYn1z10zaeIFjrxmkdq/X8HH98KAupvt4zV7smRcyXA8grLOPiX+aIrTbH4gmnQiAOBRDfSGW8o1neBCoxB4mK607+Fes+0KzIj8JLSuPPgZ95A1z5ucNCRTT1woIjNe5kvNYLm9hEEQicNVNcAZN29NSFF X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(396003)(346002)(39860400002)(376002)(136003)(230922051799003)(451199024)(1800799009)(186009)(82310400011)(64100799003)(40470700004)(46966006)(36840700001)(110136005)(2906002)(86362001)(41300700001)(40460700003)(4326008)(8936002)(8676002)(5660300002)(36756003)(478600001)(55016003)(40480700001)(47076005)(7696005)(6666004)(1076003)(107886003)(7636003)(6636002)(54906003)(316002)(16526019)(6286002)(70206006)(26005)(426003)(70586007)(336012)(82740400003)(83380400001)(36860700001)(2616005)(356005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 14:28:05.6655 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7cc43abb-dba6-47e9-c616-08dbda1d98ee X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD5.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6302 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Itamar Gozlan This commit fixes two leaks in flow table management. The first leak was when the default miss table of a flow table was not reset to the default action when setting a new first matcher. The second leak was caused by a missing free for an RTC in the case of disconnecting the last matcher in a table's matcher list. Fixes: b81f95ca770d ("net/mlx5/hws: support default miss table") Signed-off-by: Itamar Gozlan Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/hws/mlx5dr_matcher.c | 41 +++++++-------------------- 1 file changed, 10 insertions(+), 31 deletions(-) diff --git a/drivers/net/mlx5/hws/mlx5dr_matcher.c b/drivers/net/mlx5/hws/mlx5dr_matcher.c index a82c182460..ebe42c44c6 100644 --- a/drivers/net/mlx5/hws/mlx5dr_matcher.c +++ b/drivers/net/mlx5/hws/mlx5dr_matcher.c @@ -253,15 +253,15 @@ static int mlx5dr_matcher_connect(struct mlx5dr_matcher *matcher) goto remove_from_list; } - if (prev) { - /* Reset next miss FT to default (drop refcount) */ - ret = mlx5dr_table_ft_set_default_next_ft(tbl, prev->end_ft); - if (ret) { - DR_LOG(ERR, "Failed to reset matcher ft default miss"); - goto remove_from_list; - } - } else { - /* Update tables missing to current table */ + /* Reset next miss FT to default (drop refcount) */ + ret = mlx5dr_table_ft_set_default_next_ft(tbl, prev ? prev->end_ft : tbl->ft); + if (ret) { + DR_LOG(ERR, "Failed to reset matcher ft default miss"); + goto remove_from_list; + } + + if (!prev) { + /* Update tables missing to current matcher in the table */ ret = mlx5dr_table_update_connected_miss_tables(tbl); if (ret) { DR_LOG(ERR, "Fatal error, failed to update connected miss table"); @@ -276,27 +276,6 @@ static int mlx5dr_matcher_connect(struct mlx5dr_matcher *matcher) return ret; } -static int mlx5dr_last_matcher_disconnect(struct mlx5dr_table *tbl, - struct mlx5dr_devx_obj *prev_ft) -{ - struct mlx5dr_cmd_ft_modify_attr ft_attr = {0}; - - if (tbl->default_miss.miss_tbl) { - /* Connect new last matcher to next miss_tbl if exists */ - return mlx5dr_table_connect_to_miss_table(tbl, - tbl->default_miss.miss_tbl); - } else { - ft_attr.modify_fs = MLX5_IFC_MODIFY_FLOW_TABLE_RTC_ID; - ft_attr.type = tbl->fw_ft_type; - /* Matcher is last, point prev end FT to default miss */ - mlx5dr_cmd_set_attr_connect_miss_tbl(tbl->ctx, - tbl->fw_ft_type, - tbl->type, - &ft_attr); - return mlx5dr_cmd_flow_table_modify(prev_ft, &ft_attr); - } -} - static int mlx5dr_matcher_disconnect(struct mlx5dr_matcher *matcher) { struct mlx5dr_matcher *tmp_matcher, *prev_matcher; @@ -330,7 +309,7 @@ static int mlx5dr_matcher_disconnect(struct mlx5dr_matcher *matcher) goto matcher_reconnect; } } else { - ret = mlx5dr_last_matcher_disconnect(tbl, prev_ft); + ret = mlx5dr_table_connect_to_miss_table(tbl, tbl->default_miss.miss_tbl); if (ret) { DR_LOG(ERR, "Failed to disconnect last matcher"); goto matcher_reconnect; From patchwork Tue Oct 31 14:27:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 133672 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3C89643252; Tue, 31 Oct 2023 15:28:49 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 282614113D; Tue, 31 Oct 2023 15:28:25 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2085.outbound.protection.outlook.com [40.107.93.85]) by mails.dpdk.org (Postfix) with ESMTP id 3559940E64; Tue, 31 Oct 2023 15:28:23 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KADOs1kKNsLWrVVb3yS45OnT8Td7eKEFyfIJuJASixC0lH1TgEI4BJ6PzhqttiM5HSPAzp5LVY7OeFMjwgkF97UN+MnKFkqI2u/4PllATLSYTSXIjjs+OZE1zsc3plz2TS9QCc/SHRPFtIpNXd2iVD6v7hJSh8/unEcdlHLVpjk5kkMjlBC3hq74Dj8A2SKyUMQXwRtkCKIn15BQE4oOeIkKUeGJNtnU10kuFwvwnSAY6QsmqFE2O7IactA+j83qBl90UGXfGr2fTd/QtWrf4M2R/bFLwahTFlMU5kDi61jivQvNPtHXJKoduu/vQ0l13S+b1Nt7pg+hvqTgUzGLWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fEcjYrsoVwRfouF9+qytFT/QvoVk/fgbGRYiLCJky0M=; b=lRUQjf/FQiSsxT8UlnozoGEXqImvX4v7O+WTPh9jr7+6YHz5hcSJiPKwCbIzXuB2FFqNmbMAxJaTA8TjZkKoUyLXMLc9MUB0AyFHYwqFpNtV+SOiccSdP83g+GZyl089q8R9Pu87NzAMBnTNHv0SHXT1gjW80ACwPPFnAlG0fzXi8cPJCsMEC+o5wZM8wnDsrQqcQNHUY4LMbTd0OdDLhqhnSalwMT6Xc1+oMHjHdZgciJvw6/zCD+202cLvxzKIByBMxKpDL/Aw+FuwT0tPbqoRMVOBvR7v3BEdfzjcl7EmzwjrTYsoecuC10txTeTHj7oN9a9xfDdc71ZsYSK93g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fEcjYrsoVwRfouF9+qytFT/QvoVk/fgbGRYiLCJky0M=; b=QvRvfK+K4WpjofofUQJW7XGHoSgbeMkpgmhWRPukstRxOwfpNjuSI0KsIkOg7qAZCpKhE9jEPjqf0x0QahibsoJfwB/S3wfzUUTnZ4UERcX4RtMmpNsLLvciP9biZuk7SNfQG2HTdfB2z9fn8sDJKd5vT8Kaqs1R8QWLbocZtZOTT3qn7+2rx3e+TkYxmmgjtICoYk/5ttIXP9ys+QFmPguEG1xEzxkGkceIUUMjQ7TyazRqrGYUr3+frFtQ1/XncA3Eni+8qhXLqM9vJ0Yxftj52o3DbKxFK1kH0YjSDxLN95l+JP1xCoqBqKeF7BNwyobIObPSDVHvItEjOw1UBw== Received: from MW4PR04CA0076.namprd04.prod.outlook.com (2603:10b6:303:6b::21) by DM4PR12MB6039.namprd12.prod.outlook.com (2603:10b6:8:aa::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.29; Tue, 31 Oct 2023 14:28:19 +0000 Received: from MWH0EPF000971E4.namprd02.prod.outlook.com (2603:10b6:303:6b:cafe::47) by MW4PR04CA0076.outlook.office365.com (2603:10b6:303:6b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.29 via Frontend Transport; Tue, 31 Oct 2023 14:28:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MWH0EPF000971E4.mail.protection.outlook.com (10.167.243.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Tue, 31 Oct 2023 14:28:19 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:27:56 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:27:54 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou , Xueming Li CC: , Raslan Darawsheh , Subject: [PATCH 2/8] common/mlx5: fix controller index parsing Date: Tue, 31 Oct 2023 16:27:27 +0200 Message-ID: <20231031142733.2009166-3-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231031142733.2009166-1-dsosnowski@nvidia.com> References: <20231031142733.2009166-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E4:EE_|DM4PR12MB6039:EE_ X-MS-Office365-Filtering-Correlation-Id: d9cc685a-c331-4f9d-29b7-08dbda1da0ec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fAuYG14Gf6dNo53h6EB+X9EpK0VgR96aFtCNGhte+/BlONUs7EB76h5HeCfZTO1eJx/e6wx+C4Oh2t9qhtBUgLCp2Qf4B+98tj4mOrZONoaAs3sM8bXtceJE2y1Qw+JgiiBudnzo8f9RqnmfwUn5qCfWg6mtRxB9fwLqw7dFOQWTgwpHVIhmBtze+Tm8zVOxMAuYmOGMpjfU73aNnF5rUxx6guhBVcTo4rf0nbWtivMA5pAMfzyZc2XF7TZCM1isH+3xrMKsI+nmG5k+v3WDmenkzfX7nfU1z/R1BWjU3oxD+uiE7f19z4DAc7oXftme0Ccj2k7O+dCoPZORVNS6aEHvaMpt9g3SbuvqP7TuLKgdLHRbkqMgLsEDvftSInLisbaOUoTRqg+BjiHnCAw2rfEUKc7k7abuQKFOzeHHrv6jcy23WpP4KZrxnp6MA2iU3pPSNrIV3VUyjoRhRWfMRUzxGqOuefW8EyFlv6xNqfshFdySIrdPCpin8HcTa4ig9Pi3GJ5otE00s260BaeqdSX0LSS86BHkQXfipoztICldomBAkp6ikgfOPScepuh67EDXVpuUQOn3yW0D6IuP1/qv8lF6+iGP8jWR60c8+xY93Nh5AdD9O4hd0HIBzEB1eGbaOcxcX/MK3Y0ewATEOZY66J30l/uZ93OrJY/+Ybr0J6LnmjRZlvymaHuwcjukB3cUKMXcikfnsIrqn9ifJ7OTvFzAVaNGM67jAsmO57U0tmZ7jvjGBJ/NAHNCHEVL X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(396003)(39860400002)(376002)(136003)(230922051799003)(1800799009)(186009)(64100799003)(82310400011)(451199024)(40470700004)(36840700001)(46966006)(356005)(7636003)(47076005)(36860700001)(55016003)(36756003)(2616005)(26005)(336012)(426003)(1076003)(82740400003)(6286002)(16526019)(83380400001)(86362001)(6666004)(7696005)(40480700001)(2906002)(478600001)(70206006)(54906003)(110136005)(70586007)(4326008)(8676002)(450100002)(41300700001)(5660300002)(8936002)(40460700003)(6636002)(316002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 14:28:19.0389 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d9cc685a-c331-4f9d-29b7-08dbda1da0ec X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E4.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6039 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When probing the Linux kernel network interfaces attached to E-Switch, mlx5 PMD decides the representor type and represented entity using phys_port_name exposed by the mlx5 kernel driver in sysfs. mlx5 PMD first checks this name for multihost controller index. In multihost scenarios, phys_port_name is prefixed with "c[0-9]+" string. Included integer is the controller index. Assuming that phys_port_name contains a string representing a physical port, i.e. "p[0-9]+" string, the parsing logic is incorrect. Both "p[0-9]+" and "c[0-9]+" match the formatting string used to parse phys_port_name, but controller index is still filled out. This patch fixes this behavior by storing the parsed index in a temporary variable and setting controller index if and only if phys_port_name matches multihost controller syntax. Fixes: 59df97f1a832 ("common/mlx5: support sub-function representor parsing") Cc: xuemingl@nvidia.com Cc: stable@dpdk.org Signed-off-by: Dariusz Sosnowski Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/mlx5_common_os.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 7260c1a19f..41345e1597 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -96,10 +96,11 @@ mlx5_translate_port_name(const char *port_name_in, char ctrl = 0, pf_c1, pf_c2, vf_c1, vf_c2, eol; char *end; int sc_items; + int32_t ctrl_num = -1; - sc_items = sscanf(port_name_in, "%c%d", - &ctrl, &port_info_out->ctrl_num); + sc_items = sscanf(port_name_in, "%c%d", &ctrl, &ctrl_num); if (sc_items == 2 && ctrl == 'c') { + port_info_out->ctrl_num = ctrl_num; port_name_in++; /* 'c' */ port_name_in += snprintf(NULL, 0, "%d", port_info_out->ctrl_num); From patchwork Tue Oct 31 14:27:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 133673 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2060543252; Tue, 31 Oct 2023 15:28:56 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 490EC41156; Tue, 31 Oct 2023 15:28:27 +0100 (CET) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2088.outbound.protection.outlook.com [40.107.102.88]) by mails.dpdk.org (Postfix) with ESMTP id 62C6641143 for ; Tue, 31 Oct 2023 15:28:25 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=P1rzKC9+hG7FRxJIvvib5YCJzP5T2mSkWWXHuwH30shyi8Nfpb2cAiI123KCknwrT47YJB1nrk6vxoMG8V+0GK/VCYXiYRYkmtKIg6UssH3TxY7CsHtj/xMIlaZpyRmqapX3qiNb2ExU9pL9x4rSN+XaAir91sTyejyxMnwkRLWaFa75/n66ngbeIAfZbXmeaYnDKYcmPzajCxEbrq9liy2yeTCI2Q7rhOEWGvtlovNCLtAlf9cEhgRIA7QSLtRsOHwk+7hD8/j9RSGwjB8NEuH3zx4UM5rhRuksypuu5ZAYDhQvLxCwHTeXGgTYUz/nOLXLXBMZcO0l+B6a28HI1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f2IWVZ+r4heu6N2b1j5k7HjoUmSHE8myAezAUanMOgY=; b=nMZX/prWoRURHY4ErdvPM2jWQMIrGhSzrqE2ITgZsn01qkTLfR5iijo+M9+s5gkJspZQ1fm3TN/io6Mvh3lVNyyBCQ7uI36n6WCD5DQtSyvJ/vk5PahohgzfsCu7/CHL890sxFa5yRx87cqoOzQrqiz9BKSF5nNW7SuUSspeZQ3LwgCn5MfFfEwvzSs8Nz4FikTDnRD/QyKYjmdn2xy12gkYZsiJbZEu5As7BGv8x6vQAT5MBVE43KXFLlm+2JwveS8mXX9Hg9IuzZaFc/DiII8TYOgNfjQ1O6nRB9O4JwpI9acesxXFEtt7LfswmHCYBu2TkhAd2JhVuGOpQYinPQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f2IWVZ+r4heu6N2b1j5k7HjoUmSHE8myAezAUanMOgY=; b=eD8DMGdU0RcVs+B4pnZu4zrfKfuYBcpAgB1EwObTJ254jcBUKD7RSdq/m3G06WWO2vPAOn/vx71H+IEsSv/N1s9v5lRMMifwDqDCJfS/LSknVhtGc5Pv6D9tzyjh2n3gAoFz1SrrZZIMSMLqED6hl7x+IRpjE4NgpJl2+qUGMZAZz9vzW6dypCTON7kMUzNC7cwDONRLu0B1sK7riqVpQNS89AluZ0MzY72CF66EkqnDoT+Iz80GZQYv6L+gRfBlpt5iD+cV2RT8Mzj2/XoZMXQUOJ9dpudnIooksS3y+paYIOtSkZF4b5rYiRhxliMnBskdjv+5oYoRxq1y8ZO8KA== Received: from MWH0EPF00056D1A.namprd21.prod.outlook.com (2603:10b6:30f:fff2:0:1:0:17) by PH8PR12MB7183.namprd12.prod.outlook.com (2603:10b6:510:228::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.25; Tue, 31 Oct 2023 14:28:22 +0000 Received: from MWH0EPF000971E7.namprd02.prod.outlook.com (2a01:111:f403:c903::1) by MWH0EPF00056D1A.outlook.office365.com (2603:1036:d20::b) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.8 via Frontend Transport; Tue, 31 Oct 2023 14:28:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MWH0EPF000971E7.mail.protection.outlook.com (10.167.243.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Tue, 31 Oct 2023 14:28:21 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:27:59 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:27:57 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou CC: , Raslan Darawsheh Subject: [PATCH 3/8] common/mlx5: add Netlink check for Multiport E-Switch Date: Tue, 31 Oct 2023 16:27:28 +0200 Message-ID: <20231031142733.2009166-4-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231031142733.2009166-1-dsosnowski@nvidia.com> References: <20231031142733.2009166-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E7:EE_|PH8PR12MB7183:EE_ X-MS-Office365-Filtering-Correlation-Id: 198f5600-e0f1-423c-f42d-08dbda1da2a4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sZ5ApEgBpvNnEX86ZgDLwNbABgrU7ttbEU3B+vUMUWKDV47XGFsP96QIygbkrkJmySjcg+Q71ovAbXa+gpVtPvQy6hDGq8Ltfu88YtdFWugzRBr8UfM0HtARGrV8ecEyP6JEHC7P5rqWvnEQMvTkBCqgjeaXvS3oJpo0nfAhdecGMtf4GvBIgT4rc9ffafYmcQDfpfC8f7FVsIMmAO9PaZrJMFLjcGh8laalYxz+XmI2EmckGVxrEj7xddTHzo7Hijm1N8hYiE9GTNRPIEfuqkAmEMWWMhQnmy1uL9qRBbuMhI33bMXhQJR3LAZ8A2QtfSnkloJgN22HvsWSeQxDRKdKfyq4fvI+hVSWNcOXZCbtMuppPGV1bCOmccLyMWxVfTmRISDB3rFxvskSHxPILuS1y3VjECl7hGt1Qknts3AZPDtBs0yhiZ1uZVV1LKxklFeX5tMC3w94wTdDSoUE0ZDNRcW7ZXV7UvgUnJ6YTL4HwoRz/Vv8IwP9wbL9JpdOkGupO2IbZWCk+IgF0N++2IQcWhZaRMILCY849rcLXmjZacbqWGDQ4sZyGnATMkAy76rUK8WDkJnshq7phQ9VFTOuzOAO4ZQjkv5v/zp3H4/SpVncyrOD1fUvQsdH06NWZIUvvwQx+OcR/zPsXZLZBxBks3eAWep9nh7UtOEWNm8Vqo/t0PBG82Fr4DpPxLv7tyxiCVsPS5S32Vb6uDkSj8+InoTg3bu8LEpL0c/RYVnRgQgMh3ux5wZrlJrA/OAc X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(136003)(39860400002)(376002)(396003)(346002)(230922051799003)(82310400011)(1800799009)(451199024)(186009)(64100799003)(40470700004)(46966006)(36840700001)(2906002)(6286002)(26005)(1076003)(16526019)(2616005)(336012)(83380400001)(478600001)(7696005)(36860700001)(6666004)(426003)(47076005)(5660300002)(41300700001)(70206006)(70586007)(110136005)(107886003)(8676002)(4326008)(8936002)(316002)(54906003)(6636002)(40460700003)(356005)(7636003)(82740400003)(86362001)(36756003)(55016003)(40480700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 14:28:21.9120 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 198f5600-e0f1-423c-f42d-08dbda1da2a4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E7.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7183 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch implements checking if Multiport E-Switch is enabled on a given PCI device using Devlink Linux kernel interface. This facility will be used in follow up commits, which add support for such configuration to mlx5 PMD. If mlx5_core Linux kernel module supports Multiport E-Switch, then it can be configured through a Devlink boolean parameter "esw_multiport". Checking the value of this parameter is implemented in mlx5_nl_devlink_esw_multiport_get() function. If such parameter does not exist, this function returns -EINVAL. To manually check if mlx5_core kernel module supports "esw_multiport" parameter, and check if Multiport E-Switch is enabled, one can use the following command: # should be substituted with PCI device address # in format :::. $ devlink dev param show pci/0000:08:00.0 name esw_multiport pci/0000:08:00.0: name esw_multiport type driver-specific values: cmode runtime value true If parameter is not supported, Devlink command fails with "Invalid argument" error. Signed-off-by: Dariusz Sosnowski Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/mlx5_nl.c | 70 +++++++++++++++++++++++++++++ drivers/common/mlx5/linux/mlx5_nl.h | 5 +++ drivers/common/mlx5/version.map | 2 + 3 files changed, 77 insertions(+) diff --git a/drivers/common/mlx5/linux/mlx5_nl.c b/drivers/common/mlx5/linux/mlx5_nl.c index 33670bb5c8..28a1f56dba 100644 --- a/drivers/common/mlx5/linux/mlx5_nl.c +++ b/drivers/common/mlx5/linux/mlx5_nl.c @@ -1962,3 +1962,73 @@ mlx5_nl_read_events(int nlsk_fd, mlx5_nl_event_cb *cb, void *cb_arg) } return 0; } + +static int +mlx5_nl_esw_multiport_cb(struct nlmsghdr *nh, void *arg) +{ + + int ret = -EINVAL; + int *enable = arg; + struct nlattr *tail = RTE_PTR_ADD(nh, nh->nlmsg_len); + struct nlattr *nla = RTE_PTR_ADD(nh, NLMSG_ALIGN(sizeof(*nh)) + + NLMSG_ALIGN(sizeof(struct genlmsghdr))); + + while (nla->nla_len && nla < tail) { + switch (nla->nla_type) { + /* Expected nested attributes case. */ + case DEVLINK_ATTR_PARAM: + case DEVLINK_ATTR_PARAM_VALUES_LIST: + case DEVLINK_ATTR_PARAM_VALUE: + ret = 0; + nla += 1; + break; + case DEVLINK_ATTR_PARAM_VALUE_DATA: + *enable = 1; + return 0; + default: + nla = RTE_PTR_ADD(nla, NLMSG_ALIGN(nla->nla_len)); + } + } + *enable = 0; + return ret; +} + +#define NL_ESW_MULTIPORT_PARAM "esw_multiport" + +int +mlx5_nl_devlink_esw_multiport_get(int nlsk_fd, int family_id, const char *pci_addr, int *enable) +{ + struct nlmsghdr *nlh; + struct genlmsghdr *genl; + uint32_t sn = MLX5_NL_SN_GENERATE; + int ret; + uint8_t buf[NLMSG_ALIGN(sizeof(struct nlmsghdr)) + + NLMSG_ALIGN(sizeof(struct genlmsghdr)) + + NLMSG_ALIGN(sizeof(struct nlattr)) * 4 + + NLMSG_ALIGN(MLX5_NL_MAX_ATTR_SIZE) * 4]; + + memset(buf, 0, sizeof(buf)); + nlh = (struct nlmsghdr *)buf; + nlh->nlmsg_len = sizeof(struct nlmsghdr); + nlh->nlmsg_type = family_id; + nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK; + genl = (struct genlmsghdr *)nl_msg_tail(nlh); + nlh->nlmsg_len += sizeof(struct genlmsghdr); + genl->cmd = DEVLINK_CMD_PARAM_GET; + genl->version = DEVLINK_GENL_VERSION; + nl_attr_put(nlh, DEVLINK_ATTR_BUS_NAME, "pci", 4); + nl_attr_put(nlh, DEVLINK_ATTR_DEV_NAME, pci_addr, strlen(pci_addr) + 1); + nl_attr_put(nlh, DEVLINK_ATTR_PARAM_NAME, + NL_ESW_MULTIPORT_PARAM, sizeof(NL_ESW_MULTIPORT_PARAM)); + ret = mlx5_nl_send(nlsk_fd, nlh, sn); + if (ret >= 0) + ret = mlx5_nl_recv(nlsk_fd, sn, mlx5_nl_esw_multiport_cb, enable); + if (ret < 0) { + DRV_LOG(DEBUG, "Failed to get Multiport E-Switch enable on device %s: %d.", + pci_addr, ret); + return ret; + } + DRV_LOG(DEBUG, "Multiport E-Switch is %sabled for device \"%s\".", + *enable ? "en" : "dis", pci_addr); + return ret; +} diff --git a/drivers/common/mlx5/linux/mlx5_nl.h b/drivers/common/mlx5/linux/mlx5_nl.h index db01d7323e..580de3b769 100644 --- a/drivers/common/mlx5/linux/mlx5_nl.h +++ b/drivers/common/mlx5/linux/mlx5_nl.h @@ -71,6 +71,7 @@ __rte_internal uint32_t mlx5_nl_vlan_vmwa_create(struct mlx5_nl_vlan_vmwa_context *vmwa, uint32_t ifindex, uint16_t tag); +__rte_internal int mlx5_nl_devlink_family_id_get(int nlsk_fd); int mlx5_nl_enable_roce_get(int nlsk_fd, int family_id, const char *pci_addr, int *enable); @@ -82,4 +83,8 @@ int mlx5_nl_read_events(int nlsk_fd, mlx5_nl_event_cb *cb, void *cb_arg); __rte_internal int mlx5_nl_parse_link_status_update(struct nlmsghdr *hdr, uint32_t *ifindex); +__rte_internal +int mlx5_nl_devlink_esw_multiport_get(int nlsk_fd, int family_id, + const char *pci_addr, int *enable); + #endif /* RTE_PMD_MLX5_NL_H_ */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 0758ba76de..074eed46fd 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -125,6 +125,8 @@ INTERNAL { mlx5_mr_addr2mr_bh; mlx5_nl_allmulti; # WINDOWS_NO_EXPORT + mlx5_nl_devlink_esw_multiport_get; # WINDOWS_NO_EXPORT + mlx5_nl_devlink_family_id_get; # WINDOWS_NO_EXPORT mlx5_nl_ifindex; # WINDOWS_NO_EXPORT mlx5_nl_init; # WINDOWS_NO_EXPORT mlx5_nl_mac_addr_add; # WINDOWS_NO_EXPORT From patchwork Tue Oct 31 14:27:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 133669 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 81D4E43252; Tue, 31 Oct 2023 15:28:15 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 97075406A2; Tue, 31 Oct 2023 15:28:14 +0100 (CET) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2067.outbound.protection.outlook.com [40.107.220.67]) by mails.dpdk.org (Postfix) with ESMTP id DED6D40284 for ; Tue, 31 Oct 2023 15:28:13 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SX8TcSq4BUNZ1gIQY5xTPcoi+fDPWLNGXRmLxhikEN5vxWE4n9Btqc1LeNUAg3gJ0a8ge12EdPcfxk/ZoX6eFg78D0dVmm3JlThhdt1XtZofP8YnGRO9pVoOC3J77wWo3SQGyfousL++65p5Rq2hq2ZM2FG6YHBtDFl2CvHQ/ucjfF6R1BemAf40n0oZ28sXlzk63L60Ib8EsfeSh+YTHQwlG4BhHn/Bd0JmFz0qvicDsoSfUGxqBqrn+sr8KSPBg2sq0sub2F4Bf2F0EkwBkScZsg4ZHrAjFwV4E1+yca5eQeCsI095H9FUhKPeeDuYw3JgQM+AnHg8CXNRFixETg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eoTQt3AIDJZvuYXeRtKMQoa+ieQz89mUoi0czRaB0wo=; b=l95pXj9vaPdEraOv98sqQQp2OEFEr0VXTXF4MsfF9G6zOpcEw97oatH6b4mXLoTMxU7SwBt3933EDU4wdL/Fwoemi35VjIbviLJ2bvwWTP11MUegxAVwnO9TbXgiIUScbfI41rG9VHRIQNxbfrCM9UrDXRQZFRI5P+zfDZi9X6Jg95FQRxlw9/apsZbTU+4NdY4B8KeJLeCPOSQiqgYT49P3E1AfFAgKj/myUwY8bJ5HCilE0Nt2uUts3Z431W85v+jkMZ6i7MEZ2zTi+atXjMj9TdcSylO3i3rqBYA5LCt2DLnS4SFFf/PkjeLsYl456MEkTPgRi9VFUIU5gRl6Fw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eoTQt3AIDJZvuYXeRtKMQoa+ieQz89mUoi0czRaB0wo=; b=VRjzLRlsRnxHi5ArhBVu4ju6MJgv9Gb9ujJeele2hYYudnNVQ02k7LE/vRzNGiGhM13tV706DW0+XrfGFJNEC30XKZgxMwFGQUHZRh1S8BKEhu4mZi1iNS6h+u0sCx1QkD2g/fnU0cFFFJ6MgZwzdMfvbX5A1oL3q6j5DQbJDgbNCHlz71sX67W4nuM2k2o1dLPZtBZJH0i31fW3KHwK4cGpCWNO3PqYghlRLr47KVRuGWIXBpepX9mePoNsni9XXoPGIbWRR0FFxh9M47uk687V7Ci98n+0/YMyw048ULBTnq2DWmB6SjyPPxhdYNmO1g8a3GiOHboSwGYFm5fo3w== Received: from CYZPR12CA0018.namprd12.prod.outlook.com (2603:10b6:930:8b::10) by CY8PR12MB8194.namprd12.prod.outlook.com (2603:10b6:930:76::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.28; Tue, 31 Oct 2023 14:28:11 +0000 Received: from CY4PEPF0000EDD2.namprd03.prod.outlook.com (2603:10b6:930:8b:cafe::26) by CYZPR12CA0018.outlook.office365.com (2603:10b6:930:8b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Tue, 31 Oct 2023 14:28:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CY4PEPF0000EDD2.mail.protection.outlook.com (10.167.241.206) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Tue, 31 Oct 2023 14:28:10 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:28:01 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:27:59 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou CC: , Raslan Darawsheh Subject: [PATCH 4/8] net/mlx5: add sysfs check for Multiport E-Switch Date: Tue, 31 Oct 2023 16:27:29 +0200 Message-ID: <20231031142733.2009166-5-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231031142733.2009166-1-dsosnowski@nvidia.com> References: <20231031142733.2009166-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD2:EE_|CY8PR12MB8194:EE_ X-MS-Office365-Filtering-Correlation-Id: 23bd9477-7c79-4f4f-60b2-08dbda1d9c0a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PtRLrmgveBtLwf4MC12UMA6iq08UmRoo/96lLzLZTy1W/xxaa1iZuwnwTSq7dxvxUVY66gvcJuNRoh6qDhOtKwY8KQ47zAQPSt+weg8pR+28WfJqAsyAEY5Flg3iaDQ3y6rXCB4/Mvh+5v3W6m3qlCeFXmZB6mWEXkd5uKWPFGihi2Tb6nEhUnx6e40MTtoZD4nNTrfODkqmQipKUF412U6Rbi+UNxgco8rSk95+T3FaLtbrr74FT+5ON5En3JzemjvXkLaGLfkRhAcrs4fMT6gSVOv/3K2vSP4zrLtI3siXgxAimszgsf/VOrTIY3E/PZ3mDJidOOVIzdArbiHi3hvgtWe5/J1M72GKE3YvPo8Ea0iCFqWRqRot7SVM4tXyGbiYYLUo02KV8SQwY76u3UPYJA0Tpz4FcIyPFh/Cm+aBkSZb/tCyEiG560vXxvI8a8QnVlrAQ11FHsPqlXOA/zk6rX7vWTIySDHarFvkH84ZtnRoEEvo45k7tEmBQ5tq28s4ln4Z0syDlr6hoWPWWpwwIq1/zOn8kkqh+JSCkQPabTc/czupaEOOwk5Kla0nw1Ij0bo0mR5gtJWAf3HHQnjGSDRn0yWtb6whFLP5gnrtrPA8IJ/EG9SCW9sxGX/qyOycMxLB+MAIMbp7Re5EW119ye636LpwSNhsIDfbmAtuI2rRLgqyktNg/fnMbl/n1W2a4043s+t3wp3F9RnbpnKYsq74DJTD35DCSaimm8nCYUYjirxv5DGUqlCyumhT X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(396003)(346002)(376002)(136003)(39860400002)(230922051799003)(82310400011)(1800799009)(186009)(64100799003)(451199024)(36840700001)(46966006)(40470700004)(426003)(6286002)(16526019)(26005)(1076003)(55016003)(40460700003)(86362001)(36756003)(40480700001)(82740400003)(356005)(7636003)(6666004)(83380400001)(2906002)(5660300002)(7696005)(107886003)(36860700001)(478600001)(336012)(47076005)(8936002)(8676002)(316002)(4326008)(6636002)(54906003)(70586007)(41300700001)(2616005)(110136005)(70206006); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 14:28:10.8639 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 23bd9477-7c79-4f4f-60b2-08dbda1d9c0a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD2.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8194 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch implements checking if Multiport E-Switch is enabled on a given PCI device, using sysfs Linux kernel interface. This facility will be used in follow up commits, which add support for such configuration to mlx5 PMD. MLNX_OFED mlx5_core kernel module versions which support Multiport E-Switch do not expose this configuration through Devlink, but through sysfs interface. If such a version is used, then Multiport E-Switch can be enabled (or its state can be probed) through a sysfs file under path: # should be substituted with Linux interface name. /sys/class/net//compat/devlink/lag_port_select_mode Writing "multiport_esw" to this file enables Multiport E-Switch. If "multiport_esw" is read from this file, then Multiport E-Switch is enabled. If this file does not exist or writing "multiport_esw" to this file, raises an error, then Multiport E-Switch is not supported. Signed-off-by: Dariusz Sosnowski Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 69 ++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 2f08f2354e..7a656a7237 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1931,6 +1931,75 @@ mlx5_device_bond_pci_match(const char *ibdev_name, return pf; } +#define SYSFS_MPESW_PARAM_MAX_LEN 16 + +static __rte_unused int +mlx5_sysfs_esw_multiport_get(struct ibv_device *ibv, struct rte_pci_addr *pci_addr, int *enabled) +{ + int nl_rdma; + unsigned int n_ports; + unsigned int i; + int ret; + + /* Provide correct value to have defined enabled state in case of an error. */ + *enabled = 0; + nl_rdma = mlx5_nl_init(NETLINK_RDMA, 0); + if (nl_rdma < 0) + return nl_rdma; + n_ports = mlx5_nl_portnum(nl_rdma, ibv->name); + if (!n_ports) { + ret = -rte_errno; + goto close_nl_rdma; + } + for (i = 1; i <= n_ports; ++i) { + unsigned int ifindex; + char ifname[IF_NAMESIZE + 1]; + struct rte_pci_addr if_pci_addr; + char mpesw[SYSFS_MPESW_PARAM_MAX_LEN + 1]; + FILE *sysfs; + int n; + + ifindex = mlx5_nl_ifindex(nl_rdma, ibv->name, i); + if (!ifindex) + continue; + if (!if_indextoname(ifindex, ifname)) + continue; + MKSTR(sysfs_if_path, "/sys/class/net/%s", ifname); + if (mlx5_get_pci_addr(sysfs_if_path, &if_pci_addr)) + continue; + if (pci_addr->domain != if_pci_addr.domain || + pci_addr->bus != if_pci_addr.bus || + pci_addr->devid != if_pci_addr.devid || + pci_addr->function != if_pci_addr.function) + continue; + MKSTR(sysfs_mpesw_path, + "/sys/class/net/%s/compat/devlink/lag_port_select_mode", ifname); + sysfs = fopen(sysfs_mpesw_path, "r"); + if (!sysfs) + continue; + n = fscanf(sysfs, "%" RTE_STR(SYSFS_MPESW_PARAM_MAX_LEN) "s", mpesw); + fclose(sysfs); + if (n != 1) + continue; + ret = 0; + if (strcmp(mpesw, "multiport_esw") == 0) { + *enabled = 1; + break; + } + *enabled = 0; + break; + } + if (i > n_ports) { + DRV_LOG(DEBUG, "Unable to get Multiport E-Switch state by sysfs."); + rte_errno = ENOENT; + ret = -rte_errno; + } + +close_nl_rdma: + close(nl_rdma); + return ret; +} + /** * Register a PCI device within bonding. * From patchwork Tue Oct 31 14:27:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 133670 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0E9A543252; Tue, 31 Oct 2023 15:28:22 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CA86240A81; Tue, 31 Oct 2023 15:28:17 +0100 (CET) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2041.outbound.protection.outlook.com [40.107.212.41]) by mails.dpdk.org (Postfix) with ESMTP id 7FD2A40A7F for ; Tue, 31 Oct 2023 15:28:16 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=R81e+0mXltx7HBT5t0iGEhNoBgxABVtgZRWx6C9v5gT3VFfu143GlKB/Zp89/L7E4BVd0ZOsdh3Fgv5pVD7wdqwg1fqlnV+ds/nkr23Oqomjj732nvu2qBwdBgSuAZ3qAN0km02kc5CxeWILoyrLZvpvolq2z0B97SUCvU26oapJ7gLnPEg316D09H7Y4iPnVcHce72FL7ppB/HEgZDPxJNSe6X/+xc7LzRVn0hjGFJAaQav+CGrA1WUg4dox5yV9/ijg9q5R7DBdW4QFMI0WU2miruio3ic7v6u74+lUupKxKHBKtligszG326OrMROtk8xfr7qVPWR7cApNX8wrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zOKjEQafrLqsslnyzkHPVnIJ/xff7ZrPPWuQ4NnbUBY=; b=FrElpFxjik6WJIXZOLyk2Sk/BRKV2cdWU1XJaj7TtUG30fyI50wyblmpuq4pBd73er3VvnoAgO5qwXNFn5UTdsKvPpGRUxDGIr6qKU26L2M1SPdg5wx1ScwbvOPuPH0krBwlHRuQ+DEuTwj4Al8Y/PbBaObyLMG76J9qlZpF/h5zIkPqNN6VeDNX/HszEhzLu7usQKorbf7PreZlkwqotSDZGpwNBTWFz0ZkGd12T2tWVAfdZRTJ/mKc3kb5t6XRUZGuHytmP+xMZmZbGl+Qijil3N6JOSsbJ09Q7UrP2QRf/E/29eTLuQUFythxc4Y3XQkBmSmYFJUW5n/sISriIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zOKjEQafrLqsslnyzkHPVnIJ/xff7ZrPPWuQ4NnbUBY=; b=JDXXEl8zjJmIa41UmI6GOVIiNbxE80JAJVIj5r99p52P3cyuqk06LsUtsTyqh28XFopK9lsteri3wYe8GAp9ZCMzd2Tru3Rihto+hifeJ7EQkbFJJTgnwrhmoEN2XsyvRQGNTmoDSgkE4LqZrBX3s0W1Xif+fZsN98d4nY7/CUdlnxgTIqRMeT60BffO9WGdSNGUufLAWsS2c4bzU+hpgKixtTebzy4J7w2OTse2KEvsUMjzvhm9h7jZNpA89Q7rjVH2+xUciXLleTcxN0a/8nj9dz+CuYfhKHwNBj6Qtxjq8HTTv/S6/+MF8LBDZ+HlEDShFI/IIzHVqyxErrmr+w== Received: from DS7PR03CA0296.namprd03.prod.outlook.com (2603:10b6:5:3ad::31) by CY8PR12MB7756.namprd12.prod.outlook.com (2603:10b6:930:85::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.28; Tue, 31 Oct 2023 14:28:14 +0000 Received: from CY4PEPF0000EDD3.namprd03.prod.outlook.com (2603:10b6:5:3ad:cafe::60) by DS7PR03CA0296.outlook.office365.com (2603:10b6:5:3ad::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.29 via Frontend Transport; Tue, 31 Oct 2023 14:28:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CY4PEPF0000EDD3.mail.protection.outlook.com (10.167.241.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Tue, 31 Oct 2023 14:28:14 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:28:03 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:28:01 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou CC: , Raslan Darawsheh Subject: [PATCH 5/8] net/mlx5: add checking Multiport E-Switch state Date: Tue, 31 Oct 2023 16:27:30 +0200 Message-ID: <20231031142733.2009166-6-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231031142733.2009166-1-dsosnowski@nvidia.com> References: <20231031142733.2009166-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD3:EE_|CY8PR12MB7756:EE_ X-MS-Office365-Filtering-Correlation-Id: 8bf4fa95-d234-4f78-1a83-08dbda1d9e0f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4tw8FZk2e4qF/jlmBY2Xep1IvBqouFgBKPRXYnn7ZdoHsRgX3+DBnedCo/dqByS0xcuTzMj3p7OcGz1NriwmgFDuudmaVFM0h0/GaboK/hAax1y2IxjbETz2OkOGWOVtBKjM7l2li+wJD8XvoakGyp8OQBLMYITOrJDl32KkCvi/PqOhuf6MNK0Aro9j0e7hfTpIpsqVPp8+YEZF0KzrQfi75xhdf48Zuo+KD3+2aNTLcDOe7tXifawdGktFKOLBgSq5E9w0aU7u8T99UZiKae/dmS4yV7R6HlmtuqVdhLN9GY6SCOI9kztOjuvnnW/XFFJDxtexO7I3GMrKZbAVkDAUfTv2V7gZIyvGUWZC6p6y3t5atoPf8yCTSMUFyRkNXZsiWhVSgeeoRRzrLtVl3XniTRVMB5Rq4/WSmi6OACej5KYsz4G893sDozSCeTORskGwGWQzrSdnNACCMKvHHzX9/oat7y6RjbzbcYp0I/sCRsJbG2+AvqWUDsG/GouAjQDnMDcLpC1wAnHNGlcCqUAlDHCgic5uEfb5jSO+tjbvvyBLShAmUxk//4DyIGlZDDubjcghYoZekaXsDPfW1BuXg+dix75rR8S4vfwUBwLLrVWEhH4IFLTFem8ZaKpAt44TqZo9BQygpl2Y/JCaFdG8tY+fBQ/Vbg6taMPl4fBHe9Bb4GCEYDZsbGAcluWBoPt0lfrIS5uy5s0rPUoOrwR8azb4jceH+aDj9etxgAQLmShbKjUiSKUKqJnHpoE/ X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(39860400002)(136003)(376002)(396003)(230922051799003)(451199024)(1800799009)(82310400011)(64100799003)(186009)(36840700001)(46966006)(40470700004)(40480700001)(55016003)(40460700003)(6286002)(2616005)(1076003)(26005)(16526019)(478600001)(6666004)(7696005)(36860700001)(426003)(336012)(47076005)(83380400001)(2906002)(70586007)(110136005)(41300700001)(70206006)(107886003)(8676002)(4326008)(54906003)(316002)(6636002)(5660300002)(8936002)(356005)(7636003)(82740400003)(86362001)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 14:28:14.2660 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8bf4fa95-d234-4f78-1a83-08dbda1d9e0f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD3.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7756 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch implements checking if Multiport E-Switch is enabled on a given PCI device. mlx5_is_mpesw_enabled() implements this functionality and it will be used in a follow up commit. mlx5_is_mpesw_enabled() first checks if E-Switch state can be probed using Devlink device parameter. If it cannot be checked or there was an error, then sysfs interface will be probed. If that fails, mlx5_is_mpesw_enabled() assumes that Multiport E-Switch is disabled and returns an error. Signed-off-by: Dariusz Sosnowski Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 51 +++++++++++++++++++++++++++++++- 1 file changed, 50 insertions(+), 1 deletion(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 7a656a7237..8a57edc470 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1931,9 +1931,38 @@ mlx5_device_bond_pci_match(const char *ibdev_name, return pf; } +static int +mlx5_nl_esw_multiport_get(struct rte_pci_addr *pci_addr, int *enabled) +{ + char pci_addr_str[PCI_PRI_STR_SIZE] = { 0 }; + int nlsk_fd; + int devlink_id; + int ret; + + /* Provide correct value to have defined enabled state in case of an error. */ + *enabled = 0; + rte_pci_device_name(pci_addr, pci_addr_str, sizeof(pci_addr_str)); + nlsk_fd = mlx5_nl_init(NETLINK_GENERIC, 0); + if (nlsk_fd < 0) + return nlsk_fd; + devlink_id = mlx5_nl_devlink_family_id_get(nlsk_fd); + if (devlink_id < 0) { + ret = devlink_id; + DRV_LOG(DEBUG, "Unable to get devlink family id for Multiport E-Switch checks " + "by netlink, for PCI device %s", pci_addr_str); + goto close_nlsk_fd; + } + ret = mlx5_nl_devlink_esw_multiport_get(nlsk_fd, devlink_id, pci_addr_str, enabled); + if (ret < 0) + DRV_LOG(DEBUG, "Unable to get Multiport E-Switch state by Netlink."); +close_nlsk_fd: + close(nlsk_fd); + return ret; +} + #define SYSFS_MPESW_PARAM_MAX_LEN 16 -static __rte_unused int +static int mlx5_sysfs_esw_multiport_get(struct ibv_device *ibv, struct rte_pci_addr *pci_addr, int *enabled) { int nl_rdma; @@ -2000,6 +2029,26 @@ mlx5_sysfs_esw_multiport_get(struct ibv_device *ibv, struct rte_pci_addr *pci_ad return ret; } +static __rte_unused int +mlx5_is_mpesw_enabled(struct ibv_device *ibv, struct rte_pci_addr *ibv_pci_addr, int *enabled) +{ + /* + * Try getting Multiport E-Switch state through netlink interface + * If unable, try sysfs interface. If that is unable as well, + * assume that Multiport E-Switch is disabled and return an error. + */ + if (mlx5_nl_esw_multiport_get(ibv_pci_addr, enabled) >= 0 || + mlx5_sysfs_esw_multiport_get(ibv, ibv_pci_addr, enabled) >= 0) + return 0; + DRV_LOG(DEBUG, "Unable to check MPESW state for IB device %s " + "(PCI: " PCI_PRI_FMT ")", + ibv->name, + ibv_pci_addr->domain, ibv_pci_addr->bus, + ibv_pci_addr->devid, ibv_pci_addr->function); + *enabled = 0; + return -rte_errno; +} + /** * Register a PCI device within bonding. * From patchwork Tue Oct 31 14:27:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 133671 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0879843252; Tue, 31 Oct 2023 15:28:38 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7CE6940ED9; Tue, 31 Oct 2023 15:28:23 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by mails.dpdk.org (Postfix) with ESMTP id A58C540E54 for ; Tue, 31 Oct 2023 15:28:20 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K1kWGSKXIPAW2SVkZR3yzYJelkwKiQMccO9EXlwUfVAi474isoG/a2R+z3n1jM/F1SbN2LY6WhEuQ+jRirjlVGRuJIs8NptY0Vf7xdFoGb2YGDLqC8Oo7f+gzyORr+yTQAK9jIDdmpUR2dkXs8Nfss6+bk4tV+B9SttAGtmXv40a1cS+T7xruTWdAwtAMdbDY0Yag21NxE5vfkzMtb3sEEbaTAX0gyG01HAeRKVi1tp9OSO2rBbGuHfnsouQMk2uOqu2jlu5yO3Iw3ry9estV8DugbliC+Ekr++yMnPi9UoNQJrqEahZdoLyMFHipEomZbMWVUsY/nzH4HrGA4ogYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jjgZdKTBjxgmIKtarOxCKhf8ybBmBW2FY07G8GM5nQM=; b=KkG633QnvOZHJlf15utxQOCzT8QX6DIwn4sW5o5bTRKitjjLzykabrVIK6atCEOdwIYaJgn0KxPtk2BpqRT2O3luSHnoLrFrcbriK9reRxE3wznZTkwJa4L2ZhDfBbLHb/Vg8X1pE5iea4Rz7//OZf/HYMRRz1sZGyMLHrUnsj2WUldprmnkqGGRFmf2+oqv5B3f6fSiqRWATHjQqPjsSqYDKBPa19wZeclkjZdW/TEj7frTMSVM1/yU1po09sj/NTzWSbXGxKsyDW+Kk3h6n7d7Vi3T/r62x4VeDPD43OWPxLR2IJL15oM127dtWnNLGryfG/YGtyhg09ZYe4zcbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jjgZdKTBjxgmIKtarOxCKhf8ybBmBW2FY07G8GM5nQM=; b=Jxw3IcNoKujFrt8rFBPsB3mJmdFt50uWsSlqgC3zxugHiiDATbD/Omz0spMy6R0fEFsKT9N1bQZHQIjHBkG0T/dGFoEAeDmHq3qqoe/1w1Kp8ihcCMcGuBdxiW7CQV87LkuKff3/F9sT0XCXacY2XfhTlIqdpOi0MmVm+8fePTQECYdXZA+TbRfOHWn9rC/lyjh7sIP1m8r594j4Ox8gHwKX0AekbAxp9G2G11PaxJL2NGrqOBloYD31k59o9C2oOI0KF0S6kvI416nvrc5M2Y7UQF18/InBla/WImU+8pbNlDNeavSqj964pIg1f6EnZi//zGfhRqqrwhaxjEJ37w== Received: from DS7PR03CA0288.namprd03.prod.outlook.com (2603:10b6:5:3ad::23) by DS0PR12MB8044.namprd12.prod.outlook.com (2603:10b6:8:148::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.28; Tue, 31 Oct 2023 14:28:18 +0000 Received: from CY4PEPF0000EDD3.namprd03.prod.outlook.com (2603:10b6:5:3ad:cafe::25) by DS7PR03CA0288.outlook.office365.com (2603:10b6:5:3ad::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.29 via Frontend Transport; Tue, 31 Oct 2023 14:28:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CY4PEPF0000EDD3.mail.protection.outlook.com (10.167.241.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Tue, 31 Oct 2023 14:28:17 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:28:06 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:28:03 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou CC: , Raslan Darawsheh , Bing Zhao Subject: [PATCH 6/8] net/mlx5: support port probing of Multiport E-Switch device Date: Tue, 31 Oct 2023 16:27:31 +0200 Message-ID: <20231031142733.2009166-7-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231031142733.2009166-1-dsosnowski@nvidia.com> References: <20231031142733.2009166-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD3:EE_|DS0PR12MB8044:EE_ X-MS-Office365-Filtering-Correlation-Id: ade21f74-67c6-4af9-1434-08dbda1da031 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9p1esC9Hf2HO0XbhlDrrOlqWWJAKKi6NZ34GQQPP1s+Ng+BuLRQe0mSaVV1ahuZfhYOLJLz6LTZIXXRmIyQK20dg45nza68lAfDezrIVQb/YhfzJ9oWRxzzXD7ZGorOLnbsTtV8lurWcTg+sPnHB3UzK1ypKnhyoLX6IQRHlUmeIXvbBPGplUJSV2bcqEfju4VDYAaXAltfJzobUxC4j0d4xQ1X/TVAeiq9tuhJpSkS/+kWB5tcto0SXr5y992O4GcQ0yb33hBubBVWUcSaWMtKJZ5DDUVepH5WQ6g5IczFNpcJZRq43+A/I5e1qhmSPgznVgeemWgrGhGJ5zsvKaYhJeAjotmeD9sSqy8oLm+ZpjwGVuk02Fu33/kQRw2HlmQ60g8TfxcDPC2Fenv2QPY6j9uFHL7WorF1IkHW0V/TIfm5iW8pRizp/U/65BNVNoX9iEaL738TecSwLCC9qlNDUg3qbrhslfQUv85WOlIt8O5WTULyUU73W/U57UsPrwItJ9wC2+yCp5oRR8J29DxLXZ/X3H6oB2sGYQCZpvIOJjGVZ5YvhnlWw6QkdMbYU+wlCFsULd9RTfy0Xn98JCKGT1/Q73ut9sFqGcJHYBFQYY9AzZ30327VTe5hlxvzYsFeQDyfw7ALXRyqajVEqAgNHE5O4pUMfNw3ShfRU6PohsbAdPdnJ7doi9YvF+ivlRnGlqN2v4KUr27RIrOd9GdGFPwHfpQDkZKCcvPB2JmQeXwPI3AP1Bm4Ghkg0rC2LK/4yrXhq9UZinuzTLcQbg1MudYycrpRAJvnADtKgLDRttiO40WE+XsvJ4Wph7R04 X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(39860400002)(136003)(376002)(396003)(230173577357003)(230922051799003)(230273577357003)(451199024)(1800799009)(82310400011)(64100799003)(186009)(36840700001)(46966006)(40470700004)(40480700001)(55016003)(40460700003)(6286002)(2616005)(1076003)(26005)(16526019)(478600001)(6666004)(7696005)(36860700001)(426003)(336012)(47076005)(83380400001)(2906002)(70586007)(110136005)(41300700001)(70206006)(107886003)(8676002)(4326008)(54906003)(316002)(6636002)(5660300002)(8936002)(356005)(7636003)(82740400003)(86362001)(36756003)(30864003)(309714004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 14:28:17.8442 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ade21f74-67c6-4af9-1434-08dbda1da031 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD3.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8044 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds support for probing ports of a Multiport E-Switch device to mlx5 PMD. Multiport E-Switch is a configuration of NVIDIA ConnectX/BlueField HCAs where all connected entities (i.e. physical ports, VFs and SFs) share the same switch domain. In this mode, applications are allowed to create transfer flow rules which explicitly match on the physical port on which traffic arrives and/or on VFs and SFs, regardless of the root PF. On top of that, forwarding to any of these entities is allowed. Notably, applications are allowed to explicitly forward traffic to any of the physical ports of the HCA. This patch implements the following procedure for probing ports of the device configured as Multiport E-Switch: 1. EAL calls mlx5 PMD to probe certain PCI device (with address BDF). 2. mlx5 PMD iterates over all existing IB devices: 2.1. Check if IB device has a PCI address which matches BDF. 2.2. Check if IB device is configured as Multiport E-Switch device, using devlink interface. 2.3. Iterate over all IB ports of this device to find a netdev with matching PCI address. If any is found, IB device is chosen to instantiate DPDK ports from it. 3. Iterate over all IB ports of the selected IB device, to choose which ports to instantiate: 3.1. Choose IB ports which match the selected representor ports (selected through representor devarg). Instantiate DPDK ports based on those. 3.2. If IB port represented an uplink port and this port corresponds to the probed PCI device, instantiated DPDK port is selected as a switch master port. Bulk of this work was done in mlx5_os_pci_probe_pf(). To properly enable support for Multiport E-Switch, this patch also changes the following: - Probing of representors of type RTE_ETH_REPRESENTOR_PF is allowed, but if and only if Multiport E-Switch is enabled. - Uplink ports have a representor type NONE and have representor ID equal to UINT16_MAX. rte_eth_dev_representor_info struct returned for uplink ports have their index stored in `pf` field. - flow_hw_set_port_info() used by HWS steering layer sets `is_wire` field to true if a port is an uplink port, if Multiport E-Switch is enabled. - Changing MAC address of a port marked as representor is done directly through its corresponding netdev if it is a Multiport E-Switch uplink. Signed-off-by: Dariusz Sosnowski Signed-off-by: Bing Zhao Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 141 ++++++++++++++ doc/guides/rel_notes/release_23_11.rst | 1 + drivers/common/mlx5/mlx5_common.h | 1 + drivers/net/mlx5/linux/mlx5_os.c | 255 ++++++++++++++++++++++--- drivers/net/mlx5/mlx5.h | 39 ++++ drivers/net/mlx5/mlx5_ethdev.c | 53 ++++- drivers/net/mlx5/mlx5_flow_hw.c | 4 +- drivers/net/mlx5/mlx5_mac.c | 8 +- 8 files changed, 464 insertions(+), 38 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index be5054e68a..584f592433 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1759,6 +1759,147 @@ behavior as librte_net_mlx4:: > port config all rss all > port start all + +Multiport E-Switch +------------------ + +In standard deployments of NVIDIA ConnectX and BlueField HCAs, where embedded switch is enabled, +each physical port is associated with a single switching domain. +Only PFs, VFs and SFs related to that physical port are connected to this domain +and offloaded flow rules are allowed to steer traffic only between the entities in the given domain. + +The following diagram pictures the high level overview of this architecture: + +:: + + .---. .------. .------. .---. .------. .------. + |PF0| |PF0VFi| |PF0SFi| |PF1| |PF1VFi| |PF1SFi| + .-+-. .--+---. .--+---. .-+-. .--+---. .--+---. + | | | | | | + .---|------|--------|-------|------|--------|---------. + | | | | | | | HCA| + | .-+------+--------+---. .-+------+--------+---. | + | | | | | | + | | E-Switch | | E-Switch | | + | | PF0 | | PF1 | | + | | | | | | + | .---------+-----------. .--------+------------. | + | | | | + .--------+--+---+---------------+--+---+--------------. + | | | | + | PHY0 | | PHY1 | + | | | | + .------. .------. + +Multiport E-Switch is a deployment scenario where: + +- All physical ports, PFs, VFs and SFs share the same switching domain. +- Each physical port gets a separate representor port. +- Traffic can be matched or forwarded explicitly between any of the entities + connected to the domain. + +The following diagram pictures the high level overview of this architecture: + +:: + + + .---. .------. .------. .---. .------. .------. + |PF0| |PF0VFi| |PF0SFi| |PF1| |PF1VFi| |PF1SFi| + .-+-. .--+---. .--+---. .-+-. .--+---. .--+---. + | | | | | | + .---|------|--------|-------|------|--------|---------. + | | | | | | | HCA| + | .-+------+--------+-------+------+--------+---. | + | | | | + | | Shared | | + | | E-Switch | | + | | | | + | .---------+----------------------+------------. | + | | | | + .--------+--+---+---------------+--+---+--------------. + | | | | + | PHY0 | | PHY1 | + | | | | + .------. .------. + + +In this deployment a single application can control the switching and forwarding behavior for all +entities on the HCA. + +With this configuration, mlx5 PMD supports: + +- matching traffic coming from physical port, PF, VF or SF using REPRESENTED_PORT items; +- forwarding traffic to physical port, PF, VF or SF using REPRESENTED_PORT actions; + + +Requirements +~~~~~~~~~~~~ + +Supported HCAs: + +- ConnectX family: ConnectX-6 Dx and above. +- BlueField family: BlueField-2 and above. +- FW version: at least ``XX.37.1014``. + +Supported mlx5 kernel modules versions: + +- Upstream Linux - from version 6.3. +- Modules packaged in MLNX_OFED - from version v23.04-0.5.3.3. + + +Configuration +~~~~~~~~~~~~~ + +#. Apply required FW configuration:: + + sudo mlxconfig -d /dev/mst/mt4125_pciconf0 set LAG_RESOURCE_ALLOCATION=1 + +#. Reset FW or cold reboot the host. +#. Switch E-Switch mode on all of the PFs to ``switchdev`` mode:: + + sudo devlink dev eswitch set pci/0000:08:00.0 mode switchdev + sudo devlink dev eswitch set pci/0000:08:00.1 mode switchdev + +#. Enable Multiport E-Switch on all of the PFs:: + + sudo devlink dev param set pci/0000:08:00.0 name esw_multiport value true cmode runtime + sudo devlink dev param set pci/0000:08:00.1 name esw_multiport value true cmode runtime + +#. Configure required number of VFs/SFs:: + + echo 4 | sudo tee /sys/class/net/eth2/device/sriov_numvfs + echo 4 | sudo tee /sys/class/net/eth3/device/sriov_numvfs + +#. Start testpmd and verify that all ports are visible:: + + $ sudo dpdk-testpmd -a 08:00.0,dv_flow_en=2,representor=pf0-1vf0-3 -- -i + testpmd> show port summary all + Number of available ports: 10 + Port MAC Address Name Driver Status Link + 0 E8:EB:D5:18:22:BC 08:00.0_p0 mlx5_pci up 200 Gbps + 1 E8:EB:D5:18:22:BD 08:00.0_p1 mlx5_pci up 200 Gbps + 2 D2:F6:43:0B:9E:19 08:00.0_representor_c0pf0vf0 mlx5_pci up 200 Gbps + 3 E6:42:27:B7:68:BD 08:00.0_representor_c0pf0vf1 mlx5_pci up 200 Gbps + 4 A6:5B:7F:8B:B8:47 08:00.0_representor_c0pf0vf2 mlx5_pci up 200 Gbps + 5 12:93:50:45:89:02 08:00.0_representor_c0pf0vf3 mlx5_pci up 200 Gbps + 6 06:D3:B2:79:FE:AC 08:00.0_representor_c0pf1vf0 mlx5_pci up 200 Gbps + 7 12:FC:08:E4:C2:CA 08:00.0_representor_c0pf1vf1 mlx5_pci up 200 Gbps + 8 8E:A9:9A:D0:35:4C 08:00.0_representor_c0pf1vf2 mlx5_pci up 200 Gbps + 9 E6:35:83:1F:B0:A9 08:00.0_representor_c0pf1vf3 mlx5_pci up 200 Gbps + + +Limitations +~~~~~~~~~~~ + +- Multiport E-Switch is not supported on Windows. +- Multiport E-Switch is supported only with HW Steering flow engine (``dv_flow_en=2``). +- Matching traffic coming from a physical port and forwarding it to a physical port + (either the same or other one) is not supported. + + In order to achieve such a functionality, an application has to setup hairpin queues between + physical port representors and forward the traffic using hairpin queues. + + Usage example ------------- diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst index 93999893bd..f337db19f0 100644 --- a/doc/guides/rel_notes/release_23_11.rst +++ b/doc/guides/rel_notes/release_23_11.rst @@ -157,6 +157,7 @@ New Features * Added support for ``RTE_FLOW_ACTION_TYPE_INDIRECT_LIST`` flow action. * Added support for ``RTE_FLOW_ITEM_TYPE_PTYPE`` flow item. * Added support for ``RTE_FLOW_ACTION_TYPE_PORT_REPRESENTOR`` flow action and mirror. + * Added support for Multiport E-Switch. * **Updated Solarflare net driver.** diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 28f9f41528..9c80277d74 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -169,6 +169,7 @@ struct mlx5_switch_info { int32_t ctrl_num; /**< Controller number (valid for c#pf#vf# format). */ int32_t pf_num; /**< PF number (valid for pfxvfx format only). */ int32_t port_name; /**< Representor port name. */ + int32_t mpesw_owner; /**< MPESW owner port number. */ uint64_t switch_id; /**< Switch identifier. */ }; diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 8a57edc470..8ddf38288e 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -959,7 +959,30 @@ mlx5_representor_match(struct mlx5_dev_spawn_data *spawn, uint16_t repr_id = mlx5_representor_id_encode(switch_info, eth_da->type); + /* + * Assuming Multiport E-Switch device was detected, + * if spawned port is an uplink, check if the port + * was requested through representor devarg. + */ + if (mlx5_is_probed_port_on_mpesw_device(spawn) && + switch_info->name_type == MLX5_PHYS_PORT_NAME_TYPE_UPLINK) { + for (p = 0; p < eth_da->nb_ports; ++p) + if (switch_info->port_name == eth_da->ports[p]) + return true; + rte_errno = EBUSY; + return false; + } switch (eth_da->type) { + case RTE_ETH_REPRESENTOR_PF: + /* + * PF representors provided in devargs translate to uplink ports, but + * if and only if the device is a part of MPESW device. + */ + if (!mlx5_is_probed_port_on_mpesw_device(spawn)) { + rte_errno = EBUSY; + return false; + } + break; case RTE_ETH_REPRESENTOR_SF: if (!(spawn->info.port_name == -1 && switch_info->name_type == @@ -989,7 +1012,7 @@ mlx5_representor_match(struct mlx5_dev_spawn_data *spawn, } /* Check representor ID: */ for (p = 0; p < eth_da->nb_ports; ++p) { - if (spawn->pf_bond < 0) { + if (!mlx5_is_probed_port_on_mpesw_device(spawn) && spawn->pf_bond < 0) { /* For non-LAG mode, allow and ignore pf. */ switch_info->pf_num = eth_da->ports[p]; repr_id = mlx5_representor_id_encode(switch_info, @@ -1051,17 +1074,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, !mlx5_representor_match(spawn, eth_da)) return NULL; /* Build device name. */ - if (spawn->pf_bond < 0) { - /* Single device. */ - if (!switch_info->representor) - strlcpy(name, dpdk_dev->name, sizeof(name)); - else - err = snprintf(name, sizeof(name), "%s_representor_%s%u", - dpdk_dev->name, - switch_info->name_type == - MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf", - switch_info->port_name); - } else { + if (spawn->pf_bond >= 0) { /* Bonding device. */ if (!switch_info->representor) { err = snprintf(name, sizeof(name), "%s_%s", @@ -1075,6 +1088,30 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf", switch_info->port_name); } + } else if (mlx5_is_probed_port_on_mpesw_device(spawn)) { + /* MPESW device. */ + if (switch_info->name_type == MLX5_PHYS_PORT_NAME_TYPE_UPLINK) { + err = snprintf(name, sizeof(name), "%s_p%d", + dpdk_dev->name, spawn->mpesw_port); + } else { + err = snprintf(name, sizeof(name), "%s_representor_c%dpf%d%s%u", + dpdk_dev->name, + switch_info->ctrl_num, + switch_info->pf_num, + switch_info->name_type == + MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf", + switch_info->port_name); + } + } else { + /* Single device. */ + if (!switch_info->representor) + strlcpy(name, dpdk_dev->name, sizeof(name)); + else + err = snprintf(name, sizeof(name), "%s_representor_%s%u", + dpdk_dev->name, + switch_info->name_type == + MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf", + switch_info->port_name); } if (err >= (int)sizeof(name)) DRV_LOG(WARNING, "device name overflow %s", name); @@ -1202,13 +1239,25 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, priv->vport_meta_tag = 0; priv->vport_meta_mask = 0; priv->pf_bond = spawn->pf_bond; + priv->mpesw_port = spawn->mpesw_port; + priv->mpesw_uplink = false; + priv->mpesw_owner = spawn->info.mpesw_owner; + if (mlx5_is_port_on_mpesw_device(priv)) + priv->mpesw_uplink = (spawn->info.name_type == MLX5_PHYS_PORT_NAME_TYPE_UPLINK); DRV_LOG(DEBUG, - "dev_port=%u bus=%s pci=%s master=%d representor=%d pf_bond=%d\n", + "dev_port=%u bus=%s pci=%s master=%d representor=%d pf_bond=%d " + "mpesw_port=%d mpesw_uplink=%d", priv->dev_port, dpdk_dev->bus->name, priv->pci_dev ? priv->pci_dev->name : "NONE", - priv->master, priv->representor, priv->pf_bond); + priv->master, priv->representor, priv->pf_bond, + priv->mpesw_port, priv->mpesw_uplink); + if (mlx5_is_port_on_mpesw_device(priv) && priv->sh->config.dv_flow_en != 2) { + DRV_LOG(ERR, "MPESW device is supported only with HWS"); + err = ENOTSUP; + goto error; + } /* * If we have E-Switch we should determine the vport attributes. * E-Switch may use either source vport field or reg_c[0] metadata @@ -2029,7 +2078,7 @@ mlx5_sysfs_esw_multiport_get(struct ibv_device *ibv, struct rte_pci_addr *pci_ad return ret; } -static __rte_unused int +static int mlx5_is_mpesw_enabled(struct ibv_device *ibv, struct rte_pci_addr *ibv_pci_addr, int *enabled) { /* @@ -2049,6 +2098,84 @@ mlx5_is_mpesw_enabled(struct ibv_device *ibv, struct rte_pci_addr *ibv_pci_addr, return -rte_errno; } +static int +mlx5_device_mpesw_pci_match(struct ibv_device *ibv, + const struct rte_pci_addr *owner_pci, + int nl_rdma) +{ + struct rte_pci_addr ibdev_pci_addr = { 0 }; + char ifname[IF_NAMESIZE + 1] = { 0 }; + unsigned int ifindex; + unsigned int np; + unsigned int i; + int enabled = 0; + int ret; + + /* Check if IB device's PCI address matches the probed PCI address. */ + if (mlx5_get_pci_addr(ibv->ibdev_path, &ibdev_pci_addr)) { + DRV_LOG(DEBUG, "Skipping MPESW check for IB device %s since " + "there is no underlying PCI device", ibv->name); + rte_errno = ENOENT; + return -rte_errno; + } + if (ibdev_pci_addr.domain != owner_pci->domain || + ibdev_pci_addr.bus != owner_pci->bus || + ibdev_pci_addr.devid != owner_pci->devid || + ibdev_pci_addr.function != owner_pci->function) { + return -1; + } + /* Check if IB device has MPESW enabled. */ + if (mlx5_is_mpesw_enabled(ibv, &ibdev_pci_addr, &enabled)) + return -1; + if (!enabled) + return -1; + /* Iterate through IB ports to find MPESW master uplink port. */ + if (nl_rdma < 0) + return -1; + np = mlx5_nl_portnum(nl_rdma, ibv->name); + if (!np) + return -1; + for (i = 1; i <= np; ++i) { + struct rte_pci_addr pci_addr; + FILE *file; + char port_name[IF_NAMESIZE + 1]; + struct mlx5_switch_info info; + + /* Check whether IB port has a corresponding netdev. */ + ifindex = mlx5_nl_ifindex(nl_rdma, ibv->name, i); + if (!ifindex) + continue; + if (!if_indextoname(ifindex, ifname)) + continue; + /* Read port name and determine its type. */ + MKSTR(ifphysportname, "/sys/class/net/%s/phys_port_name", ifname); + file = fopen(ifphysportname, "rb"); + if (!file) + continue; + ret = fscanf(file, "%16s", port_name); + fclose(file); + if (ret != 1) + continue; + memset(&info, 0, sizeof(info)); + mlx5_translate_port_name(port_name, &info); + if (info.name_type != MLX5_PHYS_PORT_NAME_TYPE_UPLINK) + continue; + /* Fetch PCI address of the device to which the netdev is bound. */ + MKSTR(ifpath, "/sys/class/net/%s", ifname); + if (mlx5_get_pci_addr(ifpath, &pci_addr)) + continue; + if (pci_addr.domain == ibdev_pci_addr.domain && + pci_addr.bus == ibdev_pci_addr.bus && + pci_addr.devid == ibdev_pci_addr.devid && + pci_addr.function == ibdev_pci_addr.function) { + MLX5_ASSERT(info.port_name >= 0); + return info.port_name; + } + } + /* No matching MPESW uplink port was found. */ + return -1; +} + /** * Register a PCI device within bonding. * @@ -2097,6 +2224,12 @@ mlx5_os_pci_probe_pf(struct mlx5_common_device *cdev, * >= 0 - bonding device (value is slave PF index) */ int bd = -1; + /* + * Multiport E-Switch (MPESW) device: + * < 0 - no MPESW device or could not determine if it is MPESW device, + * >= 0 - MPESW device. Value is the port index of the MPESW owner. + */ + int mpesw = MLX5_MPESW_PORT_INVALID; struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(cdev->dev); struct mlx5_dev_spawn_data *list = NULL; struct rte_eth_devargs eth_da = *req_eth_da; @@ -2150,17 +2283,38 @@ mlx5_os_pci_probe_pf(struct mlx5_common_device *cdev, bd, ibv_list[ret]->name); ibv_match[nd++] = ibv_list[ret]; break; - } else { - /* Bonding device not found. */ - if (mlx5_get_pci_addr(ibv_list[ret]->ibdev_path, - &pci_addr)) - continue; - if (rte_pci_addr_cmp(&owner_pci, &pci_addr) != 0) - continue; - DRV_LOG(INFO, "PCI information matches for device \"%s\"", + } + mpesw = mlx5_device_mpesw_pci_match(ibv_list[ret], &owner_pci, nl_rdma); + if (mpesw >= 0) { + /* + * MPESW device detected. Only one matching IB device is allowed, + * so if any matches were found previously, fail gracefully. + */ + if (nd) { + DRV_LOG(ERR, + "PCI information matches MPESW device \"%s\", " + "but multiple matching PCI devices were found. " + "Probing failed.", + ibv_list[ret]->name); + rte_errno = ENOENT; + ret = -rte_errno; + goto exit; + } + DRV_LOG(INFO, + "PCI information matches MPESW device \"%s\"", ibv_list[ret]->name); ibv_match[nd++] = ibv_list[ret]; + break; } + /* Bonding or MPESW device was not found. */ + if (mlx5_get_pci_addr(ibv_list[ret]->ibdev_path, + &pci_addr)) + continue; + if (rte_pci_addr_cmp(&owner_pci, &pci_addr) != 0) + continue; + DRV_LOG(INFO, "PCI information matches for device \"%s\"", + ibv_list[ret]->name); + ibv_match[nd++] = ibv_list[ret]; } ibv_match[nd] = NULL; if (!nd) { @@ -2192,6 +2346,12 @@ mlx5_os_pci_probe_pf(struct mlx5_common_device *cdev, ret = -rte_errno; goto exit; } + if (mpesw >= 0 && !np) { + DRV_LOG(ERR, "Cannot get ports for MPESW device."); + rte_errno = ENOENT; + ret = -rte_errno; + goto exit; + } } /* Now we can determine the maximal amount of devices to be spawned. */ list = mlx5_malloc(MLX5_MEM_ZERO, @@ -2203,7 +2363,7 @@ mlx5_os_pci_probe_pf(struct mlx5_common_device *cdev, ret = -rte_errno; goto exit; } - if (bd >= 0 || np > 1) { + if (bd >= 0 || mpesw >= 0 || np > 1) { /* * Single IB device with multiple ports found, * it may be E-Switch master device and representors. @@ -2222,6 +2382,7 @@ mlx5_os_pci_probe_pf(struct mlx5_common_device *cdev, list[ns].pci_dev = pci_dev; list[ns].cdev = cdev; list[ns].pf_bond = bd; + list[ns].mpesw_port = MLX5_MPESW_PORT_INVALID; list[ns].ifindex = mlx5_nl_ifindex(nl_rdma, ibv_match[0]->name, i); @@ -2278,6 +2439,46 @@ mlx5_os_pci_probe_pf(struct mlx5_common_device *cdev, } continue; } + if (!ret && mpesw >= 0) { + switch (list[ns].info.name_type) { + case MLX5_PHYS_PORT_NAME_TYPE_UPLINK: + /* Owner port is treated as master port. */ + if (list[ns].info.port_name == mpesw) { + list[ns].info.master = 1; + list[ns].info.representor = 0; + } else { + list[ns].info.master = 0; + list[ns].info.representor = 1; + } + /* + * Ports of this type have uplink port index + * encoded in the name. This index is also a PF index. + */ + list[ns].info.pf_num = list[ns].info.port_name; + list[ns].mpesw_port = list[ns].info.port_name; + list[ns].info.mpesw_owner = mpesw; + ns++; + break; + case MLX5_PHYS_PORT_NAME_TYPE_PFHPF: + case MLX5_PHYS_PORT_NAME_TYPE_PFVF: + case MLX5_PHYS_PORT_NAME_TYPE_PFSF: + /* Only spawn representors related to the probed PF. */ + if (list[ns].info.pf_num == owner_id) { + /* + * Ports of this type have PF index encoded in name, + * which translate to the related uplink port index. + */ + list[ns].mpesw_port = list[ns].info.pf_num; + /* MPESW owner is also saved but not used now. */ + list[ns].info.mpesw_owner = mpesw; + ns++; + } + break; + default: + break; + } + continue; + } if (!ret && (list[ns].info.representor ^ list[ns].info.master)) ns++; @@ -2317,6 +2518,7 @@ mlx5_os_pci_probe_pf(struct mlx5_common_device *cdev, list[ns].pci_dev = pci_dev; list[ns].cdev = cdev; list[ns].pf_bond = -1; + list[ns].mpesw_port = MLX5_MPESW_PORT_INVALID; list[ns].ifindex = 0; if (nl_rdma >= 0) list[ns].ifindex = mlx5_nl_ifindex @@ -2597,7 +2799,10 @@ mlx5_os_auxiliary_probe(struct mlx5_common_device *cdev, struct mlx5_kvargs_ctrl *mkvlist) { struct rte_eth_devargs eth_da = { .nb_ports = 0 }; - struct mlx5_dev_spawn_data spawn = { .pf_bond = -1 }; + struct mlx5_dev_spawn_data spawn = { + .pf_bond = -1, + .mpesw_port = MLX5_MPESW_PORT_INVALID, + }; struct rte_device *dev = cdev->dev; struct rte_auxiliary_device *adev = RTE_DEV_TO_AUXILIARY(dev); struct rte_eth_dev *eth_dev; diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index a20acb6ca8..484c5eb3df 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -186,12 +186,15 @@ struct mlx5_dev_cap { char fw_ver[64]; /* Firmware version of this device. */ }; +#define MLX5_MPESW_PORT_INVALID (-1) + /** Data associated with devices to spawn. */ struct mlx5_dev_spawn_data { uint32_t ifindex; /**< Network interface index. */ uint32_t max_port; /**< Device maximal port index. */ uint32_t phys_port; /**< Device physical port index. */ int pf_bond; /**< bonding device PF index. < 0 - no bonding */ + int mpesw_port; /**< MPESW uplink index. Valid if mpesw_owner_port >= 0. */ struct mlx5_switch_info info; /**< Switch information. */ const char *phys_dev_name; /**< Name of physical device. */ struct rte_eth_dev *eth_dev; /**< Associated Ethernet device. */ @@ -200,6 +203,23 @@ struct mlx5_dev_spawn_data { struct mlx5_bond_info *bond_info; }; +/** + * Check if the port requested to be probed is MPESW physical device + * or a representor port. + * + * @param spawn + * Parameters of the probed port. + * + * @return + * True if the probed port is a physical device or representor in MPESW setup. + * False otherwise or MPESW was not configured. + */ +static inline bool +mlx5_is_probed_port_on_mpesw_device(struct mlx5_dev_spawn_data *spawn) +{ + return spawn->mpesw_port >= 0; +} + /** Data associated with socket messages. */ struct mlx5_flow_dump_req { uint32_t port_id; /**< There are plans in DPDK to extend port_id. */ @@ -1768,6 +1788,9 @@ struct mlx5_priv { uint32_t vport_meta_mask; /* Used for vport index field match mask. */ uint16_t representor_id; /* UINT16_MAX if not a representor. */ int32_t pf_bond; /* >=0, representor owner PF index in bonding. */ + int32_t mpesw_owner; /* >=0, representor owner PF index in MPESW. */ + int32_t mpesw_port; /* Related port index of MPESW device. < 0 - no MPESW. */ + bool mpesw_uplink; /* If true, port is an uplink port. */ unsigned int if_index; /* Associated kernel network device index. */ /* RX/TX queues. */ unsigned int rxqs_n; /* RX queues array size. */ @@ -1933,6 +1956,22 @@ mlx5_devx_obj_ops_en(struct mlx5_dev_ctx_shared *sh) sh->dev_cap.dest_tir); } +/** + * Check if the port is either MPESW physical device or a representor port. + * + * @param priv + * Pointer to port's private data. + * + * @return + * True if the port is a physical device or representor in MPESW setup. + * False otherwise or MPESW was not configured. + */ +static inline bool +mlx5_is_port_on_mpesw_device(struct mlx5_priv *priv) +{ + return priv->mpesw_port >= 0; +} + /* mlx5.c */ int mlx5_getenv_int(const char *); diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 4a85415ff3..cd84960b7e 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -395,18 +395,30 @@ uint16_t mlx5_representor_id_encode(const struct mlx5_switch_info *info, enum rte_eth_representor_type hpf_type) { - enum rte_eth_representor_type type = RTE_ETH_REPRESENTOR_VF; + enum rte_eth_representor_type type; uint16_t repr = info->port_name; - - if (info->representor == 0) - return UINT16_MAX; - if (info->name_type == MLX5_PHYS_PORT_NAME_TYPE_PFSF) + int32_t pf = info->pf_num; + + switch (info->name_type) { + case MLX5_PHYS_PORT_NAME_TYPE_UPLINK: + if (!info->representor) + return UINT16_MAX; + type = RTE_ETH_REPRESENTOR_PF; + pf = info->mpesw_owner; + break; + case MLX5_PHYS_PORT_NAME_TYPE_PFSF: type = RTE_ETH_REPRESENTOR_SF; - if (info->name_type == MLX5_PHYS_PORT_NAME_TYPE_PFHPF) { + break; + case MLX5_PHYS_PORT_NAME_TYPE_PFHPF: type = hpf_type; repr = UINT16_MAX; + break; + case MLX5_PHYS_PORT_NAME_TYPE_PFVF: + default: + type = RTE_ETH_REPRESENTOR_VF; + break; } - return MLX5_REPRESENTOR_ID(info->pf_num, type, repr); + return MLX5_REPRESENTOR_ID(pf, type, repr); } /** @@ -430,7 +442,7 @@ mlx5_representor_info_get(struct rte_eth_dev *dev, struct rte_eth_representor_info *info) { struct mlx5_priv *priv = dev->data->dev_private; - int n_type = 4; /* Representor types, VF, HPF@VF, SF and HPF@SF. */ + int n_type = 5; /* Representor types: PF, VF, HPF@VF, SF and HPF@SF. */ int n_pf = 2; /* Number of PFs. */ int i = 0, pf; int n_entries; @@ -443,7 +455,30 @@ mlx5_representor_info_get(struct rte_eth_dev *dev, n_entries = info->nb_ranges_alloc; info->controller = 0; - info->pf = priv->pf_bond >= 0 ? priv->pf_bond : 0; + info->pf = 0; + if (mlx5_is_port_on_mpesw_device(priv)) { + info->pf = priv->mpesw_port; + /* PF range, both ports will show the same information. */ + info->ranges[i].type = RTE_ETH_REPRESENTOR_PF; + info->ranges[i].controller = 0; + info->ranges[i].pf = priv->mpesw_owner + 1; + info->ranges[i].vf = 0; + /* + * The representor indexes should be the values set of "priv->mpesw_port". + * In the real case now, only 1 PF/UPLINK representor is supported. + * The port index will always be the value of "owner + 1". + */ + info->ranges[i].id_base = + MLX5_REPRESENTOR_ID(priv->mpesw_owner, info->ranges[i].type, + info->ranges[i].pf); + info->ranges[i].id_end = + MLX5_REPRESENTOR_ID(priv->mpesw_owner, info->ranges[i].type, + info->ranges[i].pf); + snprintf(info->ranges[i].name, sizeof(info->ranges[i].name), + "pf%d", info->ranges[i].pf); + i++; + } else if (priv->pf_bond >= 0) + info->pf = priv->pf_bond; for (pf = 0; pf < n_pf; ++pf) { /* VF range. */ info->ranges[i].type = RTE_ETH_REPRESENTOR_VF; diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index 977751394e..9700f0342a 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -1331,7 +1331,7 @@ flow_hw_represented_port_compile(struct rte_eth_dev *dev, if (!priv->master) return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, - "represented_port acton must" + "represented_port action must" " be used on proxy port"); if (m && !!m->port_id) { struct mlx5_priv *port_priv; @@ -9188,7 +9188,7 @@ flow_hw_set_port_info(struct rte_eth_dev *dev) info = &mlx5_flow_hw_port_infos[port_id]; info->regc_mask = priv->vport_meta_mask; info->regc_value = priv->vport_meta_tag; - info->is_wire = priv->master; + info->is_wire = mlx5_is_port_on_mpesw_device(priv) ? priv->mpesw_uplink : priv->master; } /* Clears vport tag and mask used for HWS rules. */ diff --git a/drivers/net/mlx5/mlx5_mac.c b/drivers/net/mlx5/mlx5_mac.c index b9d1e33ac3..22a756a52b 100644 --- a/drivers/net/mlx5/mlx5_mac.c +++ b/drivers/net/mlx5/mlx5_mac.c @@ -157,9 +157,13 @@ mlx5_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr) /* * Configuring the VF instead of its representor, - * need to skip the special case of HPF on BlueField. + * need to skip the special cases: + * - HPF on BlueField, + * - SF representors, + * - uplink ports when running in MPESW mode. */ - if (priv->representor && !mlx5_is_hpf(dev) && !mlx5_is_sf_repr(dev)) { + if (priv->representor && !mlx5_is_hpf(dev) && !mlx5_is_sf_repr(dev) && + !priv->mpesw_uplink) { DRV_LOG(DEBUG, "VF represented by port %u setting primary MAC address", dev->data->port_id); if (priv->pf_bond >= 0) { From patchwork Tue Oct 31 14:27:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 133675 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B272E43252; Tue, 31 Oct 2023 15:29:11 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AEB99427D7; Tue, 31 Oct 2023 15:28:36 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2045.outbound.protection.outlook.com [40.107.244.45]) by mails.dpdk.org (Postfix) with ESMTP id 9ADA340DDE for ; Tue, 31 Oct 2023 15:28:32 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CDXhq+l+zUFqNoAi/9kBF3q5xH1Dtxx63bEzkiuxh2xAz5j/PZkEjr+r0yfst7NuxrnNVyWxFHEpAnQuwkFZ91h+TapnOm+PPQTVz7WTh/WG3dgpitrb41gMVEL3+g9A0baQfoIntTruFDwrquPPpLq5k5Ak5JcS6InIXXbNp7NO4NjMiaMzTyhN6wYccXktbKs2F3Xl2txjLF9L21S0V801IhF8ZgLiMi96Emea8gkTd/I0iUNAsz1xkoSf73TY8CgeCeCr2NzrW/inNOx91rnK9ja10DZ9TZ2HW39iPPNoX+fsRvdi8WEiimiy76QVU6hgRiDlfFazlhu/FZDxFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=y3OHLAgipFBSsniTjKPMa1KUodxOVyq9Vfa9i7A/N6M=; b=C6/lqRSs4VikYQJ9lY5Ra6e2DfZhH+8WyBYPnR2ZieFAFEgjeaBhQ8/pjbVAHj1DGNYv059ZpiWoo9fz8qE09UTrKAYvrIjUzzk69MozsjuJCDoXYCsLrK1WvRrb7que/8P335ov+tL1ur4vIkANrk3jarfngap1wzJIFR4z8XeWsYxPxLPb2fTgn2g7VqyDlTCGsHBLEmk24p14G988wA8T8Qofbp18FLZxkOtOu1YR/hJDCIzYuzfKTL4LM/F01DLUlD+Gh1Ake/LNRjIA4oFPfKx2FACblZ1/xH29/FhMz0tOH0kyixVSSRw61AfvNfcgLx3AHAAUyqjYei/B6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y3OHLAgipFBSsniTjKPMa1KUodxOVyq9Vfa9i7A/N6M=; b=pWojvNXY1fkA3j2mTyix7OTX0KQ0joFM2myFSNrpr5SV2kC0PqK76XA3P8LcRjCSZOpWsq4vp+4zKubaEge4G3Mzx22A6Wv1C4f1xNXGneqQvdZCdg0SttyFs3a+uzCliZyBjUEUko3xvHbZddv/7C5i0WfPbGB//dkOFpQ04QX5g6GeGS9ruOJIRmynYx4Xt+2ZbzEUG1g97xGlwZ2SGfulr25Ivxtm0jgxwLVlo5lcs8P3/PszgzPL6Q0lrHpiTIbpFwee9EDrcJ5u6x4p4e+2CIKYfdcLzkBS/6UCAsWLkqXIkiIu4Q+IDsWDdGS22qmfolJBWJ0q1VjrkubYdQ== Received: from MW4PR04CA0242.namprd04.prod.outlook.com (2603:10b6:303:88::7) by CH3PR12MB8535.namprd12.prod.outlook.com (2603:10b6:610:160::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.29; Tue, 31 Oct 2023 14:28:30 +0000 Received: from MWH0EPF000971E6.namprd02.prod.outlook.com (2603:10b6:303:88:cafe::a) by MW4PR04CA0242.outlook.office365.com (2603:10b6:303:88::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.29 via Frontend Transport; Tue, 31 Oct 2023 14:28:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MWH0EPF000971E6.mail.protection.outlook.com (10.167.243.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Tue, 31 Oct 2023 14:28:30 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:28:08 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:28:06 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou CC: , Raslan Darawsheh Subject: [PATCH 7/8] net/mlx5: sort port spawn data with uplink ports first Date: Tue, 31 Oct 2023 16:27:32 +0200 Message-ID: <20231031142733.2009166-8-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231031142733.2009166-1-dsosnowski@nvidia.com> References: <20231031142733.2009166-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E6:EE_|CH3PR12MB8535:EE_ X-MS-Office365-Filtering-Correlation-Id: a9b1553b-42bd-4928-172c-08dbda1da7a1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oks3NSJKjvR7iawyxLsKxsE52r/R+AUpvw2UJgFvOUOkpFda9aqSyBg7nLmLiiwE1T/SWO9KkMl0XjqNK0cAZbxS+SkhwLpmlNMCs0XURogvnLsi3ovNKKkfTIu1Zh7xjNNuHQjEUPAyjzMND5QhNa24EBeFtyaqA2H5Ce4KF4zpgYr+EXcbq480rfTxXQsDGfd/Pg3D5FInXouZcWgofdIk050OigXjEGZd9MpZ+CZFNEISOkUuXXtngVmw+cCiSRbDxB2L6g6gdxNe+QD/LdrkO1XDIc+Eo4zPGZfzlWU+XlKHqPn0mJhm5aHzpGuUPwjrHGL5JaQ6PpdlSvMm1zav5Be5IG9JDV36E0Pig9UyekPwpUL0o4AJQLcclclTwmZe8O6H1QGv+ljRRAcPRLICCgvD0ESpZfdBIaYThIKq5H4ssKDFA/4ToAyj2abBY3Ulqop5cSwJIiQ2Pwr0oRXWIMMh08W523HkLF4IhuyREbb8r74zny8NzfRCJzp6KH4D1ZA3pUP4u+jkiJoxFXh5EoDKiYBuJJSDV9jM4OEW7vLhGANtUWJrO002t4KKLyTdYpEw0RqGFfKwCM0ixUWnNlpexOxCdDjct9dP7y57N5N8TPaD1mxYqAUnc5+v/RiuH13ua1MullNaIVqBA6O+YHeEiGmTTEA/qoztazUd/cJnB+zt+N9zhlt71m5syj+bP+KjKZ2kubG422QUX+hdDDWz8SMTbRFQoYYTp1JYY1camMgBDyyCfu5l2w9t X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(396003)(376002)(136003)(39860400002)(230922051799003)(1800799009)(82310400011)(451199024)(186009)(64100799003)(40470700004)(36840700001)(46966006)(2906002)(40460700003)(40480700001)(478600001)(55016003)(6666004)(2616005)(107886003)(1076003)(16526019)(6286002)(26005)(426003)(336012)(82740400003)(47076005)(7636003)(356005)(36756003)(83380400001)(86362001)(36860700001)(7696005)(54906003)(110136005)(41300700001)(6636002)(316002)(70586007)(70206006)(5660300002)(8936002)(8676002)(4326008); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 14:28:30.2753 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a9b1553b-42bd-4928-172c-08dbda1da7a1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E6.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8535 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch changes the behavior of the comparator used to sort mlx5_dev_spawn_data structures, to put them in a more user friendly order Before this patch, ports were sorted assuming there is only a single master port. It resulted in an order where master port first comes second, then representors in ascending order of IDs. This approach however is not desirable with devices configured for Multiport E-Switch, since uplink ports which do not correspond to the owning PCI device are representors as well and they will be mixed with VF/SF representors. To change that, this patch amends the comparator to force uplink ports to be first. If there are many uplink ports, the master port will come first and the rest will be sorted by port index. Signed-off-by: Dariusz Sosnowski Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 8ddf38288e..07f31de5ae 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1797,9 +1797,15 @@ mlx5_dev_spawn_data_cmp(const void *a, const void *b) &((const struct mlx5_dev_spawn_data *)a)->info; const struct mlx5_switch_info *si_b = &((const struct mlx5_dev_spawn_data *)b)->info; + int uplink_a = si_a->name_type == MLX5_PHYS_PORT_NAME_TYPE_UPLINK; + int uplink_b = si_b->name_type == MLX5_PHYS_PORT_NAME_TYPE_UPLINK; int ret; - /* Master device first. */ + /* Uplink ports first. */ + ret = uplink_b - uplink_a; + if (ret) + return ret; + /* Then master devices. */ ret = si_b->master - si_a->master; if (ret) return ret; From patchwork Tue Oct 31 14:27:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 133674 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 336C643252; Tue, 31 Oct 2023 15:29:04 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 918CE40EF0; Tue, 31 Oct 2023 15:28:35 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2047.outbound.protection.outlook.com [40.107.94.47]) by mails.dpdk.org (Postfix) with ESMTP id 7737A40EF0 for ; Tue, 31 Oct 2023 15:28:33 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Nn6h1OILt92G3bZRQYkycyRpLKQAhC1Gb7e4kL3xT6QgHMJ4dmLwCPEyj31XwcOxuSSLnpxiY2vQSg9kSlfr775qnIQLUyWZQKkDYU4C5xB2y7kZqEqD4EBN7hXuribzwykWfqgqVcb7p/PNen1kXwxlsaXm6VrLHrGnSQrkdZyIL2/aE0XKr+a31HZDfwjLr/JZvy5L5ehTpKkuYs65J8IKbIE9EhQuoLcwQs71nfrf+oC/8XfcAeilDZ6lLc5AICmLWWmbYxmLAKA1iiUux2SO5GnDRcJCZZS1QF4PBlMwYqtRPCR4aE1JSncZ6UmdiezFI5FOgt6ChE3Zck6E1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YbYFdSQjNoOdkopzZMZUryu4PtrLu7YPd/FXl9McxuI=; b=Pf3Dq11XXhB1xoLxVUlMQij9LSOXOfKrTVbZlXomgSCeFboazNM+ofv/FjmYhaRs9NTl6K/AuhVt1pIi1oNcHrCHmWeQOi4z7AkKJo/Q4PTiscte/CLkv2WdjNc89nh75U1uzhcBmyiYhp01MFlnTReVEE+Wc0cQSjwXXCb7U8c/10TzRPrWjvEKeKh9qpO1EDKoxbgiE6WF2Xv8H9L/EYayD2S1rAPNd6pjTe9fQ113zBsbqXBj2LcCLmByOD6eKTUO+JxV7190s40dBWYxViWInIteTdm19RaJPbmnTogwCOuEyar5cVcRu0h3u0oMS4tEy4GVwoML12Bn7VdsAA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YbYFdSQjNoOdkopzZMZUryu4PtrLu7YPd/FXl9McxuI=; b=r9N/onpDgwrKGNGT/hZRxr7jJtq9gX+9jUeHl/EsXDOk86R4tIia43WG/0gVnqbsodF+m9Sv9RzN+red6FLYJoBTFmFSLkGqvLI4gN55JfRau825WwpM8KgfJRMgryHJ3wY3X+m+hFAJt/QwR6o6WCCx1ZexdXyADF13mfVzzAZRIR7EtqW9yirsJQRU//nnQ5O7sQ+gHsOhfkteiOZZKcKEAtwunm2V6+ed0lxLAO9Xr+anqKR2wxbZrAbkyWf/SlN9vkAzky+azDcDwzn7kxkegU7TfhZU8c+w1MsR/28x3ftgFDBT4rlBcITBK0tQ7ASNIoa2JUDxkWEyvan6GA== Received: from MW4PR04CA0262.namprd04.prod.outlook.com (2603:10b6:303:88::27) by CH0PR12MB8551.namprd12.prod.outlook.com (2603:10b6:610:186::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.28; Tue, 31 Oct 2023 14:28:31 +0000 Received: from MWH0EPF000971E6.namprd02.prod.outlook.com (2603:10b6:303:88:cafe::95) by MW4PR04CA0262.outlook.office365.com (2603:10b6:303:88::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.29 via Frontend Transport; Tue, 31 Oct 2023 14:28:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MWH0EPF000971E6.mail.protection.outlook.com (10.167.243.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Tue, 31 Oct 2023 14:28:31 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:28:11 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 31 Oct 2023 07:28:09 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou CC: , Raslan Darawsheh , Bing Zhao Subject: [PATCH 8/8] net/mlx5: add support for vport match selection Date: Tue, 31 Oct 2023 16:27:33 +0200 Message-ID: <20231031142733.2009166-9-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231031142733.2009166-1-dsosnowski@nvidia.com> References: <20231031142733.2009166-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E6:EE_|CH0PR12MB8551:EE_ X-MS-Office365-Filtering-Correlation-Id: f8225162-1a4b-4818-118b-08dbda1da81d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Sj/snNbVTiYc8NfirdSRcyb3hl1JOZR6msJEPORKG+sQhgWLe2CFfTyk+FIOnHSL0G9wraRF1qg/tYJPwNDyYE6KD3aZ0Bcxn+HH+qgYoNqC7HbGLlm9mMM4bKciG3CAQCr++Uz1AYACdOUBcCMZH4mBBRzYW85RyZz8TBZpkzFiEC7gbOlHa85rYLdEjN6JHH9PVOTVNInZPvdx9RAAu3BFn/eMBmzb56ZYeZvrA8v6SSh5P5240jkaKUxhF0jzR0NHBFCnFNNdpplk5XkaAxEWQOODavq2/AnvZtnghUIyNYnNyvfzSZvK6WQuWSiRP7iaHTftrAi8FKvOWsCIxEHKlEjxIcvIVUF3Z3WrTU7IfwK7YzfzV4Jn0/wccThOS0AGU3BQCUd/jzgPs9jarVRVi0v2OJXVavibUgI07pyODU34iiNsJTM5ZyLO1j2iJih3CMnGhN4Xov8WeoSdXmP6MWP2VjF/AULsewhteEXOZuNQYCIlJhjzi9GVPGv/wqxP7vlxdmUT9/+XyhrcwHemELAi6sDY5MPsY98GjjFEoK8v+vHtXswLmHCG8aZ05lurcMJBkNxy/dxuQMRzDQXZ59RqyCO9WZUoLtZPzGgmDlFmxET1Y42DQJc2AfCYdHAQftzKOXRQXQHTEe6OAdb1xmQOjb3IKWaWhFRz864cnbzMYS9wFV4oK74SHmFOz3PN11yp/nBN2O4zmsTkTHslLuZAKAGcKj675fbMIvohfe7e5K+k83tVVvSiuIyyhorOMY+tmnCdnTH1yWkioEE5lw+bmgg7ITNYM5ryeQ9mJjdilbPBxqp2IWqqki24 X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(376002)(346002)(136003)(396003)(39860400002)(230173577357003)(230273577357003)(230922051799003)(64100799003)(1800799009)(82310400011)(186009)(451199024)(36840700001)(46966006)(40470700004)(478600001)(7696005)(110136005)(6666004)(70206006)(70586007)(336012)(426003)(16526019)(6286002)(26005)(1076003)(107886003)(41300700001)(2616005)(2906002)(86362001)(5660300002)(54906003)(6636002)(4326008)(8676002)(8936002)(316002)(82740400003)(36756003)(36860700001)(83380400001)(47076005)(356005)(7636003)(40480700001)(40460700003)(55016003)(309714004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 14:28:31.0878 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f8225162-1a4b-4818-118b-08dbda1da81d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E6.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB8551 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Bing Zhao A new devarg "vport_match" is introduced for the application to use. If set to 1, then matching using REPRESENTED_PORT items on group 0 will be forced to use "misc.source_port", instead of matching on the vport metadata in HWS mode. It allows the user to match on the traffic from E-Switch manager. A new devarg "vport_match" is introduced for the application to use. This enables the force matching on "misc.source_port" for item REPRESENTED_PORT on group 0, instead of matching on the metadata REG_C_0 bits in HWS mode. It will allow the user to match on the traffic from E-Switch manager. By default, this is set to 0. When enable it with 1, the default FDB jump rule should be disabled by set "fdb_def_rule_en=0". Signed-off-by: Bing Zhao Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 16 ++++++++++++++++ drivers/net/mlx5/mlx5.c | 17 +++++++++++++++++ drivers/net/mlx5/mlx5.h | 2 ++ drivers/net/mlx5/mlx5_flow_dv.c | 2 +- drivers/net/mlx5/mlx5_trigger.c | 5 ++++- 5 files changed, 40 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 584f592433..8c65f16db8 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1369,6 +1369,22 @@ for an additional list of options shared with other mlx5 drivers. By default, the PMD will set this value to 1. +- ``vport_match`` parameter [int] + + Controls the underlying matching mechanism for REPRESENTED_PORT items when they are used for + flow rules in E-Switch root flow table. + + If set to 1, then ``source_vport`` matching is used. This allows applications to match whole + traffic coming from the application by using REPRESENTED_PORT item with ``port_id == UINT16_MAX``. + As a side effect, flow rules in root flow table will not be able match physical ports explicitly, + when running on Multiport E-Switch. + Matching in non-root flow tables (group bigger than 1) is not affected. + + If set to 0, then ``vport_metadata`` matching is used. This is the default mechanism. + + By default, the PMD will set this value to 0. Setting ``vport_match`` to 1 requires that + ``fdb_def_rule_en`` is set to 0, so that E-Switch root flow table is exposed to the application. + Sub-Function ------------ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index f929d6547c..c275cdfee8 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -184,6 +184,9 @@ /* Device parameter to control representor matching in ingress/egress flows with HWS. */ #define MLX5_REPR_MATCHING_EN "repr_matching_en" +/* Representor matching field selection: 0 - meta_vport, 1 - misc.vport */ +#define MLX5_HWS_ROOT_VPORT_MATCH "vport_match" + /* Shared memory between primary and secondary processes. */ struct mlx5_shared_data *mlx5_shared_data; @@ -1425,6 +1428,8 @@ mlx5_dev_args_check_handler(const char *key, const char *val, void *opaque) config->cnt_svc.cycle_time = tmp; } else if (strcmp(MLX5_REPR_MATCHING_EN, key) == 0) { config->repr_matching = !!tmp; + } else if (strcmp(MLX5_HWS_ROOT_VPORT_MATCH, key) == 0) { + config->vport_match = !!tmp; } return 0; } @@ -1464,6 +1469,7 @@ mlx5_shared_dev_ctx_args_config(struct mlx5_dev_ctx_shared *sh, MLX5_HWS_CNT_SERVICE_CORE, MLX5_HWS_CNT_CYCLE_TIME, MLX5_REPR_MATCHING_EN, + MLX5_HWS_ROOT_VPORT_MATCH, NULL, }; int ret = 0; @@ -1522,6 +1528,11 @@ mlx5_shared_dev_ctx_args_config(struct mlx5_dev_ctx_shared *sh, rte_errno = ENODEV; return -rte_errno; } + if (config->dv_flow_en == 2 && config->fdb_def_rule && config->vport_match) { + DRV_LOG(DEBUG, "vport_match=1 is incompatible with FDB default rule " + "(fdb_def_rule-en=1). Setting vport_match=0."); + config->vport_match = 0; + } if (!config->tx_pp && config->tx_skew && !sh->cdev->config.hca_attr.wait_on_time) { DRV_LOG(WARNING, @@ -1562,6 +1573,7 @@ mlx5_shared_dev_ctx_args_config(struct mlx5_dev_ctx_shared *sh, config->allow_duplicate_pattern); DRV_LOG(DEBUG, "\"fdb_def_rule_en\" is %u.", config->fdb_def_rule); DRV_LOG(DEBUG, "\"repr_matching_en\" is %u.", config->repr_matching); + DRV_LOG(DEBUG, "\"vport_match\" is %u.", config->vport_match); return 0; } @@ -3003,6 +3015,11 @@ mlx5_probe_again_args_validate(struct mlx5_common_device *cdev, sh->ibdev_name); goto error; } + if (sh->config.vport_match ^ config->vport_match) { + DRV_LOG(ERR, "\"vport_match\" configuration mismatch for shared %s context.", + sh->ibdev_name); + goto error; + } mlx5_free(config); return 0; error: diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 484c5eb3df..5299b1321a 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -352,6 +352,7 @@ struct mlx5_sh_config { /* Allow/Prevent the duplicate rules pattern. */ uint32_t fdb_def_rule:1; /* Create FDB default jump rule */ uint32_t repr_matching:1; /* Enable implicit vport matching in HWS FDB. */ + uint32_t vport_match:1; /* Root table representor matching field selection. */ }; /* Structure for VF VLAN workaround. */ @@ -1782,6 +1783,7 @@ struct mlx5_priv { uint32_t mark_enabled:1; /* If mark action is enabled on rxqs. */ uint32_t num_lag_ports:4; /* Number of ports can be bonded. */ uint32_t tunnel_enabled:1; /* If tunnel offloading is enabled on rxqs. */ + uint32_t vport_match:1; /* vport match field. */ uint16_t domain_id; /* Switch domain identifier. */ uint16_t vport_id; /* Associated VF vport index (if any). */ uint32_t vport_meta_tag; /* Used for vport index match ove VF LAG. */ diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index a39b4600e6..5b5716692c 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -10594,7 +10594,7 @@ flow_dv_translate_item_represented_port(struct rte_eth_dev *dev, void *key, * Kernel can use either misc.source_port or half of C0 metadata * register. */ - if (priv->vport_meta_mask) { + if (priv->vport_meta_mask && !priv->vport_match) { /* * Provide the hint for SW steering library * to insert the flow into ingress domain and diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index 7bdb897612..d28cbe1dfd 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -1515,7 +1515,10 @@ mlx5_traffic_enable_hws(struct rte_eth_dev *dev) goto error; } } else { - DRV_LOG(INFO, "port %u FDB default rule is disabled", dev->data->port_id); + DRV_LOG(INFO, "port %u FDB default rule is disabled with vport_match %u", + dev->data->port_id, config->vport_match); + /* vport_match is only interesting in no default FDB rule mode. */ + priv->vport_match = config->vport_match; } if (priv->isolated) return 0;