From patchwork Thu Oct 14 02:58:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Liu X-Patchwork-Id: 101520 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 59D1FA0C4B; Thu, 14 Oct 2021 04:58:24 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 57A9341134; Thu, 14 Oct 2021 04:58:21 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2050.outbound.protection.outlook.com [40.107.223.50]) by mails.dpdk.org (Postfix) with ESMTP id C00D94112E for ; Thu, 14 Oct 2021 04:58:20 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dNIT7++wjQcxQ8OF8EMv9m0kgq8dE21XaYI6mgRrq+oGoL4B7Pj86D9sGwKQDA7U5pAdDPSsa7XCtCl6BznzsQKuhNUHOMLMN2Ef3hEtz5ZCV+rPuMo5+IOz6jKJB146c7UFmxiFKQKsrrc3sqi0DNSpzOhypUDYdHShTtPU0eg6Ho6bFTFT/S79Ie6qOQEA0bLeHH0fdj/So1Nt1WmqFrz9aJEs3N+7wAxMqnMvVk8CytC+8gp7BnVFigqWluc70BzBewhKBhQI2cW6Ofr7Gwx7P6NEhMozXJscfgsVs/Sva+GjcueZb1M4kwuaTGPSKVo2Vh0f/b5ZUCWRUdKZ1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HeStO2a21BwaQn3Bz9o7trAiTpnll9djHZzGKS2D8yo=; b=I0iD8yT4yd6AJXcDZaXhOADuttdlOk3iC9PsDHY123+GuTJF5dV4idFRYIPYT/uOSuDebf/qqQzRlX9Kmgr3HbTuSfOMpGcHZTzsd0UCHwdJxsNPbkbSgJ2CEZti4vsyrnE93cRn5FDXEJS22jf/sNciW+BblHFQKqlnua+mn2K5wX6fiDl9aOeXrTT0z/z25I3PtBCTCzyUQHij5zwwO4a4S892KxDLKiL8kL/EQWmSMyArx3H/an7jXqPyVYPSuJ7s2QK6v8rrE98M/rlrB04raMWv4+W9TH/jnzHja6K6eijRH6u9oTmLLPdBLP4IkD6t/XWOh203YYow0czN+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HeStO2a21BwaQn3Bz9o7trAiTpnll9djHZzGKS2D8yo=; b=ulZtqI4IEVMIWj5y5aNtIXlcHh8J8jwi7HljUytq9GrD7hZdWPHeD22qkuslsucH1jFKbs3R865ZxWX3uvG8zt5oC9iT5GI5NAcy/vLjLcELhmrnB7+xU89GRw+9USGhTJG4bb+cG4k3/vMglYzNf3JAj8GQ94Q9T0057chb8/qpDCHYefyxqaM7XrAgyMPrbvZeMug4HZ/pL9wnxkRmGiFwfQilPWQUF5PsH1hbAWSmxvJW19X8licS2YssRr2lP0v2mKrbTXKd7uvKeyWfNwXnw8Itbl5Ws9W6wsUkIFaifbAyfOgXD6gExaTQnW6SMmzgegh4LKMsCUmgnqqmvA== Received: from CO2PR04CA0176.namprd04.prod.outlook.com (2603:10b6:104:4::30) by MW3PR12MB4427.namprd12.prod.outlook.com (2603:10b6:303:52::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.22; Thu, 14 Oct 2021 02:58:18 +0000 Received: from CO1NAM11FT051.eop-nam11.prod.protection.outlook.com (2603:10b6:104:4:cafe::b5) by CO2PR04CA0176.outlook.office365.com (2603:10b6:104:4::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.15 via Frontend Transport; Thu, 14 Oct 2021 02:58:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT051.mail.protection.outlook.com (10.13.174.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4608.15 via Frontend Transport; Thu, 14 Oct 2021 02:58:18 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 14 Oct 2021 02:58:16 +0000 From: Rongwei Liu To: , , , CC: , Date: Thu, 14 Oct 2021 05:58:00 +0300 Message-ID: <20211014025801.3055244-2-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211014025801.3055244-1-rongweil@nvidia.com> References: <2308701.4c83qqXp4g@thomas> <20211014025801.3055244-1-rongweil@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 45c37d77-e477-4be4-6e92-08d98ebe7994 X-MS-TrafficTypeDiagnostic: MW3PR12MB4427: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:595; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MdnpxppIz2K5W6ptKi1LCkWIqDNU+LAdmxpmi6MWMabem8RoCzaDaPRGQ4dOfSHUNvO7yFS+TYzgJzGkokCIju6BHmFNx3R2O/WOAv8QdCJIYyUkAssQ6VT4OeOXfzVIcnc9I+c2vIOWuYERPnnwzckZrk9HrA50juWQjeVdRmdX3uO5m5CfnH5Ohv5ZH6pSJOHcpTOKoQ/2hWhTjIBw3ujMjRwuYOe6ZKseXtrezGFEocEPAtQfRasH3YNKOPRQ1D0caW0U9f86I0xQAymxC4SnSEuabQFHTSrIDopbLZ5fwCGR2jKoLaqIpNR7gkhipEu3cx6hpckTT46G4ZFknZjplk6/4sljXKy5pzxQRK0gPnlVVv/jqudx8dwFMeIJfQSN86GNg/uvd1XTO/nbhM6dAK0EqsviFJoe8UjN7S+19dDElfu7v/JKNB2OBA7Xd3ITYQWXD5TqIRSKYqxN0rjHw4SZ2oe9eXcbRpCgFWVlNLQgAqPzBeRhfR0QmaWK+DPXXUF/ZVMd89OOBlkm5gzfSBmSD6ZI7Wvel6KkRm4KFbqwdD/Gfuiy04Q1yxn4arL/0crxTweEb0QaBkHtle6ZTnx6TVx2ieEx8aBKIlVNfmEDfzaCHFvhiinXZgjVj/19KoB+dOoPVvafxO10e57qbcFltOEMxQnhL20xlVKALnNhU2uLkAgSV/iSm0xiYdJcXiMO810FPoAQ6Wp5iA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(8676002)(6666004)(86362001)(47076005)(508600001)(36756003)(316002)(426003)(4326008)(83380400001)(55016002)(1076003)(36860700001)(110136005)(82310400003)(336012)(186003)(8936002)(6286002)(2616005)(70206006)(5660300002)(70586007)(16526019)(107886003)(7696005)(54906003)(26005)(356005)(7636003)(2906002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Oct 2021 02:58:18.3217 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 45c37d77-e477-4be4-6e92-08d98ebe7994 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4427 Subject: [dpdk-dev] [PATCH v4 1/2] common/mlx5: support pcie device guid query X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" sysfs entry "phys_switch_id" holds each PCIe device' guid. The devices which reside in the same physical NIC should have the same guid. Signed-off-by: Rongwei Liu Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/mlx5_common_os.c | 41 ++++++++++++++++++++++ drivers/common/mlx5/linux/mlx5_common_os.h | 19 ++++++++++ 2 files changed, 60 insertions(+) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 9e0c823c97..8b3ee2baea 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -2,6 +2,7 @@ * Copyright 2020 Mellanox Technologies, Ltd */ +#include #include #include #include @@ -428,3 +429,43 @@ mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) mlx5_glue->free_device_list(ibv_list); return ibv_match; } + +int +mlx5_get_device_guid(const struct rte_pci_addr *dev, uint8_t *guid, size_t len) +{ + char tmp[512]; + char cur_ifname[IF_NAMESIZE + 1]; + FILE *id_file; + DIR *dir; + struct dirent *ptr; + int ret; + + if (guid == NULL || len < sizeof(u_int64_t) + 1) + return -1; + memset(guid, 0, len); + snprintf(tmp, sizeof(tmp), "/sys/bus/pci/devices/%04x:%02x:%02x.%x/net", + dev->domain, dev->bus, dev->devid, dev->function); + dir = opendir(tmp); + if (dir == NULL) + return -1; + /* Traverse to identify PF interface */ + do { + ptr = readdir(dir); + if (ptr == NULL || ptr->d_type != DT_DIR) { + closedir(dir); + return -1; + } + } while (strchr(ptr->d_name, '.') || strchr(ptr->d_name, '_') || + strchr(ptr->d_name, 'v')); + snprintf(cur_ifname, sizeof(cur_ifname), "%s", ptr->d_name); + closedir(dir); + snprintf(tmp + strlen(tmp), sizeof(tmp) - strlen(tmp), + "/%s/phys_switch_id", cur_ifname); + /* Older OFED like 5.3 doesn't support read */ + id_file = fopen(tmp, "r"); + if (!id_file) + return 0; + ret = fscanf(id_file, "%16s", guid); + fclose(id_file); + return ret; +} diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h index c3202b6786..3cdea75373 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.h +++ b/drivers/common/mlx5/linux/mlx5_common_os.h @@ -296,4 +296,23 @@ __rte_internal struct ibv_device * mlx5_os_get_ibv_dev(const struct rte_device *dev); +/** + * This is used to query system_image_guid as describing in PRM. + * + * @param dev[in] + * Pointer to a device instance as PCIe id. + * @param guid[out] + * Pointer to the buffer to hold device guid. + * Guid is uint64_t and corresponding to 17 bytes string. + * @param len[in] + * Guid buffer length, 17 bytes at least. + * + * @return + * -1 if internal failure. + * 0 if OFED doesn't support. + * >0 if success. + */ +int +mlx5_get_device_guid(const struct rte_pci_addr *dev, uint8_t *guid, size_t len); + #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */ From patchwork Thu Oct 14 02:58:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Liu X-Patchwork-Id: 101521 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 45120A0C4B; Thu, 14 Oct 2021 04:58:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 728524115A; Thu, 14 Oct 2021 04:58:24 +0200 (CEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2074.outbound.protection.outlook.com [40.107.220.74]) by mails.dpdk.org (Postfix) with ESMTP id EDA3541159 for ; Thu, 14 Oct 2021 04:58:22 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jXDj6XOexIFxIVO8h1UUxtNqZmAeaihmvckhspa+BoXMseGG5vNMjDCgmswMYSdvLakwvS8HzulAHTBAGSy0oRl+IKFuvtWh4XfpWa9JZyiGWKLIETzCP1hzTRDe2A2VwAT17hgcfUWXZQwRAAjuxie9ZCvUmVpGNk3EAsGgKu1MJ9lgX/g+Rza7bZHzkia3h2dBfiNbjckcSoxqUiZvz5EUsljzu6ib931eM/OYPbMi+trihvKR/3u99oqh1ov34OrUU3lPN7rKSdECpboGlvE76TQjSaMIo1gU3qc1jOUeE9czPes77Rib5PyRZjyd4qm8qtwEU+EqqCclsHWOYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=emHE+bisAqubvi39CKPiy/WMU0HyxduBC7wuI3639q8=; b=Y9V8g4b6osnK43J2q8nqKhQG0v1cX+TmK8NW3lwpHqj69ASbWDvDMMA1+GhImrxhHpKYYtKxBDn/pU9r/YzATENiykyU032C/xEl/yFxfh8/7NffoBXAV+a/vnWKX4rL62KKPVU3+UGlf9vhgWEoOPVkbj+t0aW2Rcv+pOomjLV1d5m4NbGX+o2KbaC9vljoYufV1iehsaGItyO5AwiJLosNOba0WnzVu8GAb5096c2Eh6UYv92zhlrILrITXf3R2vCdz+XbHYtoCe9haCK24XjUhpw4t9yW4WUpTxFLvXnGlt5rm1pG3pRPv68NGwZwupKgLw2rmINKS8752ctznw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=emHE+bisAqubvi39CKPiy/WMU0HyxduBC7wuI3639q8=; b=dmW8d/5I9kZz+msCIfbdPV0y0H1//3yFMwkMI2jvIbk8kC86zwQ8JwwjYnEZKXHECsN+V3ozqP7HVQhljCJ9RGM/qhignbI5qGil9p9drkbjKcBpMiHpFfeXAF8wlGoIW4j2bK7gBRTvuqQGmgUKv4mijB3FoRj7zr7utk0SWn15dXE51j3wD9x29LMM+CwHlgy95mESG/b0dAWUSrX7Yv5Jjbe4c2znIDohfZKHGWSnRLleiZF45rQdaPFUVJoHkJIGQRqqdJyt8b5HS/K59VxIv/6WpIRGm3YFWdX/yoBIv5H/n/J2XxlEGrVJ3jbn34lvEWd/4YNY/Dme2Mfqvw== Received: from MW2PR2101CA0012.namprd21.prod.outlook.com (2603:10b6:302:1::25) by DM6PR12MB4313.namprd12.prod.outlook.com (2603:10b6:5:21e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.20; Thu, 14 Oct 2021 02:58:21 +0000 Received: from CO1NAM11FT047.eop-nam11.prod.protection.outlook.com (2603:10b6:302:1:cafe::9c) by MW2PR2101CA0012.outlook.office365.com (2603:10b6:302:1::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.3 via Frontend Transport; Thu, 14 Oct 2021 02:58:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT047.mail.protection.outlook.com (10.13.174.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4608.15 via Frontend Transport; Thu, 14 Oct 2021 02:58:20 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 14 Oct 2021 02:58:18 +0000 From: Rongwei Liu To: , , , CC: , Date: Thu, 14 Oct 2021 05:58:01 +0300 Message-ID: <20211014025801.3055244-3-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211014025801.3055244-1-rongweil@nvidia.com> References: <2308701.4c83qqXp4g@thomas> <20211014025801.3055244-1-rongweil@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8f01b1b5-e0f5-45c1-5b10-08d98ebe7acc X-MS-TrafficTypeDiagnostic: DM6PR12MB4313: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1013; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WJNetvzQRPM5kGPcr6KtYmHvFswlgNVQHJWaeChucaXJndMYAGHuTToo+2RyoRdBuiXp6tAUGiN421uTpe4Nu+iDX+BslLxqZg80e4pQ8bJRWqMcQ/wg81dPctmAFJT3wNUQF7JRGrf0WAb2cgMnsEttNzjm26/osTH1E6EHH912ntRcWfeEk2NmwP8fB12euSOC9Ay3s9jYJeJaUo0NzYxbO1IE81A1OQyS5LTXg/7L/FVym5oo/PxO4j6t+bGbuqn2CALGxL/IV052Yyh6AVd9raeMJhIgPdkiRz3GN0MjjOB6/U9xAXMJMQfCsW/Opd/bND3ZztwIR87rOEOVqWTdI+TKT0JWTXoh9ViQAePG6th39OsuQycu42bLPjyh2ZWJbtdy/XzdaOUtUsxZY0FognvASU4O11xuljeAAVL+wqsnWaS02iDdInTHsc1BEQlOUnBN2jezXKKYQqEVrjR2eGJ/qUWk+kpbPMgZD0Nk7q7Tf1YPpKx720ieo7MvQitEkszqGO+/RFzbuV5jExoetZg9d0RhsEU5tVCIm1jDfnEO3X9W8Tae/3HpvrpYY+7LLsP0hNL6XhlTpH9Wrd6y83BH9KrouUY3nxyPhaqLwieJkPWI1yyTuoHcl1dgOKQGnOP5Fp56DHtzxbZY1btGVAYWDmdhV7eYJj6X/sgOrB6rZ188ppPC3vQIKf0IeoukUnzoU8b25hp+7Lr/Xg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(2616005)(426003)(26005)(2906002)(316002)(336012)(5660300002)(107886003)(508600001)(47076005)(1076003)(110136005)(8936002)(83380400001)(6286002)(356005)(16526019)(70586007)(70206006)(82310400003)(36756003)(86362001)(7696005)(7636003)(4326008)(186003)(54906003)(55016002)(36860700001)(6666004)(8676002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Oct 2021 02:58:20.3629 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8f01b1b5-e0f5-45c1-5b10-08d98ebe7acc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT047.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4313 Subject: [dpdk-dev] [PATCH v4 2/2] net/mlx5: support socket direct mode bonding X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In socket direct mode, it's possible to bind any two (maybe four in future) PCIe devices with IDs like xxxx:xx:xx.x and yyyy:yy:yy.y. Bonding member interfaces are unnecessary to have the same PCIe domain/bus/device ID anymore, Kernel driver uses "system_image_guid" to identify if devices can be bound together or not. Sysfs "phys_switch_id" is used to get "system_image_guid" of each network interface. OFED 5.4+ is required to support "phys_switch_id". Signed-off-by: Rongwei Liu Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 4 +++ doc/guides/rel_notes/release_21_11.rst | 4 +++ drivers/net/mlx5/linux/mlx5_os.c | 43 ++++++++++++++++++++------ 3 files changed, 42 insertions(+), 9 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index bae73f42d8..b58236e00a 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -464,6 +464,10 @@ Limitations - In order to achieve best insertion rate, application should manage the flows per lcore. - Better to disable memory reclaim by setting ``reclaim_mem_mode`` to 0 to accelerate the flow object allocation and release with cache. +- Bonding under socket direct mode + + - Needs OFED 5.4+. + Statistics ---------- diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index dfc2cbdeed..2a6cc765c2 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -106,6 +106,10 @@ New Features * Added DES-CBC, AES-XCBC-MAC, AES-CMAC and non-HMAC algo support. * Added PDCP short MAC-I support. +* **Updated Mellanox mlx5 driver.** + + * Added socket direct mode bonding support. + * **Updated NXP dpaa2_sec crypto PMD.** * Added PDCP short MAC-I support. diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 3746057673..1d57b934fc 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -2008,6 +2008,8 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, FILE *bond_file = NULL, *file; int pf = -1; int ret; + uint8_t cur_guid[32] = {0}; + uint8_t guid[32] = {0}; /* * Try to get master device name. If something goes @@ -2022,6 +2024,8 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, np = mlx5_nl_portnum(nl_rdma, ibv_dev->name); if (!np) return -1; + if (mlx5_get_device_guid(pci_dev, cur_guid, sizeof(cur_guid)) < 0) + return -1; /* * The Master device might not be on the predefined * port (not on port index 1, it is not garanted), @@ -2050,6 +2054,7 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, char tmp_str[IF_NAMESIZE + 32]; struct rte_pci_addr pci_addr; struct mlx5_switch_info info; + int ret; /* Process slave interface names in the loop. */ snprintf(tmp_str, sizeof(tmp_str), @@ -2080,15 +2085,6 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, tmp_str); break; } - /* Match PCI address, allows BDF0+pfx or BDFx+pfx. */ - if (pci_dev->domain == pci_addr.domain && - pci_dev->bus == pci_addr.bus && - pci_dev->devid == pci_addr.devid && - ((pci_dev->function == 0 && - pci_dev->function + owner == pci_addr.function) || - (pci_dev->function == owner && - pci_addr.function == owner))) - pf = info.port_name; /* Get ifindex. */ snprintf(tmp_str, sizeof(tmp_str), "/sys/class/net/%s/ifindex", ifname); @@ -2105,6 +2101,30 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, bond_info->ports[info.port_name].pci_addr = pci_addr; bond_info->ports[info.port_name].ifindex = ifindex; bond_info->n_port++; + /* + * Under socket direct mode, bonding will use + * system_image_guid as identification. + * After OFED 5.4, guid is readable (ret >= 0) under sysfs. + * All bonding members should have the same guid even if driver + * is using PCIe BDF. + */ + ret = mlx5_get_device_guid(&pci_addr, guid, sizeof(guid)); + if (ret < 0) + break; + else if (ret > 0) { + if (!memcmp(guid, cur_guid, sizeof(guid)) && + owner == info.port_name && + (owner != 0 || (owner == 0 && + !rte_pci_addr_cmp(pci_dev, &pci_addr)))) + pf = info.port_name; + } else if (pci_dev->domain == pci_addr.domain && + pci_dev->bus == pci_addr.bus && + pci_dev->devid == pci_addr.devid && + ((pci_dev->function == 0 && + pci_dev->function + owner == pci_addr.function) || + (pci_dev->function == owner && + pci_addr.function == owner))) + pf = info.port_name; } if (pf >= 0) { /* Get bond interface info */ @@ -2117,6 +2137,11 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, DRV_LOG(INFO, "PF device %u, bond device %u(%s)", ifindex, bond_info->ifindex, bond_info->ifname); } + if (owner == 0 && pf != 0) { + DRV_LOG(INFO, "PCIe instance %04x:%02x:%02x.%x isn't bonding owner", + pci_dev->domain, pci_dev->bus, pci_dev->devid, + pci_dev->function); + } return pf; }