From patchwork Tue Sep 28 08:50:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Liu X-Patchwork-Id: 99872 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 14852A0032; Tue, 28 Sep 2021 10:51:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2F03E410F2; Tue, 28 Sep 2021 10:51:53 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2084.outbound.protection.outlook.com [40.107.93.84]) by mails.dpdk.org (Postfix) with ESMTP id 7A3FF410D7 for ; Tue, 28 Sep 2021 10:51:51 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fo3mWWjAoEPQh7Dlttb5DwLU83/7dVKr+icz5Mf+qyzwv6bdD4kFfy+mjZcFEjOtYLE9t612k+9mzXKwHMuBPn2PPPGZUAbqhUBiVv/UyLskb94fKiXoTXpkKY3HVNmWUhJkG/l3OL10q3C3H0GO5jvRXk4Dt3ZtxB8xlJOEiFsfFQGrkf8wE9SJl77ul8Q6LANtvA6/ZF7gLBXwlCeHtoQZOSdGCoioYDoVP99Ro2IHJBXiYU3Gl7VS1K/cLYUg/Uaa5rCjxtbVmpsPt/igrVAjKuCeeMK6HsvGigmTt1Cxkl+DV1GYvfctbN85miWvE3J8PGUjbqROenYEEbVisQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=HeStO2a21BwaQn3Bz9o7trAiTpnll9djHZzGKS2D8yo=; b=IlPgBkU4PBYX/owjk13UEGLi+1Kvj7vZLDnE8tpZwGgsyw6DQ3lw4iZ+m2ajJdYNCf8+29lx8uW54WyYMmtXtTOoV8iw28lfSz+ELzhVnnVT1X6zg6v9ZaXN4dCB6askcNe0i0YEIBlXk9oOcP8YgTQVjr7nCQFsK4clnV430Y7dAX/NzXrITC5To13o6pFElyqmI79wgQUoY5+WAQ5VqQoEbB/K85XgebDDBOwJzMu0HDNY+5yoUa3Mzz+9pNZwkNiGXSAvg0LRhlKL7ZykYUcIbrRI53hZzUY1Er5+Q2FBGWuV2NIbFOWbIdblgqwGNRjECImeCiKDZoNsu1XoKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HeStO2a21BwaQn3Bz9o7trAiTpnll9djHZzGKS2D8yo=; b=g1Vm2gDQOWz50oL5R1vLbVCDp+4gChmND9JlchjJv58Qjh6FkY54ri9shvdfzS/HPf0EXPZOyLEc3ZVQR8aEbdJKbHjGdrTKWwOsYMYqdl332iSmQfz3F4N8LTQRGRlZ0mLZH19SlhxOFDj8RRQ/CAGY/QEOcQWsgKkB+K/5kcYOtYmsNZoxAvT5axTxm8IAgJ9KacUPf4RyFoBRCUjX4pAxsoQJQsOwH8KEfymYkO3dJJ9nIVLQrf3cbAkhUE3sVKrPYtLit8/9/82DNvA7PRP73NTVp1ss4ugeQ1cVphebdnBGspgKUaxU2zPARkohfvA0oES7G8+QPIGbX7H2uQ== Received: from DM6PR07CA0050.namprd07.prod.outlook.com (2603:10b6:5:74::27) by DM5PR12MB1370.namprd12.prod.outlook.com (2603:10b6:3:76::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Tue, 28 Sep 2021 08:51:50 +0000 Received: from DM6NAM11FT021.eop-nam11.prod.protection.outlook.com (2603:10b6:5:74:cafe::f5) by DM6PR07CA0050.outlook.office365.com (2603:10b6:5:74::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 08:51:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by DM6NAM11FT021.mail.protection.outlook.com (10.13.173.76) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 08:51:50 +0000 Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 28 Sep 2021 08:51:49 +0000 Received: from nvidia.com (172.20.187.6) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 28 Sep 2021 08:51:47 +0000 From: Rongwei Liu To: , , , CC: , Date: Tue, 28 Sep 2021 11:50:53 +0300 Message-ID: <20210928085054.1619915-2-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210928085054.1619915-1-rongweil@nvidia.com> References: <20210928085054.1619915-1-rongweil@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To DRHQMAIL107.nvidia.com (10.27.9.16) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 92061eab-8650-4449-3135-08d9825d361f X-MS-TrafficTypeDiagnostic: DM5PR12MB1370: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:595; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MG9mLp+TyD8HiDKtbQjxl1NtAAzeeKPycSeaB8ZNq3J5sjAdAyUsBEf1uDmiKhUsmtCWuH8l//623uSKyaO7QvVKxyr0XmxpCF7yj1Di7V2MN2jSYmF2rmnYAeVCg1907kFqoXlkv5QnRSBjwtfpKxtJrCzHhRWvFBN74TthOHMATdARZWNzTZlfrb5JK63ih7F6oD9IP1mnl5Ntg8r0N5m8ayP2Ow8876fuUztsNcVOSK6wae6xendTegYnG8VHxxB+xnT63f7lFSVbAsNwgMHTqPpiAfmCs8iucdOCrNBIkR12nlOy+GATuiOKeqOWIfAUJiFHjocXMd8i0C+bgjQXjR6xOvEWe40FmsSMUMr4Hk4YmaITz0bPICqobJNGCZnSHwhJaw4QuRNPdLUuzIs2K/lGKx17MYB0WeQvjv2BROyXAVqJdqpqQnan8bnkkFue8ZTBr0y36X0p1LhkSasATbiVwfEWmtJ8cj6UEn6G2Fvrurt4USbDnYUW0EtrVHmxON4fSKboR7yYukxUSa1K5B4zJudDpUxYVUI9KAyi/buCCHb9RHWagmbPxPZLGd9188GlKg54XRj48LotbVeRcZFUsO6QxIbCG3TeXk4Z5lBcUyYzAHRnNMCL2LNx3X5RLoEPWbT3bokzzfzJq5x8lpnyoPQW9L+K67u3hJkiPghWAyt0ZfG9raJHF4t21r2VQu9ugz0tKdUVRuCtKg== X-Forefront-Antispam-Report: CIP:216.228.112.35; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid02.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(6666004)(83380400001)(8676002)(16526019)(82310400003)(426003)(36756003)(1076003)(26005)(107886003)(54906003)(110136005)(2616005)(6286002)(47076005)(36860700001)(5660300002)(186003)(7696005)(55016002)(86362001)(316002)(70586007)(4326008)(2906002)(70206006)(8936002)(7636003)(356005)(508600001)(336012); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 08:51:50.0139 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 92061eab-8650-4449-3135-08d9825d361f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.35]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT021.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1370 Subject: [dpdk-dev] [PATCH v2 1/2] common/mlx5: support pcie device guid query X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" sysfs entry "phys_switch_id" holds each PCIe device' guid. The devices which reside in the same physical NIC should have the same guid. Signed-off-by: Rongwei Liu Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/mlx5_common_os.c | 41 ++++++++++++++++++++++ drivers/common/mlx5/linux/mlx5_common_os.h | 19 ++++++++++ 2 files changed, 60 insertions(+) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 9e0c823c97..8b3ee2baea 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -2,6 +2,7 @@ * Copyright 2020 Mellanox Technologies, Ltd */ +#include #include #include #include @@ -428,3 +429,43 @@ mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) mlx5_glue->free_device_list(ibv_list); return ibv_match; } + +int +mlx5_get_device_guid(const struct rte_pci_addr *dev, uint8_t *guid, size_t len) +{ + char tmp[512]; + char cur_ifname[IF_NAMESIZE + 1]; + FILE *id_file; + DIR *dir; + struct dirent *ptr; + int ret; + + if (guid == NULL || len < sizeof(u_int64_t) + 1) + return -1; + memset(guid, 0, len); + snprintf(tmp, sizeof(tmp), "/sys/bus/pci/devices/%04x:%02x:%02x.%x/net", + dev->domain, dev->bus, dev->devid, dev->function); + dir = opendir(tmp); + if (dir == NULL) + return -1; + /* Traverse to identify PF interface */ + do { + ptr = readdir(dir); + if (ptr == NULL || ptr->d_type != DT_DIR) { + closedir(dir); + return -1; + } + } while (strchr(ptr->d_name, '.') || strchr(ptr->d_name, '_') || + strchr(ptr->d_name, 'v')); + snprintf(cur_ifname, sizeof(cur_ifname), "%s", ptr->d_name); + closedir(dir); + snprintf(tmp + strlen(tmp), sizeof(tmp) - strlen(tmp), + "/%s/phys_switch_id", cur_ifname); + /* Older OFED like 5.3 doesn't support read */ + id_file = fopen(tmp, "r"); + if (!id_file) + return 0; + ret = fscanf(id_file, "%16s", guid); + fclose(id_file); + return ret; +} diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h index c3202b6786..3cdea75373 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.h +++ b/drivers/common/mlx5/linux/mlx5_common_os.h @@ -296,4 +296,23 @@ __rte_internal struct ibv_device * mlx5_os_get_ibv_dev(const struct rte_device *dev); +/** + * This is used to query system_image_guid as describing in PRM. + * + * @param dev[in] + * Pointer to a device instance as PCIe id. + * @param guid[out] + * Pointer to the buffer to hold device guid. + * Guid is uint64_t and corresponding to 17 bytes string. + * @param len[in] + * Guid buffer length, 17 bytes at least. + * + * @return + * -1 if internal failure. + * 0 if OFED doesn't support. + * >0 if success. + */ +int +mlx5_get_device_guid(const struct rte_pci_addr *dev, uint8_t *guid, size_t len); + #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */ From patchwork Tue Sep 28 08:50:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Liu X-Patchwork-Id: 99873 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3EF82A0032; Tue, 28 Sep 2021 10:52:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 53811410ED; Tue, 28 Sep 2021 10:51:56 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2065.outbound.protection.outlook.com [40.107.244.65]) by mails.dpdk.org (Postfix) with ESMTP id 94B0B410EB for ; Tue, 28 Sep 2021 10:51:55 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Tf05wqwi1G9CKikj/LrHtNuqgWpLPIwgDbIhbablH6UnWzbGdIum00OMeWCHXepSZy/MIFh2L0nnHRKcKBRN9ar8nTygW61oi7fqbyQDHM0tnk/qn4Ln59njMyb72Jg6zVTg8QrF5xo/lbzYdkz1BaYp0s4aaAjwZSE7nbrs1kCgjwf89FYXdkRqf5Pw+7C/KxCr9Zq9/9qzuu/U/i5ioD7Qpazn7GVJXQP1go+V7SQMgF84gaL6MZY0qkkw5Fo5uuXaFzQUr/dCVqsWNr2+WO79Z11KkctwxaRRTOKP+Zgw/XWiDkM3INdjMKttgltMjAYwrnVjSonCoJQdw4ZJJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=/BNiTOfdUQ10XFDChjlXxGUs3wYiNosmHg57vcALi7w=; b=KTndy0S+onij88OHR8xSj3G9nZa+UVHLV+mXazUY/03vJUFYIuZAcTZPZKHatN/YDjnetBMtEWma3l6R8HfnLf6JCTB09XsxnhyHBG+GbpXlMozswhOEmRJtUQTomlp0ashLcBR1NARRvWurze8EkuHOZYtBFlSTci7YYUoj32ZBmB84PWNLgHjbw0DrgkKdLek+7z0u5YVmkkvh0tRqb7xr8yfxfa2/kCZN0fZqRdk5pJanXt+ANTQn0ohWn3YmLESOVSszwgO2p2hQoJuknKUcAEq4OntHYUPqUkoDNuQTloWr6CpN2KNSiRyeie4EGMI6E6iK+FkvX0EyGu9Lyg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.32) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/BNiTOfdUQ10XFDChjlXxGUs3wYiNosmHg57vcALi7w=; b=stdqDeHxt53c26gtrdkFA7tRUjUxHSxJJLz9t0QYv+F4Txat20CLITXwmobzsNjehBeg8KbucQrhUDc2jvOC+zss9SeyiNymY9BW8T5JwcKsowJbnHsjRdlmsFr0uMWFveHovWqeVtVcOcXHDsutzuOoHU4bnp65OBNb/HJGpgUzZ0/ytqgooXEsc6X7aqndV9tMIfVu9d0jaAGe/Wa7eJThg1bbmGdjVDyUlFA4Wwto36rutQdXkafbcukHGkwr92ISNgg993Nn38aKXLPABiR1QE8zNnBHuzRHzf/SKXwEPtN0FYBD6A7XKN/Nd4CXwg384h5E0ACD5fr0cwSMPA== Received: from DM6PR17CA0011.namprd17.prod.outlook.com (2603:10b6:5:1b3::24) by BY5PR12MB3924.namprd12.prod.outlook.com (2603:10b6:a03:1af::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.18; Tue, 28 Sep 2021 08:51:53 +0000 Received: from DM6NAM11FT040.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1b3:cafe::df) by DM6PR17CA0011.outlook.office365.com (2603:10b6:5:1b3::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend Transport; Tue, 28 Sep 2021 08:51:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.32) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.32 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.32; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.32) by DM6NAM11FT040.mail.protection.outlook.com (10.13.173.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4544.13 via Frontend Transport; Tue, 28 Sep 2021 08:51:52 +0000 Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 28 Sep 2021 01:51:51 -0700 Received: from nvidia.com (172.20.187.6) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 28 Sep 2021 08:51:49 +0000 From: Rongwei Liu To: , , , CC: , Date: Tue, 28 Sep 2021 11:50:54 +0300 Message-ID: <20210928085054.1619915-3-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210928085054.1619915-1-rongweil@nvidia.com> References: <20210928085054.1619915-1-rongweil@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To DRHQMAIL107.nvidia.com (10.27.9.16) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 113ef212-0bb0-44a5-e678-08d9825d3755 X-MS-TrafficTypeDiagnostic: BY5PR12MB3924: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1013; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: g+rvGloe+5Ti0NdNJaQE3Qf7z2hroxzebzsLX8ksz5cNZ8aQ5qJBzHqbl9cWIYn2A5y2DfKtrf7nLbFRbj7aEu0zKyB53pq8gAzKKgOJU5+RH5NCLJGHXGs0yh7aDVHI/djJWTshLyaVLw6Nly9UNQc9pGau19PkdNn+1k7ttzB+Gsym4gpaSEgdAUwCb+sXVDbtddufs4afNdIlYPdR06purk8F/5H9C/a5TOSQ2jH9GVPb2vaVTu5cjnYcscXat1Za5TZGwdIiAOlusjbEgfEt4nrPYVyCMPkCO43PGU68hzF3OwvnAZLdQSnLWrbuRxT3VDvoF7mx4WZhjZ+hJU5wNtAO6bJS6u6gL5U7hMu9T4LxRQ7zadzZZiK2xGwVKw4Fc/mc607x1xyAU/wepcDEFwCrWFfvCRmZZ8Vu83CwDsc1wULG7hn0B2s7BGTZPnwSp+xMCmsJpwglsPpqKRsGUG2nn2lixqd822PRfWI1L1m9eoMLrf8cYSgkBhFwLShnZ4D/eSsXqJLUd1cknDR/VCsA12FR1nzEpB+X9P4mtm2QMKHmina1prhadXF4p5b9K963L7ubok+46yecnq+3fwlCovutBLoddWcTos1dujERmlj9jt7iC8jz4qobUszhea83fNt6ySija/fHT4SxWwrbxKl5l6M6Qhbemoos8ZgzI5g6DgxgXEUMuFelF36wFWpZK4tzhE/mtVc3WA== X-Forefront-Antispam-Report: CIP:216.228.112.32; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid01.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(7636003)(316002)(36756003)(82310400003)(356005)(5660300002)(54906003)(336012)(55016002)(4326008)(110136005)(6666004)(6286002)(186003)(36860700001)(1076003)(107886003)(2616005)(16526019)(508600001)(47076005)(83380400001)(26005)(70206006)(70586007)(8936002)(2906002)(426003)(7696005)(8676002)(86362001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2021 08:51:52.0452 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 113ef212-0bb0-44a5-e678-08d9825d3755 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.32]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT040.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB3924 Subject: [dpdk-dev] [PATCH v2 2/2] net/mlx5: support socket direct mode bonding X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In socket direct mode, it's possible to bind any two (maybe four in future) PCIe devices with IDs like xxxx:xx:xx.x and yyyy:yy:yy.y. Bonding member interfaces are unnecessary to have the same PCIe domain/bus/device ID anymore, Kernel driver uses "system_image_guid" to identify if devices can be bound together or not. Sysfs "phys_switch_id" is used to get "system_image_guid" of each network interface. OFED 5.4+ is required to support "phys_switch_id". Centos 8.1 needs to enable switch_dev mode first. Signed-off-by: Rongwei Liu Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 43 +++++++++++++++++++++++++------- 1 file changed, 34 insertions(+), 9 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 3746057673..1d57b934fc 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -2008,6 +2008,8 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, FILE *bond_file = NULL, *file; int pf = -1; int ret; + uint8_t cur_guid[32] = {0}; + uint8_t guid[32] = {0}; /* * Try to get master device name. If something goes @@ -2022,6 +2024,8 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, np = mlx5_nl_portnum(nl_rdma, ibv_dev->name); if (!np) return -1; + if (mlx5_get_device_guid(pci_dev, cur_guid, sizeof(cur_guid)) < 0) + return -1; /* * The Master device might not be on the predefined * port (not on port index 1, it is not garanted), @@ -2050,6 +2054,7 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, char tmp_str[IF_NAMESIZE + 32]; struct rte_pci_addr pci_addr; struct mlx5_switch_info info; + int ret; /* Process slave interface names in the loop. */ snprintf(tmp_str, sizeof(tmp_str), @@ -2080,15 +2085,6 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, tmp_str); break; } - /* Match PCI address, allows BDF0+pfx or BDFx+pfx. */ - if (pci_dev->domain == pci_addr.domain && - pci_dev->bus == pci_addr.bus && - pci_dev->devid == pci_addr.devid && - ((pci_dev->function == 0 && - pci_dev->function + owner == pci_addr.function) || - (pci_dev->function == owner && - pci_addr.function == owner))) - pf = info.port_name; /* Get ifindex. */ snprintf(tmp_str, sizeof(tmp_str), "/sys/class/net/%s/ifindex", ifname); @@ -2105,6 +2101,30 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, bond_info->ports[info.port_name].pci_addr = pci_addr; bond_info->ports[info.port_name].ifindex = ifindex; bond_info->n_port++; + /* + * Under socket direct mode, bonding will use + * system_image_guid as identification. + * After OFED 5.4, guid is readable (ret >= 0) under sysfs. + * All bonding members should have the same guid even if driver + * is using PCIe BDF. + */ + ret = mlx5_get_device_guid(&pci_addr, guid, sizeof(guid)); + if (ret < 0) + break; + else if (ret > 0) { + if (!memcmp(guid, cur_guid, sizeof(guid)) && + owner == info.port_name && + (owner != 0 || (owner == 0 && + !rte_pci_addr_cmp(pci_dev, &pci_addr)))) + pf = info.port_name; + } else if (pci_dev->domain == pci_addr.domain && + pci_dev->bus == pci_addr.bus && + pci_dev->devid == pci_addr.devid && + ((pci_dev->function == 0 && + pci_dev->function + owner == pci_addr.function) || + (pci_dev->function == owner && + pci_addr.function == owner))) + pf = info.port_name; } if (pf >= 0) { /* Get bond interface info */ @@ -2117,6 +2137,11 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, DRV_LOG(INFO, "PF device %u, bond device %u(%s)", ifindex, bond_info->ifindex, bond_info->ifname); } + if (owner == 0 && pf != 0) { + DRV_LOG(INFO, "PCIe instance %04x:%02x:%02x.%x isn't bonding owner", + pci_dev->domain, pci_dev->bus, pci_dev->devid, + pci_dev->function); + } return pf; }