From patchwork Tue Sep 14 05:38:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tal Shnaiderman X-Patchwork-Id: 98826 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B05D7A0C47; Tue, 14 Sep 2021 07:40:12 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 00814410ED; Tue, 14 Sep 2021 07:40:03 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2076.outbound.protection.outlook.com [40.107.243.76]) by mails.dpdk.org (Postfix) with ESMTP id 1ADAA410DC for ; Tue, 14 Sep 2021 07:40:00 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S7lCc3O2gH+O31KTNvzdLXPtN2d75TZgHE7dZSAL5dvADnE1AnYoIq+aArfZMIwDOwkhisXgjXvdqqJKOXHFuEjOJv8mvD3IwIQ2s2pggnc929t9AHWJzaI2iL+JN2E7BJYKq4bsftf/DoYKvsfvOvqxGZ8E7nKKeJoRfH7Dnz98uyRLA54rv3ASx4X1MdsmMNTDMjfawQ8jCEXg5S7ZYv8PHGgNXCZ9Vl2EVCduFQF6FuoV28aJIUuwLf+UutW1+WUEDbyWUC3jMdn2+e/1jfVQb2EIsnu43rwKxS78npe+nOfBmZPOnrpum7NlAnEy0AVarj5idsWlMgy7X0+r+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=si7iQQ+p8RDpxCCZ/h4ulU3zZyd/9APYWx0YG/rfeRw=; b=a2ECDBN4yCzYcFZB4YCzfzNF2KE6DdOnfBvNwFjwqsUKoVaJ8wk4N/98LpRD5U6kXLNBp0UTYa3qnX1P2BBvQdbUB/ZR215t72lUGvA1/dOtTwDJrqC/yQSQyQ0SPhJTvm2xzNq6olwx+TLkeeRha7i9Hvq+LsXBaq3sG7UXKW7Peen6ggVFenpfUamB2KL+VhVKlxCNIhqF77MAModSJy31UAFxgSqJzPFclgrSMxpuKlUnVIgsJmpjV0rvQKrRBeLj2uTYyrrWeOaLxTVGra/qoDEffhcApAXwc2bz0IReBponwgeP4H7RmIG98bZFkX++CHKYthWl6R0IuRysqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.36) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=si7iQQ+p8RDpxCCZ/h4ulU3zZyd/9APYWx0YG/rfeRw=; b=tauke9d+bAY1upKhQChLbbZHdeN99u8m2hYyma73axH0ngyObIm2dTwyDCFbDlZ64nAgPjcgq+s0qx2sHemJkQMycV717W3jC9rKaB7MGkpSmpW3YxYXUy36JPWRo3WVljnEDHKmS6725MqnygtLfsiw5g/XrzDVSV1XrtktAEmF7SxxsYLDLF/TuUyw7GjbVVUh1fWotBloIGG1crvSQ+pGOwSi7sg1L+UixNEj/BmsT1xUisMGvnmH6HsAVf0Xts5wu6Wb5d5Cw3cF6mEaKsKNRXqsj7A/uaYsGbYHraZCOc9Cvs3CtdZHNZPHA/9EuaOR9BDk9YlIVD3vrTiqOw== Received: from DS7PR03CA0324.namprd03.prod.outlook.com (2603:10b6:8:2b::24) by BN6PR1201MB0100.namprd12.prod.outlook.com (2603:10b6:405:59::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Tue, 14 Sep 2021 05:39:58 +0000 Received: from DM6NAM11FT026.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2b:cafe::d) by DS7PR03CA0324.outlook.office365.com (2603:10b6:8:2b::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 05:39:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.36) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.36 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.36; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.36) by DM6NAM11FT026.mail.protection.outlook.com (10.13.172.161) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4500.14 via Frontend Transport; Tue, 14 Sep 2021 05:39:58 +0000 Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 14 Sep 2021 05:39:57 +0000 Received: from nvidia.com (172.20.187.6) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 14 Sep 2021 05:39:54 +0000 From: Tal Shnaiderman To: CC: , , , , , , , Date: Tue, 14 Sep 2021 08:38:25 +0300 Message-ID: <20210914053833.7760-3-talshn@nvidia.com> X-Mailer: git-send-email 2.16.1.windows.4 In-Reply-To: <20210914053833.7760-1-talshn@nvidia.com> References: <20210914053833.7760-1-talshn@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To DRHQMAIL107.nvidia.com (10.27.9.16) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 340c7b75-e2cf-45f5-baa2-08d9774216bb X-MS-TrafficTypeDiagnostic: BN6PR1201MB0100: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7wcJnCcubGHapS2xfJdDQlsIZmZsUnby9W+Ipa4Z9cl4PP0AI/7xLY3rHJVmEAM6M/kmjUkWEYsyh/OSbf6Q8/TL0DQZPRnPp2C/06FMdkSxyD8VOx9cOH/FUPB1UqSHrW6JVkNdEnJgJH+n6cMETeg9HaFniSwDqxj+LjqUVMFteMJF9RwJ3LnXtz976YeoC3NU+odsI6hxfajxmZaL4e9Dieov7IS4XrOqcmbIUSRRm+KunzLD6BIbr6/PjMAIGY7zD4pU12Ry82uqFK9t0yKqtRLxX42yMhg75RTOQ1ab1HNDdsQm26oqFi6g1Wqz1Syd6+lwRr0zB2pJT79J/P4GVyf0O8luvayfYLAdhrFWUryIFpJD4rA6Jny3x1D+4gSvfAp/+qYxRNH02i05sLyAMkwG6hizzZejmwgq/oH8HxZTbL1GlqukCC/mHU5zTUoymkna3dxbZFQsdREhHav4MYKNBew9fW4iblcaHf3s/vafrOIm9Lp38sErwb7hiZQBZnJnFIaECWmETfpB86NXtVCNBX9zVlrmIKC6yAeY+qDJ7D7pLJMZ4JnuSODj5Wvx8CfshQJeSr9BY2GPxJFX51Klq3VSUpJDLN66zIV5RrgD4q20h0+ch+RGa/EbhyS10VQDaDeryhgSQ7HphspPU79MN/kOv1Y6rju1xrfRhTj3am+h0Rsuhp1IJEeNVXp6IF1dbz8G0RCXyzAyNw== X-Forefront-Antispam-Report: CIP:216.228.112.36; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid05.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(7696005)(2906002)(83380400001)(36860700001)(54906003)(86362001)(8676002)(6916009)(55016002)(16526019)(186003)(26005)(6286002)(426003)(107886003)(70586007)(70206006)(336012)(1076003)(2616005)(82310400003)(508600001)(8936002)(4326008)(6666004)(36906005)(356005)(5660300002)(316002)(7636003)(47076005)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2021 05:39:58.1055 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 340c7b75-e2cf-45f5-baa2-08d9774216bb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.36]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT026.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1201MB0100 Subject: [dpdk-dev] [RFC PATCH 02/10] common/mlx5: add an agnostic OS function to open device context X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a function to open device context from a rte_device. Function mlx5_os_open_device_context can be used both on Windows and Linux OS. Signed-off-by: Tal Shnaiderman --- drivers/common/mlx5/linux/mlx5_common_os.c | 28 +++++++ drivers/common/mlx5/mlx5_common.h | 4 + drivers/common/mlx5/version.map | 2 + drivers/common/mlx5/windows/mlx5_common_os.c | 118 +++++++++++++++++++++++++++ drivers/common/mlx5/windows/mlx5_common_os.h | 2 + drivers/net/mlx5/windows/mlx5_os.c | 64 +-------------- 6 files changed, 155 insertions(+), 63 deletions(-) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 9e0c823c97..3ef507944f 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -428,3 +428,31 @@ mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) mlx5_glue->free_device_list(ibv_list); return ibv_match; } + +/** + * Open device context from a rte_device. + * + * @param[in] dev + * Pointer to an rte_device struct. + * @return + * Pointer to device context or NULL in case context cannot be found. + */ +void * +mlx5_os_open_device_context(struct rte_device *dev) +{ + struct ibv_device *ibv; + void *ctx; + + ibv = mlx5_os_get_ibv_dev(dev); + if (ibv == NULL) { + DRV_LOG(ERR, "Failed getting ibv_dev"); + return NULL; + } + ctx = mlx5_glue->dv_open_device(ibv); + if (ctx == NULL) { + DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); + rte_errno = ENODEV; + return NULL; + } + return ctx; +} diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index a772371200..249804b00c 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -419,4 +419,8 @@ __rte_internal bool mlx5_dev_is_pci(const struct rte_device *dev); +__rte_internal +void * +mlx5_os_open_device_context(struct rte_device *dev); + #endif /* RTE_PMD_MLX5_COMMON_H_ */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index e5cb6b7060..6d4258dd25 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -141,6 +141,8 @@ INTERNAL { mlx5_os_alloc_pd; mlx5_os_dealloc_pd; mlx5_os_dereg_mr; + mlx5_os_match_devx_devices_to_addr; + mlx5_os_open_device_context; mlx5_os_get_ibv_dev; # WINDOWS_NO_EXPORT mlx5_os_reg_mr; mlx5_os_umem_dereg; diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c index 5031bdca26..3b59e57e57 100644 --- a/drivers/common/mlx5/windows/mlx5_common_os.c +++ b/drivers/common/mlx5/windows/mlx5_common_os.c @@ -6,6 +6,7 @@ #include #include +#include #include #include #include @@ -205,3 +206,120 @@ mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr) claim_zero(mlx5_os_umem_dereg(pmd_mr->obj)); memset(pmd_mr, 0, sizeof(*pmd_mr)); } + +/** + * Detect if a devx_device_bdf object has identical DBDF values to the + * rte_pci_addr found in bus/pci probing + * + * @param[in] devx_bdf + * Pointer to the devx_device_bdf structure. + * @param[in] addr + * Pointer to the rte_pci_addr structure. + * + * @return + * 1 on Device match, 0 on mismatch. + */ +static int +mlx5_os_match_devx_bdf_to_addr(struct devx_device_bdf *devx_bdf, + struct rte_pci_addr *addr) +{ + if (addr->domain != (devx_bdf->bus_id >> 8) || + addr->bus != (devx_bdf->bus_id & 0xff) || + addr->devid != devx_bdf->dev_id || + addr->function != devx_bdf->fnc_id) { + return 0; + } + return 1; +} + +/** + * Detect if a devx_device_bdf object matches the rte_pci_addr + * found in bus/pci probing + * Compare both the Native/PF BDF and the raw_bdf representing a VF BDF. + * + * @param[in] devx_bdf + * Pointer to the devx_device_bdf structure. + * @param[in] addr + * Pointer to the rte_pci_addr structure. + * + * @return + * 1 on Device match, 0 on mismatch, rte_errno code on failure. + */ +int +mlx5_os_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf, + struct rte_pci_addr *addr) +{ + int err; + struct devx_device mlx5_dev; + + if (mlx5_os_match_devx_bdf_to_addr(devx_bdf, addr)) + return 1; + /** + * Didn't match on Native/PF BDF, could still + * Match a VF BDF, check it next + */ + err = mlx5_glue->query_device(devx_bdf, &mlx5_dev); + if (err) { + DRV_LOG(ERR, "query_device failed"); + rte_errno = err; + return rte_errno; + } + if (mlx5_os_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr)) + return 1; + return 0; +} + +/** + * Open device context from a rte_device. + * + * @param[in] dev + * Pointer to an rte_device struct. + * @return + * Pointer to device context or NULL in case context cannot be found. + */ +void * +mlx5_os_open_device_context(struct rte_device *dev) +{ + struct devx_device_bdf *devx_bdf_devs, *orig_devx_bdf_devs; + struct mlx5_context *devx_ctx_match = NULL; + int ret, err; + struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev); + + devx_bdf_devs = mlx5_glue->get_device_list(&ret); + orig_devx_bdf_devs = devx_bdf_devs; + if (!devx_bdf_devs) { + rte_errno = errno ? errno : ENOSYS; + DRV_LOG(ERR, "cannot list devices, is DevX configured?"); + return NULL; + } + while (ret-- > 0) { + err = mlx5_os_match_devx_devices_to_addr(devx_bdf_devs, + &pci_dev->addr); + if (err == 1) { + devx_ctx_match = mlx5_glue->open_device(devx_bdf_devs); + if (!devx_ctx_match) { + DRV_LOG(ERR, "open_device failed"); + err = errno; + goto exit; + } + err = mlx5_glue->query_device(devx_bdf_devs, + &devx_ctx_match->mlx5_dev); + if (err) { + DRV_LOG(ERR, "query device context failed."); + rte_errno = errno ? errno : ENOSYS; + claim_zero(mlx5_glue->close_device( + devx_ctx_match)); + goto exit; + } + } + if (err != 0) { + DRV_LOG(ERR, "failed to match addr to rte_device"); + ret = -err; + goto exit; + } + devx_bdf_devs++; + } +exit: + mlx5_glue->free_device_list(orig_devx_bdf_devs); + return devx_ctx_match; +} diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h index 3756e1959b..c3d74d3b67 100644 --- a/drivers/common/mlx5/windows/mlx5_common_os.h +++ b/drivers/common/mlx5/windows/mlx5_common_os.h @@ -261,4 +261,6 @@ int mlx5_os_reg_mr(void *pd, void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr); __rte_internal void mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr); +int mlx5_os_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf, + struct rte_pci_addr *addr); #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */ diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index 26fa927039..9dea6d639e 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -901,68 +901,6 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable) return -ENOTSUP; } -/** - * Detect if a devx_device_bdf object has identical DBDF values to the - * rte_pci_addr found in bus/pci probing - * - * @param[in] devx_bdf - * Pointer to the devx_device_bdf structure. - * @param[in] addr - * Pointer to the rte_pci_addr structure. - * - * @return - * 1 on Device match, 0 on mismatch. - */ -static int -mlx5_match_devx_bdf_to_addr(struct devx_device_bdf *devx_bdf, - struct rte_pci_addr *addr) -{ - if (addr->domain != (devx_bdf->bus_id >> 8) || - addr->bus != (devx_bdf->bus_id & 0xff) || - addr->devid != devx_bdf->dev_id || - addr->function != devx_bdf->fnc_id) { - return 0; - } - return 1; -} - -/** - * Detect if a devx_device_bdf object matches the rte_pci_addr - * found in bus/pci probing - * Compare both the Native/PF BDF and the raw_bdf representing a VF BDF. - * - * @param[in] devx_bdf - * Pointer to the devx_device_bdf structure. - * @param[in] addr - * Pointer to the rte_pci_addr structure. - * - * @return - * 1 on Device match, 0 on mismatch, rte_errno code on failure. - */ -static int -mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf, - struct rte_pci_addr *addr) -{ - int err; - struct devx_device mlx5_dev; - - if (mlx5_match_devx_bdf_to_addr(devx_bdf, addr)) - return 1; - /** - * Didn't match on Native/PF BDF, could still - * Match a VF BDF, check it next - */ - err = mlx5_glue->query_device(devx_bdf, &mlx5_dev); - if (err) { - DRV_LOG(ERR, "query_device failed"); - rte_errno = err; - return rte_errno; - } - if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr)) - return 1; - return 0; -} - /** * DPDK callback to register a PCI device. * @@ -1036,7 +974,7 @@ mlx5_os_net_probe(struct rte_device *dev) struct devx_device_bdf *devx_bdf_match[ret + 1]; while (ret-- > 0) { - err = mlx5_match_devx_devices_to_addr(devx_bdf_devs, + err = mlx5_os_match_devx_devices_to_addr(devx_bdf_devs, &pci_dev->addr); if (!err) { devx_bdf_devs++;