From patchwork Thu May 27 13:37:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93482 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 38CC2A0548; Thu, 27 May 2021 15:38:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A34B941101; Thu, 27 May 2021 15:38:57 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2061.outbound.protection.outlook.com [40.107.237.61]) by mails.dpdk.org (Postfix) with ESMTP id 29991410F9 for ; Thu, 27 May 2021 15:38:56 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AZjXFEtZD3W8vvBIVjcdFCNDlqA41UQMQ/MixLzt2iU2oVMCWUIajTbIvbXmh5P63z8BbyYs+seTJaXg2VKHsdeXze3YtNC3z6byae6Mx6jIvLhyKzKq1E5JSMFk20nSUoHMWLV1KoKY4dXz7nZ55qBtC26CWBJwiK9kK35Xj9teH/ZafJglLt9SWwGDDvjc/uBrgNsqY780X4MXOX1TwIdIRrU0fR6uyDksKEjV08FA/8cYgz24cMoEerPLAjbjUROia2MjWv6g+d2IQz8BvpMUxxox5cpFmZTZNOUDRwsikuWjZpv3czdpW8WvYYdesVXsAzWQPKNsHACwPav7+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Oqt8AtawMf/fTmSFD8JvOCYXPzxC0OxAQf+t6iFkAfo=; b=IFX2eq7TeCO6jRYcOkdKUzWzDz7xWth5sYeQTYUNCH/Wl0DdZoGP/rQFmv8IxBdv7LUGEpcakkDOAXJkUULdKb+2W7cZ5tFj6+AtTh8KrSQK///uHbtBgJ/3ZiRfdhskqBxku42AeYFKnfYH6GNSEEKWyH8TXU9uurNliqXZnEahvbG9Y1yO4tDkckS0CiiDluJWfqOXMbMJEJqnbV5HCOEq2VegcT4qyGigGSov3uZld1tWTk6+33Nr8PisGac/Lu+5+D4/NLzQ5SyCQQnK8lES7UuHqV9cwhNOuMwvi1yF/xr2CvEntRlV9itw0OwrPK5RSe1GrmrhncHPPC6uAQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=tuxdriver.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Oqt8AtawMf/fTmSFD8JvOCYXPzxC0OxAQf+t6iFkAfo=; b=nMYT5bGl7DotGFFaHEs/KgzaYEP4zrHo20VIcJAIievAcSck/+K9h/awUerH41lrWbeKELFoayiR0psHTskWW3EBfAZnYYd9/S3V9QUvPgs9O+gewg0U7gisxGTaiF8UfZKiZsAp2BGIpbpBK5FCtkagJapSuy3DNb1yA9xY25v3GwXsJxfkUVY6Bw541cD/4atCNzO1nuKsASp9r2D5eijil7h5dUkK/wnTDLi4JqUX1weWx4HuzHtTIe1jn4xZ+5OLbBR+qCSuVSHYV6kshONlFnNRc+ui9R9WgEtuxvaHc771UPPFt+XnTGdhg++d0lq9PQwG7k5USz9eKrQfBw== Received: from CO2PR05CA0055.namprd05.prod.outlook.com (2603:10b6:102:2::23) by CH2PR12MB4278.namprd12.prod.outlook.com (2603:10b6:610:ab::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20; Thu, 27 May 2021 13:38:54 +0000 Received: from CO1NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:102:2:cafe::ab) by CO2PR05CA0055.outlook.office365.com (2603:10b6:102:2::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.10 via Frontend Transport; Thu, 27 May 2021 13:38:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; tuxdriver.com; dkim=none (message not signed) header.d=none;tuxdriver.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT063.mail.protection.outlook.com (10.13.175.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4150.30 via Frontend Transport; Thu, 27 May 2021 13:38:53 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:38:50 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Ray Kinsella , Neil Horman Date: Thu, 27 May 2021 16:37:46 +0300 Message-ID: <20210527133759.17401-2-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 96ea216a-faeb-4b0d-bc57-08d92114c4fc X-MS-TrafficTypeDiagnostic: CH2PR12MB4278: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cpFSOi+Bi2T0kkqVu9b4aWpLGo/sNYyF/NRLkxKPucbcD7eSkiwfxbGTpiVct6QzejJkLobQ6O0LYIQQ6UKnewhvqq+41RRMRjiI8DcsLw7XxH+YYDu0QgZD5gay53HnaxdmO/XtJ5n3El/YK6tWMIDF4Wy6aStxkcCF1BKpuDydnlq5uaU5Z3gede9+pqlg8gSlcmpyejlzUt3eDNF8c0D9+hZlaCSuOIWuNeS3iqBqWol3lgEXu4ehZLwShVu/GOGimYmwZx9gEPNc15LDMKPgmlM4+t2cUexnnXLrz4JkHvhvyKHS0G7bpep9sHibzkfTiOYMdEN2AcimMvCf2eZQNJv/Vnk9Ygr3ATngMQ/HBl8Is42kHX2waEnjdAMn+Ua2w4cj6SraY+gIQDr9/IdPIZsDOQ4GHrOFXE11xhhNpvGuKmWSA86KLDTbI4nAk/rvzf1GNhOk2KYYDnPfO2F0biOCddzIcORfGYHh+T155rDpBsPe1cfgkhtoHHevftKh15gFlTtzA0e0gGpohkZxt9Kb3Vnk8YpuJ85M+CyIJojc24c/BsHtVXsGuHS3zbktdB7XhJmXe86ZSQC2I0EFFvrDzSFKWBETuLtdgmzbJV8MS8z1LnL51uWbGJUuOUCL3DQ/Rraiqiq7gccapT4BryE4mzTGXnyQlB0G4Eo= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(39860400002)(396003)(36840700001)(46966006)(83380400001)(6636002)(8676002)(6286002)(478600001)(30864003)(6862004)(8936002)(82310400003)(36906005)(55016002)(336012)(426003)(5660300002)(36860700001)(7696005)(86362001)(82740400003)(7636003)(1076003)(37006003)(356005)(70586007)(316002)(26005)(36756003)(6666004)(54906003)(2906002)(16526019)(186003)(2616005)(4326008)(47076005)(70206006); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:38:53.6546 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 96ea216a-faeb-4b0d-bc57-08d92114c4fc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4278 Subject: [dpdk-dev] [RFC 01/14] common/mlx5: add common device driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support auxiliary bus, introduces common device driver and callbacks, suppose to replace current mlx5 common PCI bus driver. mlx5 common PCI bus driver still used by mlx5 eth, vDPA and regex PMD, will remove once all PMD drivers adapt to new common driver. Signed-off-by: Xueming Li --- drivers/common/mlx5/linux/mlx5_common_verbs.c | 21 +- drivers/common/mlx5/mlx5_common.c | 322 +++++++++++++++++- drivers/common/mlx5/mlx5_common.h | 133 ++++++++ drivers/common/mlx5/mlx5_common_pci.c | 165 ++++++++- drivers/common/mlx5/mlx5_common_private.h | 42 +++ drivers/common/mlx5/version.map | 5 + 6 files changed, 672 insertions(+), 16 deletions(-) create mode 100644 drivers/common/mlx5/mlx5_common_private.h diff --git a/drivers/common/mlx5/linux/mlx5_common_verbs.c b/drivers/common/mlx5/linux/mlx5_common_verbs.c index aa560f05f2..a49440ef72 100644 --- a/drivers/common/mlx5/linux/mlx5_common_verbs.c +++ b/drivers/common/mlx5/linux/mlx5_common_verbs.c @@ -10,11 +10,31 @@ #include #include +#include +#include + +#include "mlx5_common_log.h" +#include "mlx5_common_utils.h" +#include "mlx5_common_private.h" #include "mlx5_autoconf.h" #include #include #include +struct ibv_device * +mlx5_get_ibv_device(const struct rte_device *dev) +{ + struct ibv_device *ibv = NULL; + + if (mlx5_dev_is_pci(dev)) + ibv = mlx5_get_pci_ibv_device(&RTE_DEV_TO_PCI_CONST(dev)->addr); + if (ibv == NULL) { + rte_errno = ENODEV; + DRV_LOG(ERR, "Verbs device not found: %s", dev->name); + } + return ibv; +} + /** * Register mr. Given protection domain pointer, pointer to addr and length * register the memory region. @@ -68,4 +88,3 @@ mlx5_common_verbs_dereg_mr(struct mlx5_pmd_mr *pmd_mr) memset(pmd_mr, 0, sizeof(*pmd_mr)); } } - diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 25e9f09108..f2e2a95ae0 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -8,11 +8,14 @@ #include #include +#include +#include #include "mlx5_common.h" #include "mlx5_common_os.h" #include "mlx5_common_log.h" #include "mlx5_common_pci.h" +#include "mlx5_common_private.h" uint8_t haswell_broadwell_cpu; @@ -41,6 +44,321 @@ static inline void mlx5_cpu_id(unsigned int level, RTE_LOG_REGISTER_DEFAULT(mlx5_common_logtype, NOTICE) +/* Head of list of drivers. */ +static TAILQ_HEAD(mlx5_drivers, mlx5_class_driver) drivers_list = + TAILQ_HEAD_INITIALIZER(drivers_list); + +/* Head of devices. */ +static TAILQ_HEAD(mlx5_devices, mlx5_common_device) devices_list = + TAILQ_HEAD_INITIALIZER(devices_list); + +static const struct { + const char *name; + unsigned int drv_class; +} mlx5_classes[] = { + { .name = "vdpa", .drv_class = MLX5_CLASS_VDPA }, + { .name = "net", .drv_class = MLX5_CLASS_NET }, + { .name = "regex", .drv_class = MLX5_CLASS_REGEX }, + { .name = "compress", .drv_class = MLX5_CLASS_COMPRESS }, +}; + +static int +class_name_to_value(const char *class_name) +{ + unsigned int i; + + for (i = 0; i < RTE_DIM(mlx5_classes); i++) { + if (strcmp(class_name, mlx5_classes[i].name) == 0) + return mlx5_classes[i].drv_class; + } + return -EINVAL; +} + +static struct mlx5_class_driver * +driver_get(uint32_t class) +{ + struct mlx5_class_driver *driver; + + TAILQ_FOREACH(driver, &drivers_list, next) { + if ((uint32_t)driver->drv_class == class) + return driver; + } + return NULL; +} + +static int +parse_class_option(const struct rte_devargs *devargs) +{ + const char *cls; + struct rte_kvargs *kvlist = NULL; + int ret = 0; + + if (devargs == NULL) + return 0; + if (devargs->cls != NULL) { + /* Global syntax. */ + cls = devargs->cls->name; + } else { + /* Legacy syntax. */ + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + return -EINVAL; + cls = rte_kvargs_get(kvlist, RTE_DEVARGS_KEY_CLASS); + } + if (cls != NULL) + ret = class_name_to_value(cls); + if (kvlist != NULL) + rte_kvargs_free(kvlist); + return ret; +} + +static const unsigned int mlx5_class_invalid_combinations[] = { + MLX5_CLASS_NET | MLX5_CLASS_VDPA, + /* New class combination should be added here. */ +}; + +static int +is_valid_class_combination(uint32_t user_classes) +{ + unsigned int i; + + /* Verify if user specified unsupported combination. */ + for (i = 0; i < RTE_DIM(mlx5_class_invalid_combinations); i++) { + if ((mlx5_class_invalid_combinations[i] & user_classes) == + mlx5_class_invalid_combinations[i]) + return -EINVAL; + } + /* Not found any invalid class combination. */ + return 0; +} + +static bool +device_class_enabled(const struct mlx5_common_device *device, uint32_t class) +{ + return (device->classes_loaded & class) > 0; +} + +static bool +mlx5_bus_match(const struct mlx5_class_driver *drv, + const struct rte_device *dev) +{ + if (mlx5_dev_is_pci(dev)) + return mlx5_dev_pci_match(drv, dev); + return true; +} + +static struct mlx5_common_device * +to_mlx5_device(const struct rte_device *rte_dev) +{ + struct mlx5_common_device *dev; + + TAILQ_FOREACH(dev, &devices_list, next) { + if (rte_dev == dev->dev) + return dev; + } + return NULL; +} + +static void +dev_release(struct mlx5_common_device *dev) +{ + TAILQ_REMOVE(&devices_list, dev, next); + rte_free(dev); +} + +static int +drivers_remove(struct mlx5_common_device *dev, uint32_t enabled_classes) +{ + struct mlx5_class_driver *driver; + int local_ret = -ENODEV; + unsigned int i = 0; + int ret = 0; + + enabled_classes &= dev->classes_loaded; + while (enabled_classes) { + driver = driver_get(RTE_BIT64(i)); + if (driver != NULL) { + local_ret = driver->remove(dev->dev); + if (local_ret == 0) + dev->classes_loaded &= ~RTE_BIT64(i); + else if (ret == 0) + ret = local_ret; + } + enabled_classes &= ~RTE_BIT64(i); + i++; + } + if (local_ret != 0 && ret == 0) + ret = local_ret; + return ret; +} + +static int +drivers_probe(struct mlx5_common_device *dev, uint32_t user_classes) +{ + struct mlx5_class_driver *driver; + uint32_t enabled_classes = 0; + bool already_loaded; + int ret; + + TAILQ_FOREACH(driver, &drivers_list, next) { + if ((driver->drv_class & user_classes) == 0) + continue; + if (!mlx5_bus_match(driver, dev->dev)) + continue; + already_loaded = dev->classes_loaded & driver->drv_class; + if (already_loaded && driver->probe_again == 0) { + DRV_LOG(ERR, "Device %s is already probed", + dev->dev->name); + ret = -EEXIST; + goto probe_err; + } + ret = driver->probe(dev->dev); + if (ret < 0) { + DRV_LOG(ERR, "Failed to load driver %s", + driver->name); + goto probe_err; + } + enabled_classes |= driver->drv_class; + } + dev->classes_loaded |= enabled_classes; + return 0; +probe_err: + /* Only unload drivers which are enabled which were enabled + * in this probe instance. + */ + drivers_remove(dev, enabled_classes); + return ret; +} + +int +mlx5_common_dev_probe(struct rte_device *eal_dev) +{ + struct mlx5_common_device *dev; + uint32_t user_class = 0; + bool new_device = false; + int ret; + + DRV_LOG(INFO, "probe device \"%s\".", eal_dev->name); + ret = parse_class_option(eal_dev->devargs); + if (ret < 0) + return ret; + user_class = ret; + if (user_class == 0) + /* Default to net class. */ + user_class = MLX5_CLASS_NET; + dev = to_mlx5_device(eal_dev); + if (!dev) { + dev = rte_zmalloc("mlx5_common_device", sizeof(*dev), 0); + if (!dev) + return -ENOMEM; + dev->dev = eal_dev; + TAILQ_INSERT_HEAD(&devices_list, dev, next); + new_device = true; + } else { + /* Validate combination here. */ + ret = is_valid_class_combination(user_class | + dev->classes_loaded); + if (ret != 0) { + DRV_LOG(ERR, "Unsupported mlx5 classes supplied."); + return ret; + } + } + ret = drivers_probe(dev, user_class); + if (ret) + goto class_err; + return 0; +class_err: + if (new_device) + dev_release(dev); + return ret; +} + +int +mlx5_common_dev_remove(struct rte_device *eal_dev) +{ + struct mlx5_common_device *dev; + int ret; + + dev = to_mlx5_device(eal_dev); + if (!dev) + return -ENODEV; + /* Matching device found, cleanup and unload drivers. */ + ret = drivers_remove(dev, dev->classes_loaded); + if (ret != 0) + dev_release(dev); + return ret; +} + +int +mlx5_common_dev_dma_map(struct rte_device *dev, void *addr, uint64_t iova, + size_t len) +{ + struct mlx5_class_driver *driver = NULL; + struct mlx5_class_driver *temp; + struct mlx5_common_device *mdev; + int ret = -EINVAL; + + mdev = to_mlx5_device(dev); + if (!mdev) + return -ENODEV; + TAILQ_FOREACH(driver, &drivers_list, next) { + if (!device_class_enabled(mdev, driver->drv_class) || + driver->dma_map == NULL) + continue; + ret = driver->dma_map(dev, addr, iova, len); + if (ret) + goto map_err; + } + return ret; +map_err: + TAILQ_FOREACH(temp, &drivers_list, next) { + if (temp == driver) + break; + if (device_class_enabled(mdev, temp->drv_class) && + temp->dma_map && temp->dma_unmap) + temp->dma_unmap(dev, addr, iova, len); + } + return ret; +} + +int +mlx5_common_dev_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, + size_t len) +{ + struct mlx5_class_driver *driver; + struct mlx5_common_device *mdev; + int local_ret = -EINVAL; + int ret = 0; + + mdev = to_mlx5_device(dev); + if (!mdev) + return -ENODEV; + /* There is no unmap error recovery in current implementation. */ + TAILQ_FOREACH_REVERSE(driver, &drivers_list, mlx5_drivers, next) { + if (!device_class_enabled(mdev, driver->drv_class) || + driver->dma_unmap == NULL) + continue; + local_ret = driver->dma_unmap(dev, addr, iova, len); + if (local_ret && (ret == 0)) + ret = local_ret; + } + if (local_ret) + ret = local_ret; + return ret; +} + +void +mlx5_class_driver_register(struct mlx5_class_driver *driver) +{ + mlx5_common_driver_on_register_pci(driver); + TAILQ_INSERT_TAIL(&drivers_list, driver, next); +} + +static void mlx5_common_driver_init(void) +{ + mlx5_common_pci_init(); +} + static bool mlx5_common_initialized; /** @@ -55,7 +373,7 @@ mlx5_common_init(void) return; mlx5_glue_constructor(); - mlx5_common_pci_init(); + mlx5_common_driver_init(); mlx5_common_initialized = true; } @@ -214,3 +532,5 @@ mlx5_devx_alloc_uar(void *ctx, int mapping) exit: return uar; } + +RTE_PMD_EXPORT_NAME(mlx5_class_driver, __COUNTER__); diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 306f2f1ab7..0e67a36d0c 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -134,6 +135,11 @@ enum { PCI_DEVICE_ID_MELLANOX_CONNECTX7BF = 0Xa2dc, }; +struct ibv_device; + +__rte_internal +struct ibv_device *mlx5_get_ibv_device(const struct rte_device *dev); + /* Maximum number of simultaneous unicast MAC addresses. */ #define MLX5_MAX_UC_MAC_ADDRESSES 128 /* Maximum number of simultaneous Multicast MAC addresses. */ @@ -242,4 +248,131 @@ extern uint8_t haswell_broadwell_cpu; __rte_internal void mlx5_common_init(void); +/** + * Common Driver Interface + * ConnectX common driver supports multiple classes: net,vdpa,regex,crypto and + * compress devices. This layer enables creating such multiple class of devices + * on a single device by allowing to bind multiple class specific device + * driver to attach to common driver. + * + * ---------------- + * | mlx5 class | + * | drivers | + * ---------------- + * || + * ----------------- + * | mlx5 | + * | common driver | + * ----------------- + * | | + * ----------- ----------------- + * | mlx5 | | mlx5 | + * | pci dev | | auxiliary dev | + * ----------- ----------------- + * + * - mlx5 pci bus driver binds to mlx5 PCI devices defined by PCI + * ID table of all related mlx5 PCI devices. + * - mlx5 class driver such as net, vdpa, regex PMD defines its + * specific PCI ID table and mlx5 bus driver probes matching + * class drivers. + * - mlx5 common driver is central place that validates supported + * class combinations. + * - mlx5 common driver hide bus difference by resolving device address + * from devargs, locating target RDMA device and probing with it. + */ + +/** + * Initialization function for the driver called during device probing. + */ +typedef int (mlx5_class_driver_probe_t)(struct rte_device *dev); + +/** + * Uninitialization function for the driver called during hot-unplugging. + */ +typedef int (mlx5_class_driver_remove_t)(struct rte_device *dev); + +/** + * Driver-specific DMA mapping. After a successful call the device + * will be able to read/write from/to this segment. + * + * @param dev + * Pointer to the device. + * @param addr + * Starting virtual address of memory to be mapped. + * @param iova + * Starting IOVA address of memory to be mapped. + * @param len + * Length of memory segment being mapped. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int (mlx5_class_driver_dma_map_t)(struct rte_device *dev, void *addr, + uint64_t iova, size_t len); + +/** + * Driver-specific DMA un-mapping. After a successful call the device + * will not be able to read/write from/to this segment. + * + * @param dev + * Pointer to the device. + * @param addr + * Starting virtual address of memory to be unmapped. + * @param iova + * Starting IOVA address of memory to be unmapped. + * @param len + * Length of memory segment being unmapped. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int (mlx5_class_driver_dma_unmap_t)(struct rte_device *dev, void *addr, + uint64_t iova, size_t len); + +/** Device already probed can be probed again to check for new ports. */ +#define MLX5_DRV_PROBE_AGAIN 0x0004 + +/** + * A structure describing a mlx5 common class driver. + */ +struct mlx5_class_driver { + TAILQ_ENTRY(mlx5_class_driver) next; + enum mlx5_class drv_class; /**< Class of this driver. */ + const char *name; /**< Driver name. */ + mlx5_class_driver_probe_t *probe; /**< Device Probe function. */ + mlx5_class_driver_remove_t *remove; /**< Device Remove function. */ + mlx5_class_driver_dma_map_t *dma_map; /**< device dma map function. */ + mlx5_class_driver_dma_unmap_t *dma_unmap; /**< device dma unmap function. */ + const struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ + uint32_t probe_again:1; + /**< Device already probed can be probed again to check new device. */ + uint32_t intr_lsc:1; /**< Supports link state interrupt. */ + uint32_t intr_rmv:1; /**< Supports device remove interrupt. */ +}; + +/** + * Register a mlx5 device driver. + * + * @param driver + * A pointer to a mlx5_driver structure describing the driver + * to be registered. + */ +__rte_internal +void +mlx5_class_driver_register(struct mlx5_class_driver *driver); + +/** + * Test a device is PCI bus device. + * + * @param dev + * Pointer to device. + * + * @return + * - True on device devargs is a PCI bus device. + * - False otherwise. + */ +__rte_internal +bool +mlx5_dev_is_pci(const struct rte_device *dev); + #endif /* RTE_PMD_MLX5_COMMON_H_ */ diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 34747c4e07..53090173a2 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -3,11 +3,19 @@ */ #include + #include +#include +#include #include #include "mlx5_common_log.h" #include "mlx5_common_pci.h" +#include "mlx5_common_private.h" + +static struct rte_pci_driver mlx5_common_pci_driver; + +/********** Legacy PCI bus driver, to be removed ********/ struct mlx5_pci_device { struct rte_pci_device *pci_dev; @@ -282,8 +290,8 @@ drivers_probe(struct mlx5_pci_device *dev, struct rte_pci_driver *pci_drv, * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) +mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) { struct mlx5_pci_device *dev; uint32_t user_classes = 0; @@ -336,7 +344,7 @@ mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, * 0 on success, the function cannot fail. */ static int -mlx5_common_pci_remove(struct rte_pci_device *pci_dev) +mlx5_pci_remove(struct rte_pci_device *pci_dev) { struct mlx5_pci_device *dev; int ret; @@ -352,8 +360,8 @@ mlx5_common_pci_remove(struct rte_pci_device *pci_dev) } static int -mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) +mlx5_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) { struct mlx5_pci_driver *driver = NULL; struct mlx5_pci_driver *temp; @@ -385,8 +393,8 @@ mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, } static int -mlx5_common_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) +mlx5_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) { struct mlx5_pci_driver *driver; struct mlx5_pci_device *dev; @@ -419,10 +427,10 @@ static struct rte_pci_driver mlx5_pci_driver = { .driver = { .name = MLX5_PCI_DRIVER_NAME, }, - .probe = mlx5_common_pci_probe, - .remove = mlx5_common_pci_remove, - .dma_map = mlx5_common_pci_dma_map, - .dma_unmap = mlx5_common_pci_dma_unmap, + .probe = mlx5_pci_probe, + .remove = mlx5_pci_remove, + .dma_map = mlx5_pci_dma_map, + .dma_unmap = mlx5_pci_dma_unmap, }; static int @@ -486,7 +494,7 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) updated_table = calloc(num_ids, sizeof(*updated_table)); if (!updated_table) return -ENOMEM; - if (TAILQ_EMPTY(&drv_list)) { + if (old_table == NULL) { /* Copy the first driver's ID table. */ for (id_iter = driver_id_table; id_iter->vendor_id != 0; id_iter++, i++) @@ -502,6 +510,7 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) /* Terminate table with empty entry. */ updated_table[i].vendor_id = 0; mlx5_pci_driver.id_table = updated_table; + mlx5_common_pci_driver.id_table = updated_table; mlx5_pci_id_table = updated_table; if (old_table) free(old_table); @@ -520,6 +529,133 @@ mlx5_pci_driver_register(struct mlx5_pci_driver *driver) TAILQ_INSERT_TAIL(&drv_list, driver, next); } +/********** New common PCI bus driver ********/ + +bool +mlx5_dev_is_pci(const struct rte_device *dev) +{ + struct rte_devargs *da = dev->devargs; + + if (da == NULL || da->bus == NULL) + return false; + return strcmp(da->bus->name, "pci") == 0; +} + +struct ibv_device * +mlx5_get_pci_ibv_device(const struct rte_pci_addr *addr) +{ + int n; + struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); + struct ibv_device *ibv_match = NULL; + + if (!ibv_list) { + rte_errno = ENOSYS; + return NULL; + } + while (n-- > 0) { + struct rte_pci_addr pci_addr; + + DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name); + if (mlx5_get_pci_addr(ibv_list[n]->ibdev_path, &pci_addr)) + continue; + if (rte_pci_addr_cmp(addr, &pci_addr)) + continue; + ibv_match = ibv_list[n]; + break; + } + if (!ibv_match) + rte_errno = ENOENT; + mlx5_glue->free_device_list(ibv_list); + return ibv_match; +} + +bool +mlx5_dev_pci_match(const struct mlx5_class_driver *drv, + const struct rte_device *dev) +{ + const struct rte_pci_device *pci_dev; + const struct rte_pci_id *id_table; + + if (!mlx5_dev_is_pci(dev)) + return false; + pci_dev = RTE_DEV_TO_PCI_CONST(dev); + for (id_table = drv->id_table; id_table->vendor_id != 0; + id_table++) { + /* Check if device's ids match the class driver's ids. */ + if (id_table->vendor_id != pci_dev->id.vendor_id && + id_table->vendor_id != RTE_PCI_ANY_ID) + continue; + if (id_table->device_id != pci_dev->id.device_id && + id_table->device_id != RTE_PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + pci_dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != RTE_PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + pci_dev->id.subsystem_device_id && + id_table->subsystem_device_id != RTE_PCI_ANY_ID) + continue; + if (id_table->class_id != pci_dev->id.class_id && + id_table->class_id != RTE_CLASS_ANY_ID) + continue; + return true; + } + return false; +} + +static int +mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) +{ + return mlx5_common_dev_probe(&pci_dev->device); +} + +static int +mlx5_common_pci_remove(struct rte_pci_device *pci_dev) +{ + return mlx5_common_dev_remove(&pci_dev->device); +} + +static int +mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_map(&pci_dev->device, addr, iova, len); +} + +static int +mlx5_common_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_unmap(&pci_dev->device, addr, iova, len); +} + +void +mlx5_common_driver_on_register_pci(struct mlx5_class_driver *driver) +{ + if (driver->id_table != NULL) { + if (pci_ids_table_update(driver->id_table) != 0) + return; + } + if (driver->probe_again) + mlx5_common_pci_driver.drv_flags |= RTE_PCI_DRV_PROBE_AGAIN; + if (driver->intr_lsc) + mlx5_common_pci_driver.drv_flags |= RTE_PCI_DRV_INTR_LSC; + if (driver->intr_rmv) + mlx5_common_pci_driver.drv_flags |= RTE_PCI_DRV_INTR_RMV; +} + +static struct rte_pci_driver mlx5_common_pci_driver = { + .driver = { + .name = MLX5_PCI_DRIVER_NAME, + }, + .probe = mlx5_common_pci_probe, + .remove = mlx5_common_pci_remove, + .dma_map = mlx5_common_pci_dma_map, + .dma_unmap = mlx5_common_pci_dma_unmap, +}; + void mlx5_common_pci_init(void) { const struct rte_pci_id empty_table[] = { @@ -535,7 +671,7 @@ void mlx5_common_pci_init(void) */ if (mlx5_pci_id_table == NULL && pci_ids_table_update(empty_table)) return; - rte_pci_register(&mlx5_pci_driver); + rte_pci_register(&mlx5_common_pci_driver); } RTE_FINI(mlx5_common_pci_finish) @@ -544,8 +680,9 @@ RTE_FINI(mlx5_common_pci_finish) /* Constructor doesn't register with PCI bus if it failed * to build the table. */ - rte_pci_unregister(&mlx5_pci_driver); + rte_pci_unregister(&mlx5_common_pci_driver); free(mlx5_pci_id_table); } } + RTE_PMD_EXPORT_NAME(mlx5_common_pci, __COUNTER__); diff --git a/drivers/common/mlx5/mlx5_common_private.h b/drivers/common/mlx5/mlx5_common_private.h new file mode 100644 index 0000000000..72df9aef35 --- /dev/null +++ b/drivers/common/mlx5/mlx5_common_private.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Mellanox Technologies, Ltd + */ + +#ifndef _MLX5_COMMON_PRIVATE_H_ +#define _MLX5_COMMON_PRIVATE_H_ + +#include + +#include "mlx5_common.h" + +#ifdef __cplusplus +extern "C" { +#endif /* __cplusplus */ + +/* Common bus driver: */ + +struct mlx5_common_device { + struct rte_device *dev; + TAILQ_ENTRY(mlx5_common_device) next; + uint32_t classes_loaded; +}; + +int mlx5_common_dev_probe(struct rte_device *eal_dev); +int mlx5_common_dev_remove(struct rte_device *eal_dev); +int mlx5_common_dev_dma_map(struct rte_device *dev, void *addr, uint64_t iova, + size_t len); +int mlx5_common_dev_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, + size_t len); + +/* Common PCI bus driver: */ + +void mlx5_common_driver_on_register_pci(struct mlx5_class_driver *driver); +bool mlx5_dev_pci_match(const struct mlx5_class_driver *drv, + const struct rte_device *dev); +struct ibv_device *mlx5_get_pci_ibv_device(const struct rte_pci_addr *); + +#ifdef __cplusplus +} +#endif /* __cplusplus */ + +#endif /* _MLX5_COMMON_PRIVATE_H_ */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index db4f13f1f7..3c21719975 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -3,6 +3,8 @@ INTERNAL { haswell_broadwell_cpu; + mlx5_class_driver_register; + mlx5_common_init; mlx5_common_verbs_reg_mr; # WINDOWS_NO_EXPORT @@ -10,6 +12,7 @@ INTERNAL { mlx5_create_mr_ext; + mlx5_dev_is_pci; mlx5_dev_to_pci_addr; # WINDOWS_NO_EXPORT mlx5_devx_alloc_uar; # WINDOWS_NO_EXPORT @@ -69,6 +72,8 @@ INTERNAL { mlx5_free; + mlx5_get_ibv_device; # WINDOWS_NO_EXPORT + mlx5_get_ifname_sysfs; # WINDOWS_NO_EXPORT mlx5_glue; From patchwork Thu May 27 13:37:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93483 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 23932A0548; Thu, 27 May 2021 15:39:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0E67A4110F; Thu, 27 May 2021 15:38:59 +0200 (CEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2070.outbound.protection.outlook.com [40.107.220.70]) by mails.dpdk.org (Postfix) with ESMTP id A32DE410F9 for ; Thu, 27 May 2021 15:38:56 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Bc1bJyAg9KgdFBKeV0Bdqgwpw3fuxP3PU3SAHGf3WKa/Ar4rdvVK6lEW2XCHBpoelxD8z02R++kBftZ4Pbv1Kr0CntNsS2yLVCbUKR4wfcVpTU4UvT56R4+HlU9OxaDtZQ9wuuTAIBch8C81K5dtqCjz4shjqLdCXhmg9TmIz32gJWRcVZachfHXRkjnV+7b5frtC+IF+8qlVm91tBFLFRTP/niM2xajsIg7KJqKRLKHhdOnWyoNqxxsb4Di4RxVpEAtv1QhkGz3ljdVmdsRDiX0iM+6Wdm1CmfcTNRV/RsMs2M5ElB7J2BshJ2eEHjaEs/hNqiTURVceUsx2OeiTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=geuzaiPLSwWztewXrfbHVLKXwLV81veeSz22ZuWlIVI=; b=h6lho5vCtFHztIfWZHIgMeWd/TauvCfi6wQRVv4mlVfq9mV/miGJuuC0zKbHsu3/uovD4lAf1j6YDKfqAMMDXBV+L8jXVK5s1PpG8c1HqJk+WjqI4EvrjPSwnbmi3dalZmnqmJUgIVV54LkxsqTm4ENNyjKX7gsgnsH4MqWz8dhFHqrt9HsWfPRA0n5iuJz7yaDODLC6tTr4dylr0gNXlbxGAhVZX3LeM77LQR6VxK5H4H9k5eso52H4t8eeVOtYKJ2b8stmHL72hZWVLxYV0XKzhPxOt7dpyAI8z5HesJM1YC/E1fvaGvZKOZ2fbDtaurFVEobZNjzUMs+LYjgFuQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=geuzaiPLSwWztewXrfbHVLKXwLV81veeSz22ZuWlIVI=; b=DPzt2vGXwYpjEdcvHcka8eh97WJRI4fcCSK9MHv9JzMTwT7Mei/3ySe09xc8XNZlOWflPYReGkLFL4EEGx43G+1nWBC0kFMveZk0gxMfe5Guh+V6owZtDF18ZZsPtoLKo8ttnStjCyQSzVkADPQwJ5ToGqU2IKP7pYi2rzFU/OrpMaGmfBpf27uydOMOIj6eEBvtfEhe14fi8/Oc8F4xF533JDa9r7Tgwvl5k2V9mDYMjkROhwbSn4gNnw/jsFqQQD7v6ZcJ+ExIxc9iRRXLIKWE2jqGcCivlyLdn7/orVYwBDYnSLHxmFIh/AB3ySxI9HNDfDeT9pcyru7pRVK/iw== Received: from CO2PR05CA0058.namprd05.prod.outlook.com (2603:10b6:102:2::26) by BL1PR12MB5144.namprd12.prod.outlook.com (2603:10b6:208:316::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20; Thu, 27 May 2021 13:38:55 +0000 Received: from CO1NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:102:2:cafe::8a) by CO2PR05CA0058.outlook.office365.com (2603:10b6:102:2::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.9 via Frontend Transport; Thu, 27 May 2021 13:38:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT063.mail.protection.outlook.com (10.13.175.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4150.30 via Frontend Transport; Thu, 27 May 2021 13:38:54 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:38:52 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad , Shahaf Shuler Date: Thu, 27 May 2021 16:37:47 +0300 Message-ID: <20210527133759.17401-3-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e06cc344-b2a2-4a2f-7d7c-08d92114c5b4 X-MS-TrafficTypeDiagnostic: BL1PR12MB5144: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:923; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /EZYA2GduTjZ7rAoBpeqJrv8te4ZyPAeqs+x56gl5EXNTK6l97riXgkAhFmSFuhHsHQMl5hteah3kkRFfl23qLWXwDsENSTBTyzgFhJuIEcck8CK56hqbp6q07Hs8tszXo0dmvVBiu42AKDBic60r7QiDtDszwu6UDQh4P12ZHNas8+OhF4LSIjmUPfkvk/2I48EkOScDulWulYFExakNL6AShhrVH6E23GXcuxL0AUEEZxbO0UCA2PaEEZScP1y7HZAVxtveRsJrnD2wefMFhUQwnuDRDkjyAeeseE4B05KnHer4zBTMsGE4m3c5F4h+v9hcV6TU7nue7Y1D0wfchg+3BNjgzMYRoaLGESxavdTCnxQTQlbgtXvPjTffT30GEsId4P2ZII1OUl8/MsDDmzOu1nZh8ZjAHWSsGXOfHCF4p+RX4s4wDtdRkxoYz6MxC82Hp93snL06ecQmjf2ZUJiVC2jcjHac2N8fFB0akOzklg+qy/524pvIlh1pp4r2Mkj/SvkYcZ7dLRssPPTutQrQwBIOTeLjcxjV05UiRzb2qU6A3BVtaKCaZ4A/n34qbR+nC+6p1xp0g0J22wp08Zy2xgfmLXxW9mY0rp77VEytEeSZfOUGDME6pEFv00xrl5+QFhh1FgsVBlwje9MwNLK6mQfblXPLn8PCbb9/9c= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(136003)(39860400002)(396003)(376002)(46966006)(36840700001)(6636002)(478600001)(8936002)(82310400003)(8676002)(6862004)(36860700001)(55016002)(36906005)(83380400001)(5660300002)(426003)(7696005)(336012)(1076003)(82740400003)(86362001)(7636003)(54906003)(70206006)(26005)(356005)(2906002)(6286002)(36756003)(107886003)(6666004)(316002)(37006003)(2616005)(186003)(47076005)(16526019)(4326008)(70586007); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:38:54.8609 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e06cc344-b2a2-4a2f-7d7c-08d92114c5b4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5144 Subject: [dpdk-dev] [RFC 02/14] common/mlx5: move description of PCI sysfs functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon The Linux-specific functions mlx5_get_pci_addr() and mlx5_get_ifname_sysfs() are better described in the .h file. The requirement for using mlx5_get_pci_addr() is explicited: the node /device must exist in the provided sysfs path. Signed-off-by: Thomas Monjalon --- drivers/common/mlx5/linux/mlx5_common_os.c | 22 ------------------ drivers/common/mlx5/mlx5_common.h | 26 ++++++++++++++++++++++ 2 files changed, 26 insertions(+), 22 deletions(-) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index ea0b71e425..ea6001e6b2 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -23,17 +23,6 @@ const struct mlx5_glue *mlx5_glue; #endif -/** - * Get PCI information by sysfs device path. - * - * @param dev_path - * Pointer to device sysfs folder name. - * @param[out] pci_addr - * PCI bus address output buffer. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ int mlx5_dev_to_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr) @@ -159,17 +148,6 @@ mlx5_translate_port_name(const char *port_name_in, port_info_out->name_type = MLX5_PHYS_PORT_NAME_TYPE_UNKNOWN; } -/** - * Get kernel interface name from IB device path. - * - * @param[in] ibdev_path - * Pointer to IB device path. - * @param[out] ifname - * Interface name output buffer. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ int mlx5_get_ifname_sysfs(const char *ibdev_path, char *ifname) { diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 0e67a36d0c..62a0dc4bad 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -208,8 +208,34 @@ check_cqe(volatile struct mlx5_cqe *cqe, const uint16_t cqes_n, return MLX5_CQE_STATUS_SW_OWN; } +/* + * Get PCI address from sysfs of a PCI-related device. + * + * @param[in] dev_path + * The sysfs path should not point to the direct plain PCI device. + * Instead, the node "/device/" is used to access the real device. + * @param[out] pci_addr + * Parsed PCI address. + * + * @return + * - 0 on success. + * - Negative value and rte_errno is set otherwise. + */ __rte_internal int mlx5_dev_to_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr); + +/* + * Get kernel network interface name from sysfs IB device path. + * + * @param[in] ibdev_path + * The sysfs path to IB device. + * @param[out] ifname + * Interface name output of size IF_NAMESIZE. + * + * @return + * - 0 on success. + * - Negative value and rte_errno is set otherwise. + */ __rte_internal int mlx5_get_ifname_sysfs(const char *ibdev_path, char *ifname); From patchwork Thu May 27 13:37:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93485 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BA236A0548; Thu, 27 May 2021 15:39:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 46C3741116; Thu, 27 May 2021 15:39:03 +0200 (CEST) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1anam02on2048.outbound.protection.outlook.com [40.107.96.48]) by mails.dpdk.org (Postfix) with ESMTP id EF28941108 for ; Thu, 27 May 2021 15:39:00 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=amlNhdot8iSdLoGAZeGY5DfjpECfvpKMYVyuobHQ0LcUkfbtDVBgacJSXi4rFdwzXORrmASAD7/xPXeJtuGVH5T8tFB1zYtzZSZA85j/u/lomzqh5AmhDyQmCOqE27LmjYCuxDL+aU6dcv1HliN6VCzAESFspCnFMNkTGtxXair09n9AhIAgiZnb8WRcLXxhKtRCWFazRBeMpoRCgyC5QDuYDo5tehdmaKjovvEQBDCSLjR2Q/DnsLB3Mk6RQ2z2GvPuJWwYVWd6WfP+oh0EBN5p7n/uYXL/QK/YUt7QsEk99gj9pYfm+fws2C3WbqMmV+VghGMhW63rnKLqYEy62w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ygJtUsWkzw9JeGHR+Z1k1HRo4oNdOcMbkHPzP7l5Kz8=; b=huLqWXGQ2lYNASIVoDKNExGzzt7YGE1WvL48sewRT85SZ5t0Nr2ThUUgSeRFMi1SPOPmrV3LxZUjwdD+qnLVTHdAvktMUl8Lx6TxPj/R8K9VwRWS3uJfmIw+VG6f1OE6BNz4hZ19YVJPKK8rBjMZuZLHbAm6EUttcInObwpE9indbdQmlTU29l5H9w1jJn55I/IklETGBhHZgIc0/YN+u4PSOWda2AEP53JB4iIN+x1DNvnxNemzBFdNwXYtUZVsbHqMiVqeAIVKVxKwCmTN56BPzhTjbosZLFiScVJljADKmzv6QDhRGwFBrq33IvQAyvu72WuBgmMb3WKaK70vXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ygJtUsWkzw9JeGHR+Z1k1HRo4oNdOcMbkHPzP7l5Kz8=; b=rOk0fwMXXlUX+npVomVsPR9asJ+1xTFQTVfOu4eYLfO6BS0ookOd7ufDmA7hgdPFQdgkVGWh0wo3JaQ5psjEvPH0B5rEPCeQfpzy57GI/LEiZ7IDLx8nkYo1LMYgwKsQIkEThReQXU4QceYSqDFDy/jveLrcR4bBR8YGIAJracYEDjhRJQwDdVZsLtaPQofW72Rzd1527y3ntMtfS6PIqwQrFA7uoc3vIAHHCRL4bpMgVtX878/sJ3ShSRAD4TGYV8RbPjFd8uShm9NEqIvTKHOavEd057RWwWZO+JcsFdiZ9fcWUUycP9AC+re5rfxhckVazgpaJnQL/4O4HoVkcQ== Received: from CO2PR05CA0054.namprd05.prod.outlook.com (2603:10b6:102:2::22) by BYAPR12MB3110.namprd12.prod.outlook.com (2603:10b6:a03:d7::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4150.25; Thu, 27 May 2021 13:38:58 +0000 Received: from CO1NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:102:2:cafe::30) by CO2PR05CA0054.outlook.office365.com (2603:10b6:102:2::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.9 via Frontend Transport; Thu, 27 May 2021 13:38:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT063.mail.protection.outlook.com (10.13.175.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4150.30 via Frontend Transport; Thu, 27 May 2021 13:38:56 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:38:54 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler Date: Thu, 27 May 2021 16:37:48 +0300 Message-ID: <20210527133759.17401-4-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 11e79f94-00d1-4eb7-2477-08d92114c6d7 X-MS-TrafficTypeDiagnostic: BYAPR12MB3110: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:69; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4tU0rcAQGq95YmIKy3ERqGNFuCyFMyAL+H9tmwWhPZ4fxP6vCN4p9Lx8InHOKiZO2sbGwuCIMLaY1MPyvAMazQ21FJgN3phxvcMDmc56ppsEbpUrfn46EEmZkwVXf8sI0f0QUVRqP/+2NRJydsQ+uaopmrFhs+Y8D227BPpdrOwXrN0qSZVKUaxTSDPNFaBGMQkdf4MUEuSSyywBnSjIHxsNs+LQ+kJh+w4q/2wagnDL8N6jyBqL27gTEO+ChJ9EJW9XKL3VF+44FxtMbznuzbew7wxzojUXrNpwuf+WU+hhdRoD+fvr0AtnTlODoRVuSyBA5VSLq6jNUJ/9qjoKWjmiVq6mtQtf61FwrP4q5t9G5xzRumPixYjcwk8SfTtQDVqg2Hh4SB0/748GbfZQbEu/PMdckO+XgZILFDPdU/456M7IBK6Jl3h9sAHKFeR2qv2hVudCPUim5zfWN1z10P+t6wiwkXk3MWfMQuXufo+sCfRDTyFvBpj7LnSmXUH1WMvgL2p87ZoqgX9lISDjbMO9bcyjmTWLCabHoaFeui01dexXKk7ImhoJ092lCXFG1O1eVREtXRj3DgSJnW6XQIXqJXkRWVOjybiFKJIwBzgtj6G+Rcl1hz+DRpsRtX+21SZdVHUkpOvrHK4XFLBUfeb/NrRCn7AIgZSG+H+Z3pY= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(376002)(396003)(346002)(136003)(36840700001)(46966006)(6286002)(37006003)(16526019)(36860700001)(316002)(7636003)(36906005)(82740400003)(54906003)(478600001)(47076005)(30864003)(86362001)(70586007)(70206006)(26005)(7696005)(6666004)(1076003)(8936002)(6862004)(2616005)(107886003)(5660300002)(4326008)(8676002)(55016002)(356005)(426003)(336012)(6636002)(2906002)(186003)(36756003)(83380400001)(82310400003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:38:56.7647 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 11e79f94-00d1-4eb7-2477-08d92114c6d7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3110 Subject: [dpdk-dev] [RFC 03/14] net/mlx5: remove PCI dependency X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support more bus types, remove PCI dependency where possible. Signed-off-by: Xueming Li --- drivers/common/mlx5/mlx5_common_pci.c | 6 +-- drivers/net/mlx5/linux/mlx5_ethdev_os.c | 2 +- drivers/net/mlx5/linux/mlx5_os.c | 4 +- drivers/net/mlx5/mlx5.c | 53 ++++++++++++++++--------- drivers/net/mlx5/mlx5.h | 8 ++-- drivers/net/mlx5/mlx5_ethdev.c | 2 +- drivers/net/mlx5/mlx5_mr.c | 14 +++---- drivers/net/mlx5/mlx5_trigger.c | 12 +++--- drivers/net/mlx5/mlx5_txq.c | 2 +- drivers/net/mlx5/windows/mlx5_os.c | 6 +-- 10 files changed, 58 insertions(+), 51 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 53090173a2..5a824dd50f 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -534,11 +534,7 @@ mlx5_pci_driver_register(struct mlx5_pci_driver *driver) bool mlx5_dev_is_pci(const struct rte_device *dev) { - struct rte_devargs *da = dev->devargs; - - if (da == NULL || da->bus == NULL) - return false; - return strcmp(da->bus->name, "pci") == 0; + return strcmp(dev->bus->name, "pci") == 0; } struct ibv_device * diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c index ddc1371aa9..6fdb310129 100644 --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c @@ -346,7 +346,7 @@ mlx5_find_master_dev(struct rte_eth_dev *dev) priv = dev->data->dev_private; domain_id = priv->domain_id; MLX5_ASSERT(priv->representor); - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, dev) { struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; if (opriv && diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 534a56a555..e8a97d4337 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -925,6 +925,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, sh = mlx5_alloc_shared_dev_ctx(spawn, config); if (!sh) return NULL; + sh->numa_node = dpdk_dev->numa_node; config->devx = sh->devx; #ifdef HAVE_MLX5DV_DR_ACTION_DEST_DEVX_TIR config->dest_tir = 1; @@ -1133,7 +1134,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, * Look for sibling devices in order to reuse their switch domain * if any, otherwise allocate one. */ - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, NULL) { const struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; @@ -2556,7 +2557,6 @@ mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn, int dbmap_env; int err = 0; - sh->numa_node = spawn->pci_dev->device.numa_node; pthread_mutex_init(&sh->txpp.mutex, NULL); /* * Configure environment variable "MLX5_BF_SHUT_UP" diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index d0faa45944..95ac43268b 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1185,7 +1185,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, */ err = mlx5_mr_btree_init(&sh->share_cache.cache, MLX5_MR_BTREE_CACHE_N * 2, - spawn->pci_dev->device.numa_node); + sh->numa_node); if (err) { err = rte_errno; goto error; @@ -1620,7 +1620,7 @@ mlx5_dev_close(struct rte_eth_dev *dev) unsigned int c = 0; uint16_t port_id; - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, dev) { struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; @@ -2057,7 +2057,8 @@ void mlx5_set_min_inline(struct mlx5_dev_spawn_data *spawn, struct mlx5_dev_config *config) { - if (config->txq_inline_min != MLX5_ARG_UNSET) { + if (config->txq_inline_min != MLX5_ARG_UNSET && + spawn->pci_dev != NULL) { /* Application defines size of inlined data explicitly. */ switch (spawn->pci_dev->id.device_id) { case PCI_DEVICE_ID_MELLANOX_CONNECTX4: @@ -2124,6 +2125,11 @@ mlx5_set_min_inline(struct mlx5_dev_spawn_data *spawn, } } } + if (spawn->pci_dev == NULL) { + if (config->txq_inline_min == MLX5_ARG_UNSET) + config->txq_inline_min = MLX5_INLINE_HSIZE_NONE; + goto exit; + } /* * We get here if we are unable to deduce * inline data size with DevX. Try PCI ID @@ -2258,7 +2264,7 @@ mlx5_dev_check_sibling_config(struct mlx5_priv *priv, if (sh->refcnt == 1) return 0; /* Find the device with shared context. */ - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, NULL) { struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; @@ -2286,33 +2292,42 @@ mlx5_dev_check_sibling_config(struct mlx5_priv *priv, /** * Look for the ethernet device belonging to mlx5 driver. + * If device specified, look for ports belong to same PCI/bonding. * * @param[in] port_id * port_id to start looking for device. - * @param[in] pci_dev - * Pointer to the hint PCI device. When device is being probed - * the its siblings (master and preceding representors might - * not have assigned driver yet (because the mlx5_os_pci_probe() - * is not completed yet, for this case match on hint PCI - * device may be used to detect sibling device. + * @param[in] odev + * Device to detect sibling. * * @return * port_id of found device, RTE_MAX_ETHPORT if not found. */ uint16_t -mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev) +mlx5_eth_find_next(uint16_t port_id, struct rte_eth_dev *odev) { - while (port_id < RTE_MAX_ETHPORTS) { + const struct mlx5_priv *opriv = NULL; + + if (odev) + opriv = odev->data->dev_private; + for ( ; port_id < RTE_MAX_ETHPORTS; port_id++) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct mlx5_priv *priv; - if (dev->state != RTE_ETH_DEV_UNUSED && - dev->device && - (dev->device == &pci_dev->device || - (dev->device->driver && - dev->device->driver->name && - !strcmp(dev->device->driver->name, MLX5_PCI_DRIVER_NAME)))) + if (dev->state == RTE_ETH_DEV_UNUSED) + continue; + priv = dev->data->dev_private; + if (odev != NULL) { + /* odev specified, find devices on same PCI/bonding. */ + if (opriv->sh == priv->sh || + odev->device == dev->device) + break; + } else if (dev->device != NULL && dev->device->driver && + dev->device->driver->name && + !strcmp(dev->device->driver->name, + MLX5_PCI_DRIVER_NAME)) { + /* odev not specified, found all mlx5 devices. */ break; - port_id++; + } } if (port_id >= RTE_MAX_ETHPORTS) return RTE_MAX_ETHPORTS; diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 32b2817bf2..29a9b18887 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1404,16 +1404,16 @@ int mlx5_proc_priv_init(struct rte_eth_dev *dev); void mlx5_proc_priv_uninit(struct rte_eth_dev *dev); int mlx5_udp_tunnel_port_add(struct rte_eth_dev *dev, struct rte_eth_udp_tunnel *udp_tunnel); -uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_pci_device *pci_dev); +uint16_t mlx5_eth_find_next(uint16_t port_id, struct rte_eth_dev *odev); int mlx5_dev_close(struct rte_eth_dev *dev); bool mlx5_is_hpf(struct rte_eth_dev *dev); void mlx5_age_event_prepare(struct mlx5_dev_ctx_shared *sh); /* Macro to iterate over all valid ports for mlx5 driver. */ -#define MLX5_ETH_FOREACH_DEV(port_id, pci_dev) \ - for (port_id = mlx5_eth_find_next(0, pci_dev); \ +#define MLX5_ETH_FOREACH_DEV(port_id, dev) \ + for (port_id = mlx5_eth_find_next(0, dev); \ port_id < RTE_MAX_ETHPORTS; \ - port_id = mlx5_eth_find_next(port_id + 1, pci_dev)) + port_id = mlx5_eth_find_next(port_id + 1, dev)) int mlx5_args(struct mlx5_dev_config *config, struct rte_devargs *devargs); struct mlx5_dev_ctx_shared * mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 90baee5aa4..4654b85844 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -335,7 +335,7 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) if (priv->representor) { uint16_t port_id; - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, dev) { struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index e791b6338d..fcb475582d 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -290,23 +290,23 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, } /** - * Finds the first ethdev that match the pci device. + * Finds the first ethdev that match the device. * The existence of multiple ethdev per pci device is only with representors. * On such case, it is enough to get only one of the ports as they all share * the same ibv context. * - * @param pdev - * Pointer to the PCI device. + * @param dev + * Pointer to the device. * * @return * Pointer to the ethdev if found, NULL otherwise. */ static struct rte_eth_dev * -pci_dev_to_eth_dev(struct rte_pci_device *pdev) +dev_to_eth_dev(struct rte_device *dev) { uint16_t port_id; - port_id = rte_eth_find_next_of(0, &pdev->device); + port_id = rte_eth_find_next_of(0, dev); if (port_id == RTE_MAX_ETHPORTS) return NULL; return &rte_eth_devices[port_id]; @@ -336,7 +336,7 @@ mlx5_dma_map(struct rte_pci_device *pdev, void *addr, struct mlx5_priv *priv; struct mlx5_dev_ctx_shared *sh; - dev = pci_dev_to_eth_dev(pdev); + dev = dev_to_eth_dev(&pdev->device); if (!dev) { DRV_LOG(WARNING, "unable to find matching ethdev " "to PCI device %p", (void *)pdev); @@ -386,7 +386,7 @@ mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, struct mlx5_mr *mr; struct mr_cache_entry entry; - dev = pci_dev_to_eth_dev(pdev); + dev = dev_to_eth_dev(&pdev->device); if (!dev) { DRV_LOG(WARNING, "unable to find matching ethdev " "to PCI device %p", (void *)pdev); diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index ae7fcca229..6c8a64ce03 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -697,7 +697,7 @@ mlx5_hairpin_bind_single_port(struct rte_eth_dev *dev, uint16_t rx_port) uint32_t explicit; uint16_t rx_queue; - if (mlx5_eth_find_next(rx_port, priv->pci_dev) != rx_port) { + if (mlx5_eth_find_next(rx_port, dev) != rx_port) { rte_errno = ENODEV; DRV_LOG(ERR, "Rx port %u does not belong to mlx5", rx_port); return -rte_errno; @@ -835,7 +835,7 @@ mlx5_hairpin_unbind_single_port(struct rte_eth_dev *dev, uint16_t rx_port) int ret; uint16_t cur_port = priv->dev_data->port_id; - if (mlx5_eth_find_next(rx_port, priv->pci_dev) != rx_port) { + if (mlx5_eth_find_next(rx_port, dev) != rx_port) { rte_errno = ENODEV; DRV_LOG(ERR, "Rx port %u does not belong to mlx5", rx_port); return -rte_errno; @@ -893,7 +893,6 @@ mlx5_hairpin_bind(struct rte_eth_dev *dev, uint16_t rx_port) { int ret = 0; uint16_t p, pp; - struct mlx5_priv *priv = dev->data->dev_private; /* * If the Rx port has no hairpin configuration with the current port, @@ -902,7 +901,7 @@ mlx5_hairpin_bind(struct rte_eth_dev *dev, uint16_t rx_port) * information updating. */ if (rx_port == RTE_MAX_ETHPORTS) { - MLX5_ETH_FOREACH_DEV(p, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(p, dev) { ret = mlx5_hairpin_bind_single_port(dev, p); if (ret != 0) goto unbind; @@ -912,7 +911,7 @@ mlx5_hairpin_bind(struct rte_eth_dev *dev, uint16_t rx_port) return mlx5_hairpin_bind_single_port(dev, rx_port); } unbind: - MLX5_ETH_FOREACH_DEV(pp, priv->pci_dev) + MLX5_ETH_FOREACH_DEV(pp, dev) if (pp < p) mlx5_hairpin_unbind_single_port(dev, pp); return ret; @@ -927,10 +926,9 @@ mlx5_hairpin_unbind(struct rte_eth_dev *dev, uint16_t rx_port) { int ret = 0; uint16_t p; - struct mlx5_priv *priv = dev->data->dev_private; if (rx_port == RTE_MAX_ETHPORTS) - MLX5_ETH_FOREACH_DEV(p, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(p, dev) { ret = mlx5_hairpin_unbind_single_port(dev, p); if (ret != 0) return ret; diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 3e5e94444b..f68c0c61a9 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -816,7 +816,7 @@ txq_set_params(struct mlx5_txq_ctrl *txq_ctrl) if (config->txqs_inline == MLX5_ARG_UNSET) txqs_inline = #if defined(RTE_ARCH_ARM64) - (priv->pci_dev->id.device_id == + (priv->pci_dev && priv->pci_dev->id.device_id == PCI_DEVICE_ID_MELLANOX_CONNECTX5BF) ? MLX5_INLINE_MAX_TXQS_BLUEFIELD : #endif diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index 3fe3f55f49..1e3260c6b5 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -229,9 +229,6 @@ mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn, struct mlx5_context *mlx5_ctx; pthread_mutex_init(&sh->txpp.mutex, NULL); - /* Set numa node from pci probe */ - sh->numa_node = spawn->pci_dev->device.numa_node; - /* Try to open device with DevX */ rte_errno = 0; sh->ctx = mlx5_glue->open_device(spawn->phys_dev); @@ -344,6 +341,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, sh = mlx5_alloc_shared_dev_ctx(spawn, config); if (!sh) return NULL; + sh->numa_node = dpdk_dev->numa_node; config->devx = sh->devx; /* Initialize the shutdown event in mlx5_dev_spawn to * support mlx5_is_removed for Windows. @@ -393,7 +391,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, * Look for sibling devices in order to reuse their switch domain * if any, otherwise allocate one. */ - MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) { + MLX5_ETH_FOREACH_DEV(port_id, NULL) { const struct mlx5_priv *opriv = rte_eth_devices[port_id].data->dev_private; From patchwork Thu May 27 13:37:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93484 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5F513A0548; Thu, 27 May 2021 15:39:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2A85641108; Thu, 27 May 2021 15:39:02 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2042.outbound.protection.outlook.com [40.107.237.42]) by mails.dpdk.org (Postfix) with ESMTP id D207141107 for ; Thu, 27 May 2021 15:39:00 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LXJJpz6f0c1Z9j0pd8bx8ePA+8HWMTHIdDao+OgQ7J4WXRJNJjmDQzgICp4TRy5UjL8bEPduAN5PxyuQetI71QM0ZYqCpiUaAvHmcKQgias+C9Xtcihz40w0z1rYj7n1o77mfvtXJB5WcYKjgUma+uQP6Ahx8/Z6RadmMSeQhbPZFDGCXdwGYaFSdDY176LjZr13UeQhNsdiS+uUFFJsvZ0P1J9CmRcrk/1nGjhIqlmv62jNf926rBEcMcLa1fbcby97Zm+OJJDxNWiRBpGEurAlUTvnZQjwvHsG0el3Sm6qMxIcQDQaDnL9mibQka5uTsDu7vMY8xaVLpHCPefPFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yxsZyRUAmzxvPaFxymyfmIWA7kl5nIz7Kj3AzAtODPE=; b=DPGVi5wHx3CXEpw9IHW5O23dLEHgl+OlMBDqwxQK5a4dFhPlUPVtBmiWsSixvrBYAvGVpVhgv9raR0O9T1WkVPQIPzhoH42iSrS503vorfMQKxc2lY3/dcLhTXYKiYHl1c5aM7m/ldymuyBOdq4vw4jUIBqIwXY7eJqCQdHrgr6rUvPZ+dxrLuadkapO76+YplfLXN5SQxytoZBXLbbbdznet8CnjeP5jfn/ViP3iq0akKGWOjywy//Sgk/+L+/MRqOoCRYnm747xFv5Rg2EWCR7UTtsbyNrmJUbydsa8oxOrJC4KZ1ZxWwtUs8rHxTXVndXZPOkAaTQr50k+ZeFxQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yxsZyRUAmzxvPaFxymyfmIWA7kl5nIz7Kj3AzAtODPE=; b=osTn3DO1U6IpwZvsTw76EUhqDktmEBRp1/e17jEQp78tsjmVFfoN0msgZwSt3rFRLmcAYvEBUIenqtlDtqztUyt3FWSnmnys5RhAYYW0sKYeQ/ZGzJvjOjOXPAvmSIGXsJeppvVTPTFpR7Ye4B0luVIFu9DvnlK6fIjKcRdrNiklCztK+r6xfIUMc+R4lPP3qDADjdxvlLZSbexRRFdEcu5g0NmpVwoeVAPpgUNEpGjLD2bB9nM9YqR71roGJZAyYjArV4DtOcKC8saVBNOiNwBRn3WOF+V2nDTlANaiR7ullcO+9F6XVzUitqEzM/V6C/U5nmEMtQbor3Msmg43BQ== Received: from CO2PR05CA0055.namprd05.prod.outlook.com (2603:10b6:102:2::23) by BY5PR12MB4856.namprd12.prod.outlook.com (2603:10b6:a03:1d5::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20; Thu, 27 May 2021 13:38:59 +0000 Received: from CO1NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:102:2:cafe::ab) by CO2PR05CA0055.outlook.office365.com (2603:10b6:102:2::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.10 via Frontend Transport; Thu, 27 May 2021 13:38:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT063.mail.protection.outlook.com (10.13.175.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4150.30 via Frontend Transport; Thu, 27 May 2021 13:38:58 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:38:56 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Anatoly Burakov Date: Thu, 27 May 2021 16:37:49 +0300 Message-ID: <20210527133759.17401-5-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f941a489-5a56-492a-c3fd-08d92114c7eb X-MS-TrafficTypeDiagnostic: BY5PR12MB4856: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:185; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: J5dBUSPPARZk2QIqF2SL/Pjmj8XLanD4CGzN5JmFcGKSXNw+3MsDOowl8gjVoBaMEgQO3daTtW7MvkBvUobrvqzuRCk80RiLLeLE+Tq7IGkQMR2puzhFi3WV8FOozNIKqRFVIf7ldz3Ox4aCEq77sPa0CaS3sVN51evK8fsdkozDeQBmh5IJ5FMIGUj2N76PTh1VymlKzQM7D7m1d5grsueZp2Zbz6/aLggql/Zl+eJQn/fKKa6Nh33E3x8N/saty/xXVSF6PJ0cOeFmOiDQtCLuBQjsMcmdDKgQffCFT5JgGI6t4tjdwRGbXHlJV3VP8C1jBd94QXLFOBo1PUhXdAhDMs/Tq46scm8p56R7U4hffg/WhD1379aKwqwHCAoV750ZdSZwZpXT0Yw0Cb0UMBgeps4UywYwHr39h51OfdSF5Nz2QGfZ+qbIU04QE5NS/q49f3w8BYQHkM+oWtc4pDRhndG13Rc4vCvIfV0v8T59WBX4RqmsWWsCCDFdfZO9ZD30KslUKcONL2iz0j/yN46BLmyWR0tirhPqDIOOfRKZPeB/CdhYU7uQH0yj55EUsvK6byAQqU9DsDSdzmdZicP+0+Q2vcl8ND9axqGsYdUGb0/1X2EQcxwYDprtVNht//mMghrskUNj4RmwTYMb4kjc2yYEcz82wz5WoLCkB78= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(396003)(346002)(39860400002)(136003)(36840700001)(46966006)(82740400003)(426003)(2616005)(336012)(86362001)(82310400003)(1076003)(356005)(16526019)(30864003)(7696005)(186003)(6666004)(70586007)(70206006)(7636003)(36860700001)(54906003)(5660300002)(26005)(47076005)(8936002)(36906005)(8676002)(6636002)(4326008)(55016002)(2906002)(6862004)(37006003)(83380400001)(478600001)(316002)(6286002)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:38:58.5696 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f941a489-5a56-492a-c3fd-08d92114c7eb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4856 Subject: [dpdk-dev] [RFC 04/14] net/mlx5: migrate to bus-agnostic common driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li --- drivers/net/mlx5/linux/mlx5_os.c | 46 +++++++++++++++++++++--------- drivers/net/mlx5/linux/mlx5_os.h | 3 -- drivers/net/mlx5/mlx5.c | 42 ++++++++++++--------------- drivers/net/mlx5/mlx5.h | 3 +- drivers/net/mlx5/mlx5_mr.c | 36 +++++++++++------------ drivers/net/mlx5/mlx5_rxtx.h | 9 +++--- drivers/net/mlx5/windows/mlx5_os.c | 14 ++++----- 7 files changed, 80 insertions(+), 73 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index e8a97d4337..e8e6b0d5c9 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1977,14 +1977,6 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, struct mlx5_bond_info bond_info; int ret = -1; - if (rte_eal_process_type() == RTE_PROC_PRIMARY) - mlx5_pmd_socket_init(); - ret = mlx5_init_once(); - if (ret) { - DRV_LOG(ERR, "unable to init PMD global data: %s", - strerror(rte_errno)); - return -rte_errno; - } errno = 0; ibv_list = mlx5_glue->get_device_list(&ret); if (!ibv_list) { @@ -2417,21 +2409,18 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, } /** - * DPDK callback to register a PCI device. + * Callback to register a PCI device. * * This function spawns Ethernet devices out of a given PCI device. * - * @param[in] pci_drv - * PCI driver structure (mlx5_driver). * @param[in] pci_dev * PCI device information. * * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ -int -mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) +static int +mlx5_os_pci_probe(struct rte_pci_device *pci_dev) { struct rte_eth_devargs eth_da = { .type = RTE_ETH_REPRESENTOR_NONE }; int ret = 0; @@ -2470,6 +2459,35 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, return ret; } +/** + * Common bus driver callback to probe a device. + * + * This function probe PCI bus device(s). + * + * @param[in] dev + * Pointer to the generic device. + * + * @return + * 0 on success, the function cannot fail. + */ +int +mlx5_os_net_probe(struct rte_device *dev) +{ + int ret; + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + mlx5_pmd_socket_init(); + ret = mlx5_init_once(); + if (ret) { + DRV_LOG(ERR, "unable to init PMD global data: %s", + strerror(rte_errno)); + return -rte_errno; + } + if (mlx5_dev_is_pci(dev)) + return mlx5_os_pci_probe(RTE_DEV_TO_PCI(dev)); + return 0; +} + static int mlx5_config_doorbell_mapping_env(const struct mlx5_dev_config *config) { diff --git a/drivers/net/mlx5/linux/mlx5_os.h b/drivers/net/mlx5/linux/mlx5_os.h index 4ae7d0ef47..af7cbeb418 100644 --- a/drivers/net/mlx5/linux/mlx5_os.h +++ b/drivers/net/mlx5/linux/mlx5_os.h @@ -19,7 +19,4 @@ enum { #define MLX5_NAMESIZE IF_NAMESIZE -#define PCI_DRV_FLAGS (RTE_PCI_DRV_INTR_LSC | \ - RTE_PCI_DRV_INTR_RMV | \ - RTE_PCI_DRV_PROBE_AGAIN) #endif /* RTE_PMD_MLX5_OS_H_ */ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 95ac43268b..3defdb2db3 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -12,9 +12,7 @@ #include #include -#include #include -#include #include #include #include @@ -22,13 +20,13 @@ #include #include #include +#include #include #include #include #include #include -#include #include #include "mlx5_defs.h" @@ -2335,23 +2333,23 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_eth_dev *odev) } /** - * DPDK callback to remove a PCI device. + * Callback to remove a device. * - * This function removes all Ethernet devices belong to a given PCI device. + * This function removes all Ethernet devices belong to a given device. * - * @param[in] pci_dev - * Pointer to the PCI device. + * @param[in] dev + * Pointer to the generic device. * * @return * 0 on success, the function cannot fail. */ static int -mlx5_pci_remove(struct rte_pci_device *pci_dev) +mlx5_net_remove(struct rte_device *dev) { uint16_t port_id; int ret = 0; - RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) { + RTE_ETH_FOREACH_DEV_OF(port_id, dev) { /* * mlx5_dev_close() is not registered to secondary process, * call the close function explicitly for secondary process. @@ -2442,19 +2440,17 @@ static const struct rte_pci_id mlx5_pci_id_map[] = { } }; -static struct mlx5_pci_driver mlx5_driver = { - .driver_class = MLX5_CLASS_NET, - .pci_driver = { - .driver = { - .name = MLX5_PCI_DRIVER_NAME, - }, - .id_table = mlx5_pci_id_map, - .probe = mlx5_os_pci_probe, - .remove = mlx5_pci_remove, - .dma_map = mlx5_dma_map, - .dma_unmap = mlx5_dma_unmap, - .drv_flags = PCI_DRV_FLAGS, - }, +static struct mlx5_class_driver mlx5_net_driver = { + .drv_class = MLX5_CLASS_NET, + .name = "mlx5_eth", + .id_table = mlx5_pci_id_map, + .probe = mlx5_os_net_probe, + .remove = mlx5_net_remove, + .dma_map = mlx5_net_dma_map, + .dma_unmap = mlx5_net_dma_unmap, + .probe_again = 1, + .intr_lsc = 1, + .intr_rmv = 1, }; /* Initialize driver log type. */ @@ -2472,7 +2468,7 @@ RTE_INIT(rte_mlx5_pmd_init) mlx5_set_cksum_table(); mlx5_set_swp_types_table(); if (mlx5_glue) - mlx5_pci_driver_register(&mlx5_driver); + mlx5_class_driver_register(&mlx5_net_driver); } RTE_PMD_EXPORT_NAME(net_mlx5, __COUNTER__); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 29a9b18887..27bb34e827 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1680,8 +1680,7 @@ int mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn, const struct mlx5_dev_config *config, struct mlx5_dev_ctx_shared *sh); int mlx5_os_get_pdn(void *pd, uint32_t *pdn); -int mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev); +int mlx5_os_net_probe(struct rte_device *dev); void mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh); void mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh); void mlx5_os_set_reg_mr_cb(mlx5_reg_mr_t *reg_mr_cb, diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index fcb475582d..2bce302eb5 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -7,7 +7,6 @@ #include #include #include -#include #include #include @@ -313,10 +312,10 @@ dev_to_eth_dev(struct rte_device *dev) } /** - * DPDK callback to DMA map external memory to a PCI device. + * Callback to DMA map external memory to a device. * - * @param pdev - * Pointer to the PCI device. + * @param rte_dev + * Pointer to the generic device. * @param addr * Starting virtual address of memory to be mapped. * @param iova @@ -328,18 +327,18 @@ dev_to_eth_dev(struct rte_device *dev) * 0 on success, negative value on error. */ int -mlx5_dma_map(struct rte_pci_device *pdev, void *addr, - uint64_t iova __rte_unused, size_t len) +mlx5_net_dma_map(struct rte_device *rte_dev, void *addr, + uint64_t iova __rte_unused, size_t len) { struct rte_eth_dev *dev; struct mlx5_mr *mr; struct mlx5_priv *priv; struct mlx5_dev_ctx_shared *sh; - dev = dev_to_eth_dev(&pdev->device); + dev = dev_to_eth_dev(rte_dev); if (!dev) { DRV_LOG(WARNING, "unable to find matching ethdev " - "to PCI device %p", (void *)pdev); + "to device %s", rte_dev->name); rte_errno = ENODEV; return -1; } @@ -362,10 +361,10 @@ mlx5_dma_map(struct rte_pci_device *pdev, void *addr, } /** - * DPDK callback to DMA unmap external memory to a PCI device. + * Callback to DMA unmap external memory to a device. * - * @param pdev - * Pointer to the PCI device. + * @param rte_dev + * Pointer to the generic device. * @param addr * Starting virtual address of memory to be unmapped. * @param iova @@ -377,8 +376,8 @@ mlx5_dma_map(struct rte_pci_device *pdev, void *addr, * 0 on success, negative value on error. */ int -mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, - uint64_t iova __rte_unused, size_t len __rte_unused) +mlx5_net_dma_unmap(struct rte_device *rte_dev, void *addr, + uint64_t iova __rte_unused, size_t len __rte_unused) { struct rte_eth_dev *dev; struct mlx5_priv *priv; @@ -386,10 +385,10 @@ mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, struct mlx5_mr *mr; struct mr_cache_entry entry; - dev = dev_to_eth_dev(&pdev->device); + dev = dev_to_eth_dev(rte_dev); if (!dev) { - DRV_LOG(WARNING, "unable to find matching ethdev " - "to PCI device %p", (void *)pdev); + DRV_LOG(WARNING, "unable to find matching ethdev to device %s", + rte_dev->name); rte_errno = ENODEV; return -1; } @@ -399,9 +398,8 @@ mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, mr = mlx5_mr_lookup_list(&sh->share_cache, &entry, (uintptr_t)addr); if (!mr) { rte_rwlock_read_unlock(&sh->share_cache.rwlock); - DRV_LOG(WARNING, "address 0x%" PRIxPTR " wasn't registered " - "to PCI device %p", (uintptr_t)addr, - (void *)pdev); + DRV_LOG(WARNING, "address 0x%" PRIxPTR " wasn't registered to device %s", + (uintptr_t)addr, rte_dev->name); rte_errno = EINVAL; return -1; } diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index e168dd46f9..ad1144e218 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -16,7 +16,6 @@ #include #include #include -#include #include #include @@ -48,10 +47,10 @@ int mlx5_queue_state_modify(struct rte_eth_dev *dev, /* mlx5_mr.c */ void mlx5_mr_flush_local_cache(struct mlx5_mr_ctrl *mr_ctrl); -int mlx5_dma_map(struct rte_pci_device *pdev, void *addr, uint64_t iova, - size_t len); -int mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, uint64_t iova, - size_t len); +int mlx5_net_dma_map(struct rte_device *rte_dev, void *addr, uint64_t iova, + size_t len); +int mlx5_net_dma_unmap(struct rte_device *rte_dev, void *addr, uint64_t iova, + size_t len); /** * Get Memory Pool (MP) from mbuf. If mbuf is indirect, the pool from which the diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index 1e3260c6b5..2dfc957412 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -917,22 +917,22 @@ mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf, } /** - * DPDK callback to register a PCI device. + * Driver callback to register a device. * * This function spawns Ethernet devices out of a given PCI device. * - * @param[in] pci_drv - * PCI driver structure (mlx5_driver). - * @param[in] pci_dev - * PCI device information. + * @param[in] rte_dev + * Pointer to generic device. + * @param[in] ib_dev + * Pointer to Verbs device. * * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) +mlx5_os_net_probe(struct rte_device *rte_dev) { + struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(rte_dev); struct devx_device_bdf *devx_bdf_devs, *orig_devx_bdf_devs; /* * Number of found IB Devices matching with requested PCI BDF. From patchwork Thu May 27 13:37:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93486 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 921BDA0548; Thu, 27 May 2021 15:39:34 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5AA524111D; Thu, 27 May 2021 15:39:04 +0200 (CEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2069.outbound.protection.outlook.com [40.107.220.69]) by mails.dpdk.org (Postfix) with ESMTP id 5CBE440150 for ; Thu, 27 May 2021 15:39:01 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W/OF1ub1MoieLIQf65YaTn5WcD32Xi1P4jbMQRzy5H62aYcqIfAYIZxBsy2pCvOTWL/742JRSEWXA1ceblt/VHEgkzKonJVKryBJE0Ie7PkSPCwF+jP1EIZgjC9W2/x1kfaml+1TTdQecrzHZ4zBw5UWgCdHR7Lwod7VV46/P5b7y+YTrnEaRwKthAydql6vNoI5GS9mBZeUfd9yhUtPO8BJfPjMTA0cVY+jAUlrNSSiWDV4U9Rqy66przwjb5R015N3vg6qTyH7J+ftt+tcUXmT4wQjcNhw+qP1B6EXkRPwdv1CrtFf6wvMiYl9ClnYfT1FszTBaBsdVX+puDpLgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WTQ8qOGUIprfsNwvgTddMJLNS2AHuHo59ElR3fykCCo=; b=YDwQf75btJYpijjNp2Jm8ZAU3JBhrtNPaniLUP4O3cO/ReQGEUIplrEs9lKqAlDAGpwCMDp0KRGjuLRXxNdVzPGzF9a+IewdsNIl4qlksJSEHp5l1sq/2Uo+x9UQ0d1CW4moanIXRVUgDJfNAMQ2bNj8McvzX2sIGBovoWWxCKyT7kCC0VQyczSpZj73RPLAMQ/48X/ZOkOI03jjc840QbrCTGUDAOFQCefcv8FxlJ+rh9UgMQWCVsJ6asgDWxiyG8Z3pJYGyy4XXTEIY2vJDMVNGiL4W9PMr5rfdpoFFQ/g950bBFItI47l1ch+vnJngRRvqvcJO63pEDszUWXS+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WTQ8qOGUIprfsNwvgTddMJLNS2AHuHo59ElR3fykCCo=; b=PqdvyNP9ahM0DaJMxB2sUqPuW7Z6tkuIy1UWgTYSMm4168eR5GwMJUN1ksblWbNPPxVublE2O4SVgor3j1ZlLqopom+wppgcfWBUBsBMHi2uoxd3JGXLFrpQMEAT3fwXbw3KlPYCtcjD0Br2sN+65ngurAty/BNUkdOWoluLN9gEoGJxQ2xQ/yYmG/kouHGLdYAUsRMpwpbohbTGjc/+DmOWw1FccRDVFyeY/wa5ctRdKAZ8XKJZ7uFKmB8SgwnYZDAn/kRvL4c+u0uc+ic0qhcwLFYGX2neJGyBqavCFxGJGWg244WJdXiBu5yB+2L4J/WLcSn2wR1i6rQ0JtNm7A== Received: from CO2PR04CA0175.namprd04.prod.outlook.com (2603:10b6:104:4::29) by BYAPR12MB2872.namprd12.prod.outlook.com (2603:10b6:a03:12e::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.21; Thu, 27 May 2021 13:39:00 +0000 Received: from CO1NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:104:4:cafe::40) by CO2PR04CA0175.outlook.office365.com (2603:10b6:104:4::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20 via Frontend Transport; Thu, 27 May 2021 13:39:00 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT035.mail.protection.outlook.com (10.13.175.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 13:38:59 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:38:58 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Ori Kam Date: Thu, 27 May 2021 16:37:50 +0300 Message-ID: <20210527133759.17401-6-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a521fc46-4003-43e0-8b7d-08d92114c898 X-MS-TrafficTypeDiagnostic: BYAPR12MB2872: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2582; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XmMUaQKqaR5AbspupRuLE2TUQgnnq3gF613cy1tPYsJrmIp3nh01jE06UWMUvt17Q6Nx2+dwlRZMT+Y3rOJPjM5tXEnwZQCywAfaSC0DVEVweygrjDgrlh17417nFYi3Xxwpk8JNodLpdVsrrC/Zw/9+7C/MQ6jZhf+CYpvllTlKt7HXIh/m0rZbV23I6oF2gZqVuYXZOgxbApA2A1vDNpdc4AAaGDA+C+zeKB89A6xbgeKKeTgKiioMaKpu8eNq/szxVBrGb4Ir9L5OqI09FnKFgp+4wfqxkSTHOULe5woMl5ZahaoWbqxcfe5NWdbQjmqyGuwDwcrDoQDOGq0w5UH2BM+k5rxdXfBlVrt2FfitXkFwMwE/2/pUc96+3yOFDVq5bLc4hfJMzN1aPn7P5Bv1ZBl7JmIXv1PaCQ9SdmuDAkgrNTHCWkU9CdlGs0MH91UcopN0D5OW+26UlMQEpO5Rk3gcB+gz0k4gYc85Ru3ya5tg8IeI6Kcsf/JfsUdZQhkAh3LgLvlT7XIsa3McnjHa7L+dUaMrYuBC5+M6a4Ffjc+J1pDvjbXM+78kg8WbKBhBBTGXTbL6qYTyhICMsRo16cHYBphZYkYKIaq3VTpkjQ/F/uOazOVPb0+vBF+eFxxzjt/Af9rBCd8YRtJMdltVN7FFPystPTLko5GvHNs= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(346002)(136003)(376002)(396003)(46966006)(36840700001)(186003)(16526019)(426003)(336012)(356005)(7696005)(54906003)(26005)(82310400003)(47076005)(55016002)(6666004)(36756003)(2616005)(86362001)(6636002)(8936002)(8676002)(2906002)(107886003)(4326008)(70586007)(6862004)(37006003)(70206006)(5660300002)(82740400003)(1076003)(6286002)(7636003)(316002)(36906005)(478600001)(83380400001)(36860700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:38:59.7089 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a521fc46-4003-43e0-8b7d-08d92114c898 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2872 Subject: [dpdk-dev] [RFC 05/14] regex/mlx5: migrate to common driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support auxiliary bus, upgrades driver to use mlx5 common driver structure. Signed-off-by: Xueming Li --- drivers/regex/mlx5/mlx5_regex.c | 49 ++++++++++++--------------------- drivers/regex/mlx5/mlx5_regex.h | 1 - 2 files changed, 18 insertions(+), 32 deletions(-) diff --git a/drivers/regex/mlx5/mlx5_regex.c b/drivers/regex/mlx5/mlx5_regex.c index dcb2ced88e..9151e477c0 100644 --- a/drivers/regex/mlx5/mlx5_regex.c +++ b/drivers/regex/mlx5/mlx5_regex.c @@ -9,8 +9,8 @@ #include #include #include +#include -#include #include #include #include @@ -76,15 +76,13 @@ mlx5_regex_engines_status(struct ibv_context *ctx, int num_engines) } static void -mlx5_regex_get_name(char *name, struct rte_pci_device *pci_dev __rte_unused) +mlx5_regex_get_name(char *name, struct rte_device *dev) { - sprintf(name, "mlx5_regex_%02x:%02x.%02x", pci_dev->addr.bus, - pci_dev->addr.devid, pci_dev->addr.function); + sprintf(name, "mlx5_regex_%s", dev->name); } static int -mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) +mlx5_regex_dev_probe(struct rte_device *rte_dev) { struct ibv_device *ibv; struct mlx5_regex_priv *priv = NULL; @@ -94,16 +92,10 @@ mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, int ret; uint32_t val; - ibv = mlx5_os_get_ibv_device(&pci_dev->addr); - if (!ibv) { - DRV_LOG(ERR, "No matching IB device for PCI slot " - PCI_PRI_FMT ".", pci_dev->addr.domain, - pci_dev->addr.bus, pci_dev->addr.devid, - pci_dev->addr.function); + ibv = mlx5_get_ibv_device(rte_dev); + if (ibv == NULL) return -rte_errno; - } - DRV_LOG(INFO, "PCI information matches for device \"%s\".", - ibv->name); + DRV_LOG(INFO, "Probe device \"%s\".", ibv->name); ctx = mlx5_glue->dv_open_device(ibv); if (!ctx) { DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); @@ -146,7 +138,7 @@ mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, priv->is_bf2 = 1; /* Default RXP programming mode to Shared. */ priv->prog_mode = MLX5_RXP_SHARED_PROG_MODE; - mlx5_regex_get_name(name, pci_dev); + mlx5_regex_get_name(name, rte_dev); priv->regexdev = rte_regexdev_register(name); if (priv->regexdev == NULL) { DRV_LOG(ERR, "Failed to register RegEx device."); @@ -180,7 +172,7 @@ mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, priv->regexdev->enqueue = mlx5_regexdev_enqueue_gga; #endif priv->regexdev->dequeue = mlx5_regexdev_dequeue; - priv->regexdev->device = (struct rte_device *)pci_dev; + priv->regexdev->device = rte_dev; priv->regexdev->data->dev_private = priv; priv->regexdev->state = RTE_REGEXDEV_READY; priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr; @@ -213,13 +205,13 @@ mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, } static int -mlx5_regex_pci_remove(struct rte_pci_device *pci_dev) +mlx5_regex_dev_remove(struct rte_device *rte_dev) { char name[RTE_REGEXDEV_NAME_MAX_LEN]; struct rte_regexdev *dev; struct mlx5_regex_priv *priv = NULL; - mlx5_regex_get_name(name, pci_dev); + mlx5_regex_get_name(name, rte_dev); dev = rte_regexdev_get_device_by_name(name); if (!dev) return 0; @@ -254,24 +246,19 @@ static const struct rte_pci_id mlx5_regex_pci_id_map[] = { } }; -static struct mlx5_pci_driver mlx5_regex_driver = { - .driver_class = MLX5_CLASS_REGEX, - .pci_driver = { - .driver = { - .name = RTE_STR(MLX5_REGEX_DRIVER_NAME), - }, - .id_table = mlx5_regex_pci_id_map, - .probe = mlx5_regex_pci_probe, - .remove = mlx5_regex_pci_remove, - .drv_flags = 0, - }, +static struct mlx5_class_driver mlx5_regex_driver = { + .drv_class = MLX5_CLASS_REGEX, + .name = RTE_STR(MLX5_REGEX_DRIVER_NAME), + .id_table = mlx5_regex_pci_id_map, + .probe = mlx5_regex_dev_probe, + .remove = mlx5_regex_dev_remove, }; RTE_INIT(rte_mlx5_regex_init) { mlx5_common_init(); if (mlx5_glue) - mlx5_pci_driver_register(&mlx5_regex_driver); + mlx5_class_driver_register(&mlx5_regex_driver); } RTE_LOG_REGISTER_DEFAULT(mlx5_regex_logtype, NOTICE) diff --git a/drivers/regex/mlx5/mlx5_regex.h b/drivers/regex/mlx5/mlx5_regex.h index 51a2101e53..45200bf937 100644 --- a/drivers/regex/mlx5/mlx5_regex.h +++ b/drivers/regex/mlx5/mlx5_regex.h @@ -59,7 +59,6 @@ struct mlx5_regex_db { struct mlx5_regex_priv { TAILQ_ENTRY(mlx5_regex_priv) next; struct ibv_context *ctx; /* Device context. */ - struct rte_pci_device *pci_dev; struct rte_regexdev *regexdev; /* Pointer to the RegEx dev. */ uint16_t nb_queues; /* Number of queues. */ struct mlx5_regex_qp *qps; /* Pointer to the qp array. */ From patchwork Thu May 27 13:37:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93487 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3218CA0548; Thu, 27 May 2021 15:39:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AB09F41129; Thu, 27 May 2021 15:39:05 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2043.outbound.protection.outlook.com [40.107.236.43]) by mails.dpdk.org (Postfix) with ESMTP id 9AC7D410F9 for ; Thu, 27 May 2021 15:39:02 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HnAHwZuCfgcjhCEI9r5r5eLjE8gsCTtzjYdxLDTqLGWDeQaCokSXr+ojFpZu1UVEVkh0LS9TCE5KKhNwMxLoy7Az4WS88N3WR05UhEoEiFjXBRxjNBKSZX3Ir2I8yzFz9sveAYOQyrh4AUBACgx9xwPhpe85shOnl9s6mU8gc9OzQzl9QdBbW2BuUVw1IbKtlREBQEuRpsdMnmNNOH6PVnFrRQ3CMej1BgNn6PUQNegdrFZVPzp5xBk6RSprAaRJPLuGTvVU8I9sNG+YV8ozq+SzsxZHUdutivqU69jXum/Y+RgdeNd21+RnD6CAvh10ayVCA/ZYrEoaMO0zl3pxHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ma6lusVONP5FdWPz30sm7ibDLqkbm2L/sYoDc4GyxF8=; b=n0EHfkljbw0KZIiRqUbs4ORwndpmvO/hmyC11LuJXudvxS3Ge0iSG89D0KLCjoe9nZVi9hP1M5+WRAXCldo9wS1VOWo0CrCAEBSmLixsOgqDD9WezcB/VWAwM1aqjm3pTi4jdw6H416dvOAiuWCQ9N4xlmu4s1blxyMhRpq6RAvXXrwOWLK6OjsSLrgKYj7+wPog69ebpDL5aVj1q/Lu/w1ZUu2cCUQGfWZ3/LSsAf59bI+6Yv3JtF6fCffe3VCdiLKENg55xidNwfNgnCWdy6R9T80Vp6SA3hXiGshW8egnHpzuUpwfGv4DP2aJmhftOswXJP2xL+zg9XvAo8b6zQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=marvell.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ma6lusVONP5FdWPz30sm7ibDLqkbm2L/sYoDc4GyxF8=; b=hnTO3KSJAIWC8QNksPD3lD4RrvtHMbGIE/dK4p0iD46yySOyijlb0hB4uclasDrCxKZM86NwCdxTN7pyr4fYIVtBzwLw3jDy8M08Ztl9qC5P8QbepjC9Xv4SUdxspFyLigXvl/LhZVw1etFZaf+cYHrqxyw31MF+mwMp+ik2grHGiWf+rPssprA7DLSsIA/ProlryVUzqszU468KyvB3sxrfZjcLYBsagnc9zGTFmkEutYcQ8MIFt7b7iyco/CI+dtCb2hIFTOhFSdg+frG/fBXaYoqZkt0D89O3S5RdaCAqOcI+4fXt8NJEwpPQ2eMdzHgzJXzktpt+/2Jb5JQf5g== Received: from CO2PR04CA0165.namprd04.prod.outlook.com (2603:10b6:104:4::19) by MWHPR1201MB0128.namprd12.prod.outlook.com (2603:10b6:301:56::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.22; Thu, 27 May 2021 13:39:01 +0000 Received: from CO1NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:104:4:cafe::e3) by CO2PR04CA0165.outlook.office365.com (2603:10b6:104:4::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20 via Frontend Transport; Thu, 27 May 2021 13:39:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT035.mail.protection.outlook.com (10.13.175.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 13:39:01 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:38:59 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Fiona Trahe , Ashish Gupta Date: Thu, 27 May 2021 16:37:51 +0300 Message-ID: <20210527133759.17401-7-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c64243e3-7c33-4279-0621-08d92114c982 X-MS-TrafficTypeDiagnostic: MWHPR1201MB0128: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4303; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YgNGqxj8q+WbGxa6eIhPCDJMzv7IO/pY3pzPl+6cE5zIHyTlKZZmqGxfvY2mRnlgTsRLNG9cAGO4XKYXda65HunLBN6agQYw8cLvkqPqJkgWt6HF1TXVyCong+LBzhQx23X8x1Y1MF8MrVWHF8rv4Os7o4XlEzidVrTKVjfdisRaM28vl2kg6StvBJv82QmoThJPhrMOjUssNmt34ITcRY3Ik6lSKqESJYr3pUmCr4ZT1lYK/5dnWQjWA4kotxKK/PAkoJmydKqbBnKc+E6nA4qJmKsh6nI1rdlLLw1uYHQUaO/qANjm4lGr+b1eza/DYnJml3hMGBLJkgi4jbPlXihEzt+EaLnn5umri0csYEC+lt8h4IwpJE2lLQKRrEbRkeKMlhJi/1nMp1kb9/73Vgx0ngePUfLIChukS8HXKoZxm3GKNJbApWNLz6q8drqWUOGm8DMkp4P+Xr0STYI6ue+np46pxJSZAM3xjGoU6wJlETIbuH5cwda9YEHOsliEP6inyoAmsQN72NIo2mT6bgnnmtRCb9ALJ5zS9T42wKNV4TsfzLUZpbMX7xXE3/Glz/DGJSCa2dTz/eA2Dur9C7EF/idMqab50TdswXI+fB2kqKdU1NZHgdnwkOVnu5uUAYNPlMLAoAPCLSKYuqJ4GQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(136003)(39860400002)(396003)(346002)(36840700001)(46966006)(36860700001)(7696005)(54906003)(36906005)(82310400003)(86362001)(7636003)(83380400001)(37006003)(316002)(6862004)(6286002)(55016002)(8936002)(478600001)(4326008)(8676002)(6666004)(6636002)(82740400003)(47076005)(2906002)(2616005)(186003)(336012)(16526019)(26005)(426003)(36756003)(70586007)(70206006)(1076003)(356005)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:39:01.2380 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c64243e3-7c33-4279-0621-08d92114c982 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0128 Subject: [dpdk-dev] [RFC 06/14] compress/mlx5: migrate to common driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support auxiliary bus, upgrades driver to use mlx5 common driver structure. Signed-off-by: Xueming Li --- drivers/compress/mlx5/mlx5_compress.c | 71 ++++++--------------------- 1 file changed, 15 insertions(+), 56 deletions(-) diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c index 80c564f10b..10d64a72d6 100644 --- a/drivers/compress/mlx5/mlx5_compress.c +++ b/drivers/compress/mlx5/mlx5_compress.c @@ -5,7 +5,7 @@ #include #include #include -#include +#include #include #include #include @@ -13,7 +13,6 @@ #include #include -#include #include #include #include @@ -37,7 +36,6 @@ struct mlx5_compress_xform { struct mlx5_compress_priv { TAILQ_ENTRY(mlx5_compress_priv) next; struct ibv_context *ctx; /* Device context. */ - struct rte_pci_device *pci_dev; struct rte_compressdev *cdev; void *uar; uint32_t pdn; /* Protection Domain number. */ @@ -711,23 +709,8 @@ mlx5_compress_hw_global_prepare(struct mlx5_compress_priv *priv) return 0; } -/** - * DPDK callback to register a PCI device. - * - * This function spawns compress device out of a given PCI device. - * - * @param[in] pci_drv - * PCI driver structure (mlx5_compress_driver). - * @param[in] pci_dev - * PCI device information. - * - * @return - * 0 on success, 1 to skip this driver, a negative errno value otherwise - * and rte_errno is set. - */ static int -mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, - struct rte_pci_device *pci_dev) +mlx5_compress_dev_probe(struct rte_device *dev) { struct ibv_device *ibv; struct rte_compressdev *cdev; @@ -736,24 +719,17 @@ mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, struct mlx5_hca_attr att = { 0 }; struct rte_compressdev_pmd_init_params init_params = { .name = "", - .socket_id = pci_dev->device.numa_node, + .socket_id = dev->numa_node, }; - RTE_SET_USED(pci_drv); if (rte_eal_process_type() != RTE_PROC_PRIMARY) { DRV_LOG(ERR, "Non-primary process type is not supported."); rte_errno = ENOTSUP; return -rte_errno; } - ibv = mlx5_os_get_ibv_device(&pci_dev->addr); - if (ibv == NULL) { - DRV_LOG(ERR, "No matching IB device for PCI slot " - PCI_PRI_FMT ".", pci_dev->addr.domain, - pci_dev->addr.bus, pci_dev->addr.devid, - pci_dev->addr.function); + ibv = mlx5_get_ibv_device(dev); + if (ibv == NULL) return -rte_errno; - } - DRV_LOG(INFO, "PCI information matches for device \"%s\".", ibv->name); ctx = mlx5_glue->dv_open_device(ibv); if (ctx == NULL) { DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); @@ -769,7 +745,7 @@ mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, rte_errno = ENOTSUP; return -ENOTSUP; } - cdev = rte_compressdev_pmd_create(ibv->name, &pci_dev->device, + cdev = rte_compressdev_pmd_create(ibv->name, dev, sizeof(*priv), &init_params); if (cdev == NULL) { DRV_LOG(ERR, "Failed to create device \"%s\".", ibv->name); @@ -784,7 +760,6 @@ mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, cdev->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED; priv = cdev->data->dev_private; priv->ctx = ctx; - priv->pci_dev = pci_dev; priv->cdev = cdev; priv->min_block_size = att.compress_min_block_size; priv->sq_ts_format = att.sq_ts_format; @@ -810,25 +785,14 @@ mlx5_compress_pci_probe(struct rte_pci_driver *pci_drv, return 0; } -/** - * DPDK callback to remove a PCI device. - * - * This function removes all compress devices belong to a given PCI device. - * - * @param[in] pci_dev - * Pointer to the PCI device. - * - * @return - * 0 on success, the function cannot fail. - */ static int -mlx5_compress_pci_remove(struct rte_pci_device *pdev) +mlx5_compress_dev_remove(struct rte_device *dev) { struct mlx5_compress_priv *priv = NULL; pthread_mutex_lock(&priv_list_lock); TAILQ_FOREACH(priv, &mlx5_compress_priv_list, next) - if (rte_pci_addr_cmp(&priv->pci_dev->addr, &pdev->addr) != 0) + if (priv->cdev->device == dev) break; if (priv) TAILQ_REMOVE(&mlx5_compress_priv_list, priv, next); @@ -852,24 +816,19 @@ static const struct rte_pci_id mlx5_compress_pci_id_map[] = { } }; -static struct mlx5_pci_driver mlx5_compress_driver = { - .driver_class = MLX5_CLASS_COMPRESS, - .pci_driver = { - .driver = { - .name = RTE_STR(MLX5_COMPRESS_DRIVER_NAME), - }, - .id_table = mlx5_compress_pci_id_map, - .probe = mlx5_compress_pci_probe, - .remove = mlx5_compress_pci_remove, - .drv_flags = 0, - }, +static struct mlx5_class_driver mlx5_compress_driver = { + .drv_class = MLX5_CLASS_COMPRESS, + .name = RTE_STR(MLX5_COMPRESS_DRIVER_NAME), + .id_table = mlx5_compress_pci_id_map, + .probe = mlx5_compress_dev_probe, + .remove = mlx5_compress_dev_remove, }; RTE_INIT(rte_mlx5_compress_init) { mlx5_common_init(); if (mlx5_glue != NULL) - mlx5_pci_driver_register(&mlx5_compress_driver); + mlx5_class_driver_register(&mlx5_compress_driver); } RTE_LOG_REGISTER_DEFAULT(mlx5_compress_logtype, NOTICE) From patchwork Thu May 27 13:37:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93488 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 43E06A0548; Thu, 27 May 2021 15:39:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D3A6C4110D; Thu, 27 May 2021 15:39:08 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2085.outbound.protection.outlook.com [40.107.94.85]) by mails.dpdk.org (Postfix) with ESMTP id 373A341121 for ; Thu, 27 May 2021 15:39:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=io3/KbDXT7sIEJdfunmrl4spSg3lqw5O7hV+IOsfiKZCNVJZ2O9h39m1DH9R4vpDI53dbg6F6oLKcX/w0US42iEOCZubnGymCDHENrBZ90qq2KUrf4K3b4QQ2QQfSIE6rbrXimVyCaZsuFU/8un5YqV5SKY/uT+Pr40ZaM6Ufgwzg9Fsrr2ce0BlSgvkXk/l7L1c4xjxBYWy93AJV/2ZKiEw6v3p9H1bKLRrrteTgD1qHEuIHA4OX4ZNazg6abdJaRVuXdtSbDEn0XaRftemC9jUMEBbj2fsQ/fXMIgvvEYITGg+fV4OlGGRdOXv0znHMSq4gEL1gpr3IOewMiDo9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=63tgzkCpS5kqcWRuaaFZ0Y1loI1UOkWNb4qprz0K25M=; b=oAdD5hujXys16G9JkLnu9MgcGe6/no89yGtyJVdhuN7yVbjY/kez+MIjlzivvmnzC0A01xCpIu6IXSx3PmNtmkL2JGPiK2IJslcIT2PD+z0hPR6wbemjtMo6BidhoP5KD3309j0d82cKb4Coj4FwwTd+FIuWupBiQEXxQutMpmN6lImM4M0Re8UJxWtuFmfvarH8qlnOojwvTZ1khjrEHqAAHR0vchX/JU00a3oIdN/i7zEQGSON7JXQS5r0PQKqJk1fCAMStw4gBNeDov7/Jm0NOjbO+BYM/Z8nzFUyV57SaBArDK3poOmGxGONE1AM84m/mNuMcbYZ0hD8wMLY3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=63tgzkCpS5kqcWRuaaFZ0Y1loI1UOkWNb4qprz0K25M=; b=SqL5t9Icrs4ihWuHsbRuRYQV8wwIFbWkaASH9osaq++wg+WZpxh4QpgdbNmJcHqnt3nHwpJCSxfD+hBsZTfCNHYfvocEup9kzZuTL/8QlxHqgAKKxJFZqzArKwizZ+4ZhnlLfCwH+HYbqQJkKu2RN06vB5kkRSVOCv9fWyw/kbdVBh9/tdCQwHx25yR1YU4r14X2Z24+YPMX4Tzfd2oWac+m8FN5UF288dzEdqA2robKoLCkMuCIBW+SSCOzcyA91zjvrPpUB7Pvl/iQISoiwekHvMVKGu7br+a55hO7VO5eClvdzCwFdK+U1+hW573cz4UZ+mK0Xtat7QRVgFEyNA== Received: from MWHPR14CA0040.namprd14.prod.outlook.com (2603:10b6:300:12b::26) by CY4PR12MB1494.namprd12.prod.outlook.com (2603:10b6:910:f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4150.27; Thu, 27 May 2021 13:39:03 +0000 Received: from CO1NAM11FT053.eop-nam11.prod.protection.outlook.com (2603:10b6:300:12b:cafe::73) by MWHPR14CA0040.outlook.office365.com (2603:10b6:300:12b::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.21 via Frontend Transport; Thu, 27 May 2021 13:39:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT053.mail.protection.outlook.com (10.13.175.63) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 13:39:03 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:39:01 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad Date: Thu, 27 May 2021 16:37:52 +0300 Message-ID: <20210527133759.17401-8-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 89158e14-0359-423a-5bc3-08d92114caa4 X-MS-TrafficTypeDiagnostic: CY4PR12MB1494: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:296; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WMxG99BcEHNNDQrFUsFqoP5US/lLgeG6gMTnDunhouaakSxu4yXRdmM3g0ZJ7E5i0YlSeIJ5eLkZlKH1CeUvoEmD2VyoFvlY0uyzCKG0C1x1FPz5iG3uCTPVP02ezhQ9rCs32oLjd2leJ67O96DFK5Mfio+XSr3GpvZqbKjxY/CfxB/2ks15yQweCTPL3K775mvH1DYtXsE21p8bJOu7XTnm3eci8vjPvDcDDIAPb78bM8fmQ1kwZS1337pYpW5yWDMFwnrC1vjJd4niJE5IwqoxDxFITUQmIbEorujvPVfPUNAmdgMpfqJXYryoTbnH8QrOpdkuJq0+dMAOz4Gni4b2Y12r9y5fsOy08f463OSDyxhbgLJz92EDNeDJXs5t9lAyyIooOYiqfpWUe/Pzsqtp8CaZAgChTwWRpIJX3sU0uwAYGpM+MluKNT+fhcgWeK27+O2xHSXNvOy5yAqwcDfjdlDizB/sXFhIf0llKroEm5fJxJC1iyRidCOmHmkfVCPJzeg5w/0QpOdQUklIQld6f8Ie41NiLHEE+F2J/BiqNdYlO133w79h7pGilsg+kNdiuodu1EHEQvGphXd4CP78qG7z7nopCUezXikdqS//BpqVW0RCnBNSNyAvAkf4ejHrsi30KnEMdpfZFjXUOEri3ZYY4336Di8C9OEA0TA= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(346002)(396003)(136003)(376002)(46966006)(36840700001)(8676002)(4326008)(107886003)(36860700001)(26005)(55016002)(70206006)(5660300002)(47076005)(186003)(316002)(6862004)(6666004)(1076003)(426003)(16526019)(36906005)(37006003)(36756003)(6636002)(8936002)(54906003)(478600001)(7636003)(6286002)(356005)(83380400001)(2616005)(7696005)(82740400003)(82310400003)(70586007)(336012)(2906002)(86362001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:39:03.1348 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 89158e14-0359-423a-5bc3-08d92114caa4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT053.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1494 Subject: [dpdk-dev] [RFC 07/14] vdpa/mlx5: fix driver name X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon Signed-off-by: Thomas Monjalon --- drivers/vdpa/mlx5/mlx5_vdpa.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 8b5bfd8c3d..5ab7c525c2 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -24,6 +24,7 @@ #include "mlx5_vdpa_utils.h" #include "mlx5_vdpa.h" +#define MLX5_VDPA_DRIVER_NAME vdpa_mlx5 #define MLX5_VDPA_DEFAULT_FEATURES ((1ULL << VHOST_USER_F_PROTOCOL_FEATURES) | \ (1ULL << VIRTIO_F_ANY_LAYOUT) | \ @@ -834,7 +835,7 @@ static struct mlx5_pci_driver mlx5_vdpa_driver = { .driver_class = MLX5_CLASS_VDPA, .pci_driver = { .driver = { - .name = "mlx5_vdpa", + .name = RTE_STR(MLX5_VDPA_DRIVER_NAME), }, .id_table = mlx5_vdpa_pci_id_map, .probe = mlx5_vdpa_pci_probe, @@ -855,6 +856,6 @@ RTE_INIT(rte_mlx5_vdpa_init) mlx5_pci_driver_register(&mlx5_vdpa_driver); } -RTE_PMD_EXPORT_NAME(net_mlx5_vdpa, __COUNTER__); -RTE_PMD_REGISTER_PCI_TABLE(net_mlx5_vdpa, mlx5_vdpa_pci_id_map); -RTE_PMD_REGISTER_KMOD_DEP(net_mlx5_vdpa, "* ib_uverbs & mlx5_core & mlx5_ib"); +RTE_PMD_EXPORT_NAME(MLX5_VDPA_DRIVER_NAME, __COUNTER__); +RTE_PMD_REGISTER_PCI_TABLE(MLX5_VDPA_DRIVER_NAME, mlx5_vdpa_pci_id_map); +RTE_PMD_REGISTER_KMOD_DEP(MLX5_VDPA_DRIVER_NAME, "* ib_uverbs & mlx5_core & mlx5_ib"); From patchwork Thu May 27 13:37:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93489 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CF3C4A0548; Thu, 27 May 2021 15:39:56 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EE4C141135; Thu, 27 May 2021 15:39:09 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2069.outbound.protection.outlook.com [40.107.244.69]) by mails.dpdk.org (Postfix) with ESMTP id CDB2041121 for ; Thu, 27 May 2021 15:39:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ErwQPjsg7AGXlXAURhJB/3k4M1zWoTPDVVooMTe9kS9unSnwYSxm4wgWhk4JDRazo3CuCWCnmf4RMYp9k42SR0sz5UfycnT3LgZZT8rlQ4F2gwWzPhMyfhjwZVAQoLuVN26ZMk20UE059ddL+KX6suBDRiBxLLqJMVCLnWi/f4b8Dt+0DQdiFPLfjJ9YDR52Z98wlv4RH/1xRGuklkSKJyBZx00qFq6LExpOQ+L7+99m5tPNnVPZLm+xOlk6fb2L3w7oM/S39cN4j+eSkFGrigM3UxAYLGiFc2oXEA8gig+2Ml/0FdxE8QUt486z8KJjdmCi/yvt1M72av4kj5oXtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cU0R+8zcVx82stiyIirQFBA9ZrQyzP9bFbh81l/h+0E=; b=H5AhaZwnFv0nzYcsQTuJTxd/OYU0P/Vy1S2liPCliRIFScefbUANQzAzO9tBurLZOUyK/bhEogPZtp+h2PeINKGnagW9KWocXfMZ7BDwr0vQTmVQV3DdgHbw+FQduHGP1KS9CMd/7ZnvmFbLwXJS6NLauB9Is8xB+uTdnFjsni6OuUziCriN0PZ84YwD899Sv/c2z4RENS/QuaY3EKAnM6hlodtKVNrPzqDAICH6AGz4kHuACK4yyrqB1jro/hayEi+c9shUmOsao0VYLWjFlv65o+r/B7wTBnQaMZ3qVImBws+h1F0R69vS1K+59mOegkNY1rSbDTHu/EXyQhOR+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cU0R+8zcVx82stiyIirQFBA9ZrQyzP9bFbh81l/h+0E=; b=tCpKYBO9Xw3MIA3MCAh/9r8TU4RifdemCObr0XZ/PWeFWa5lFP2Hq86yRrCCBOanQGMo4IR025cjQo2OSFc1XMzrI3llSwkPUcZvDqVDtvHqwEXV3UybcET0bkSIAgjdrwXpThEx8bZm/wjaDDElTMuvYK3oaFKINImKfksYQvYKlK8nyteATCbV35acvr5er1PeNYrtdX9BXsKJOiJM4OYqoKg+0vCAWsfRI5YV9EJrbG7C/joIlR9sNZgYollM0T6YKzBYLLs55nGhjpGixIcwAZlI+M3WMdDJ+qCBOayNTEf/ptQszACt+v32dl34b590d6cHZCKA0gFPLg5lSA== Received: from MW4P220CA0019.NAMP220.PROD.OUTLOOK.COM (2603:10b6:303:115::24) by DM6PR12MB4369.namprd12.prod.outlook.com (2603:10b6:5:2a1::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.21; Thu, 27 May 2021 13:39:06 +0000 Received: from CO1NAM11FT005.eop-nam11.prod.protection.outlook.com (2603:10b6:303:115:cafe::7d) by MW4P220CA0019.outlook.office365.com (2603:10b6:303:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20 via Frontend Transport; Thu, 27 May 2021 13:39:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT005.mail.protection.outlook.com (10.13.174.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 13:39:05 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:39:03 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad Date: Thu, 27 May 2021 16:37:53 +0300 Message-ID: <20210527133759.17401-9-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 817e624a-3c0b-4c18-c912-08d92114cc56 X-MS-TrafficTypeDiagnostic: DM6PR12MB4369: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1850; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Gvjp0+jK3VGpMXIxVPNx0ms/pmNJIYVkRvhuTUn3kMMhxiTqX1nA808FaBa9Lx+g4zO6yOF+t5FMTQbVIXzsxypUNq8Jx6VvmLM8cdkWwTHEB+TNzWBNekxBVFD3u+n0zlwcd3C5wMbg8VlfNjlp+iVhjfY1RVJlT+KpxQ9El0W4NXg1xfRrGTu4+n9x6NeMGTvahqnt7mPASQ73PptVnmfn7HrAjvn4wx3yuMefdICGlrtW56Vx5SPHvn3wvk5hoCPtbj/vw1EsOhGQwyO3H5rF00J0KxSKY96aCk9MA9AoZGRQulmYX7VOvZkTm/xiYPW13UjFA+vB1ovA6zyjXXR9L4sPo6pVupWZ6Zj0oWoPRprcZbuOanB5Fz58eN4+Ah0K/k7ESNdcQiUPdwSQpB0FahmTZp8QdLbdZvlntSk4k71+RTtNwLhL/TgxK3K7530tj1Mm3h8M4oDrfKHxdDedamgG3RncRaZ9CNcZ+X4+zVA44gC9yT4JZrSVzTmO2RwK9ioFJVVnv9EHUCW1Ks3j4speNAq6f2zspSIACbkbNy8BLZNI2OMPdUYBYFYVBmlsGl13f5BPWbufncupmXPrb3oAZYZNozaijVFHkkCi/aG96vlgu8ZNOEAv1YEPmfrcojzlMsskxtod7GwaBg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(376002)(39860400002)(346002)(136003)(36840700001)(46966006)(83380400001)(2616005)(6636002)(1076003)(2906002)(426003)(316002)(186003)(16526019)(36906005)(336012)(82740400003)(82310400003)(478600001)(54906003)(5660300002)(356005)(7636003)(36860700001)(6286002)(86362001)(107886003)(8936002)(55016002)(26005)(37006003)(8676002)(7696005)(6666004)(47076005)(4326008)(70586007)(36756003)(70206006)(6862004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:39:05.9958 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 817e624a-3c0b-4c18-c912-08d92114cc56 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT005.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4369 Subject: [dpdk-dev] [RFC 08/14] vdpa/mlx5: remove PCI specifics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon Signed-off-by: Thomas Monjalon --- drivers/vdpa/mlx5/mlx5_vdpa.c | 119 ++++++++++------------------------ drivers/vdpa/mlx5/mlx5_vdpa.h | 1 - 2 files changed, 34 insertions(+), 86 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 5ab7c525c2..967234193f 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -11,12 +11,11 @@ #include #include #include -#include #include +#include #include #include -#include #include #include #include @@ -552,34 +551,13 @@ mlx5_vdpa_sys_roce_disable(const char *addr) } static int -mlx5_vdpa_roce_disable(struct rte_pci_addr *addr, struct ibv_device **ibv) +mlx5_vdpa_roce_disable(struct rte_device *dev) { - char addr_name[64] = {0}; - - rte_pci_device_name(addr, addr_name, sizeof(addr_name)); /* Firstly try to disable ROCE by Netlink and fallback to sysfs. */ - if (mlx5_vdpa_nl_roce_disable(addr_name) == 0 || - mlx5_vdpa_sys_roce_disable(addr_name) == 0) { - /* - * Succeed to disable ROCE, wait for the IB device to appear - * again after reload. - */ - int r; - struct ibv_device *ibv_new; - - for (r = MLX5_VDPA_MAX_RETRIES; r; r--) { - ibv_new = mlx5_os_get_ibv_device(addr); - if (ibv_new) { - *ibv = ibv_new; - return 0; - } - usleep(MLX5_VDPA_USEC); - } - DRV_LOG(ERR, "Cannot much device %s after ROCE disable, " - "retries exceed %d", addr_name, MLX5_VDPA_MAX_RETRIES); - rte_errno = EAGAIN; - } - return -rte_errno; + if (mlx5_vdpa_nl_roce_disable(dev->name) != 0 && + mlx5_vdpa_sys_roce_disable(dev->name) != 0) + return -rte_errno; + return 0; } static int @@ -647,44 +625,33 @@ mlx5_vdpa_config_get(struct rte_devargs *devargs, struct mlx5_vdpa_priv *priv) DRV_LOG(DEBUG, "no traffic max is %u.", priv->no_traffic_max); } -/** - * DPDK callback to register a mlx5 PCI device. - * - * This function spawns vdpa device out of a given PCI device. - * - * @param[in] pci_drv - * PCI driver structure (mlx5_vpda_driver). - * @param[in] pci_dev - * PCI device information. - * - * @return - * 0 on success, 1 to skip this driver, a negative errno value otherwise - * and rte_errno is set. - */ static int -mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev __rte_unused) +mlx5_vdpa_dev_probe(struct rte_device *dev) { struct ibv_device *ibv; struct mlx5_vdpa_priv *priv = NULL; struct ibv_context *ctx = NULL; struct mlx5_hca_attr attr; + int retry; int ret; - ibv = mlx5_os_get_ibv_device(&pci_dev->addr); - if (!ibv) { - DRV_LOG(ERR, "No matching IB device for PCI slot " - PCI_PRI_FMT ".", pci_dev->addr.domain, - pci_dev->addr.bus, pci_dev->addr.devid, - pci_dev->addr.function); + if (mlx5_vdpa_roce_disable(dev) != 0) { + DRV_LOG(WARNING, "Failed to disable ROCE for \"%s\".", + dev->name); return -rte_errno; - } else { - DRV_LOG(INFO, "PCI information matches for device \"%s\".", - ibv->name); } - if (mlx5_vdpa_roce_disable(&pci_dev->addr, &ibv) != 0) { - DRV_LOG(WARNING, "Failed to disable ROCE for \"%s\".", - ibv->name); + /* Wait for the IB device to appear again after reload. */ + for (retry = MLX5_VDPA_MAX_RETRIES; retry > 0; --retry) { + ibv = mlx5_get_ibv_device(dev); + if (ibv != NULL) + break; + usleep(MLX5_VDPA_USEC); + } + if (ibv == NULL) { + DRV_LOG(ERR, "Cannot get IB device after disabling RoCE for " + "\"%s\", retries exceed %d.", + dev->name, MLX5_VDPA_MAX_RETRIES); + rte_errno = EAGAIN; return -rte_errno; } ctx = mlx5_glue->dv_open_device(ibv); @@ -722,20 +689,18 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, if (attr.num_lag_ports == 0) priv->num_lag_ports = 1; priv->ctx = ctx; - priv->pci_dev = pci_dev; priv->var = mlx5_glue->dv_alloc_var(ctx, 0); if (!priv->var) { DRV_LOG(ERR, "Failed to allocate VAR %u.", errno); goto error; } - priv->vdev = rte_vdpa_register_device(&pci_dev->device, - &mlx5_vdpa_ops); + priv->vdev = rte_vdpa_register_device(dev, &mlx5_vdpa_ops); if (priv->vdev == NULL) { DRV_LOG(ERR, "Failed to register vDPA device."); rte_errno = rte_errno ? rte_errno : EINVAL; goto error; } - mlx5_vdpa_config_get(pci_dev->device.devargs, priv); + mlx5_vdpa_config_get(dev->devargs, priv); SLIST_INIT(&priv->mr_list); pthread_mutex_init(&priv->vq_config_lock, NULL); pthread_mutex_lock(&priv_list_lock); @@ -754,26 +719,15 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, return -rte_errno; } -/** - * DPDK callback to remove a PCI device. - * - * This function removes all vDPA devices belong to a given PCI device. - * - * @param[in] pci_dev - * Pointer to the PCI device. - * - * @return - * 0 on success, the function cannot fail. - */ static int -mlx5_vdpa_pci_remove(struct rte_pci_device *pci_dev) +mlx5_vdpa_dev_remove(struct rte_device *dev) { struct mlx5_vdpa_priv *priv = NULL; int found = 0; pthread_mutex_lock(&priv_list_lock); TAILQ_FOREACH(priv, &priv_list, next) { - if (!rte_pci_addr_cmp(&priv->pci_dev->addr, &pci_dev->addr)) { + if (priv->vdev->device == dev) { found = 1; break; } @@ -831,17 +785,12 @@ static const struct rte_pci_id mlx5_vdpa_pci_id_map[] = { } }; -static struct mlx5_pci_driver mlx5_vdpa_driver = { - .driver_class = MLX5_CLASS_VDPA, - .pci_driver = { - .driver = { - .name = RTE_STR(MLX5_VDPA_DRIVER_NAME), - }, - .id_table = mlx5_vdpa_pci_id_map, - .probe = mlx5_vdpa_pci_probe, - .remove = mlx5_vdpa_pci_remove, - .drv_flags = 0, - }, +static struct mlx5_class_driver mlx5_vdpa_driver = { + .drv_class = MLX5_CLASS_VDPA, + .name = RTE_STR(MLX5_VDPA_DRIVER_NAME), + .id_table = mlx5_vdpa_pci_id_map, + .probe = mlx5_vdpa_dev_probe, + .remove = mlx5_vdpa_dev_remove, }; RTE_LOG_REGISTER_DEFAULT(mlx5_vdpa_logtype, NOTICE) @@ -853,7 +802,7 @@ RTE_INIT(rte_mlx5_vdpa_init) { mlx5_common_init(); if (mlx5_glue) - mlx5_pci_driver_register(&mlx5_vdpa_driver); + mlx5_class_driver_register(&mlx5_vdpa_driver); } RTE_PMD_EXPORT_NAME(MLX5_VDPA_DRIVER_NAME, __COUNTER__); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h index 722c72b65e..2a04e36607 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/mlx5_vdpa.h @@ -133,7 +133,6 @@ struct mlx5_vdpa_priv { struct rte_vdpa_device *vdev; /* vDPA device. */ int vid; /* vhost device id. */ struct ibv_context *ctx; /* Device context. */ - struct rte_pci_device *pci_dev; struct mlx5_hca_vdpa_attr caps; uint32_t pdn; /* Protection Domain number. */ struct ibv_pd *pd; From patchwork Thu May 27 13:37:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93490 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 934CDA0548; Thu, 27 May 2021 15:40:05 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7C2BF41140; Thu, 27 May 2021 15:39:11 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2085.outbound.protection.outlook.com [40.107.92.85]) by mails.dpdk.org (Postfix) with ESMTP id 77AE04112F for ; Thu, 27 May 2021 15:39:09 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j2WPUV/dX2CGiSxPz2wm2JTQDcqiysAVxwIIyBn27/jC2ztZphzqaq+zsNGSD9xT13Ai2QLqxLOH7eEoi/Km01gEbGR6JbXNXl7oXmMLsPqCW2u8CBnCmttfQ0Yp/KvP9FweHuaCevprIEo6dRWgqnud7lAaVuD2M+65NCVvGFx72HbYsaRyHU4pBPQPhVvJL1cyOgRdOY4ERaAzQyT1kqMZibGY5wArvVLhh+AbsYHD7SWF+ByhoSxaRV5CN4cXGwrW9vaYrd/iemE1qixLBpnn475N8BTaVlGBIxDvuCeikYummyKcLjsVN3tToRY6krqsf0gCITrzNKUd0fRdpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=otp8g7rCiwvUZzzbdNDwvn8qYgvifJM0ir8LBsj73KM=; b=IYPcfYcQBZNE3JtL+Q9NdzqtrwNGP7R7q+xVcIgfPq2QKXhICx7ndLJ5vnTG5Qroo6GqodN83m+/zEWctcDCQzM3C+Vk4O9TC2vGvmqMRNJI/joASm6nwkg8MmYtAfJelcTHCnbqbxsrpHrMSc9ap/voE/aSvG7C2TqAbATP5E+CJUuKNhqZlkbspP5dsxuUJVGfQWAU5BSF7fx7sY80sACjO0HAihF8FEd0jCfln+Pm7eBsL72XTCRCqn1Dn7gsF4CLU598zKZoOSg+SntHf/Ml0hLEwN39XWyn13rF7h1KpK+98Yhrir0eA6MuITgWKMFmRiu+o4xcTL+r0r/1Rg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=tuxdriver.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=otp8g7rCiwvUZzzbdNDwvn8qYgvifJM0ir8LBsj73KM=; b=NrF5aTLAImgnah/VQEnnCcO355tcfKnwskboBEPK1YcxyuT70DU+i+7nsOEDqBX0sUrPRWnbSA9KP5nl2HaO6uybmH8VANiqNX6Jgc3nSSvm/LQFvDuWoCNKS69Fyx+gKtRYtrMMLbytHAzfoaSbbX6J/EZvbYVxHNv71zGhO8//1fBmGz6rTc7lem2LqQlL2LKInIamh7bfWTT+JypgdOtMNR4dytqgoPu0cGxcqTElKbTLemrav0HUSdJ1T3uv7dd2u8R1XyqEyfWLOiUMowrKOm2grMPdRmLxj5v0V/YdjfIVj5ZkDOmCqw2/FGpq6s0Ks2cSZgDMl8i4oDMqeQ== Received: from MW4P220CA0029.NAMP220.PROD.OUTLOOK.COM (2603:10b6:303:115::34) by DM6PR12MB4435.namprd12.prod.outlook.com (2603:10b6:5:2a6::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.21; Thu, 27 May 2021 13:39:08 +0000 Received: from CO1NAM11FT005.eop-nam11.prod.protection.outlook.com (2603:10b6:303:115:cafe::de) by MW4P220CA0029.outlook.office365.com (2603:10b6:303:115::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20 via Frontend Transport; Thu, 27 May 2021 13:39:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; tuxdriver.com; dkim=none (message not signed) header.d=none;tuxdriver.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT005.mail.protection.outlook.com (10.13.174.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 13:39:07 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:39:04 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Ray Kinsella , Neil Horman Date: Thu, 27 May 2021 16:37:54 +0300 Message-ID: <20210527133759.17401-10-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e249ea26-0793-42df-c1c8-08d92114cd79 X-MS-TrafficTypeDiagnostic: DM6PR12MB4435: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:586; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XZhYJGq52vUpHVu0RVqJZBP6/lqjYB0K3T8z46TgyWxcAJOuSQiDjn6YzaMNfMnUS+5tdnapYHwDYI/Pi8f6oj3vAdPWk+6GPg3p8boCB6gSs34KpbrqHvZkeOkbmhA7xmq8cuTmHKTihAgRNW7H7XIN+FZJ/fglEAH3d5NtC/KbTWh9/RSoGYxV5VG4b/mroQ9g6Aoq185F4xa30wRtB/Fitn04TiJ4UOPoUFF4vwhqAHref4kHVtA4OfvLWoS9ibBNRMpDDvtsco31UptfDliZ+4y4ZB9z9UWsf1HYsonaDyDlvL7URh2L1QSDUbQp1TOownM5O2PVY+UaTj/WSoAcI07B+zXobP9rAFa9tNnLWbuyfJgnmOCThShOUFLlvbOWHDHLjXpThT+EwzBB1te7CXtNCk0JVEuvLYShFP0O2dGQfb83aSD0beHkhELDvorL/MK/kfZK472scotpBqZR5EpwUUEHBJmgeda9X7R3zaMYc7Av1bhmpHIL5qvcdfRgsxt5LYBII5xaVnmhH7D7NCiPlBQ4oZUY9ciYpX2Y3cADr07nGmOYbOGlnsUVYMFqYYrj0tTRGsBb4hAOTZzMk+s/M7/0Y2HJBWZAWA+f57nMKEnR4Tk3xFsCurV2prpJgKaaBV21FFKDi+QXEQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(136003)(46966006)(36840700001)(83380400001)(36906005)(6636002)(2906002)(1076003)(316002)(16526019)(186003)(2616005)(426003)(336012)(5660300002)(30864003)(478600001)(36860700001)(82310400003)(54906003)(7636003)(82740400003)(356005)(6286002)(8676002)(37006003)(26005)(55016002)(8936002)(86362001)(4326008)(70586007)(6862004)(36756003)(6666004)(47076005)(7696005)(70206006); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:39:07.8827 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e249ea26-0793-42df-c1c8-08d92114cd79 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT005.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4435 Subject: [dpdk-dev] [RFC 09/14] common/mlx5: clean up legacy PCI bus driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Clean up legacy PCI bus driver since all mlx5 PMDs moved to new common PCI bus driver. Signed-off-by: Xueming Li --- drivers/common/mlx5/linux/mlx5_common_os.c | 28 -- drivers/common/mlx5/linux/mlx5_common_os.h | 4 - drivers/common/mlx5/mlx5_common.c | 1 - drivers/common/mlx5/mlx5_common_pci.c | 433 +-------------------- drivers/common/mlx5/mlx5_common_pci.h | 77 ---- drivers/common/mlx5/mlx5_common_private.h | 1 + drivers/common/mlx5/version.map | 4 - 7 files changed, 3 insertions(+), 545 deletions(-) delete mode 100644 drivers/common/mlx5/mlx5_common_pci.h diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index ea6001e6b2..cd1c305cc1 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -401,31 +401,3 @@ mlx5_glue_constructor(void) " libmlx5)"); mlx5_glue = NULL; } - -struct ibv_device * -mlx5_os_get_ibv_device(struct rte_pci_addr *addr) -{ - int n; - struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); - struct ibv_device *ibv_match = NULL; - - if (ibv_list == NULL) { - rte_errno = ENOSYS; - return NULL; - } - while (n-- > 0) { - struct rte_pci_addr paddr; - - DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name); - if (mlx5_dev_to_pci_addr(ibv_list[n]->ibdev_path, &paddr) != 0) - continue; - if (rte_pci_addr_cmp(addr, &paddr) != 0) - continue; - ibv_match = ibv_list[n]; - break; - } - if (ibv_match == NULL) - rte_errno = ENOENT; - mlx5_glue->free_device_list(ibv_list); - return ibv_match; -} diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h index 72d6bf828b..bce5a11c0f 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.h +++ b/drivers/common/mlx5/linux/mlx5_common_os.h @@ -289,8 +289,4 @@ mlx5_os_free(void *addr) free(addr); } -__rte_internal -struct ibv_device * -mlx5_os_get_ibv_device(struct rte_pci_addr *addr); - #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */ diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index f2e2a95ae0..875668d72b 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -14,7 +14,6 @@ #include "mlx5_common.h" #include "mlx5_common_os.h" #include "mlx5_common_log.h" -#include "mlx5_common_pci.h" #include "mlx5_common_private.h" uint8_t haswell_broadwell_cpu; diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 5a824dd50f..c1500f3a2b 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -8,431 +8,17 @@ #include #include #include +#include +#include #include "mlx5_common_log.h" -#include "mlx5_common_pci.h" #include "mlx5_common_private.h" static struct rte_pci_driver mlx5_common_pci_driver; -/********** Legacy PCI bus driver, to be removed ********/ - -struct mlx5_pci_device { - struct rte_pci_device *pci_dev; - TAILQ_ENTRY(mlx5_pci_device) next; - uint32_t classes_loaded; -}; - -/* Head of list of drivers. */ -static TAILQ_HEAD(mlx5_pci_bus_drv_head, mlx5_pci_driver) drv_list = - TAILQ_HEAD_INITIALIZER(drv_list); - -/* Head of mlx5 pci devices. */ -static TAILQ_HEAD(mlx5_pci_devices_head, mlx5_pci_device) devices_list = - TAILQ_HEAD_INITIALIZER(devices_list); - -static const struct { - const char *name; - unsigned int driver_class; -} mlx5_classes[] = { - { .name = "vdpa", .driver_class = MLX5_CLASS_VDPA }, - { .name = "net", .driver_class = MLX5_CLASS_NET }, - { .name = "regex", .driver_class = MLX5_CLASS_REGEX }, - { .name = "compress", .driver_class = MLX5_CLASS_COMPRESS }, -}; - -static const unsigned int mlx5_class_combinations[] = { - MLX5_CLASS_NET, - MLX5_CLASS_VDPA, - MLX5_CLASS_REGEX, - MLX5_CLASS_COMPRESS, - MLX5_CLASS_NET | MLX5_CLASS_REGEX, - MLX5_CLASS_VDPA | MLX5_CLASS_REGEX, - MLX5_CLASS_NET | MLX5_CLASS_COMPRESS, - MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS, - MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, - MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, - MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, - /* New class combination should be added here. */ -}; - -static int -class_name_to_value(const char *class_name) -{ - unsigned int i; - - for (i = 0; i < RTE_DIM(mlx5_classes); i++) { - if (strcmp(class_name, mlx5_classes[i].name) == 0) - return mlx5_classes[i].driver_class; - } - return -EINVAL; -} - -static struct mlx5_pci_driver * -driver_get(uint32_t class) -{ - struct mlx5_pci_driver *driver; - - TAILQ_FOREACH(driver, &drv_list, next) { - if (driver->driver_class == class) - return driver; - } - return NULL; -} - -static int -bus_cmdline_options_handler(__rte_unused const char *key, - const char *class_names, void *opaque) -{ - int *ret = opaque; - char *nstr_org; - int class_val; - char *found; - char *nstr; - char *refstr = NULL; - - *ret = 0; - nstr = strdup(class_names); - if (!nstr) { - *ret = -ENOMEM; - return *ret; - } - nstr_org = nstr; - found = strtok_r(nstr, ":", &refstr); - if (!found) - goto err; - do { - /* Extract each individual class name. Multiple - * class key,value is supplied as class=net:vdpa:foo:bar. - */ - class_val = class_name_to_value(found); - /* Check if its a valid class. */ - if (class_val < 0) { - *ret = -EINVAL; - goto err; - } - *ret |= class_val; - found = strtok_r(NULL, ":", &refstr); - } while (found); -err: - free(nstr_org); - if (*ret < 0) - DRV_LOG(ERR, "Invalid mlx5 class options %s." - " Maybe typo in device class argument setting?", - class_names); - return *ret; -} - -static int -parse_class_options(const struct rte_devargs *devargs) -{ - const char *key = RTE_DEVARGS_KEY_CLASS; - struct rte_kvargs *kvlist; - int ret = 0; - - if (devargs == NULL) - return 0; - kvlist = rte_kvargs_parse(devargs->args, NULL); - if (kvlist == NULL) - return 0; - if (rte_kvargs_count(kvlist, key)) - rte_kvargs_process(kvlist, key, bus_cmdline_options_handler, - &ret); - rte_kvargs_free(kvlist); - return ret; -} - -static bool -mlx5_bus_match(const struct mlx5_pci_driver *drv, - const struct rte_pci_device *pci_dev) -{ - const struct rte_pci_id *id_table; - - for (id_table = drv->pci_driver.id_table; id_table->vendor_id != 0; - id_table++) { - /* Check if device's ids match the class driver's ids. */ - if (id_table->vendor_id != pci_dev->id.vendor_id && - id_table->vendor_id != RTE_PCI_ANY_ID) - continue; - if (id_table->device_id != pci_dev->id.device_id && - id_table->device_id != RTE_PCI_ANY_ID) - continue; - if (id_table->subsystem_vendor_id != - pci_dev->id.subsystem_vendor_id && - id_table->subsystem_vendor_id != RTE_PCI_ANY_ID) - continue; - if (id_table->subsystem_device_id != - pci_dev->id.subsystem_device_id && - id_table->subsystem_device_id != RTE_PCI_ANY_ID) - continue; - if (id_table->class_id != pci_dev->id.class_id && - id_table->class_id != RTE_CLASS_ANY_ID) - continue; - return true; - } - return false; -} - -static int -is_valid_class_combination(uint32_t user_classes) -{ - unsigned int i; - - /* Verify if user specified valid supported combination. */ - for (i = 0; i < RTE_DIM(mlx5_class_combinations); i++) { - if (mlx5_class_combinations[i] == user_classes) - return 0; - } - /* Not found any valid class combination. */ - return -EINVAL; -} - -static struct mlx5_pci_device * -pci_to_mlx5_device(const struct rte_pci_device *pci_dev) -{ - struct mlx5_pci_device *dev; - - TAILQ_FOREACH(dev, &devices_list, next) { - if (dev->pci_dev == pci_dev) - return dev; - } - return NULL; -} - -static bool -device_class_enabled(const struct mlx5_pci_device *device, uint32_t class) -{ - return (device->classes_loaded & class) ? true : false; -} - -static void -dev_release(struct mlx5_pci_device *dev) -{ - TAILQ_REMOVE(&devices_list, dev, next); - rte_free(dev); -} - -static int -drivers_remove(struct mlx5_pci_device *dev, uint32_t enabled_classes) -{ - struct mlx5_pci_driver *driver; - int local_ret = -ENODEV; - unsigned int i = 0; - int ret = 0; - - enabled_classes &= dev->classes_loaded; - while (enabled_classes) { - driver = driver_get(RTE_BIT64(i)); - if (driver) { - local_ret = driver->pci_driver.remove(dev->pci_dev); - if (!local_ret) - dev->classes_loaded &= ~RTE_BIT64(i); - else if (ret == 0) - ret = local_ret; - } - enabled_classes &= ~RTE_BIT64(i); - i++; - } - if (local_ret) - ret = local_ret; - return ret; -} - -static int -drivers_probe(struct mlx5_pci_device *dev, struct rte_pci_driver *pci_drv, - struct rte_pci_device *pci_dev, uint32_t user_classes) -{ - struct mlx5_pci_driver *driver; - uint32_t enabled_classes = 0; - bool already_loaded; - int ret; - - TAILQ_FOREACH(driver, &drv_list, next) { - if ((driver->driver_class & user_classes) == 0) - continue; - if (!mlx5_bus_match(driver, pci_dev)) - continue; - already_loaded = dev->classes_loaded & driver->driver_class; - if (already_loaded && - !(driver->pci_driver.drv_flags & RTE_PCI_DRV_PROBE_AGAIN)) { - DRV_LOG(ERR, "Device %s is already probed", - pci_dev->device.name); - ret = -EEXIST; - goto probe_err; - } - ret = driver->pci_driver.probe(pci_drv, pci_dev); - if (ret < 0) { - DRV_LOG(ERR, "Failed to load driver %s", - driver->pci_driver.driver.name); - goto probe_err; - } - enabled_classes |= driver->driver_class; - } - dev->classes_loaded |= enabled_classes; - return 0; -probe_err: - /* Only unload drivers which are enabled which were enabled - * in this probe instance. - */ - drivers_remove(dev, enabled_classes); - return ret; -} - -/** - * DPDK callback to register to probe multiple drivers for a PCI device. - * - * @param[in] pci_drv - * PCI driver structure. - * @param[in] dev - * PCI device information. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) -{ - struct mlx5_pci_device *dev; - uint32_t user_classes = 0; - bool new_device = false; - int ret; - - ret = parse_class_options(pci_dev->device.devargs); - if (ret < 0) - return ret; - user_classes = ret; - if (user_classes) { - /* Validate combination here. */ - ret = is_valid_class_combination(user_classes); - if (ret) { - DRV_LOG(ERR, "Unsupported mlx5 classes supplied."); - return ret; - } - } else { - /* Default to net class. */ - user_classes = MLX5_CLASS_NET; - } - dev = pci_to_mlx5_device(pci_dev); - if (!dev) { - dev = rte_zmalloc("mlx5_pci_device", sizeof(*dev), 0); - if (!dev) - return -ENOMEM; - dev->pci_dev = pci_dev; - TAILQ_INSERT_HEAD(&devices_list, dev, next); - new_device = true; - } - ret = drivers_probe(dev, pci_drv, pci_dev, user_classes); - if (ret) - goto class_err; - return 0; -class_err: - if (new_device) - dev_release(dev); - return ret; -} - -/** - * DPDK callback to remove one or more drivers for a PCI device. - * - * This function removes all drivers probed for a given PCI device. - * - * @param[in] pci_dev - * Pointer to the PCI device. - * - * @return - * 0 on success, the function cannot fail. - */ -static int -mlx5_pci_remove(struct rte_pci_device *pci_dev) -{ - struct mlx5_pci_device *dev; - int ret; - - dev = pci_to_mlx5_device(pci_dev); - if (!dev) - return -ENODEV; - /* Matching device found, cleanup and unload drivers. */ - ret = drivers_remove(dev, dev->classes_loaded); - if (!ret) - dev_release(dev); - return ret; -} - -static int -mlx5_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) -{ - struct mlx5_pci_driver *driver = NULL; - struct mlx5_pci_driver *temp; - struct mlx5_pci_device *dev; - int ret = -EINVAL; - - dev = pci_to_mlx5_device(pci_dev); - if (!dev) - return -ENODEV; - TAILQ_FOREACH(driver, &drv_list, next) { - if (device_class_enabled(dev, driver->driver_class) && - driver->pci_driver.dma_map) { - ret = driver->pci_driver.dma_map(pci_dev, addr, - iova, len); - if (ret) - goto map_err; - } - } - return ret; -map_err: - TAILQ_FOREACH(temp, &drv_list, next) { - if (temp == driver) - break; - if (device_class_enabled(dev, temp->driver_class) && - temp->pci_driver.dma_map && temp->pci_driver.dma_unmap) - temp->pci_driver.dma_unmap(pci_dev, addr, iova, len); - } - return ret; -} - -static int -mlx5_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) -{ - struct mlx5_pci_driver *driver; - struct mlx5_pci_device *dev; - int local_ret = -EINVAL; - int ret; - - dev = pci_to_mlx5_device(pci_dev); - if (!dev) - return -ENODEV; - ret = 0; - /* There is no unmap error recovery in current implementation. */ - TAILQ_FOREACH_REVERSE(driver, &drv_list, mlx5_pci_bus_drv_head, next) { - if (device_class_enabled(dev, driver->driver_class) && - driver->pci_driver.dma_unmap) { - local_ret = driver->pci_driver.dma_unmap(pci_dev, addr, - iova, len); - if (local_ret && (ret == 0)) - ret = local_ret; - } - } - if (local_ret) - ret = local_ret; - return ret; -} - /* PCI ID table is build dynamically based on registered mlx5 drivers. */ static struct rte_pci_id *mlx5_pci_id_table; -static struct rte_pci_driver mlx5_pci_driver = { - .driver = { - .name = MLX5_PCI_DRIVER_NAME, - }, - .probe = mlx5_pci_probe, - .remove = mlx5_pci_remove, - .dma_map = mlx5_pci_dma_map, - .dma_unmap = mlx5_pci_dma_unmap, -}; - static int pci_id_table_size_get(const struct rte_pci_id *id_table) { @@ -509,7 +95,6 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) } /* Terminate table with empty entry. */ updated_table[i].vendor_id = 0; - mlx5_pci_driver.id_table = updated_table; mlx5_common_pci_driver.id_table = updated_table; mlx5_pci_id_table = updated_table; if (old_table) @@ -517,20 +102,6 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) return 0; } -void -mlx5_pci_driver_register(struct mlx5_pci_driver *driver) -{ - int ret; - - ret = pci_ids_table_update(driver->pci_driver.id_table); - if (ret) - return; - mlx5_pci_driver.drv_flags |= driver->pci_driver.drv_flags; - TAILQ_INSERT_TAIL(&drv_list, driver, next); -} - -/********** New common PCI bus driver ********/ - bool mlx5_dev_is_pci(const struct rte_device *dev) { diff --git a/drivers/common/mlx5/mlx5_common_pci.h b/drivers/common/mlx5/mlx5_common_pci.h deleted file mode 100644 index de89bb98bc..0000000000 --- a/drivers/common/mlx5/mlx5_common_pci.h +++ /dev/null @@ -1,77 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright 2020 Mellanox Technologies, Ltd - */ - -#ifndef _MLX5_COMMON_PCI_H_ -#define _MLX5_COMMON_PCI_H_ - -/** - * @file - * - * RTE Mellanox PCI Driver Interface - * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex and - * compress devices. This layer enables creating such multiple class of devices - * on a single PCI device by allowing to bind multiple class specific device - * driver to attach to mlx5_pci driver. - * - * ----------- ------------ ------------- ---------------- - * | mlx5 | | mlx5 | | mlx5 | | mlx5 | - * | net pmd | | vdpa pmd | | regex pmd | | compress pmd | - * ----------- ------------ ------------- ---------------- - * \ \ / / - * \ \ / / - * \ \_--------------_/ / - * \_______________| mlx5 |_______________/ - * | pci common | - * -------------- - * | - * ----------- - * | mlx5 | - * | pci dev | - * ----------- - * - * - mlx5 pci driver binds to mlx5 PCI devices defined by PCI - * ID table of all related mlx5 PCI devices. - * - mlx5 class driver such as net, vdpa, regex PMD defines its - * specific PCI ID table and mlx5 bus driver probes matching - * class drivers. - * - mlx5 pci bus driver is cental place that validates supported - * class combinations. - */ - -#ifdef __cplusplus -extern "C" { -#endif /* __cplusplus */ - -#include -#include - -#include - -void mlx5_common_pci_init(void); - -/** - * A structure describing a mlx5 pci driver. - */ -struct mlx5_pci_driver { - struct rte_pci_driver pci_driver; /**< Inherit core pci driver. */ - uint32_t driver_class; /**< Class of this driver, enum mlx5_class */ - TAILQ_ENTRY(mlx5_pci_driver) next; -}; - -/** - * Register a mlx5_pci device driver. - * - * @param driver - * A pointer to a mlx5_pci_driver structure describing the driver - * to be registered. - */ -__rte_internal -void -mlx5_pci_driver_register(struct mlx5_pci_driver *driver); - -#ifdef __cplusplus -} -#endif /* __cplusplus */ - -#endif /* _MLX5_COMMON_PCI_H_ */ diff --git a/drivers/common/mlx5/mlx5_common_private.h b/drivers/common/mlx5/mlx5_common_private.h index 72df9aef35..1beeaae50e 100644 --- a/drivers/common/mlx5/mlx5_common_private.h +++ b/drivers/common/mlx5/mlx5_common_private.h @@ -30,6 +30,7 @@ int mlx5_common_dev_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, /* Common PCI bus driver: */ +void mlx5_common_pci_init(void); void mlx5_common_driver_on_register_pci(struct mlx5_class_driver *driver); bool mlx5_dev_pci_match(const struct mlx5_class_driver *drv, const struct rte_device *dev); diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 3c21719975..b10e1c4646 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -129,17 +129,13 @@ INTERNAL { mlx5_nl_vlan_vmwa_create; # WINDOWS_NO_EXPORT mlx5_nl_vlan_vmwa_delete; # WINDOWS_NO_EXPORT - mlx5_pci_driver_register; - mlx5_os_alloc_pd; mlx5_os_dealloc_pd; mlx5_os_dereg_mr; - mlx5_os_get_ibv_device; # WINDOWS_NO_EXPORT mlx5_os_reg_mr; mlx5_os_umem_dereg; mlx5_os_umem_reg; mlx5_realloc; - mlx5_translate_port_name; # WINDOWS_NO_EXPORT }; From patchwork Thu May 27 14:01:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93492 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 05D47A0548; Thu, 27 May 2021 16:02:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8E07C40150; Thu, 27 May 2021 16:02:31 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2046.outbound.protection.outlook.com [40.107.94.46]) by mails.dpdk.org (Postfix) with ESMTP id 38BFD40143 for ; Thu, 27 May 2021 16:02:30 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hklF4c2mqvNPBigpUb/8Ay3yyoCi/hMn0hsuBTjICOi4MPsY9JT8NXC126O2NCH1h2xP8DuD+AKzIEZscAhss1h6+cqtdrl7TKdS79mn647pI5ula2bV5LKJzQ0ueAgIhNZzElGcA4NSVCZ+sj+MRycXRkOYZR8gk4XrZx+BGRbdspZOT3YSazS0mgKWGll2vCztu1/I2ebAW5BjZol68pMzkG5gAgImLyFm2QgXLSDiPT4S5Bk/6oqjZoKXO6sUcLD//wG7fj7P3HQ0yb3SL7LVZ7evCA3+YcjuO3ZUrhVZGxDGQtn3JcIjHTBOfma29Lnm+lFwbqxK7BtDKINHlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uiLwHZ6e4TiYvd14I2h73mMUy7vVYdY4j4QmNigEC34=; b=kCH9KE+g7Mv/51Sf/URqzDG3h994F2Ayn3nCcxxHc19JQx7193YgHZRRtmjBfDon76/EDCaeu2ptBw1k7osJkoL6YseHhxlFmkeHt2LXPZvpVAKQEb6vvZ2BMhABN5x+40W0ImTwGDD12keAGQ3t/g0OoUnfysGJOmA1G4GXIfJLn+N39GwBqJItEfc5lvdNEimMs7Bg/1EYPtZJ3eYcm+euvoMPx61VYtGo9bNIQXoUgtcszePFnQgkQXahEtySopaoigF23fQ8NfvLbaTfp0TBHp4rmhlxir3hdamSgnYy8ilLhkn4R0OMmB3Z8uenMevhIoGAtE4w6FqsfQKCxA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=ashroe.eu smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uiLwHZ6e4TiYvd14I2h73mMUy7vVYdY4j4QmNigEC34=; b=sFOWr5gAfMVL9Ne0XoGUnVvCEknWVL7ctWuwrcPhWHYgd3g0fiGm9RNIN6C7qb5hqmg1ZV7Ccdh3xoVnk/d0Y/4oLMssRCoZitDuXZ8tvQdjclJ44t3no7vENovwwGFeWEpWo8Qtqj8EeJwpA5QC5AAWwurbqLq9y1NbkU+aymPzVhifMVh1EQzCuC+8d1VylP+cUVhOMhvO7mbzkUFkbx7HFA8hy1pjA4rKur4eeUZSfU9D1Zwl6LL7shjeKYwkpLbrdpEx+HjUbhYpLEDy4zrM8BVbyqHQAT3cgc049SbpKa1EEf9AaYnIMRmm5LO90CqZ1c9qVNUqP+8scjruOw== Received: from BN1PR10CA0024.namprd10.prod.outlook.com (2603:10b6:408:e0::29) by DM4PR12MB5280.namprd12.prod.outlook.com (2603:10b6:5:39d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.22; Thu, 27 May 2021 14:02:28 +0000 Received: from BN8NAM11FT039.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e0:cafe::a3) by BN1PR10CA0024.outlook.office365.com (2603:10b6:408:e0::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20 via Frontend Transport; Thu, 27 May 2021 14:02:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; ashroe.eu; dkim=none (message not signed) header.d=none;ashroe.eu; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT039.mail.protection.outlook.com (10.13.177.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 14:02:27 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 14:02:17 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Wang Haiyue , Thomas Monjalon , Parav Pandit , Ray Kinsella , Neil Horman Date: Thu, 27 May 2021 17:01:58 +0300 Message-ID: <20210527140202.19377-1-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fca4721a-0fe9-4cdf-2d88-08d921180fdd X-MS-TrafficTypeDiagnostic: DM4PR12MB5280: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JdN4W/v9NZx5Yq6WJOEoU9Q4CCbub7Mb1FJGceg1BZbqceGtsvEbfXnUtSpVpIIFkLQzvLJcgUWWP7SeIVnVyArFCOAsKABnvHuGqYGVEIwsn2qinQ6rqTio4vmuwy6k4kKakWYZhrY/WMcjY/yTnFKlO5FEJGaOY8pzglgYl5+s8IkZda2M/Gm61DQERphAddB4705HF0aKg0wRVdXj8B6/xdrnCqI+DToCE802iaUUD2oI4opJnFdXcESTTuRzKQxTZfE5Cz8LBykg2bnHJiLGLG3SG3S3sdOC/O/caBeUKGHuyRbF6ocIVysFutjB3PsN2iwCnr1vT4NB0TmXzD+sbQwAyiTPnrTZoX6lpKaXgCa/aZGwDmLhqXZtAQbR+jCYBYXeniaBKH1YuVopfcccH/QjbEshMhUpKD/YAKQspB5Ws4A54VVTY4tasYvp2aFTS7gNQVnZ/rhgFvVxQIBiTOxHOPYMt3/BzFkEzviw3eyIPF1PoPoHyAKFd+GK7xa3QLZrF/o5r1EYXWVLnuf/G3d3aIVGU4feJk0OL8Cjc7uOF60rvYJ2lYr5wUcfsN/V6dWNwBTfgHxfsowPwqfQ6qS0QM12Z3OH92Q5hbCJNFi9GAGGUfvTInD2Z4BB9n7dAu9x8QaxxBq2rudyHx7CIRJf7HSCKO7JyTQXcEPbu7CB2MjexsCGEYagHh/5twDrNwmXQNJRJr9rK7gy0bPThnXNirO9bmFb0+3ipjThw1VO13qenmVTw//0l5tBib2DZ/atc302eNn+5WZyXQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(136003)(346002)(396003)(39860400002)(36840700001)(46966006)(2616005)(5660300002)(2906002)(16526019)(6862004)(6666004)(30864003)(336012)(4326008)(47076005)(6286002)(86362001)(8936002)(8676002)(186003)(82310400003)(82740400003)(7696005)(426003)(6636002)(83380400001)(966005)(55016002)(26005)(36906005)(54906003)(70206006)(37006003)(36860700001)(36756003)(356005)(70586007)(478600001)(1076003)(7636003)(316002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 14:02:27.6801 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fca4721a-0fe9-4cdf-2d88-08d921180fdd X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT039.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5280 Subject: [dpdk-dev] [RFC 10/14] bus/auxiliary: introduce auxiliary bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Auxiliary [1] provides a way to split function into child-devices representing sub-domains of functionality. Each auxiliary_device represents a part of its parent functionality. Auxiliary device is identified by unique device name, sysfs path: /sys/bus/auxiliary/devices/ [1] kernel auxiliary bus document: https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.html Signed-off-by: Xueming Li Cc: Wang Haiyue --- MAINTAINERS | 5 + drivers/bus/auxiliary/auxiliary_common.c | 408 ++++++++++++++++++++++ drivers/bus/auxiliary/auxiliary_params.c | 58 +++ drivers/bus/auxiliary/linux/auxiliary.c | 147 ++++++++ drivers/bus/auxiliary/meson.build | 11 + drivers/bus/auxiliary/private.h | 120 +++++++ drivers/bus/auxiliary/rte_bus_auxiliary.h | 199 +++++++++++ drivers/bus/auxiliary/version.map | 10 + drivers/bus/meson.build | 1 + 9 files changed, 959 insertions(+) create mode 100644 drivers/bus/auxiliary/auxiliary_common.c create mode 100644 drivers/bus/auxiliary/auxiliary_params.c create mode 100644 drivers/bus/auxiliary/linux/auxiliary.c create mode 100644 drivers/bus/auxiliary/meson.build create mode 100644 drivers/bus/auxiliary/private.h create mode 100644 drivers/bus/auxiliary/rte_bus_auxiliary.h create mode 100644 drivers/bus/auxiliary/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 5877a16971..eaf691ca6a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -525,6 +525,11 @@ F: doc/guides/mempool/octeontx2.rst Bus Drivers ----------- +Auxiliary bus driver +M: Parav Pandit +M: Xueming Li +F: drivers/bus/auxiliary/ + Intel FPGA bus M: Rosen Xu F: drivers/bus/ifpga/ diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c new file mode 100644 index 0000000000..cef85ae991 --- /dev/null +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -0,0 +1,408 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Mellanox Technologies, Ltd + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "private.h" +#include "rte_bus_auxiliary.h" + + +int auxiliary_bus_logtype; + +static struct rte_devargs * +auxiliary_devargs_lookup(const char *name) +{ + struct rte_devargs *devargs; + + RTE_EAL_DEVARGS_FOREACH("auxiliary", devargs) { + if (strcmp(devargs->name, name) == 0) + return devargs; + } + return NULL; +} + +/* + * Test whether the auxiliary device exist + */ +__rte_weak bool +auxiliary_dev_exists(const char *name) +{ + RTE_SET_USED(name); + return false; +} + +/* + * Scan the content of the auxiliary bus, and the devices in the devices + * list + */ +__rte_weak int +auxiliary_scan(void) +{ + return 0; +} + +void +auxiliary_on_scan(struct rte_auxiliary_device *aux_dev) +{ + aux_dev->device.devargs = auxiliary_devargs_lookup(aux_dev->name); +} + +/* + * Match the auxiliary Driver and Device using driver function. + */ +bool +auxiliary_match(const struct rte_auxiliary_driver *aux_drv, + const struct rte_auxiliary_device *aux_dev) +{ + if (aux_drv->match == NULL) + return false; + return aux_drv->match(aux_dev->name); +} + +/* + * Call the probe() function of the driver. + */ +static int +rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *dr, + struct rte_auxiliary_device *dev) +{ + enum rte_iova_mode iova_mode; + int ret; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + /* The device is not blocked; Check if driver supports it */ + if (!auxiliary_match(dr, dev)) + /* Match of device and driver failed */ + return 1; + + AUXILIARY_LOG(DEBUG, "Auxiliary device %s on NUMA socket %i\n", + dev->name, dev->device.numa_node); + + /* no initialization when marked as blocked, return without error */ + if (dev->device.devargs != NULL && + dev->device.devargs->policy == RTE_DEV_BLOCKED) { + AUXILIARY_LOG(INFO, " Device is blocked, not initializing\n"); + return -1; + } + + if (dev->device.numa_node < 0) { + AUXILIARY_LOG(WARNING, " Invalid NUMA socket, default to 0\n"); + dev->device.numa_node = 0; + } + + AUXILIARY_LOG(DEBUG, " Probe driver: %s\n", dr->driver.name); + + iova_mode = rte_eal_iova_mode(); + if ((dr->drv_flags & RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA) && + iova_mode != RTE_IOVA_VA) { + AUXILIARY_LOG(ERR, " Expecting VA IOVA mode but current mode is PA, not initializing\n"); + return -EINVAL; + } + + dev->driver = dr; + + AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (socket %i)\n", + dr->driver.name, dev->name, dev->device.numa_node); + ret = dr->probe(dr, dev); + if (ret) + dev->driver = NULL; + else + dev->device.driver = &dr->driver; + + return ret; +} + +/* + * Call the remove() function of the driver. + */ +static int +rte_auxiliary_driver_remove_dev(struct rte_auxiliary_device *dev) +{ + struct rte_auxiliary_driver *dr; + int ret = 0; + + if (dev == NULL) + return -EINVAL; + + dr = dev->driver; + + AUXILIARY_LOG(DEBUG, "Auxiliary device %s on NUMA socket %i\n", + dev->name, dev->device.numa_node); + + AUXILIARY_LOG(DEBUG, " remove driver: %s %s\n", + dev->name, dr->driver.name); + + if (dr->remove) { + ret = dr->remove(dev); + if (ret < 0) + return ret; + } + + /* clear driver structure */ + dev->driver = NULL; + dev->device.driver = NULL; + + return 0; +} + +/* + * Call the probe() function of all registered driver for the given device. + * Return < 0 if initialization failed. + * Return 1 if no driver is found for this device. + */ +static int +auxiliary_probe_all_drivers(struct rte_auxiliary_device *dev) +{ + struct rte_auxiliary_driver *dr; + int rc; + + if (dev == NULL) + return -EINVAL; + + FOREACH_DRIVER_ON_AUXILIARYBUS(dr) { + if (!dr->match(dev->name)) + continue; + + rc = rte_auxiliary_probe_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return rc; + if (rc > 0) + /* positive value means driver doesn't support it */ + continue; + return 0; + } + return 1; +} + +/* + * Scan the content of the auxiliary bus, and call the probe() function for + * + * all registered drivers that have a matching entry in its id_table + * for discovered devices. + */ +static int +auxiliary_probe(void) +{ + struct rte_auxiliary_device *dev = NULL; + size_t probed = 0, failed = 0; + int ret = 0; + + FOREACH_DEVICE_ON_AUXILIARYBUS(dev) { + probed++; + + ret = auxiliary_probe_all_drivers(dev); + if (ret < 0) { + if (ret != -EEXIST) { + AUXILIARY_LOG(ERR, "Requested device %s cannot be used\n", + dev->name); + rte_errno = errno; + failed++; + } + ret = 0; + } + } + + return (probed && probed == failed) ? -1 : 0; +} + +static int +auxiliary_parse(const char *name, void *addr) +{ + struct rte_auxiliary_driver *dr = NULL; + const char **out = addr; + + FOREACH_DRIVER_ON_AUXILIARYBUS(dr) { + if (dr->match(name)) + break; + } + if (dr != NULL && addr != NULL) + *out = name; + return dr != NULL ? 0 : -1; +} + +/* register a driver */ +void +rte_auxiliary_register(struct rte_auxiliary_driver *driver) +{ + TAILQ_INSERT_TAIL(&auxiliary_bus.driver_list, driver, next); + driver->bus = &auxiliary_bus; +} + +/* unregister a driver */ +void +rte_auxiliary_unregister(struct rte_auxiliary_driver *driver) +{ + TAILQ_REMOVE(&auxiliary_bus.driver_list, driver, next); + driver->bus = NULL; +} + +/* Add a device to auxiliary bus */ +void +auxiliary_add_device(struct rte_auxiliary_device *aux_dev) +{ + TAILQ_INSERT_TAIL(&auxiliary_bus.device_list, aux_dev, next); +} + +/* Insert a device into a predefined position in auxiliary bus */ +void +auxiliary_insert_device(struct rte_auxiliary_device *exist_aux_dev, + struct rte_auxiliary_device *new_aux_dev) +{ + TAILQ_INSERT_BEFORE(exist_aux_dev, new_aux_dev, next); +} + +/* Remove a device from auxiliary bus */ +static void +rte_auxiliary_remove_device(struct rte_auxiliary_device *auxiliary_dev) +{ + TAILQ_REMOVE(&auxiliary_bus.device_list, auxiliary_dev, next); +} + +static struct rte_device * +auxiliary_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, + const void *data) +{ + const struct rte_auxiliary_device *pstart; + struct rte_auxiliary_device *adev; + + if (start != NULL) { + pstart = RTE_DEV_TO_AUXILIARY_CONST(start); + adev = TAILQ_NEXT(pstart, next); + } else { + adev = TAILQ_FIRST(&auxiliary_bus.device_list); + } + while (adev != NULL) { + if (cmp(&adev->device, data) == 0) + return &adev->device; + adev = TAILQ_NEXT(adev, next); + } + return NULL; +} + +static int +auxiliary_plug(struct rte_device *dev) +{ + if (!auxiliary_dev_exists(dev->name)) + return -ENOENT; + return auxiliary_probe_all_drivers(RTE_DEV_TO_AUXILIARY(dev)); +} + +static int +auxiliary_unplug(struct rte_device *dev) +{ + struct rte_auxiliary_device *adev; + int ret; + + adev = RTE_DEV_TO_AUXILIARY(dev); + ret = rte_auxiliary_driver_remove_dev(adev); + if (ret == 0) { + rte_auxiliary_remove_device(adev); + rte_devargs_remove(dev->devargs); + free(adev); + } + return ret; +} + +static int +auxiliary_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len) +{ + struct rte_auxiliary_device *aux_dev = RTE_DEV_TO_AUXILIARY(dev); + + if (dev == NULL || !aux_dev->driver) { + rte_errno = EINVAL; + return -1; + } + if (aux_dev->driver->dma_map) + return aux_dev->driver->dma_map(aux_dev, addr, iova, len); + rte_errno = ENOTSUP; + return -1; +} + +static int +auxiliary_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, + size_t len) +{ + struct rte_auxiliary_device *aux_dev = RTE_DEV_TO_AUXILIARY(dev); + + if (dev == NULL || !aux_dev->driver) { + rte_errno = EINVAL; + return -1; + } + if (aux_dev->driver->dma_unmap) + return aux_dev->driver->dma_unmap(aux_dev, addr, iova, len); + rte_errno = ENOTSUP; + return -1; +} + +bool +auxiliary_ignore_device(const char *name) +{ + struct rte_devargs *devargs = auxiliary_devargs_lookup(name); + + switch (auxiliary_bus.bus.conf.scan_mode) { + case RTE_BUS_SCAN_ALLOWLIST: + if (devargs && devargs->policy == RTE_DEV_ALLOWED) + return false; + break; + case RTE_BUS_SCAN_UNDEFINED: + case RTE_BUS_SCAN_BLOCKLIST: + if (devargs == NULL || devargs->policy != RTE_DEV_BLOCKED) + return false; + break; + } + return true; +} + +static enum rte_iova_mode +auxiliary_get_iommu_class(void) +{ + const struct rte_auxiliary_driver *drv; + + FOREACH_DRIVER_ON_AUXILIARYBUS(drv) { + if (drv->drv_flags & RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA) + return RTE_IOVA_VA; + } + + return RTE_IOVA_DC; +} + +struct rte_auxiliary_bus auxiliary_bus = { + .bus = { + .scan = auxiliary_scan, + .probe = auxiliary_probe, + .find_device = auxiliary_find_device, + .plug = auxiliary_plug, + .unplug = auxiliary_unplug, + .parse = auxiliary_parse, + .dma_map = auxiliary_dma_map, + .dma_unmap = auxiliary_dma_unmap, + .get_iommu_class = auxiliary_get_iommu_class, + .dev_iterate = auxiliary_dev_iterate, + }, + .device_list = TAILQ_HEAD_INITIALIZER(auxiliary_bus.device_list), + .driver_list = TAILQ_HEAD_INITIALIZER(auxiliary_bus.driver_list), +}; + +RTE_REGISTER_BUS(auxiliary, auxiliary_bus.bus); +RTE_LOG_REGISTER(auxiliary_bus_logtype, bus.auxiliary, NOTICE); diff --git a/drivers/bus/auxiliary/auxiliary_params.c b/drivers/bus/auxiliary/auxiliary_params.c new file mode 100644 index 0000000000..5a1b029839 --- /dev/null +++ b/drivers/bus/auxiliary/auxiliary_params.c @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Mellanox Technologies, Ltd + */ + +#include + +#include +#include +#include +#include + +#include "private.h" +#include "rte_bus_auxiliary.h" + +enum auxiliary_params { + RTE_AUXILIARY_PARAM_NAME, +}; + +static const char * const auxiliary_params_keys[] = { + [RTE_AUXILIARY_PARAM_NAME] = "name", +}; + +static int +auxiliary_dev_match(const struct rte_device *dev, + const void *_kvlist) +{ + const struct rte_kvargs *kvlist = _kvlist; + int ret; + + ret = rte_kvargs_process(kvlist, + auxiliary_params_keys[RTE_AUXILIARY_PARAM_NAME], + rte_kvargs_strcmp, (void *)(uintptr_t)dev->name); + + return ret != 0 ? -1 : 0; +} + +void * +auxiliary_dev_iterate(const void *start, + const char *str, + const struct rte_dev_iterator *it __rte_unused) +{ + rte_bus_find_device_t find_device; + struct rte_kvargs *kvargs = NULL; + struct rte_device *dev; + + if (str != NULL) { + kvargs = rte_kvargs_parse(str, auxiliary_params_keys); + if (kvargs == NULL) { + RTE_LOG(ERR, EAL, "cannot parse argument list\n"); + rte_errno = EINVAL; + return NULL; + } + } + find_device = auxiliary_bus.bus.find_device; + dev = find_device(start, auxiliary_dev_match, kvargs); + rte_kvargs_free(kvargs); + return dev; +} diff --git a/drivers/bus/auxiliary/linux/auxiliary.c b/drivers/bus/auxiliary/linux/auxiliary.c new file mode 100644 index 0000000000..b75bb4d4a6 --- /dev/null +++ b/drivers/bus/auxiliary/linux/auxiliary.c @@ -0,0 +1,147 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Mellanox Technologies, Ltd + */ + +#include +#include + +#include +#include +#include +#include +#include +#include + +#include "../rte_bus_auxiliary.h" +#include "../private.h" + +#define AUXILIARY_SYSFS_PATH "/sys/bus/auxiliary/devices" + +/** + * @file + * Linux auxiliary probing. + */ + +/* Scan one auxiliary sysfs entry, and fill the devices list from it. */ +static int +auxiliary_scan_one(const char *dirname, const char *name) +{ + struct rte_auxiliary_device *dev; + struct rte_auxiliary_device *dev2; + char filename[PATH_MAX]; + unsigned long tmp; + int ret; + + dev = malloc(sizeof(*dev)); + if (dev == NULL) + return -1; + + memset(dev, 0, sizeof(*dev)); + if (rte_strscpy(dev->name, name, sizeof(dev->name)) < 0) { + free(dev); + return -1; + } + dev->device.name = dev->name; + dev->device.bus = &auxiliary_bus.bus; + + /* Get numa node, default to 0 if not present */ + snprintf(filename, sizeof(filename), "%s/%s/numa_node", + dirname, name); + if (access(filename, F_OK) != -1) { + if (eal_parse_sysfs_value(filename, &tmp) == 0) + dev->device.numa_node = tmp; + else + dev->device.numa_node = -1; + } else { + dev->device.numa_node = 0; + } + + auxiliary_on_scan(dev); + + /* Device is valid, add in list (sorted) */ + TAILQ_FOREACH(dev2, &auxiliary_bus.device_list, next) { + ret = strcmp(dev->name, dev2->name); + if (ret > 0) + continue; + if (ret < 0) { + auxiliary_insert_device(dev2, dev); + } else { /* already registered */ + if (rte_dev_is_probed(&dev2->device) && + dev2->device.devargs != dev->device.devargs) { + /* To probe device with new devargs. */ + rte_devargs_remove(dev2->device.devargs); + auxiliary_on_scan(dev2); + } + free(dev); + } + return 0; + } + auxiliary_add_device(dev); + return 0; +} + +/* + * Test whether the auxiliary device exist + */ +bool +auxiliary_dev_exists(const char *name) +{ + DIR *dir; + char dirname[PATH_MAX]; + + snprintf(dirname, sizeof(dirname), "%s/%s", + AUXILIARY_SYSFS_PATH, name); + dir = opendir(dirname); + if (dir == NULL) + return false; + closedir(dir); + return true; +} + +/* + * Scan the content of the auxiliary bus, and the devices in the devices + * list + */ +int +auxiliary_scan(void) +{ + struct dirent *e; + DIR *dir; + char dirname[PATH_MAX]; + struct rte_auxiliary_driver *drv; + + dir = opendir(AUXILIARY_SYSFS_PATH); + if (dir == NULL) { + AUXILIARY_LOG(INFO, "%s not found, is auxiliary module loaded?\n", + AUXILIARY_SYSFS_PATH); + return 0; + } + + while ((e = readdir(dir)) != NULL) { + if (e->d_name[0] == '.') + continue; + + if (auxiliary_ignore_device(e->d_name)) + continue; + + snprintf(dirname, sizeof(dirname), "%s/%s", + AUXILIARY_SYSFS_PATH, e->d_name); + + /* Ignore if no driver can handle. */ + FOREACH_DRIVER_ON_AUXILIARYBUS(drv) { + if (drv->match(e->d_name)) + break; + } + if (drv == NULL) + continue; + + if (auxiliary_scan_one(dirname, e->d_name) < 0) + goto error; + } + closedir(dir); + return 0; + +error: + closedir(dir); + return -1; +} diff --git a/drivers/bus/auxiliary/meson.build b/drivers/bus/auxiliary/meson.build new file mode 100644 index 0000000000..f85608afd0 --- /dev/null +++ b/drivers/bus/auxiliary/meson.build @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2021 Mellanox Technologies, Ltd + +headers = files('rte_bus_auxiliary.h') +sources = files('auxiliary_common.c', + 'auxiliary_params.c') +if is_linux + sources += files('linux/auxiliary.c') +endif +deps += ['kvargs'] + diff --git a/drivers/bus/auxiliary/private.h b/drivers/bus/auxiliary/private.h new file mode 100644 index 0000000000..3529348900 --- /dev/null +++ b/drivers/bus/auxiliary/private.h @@ -0,0 +1,120 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Mellanox Technologies, Ltd + */ + +#ifndef _AUXILIARY_PRIVATE_H_ +#define _AUXILIARY_PRIVATE_H_ + +#include +#include +#include "rte_bus_auxiliary.h" + +extern struct rte_auxiliary_bus auxiliary_bus; +extern int auxiliary_bus_logtype; + +#define AUXILIARY_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, auxiliary_bus_logtype, "%s(): " fmt "\n", \ + __func__, ##args) + +/* Auxiliary bus iterators */ +#define FOREACH_DEVICE_ON_AUXILIARYBUS(p) \ + TAILQ_FOREACH(p, &(auxiliary_bus.device_list), next) + +#define FOREACH_DRIVER_ON_AUXILIARYBUS(p) \ + TAILQ_FOREACH(p, &(auxiliary_bus.driver_list), next) + +/** + * Test whether the auxiliary device exist + * + * @param name + * Auxiliary device name + * @return + * true on exists, false otherwise + */ +bool auxiliary_dev_exists(const char *name); + +/** + * Scan the content of the auxiliary bus, and the devices in the devices + * list + * + * @return + * 0 on success, negative on error + */ +int auxiliary_scan(void); + +/** + * Setup or update device when being scanned. + * + * @param aux_dev + * AUXILIARY device. + */ +void auxiliary_on_scan(struct rte_auxiliary_device *aux_dev); + +/** + * Validate whether a device with given auxiliary device should be ignored + * or not. + * + * @param name + * Auxiliary name of device to be validated + * @return + * true: if device is to be ignored, + * false: if device is to be scanned, + */ +bool auxiliary_ignore_device(const char *name); + +/** + * Add an auxiliary device to the auxiliary bus (append to auxiliary Device + * list). This function also updates the bus references of the auxiliary + * Device (and the generic device object embedded within. + * + * @param aux_dev + * AUXILIARY device to add + * @return void + */ +void auxiliary_add_device(struct rte_auxiliary_device *aux_dev); + +/** + * Insert an auxiliary device in the auxiliary bus at a particular location + * in the device list. It also updates the auxiliary bus reference of the + * new devices to be inserted. + * + * @param exist_aux_dev + * Existing auxiliary device in auxiliary bus + * @param new_aux_dev + * AUXILIARY device to be added before exist_aux_dev + * @return void + */ +void auxiliary_insert_device(struct rte_auxiliary_device *exist_aux_dev, + struct rte_auxiliary_device *new_aux_dev); + +/** + * Match the auxiliary Driver and Device by driver function + * + * @param aux_drv + * auxiliary driver + * @param aux_dev + * auxiliary device to match against the driver + * @return + * the driver can handle the device + */ +bool auxiliary_match(const struct rte_auxiliary_driver *aux_drv, + const struct rte_auxiliary_device *aux_dev); + +/** + * Iterate over internal devices, matching any device against the provided + * string. + * + * @param start + * Iteration starting point. + * @param str + * Device string to match against. + * @param it + * (unused) iterator structure. + * @return + * A pointer to the next matching device if any. + * NULL otherwise. + */ +void *auxiliary_dev_iterate(const void *start, const char *str, + const struct rte_dev_iterator *it); + +#endif /* _AUXILIARY_PRIVATE_H_ */ diff --git a/drivers/bus/auxiliary/rte_bus_auxiliary.h b/drivers/bus/auxiliary/rte_bus_auxiliary.h new file mode 100644 index 0000000000..d681464602 --- /dev/null +++ b/drivers/bus/auxiliary/rte_bus_auxiliary.h @@ -0,0 +1,199 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Mellanox Technologies, Ltd + */ + +#ifndef _RTE_BUS_AUXILIARY_H_ +#define _RTE_BUS_AUXILIARY_H_ + +/** + * @file + * + * RTE Auxiliary Bus Interface. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +/* Forward declarations */ +struct rte_auxiliary_driver; +struct rte_auxiliary_bus; +struct rte_auxiliary_device; + +/** + * Match function for the driver to decide if device can be handled. + * + * @param name + * Pointer to the auxiliary device name. + * @return + * Whether the driver can handle the auxiliary device. + */ +typedef bool(*rte_auxiliary_match_t) (const char *name); + +/** + * Initialization function for the driver called during auxiliary probing. + * + * @param drv + * Pointer to the auxiliary driver. + * @param dev + * Pointer to the auxiliary device. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int(*rte_auxiliary_probe_t) (struct rte_auxiliary_driver *drv, + struct rte_auxiliary_device *dev); + +/** + * Uninitialization function for the driver called during hotplugging. + * + * @param dev + * Pointer to the auxiliary device. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int (*rte_auxiliary_remove_t)(struct rte_auxiliary_device *dev); + +/** + * Driver-specific DMA mapping. After a successful call the device + * will be able to read/write from/to this segment. + * + * @param dev + * Pointer to the auxiliary device. + * @param addr + * Starting virtual address of memory to be mapped. + * @param iova + * Starting IOVA address of memory to be mapped. + * @param len + * Length of memory segment being mapped. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int (*rte_auxiliary_dma_map_t)(struct rte_auxiliary_device *dev, + void *addr, uint64_t iova, size_t len); + +/** + * Driver-specific DMA un-mapping. After a successful call the device + * will not be able to read/write from/to this segment. + * + * @param dev + * Pointer to the auxiliary device. + * @param addr + * Starting virtual address of memory to be unmapped. + * @param iova + * Starting IOVA address of memory to be unmapped. + * @param len + * Length of memory segment being unmapped. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int (*rte_auxiliary_dma_unmap_t)(struct rte_auxiliary_device *dev, + void *addr, uint64_t iova, size_t len); + +/** + * A structure describing an auxiliary device. + */ +struct rte_auxiliary_device { + TAILQ_ENTRY(rte_auxiliary_device) next; /**< Next probed device. */ + char name[RTE_DEV_NAME_MAX_LEN + 1]; /**< ASCII device name */ + struct rte_device device; /**< Inherit core device */ + struct rte_intr_handle intr_handle; /**< Interrupt handle */ + struct rte_auxiliary_driver *driver; /**< driver used in probing */ +}; + +/** List of auxiliary devices */ +TAILQ_HEAD(rte_auxiliary_device_list, rte_auxiliary_device); +/** List of auxiliary drivers */ +TAILQ_HEAD(rte_auxiliary_driver_list, rte_auxiliary_driver); + +/** + * Structure describing the auxiliary bus + */ +struct rte_auxiliary_bus { + struct rte_bus bus; /**< Inherit the generic class */ + struct rte_auxiliary_device_list device_list; /**< List of devices */ + struct rte_auxiliary_driver_list driver_list; /**< List of drivers */ +}; + +/** + * A structure describing an auxiliary driver. + */ +struct rte_auxiliary_driver { + TAILQ_ENTRY(rte_auxiliary_driver) next; /**< Next in list. */ + struct rte_driver driver; /**< Inherit core driver. */ + struct rte_auxiliary_bus *bus; /**< Auxiliary bus reference. */ + rte_auxiliary_match_t match; /**< Device match function. */ + rte_auxiliary_probe_t probe; /**< Device Probe function. */ + rte_auxiliary_remove_t remove; /**< Device Remove function. */ + rte_auxiliary_dma_map_t dma_map; /**< Device dma map function. */ + rte_auxiliary_dma_unmap_t dma_unmap; /**< Device dma unmap function. */ + uint32_t drv_flags; /**< Flags RTE_auxiliary_DRV_*. */ +}; + +/** + * @internal + * Helper macro for drivers that need to convert to struct rte_auxiliary_device. + */ +#define RTE_DEV_TO_AUXILIARY(ptr) \ + container_of(ptr, struct rte_auxiliary_device, device) + +#define RTE_DEV_TO_AUXILIARY_CONST(ptr) \ + container_of(ptr, const struct rte_auxiliary_device, device) + +#define RTE_ETH_DEV_TO_AUXILIARY(eth_dev) \ + RTE_DEV_TO_AUXILIARY((eth_dev)->device) + +/** Device driver needs IOVA as VA and cannot work with IOVA as PA */ +#define RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA 0x002 + +/** + * Register an auxiliary driver. + * + * @param driver + * A pointer to a rte_auxiliary_driver structure describing the driver + * to be registered. + */ +__rte_experimental +void rte_auxiliary_register(struct rte_auxiliary_driver *driver); + +/** Helper for auxiliary device registration from driver instance */ +#define RTE_PMD_REGISTER_AUXILIARY(nm, auxiliary_drv) \ + RTE_INIT(auxiliaryinitfn_##nm) \ + { \ + (auxiliary_drv).driver.name = RTE_STR(nm); \ + rte_auxiliary_register(&auxiliary_drv); \ + } \ + RTE_PMD_EXPORT_NAME(nm, __COUNTER__) + +/** + * Unregister an auxiliary driver. + * + * @param driver + * A pointer to a rte_auxiliary_driver structure describing the driver + * to be unregistered. + */ +__rte_experimental +void rte_auxiliary_unregister(struct rte_auxiliary_driver *driver); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_BUS_AUXILIARY_H_ */ diff --git a/drivers/bus/auxiliary/version.map b/drivers/bus/auxiliary/version.map new file mode 100644 index 0000000000..3d270baea7 --- /dev/null +++ b/drivers/bus/auxiliary/version.map @@ -0,0 +1,10 @@ +DPDK_21 { + local: *; +}; + +EXPERIMENTAL { + global: + + rte_auxiliary_register; + rte_auxiliary_unregister; +}; diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build index 410058de3a..45eab5233d 100644 --- a/drivers/bus/meson.build +++ b/drivers/bus/meson.build @@ -2,6 +2,7 @@ # Copyright(c) 2017 Intel Corporation drivers = [ + 'auxiliary', 'dpaa', 'fslmc', 'ifpga', From patchwork Thu May 27 14:01:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93493 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D9080A0548; Thu, 27 May 2021 16:02:42 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DE22F41104; Thu, 27 May 2021 16:02:33 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2087.outbound.protection.outlook.com [40.107.94.87]) by mails.dpdk.org (Postfix) with ESMTP id 87AE8410F5 for ; Thu, 27 May 2021 16:02:32 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KseoOUDi/STLg1iNtJUBLsR3yzdNphdTFk4AuwHo5LNMC67ictG2vOUfTfO0YzgGJCb7RVulsvdhRWbV/L9Ho904jT95/YgHraHkM/s4bDQEYNXK6qSAxiJDWAnRj/Ob7rNtRnqyhYDxsq85iAZzf5kEQZ52TKShpIw5u3X2Z25hqFX+A8P5QIXDhV8fGkeRvADRCtYh0FfiMQ5H4AWkTK+DeQpOqBi+iCRvW/Ty2L7w5WN2kRCBPfUyiC5Bcz9x+ObTBtvz0bPE1uGEImATUJ3ZIfKen6IbIOtirLcA+23ck4HQgGefd1TBwvKQ4eN1ydKk86ptNFdRRqFIOv7H8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DwSiLrJzpvEkEGLRsG6xR5DZJwMQCybhN/AeQg3Ykrk=; b=RYdKWYcabFv0F5uleeReLCSZGOmZK6fZ+sKUX1KBVZgr3qoKcML8sbROK2kX77I0f9cQVk/TLLvIDjsjSTqFKazhjOior3+L66pfyMITwUDf4l0kF6LYklK1PKMhmFQ1/UJCAP94bk0mTtLvABzIgYoaChjP7Vb4SJn4P5pvezwdpBVXYh2bnsIYsfCqhBaTx6yOLh2M6iDPWXCeFqqZj8j6AxqDiLymceuyq5s3O18VaGZbM3P7fZd+beBhMDe7p1D8acHsoUhY1cQcA3hIl7IiRC+ThEOmL21M3/jNp2mX6yVL6ZReQyfqAcLND1CGccH5nJaODluDgx6ULY79AQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=tuxdriver.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DwSiLrJzpvEkEGLRsG6xR5DZJwMQCybhN/AeQg3Ykrk=; b=Hw0u+3uBl4aHJkR90kLa85bR9KEBXcgtFVwtANN1Viyy/lToYZL/XH/cWRiPKh9RITVKN29dQCIEVkDQly123agKQpoqCECerxfaUgVFG1oAbKrlNgFUB1Sq4nr4AFC39xjQOQd4tbMiZ+QrXglqyJiTGx5kuTZx1RqFIHgTOx4og4Op5a3gDJRV4khHBGpXX7sHFfU2Fj86gYpVFJH25zVFp6D79dRIBoQX8tgN5lGleVY7mws06pv6qnNaTQAiSzymnMVhh2RjhCDuUI9j0Y2hQwId2EvH4gcfs2KvsBQfpH7iyXBOFP4C/iMaFl28ZYsr1OP/B6XBCA/lqqtvRw== Received: from BN1PR10CA0008.namprd10.prod.outlook.com (2603:10b6:408:e0::13) by DM5PR1201MB2474.namprd12.prod.outlook.com (2603:10b6:3:e3::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4150.26; Thu, 27 May 2021 14:02:30 +0000 Received: from BN8NAM11FT039.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e0:cafe::bf) by BN1PR10CA0008.outlook.office365.com (2603:10b6:408:e0::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20 via Frontend Transport; Thu, 27 May 2021 14:02:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; tuxdriver.com; dkim=none (message not signed) header.d=none;tuxdriver.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT039.mail.protection.outlook.com (10.13.177.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 14:02:30 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 14:02:19 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Ray Kinsella , Neil Horman Date: Thu, 27 May 2021 17:01:59 +0300 Message-ID: <20210527140202.19377-2-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527140202.19377-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> <20210527140202.19377-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b6fd0571-37ac-4b36-0ac2-08d92118118f X-MS-TrafficTypeDiagnostic: DM5PR1201MB2474: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:147; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LQU4OJOBDQlNxJ0/jSv2XtufZt9BNYgsMIip17/EU7IeFbv/6gbKodyLdBXDVhA7ryiVHPV3Orz2y2wsPMjJojyJ9Y3mlO288pJKR3RtF8lKU1efyLokBtaMTcVKyShjOOqkIghbK+7IV2I+42TTjyjqxUFPGE4ACHyFQ4mTW3i5ao1tWHdU5C9hqwxwWUtUvIhRn0xwDg4baVD9wptUi8K1LJtaHT1gfPBo+2xcsvuahM+jjAQm7+ps0mwrRjIMI/fTJL0RypkLPvvHn3losADTmKftZShi6pbZ0UbxVx8K/bGIQ9ohyo+IAqP9Z4pToQMrGy+fASNeL9uc+u3UVt3x650g7h1pSZ3o1sa2rRDBq3wNrLAN9mmDkl9RRO770tmFDV0mlpy4CKcgB1c50z6N0t0CIKA5kALGxaNdqdHBDXJv/T56OpFNKyh/eus1rlnGmHt7Jud7Cu9rY0tYEhYNbGv/4r3Kbnk1tNrlKoaUTLsvIj0lbWH/pB0RcX5RtXtf1NasRSwJeRyKrIXQQbPqzRaPhswQDhFu8YpMv7roL4NmO9UWxFMJPY80WBxUHDBXNTxNdcr+A4pV1HEsw4NbhZVWusIOKLy6BI3jsDkv0/HBXl3+r6LFekH2ln9abkrhewzW3mXMaDviOWMlSngVgY2Tc9rFHCZjR0SdHkI= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(39860400002)(136003)(376002)(396003)(36840700001)(46966006)(70206006)(1076003)(4326008)(6636002)(8676002)(426003)(83380400001)(70586007)(5660300002)(316002)(6666004)(54906003)(2616005)(478600001)(36756003)(26005)(82310400003)(7696005)(36906005)(8936002)(356005)(336012)(55016002)(7636003)(86362001)(82740400003)(6862004)(36860700001)(47076005)(186003)(16526019)(37006003)(6286002)(2906002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 14:02:30.5245 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b6fd0571-37ac-4b36-0ac2-08d92118118f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT039.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB2474 Subject: [dpdk-dev] [RFC 11/14] common/mlx5: support auxiliary bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds auxiliary bus driver and delegate to registered internal mlx5 common device drivers, i.e. eth, vdpa... Current major target is to support SubFunction on auxiliary bus. As a limitation of current driver, numa node of device is detected from PCI bus of device symbol link, will remove once numa node file available on sysfs. Signed-off-by: Xueming Li --- drivers/common/mlx5/linux/meson.build | 3 + .../common/mlx5/linux/mlx5_common_auxiliary.c | 173 ++++++++++++++++++ drivers/common/mlx5/linux/mlx5_common_verbs.c | 5 +- drivers/common/mlx5/mlx5_common.c | 3 + drivers/common/mlx5/mlx5_common.h | 5 + drivers/common/mlx5/mlx5_common_private.h | 5 + drivers/common/mlx5/version.map | 2 + 7 files changed, 195 insertions(+), 1 deletion(-) create mode 100644 drivers/common/mlx5/linux/mlx5_common_auxiliary.c diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build index 007834a49b..a1070acb77 100644 --- a/drivers/common/mlx5/linux/meson.build +++ b/drivers/common/mlx5/linux/meson.build @@ -48,10 +48,13 @@ endif sources += files('mlx5_nl.c') sources += files('mlx5_common_os.c') sources += files('mlx5_common_verbs.c') +sources += files('mlx5_common_auxiliary.c') if not dlopen_ibverbs sources += files('mlx5_glue.c') endif +deps += ['bus_auxiliary'] + # To maintain the compatibility with the make build system # mlx5_autoconf.h file is still generated. # input array for meson member search: diff --git a/drivers/common/mlx5/linux/mlx5_common_auxiliary.c b/drivers/common/mlx5/linux/mlx5_common_auxiliary.c new file mode 100644 index 0000000000..f16fd2ee37 --- /dev/null +++ b/drivers/common/mlx5/linux/mlx5_common_auxiliary.c @@ -0,0 +1,173 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies Ltd + */ + +#include +#include +#include +#include +#include +#include +#include "eal_filesystem.h" + +#include "mlx5_common_utils.h" +#include "mlx5_common_private.h" + +#define AUXILIARY_SYSFS_PATH "/sys/bus/auxiliary/devices" +#define MLX5_AUXILIARY_PREFIX "mlx5_core.sf." + +int +mlx5_auxiliary_get_child_name(const char *dev, const char *node, + char *child, size_t size) +{ + DIR *dir; + struct dirent *dent; + MKSTR(path, "%s/%s%s", AUXILIARY_SYSFS_PATH, dev, node); + + dir = opendir(path); + if (dir == NULL) { + rte_errno = errno; + return -rte_errno; + } + /* Get the first file name. */ + while ((dent = readdir(dir)) != NULL) { + if (dent->d_name[0] != '.') + break; + } + closedir(dir); + if (dent == NULL) { + rte_errno = ENOENT; + return -rte_errno; + } + if (rte_strscpy(child, dent->d_name, size) < 0) + return -rte_errno; + return 0; +} + +static int +mlx5_auxiliary_get_pci_path(const struct rte_auxiliary_device *dev, + char *sysfs_pci, size_t size) +{ + char sysfs_real[PATH_MAX]; + char *last_slash; + MKSTR(sysfs_aux, "%s/%s", AUXILIARY_SYSFS_PATH, dev->name); + + if (realpath(sysfs_aux, sysfs_real) == NULL) { + rte_errno = errno; + return -rte_errno; + } + last_slash = strrchr(sysfs_real, '/'); + if (last_slash == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + *last_slash = '\0'; + if (rte_strscpy(sysfs_pci, sysfs_real, size) < 0) + return -rte_errno; + return 0; +} + +static int +mlx5_auxiliary_get_numa(const struct rte_auxiliary_device *dev) +{ + unsigned long numa; + char numa_path[PATH_MAX]; + + if (mlx5_auxiliary_get_pci_path(dev, numa_path, sizeof(numa_path)) != 0) + return SOCKET_ID_ANY; + if (strcat(numa_path, "/numa_node") == NULL) { + rte_errno = ENAMETOOLONG; + return SOCKET_ID_ANY; + } + if (eal_parse_sysfs_value(numa_path, &numa) != 0) { + rte_errno = EINVAL; + return SOCKET_ID_ANY; + } + return (int)numa; +} + +struct ibv_device * +mlx5_get_aux_ibv_device(const struct rte_auxiliary_device *dev) +{ + int n; + char ib_name[64]; + struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); + struct ibv_device *ibv_match = NULL; + + if (!ibv_list) { + rte_errno = ENOSYS; + return NULL; + } + if (mlx5_auxiliary_get_child_name(dev->name, "/infiniband", + ib_name, sizeof(ib_name)) != 0) + return NULL; + while (n-- > 0) { + if (strcmp(ibv_list[n]->name, ib_name) != 0) + continue; + ibv_match = ibv_list[n]; + break; + } + if (ibv_match == NULL) + rte_errno = ENOENT; + mlx5_glue->free_device_list(ibv_list); + return ibv_match; +} + +static bool +mlx5_common_auxiliary_match(const char *name) +{ + return strncmp(name, MLX5_AUXILIARY_PREFIX, + strlen(MLX5_AUXILIARY_PREFIX)) == 0; +} + +static int +mlx5_common_auxiliary_probe(struct rte_auxiliary_driver *drv __rte_unused, + struct rte_auxiliary_device *dev) +{ + dev->device.numa_node = mlx5_auxiliary_get_numa(dev); + return mlx5_common_dev_probe(&dev->device); +} + +static int +mlx5_common_auxiliary_remove(struct rte_auxiliary_device *auxiliary_dev) +{ + return mlx5_common_dev_remove(&auxiliary_dev->device); +} + +static int +mlx5_common_auxiliary_dma_map(struct rte_auxiliary_device *auxiliary_dev, + void *addr, uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_map(&auxiliary_dev->device, addr, iova, len); +} + +static int +mlx5_common_auxiliary_dma_unmap(struct rte_auxiliary_device *auxiliary_dev, + void *addr, uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_unmap(&auxiliary_dev->device, addr, iova, + len); +} + +static struct rte_auxiliary_driver mlx5_auxiliary_driver = { + .driver = { + .name = MLX5_AUXILIARY_DRIVER_NAME, + }, + .match = mlx5_common_auxiliary_match, + .probe = mlx5_common_auxiliary_probe, + .remove = mlx5_common_auxiliary_remove, + .dma_map = mlx5_common_auxiliary_dma_map, + .dma_unmap = mlx5_common_auxiliary_dma_unmap, +}; + +void mlx5_common_auxiliary_init(void) +{ + if (mlx5_auxiliary_driver.bus == NULL) + rte_auxiliary_register(&mlx5_auxiliary_driver); +} + +RTE_FINI(mlx5_common_auxiliary_driver_finish) +{ + if (mlx5_auxiliary_driver.bus != NULL) + rte_auxiliary_unregister(&mlx5_auxiliary_driver); +} diff --git a/drivers/common/mlx5/linux/mlx5_common_verbs.c b/drivers/common/mlx5/linux/mlx5_common_verbs.c index a49440ef72..856e782878 100644 --- a/drivers/common/mlx5/linux/mlx5_common_verbs.c +++ b/drivers/common/mlx5/linux/mlx5_common_verbs.c @@ -12,6 +12,7 @@ #include #include +#include #include "mlx5_common_log.h" #include "mlx5_common_utils.h" @@ -24,10 +25,12 @@ struct ibv_device * mlx5_get_ibv_device(const struct rte_device *dev) { - struct ibv_device *ibv = NULL; + struct ibv_device *ibv; if (mlx5_dev_is_pci(dev)) ibv = mlx5_get_pci_ibv_device(&RTE_DEV_TO_PCI_CONST(dev)->addr); + else + ibv = mlx5_get_aux_ibv_device(RTE_DEV_TO_AUXILIARY_CONST(dev)); if (ibv == NULL) { rte_errno = ENODEV; DRV_LOG(ERR, "Verbs device not found: %s", dev->name); diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 875668d72b..b7be713cbe 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -356,6 +356,9 @@ mlx5_class_driver_register(struct mlx5_class_driver *driver) static void mlx5_common_driver_init(void) { mlx5_common_pci_init(); +#ifdef RTE_EXEC_ENV_LINUX + mlx5_common_auxiliary_init(); +#endif } static bool mlx5_common_initialized; diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 62a0dc4bad..c9c77ce540 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -23,6 +23,7 @@ /* Reported driver name. */ #define MLX5_PCI_DRIVER_NAME "mlx5_pci" +#define MLX5_AUXILIARY_DRIVER_NAME "mlx5_auxiliary" /* Bit-field manipulation. */ #define BITFIELD_DECLARE(bf, type, size) \ @@ -140,6 +141,10 @@ struct ibv_device; __rte_internal struct ibv_device *mlx5_get_ibv_device(const struct rte_device *dev); +__rte_internal +int mlx5_auxiliary_get_child_name(const char *dev, const char *node, + char *child, size_t size); + /* Maximum number of simultaneous unicast MAC addresses. */ #define MLX5_MAX_UC_MAC_ADDRESSES 128 /* Maximum number of simultaneous Multicast MAC addresses. */ diff --git a/drivers/common/mlx5/mlx5_common_private.h b/drivers/common/mlx5/mlx5_common_private.h index 1beeaae50e..d1ab15ac43 100644 --- a/drivers/common/mlx5/mlx5_common_private.h +++ b/drivers/common/mlx5/mlx5_common_private.h @@ -6,6 +6,7 @@ #define _MLX5_COMMON_PRIVATE_H_ #include +#include #include "mlx5_common.h" @@ -36,6 +37,10 @@ bool mlx5_dev_pci_match(const struct mlx5_class_driver *drv, const struct rte_device *dev); struct ibv_device *mlx5_get_pci_ibv_device(const struct rte_pci_addr *); +/* Common auxiliary bus driver: */ +void mlx5_common_auxiliary_init(void); +struct ibv_device *mlx5_get_aux_ibv_device(const struct rte_auxiliary_device *); + #ifdef __cplusplus } #endif /* __cplusplus */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index b10e1c4646..33eb7f09bc 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -3,6 +3,8 @@ INTERNAL { haswell_broadwell_cpu; + mlx5_auxiliary_get_child_name; + mlx5_class_driver_register; mlx5_common_init; From patchwork Thu May 27 14:02:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93494 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3F57FA0548; Thu, 27 May 2021 16:02:52 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 09C234110C; Thu, 27 May 2021 16:02:35 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2055.outbound.protection.outlook.com [40.107.93.55]) by mails.dpdk.org (Postfix) with ESMTP id C136841104 for ; Thu, 27 May 2021 16:02:32 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lxp4f4KS3SO9i2fw8hOEKlcQk+ozHaqJvcpFzZ+ZiN+vP1PIVjELWA8jCfFIEHHTG06c4NtBS7qRYuDM4t0jbyNEAGiKIomJGZTOhlSR4ywgBQ1ezcoYMJHOHHWlsHLBwXcLxIQA6vsu4tnZfiMeYcsoKH9gYwotRutbRp8jt9vnhRQ0eF/XQ3cIsYOSEoOvYZ9niyCO11h5sFIW8Syz3y7HXgRojfvNmWlHmwNnipKM7Ctt9zRw1/MUU/wqcAV6eW0KSHzS25+TaD3jv4pr79qoJ/syc4HCIuMfWaETlP4QEyOpfYr7F0dSBc3ymkBWlNGiRhXxd66xejvvvLruWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ps17lqtqUffF7kUuQd/oNdqHaPmrR58DiS9pM4VVF9Y=; b=gs4fTB3/o8QahvmzEoF6v1NCrpvAksxRqVWavpivHs3AGr4vofqCWKiO/2RhfdnuemTXmcLy7Ndesdfl2X/gRaSE7grk5juDuc0aJ3e/3duXHHiA0/OGxqbJSIShdnf4Vm00lCZqChFnvln0/Bb6No66C32E2dhKe7ISM78Ar4PNvOIStFqEk8dnjPLFclQd652+vN5tSMAwbU0pc73/YAZNE9kWopxRo7+QO1BiDxRB6lO2shxTQ50KGeLZc6pqoidE/EBDkDsO9+9qrKYoYo+xsDW/QqF1D3y8x7moJTxt7OYQ4pAxPTwN8dN+zvtPUzNXp0yTGt1DaXc/OfY35A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=ashroe.eu smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ps17lqtqUffF7kUuQd/oNdqHaPmrR58DiS9pM4VVF9Y=; b=hEJoFgDWkMO98mu7W82PUS6oNUCwgtpYBh+iNfKMbrXuuCZVWzMQd3arA0gCWfVSOm4gV+Wa8euoNL2MB/TYXUwCFI3XYOiU8HhrKnHWiSF5XHFbcsinFC+6cVrWBzev2PgFqefeNf0rWf8FAJxRtMcPq/YdCZdCuuo/Z7ICZU7sSyGDr8PAIwlLEggba5fkYuq2sbJWpPF5OX56CDjtgELyx3VDmluccT96une3XyaULqw1RTegVFHadNujEDQsAcKQLY9XOdNYEHaookXeE1K2Jqjh97+tLynNNCM2uWta5rUznTN6vkPpX+LNfnZyHP7PBiiK4hIyzf2IJJmSPg== Received: from BN1PR10CA0030.namprd10.prod.outlook.com (2603:10b6:408:e0::35) by DM6PR12MB4075.namprd12.prod.outlook.com (2603:10b6:5:21d::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.21; Thu, 27 May 2021 14:02:31 +0000 Received: from BN8NAM11FT039.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e0:cafe::83) by BN1PR10CA0030.outlook.office365.com (2603:10b6:408:e0::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20 via Frontend Transport; Thu, 27 May 2021 14:02:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; ashroe.eu; dkim=none (message not signed) header.d=none;ashroe.eu; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT039.mail.protection.outlook.com (10.13.177.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 14:02:31 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 14:02:21 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad , Shahaf Shuler , Ray Kinsella , Neil Horman Date: Thu, 27 May 2021 17:02:00 +0300 Message-ID: <20210527140202.19377-3-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527140202.19377-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> <20210527140202.19377-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7eeaa84b-5cea-4206-accd-08d9211811f5 X-MS-TrafficTypeDiagnostic: DM6PR12MB4075: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:568; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4eD8IhexKYN7TxOcKhS+WObt38L+gBbA4aH3gsF6r/wJd+wL9bk64g7PPz9APIJmoAXRBHCwhZudCNACc3xV3xi98itjZdUXe6hRXttrM9NYhZsosLh2BFm0R9UIWxA3jViwthZxMIfbdQA2C3fhB33keLYhSTHnOXuRaSciIlFfMQ67i9gyy5aFB8p/afT8CHW3kHKEQQR7j44h3W9UPRZWcIiwk6PG0U/T/BySf8JR4SVVtpH492patwSaDqc/qdhDkxUxoSL/z32VRiHiKlfYq7uv+QbZ/Fp2sQGSMbBe/g8k7Lvcy7RXWjgrMJWTBD4lGhXi/q4/+owqNPlzwjYC27peSWs/uV1iKHprFIQ00APpVnwz0V5HTDtTHEKUvWA0IUAbtQOZJMRhSbt73lzfWmUP6nFF+GSvACWJFweO5srl+Kyl3DaS9oj1MULzrXjE1KL1y+d+gx96XnlyHnlEAPKlF1K6oZlvqkl5NQYzP/a9uif7nHqHC29fWZJa9i+2jmv7Do1eyox6wFzy7fVumShZPmVrQ8ptendAZN4Et6QFfj7HsRxA8h+WR1Y2PlHVkYbxDrDNR91DBOFI2XKIjmarrsVvOuGXG/inz/wbjt5JCqWGnuA390VuoE2sJSF8vW5NMnw1JjHFJdzaxw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(346002)(39860400002)(376002)(396003)(46966006)(36840700001)(26005)(7696005)(37006003)(6862004)(86362001)(55016002)(8676002)(8936002)(47076005)(6666004)(70586007)(36756003)(70206006)(316002)(16526019)(2616005)(426003)(1076003)(186003)(83380400001)(2906002)(6636002)(36906005)(7636003)(54906003)(478600001)(356005)(6286002)(336012)(5660300002)(36860700001)(82310400003)(4326008)(82740400003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 14:02:31.1951 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7eeaa84b-5cea-4206-accd-08d9211811f5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT039.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4075 Subject: [dpdk-dev] [RFC 12/14] common/mlx5: get PCI device address from any bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon A function is exported to allow retrieving the PCI address of the parent PCI device of a Sub-Function in auxiliary bus sysfs. The function mlx5_dev_to_pci_str() is accepting both PCI and auxiliary devices. In case of a PCI device, it is simply using the device name. The function mlx5_dev_to_pci_addr(), which is based on sysfs path and do not use any device object, is renamed to mlx5_get_pci_addr() for clarity purpose. Signed-off-by: Thomas Monjalon --- drivers/common/mlx5/linux/mlx5_common_auxiliary.c | 15 +++++++++++++++ drivers/common/mlx5/linux/mlx5_common_os.c | 3 +-- drivers/common/mlx5/mlx5_common.c | 14 ++++++++++++++ drivers/common/mlx5/mlx5_common.h | 12 +++++++++++- drivers/common/mlx5/mlx5_common_private.h | 2 ++ drivers/common/mlx5/version.map | 3 ++- drivers/net/mlx5/linux/mlx5_os.c | 5 ++--- 7 files changed, 47 insertions(+), 7 deletions(-) diff --git a/drivers/common/mlx5/linux/mlx5_common_auxiliary.c b/drivers/common/mlx5/linux/mlx5_common_auxiliary.c index f16fd2ee37..f97c5e7350 100644 --- a/drivers/common/mlx5/linux/mlx5_common_auxiliary.c +++ b/drivers/common/mlx5/linux/mlx5_common_auxiliary.c @@ -4,6 +4,8 @@ #include #include +#include + #include #include #include @@ -67,6 +69,19 @@ mlx5_auxiliary_get_pci_path(const struct rte_auxiliary_device *dev, return 0; } +int +mlx5_auxiliary_get_pci_str(const struct rte_auxiliary_device *dev, + char *addr, size_t size) +{ + char sysfs_pci[PATH_MAX]; + + if (mlx5_auxiliary_get_pci_path(dev, sysfs_pci, sizeof(sysfs_pci)) != 0) + return -ENODEV; + if (rte_strscpy(addr, basename(dirname(sysfs_pci)), size) < 0) + return -rte_errno; + return 0; +} + static int mlx5_auxiliary_get_numa(const struct rte_auxiliary_device *dev) { diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index cd1c305cc1..464a897072 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -24,8 +24,7 @@ const struct mlx5_glue *mlx5_glue; #endif int -mlx5_dev_to_pci_addr(const char *dev_path, - struct rte_pci_addr *pci_addr) +mlx5_get_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr) { FILE *file; char line[32]; diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index b7be713cbe..c3fdb7d23f 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -158,6 +158,20 @@ to_mlx5_device(const struct rte_device *rte_dev) return NULL; } +int +mlx5_dev_to_pci_str(const struct rte_device *dev, char *addr, size_t size) +{ + if (mlx5_dev_is_pci(dev)) + return rte_strscpy(addr, dev->name, size); +#ifdef RTE_EXEC_ENV_LINUX + return mlx5_auxiliary_get_pci_str(RTE_DEV_TO_AUXILIARY_CONST(dev), + addr, size); +#else + rte_errno = ENODEV; + return -rte_errno; +#endif +} + static void dev_release(struct mlx5_common_device *dev) { diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index c9c77ce540..4d520b16eb 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -213,6 +213,16 @@ check_cqe(volatile struct mlx5_cqe *cqe, const uint16_t cqes_n, return MLX5_CQE_STATUS_SW_OWN; } +/* + * Get PCI address string from EAL device. + * + * @return + * - Copied string length on success. + * - Negative value and rte_errno is set otherwise. + */ +__rte_internal +int mlx5_dev_to_pci_str(const struct rte_device *dev, char *addr, size_t size); + /* * Get PCI address from sysfs of a PCI-related device. * @@ -227,7 +237,7 @@ check_cqe(volatile struct mlx5_cqe *cqe, const uint16_t cqes_n, * - Negative value and rte_errno is set otherwise. */ __rte_internal -int mlx5_dev_to_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr); +int mlx5_get_pci_addr(const char *dev_path, struct rte_pci_addr *pci_addr); /* * Get kernel network interface name from sysfs IB device path. diff --git a/drivers/common/mlx5/mlx5_common_private.h b/drivers/common/mlx5/mlx5_common_private.h index d1ab15ac43..26dfc55a61 100644 --- a/drivers/common/mlx5/mlx5_common_private.h +++ b/drivers/common/mlx5/mlx5_common_private.h @@ -39,6 +39,8 @@ struct ibv_device *mlx5_get_pci_ibv_device(const struct rte_pci_addr *); /* Common auxiliary bus driver: */ void mlx5_common_auxiliary_init(void); +int mlx5_auxiliary_get_pci_str(const struct rte_auxiliary_device *dev, + char *addr, size_t size); struct ibv_device *mlx5_get_aux_ibv_device(const struct rte_auxiliary_device *); #ifdef __cplusplus diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 33eb7f09bc..afa52ec6bb 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -15,7 +15,7 @@ INTERNAL { mlx5_create_mr_ext; mlx5_dev_is_pci; - mlx5_dev_to_pci_addr; # WINDOWS_NO_EXPORT + mlx5_dev_to_pci_str; mlx5_devx_alloc_uar; # WINDOWS_NO_EXPORT @@ -77,6 +77,7 @@ INTERNAL { mlx5_get_ibv_device; # WINDOWS_NO_EXPORT mlx5_get_ifname_sysfs; # WINDOWS_NO_EXPORT + mlx5_get_pci_addr; # WINDOWS_NO_EXPORT mlx5_glue; diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index e8e6b0d5c9..4f16230fa5 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1857,7 +1857,7 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, /* Process slave interface names in the loop. */ snprintf(tmp_str, sizeof(tmp_str), "/sys/class/net/%s", ifname); - if (mlx5_dev_to_pci_addr(tmp_str, &pci_addr)) { + if (mlx5_get_pci_addr(tmp_str, &pci_addr)) { DRV_LOG(WARNING, "can not get PCI address" " for netdev \"%s\"", ifname); continue; @@ -2026,8 +2026,7 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, break; } else { /* Bonding device not found. */ - if (mlx5_dev_to_pci_addr - (ibv_list[ret]->ibdev_path, &pci_addr)) + if (mlx5_get_pci_addr(ibv_list[ret]->ibdev_path, &pci_addr)) continue; if (owner_pci.domain != pci_addr.domain || owner_pci.bus != pci_addr.bus || From patchwork Thu May 27 14:02:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93495 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A34F2A0548; Thu, 27 May 2021 16:03:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2564141112; Thu, 27 May 2021 16:02:36 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2054.outbound.protection.outlook.com [40.107.243.54]) by mails.dpdk.org (Postfix) with ESMTP id E9CDA41106 for ; Thu, 27 May 2021 16:02:32 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oI6ijxSNc85O0Q4YDQmxlVALQPpQwkMeCmHFSO1AOWhy5xxz+4FWOLWAW7RT4EWSYdlGgEN6V+INrsTqUSwHe7q6/5M5xNxdosaiZ5B2xXBfvVvN6pESn9Z1eEQEJYvZOMTF5dN3HyAs3czTrnNBk4Pp2An17WCE/zeEbWGxQDK/N9iZGVoMNw+27cw05mobyIY6drnCo9JQ7aoBSRs/sgX0Qs9jlSd0rxOy2uYOyb2z2p0yMQYZBHEK9B3wTcSoRTWcY24DU6T2QfDZbtI0QU2wGaeiS23RmA0E6fW/9P8wvn08+D4lgSulYeIRxVvrE2Rp4zolqmRVpxKH+ci9Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N6twiXeE+3ElBKdDJ3XIA9zZgGwlLdUiv974F7RXdrY=; b=P57XCO+Wwl+l89dAhYyeyqKehjkxBoredQyLHQzW0mHgNb6VAFrBAtgT8NaCegRC3uD95Ll3X155w5I4JzD7x3fKzUEyjuu5hAeJhA80s9R5DotFdxDpRlJTteBSuS7NraKdrWR+QIxjCCfcoUrk+JutSAFg9cpPPy1nZKrEimMN0cggSZHfCE//WfRSEY7QryDP/3clvorcqPMHyYA7YtRul98mlCjhPCA7rIEn1ok+Bpw5/MRm0erZuwecG2+YfHjEwjn9y2ySfBHhJtDRMd3+RItDxHSWQaBIz4FuMYpGtB5DlhPbtO3QREsq7sVX+MJp3KyOJi53M1fFnAJ3ag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N6twiXeE+3ElBKdDJ3XIA9zZgGwlLdUiv974F7RXdrY=; b=FRLpJ8l8+OHnKmUcQAMuoi4glaRBi1tXUi3UhkNgoT6HxOrRh8mgM7PFH7p4+zLUEjBfRGXDGumQ71zHSb1xbYp+zuzD4CqVY58n6jhS2NYzENj9W56IJ/92SVRnqRqlRWz8overOm3n+AoO8aq5IuNnM1N/baZsIQWKcQC97Y2pnWFhPJ/Ca5u17iu3jxryPyuDE6XL1QCrlWCHDRergFO66uu11A+xwb5FsarEx4izy2d6PIu6bdAGhjHaZm++0dCqgHWVu+k8JBFnMw99BZXQwi4u42Y7vSduqJxvyoNwqkqBSuChUOQac4h47EcWqpmRRbkvka9pxroR9e65Lw== Received: from BN9PR03CA0677.namprd03.prod.outlook.com (2603:10b6:408:10e::22) by DM5PR1201MB0075.namprd12.prod.outlook.com (2603:10b6:4:54::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20; Thu, 27 May 2021 14:02:31 +0000 Received: from BN8NAM11FT055.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10e:cafe::14) by BN9PR03CA0677.outlook.office365.com (2603:10b6:408:10e::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.21 via Frontend Transport; Thu, 27 May 2021 14:02:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT055.mail.protection.outlook.com (10.13.177.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 14:02:31 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 14:02:23 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: Thomas Monjalon , , , Matan Azrad Date: Thu, 27 May 2021 17:02:01 +0300 Message-ID: <20210527140202.19377-4-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527140202.19377-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> <20210527140202.19377-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: eba0d1bf-e263-4b31-6e75-08d92118122e X-MS-TrafficTypeDiagnostic: DM5PR1201MB0075: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:510; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TwNyrZy90rQi8l+cyfKbFMKBSPZKCKXJ5LZMRbO6Cqrn2hiRIvQ4NrtcsyLOS5yK+ZleOwvL5UEPsvMCgVCfbSuQaeBKXb4r/Bn4Xomo1jw+Q+dkbmgjpd6MUMd2ld7nwXhuOBrzq7Z9+KDJcFrDNWTDQeRzXyfgcKrQ7wGBQaJU6RrhmVBSmuT5swIhHAHQ+Pb0EGqdHFb1n7W5NcKQtWP9/kPQuVcI+ZNDhT110L8FRfROwCekmCmD/KNFFjfIHSf+2wz1JP/FRabPSqngFvkPHzT8ugwFNNRVZkU+PX9AN/rkYJzJZfL3rGlgwNdLgdsi4hxnXo7+3ba0+9uJ2DWUfwjkU31x7IHkcQaCSH8sX37J3YwMTOWFt8TI2GLeX/L3XejUVhk65XgFfHZGG09LTFoWaQ5480PZCPEx4VTnNyvTXoaT+RwL5K2SEiwNRd/AWHLIy8y7URcTbRhWYATF7cONl6vXqGan8C4mcvzs4thzH/XkCwfu+VVaTO9JQaJjTZd20davgAe8HaESVC3ct/RQzHeegFXC8sMtS4jPpdQlEawQ9Ey+viBzeJPLg/nDMLWimG9Nmhqz4J637N/51cGju8J3xAyQ+K7VtoSsIYYzCc98FTIORlNVzPy/w1LoU1H7UP2ZU+Vl3N2sDUO0SH7hEm0UC/fhD1IkmQo= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(136003)(396003)(346002)(39860400002)(36840700001)(46966006)(107886003)(6286002)(70206006)(2616005)(36906005)(1076003)(6666004)(26005)(356005)(82740400003)(70586007)(83380400001)(55016002)(8676002)(86362001)(8936002)(54906003)(37006003)(4326008)(36860700001)(316002)(5660300002)(82310400003)(478600001)(6636002)(2906002)(36756003)(6862004)(336012)(7636003)(186003)(4744005)(47076005)(16526019)(7696005)(426003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 14:02:31.5821 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: eba0d1bf-e263-4b31-6e75-08d92118122e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT055.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB0075 Subject: [dpdk-dev] [RFC 13/14] vdpa/mlx5: support SubFunction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon Signed-off-by: Thomas Monjalon --- drivers/vdpa/mlx5/mlx5_vdpa.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 967234193f..2f4420becd 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -553,9 +553,13 @@ mlx5_vdpa_sys_roce_disable(const char *addr) static int mlx5_vdpa_roce_disable(struct rte_device *dev) { + char pci_addr[PCI_PRI_STR_SIZE]; + + if (mlx5_dev_to_pci_str(dev, pci_addr, sizeof(pci_addr)) < 0) + return -rte_errno; /* Firstly try to disable ROCE by Netlink and fallback to sysfs. */ - if (mlx5_vdpa_nl_roce_disable(dev->name) != 0 && - mlx5_vdpa_sys_roce_disable(dev->name) != 0) + if (mlx5_vdpa_nl_roce_disable(pci_addr) != 0 && + mlx5_vdpa_sys_roce_disable(pci_addr) != 0) return -rte_errno; return 0; } From patchwork Thu May 27 14:02:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 93496 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 428C1A0548; Thu, 27 May 2021 16:03:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 400F241118; Thu, 27 May 2021 16:02:37 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2070.outbound.protection.outlook.com [40.107.237.70]) by mails.dpdk.org (Postfix) with ESMTP id A28234110A for ; Thu, 27 May 2021 16:02:34 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n8F7CHmy2Wa0Euj4fvXEp+TXvYksqsWxQZ4NS/no0e4hHp0MIF46VKAeAfzdeSUlvqdNwl7emmqGze9hJwi0I9sdwqH9nkRmw7sEs5HjcCCrsYEJPIzbiphU6J/ED4qnFf729ulxbrZQp2sPjEsBEeiPJ3cweVDbpuX/9HIMsocGKKOXhqka6U+6/K85sddWhYE2B2BJfhe+ZiTpZBumXowVD7XlKzRj7NHSP90n5RWrPHDD4mYS13gdKNym7F5OBOOgYD+tngc7ibUqNWvb0VNasuiZmAdXne5r1WA/ArIkUttPhlFA4lF86PMbFPfAXy4KKHxruR5VgRX8HIk7+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GYvdj8bOU7LnODTDYHYHaMjOSft4z4DU52JYvpTFeek=; b=jTURbFqZwoL1ZBQEDJoWsbfk6VAAC0ws5Knkpnry6/C1I2Gq9xp9zfJHCl1XGr8/yMKLwyG4EEJ4C0zCQUaDoBIZDAeAB3gb9QzTKoO1pIjnCkwSanh71EM0DAmkl1PU05XVQAAFFAjO+hp6ApTdemDn12XCbX/FLiSc1WsFF1iNQwDa2mqFJ68++0fnb/qz4ZsEGHAU2IEdEwcjRk9c/Kxtyd8PChNSz87tEavnRey2PxNp2+mLh0Ju39QNa7YO9AWIACiYAM1QocyXGfipWUbVbxfBPstacY/oc3EsbZGzAwZXYFwd2bKPKqtNRmuXbZX+RmrOgL+9zIpgKW6d6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GYvdj8bOU7LnODTDYHYHaMjOSft4z4DU52JYvpTFeek=; b=fBBqdGZb211cwV6KyefO9YgZ00MnLsG3EttmfjOT5wLoMjOUa52cCixlRKPia35B1GWTFbibgxgl8gJMHXbVt0ilJlkERyXkIWGPqK7oAuSmXfgwqDaYbN7bBVnmQvEhhiivYMdb2jtCql4uKuvqwsiThe/6XJVdutV64Dk1qA4lF9Xa/UCDkztnv+8vu4R/wEPesjv9QLX7laJmp7fJzuG5XqzbqJWlaqiJV5LDSkGNItaWZ2/Tti6wq3SgCNF7E1GCXKtSHZekNEzNc8L9d9i11GRIXJ69B4iA0YYoSVKCOVwvKgtIMNfiC5y/9mEE+17D4+gMGRVKJp0a7mT+kg== Received: from BN1PR10CA0005.namprd10.prod.outlook.com (2603:10b6:408:e0::10) by MN2PR12MB3599.namprd12.prod.outlook.com (2603:10b6:208:d2::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.21; Thu, 27 May 2021 14:02:32 +0000 Received: from BN8NAM11FT039.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e0:cafe::55) by BN1PR10CA0005.outlook.office365.com (2603:10b6:408:e0::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20 via Frontend Transport; Thu, 27 May 2021 14:02:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT039.mail.protection.outlook.com (10.13.177.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4129.25 via Frontend Transport; Thu, 27 May 2021 14:02:31 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 14:02:25 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Anatoly Burakov Date: Thu, 27 May 2021 17:02:02 +0300 Message-ID: <20210527140202.19377-5-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527140202.19377-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> <20210527140202.19377-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b8a4f93e-c44b-48ed-a723-08d921181246 X-MS-TrafficTypeDiagnostic: MN2PR12MB3599: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2331; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aMDJhX1pBsClsrcSIk46M7tsi8YlvTeDSFc1KADmmYEvSCQroDsTMW43xxosdWEMeSPvLfyNQnAOBOJHagMIFP519jJaXNwx6/wFsIC47htO9S8QKu8W4of6XYAmpbU9yEnKBeZCNE6OS0l2lmnCr2qncGuvbaStbre0iqIZu4yRduZnistyVutWzifrZPSFemDV4oIK1Hh8gzox1hjnshReCumxswtcJXSl15SEMLji1OmEVpEsinbhia9zh1NSsBje3edzOBbn13k/SHvb6URIxMI1eRG1gxc3BzbdYJdxFO6roBvwAorzTpfSpVV62M2UNfdKDWYyH0ow8oca6Gn9dMR7y4S3aHShCqXdweQ1BEUwqV9EGnexKTo/S7vZdbm8REvrX8GEKQQsEaFtMCOdhLNLAtdkXfybqUxyMW+x/YAx71/uSEKD3Li+wMoL9pio33AVOxMZywefRHMJ4GyGyvOpJbGzGfFazZdjfbxxGD13nMwGmRsP7BioCmHWsojwhoTA/pwNz6DLD5boSC3SxBbExkDLn5dYnnfrxqnO2tACKR3x3yhttjXcg4llbRy7YHdx8eKWX/CXwENozPvEFS1u6hU88nmsJFfJ7CxxuUNFiVSM7iT0t+G76YYgIpnYjTWJZhiUa9c4HtqDdNcYgJMsI/bwYw8Gsdyaaas= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(136003)(39860400002)(396003)(376002)(36840700001)(46966006)(47076005)(30864003)(26005)(16526019)(6286002)(186003)(7696005)(6862004)(36756003)(8676002)(6636002)(478600001)(5660300002)(86362001)(316002)(36906005)(426003)(82740400003)(70206006)(2906002)(55016002)(70586007)(82310400003)(1076003)(7636003)(8936002)(83380400001)(6666004)(36860700001)(19627235002)(336012)(54906003)(2616005)(4326008)(37006003)(356005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 14:02:31.7288 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b8a4f93e-c44b-48ed-a723-08d921181246 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT039.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3599 Subject: [dpdk-dev] [RFC 14/14] net/mlx5: support SubFunction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch introduces SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li --- doc/guides/nics/mlx5.rst | 339 +++++++++++++++++++++++- drivers/net/mlx5/linux/mlx5_ethdev_os.c | 12 +- drivers/net/mlx5/linux/mlx5_os.c | 142 +++++++--- drivers/net/mlx5/linux/mlx5_os.h | 2 + drivers/net/mlx5/mlx5.c | 10 +- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_rxmode.c | 8 +- drivers/net/mlx5/mlx5_trigger.c | 2 +- 8 files changed, 452 insertions(+), 64 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 83299646dd..3f5692038c 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -403,6 +403,300 @@ Limitations - Hairpin between two ports could only manual binding and explicit Tx flow mode. For single port hairpin, all the combinations of auto/manual binding and explicit/implicit Tx flow mode could be supported. - Hairpin in switchdev SR-IOV mode is not supported till now. +- Meter: + +Limitations +----------- + +- Windows support: + + On Windows, the features are limited: + + - Promiscuous mode is not supported + - The following rules are supported: + + - IPv4/UDP with CVLAN filtering + - Unicast MAC filtering + +- For secondary process: + + - Forked secondary process not supported. + - External memory unregistered in EAL memseg list cannot be used for DMA + unless such memory has been registered by ``mlx5_mr_update_ext_mp()`` in + primary process and remapped to the same virtual address in secondary + process. If the external memory is registered by primary process but has + different virtual address in secondary process, unexpected error may happen. + +- When using Verbs flow engine (``dv_flow_en`` = 0), flow pattern without any + specific VLAN will match for VLAN packets as well: + + When VLAN spec is not specified in the pattern, the matching rule will be created with VLAN as a wild card. + Meaning, the flow rule:: + + flow create 0 ingress pattern eth / vlan vid is 3 / ipv4 / end ... + + Will only match vlan packets with vid=3. and the flow rule:: + + flow create 0 ingress pattern eth / ipv4 / end ... + + Will match any ipv4 packet (VLAN included). + +- When using Verbs flow engine (``dv_flow_en`` = 0), multi-tagged(QinQ) match is not supported. + +- When using DV flow engine (``dv_flow_en`` = 1), flow pattern with any VLAN specification will match only single-tagged packets unless the ETH item ``type`` field is 0x88A8 or the VLAN item ``has_more_vlan`` field is 1. + The flow rule:: + + flow create 0 ingress pattern eth / ipv4 / end ... + + Will match any ipv4 packet. + The flow rules:: + + flow create 0 ingress pattern eth / vlan / end ... + flow create 0 ingress pattern eth has_vlan is 1 / end ... + flow create 0 ingress pattern eth type is 0x8100 / end ... + + Will match single-tagged packets only, with any VLAN ID value. + The flow rules:: + + flow create 0 ingress pattern eth type is 0x88A8 / end ... + flow create 0 ingress pattern eth / vlan has_more_vlan is 1 / end ... + + Will match multi-tagged packets only, with any VLAN ID value. + +- A flow pattern with 2 sequential VLAN items is not supported. + +- VLAN pop offload command: + + - Flow rules having a VLAN pop offload command as one of their actions and + are lacking a match on VLAN as one of their items are not supported. + - The command is not supported on egress traffic in NIC mode. + +- VLAN push offload is not supported on ingress traffic in NIC mode. + +- VLAN set PCP offload is not supported on existing headers. + +- A multi segment packet must have not more segments than reported by dev_infos_get() + in tx_desc_lim.nb_seg_max field. This value depends on maximal supported Tx descriptor + size and ``txq_inline_min`` settings and may be from 2 (worst case forced by maximal + inline settings) to 58. + +- Flows with a VXLAN Network Identifier equal (or ends to be equal) + to 0 are not supported. + +- L3 VXLAN and VXLAN-GPE tunnels cannot be supported together with MPLSoGRE and MPLSoUDP. + +- Match on Geneve header supports the following fields only: + + - VNI + - OAM + - protocol type + - options length + +- Match on Geneve TLV option is supported on the following fields: + + - Class + - Type + - Length + - Data + + Only one Class/Type/Length Geneve TLV option is supported per shared device. + Class/Type/Length fields must be specified as well as masks. + Class/Type/Length specified masks must be full. + Matching Geneve TLV option without specifying data is not supported. + Matching Geneve TLV option with ``data & mask == 0`` is not supported. + +- VF: flow rules created on VF devices can only match traffic targeted at the + configured MAC addresses (see ``rte_eth_dev_mac_addr_add()``). + +- Match on GTP tunnel header item supports the following fields only: + + - v_pt_rsv_flags: E flag, S flag, PN flag + - msg_type + - teid + +- Match on GTP extension header only for GTP PDU session container (next + extension header type = 0x85). +- Match on GTP extension header is not supported in group 0. + +- No Tx metadata go to the E-Switch steering domain for the Flow group 0. + The flows within group 0 and set metadata action are rejected by hardware. + +.. note:: + + MAC addresses not already present in the bridge table of the associated + kernel network device will be added and cleaned up by the PMD when closing + the device. In case of ungraceful program termination, some entries may + remain present and should be removed manually by other means. + +- Buffer split offload is supported with regular Rx burst routine only, + no MPRQ feature or vectorized code can be engaged. + +- When Multi-Packet Rx queue is configured (``mprq_en``), a Rx packet can be + externally attached to a user-provided mbuf with having EXT_ATTACHED_MBUF in + ol_flags. As the mempool for the external buffer is managed by PMD, all the + Rx mbufs must be freed before the device is closed. Otherwise, the mempool of + the external buffers will be freed by PMD and the application which still + holds the external buffers may be corrupted. + +- If Multi-Packet Rx queue is configured (``mprq_en``) and Rx CQE compression is + enabled (``rxq_cqe_comp_en``) at the same time, RSS hash result is not fully + supported. Some Rx packets may not have PKT_RX_RSS_HASH. + +- IPv6 Multicast messages are not supported on VM, while promiscuous mode + and allmulticast mode are both set to off. + To receive IPv6 Multicast messages on VM, explicitly set the relevant + MAC address using rte_eth_dev_mac_addr_add() API. + +- To support a mixed traffic pattern (some buffers from local host memory, some + buffers from other devices) with high bandwidth, a mbuf flag is used. + + An application hints the PMD whether or not it should try to inline the + given mbuf data buffer. PMD should do the best effort to act upon this request. + + The hint flag ``RTE_PMD_MLX5_FINE_GRANULARITY_INLINE`` is dynamic, + registered by application with rte_mbuf_dynflag_register(). This flag is + purely driver-specific and declared in PMD specific header ``rte_pmd_mlx5.h``, + which is intended to be used by the application. + + To query the supported specific flags in runtime, + the function ``rte_pmd_mlx5_get_dyn_flag_names`` returns the array of + currently (over present hardware and configuration) supported specific flags. + The "not inline hint" feature operating flow is the following one: + + - application starts + - probe the devices, ports are created + - query the port capabilities + - if port supporting the feature is found + - register dynamic flag ``RTE_PMD_MLX5_FINE_GRANULARITY_INLINE`` + - application starts the ports + - on ``dev_start()`` PMD checks whether the feature flag is registered and + enables the feature support in datapath + - application might set the registered flag bit in ``ol_flags`` field + of mbuf being sent and PMD will handle ones appropriately. + +- The amount of descriptors in Tx queue may be limited by data inline settings. + Inline data require the more descriptor building blocks and overall block + amount may exceed the hardware supported limits. The application should + reduce the requested Tx size or adjust data inline settings with + ``txq_inline_max`` and ``txq_inline_mpw`` devargs keys. + +- To provide the packet send scheduling on mbuf timestamps the ``tx_pp`` + parameter should be specified. + When PMD sees the RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME set on the packet + being sent it tries to synchronize the time of packet appearing on + the wire with the specified packet timestamp. It the specified one + is in the past it should be ignored, if one is in the distant future + it should be capped with some reasonable value (in range of seconds). + These specific cases ("too late" and "distant future") can be optionally + reported via device xstats to assist applications to detect the + time-related problems. + + The timestamp upper "too-distant-future" limit + at the moment of invoking the Tx burst routine + can be estimated as ``tx_pp`` option (in nanoseconds) multiplied by 2^23. + Please note, for the testpmd txonly mode, + the limit is deduced from the expression:: + + (n_tx_descriptors / burst_size + 1) * inter_burst_gap + + There is no any packet reordering according timestamps is supposed, + neither within packet burst, nor between packets, it is an entirely + application responsibility to generate packets and its timestamps + in desired order. The timestamps can be put only in the first packet + in the burst providing the entire burst scheduling. + +- E-Switch decapsulation Flow: + + - can be applied to PF port only. + - must specify VF port action (packet redirection from PF to VF). + - optionally may specify tunnel inner source and destination MAC addresses. + +- E-Switch encapsulation Flow: + + - can be applied to VF ports only. + - must specify PF port action (packet redirection from VF to PF). + +- Raw encapsulation: + + - The input buffer, used as outer header, is not validated. + +- Raw decapsulation: + + - The decapsulation is always done up to the outermost tunnel detected by the HW. + - The input buffer, providing the removal size, is not validated. + - The buffer size must match the length of the headers to be removed. + +- ICMP(code/type/identifier/sequence number) / ICMP6(code/type) matching, IP-in-IP and MPLS flow matching are all + mutually exclusive features which cannot be supported together + (see :ref:`mlx5_firmware_config`). + +- LRO: + + - Requires DevX and DV flow to be enabled. + - KEEP_CRC offload cannot be supported with LRO. + - The first mbuf length, without head-room, must be big enough to include the + TCP header (122B). + - Rx queue with LRO offload enabled, receiving a non-LRO packet, can forward + it with size limited to max LRO size, not to max RX packet length. + - LRO can be used with outer header of TCP packets of the standard format: + eth (with or without vlan) / ipv4 or ipv6 / tcp / payload + + Other TCP packets (e.g. with MPLS label) received on Rx queue with LRO enabled, will be received with bad checksum. + - LRO packet aggregation is performed by HW only for packet size larger than + ``lro_min_mss_size``. This value is reported on device start, when debug + mode is enabled. + +- CRC: + + - ``DEV_RX_OFFLOAD_KEEP_CRC`` cannot be supported with decapsulation + for some NICs (such as ConnectX-6 Dx, ConnectX-6 Lx, and BlueField-2). + The capability bit ``scatter_fcs_w_decap_disable`` shows NIC support. + +- TX mbuf fast free: + + - fast free offload assumes the all mbufs being sent are originated from the + same memory pool and there is no any extra references to the mbufs (the + reference counter for each mbuf is equal 1 on tx_burst call). The latter + means there should be no any externally attached buffers in mbufs. It is + an application responsibility to provide the correct mbufs if the fast + free offload is engaged. The mlx5 PMD implicitly produces the mbufs with + externally attached buffers if MPRQ option is enabled, hence, the fast + free offload is neither supported nor advertised if there is MPRQ enabled. + +- Sample flow: + + - Supports ``RTE_FLOW_ACTION_TYPE_SAMPLE`` action only within NIC Rx and + E-Switch steering domain. + - For E-Switch Sampling flow with sample ratio > 1, additional actions are not + supported in the sample actions list. + - For ConnectX-5, the ``RTE_FLOW_ACTION_TYPE_SAMPLE`` is typically used as + first action in the E-Switch egress flow if with header modify or + encapsulation actions. + - For NIC Rx flow, supports ``MARK``, ``COUNT``, ``QUEUE``, ``RSS`` in the + sample actions list. + - For E-Switch mirroring flow, supports ``RAW ENCAP``, ``Port ID``, + ``VXLAN ENCAP``, ``NVGRE ENCAP`` in the sample actions list. + +- Modify Field flow: + + - Supports the 'set' operation only for ``RTE_FLOW_ACTION_TYPE_MODIFY_FIELD`` action. + - Modification of an arbitrary place in a packet via the special ``RTE_FLOW_FIELD_START`` Field ID is not supported. + - Modification of the 802.1Q Tag, VXLAN Network or GENEVE Network ID's is not supported. + - Encapsulation levels are not supported, can modify outermost header fields only. + - Offsets must be 32-bits aligned, cannot skip past the boundary of a field. + +- IPv6 header item 'proto' field, indicating the next header protocol, should + not be set as extension header. + In case the next header is an extension header, it should not be specified in + IPv6 header item 'proto' field. + The last extension header item 'next header' field can specify the following + header protocol type. + +- Hairpin: + + - Hairpin between two ports could only manual binding and explicit Tx flow mode. For single port hairpin, all the combinations of auto/manual binding and explicit/implicit Tx flow mode could be supported. + - Hairpin in switchdev SR-IOV mode is not supported till now. + - Meter: - All the meter colors with drop action will be counted only by the global drop statistics. @@ -1438,13 +1732,17 @@ the DPDK application. echo switchdev > /sys/class/net//compat/devlink/mode -Sub-Function representor ------------------------- +SubFunction support +------------------- +SubFunction is a portion of the PCI device, a SF netdev has its own +dedicated queues(txq, rxq). A SF shares PCI level resources with other SFs +and/or with its parent PCI function. -Sub-Function is a portion of the PCI device, a SF netdev has its own -dedicated queues(txq, rxq). A SF netdev supports E-Switch representation -offload similar to existing PF and VF representors. A SF shares PCI -level resources with other SFs and/or with its parent PCI function. +0. Requirement:: + + kernel version >= 5.12 or OFED version >= 5.6 + + iproute2 >= 5.11 1. Configure SF feature:: @@ -1457,21 +1755,34 @@ level resources with other SFs and/or with its parent PCI function. 2: 32 SFs 3: 64 SFs -2. Reset the FW:: +2. Enable switchdev mode:: - mlxfwreset -d reset + devlink dev eswitch set pci/ mode switchdev -3. Enable switchdev mode:: +3. Add SF port:: - echo switchdev > /sys/class/net//compat/devlink/mode + devlink port add pci/ flavour pcisf pfnum 0 sfnum + + Get SFID from output: pci// + +4. Modify MAC address:: + + devlink port function set pci// hw_addr + +5. Activate SF port:: + + devlink port function set pci// state active -4. Create SF:: +6. Devargs to probe SF device:: - mlnx-sf -d -a create + auxiliary:mlx5_core.sf.9,dv_flow_en=1 -5. Probe SF representor:: +SubFunction representor support +------------------------------- +A SF netdev supports E-Switch representation offload similar to existing PF +and VF representors. Use to probe SF representor. - testpmd> port attach ,representor=sf0,dv_flow_en=1 + testpmd> port attach ,representor=sf,dv_flow_en=1 Performance tuning ------------------ diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c index 6fdb310129..8678502595 100644 --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c @@ -128,6 +128,17 @@ struct ethtool_link_settings { #define ETHTOOL_LINK_MODE_200000baseCR4_Full_BIT 2 /* 66 - 64 */ #endif +/* Get interface index from SubFunction device name. */ +int +mlx5_auxiliary_get_ifindex(const char *sf_name) +{ + char if_name[IF_NAMESIZE]; + + if (mlx5_auxiliary_get_child_name(sf_name, "/net", + if_name, sizeof(if_name)) != 0) + return -rte_errno; + return if_nametoindex(if_name); +} /** * Get interface name from private structure. @@ -1619,4 +1630,3 @@ mlx5_get_mac(struct rte_eth_dev *dev, uint8_t (*mac)[RTE_ETHER_ADDR_LEN]) memcpy(mac, request.ifr_hwaddr.sa_data, RTE_ETHER_ADDR_LEN); return 0; } - diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 4f16230fa5..d74273a7ca 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -1923,6 +1924,27 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, return pf; } +static void +mlx5_os_config_default(struct mlx5_dev_config *config) +{ + memset(config, 0, sizeof(*config)); + config->mps = MLX5_ARG_UNSET; + config->dbnc = MLX5_ARG_UNSET; + config->rx_vec_en = 1; + config->txq_inline_max = MLX5_ARG_UNSET; + config->txq_inline_min = MLX5_ARG_UNSET; + config->txq_inline_mpw = MLX5_ARG_UNSET; + config->txqs_inline = MLX5_ARG_UNSET; + config->vf_nl_en = 1; + config->mr_ext_memseg_en = 1; + config->mprq.max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN; + config->mprq.min_rxqs_num = MLX5_MPRQ_MIN_RXQS; + config->dv_esw_en = 1; + config->dv_flow_en = 1; + config->decap_en = 1; + config->log_hp_size = MLX5_ARG_UNSET; +} + /** * Register a PCI device within bonding. * @@ -2334,23 +2356,8 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, uint32_t restore; /* Default configuration. */ - memset(&dev_config, 0, sizeof(struct mlx5_dev_config)); + mlx5_os_config_default(&dev_config); dev_config.vf = dev_config_vf; - dev_config.mps = MLX5_ARG_UNSET; - dev_config.dbnc = MLX5_ARG_UNSET; - dev_config.rx_vec_en = 1; - dev_config.txq_inline_max = MLX5_ARG_UNSET; - dev_config.txq_inline_min = MLX5_ARG_UNSET; - dev_config.txq_inline_mpw = MLX5_ARG_UNSET; - dev_config.txqs_inline = MLX5_ARG_UNSET; - dev_config.vf_nl_en = 1; - dev_config.mr_ext_memseg_en = 1; - dev_config.mprq.max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN; - dev_config.mprq.min_rxqs_num = MLX5_MPRQ_MIN_RXQS; - dev_config.dv_esw_en = 1; - dev_config.dv_flow_en = 1; - dev_config.decap_en = 1; - dev_config.log_hp_size = MLX5_ARG_UNSET; list[i].eth_dev = mlx5_dev_spawn(&pci_dev->device, &list[i], &dev_config, @@ -2407,6 +2414,35 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, return ret; } +static int +mlx5_os_parse_eth_devargs(struct rte_device *dev, + struct rte_eth_devargs *eth_da) +{ + int ret = 0; + + if (dev->devargs == NULL) + return 0; + memset(eth_da, 0, sizeof(*eth_da)); + /* Parse representor information first from class argument. */ + if (dev->devargs->cls_str) + ret = rte_eth_devargs_parse(dev->devargs->cls_str, eth_da); + if (ret != 0) { + DRV_LOG(ERR, "failed to parse device arguments: %s", + dev->devargs->cls_str); + return -rte_errno; + } + if (eth_da->type == RTE_ETH_REPRESENTOR_NONE) { + /* Parse legacy device argument */ + ret = rte_eth_devargs_parse(dev->devargs->args, eth_da); + if (ret) { + DRV_LOG(ERR, "failed to parse device arguments: %s", + dev->devargs->args); + return -rte_errno; + } + } + return 0; +} + /** * Callback to register a PCI device. * @@ -2421,31 +2457,13 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, static int mlx5_os_pci_probe(struct rte_pci_device *pci_dev) { - struct rte_eth_devargs eth_da = { .type = RTE_ETH_REPRESENTOR_NONE }; + struct rte_eth_devargs eth_da = { .nb_ports = 0 }; int ret = 0; uint16_t p; - if (pci_dev->device.devargs) { - /* Parse representor information from device argument. */ - if (pci_dev->device.devargs->cls_str) - ret = rte_eth_devargs_parse - (pci_dev->device.devargs->cls_str, ð_da); - if (ret) { - DRV_LOG(ERR, "failed to parse device arguments: %s", - pci_dev->device.devargs->cls_str); - return -rte_errno; - } - if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) { - /* Support legacy device argument */ - ret = rte_eth_devargs_parse - (pci_dev->device.devargs->args, ð_da); - if (ret) { - DRV_LOG(ERR, "failed to parse device arguments: %s", - pci_dev->device.devargs->args); - return -rte_errno; - } - } - } + ret = mlx5_os_parse_eth_devargs(&pci_dev->device, ð_da); + if (ret != 0) + return ret; if (eth_da.nb_ports > 0) { /* Iterate all port if devargs pf is range: "pf[0-1]vf[...]". */ @@ -2458,10 +2476,53 @@ mlx5_os_pci_probe(struct rte_pci_device *pci_dev) return ret; } +/* Probe a single SF device on auxiliary bus, no representor support. */ +static int +mlx5_os_auxiliary_probe(struct rte_device *dev) +{ + struct rte_eth_devargs eth_da = { .nb_ports = 0 }; + struct mlx5_dev_config config; + struct mlx5_dev_spawn_data spawn = { .pf_bond = -1 }; + struct rte_auxiliary_device *adev = RTE_DEV_TO_AUXILIARY(dev); + struct rte_eth_dev *eth_dev; + int ret = 0; + + /* Parse ethdev devargs. */ + ret = mlx5_os_parse_eth_devargs(dev, ð_da); + if (ret != 0) + return ret; + /* Set default config data. */ + mlx5_os_config_default(&config); + config.sf = 1; + /* Init spawn data. */ + spawn.max_port = 1; + spawn.phys_port = 1; + spawn.phys_dev = mlx5_get_ibv_device(dev); + ret = mlx5_auxiliary_get_ifindex(dev->name); + if (ret < 0) { + DRV_LOG(ERR, "failed to get ethdev ifindex: %s", dev->name); + return ret; + } + spawn.ifindex = ret; + /* Spawn device. */ + eth_dev = mlx5_dev_spawn(dev, &spawn, &config, ð_da); + if (eth_dev == NULL) + return -rte_errno; + /* Post create. */ + eth_dev->intr_handle = &adev->intr_handle; + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC; + eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_RMV; + eth_dev->data->numa_node = dev->numa_node; + } + rte_eth_dev_probing_finish(eth_dev); + return 0; +} + /** * Common bus driver callback to probe a device. * - * This function probe PCI bus device(s). + * This function probe PCI bus device(s) or a single SF on auxiliary bus. * * @param[in] dev * Pointer to the generic device. @@ -2484,7 +2545,8 @@ mlx5_os_net_probe(struct rte_device *dev) } if (mlx5_dev_is_pci(dev)) return mlx5_os_pci_probe(RTE_DEV_TO_PCI(dev)); - return 0; + else + return mlx5_os_auxiliary_probe(dev); } static int diff --git a/drivers/net/mlx5/linux/mlx5_os.h b/drivers/net/mlx5/linux/mlx5_os.h index af7cbeb418..2991d37df2 100644 --- a/drivers/net/mlx5/linux/mlx5_os.h +++ b/drivers/net/mlx5/linux/mlx5_os.h @@ -19,4 +19,6 @@ enum { #define MLX5_NAMESIZE IF_NAMESIZE +int mlx5_auxiliary_get_ifindex(const char *sf_name); + #endif /* RTE_PMD_MLX5_OS_H_ */ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 3defdb2db3..69edd55b86 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -2319,10 +2319,12 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_eth_dev *odev) if (opriv->sh == priv->sh || odev->device == dev->device) break; - } else if (dev->device != NULL && dev->device->driver && - dev->device->driver->name && - !strcmp(dev->device->driver->name, - MLX5_PCI_DRIVER_NAME)) { + } else if (dev->device != NULL && dev->device->driver != NULL && + dev->device->driver->name != NULL && + (strcmp(dev->device->driver->name, + MLX5_PCI_DRIVER_NAME) == 0 || + strcmp(dev->device->driver->name, + MLX5_AUXILIARY_DRIVER_NAME) == 0)) { /* odev not specified, found all mlx5 devices. */ break; } diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 27bb34e827..b06f45fc54 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -220,6 +220,7 @@ struct mlx5_dev_config { unsigned int hw_fcs_strip:1; /* FCS stripping is supported. */ unsigned int hw_padding:1; /* End alignment padding is supported. */ unsigned int vf:1; /* This is a VF. */ + unsigned int sf:1; /* This is a SF. */ unsigned int tunnel_en:1; /* Whether tunnel stateless offloads are supported. */ unsigned int mpls_en:1; /* MPLS over GRE/UDP is enabled. */ diff --git a/drivers/net/mlx5/mlx5_rxmode.c b/drivers/net/mlx5/mlx5_rxmode.c index 25fb47c9ed..7f19b235c2 100644 --- a/drivers/net/mlx5/mlx5_rxmode.c +++ b/drivers/net/mlx5/mlx5_rxmode.c @@ -36,7 +36,7 @@ mlx5_promiscuous_enable(struct rte_eth_dev *dev) dev->data->port_id); return 0; } - if (priv->config.vf) { + if (priv->config.vf || priv->config.sf) { ret = mlx5_os_set_promisc(dev, 1); if (ret) return ret; @@ -69,7 +69,7 @@ mlx5_promiscuous_disable(struct rte_eth_dev *dev) int ret; dev->data->promiscuous = 0; - if (priv->config.vf) { + if (priv->config.vf || priv->config.sf) { ret = mlx5_os_set_promisc(dev, 0); if (ret) return ret; @@ -109,7 +109,7 @@ mlx5_allmulticast_enable(struct rte_eth_dev *dev) dev->data->port_id); return 0; } - if (priv->config.vf) { + if (priv->config.vf || priv->config.sf) { ret = mlx5_os_set_allmulti(dev, 1); if (ret) goto error; @@ -142,7 +142,7 @@ mlx5_allmulticast_disable(struct rte_eth_dev *dev) int ret; dev->data->all_multicast = 0; - if (priv->config.vf) { + if (priv->config.vf || priv->config.sf) { ret = mlx5_os_set_allmulti(dev, 0); if (ret) goto error; diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index 6c8a64ce03..e4e057a6f8 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -1259,7 +1259,7 @@ mlx5_traffic_enable(struct rte_eth_dev *dev) } mlx5_txq_release(dev, i); } - if (priv->config.dv_esw_en && !priv->config.vf) { + if (priv->config.dv_esw_en && !priv->config.vf && !priv->config.sf) { if (mlx5_flow_create_esw_table_zero_flow(dev)) priv->fdb_def_rule = 1; else