From patchwork Thu Jul 16 07:29:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 74173 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4D2C3A0546; Thu, 16 Jul 2020 09:31:04 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1D6631BF70; Thu, 16 Jul 2020 09:30:15 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60070.outbound.protection.outlook.com [40.107.6.70]) by dpdk.org (Postfix) with ESMTP id 24D721BF46 for ; Thu, 16 Jul 2020 09:30:13 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KS+0RGIu8+CseJ0mEwdbLjD8GmvPakVe+SalxLYeD64kuHK2S3fyarh9K2Zgxd6J4y2w0vng9sRmQXSU8oJqiWpAbf4OLPlWwTcaXLOxsjagvKu053Oactx34hSnv3ipg5EYhZ5IpfGziHy/e2jJu7zkY0rRxrHaXkTtResCRKDvV3KQhd81ToTsEoQNThAzQkVyzadmYtfU+BytbGmF3xLPL4F+3lzQv8gag87iKx0ACp6Mt7DoudzGIr5To/k0gbwgYu0K13j/UGz2T0Q+h+suSD8xD0Wwr7JOsc9HV1mVh7s82SLcbCYn3UocCNQD4duFBngE8eKsxIWdfv0jIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3yiQljzA4NxcJY/AjpQWRaqpDlUV1dDKlspCx0+4xR4=; b=N4RXt2kuHSG4tLUsukmhioHVE65eW/PgSW0VVmHR7SNETJQ9gbcr/+hsJV6b268j9MOLpaPAcOe6gWFdl9Oey9DJKzen8N1MVBnPUnqH/3RKOvKdb8V3h6a0wkRCre9C1wg6rh5WEVLnKtyVdSl7zx6zhAgSTPCrz2FRuHpJWdCpPw2+ekcay5HIZRnCxCwTL7zNAsSRn0IN4BiMnBU5sbTiRZz7GLByuFxW19e49ujxJfXSwzYRnbRGpcaFXx+iA9/UJnU1xJ0GY68UM6riIzE6uLhGT9KAh3jn//0h/GMWzQzh+TpF6wk3E43oDzLOKR1AmxHdnP++QEay3y+5zw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3yiQljzA4NxcJY/AjpQWRaqpDlUV1dDKlspCx0+4xR4=; b=EOvFey3N5I44Q4cTyIgOvywB/yl/KsgMWKH6v3dWBx63BGpFKHREBIc68fPPJzXR2goLvgxtS/dyjmesOMlztGI595fzvdCFpthY08LwYICRUqOj14cZQ03xmKLe0NiZYu8JtaEDZrL4WaVPJr5OpLx1TtrGyleghvvzqQe+rsY= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=mellanox.com; Received: from AM0PR05MB4866.eurprd05.prod.outlook.com (2603:10a6:208:c0::32) by AM0PR05MB5329.eurprd05.prod.outlook.com (2603:10a6:208:ed::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.23; Thu, 16 Jul 2020 07:30:11 +0000 Received: from AM0PR05MB4866.eurprd05.prod.outlook.com ([fe80::d44d:a804:c730:d2b7]) by AM0PR05MB4866.eurprd05.prod.outlook.com ([fe80::d44d:a804:c730:d2b7%2]) with mapi id 15.20.3174.026; Thu, 16 Jul 2020 07:30:11 +0000 From: Parav Pandit To: dev@dpdk.org, grive@u256.net, ferruh.yigit@intel.com, thomas@monjalon.net Cc: rasland@mellanox.com, orika@mellanox.com, matan@mellanox.com, joyce.kong@arm.com, Parav Pandit Date: Thu, 16 Jul 2020 10:29:35 +0300 Message-Id: <20200716072938.812733-7-parav@mellanox.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200716072938.812733-1-parav@mellanox.com> References: <20200610171728.89-2-parav@mellanox.com> <20200716072938.812733-1-parav@mellanox.com> X-ClientProxiedBy: SN6PR16CA0057.namprd16.prod.outlook.com (2603:10b6:805:ca::34) To AM0PR05MB4866.eurprd05.prod.outlook.com (2603:10a6:208:c0::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from sw-mtx-036.mtx.labs.mlnx (208.176.44.194) by SN6PR16CA0057.namprd16.prod.outlook.com (2603:10b6:805:ca::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.18 via Frontend Transport; Thu, 16 Jul 2020 07:30:09 +0000 X-Mailer: git-send-email 2.26.2 X-Originating-IP: [208.176.44.194] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 48a45cf8-a1b2-4eae-f263-08d8295a12bf X-MS-TrafficTypeDiagnostic: AM0PR05MB5329: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd,ExtAddr X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: G100yDJdVq1QSz63Q+48dFzieAboeKURGW12mmha0BbembPoZYjsPaYtGBhTbbbBJJtZGduWlUkS8imFXAkVEI1hcaIITn2cNtwOoigj2GIxA9cBCkq/hHutSkOpinsS7v+1YyePwWD7OPbk259BYTpSNaRUGBsfDMz6UNDjWuapRh62LCPlG45WpxvH654WnV/6FmwAb2IPiqnAKkXV+ZjW+ZZYiOsZOg+GhvbxjmDBjCqj0jXUa9XQiNeKgvrK/tyBtuLPJGIW+khnPn63KXepJ+vA097U0VbYLkXFeGk3NeiVWs7tfD7Kmjg0XP2cgWfo+IjWswiA0v9lkPANvA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR05MB4866.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(376002)(366004)(346002)(136003)(396003)(39860400002)(6506007)(6666004)(16526019)(186003)(66946007)(107886003)(66556008)(26005)(8936002)(86362001)(6512007)(6486002)(52116002)(83380400001)(66476007)(5660300002)(2616005)(956004)(36756003)(8676002)(478600001)(316002)(1076003)(2906002)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: ZVTa8iH5rZzoS8KIyRrtv9xoKIO3dtq4/f7BSgyWT5jnFPu6gngJXaRIatv2rS+dE+QOm0Ap52b664G6uLaA0fc8MkX4GviHwyUxF4KMdrlwP6ciMk0jC71cSKlxSHm2a1+1pIWYbYZbDgYTfd7ltQn4neF2rbdHvVJ6A6Oc/DEDpYHmmJf1xOiv7VjWC6lQ3efYtq7pFfkmfV15XKLS/66ZhoZ6CLTBDrcheovjvm0LV15CCkaZ4s0GlF4cg3FW0W/tq0+7NRN7YIo48BZzQD0u7JEk2TDY9us1cxLf51kPA0p8LYw4kNwnZvETl2NZimE8JeL0Uk4Z4vpR7Y7qO/PVwcf7t7EZq4uJ5pkkgcWmIzCVhN4iMWmUH4jJz8PwIcKF0Omj8hVM4juTtjJTYfpxZsyhbcAg20uLc7XUOLtDgs9VCF5u+RfaZLSkaqpN3gaKvjaxVe9LlLrd0y8/TYjWEzijT/J+fTPYzXMtonqNOqA88gE9GNGvZvuEwE8h X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 48a45cf8-a1b2-4eae-f263-08d8295a12bf X-MS-Exchange-CrossTenant-AuthSource: AM0PR05MB4866.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jul 2020 07:30:11.5617 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pFkWEqpJ92Y6eoejvQKdA/q79JjHdDvQq4cvM58nMVz5uc+tm/80FrUkWqlRu4b0dBZgk+Q66BxI6xDLKtJe0Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB5329 Subject: [dpdk-dev] [PATCH v6 6/9] bus/mlx5_pci: add mlx5 PCI bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add mlx5 PCI bus which enables multiple mlx5 drivers to bind to single pci device. Signed-off-by: Parav Pandit Acked-by: Matan Azrad --- Changelog: v5->v6: - Updated Makefile for parallel shared build support v4->v5: - Merged maintainers update patch with this patch v2->v3: - Addressed comments from Thomas and Asaf - Moved pci_driver structure instance as first in driver - Removed white spaces at the end of line in diagram - Address comments from Matan - Removed CONFIG_RTE_LIBRTE_MLX5_PCI_BUS from config files - Changed alignedment to mlx5 specific aligment instead of standard DPDK - Using uint32_t instead of mlx5_class enum v1->v2: - Address comments from Thomas and Gaetan - Inheriting ret_pci_driver instead of rte_driver - Added design and description of the mlx5_pci bus --- MAINTAINERS | 5 ++ drivers/bus/Makefile | 4 + drivers/bus/meson.build | 2 +- drivers/bus/mlx5_pci/Makefile | 39 +++++++++ drivers/bus/mlx5_pci/meson.build | 19 +++++ drivers/bus/mlx5_pci/mlx5_pci_bus.c | 14 ++++ drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h | 84 +++++++++++++++++++ .../bus/mlx5_pci/rte_bus_mlx5_pci_version.map | 5 ++ 8 files changed, 171 insertions(+), 1 deletion(-) create mode 100644 drivers/bus/mlx5_pci/Makefile create mode 100644 drivers/bus/mlx5_pci/meson.build create mode 100644 drivers/bus/mlx5_pci/mlx5_pci_bus.c create mode 100644 drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h create mode 100644 drivers/bus/mlx5_pci/rte_bus_mlx5_pci_version.map diff --git a/MAINTAINERS b/MAINTAINERS index 3cd402b34..9e3bd1672 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -525,6 +525,11 @@ Intel FPGA bus M: Rosen Xu F: drivers/bus/ifpga/ +Melllanox mlx5 PCI bus driver +M: Parav Pandit +M: Matan Azrad +F: drivers/bus/mlx5_pci + NXP buses M: Hemant Agrawal M: Sachin Saxena diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile index cea3b55e6..a70f213c1 100644 --- a/drivers/bus/Makefile +++ b/drivers/bus/Makefile @@ -9,6 +9,10 @@ DIRS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += fslmc endif DIRS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) += ifpga DIRS-$(CONFIG_RTE_LIBRTE_PCI_BUS) += pci +ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) +DEPDIRS-mlx5_pci := pci +DIRS-y += mlx5_pci +endif DIRS-$(CONFIG_RTE_LIBRTE_VDEV_BUS) += vdev DIRS-$(CONFIG_RTE_LIBRTE_VMBUS) += vmbus diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build index 80de2d91d..b1381838d 100644 --- a/drivers/bus/meson.build +++ b/drivers/bus/meson.build @@ -1,7 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -drivers = ['dpaa', 'fslmc', 'ifpga', 'pci', 'vdev', 'vmbus'] +drivers = ['dpaa', 'fslmc', 'ifpga', 'pci', 'mlx5_pci', 'vdev', 'vmbus'] std_deps = ['eal'] config_flag_fmt = 'RTE_LIBRTE_@0@_BUS' driver_name_fmt = 'rte_bus_@0@' diff --git a/drivers/bus/mlx5_pci/Makefile b/drivers/bus/mlx5_pci/Makefile new file mode 100644 index 000000000..dd24811a3 --- /dev/null +++ b/drivers/bus/mlx5_pci/Makefile @@ -0,0 +1,39 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_bus_mlx5_pci.a + +CFLAGS += -O3 -Wall -Wextra +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -Wno-strict-prototypes +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 +CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux +CFLAGS += -I$(BUILDDIR)/drivers/bus/pci +CFLAGS += -I$(BUILDDIR)/drivers/bus +LDLIBS += -lrte_eal +LDLIBS += -lrte_common_mlx5 +LDLIBS += -lrte_pci -lrte_bus_pci + +# versioning export map +EXPORT_MAP := rte_bus_mlx5_pci_version.map + +SRCS-y += mlx5_pci_bus.c + +# DEBUG which is usually provided on the command-line may enable +# CONFIG_RTE_LIBRTE_MLX5_DEBUG. +ifeq ($(DEBUG),1) +CONFIG_RTE_LIBRTE_MLX5_DEBUG := y +endif + +# +# Export include files +# +SYMLINK-y-include += rte_bus_mlx5_pci.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/bus/mlx5_pci/meson.build b/drivers/bus/mlx5_pci/meson.build new file mode 100644 index 000000000..64a17cbad --- /dev/null +++ b/drivers/bus/mlx5_pci/meson.build @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Mellanox Technologies Ltd + +deps += ['pci', 'bus_pci', 'common_mlx5'] +install_headers('rte_bus_mlx5_pci.h') +sources = files('mlx5_pci_bus.c') + +cflags_options = [ + '-std=c11', + '-Wno-strict-prototypes', + '-D_BSD_SOURCE', + '-D_DEFAULT_SOURCE', + '-D_XOPEN_SOURCE=600' +] +foreach option:cflags_options + if cc.has_argument(option) + cflags += option + endif +endforeach diff --git a/drivers/bus/mlx5_pci/mlx5_pci_bus.c b/drivers/bus/mlx5_pci/mlx5_pci_bus.c new file mode 100644 index 000000000..66db3c7b0 --- /dev/null +++ b/drivers/bus/mlx5_pci/mlx5_pci_bus.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#include "rte_bus_mlx5_pci.h" + +static TAILQ_HEAD(mlx5_pci_bus_drv_head, rte_mlx5_pci_driver) drv_list = + TAILQ_HEAD_INITIALIZER(drv_list); + +void +rte_mlx5_pci_driver_register(struct rte_mlx5_pci_driver *driver) +{ + TAILQ_INSERT_TAIL(&drv_list, driver, next); +} diff --git a/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h new file mode 100644 index 000000000..9f8d22e2b --- /dev/null +++ b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h @@ -0,0 +1,84 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Mellanox Technologies, Ltd + */ + +#ifndef _RTE_BUS_MLX5_PCI_H_ +#define _RTE_BUS_MLX5_PCI_H_ + +/** + * @file + * + * RTE Mellanox PCI Bus Interface + * Mellanox ConnectX PCI device supports multiple class (net/vdpa/regex) + * devices. This bus enables creating such multiple class of devices on a + * single PCI device by allowing to bind multiple class specific device + * driver to attach to mlx5_pci bus driver. + * + * ----------- ------------ ----------------- + * | mlx5 | | mlx5 | | mlx5 | + * | net pmd | | vdpa pmd | | new class pmd | + * ----------- ------------ ----------------- + * \ | / + * \ | / + * \ ------------- / + * \______| mlx5 |_____ / + * | pci bus | + * ------------- + * | + * ----------- + * | mlx5 | + * | pci dev | + * ----------- + * + * - mlx5 pci bus driver binds to mlx5 PCI devices defined by PCI + * ID table of all related mlx5 PCI devices. + * - mlx5 class driver such as net, vdpa, regex PMD defines its + * specific PCI ID table and mlx5 bus driver probes matching + * class drivers. + * - mlx5 pci bus driver is cental place that validates supported + * class combinations. + */ + +#ifdef __cplusplus +extern "C" { +#endif /* __cplusplus */ + +#include +#include + +#include + +/** + * A structure describing a mlx5 pci driver. + */ +struct rte_mlx5_pci_driver { + struct rte_pci_driver pci_driver; /**< Inherit core pci driver. */ + uint32_t dev_class; /**< Class of this driver, enum mlx5_class */ + TAILQ_ENTRY(rte_mlx5_pci_driver) next; +}; + +/** + * Register a mlx5_pci device driver. + * + * @param driver + * A pointer to a rte_mlx5_pci_driver structure describing the driver + * to be registered. + */ +__rte_internal +void +rte_mlx5_pci_driver_register(struct rte_mlx5_pci_driver *driver); + +#define RTE_PMD_REGISTER_MLX5_PCI(nm, drv) \ + static const char *mlx5_pci_drvinit_fn_ ## nm; \ + RTE_INIT(mlx5_pci_drvinit_fn_ ##drv) \ + { \ + (drv).driver.name = RTE_STR(nm); \ + rte_mlx5_pci_driver_register(&drv); \ + } \ + RTE_PMD_EXPORT_NAME(nm, __COUNTER__) + +#ifdef __cplusplus +} +#endif /* __cplusplus */ + +#endif /* _RTE_BUS_MLX5_PCI_H_ */ diff --git a/drivers/bus/mlx5_pci/rte_bus_mlx5_pci_version.map b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci_version.map new file mode 100644 index 000000000..4cfd3db10 --- /dev/null +++ b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci_version.map @@ -0,0 +1,5 @@ +INTERNAL { + global: + + rte_mlx5_pci_driver_register; +};