From patchwork Thu Oct 26 06:43:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 133359 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6382F43204; Thu, 26 Oct 2023 08:47:11 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2C2EF42E35; Thu, 26 Oct 2023 08:44:31 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2093.outbound.protection.outlook.com [40.107.93.93]) by mails.dpdk.org (Postfix) with ESMTP id 5846342E05 for ; Thu, 26 Oct 2023 08:44:22 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TlYFXgMS+hYAFqTQxbu+DHQGZhx1CL6GsrEYfIj0+WHu26icMBETivtAMtphvqnVO5TjdNnEiTqbl/b6TomAbQxArvwNM0Mpim2e8BFz74tJeY9WVTstXiRwYv756QQtQvcbb/TNLj/BN0fD69IxZf/shUcqxlVes0+Kp2gLZYAr588DtG1sTknQkbgvQnt2Cb8rcUside7MjgQE4yodW2g9PjI6M7dYtQKLKqlLpV+1sb1D3ZMGiWailjN+wzZ8DP4eXK2gG7+da/9G9DfsEtsqgMbjaV2VlG9JxbrIcp7VULw+MIypV48J4EJCtuwPFEBqIjOU7grE1t3PhVtC+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yOGvlioRLuuyiF3PvoqM9dx/VgwtqOmWT2SFrI/Pe0E=; b=CPpfx50lBltFz9LHLA9ADcDQVufOVkraGMGZz2VURlyncWyJ9LmX+kBDqt0mVzt6qe8xp5Sc/MZ+B/tbIOPGpIlYzPJ3jA1yERpJgtGEiFs949XAwifZj4rO7+g0ZgE6EWW80SgLrlweRwTRxLyulYhtUakd4x9vPB6A7j/SeFdrk1Iv7j2VHLTLYjbppOUstOS+bCa6gbOi/d/l9x3ZozkqktX7HQ1/bWGu4hkj1Cp6qvyxxwDojpjMgiUI4EjlFehgj8QLPesQwFe1eTXmMFWrezxvTP2pGx8gHAmPaDS5c4jE3E9KoQAl8Y5I/Nr4wP3CedFjJEgasZevb3XY3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yOGvlioRLuuyiF3PvoqM9dx/VgwtqOmWT2SFrI/Pe0E=; b=GEJZVVnZQVXKiLXrvFBqIt62L2bGYBWwILyvYR1Wq9E3xsFu8oU1xv7y81sXI9kZ+xPNfE2CTAU2+YknkKZx8kP4WyKo29l2R7Zi5yezK5d5KDaKFv+BQpfMYefdG+dEXAk+cnWYENPw0M5bLyuwfDpbgj19y4hJlBNy8Fi05bg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by DM4PR13MB5908.namprd13.prod.outlook.com (2603:10b6:8:50::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.19; Thu, 26 Oct 2023 06:44:20 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b%6]) with mapi id 15.20.6933.022; Thu, 26 Oct 2023 06:44:20 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Chaoyong He , Shujing Dong , Long Wu , Peng Zhang Subject: [PATCH v3 19/25] drivers: add the basic framework of vDPA PMD Date: Thu, 26 Oct 2023 14:43:18 +0800 Message-Id: <20231026064324.177531-20-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231026064324.177531-1-chaoyong.he@corigine.com> References: <20231024022849.3092506-1-chaoyong.he@corigine.com> <20231026064324.177531-1-chaoyong.he@corigine.com> X-ClientProxiedBy: BYAPR07CA0034.namprd07.prod.outlook.com (2603:10b6:a02:bc::47) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|DM4PR13MB5908:EE_ X-MS-Office365-Filtering-Correlation-Id: 0a31b275-79dc-4ca4-b311-08dbd5eefbbe X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QPqZclHh5rJmFqUK1/CNwL/caPBSJ2s62mHUP7rPPQ1vaCcBxQxKzAhgQeLWfoPPTmPySeGuw5biWTZbhggZQd8q/I2z2hJjHHn7qjMXWFZCqS2l2CvikzSwbkNJZN36vl+NvZhfc68fO9El1tTYhIrD/KwoOlgZwHXix7sTq+dRSO1RjsMjmKNEfrWbQOLF/NjnjJ78eEartEtQMQhzT6IziEkMZqhTHnEUj4xwpaYBi2hfcd5ZbTKid9k3LewrUtg166OecwtiAeBj7xvYNRyEyFWSaw2B/Z47JEKNCDUhM3E2D3atzv22+eBPlLNUGwaYULt1tQTKUtStRH8bHpRF+qcwleLSGbMsKKrfraakZ8g+O2zVpC2ASh1ke/KMUuh7jfR6ltfHh3t3OJBVNUbDujLNAdBJ2OSvhFqSdcRZbrAgOF0V7dT9m4woO1lLgYINujLQSH18qP4ChJ2vdH+UGi1H55KUdZ0SR8QKrjwuePcuc1QQYALT1GGdWjiwtfO9Oj1oA4T8Wvz/cPTRFr4aDzzhXl5KSvBp7GFqE5RsarucJstC2/GuqhX9n654wmS622cXDyDOi1QwFWCdpiRhPeh+hz4GwTLjTedwNq+JwL4rTp66qiEQQnzilNg5Nc+AeFrZtL2DBc1FnVEO+phghM9EL0fyJqIdrXTC9LU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(376002)(366004)(396003)(346002)(39840400004)(230922051799003)(186009)(1800799009)(451199024)(64100799003)(26005)(478600001)(1076003)(316002)(6512007)(86362001)(6506007)(6916009)(66946007)(66556008)(38100700002)(66476007)(54906003)(36756003)(38350700005)(2616005)(6486002)(107886003)(44832011)(2906002)(83380400001)(4326008)(8676002)(8936002)(5660300002)(41300700001)(52116002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7JTnNTeFuDLC4LDjEW2LJN1cylE68TWDNR4ki4notrAEP6XYgfPb67vFZhuJMOH16LzR99CbXdU39Hovi1krRaF2fZUKmU2f6BfyN8jEGfiVbnX1GpIH9WDXJZBoLlsri9yhD5ZPzDTCFaTH7VSn5rbInfwwrDkPBnwLmLJxp31N1VdsCN/CLGXb5s00NU20qPB7Amg8LGPCUFtQoXcnEaJpTySdgVOWG5+SqXC+JHudRR3LBnY+Heuy4KRkbTMk7OB2J1B7+Po8UaTy8++TQ/IjuKnFLcsxlpCy04T5cfT+YA1qG5GsxCS0ooMSTpjPXq33d+moMVYfJiwWUR8O0qQe6lx5JXZZRxBj1dkwcIWSJyyoVPfl6IR0tCtAW9XruKw58r85ej0wWe9d9lfbTw4+55uOL5AECI18bnlI7mQjlRIQEFpWL1adeG6TbfFHyk3C0EFdRYEAUcon8ZXNhDsKwvn9UgxdrzFsG8BfkGQoSaUQIFnc/vurBlV9Jn6qZHatuSve9JMRCxBEPPiYVpQYPgxYSl6dAZFfwVBNW0qgYMgmJDvVZUIQ/W9Eg1/CF6PiyGEvqi40nE3J1qPxXjp1zqFR6RMHS4x4XEaTSA+BMFoOJR5oxL6i2YbVUuPT1qdXt7aLl1Dv0ZXgwlBiYNuoViFlD8N+xzgRpBMevSyjddnpwuvj3IbwYg7lSSgna/crmM4xmHncVCqA1S/yP6aB5QRxsOshjEEflGqtjMLKvMuhtCQzWjbJx5uFzPfh0npifsOAUy8Yth2KKX3N/JpmYHf1evSzt7WJ+NH59anhQsbtBrxHFgn7iZU2eelUenR+XBYHxc5eksgWN4f457AjH18gdJGPguRLjKd4tUqC1i/51pLMsPygUOHvbqbyvN4kI0BjDHM+D7wHvIh9ryUdQFYUkXb1iRMOVtQV8qqvaz6uR9VzZ9z03mv8um/qd71f1Qp7zcwAyt/Gq5oDhzdO+Qn7DlXf2f7Snji9UaIYgy3809ZuB7D4m07v3vMYMQPeXg/8RitZLXSzRZjAq5SPvALUTpzZtPaOgTJiwhWOU3qH1i7pBj8MtMPkDiGWhhV1IYl2XdW6EG86mGMEP040qrXaZx3iPRDpYscOpfk2cfMlx1MyHHU8AwaEwL+Ax++47YUQvgZCX7bRLqrVWph+l0X9OY5RYLLdgaiCdcR8HhTAIghPhhKTWpT9IRUnSHIGdcSQnUGXnBUk90nFEhaPO1cLgkj35R+lGcoicOw/4IZwwT7A/75dMI4NU4Hr/hOUYwaEpc3+lsG/YXcXEg0N7JCXMwM7Eu+CisXCXLsMzAvTbVPC/qrVUaypq04ph2VfAnrqTdvNJZuEfO4RuqbXT7TijdQnc7bUNIaHf5xGAE/yTYoDeHnPBKXGtDRqWhP2dnfo8HRCOBzveKCu+S0Wnu0YTIhZ6auj7NJZEc9WhjgulN5Za8e0V5FZq3uLUnqLF2dCRRqChNTMdPCnEADx8woBvgo5A/enZphW67Gvx0SiCjZs1oqoec8vFBl4hcRJswFp4Bev3ADJr2ahe4Tv2exXPIVIGl1MNmmUcGURUK05CWkjB/UNtiXnK9zaYCTAJ4/2kXDDwIl8pkuQPQ== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0a31b275-79dc-4ca4-b311-08dbd5eefbbe X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Oct 2023 06:44:20.8305 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bqOoxzVqAtk4cqX/PaQLQOZTXQE2KcjXDcSCbw5XVkktiTXM1fU1KN8sCirrs7twX33KspWos3bHc6fJ/c/EO+rpyglDqcputLKfaLKrnYY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR13MB5908 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add the basic framework of vDPA PMD. Signed-off-by: Chaoyong He Signed-off-by: Shujing Dong Reviewed-by: Long Wu Reviewed-by: Peng Zhang --- drivers/common/nfp/nfp_common_pci.c | 1 + drivers/common/nfp/nfp_common_pci.h | 1 + drivers/vdpa/nfp/meson.build | 3 +- drivers/vdpa/nfp/nfp_vdpa.c | 153 ++++++++++++++++++++++++++++ 4 files changed, 157 insertions(+), 1 deletion(-) create mode 100644 drivers/vdpa/nfp/nfp_vdpa.c diff --git a/drivers/common/nfp/nfp_common_pci.c b/drivers/common/nfp/nfp_common_pci.c index 65f189b05e..723035d0f7 100644 --- a/drivers/common/nfp/nfp_common_pci.c +++ b/drivers/common/nfp/nfp_common_pci.c @@ -32,6 +32,7 @@ static const struct { enum nfp_class drv_class; } nfp_classes[] = { { .name = "eth", .drv_class = NFP_CLASS_ETH }, + { .name = "vdpa", .drv_class = NFP_CLASS_VDPA }, }; static enum nfp_class diff --git a/drivers/common/nfp/nfp_common_pci.h b/drivers/common/nfp/nfp_common_pci.h index 21465fca68..80c027c3a5 100644 --- a/drivers/common/nfp/nfp_common_pci.h +++ b/drivers/common/nfp/nfp_common_pci.h @@ -16,6 +16,7 @@ typedef int (nfp_class_driver_remove_t)(struct rte_pci_device *dev); enum nfp_class { NFP_CLASS_ETH, + NFP_CLASS_VDPA, NFP_CLASS_INVALID, }; diff --git a/drivers/vdpa/nfp/meson.build b/drivers/vdpa/nfp/meson.build index 5eff240055..79a0364917 100644 --- a/drivers/vdpa/nfp/meson.build +++ b/drivers/vdpa/nfp/meson.build @@ -7,7 +7,8 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') endif sources = files( + 'nfp_vdpa.c', 'nfp_vdpa_log.c', ) -deps += ['vhost'] +deps += ['vhost', 'common_nfp'] diff --git a/drivers/vdpa/nfp/nfp_vdpa.c b/drivers/vdpa/nfp/nfp_vdpa.c new file mode 100644 index 0000000000..ef4ae4946a --- /dev/null +++ b/drivers/vdpa/nfp/nfp_vdpa.c @@ -0,0 +1,153 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#include + +#include +#include +#include + +#include "nfp_vdpa_log.h" + +#define NFP_VDPA_DRIVER_NAME nfp_vdpa + +struct nfp_vdpa_dev { + struct rte_pci_device *pci_dev; + struct rte_vdpa_device *vdev; +}; + +struct nfp_vdpa_dev_node { + TAILQ_ENTRY(nfp_vdpa_dev_node) next; + struct nfp_vdpa_dev *device; +}; + +TAILQ_HEAD(vdpa_dev_list_head, nfp_vdpa_dev_node); + +static struct vdpa_dev_list_head vdpa_dev_list = + TAILQ_HEAD_INITIALIZER(vdpa_dev_list); + +static pthread_mutex_t vdpa_list_lock = PTHREAD_MUTEX_INITIALIZER; + +static struct nfp_vdpa_dev_node * +nfp_vdpa_find_node_by_pdev(struct rte_pci_device *pdev) +{ + bool found = false; + struct nfp_vdpa_dev_node *node; + + pthread_mutex_lock(&vdpa_list_lock); + + TAILQ_FOREACH(node, &vdpa_dev_list, next) { + if (pdev == node->device->pci_dev) { + found = true; + break; + } + } + + pthread_mutex_unlock(&vdpa_list_lock); + + if (found) + return node; + + return NULL; +} + +struct rte_vdpa_dev_ops nfp_vdpa_ops = { +}; + +static int +nfp_vdpa_pci_probe(struct rte_pci_device *pci_dev) +{ + struct nfp_vdpa_dev *device; + struct nfp_vdpa_dev_node *node; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + node = calloc(1, sizeof(*node)); + if (node == NULL) + return -ENOMEM; + + device = calloc(1, sizeof(*device)); + if (device == NULL) + goto free_node; + + device->pci_dev = pci_dev; + + device->vdev = rte_vdpa_register_device(&pci_dev->device, &nfp_vdpa_ops); + if (device->vdev == NULL) { + DRV_VDPA_LOG(ERR, "Failed to register device %s", pci_dev->name); + goto free_device; + } + + node->device = device; + pthread_mutex_lock(&vdpa_list_lock); + TAILQ_INSERT_TAIL(&vdpa_dev_list, node, next); + pthread_mutex_unlock(&vdpa_list_lock); + + return 0; + +free_device: + free(device); +free_node: + free(node); + + return -1; +} + +static int +nfp_vdpa_pci_remove(struct rte_pci_device *pci_dev) +{ + struct nfp_vdpa_dev *device; + struct nfp_vdpa_dev_node *node; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + node = nfp_vdpa_find_node_by_pdev(pci_dev); + if (node == NULL) { + DRV_VDPA_LOG(ERR, "Invalid device: %s", pci_dev->name); + return -ENODEV; + } + + device = node->device; + + pthread_mutex_lock(&vdpa_list_lock); + TAILQ_REMOVE(&vdpa_dev_list, node, next); + pthread_mutex_unlock(&vdpa_list_lock); + + rte_vdpa_unregister_device(device->vdev); + + free(device); + free(node); + + return 0; +} + +static const struct rte_pci_id pci_id_nfp_vdpa_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_NETRONOME, + PCI_DEVICE_ID_NFP6000_VF_NIC) + }, + { + .vendor_id = 0, + }, +}; + +static struct nfp_class_driver nfp_vdpa = { + .drv_class = NFP_CLASS_VDPA, + .name = RTE_STR(NFP_VDPA_DRIVER_NAME), + .id_table = pci_id_nfp_vdpa_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, + .probe = nfp_vdpa_pci_probe, + .remove = nfp_vdpa_pci_remove, +}; + +RTE_INIT(nfp_vdpa_init) +{ + nfp_class_driver_register(&nfp_vdpa); +} + +RTE_PMD_REGISTER_PCI_TABLE(NFP_VDPA_DRIVER_NAME, pci_id_nfp_vdpa_map); +RTE_PMD_REGISTER_KMOD_DEP(NFP_VDPA_DRIVER_NAME, "* vfio-pci");