From patchwork Fri Oct 27 02:59:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 133464 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 82A074320F; Fri, 27 Oct 2023 05:04:10 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0720342EA4; Fri, 27 Oct 2023 05:01:22 +0200 (CEST) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2116.outbound.protection.outlook.com [40.107.102.116]) by mails.dpdk.org (Postfix) with ESMTP id 5814442E25 for ; Fri, 27 Oct 2023 05:01:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TotJ5jqu00NJUQWSvDhJcCrYyV4eFH4/DJAbds66T0n7TSDC3vz7MNfd7dU1KAYRkS1++DpE8pYK/pnQ+5sfXc+NgS4KnHReK7lU7wvGU1KyXk44l0NyfXJmHI8JnNScSe/Fo3R5asC6OtBdPya8SyhCVKCuTtl2+4kzKVgmYlHEKyoPIDA+KIwDjCizAj9g7LtjL7JDte7m04PzFDolorwrtBpF+nHqs4+xcnrBZE3vdQNwTW++GNn+PjBP41u2rPGvH3iDGNe2aTEHHKMhEUkTesW8t1UCBxeHCvGSZa+25KEukyG5aICktyZiL5+1DkpLLC6zm7Bars93hHMbjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=//vcj9C4dCYeyhEFcez77fR7dZTUeDRee4UCLmkkU5I=; b=WG6P/XvbqUPZVxvMlQShq2qo14pd7B3FLqavbz3xM5c1WKYuZXmpleD4A4zcziD2knP8f2+yYzr/Yi1tqmNGdLo8tPFgOUkSrJU+cXz8nKoI++RCueNuth1Tpkf5Hr5q90Cnv8WkP4kzJxXk1vobGNaFzIEXZpgcPKFK1gtXuTb3rXN8Xim6m8oQvSxkGqrMRb44Fvk90wS+TwbAj2DM6vLFQMgso005MOiwMYw4GEWgm+pVEKaf/4QkC2FMJwOuUp5dI7Ijn0Ws1bidnS6K1wwR9452VtKsSZG+VcgD0Sc8qfdiCA220p8GL6crVWLeCk03LFGIQpkmbum0UvkmeA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=//vcj9C4dCYeyhEFcez77fR7dZTUeDRee4UCLmkkU5I=; b=J/cNgc7KsOE/l0t7MFrlrxWkLbBzRgTmQewPkepadlqeVniv6bGk7CvPat7aGr2I4XPJBwSRabe/LkSl6HI68yQO9GhxxU4ZhqnYH+hN5z0hetn0PamG09KjyKw2L99fXDc+MJsnvsOXhW2rD8mYPXIPu0U0/PrI/W1m2zrpAs8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA0PR13MB4048.namprd13.prod.outlook.com (2603:10b6:806:97::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.24; Fri, 27 Oct 2023 03:01:05 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b%6]) with mapi id 15.20.6933.022; Fri, 27 Oct 2023 03:01:05 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Chaoyong He , Shujing Dong , Long Wu , Peng Zhang Subject: [PATCH v4 21/24] vdpa/nfp: add hardware init Date: Fri, 27 Oct 2023 10:59:58 +0800 Message-Id: <20231027030001.602639-22-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231027030001.602639-1-chaoyong.he@corigine.com> References: <20231026064324.177531-1-chaoyong.he@corigine.com> <20231027030001.602639-1-chaoyong.he@corigine.com> X-ClientProxiedBy: BYAPR08CA0065.namprd08.prod.outlook.com (2603:10b6:a03:117::42) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA0PR13MB4048:EE_ X-MS-Office365-Filtering-Correlation-Id: 8d5dbfdf-80b7-4e9c-de19-08dbd698f616 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3n6bJt3lbLmSJFliod9Df8c0oKJ0B0kYixkak4YE5dU5FvE7VZrT3ojxq6C5Hlzano0OiKxRwC0trK2gDU554PcTgQLYfASE+x5ypdKENJsDTXYW6rvFZoJoHOAPkPmaxfFf6+ejesT7MZJBCjS2szfLQnmOVndQMRFpDLowbE+NSDKCuwDffmxX59d1VaNPEPwtPu5zeq3FuSr+ESdFnV/URjJVUwqTZK+5w/ELSm5tvC12zjib/GUuDjE/yy1jBxHvZmkOwMgtbaZ+Cu42ppKhyW2/wgkPlp4yVZxwglicJbA5SpOBbo9BIX1aDCGKTRj1UoD9xK873I1nuaArGuXcM3UZYNne2d//G7sIxGKqzFuyIQswKCOzHeM+Ra61CNhoOBWhgyAt2pFTup8F64BeqIo3HFrreqh+oMX9R0IAym0+pzzywK09m6KkXUdPyYlNWcBuvkhFML710Sk+ahmc/RzpQirhvh3/37XlzAmaeLdrayKwEpidefZ9f8wtgzGmL4XadGyZmzWRvG0gLbovZ9ILmkWSQT5SfNZEhbkfpA7lQ1GrWr9VqATd3VKSxbzzHAza1y/6cabk7QehKu2thyjP2qevi1NWW1Dyt/t50Ag1rJJYbB9EbkP99lcry97R/rO+xC3DAFykBOEV4TVlnIzR9ER8EJWjjO/0JGs= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376002)(396003)(39840400004)(366004)(346002)(136003)(230922051799003)(64100799003)(186009)(1800799009)(451199024)(6512007)(6666004)(52116002)(6506007)(6486002)(86362001)(36756003)(478600001)(38100700002)(2906002)(6916009)(41300700001)(83380400001)(1076003)(107886003)(26005)(2616005)(38350700005)(66946007)(316002)(66556008)(66476007)(4326008)(8936002)(8676002)(44832011)(5660300002)(54906003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cY2LNlMJoU1Tgz9OjeNCCo1l9RvoL9+pSm57ctkbF5dlblzAJFNDAzWtENeoiMOi39ZWr21hkRO2+XzaIgcVBhIPE3hfwYz9S9C8lARxg/3yYUlG5CiTh2KFjuQOlgTx7iD8DUwsfC/aTSLkod8xJ+TcgEbFNowQbcRg6AL5QtnOB6gUjDTVrDc7eUvhsltGWkQXtVWKNe/le/oXZMfjMfqB1qYmrJWU55bh2Gtky5de4pA1LVMd9OEuFsWvCLT/qi5isZYkWLWrtCaUREcH19ttvUxReLjWnht01gzkKNrNBDr9ujjBk42C3/nwQYOwFhxO1bFG0T+7LAuD/R2jAv2VgNYbZpuVaPnCfGyKiE+hkr0KFq60SoTCsImreKMN5pymL3Zu+40Ag1fLolcDEZ6ZAU1gNELmQQkiEVg191+rGRKuxF6QNVjuD7rKVVGuv7ReIgf6I1sAMWDBTxwGZYcw2Bt8++/8s5ex5zFAL3UJ22eDIoKAEysF8F7w+bJEk+NbGR50LZL9a9/I4LIFf5Exy0QbEgY+Phz3ZipEgapIOjt8BwnU1trD+EJnvp4zVBBRct+Ltcd5bwgilsz/FjociMFsYm3fUqzKrhYH9H8VPmM6raF3NqlU0h3Dun/vHWalbMK7TxmVNYXZC9iH+NU9FHkP9TBjnNQbufvKLtHp61ZESJs4VlW9CRkD7CCUlrC+M8blJybiDz7B5GxcTxnutwrcuuZsU5PhWmsIpoR3lVEYEfDuyW7V/9BOlprIdKiXa1QYiT30OmTWatbF0XQDQ9dgJC/CrKQnhekQlBI5kcoOcWINtTrYIiiOWvzt5DriRPcYqMnzE0FHZYMXRCHX45N+lQdTapgHzoKEDLtZ7lgGtztsRKjpdziK65tA9o2HRwprImBZCIN1i/WMlD7SMp3Xax2+B/wxhOV9x6ZNDxfemVIEHyvYLdWDWAvaBBuJww2J6NWHVhgGSbig3F9wwG9TU01qSgF0tcqu3aKISsEUZGYnlCVRm7dePjjugvVoeFrMdwSLT1X9aP0c18gUHU/QXPRb2F6Xh3A3QDfTdboBVk68uA51vP/t39n/PnqTB2mirNyVF2//aNm5Jwe/xwhXttiNY0BRw+mXA7akgFpaJeWAOhzrsKKqfqOnaYeQvOVIDmmGubjUWhcAZSKdYWx5QBzoTxDzq2xjK4itrYzziweblZSVUnqYstq9njsjMGNPg3t7qGdGkyD3SHYMCGhmKkq22heXljIOVvnXENzrZ2CuJpv8O2CISIydm96sEfZZR0ailGScaHhTQeTep+rHd0YZZv0GFnxe8sz0xAt6a5UVCk23D5WIDesHPK7dfmDnu9tRV5GdOtwqEGusPRj1AQ0XaEtK59F1NNLRct1lAnJvtZaMe8mt4ZkX/BOvPA+zFd8i7BKyAaDrA0Ms/NLCis5TN1ujgvTLhYU4f8WQoJzvAWdt7F124Xll0JyoPRsnAoVWVAwFe4UbWg50ujuu1zqEi+/uy4KssxatTCU17ou2HpnZPDNCB0s1q5SNCFVISYgLkxaAFSPsGlrp3e5/hIX34DliM5X97op3lDz7BhsEYUiCf221M7fKFLh4Mbi/UjhwQHwSLn004w== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8d5dbfdf-80b7-4e9c-de19-08dbd698f616 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Oct 2023 03:01:05.8236 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8a/JM8WN2Aky/lOAaiR9fVpqqZCc4vszGpN21NzgNmhssPzgRMXO/1e8FxwNmYi60nFXbqYqOeqSBvkbt7I9RoKkUMjs4aa6grqVaEi2J1w= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR13MB4048 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add the hardware initialize logic. Signed-off-by: Chaoyong He Signed-off-by: Shujing Dong Reviewed-by: Long Wu Reviewed-by: Peng Zhang --- drivers/vdpa/nfp/meson.build | 1 + drivers/vdpa/nfp/nfp_vdpa.c | 10 ++++++ drivers/vdpa/nfp/nfp_vdpa_core.c | 54 ++++++++++++++++++++++++++++++++ drivers/vdpa/nfp/nfp_vdpa_core.h | 32 +++++++++++++++++++ 4 files changed, 97 insertions(+) create mode 100644 drivers/vdpa/nfp/nfp_vdpa_core.c create mode 100644 drivers/vdpa/nfp/nfp_vdpa_core.h diff --git a/drivers/vdpa/nfp/meson.build b/drivers/vdpa/nfp/meson.build index 79a0364917..258f3eaaa0 100644 --- a/drivers/vdpa/nfp/meson.build +++ b/drivers/vdpa/nfp/meson.build @@ -8,6 +8,7 @@ endif sources = files( 'nfp_vdpa.c', + 'nfp_vdpa_core.c', 'nfp_vdpa_log.c', ) diff --git a/drivers/vdpa/nfp/nfp_vdpa.c b/drivers/vdpa/nfp/nfp_vdpa.c index 824b0d681a..00d8f7e007 100644 --- a/drivers/vdpa/nfp/nfp_vdpa.c +++ b/drivers/vdpa/nfp/nfp_vdpa.c @@ -10,6 +10,7 @@ #include #include +#include "nfp_vdpa_core.h" #include "nfp_vdpa_log.h" #define NFP_VDPA_DRIVER_NAME nfp_vdpa @@ -17,11 +18,14 @@ struct nfp_vdpa_dev { struct rte_pci_device *pci_dev; struct rte_vdpa_device *vdev; + struct nfp_vdpa_hw hw; int vfio_container_fd; int vfio_group_fd; int vfio_dev_fd; int iommu_group; + + uint16_t max_queues; }; struct nfp_vdpa_dev_node { @@ -135,6 +139,12 @@ nfp_vdpa_pci_probe(struct rte_pci_device *pci_dev) if (ret != 0) goto free_device; + ret = nfp_vdpa_hw_init(&device->hw, pci_dev); + if (ret != 0) + goto vfio_teardown; + + device->max_queues = NFP_VDPA_MAX_QUEUES; + device->vdev = rte_vdpa_register_device(&pci_dev->device, &nfp_vdpa_ops); if (device->vdev == NULL) { DRV_VDPA_LOG(ERR, "Failed to register device %s", pci_dev->name); diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.c b/drivers/vdpa/nfp/nfp_vdpa_core.c new file mode 100644 index 0000000000..a7e15fa88a --- /dev/null +++ b/drivers/vdpa/nfp/nfp_vdpa_core.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#include "nfp_vdpa_core.h" + +#include + +#include "nfp_vdpa_log.h" + +/* Available and used descs are in same order */ +#ifndef VIRTIO_F_IN_ORDER +#define VIRTIO_F_IN_ORDER 35 +#endif + +int +nfp_vdpa_hw_init(struct nfp_vdpa_hw *vdpa_hw, + struct rte_pci_device *pci_dev) +{ + uint32_t queue; + struct nfp_hw *hw; + uint8_t *notify_base; + + hw = &vdpa_hw->super; + hw->ctrl_bar = pci_dev->mem_resource[0].addr; + if (hw->ctrl_bar == NULL) { + DRV_CORE_LOG(ERR, "hw->ctrl_bar is NULL. BAR0 not configured."); + return -ENODEV; + } + + notify_base = hw->ctrl_bar + NFP_VDPA_NOTIFY_ADDR_BASE; + for (queue = 0; queue < NFP_VDPA_MAX_QUEUES; queue++) { + uint32_t idx = queue * 2; + + /* RX */ + vdpa_hw->notify_addr[idx] = notify_base; + notify_base += NFP_VDPA_NOTIFY_ADDR_INTERVAL; + /* TX */ + vdpa_hw->notify_addr[idx + 1] = notify_base; + notify_base += NFP_VDPA_NOTIFY_ADDR_INTERVAL; + + vdpa_hw->notify_region = queue; + DRV_CORE_LOG(DEBUG, "notify_addr[%d] at %p, notify_addr[%d] at %p", + idx, vdpa_hw->notify_addr[idx], + idx + 1, vdpa_hw->notify_addr[idx + 1]); + } + + vdpa_hw->features = (1ULL << VIRTIO_F_VERSION_1) | + (1ULL << VIRTIO_F_IN_ORDER) | + (1ULL << VHOST_USER_F_PROTOCOL_FEATURES); + + return 0; +} diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.h b/drivers/vdpa/nfp/nfp_vdpa_core.h new file mode 100644 index 0000000000..c9403e0ea4 --- /dev/null +++ b/drivers/vdpa/nfp/nfp_vdpa_core.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#ifndef __NFP_VDPA_CORE_H__ +#define __NFP_VDPA_CORE_H__ + +#include +#include +#include + +#define NFP_VDPA_MAX_QUEUES 1 + +#define NFP_VDPA_NOTIFY_ADDR_BASE 0x4000 +#define NFP_VDPA_NOTIFY_ADDR_INTERVAL 0x1000 + +struct nfp_vdpa_hw { + struct nfp_hw super; + + uint64_t features; + uint64_t req_features; + + uint8_t *notify_addr[NFP_VDPA_MAX_QUEUES * 2]; + + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; + uint8_t notify_region; +}; + +int nfp_vdpa_hw_init(struct nfp_vdpa_hw *vdpa_hw, struct rte_pci_device *dev); + +#endif /* __NFP_VDPA_CORE_H__ */