From patchwork Thu Oct 26 06:43:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 133363 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ECDF543204; Thu, 26 Oct 2023 08:47:52 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9D90E42E5F; Thu, 26 Oct 2023 08:44:39 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2139.outbound.protection.outlook.com [40.107.93.139]) by mails.dpdk.org (Postfix) with ESMTP id DD17E42830 for ; Thu, 26 Oct 2023 08:44:30 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QGBVyIeBMMBGoummtT590jHM+DID838lo4vFyrlJWgQwxmMsRzUMAJQqF8UfhV/u2l0M10tb110Cz2Uow3FXBKBouAui8IiBExkPZZyUnThQ2Ay8gRxR9j3Mlg4b8dsnwe52CSfrINBDZKEIMLnjAM/sjNifTqXnmh72bbvuM1DZgqIzkeufjrKKzcye3GJvLZKxPB7vEyrAaervr18ejw4j8croJOwk5V8xuyASY+/2+I4zKrnFMwHWjwA1byDwpGd+TIGNqzCMGcMsyyit0pau+Yg9qgEqg0KFKSn3Q18AIFuydueD45rGcPBFpG6d0X1RmTZVxAmyyxdZYqwGdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EkIhE+6SUYsV+j9Cgqztz8HgRPhMNijBbqLxanMivEM=; b=Az4rvCCFT5q/SHZ+oZMKBiTtyVkfBaeGWapOgwuMwoYzPrUeix+To6NXjQsU8DIVA0hfg2kqi+kJM8CMl/GPsK18FJB2RdaVw9MItCTm1ax+pjRUqEjFQ/DoXFiHxL/AqURuDXK1mVHIzIxubKept27+z8BHWi6ug1jrStswjfhCy0cq1l7HoC4dVG/t/+6RYKtDY29gFrXrc2EKq8GyCRT4tXKxVTlOTxaPFRNCPduBq2eJ5GKZnL1iSHweYmX73fLXvzqwIHUlDcDulm9fUi3h2zXbOuPLKRQzefIst5bq+Z1TrmxtwHIKcnIyuPmY2YUCDs6vrqtXuLgL9jDFYQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EkIhE+6SUYsV+j9Cgqztz8HgRPhMNijBbqLxanMivEM=; b=diknU+2LEn7zOpTnBc2kzcIzD2dds7zWu+9rB0RdmpIk+Dh3BgkU9+I6+L/bj7DZF+KuLxBm7yIEU00cg3NY8DmWaMxqOnHsev1K5YoOi1Miiv9kyPFIxibKlIGQtZabzV1w+2y6mD/5Z7e11lpBrbvH4vpon8BWHmPs33hXLL8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by DM4PR13MB5908.namprd13.prod.outlook.com (2603:10b6:8:50::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.19; Thu, 26 Oct 2023 06:44:29 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b%6]) with mapi id 15.20.6933.022; Thu, 26 Oct 2023 06:44:29 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Chaoyong He , Shujing Dong , Long Wu , Peng Zhang Subject: [PATCH v3 23/25] vdpa/nfp: add the notify related logic Date: Thu, 26 Oct 2023 14:43:22 +0800 Message-Id: <20231026064324.177531-24-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231026064324.177531-1-chaoyong.he@corigine.com> References: <20231024022849.3092506-1-chaoyong.he@corigine.com> <20231026064324.177531-1-chaoyong.he@corigine.com> X-ClientProxiedBy: BYAPR07CA0034.namprd07.prod.outlook.com (2603:10b6:a02:bc::47) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|DM4PR13MB5908:EE_ X-MS-Office365-Filtering-Correlation-Id: cbc7d79b-65a1-456c-c9bc-08dbd5ef00a4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gZfo4oaSwWdSCLCseWKO1SMPp39GO8JexZdhB4WvuY9c5PpnaYWdIgOCoTwftvAPMXz0B9AOpuHF0W3l3tRm4vUukSPnTs7wVeAkL5P+MqUMdJ3B1GvF55Jf5z98dKG1n99mDyUpyhhTEI3scMEyN6NdwMWnWAJTAOko1FNNcCYa96j1jhMdy4DxjSLON28rHTXZBI2Ey9X4q8GKynAL/JVCsvoFCXawWWesan7FVAU/U54Umu+f+xaK2e/PLGJ5wBUXTh/77xh8ybeODEKhWbnlzM5d6iaVuRLiV9S2lQ/viq7a2bPGswrMbRJVbPt68yiUMPcu8Ot36Yr7IblP2yL1qhL3FURbp881F7XqIWTJEJeyx5hH4FObhBibDdWLfcXm+TkW374Cd8GCAhfNhxxNHMYmTxNXtZpsIdtMcSWA2yef/C06rbZXcistCpBpW93NUT4WP0Uop5v41RovsySoTP/HKmis6DmpTqek2ocKQbNh26PUWMs8dmMPFD1I3rhVWu3i/xtJpqDAYzQDGakvakQSbigJH0Z5E8qhBfkZ6GKpNIDVBKV7Ik2pulUzH7ZKjV/0AH8mFeyDH9Q7SlQ8AzG686okbklcii25xlRTqOl3BfQpfgGwbPqiykHQcMk0Idv7C9I69KNs7UxWQyOnUsL8yMrlCHDoK57/D6k= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(376002)(366004)(396003)(346002)(39840400004)(230922051799003)(186009)(1800799009)(451199024)(64100799003)(26005)(478600001)(1076003)(316002)(6666004)(6512007)(86362001)(6506007)(6916009)(66946007)(66556008)(38100700002)(66476007)(54906003)(36756003)(38350700005)(2616005)(6486002)(107886003)(44832011)(2906002)(83380400001)(4326008)(8676002)(8936002)(5660300002)(41300700001)(52116002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HFnk2JxAvSZNIDyLc9L17CVxztSFpgR5Fk5x3YesBbWRoTS5+27ge96wui/hdrgMRZe44d4vzWc2/IEBJB1WNkUtULBTtM2xXJwjcPRvOi1NmGCZlF9cNdzzr4Wj3M/O5q1g4DOquc11jcj6joxERwFwM9ykghNNu1XZLqO8Bj7EUfkfevwZUo4JaRCCuKU0qDj70OLhN1SDUpreavd2s4OcjxoNWBshCCN1jG3Z8t0odEGRqreIWhwe141q1ekL8c5j6aVKmZE9BaWQXt6Fv6jQraI2sik1gTWilKMSuWHsOefxP2BS+3jFMEkrj7tkc6k284tJhJaktrwczH4igNtxIR+XLCJVisUoiUI6RxMxm/Tv2fiZFFYHzGdQr8Lb3UfKmV0a9y6R2c2tfrM0IaHOKBtlLhQVO7Fw56J+aeu5+BxD1Y9QbnNCx+XRVZpUxasI/7/p+UaAyGQd+5yXbcRTLHOE7SJ1MK+KjFpUeAAn3GWWSn43VmrWbcia3GdiXVfq11/1yuKGoT6csLpAE/j1Omt5cP9Mueg7oyq+0HjNqMICGbpfSXI3QTy2ygrxi2XYT8EYurW3LjNxBBEDFtpypyEV0bvwPBFkfbIoIKPaRrvCpf+7rnBO8FojQZdtiF3e2I/xY8FFFTR2O9J+u+tOEJEUASS14eZitVB+XuddP6gB5dwrdybQwzmJRCPC+o5e6LrD2iS60lQXimdh/dFpReGVL5Lms8CRb7P1wrJmw70IT3YAIOmP0W3m+mtxdI4kJfV1hc4C41STFAXRziaO8NUHXRY93Ve/LMRYpbqIQglHsppTq1Od8NV9YhuEEDmwbSNKKEiGlq3pcpOCh3G1RDASVulJ2NoAwstsfpAYFln3CCbzE/7Lz9L4Fb+E3d3qEHRictxAUCaS4tPZvXzgBBYLnObDBK++VBdmxb93JacxpjEcw5JHOFfdE0w+ztH3VDIdNwlb9LbvI7nJAPs94LEZiJvMJ6iH8z+AXbVeWonAkChgZjOK/+Rz156OAYgCTImZlBtk9n1ZvKnfukyTXQzxfE5dY51Xj/lA/gjhM/Est7mGB4QLeFaNWv4UBPBTh4Lv+BRQ3dMlWYIQP48a7e5tZzkGESAjy6jAhCMW0/dlw/T3EGGrvSv8uf5GNOSDFBUPvqbV54Bd1liQdjX9Aeqro1mJZ9ObwaC6J3JoMFlu2CXQdgqF+NbjxDaJU2KiOyHb/ZFnI0ae/ApM95uF4wyaDYSmZb6h8WWT+8ULJdcQPngROEPHxsmW35B+1038/E0p+TirqikaCdoDhjKle26J38IgmQxuU+fceCD0KEkz9i7GtQbY5NNe8CoeIliTpiL+hFKBaQ/GPRfjKIACUauARTybPvzAfNMRw0DSeM+KA6jMUhtZJcsoRjeePm/UpsCHEYScDXwVKnQMlCw8OiuSnrrJLEkcLKWfySyud0TE74+mICf4+bx4P+vQe00T8E3sB4YOe0PDArJZ2sDPFY0FnFM7ZVgELxEhCl4NNWqp+GSDgLgQje4+3skvn1ANpeeJPIxn0LVQgx2q4Jx5DNAu/DpSTTnx3lU8umEV+gPfXhGN2lc5y6xlbxCyoBBMuDbrZpE6I1yseyuTHg== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: cbc7d79b-65a1-456c-c9bc-08dbd5ef00a4 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Oct 2023 06:44:29.0583 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: W++l/uTSmV9TJCNe7+MiQr91B8e1e5OEuh80Omu8gQj7SZgGMCodFO5jtiuJqN/VRKcKXDjRXHo2//4u65iyneHTntL5mOcJ7G/Hv2vVB94= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR13MB5908 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add the logic to process vDPA notify relay. Signed-off-by: Chaoyong He Signed-off-by: Shujing Dong Reviewed-by: Long Wu Reviewed-by: Peng Zhang --- drivers/vdpa/nfp/nfp_vdpa.c | 155 +++++++++++++++++++++++++++++++ drivers/vdpa/nfp/nfp_vdpa_core.c | 61 ++++++++++++ drivers/vdpa/nfp/nfp_vdpa_core.h | 4 + 3 files changed, 220 insertions(+) diff --git a/drivers/vdpa/nfp/nfp_vdpa.c b/drivers/vdpa/nfp/nfp_vdpa.c index d0d8050887..f4d63b8ba9 100644 --- a/drivers/vdpa/nfp/nfp_vdpa.c +++ b/drivers/vdpa/nfp/nfp_vdpa.c @@ -4,7 +4,9 @@ */ #include +#include #include +#include #include #include @@ -29,6 +31,9 @@ struct nfp_vdpa_dev { int vfio_dev_fd; int iommu_group; + rte_thread_t tid; /**< Thread for notify relay */ + int epoll_fd; + int vid; uint16_t max_queues; uint32_t started; @@ -368,6 +373,148 @@ nfp_vdpa_disable_vfio_intr(struct nfp_vdpa_dev *device) return 0; } +static void +nfp_vdpa_read_kickfd(int kickfd) +{ + int bytes; + uint64_t buf; + + for (;;) { + bytes = read(kickfd, &buf, 8); + if (bytes >= 0) + break; + + if (errno != EINTR && errno != EWOULDBLOCK && + errno != EAGAIN) { + DRV_VDPA_LOG(ERR, "Error reading kickfd"); + break; + } + } +} + +static int +nfp_vdpa_notify_epoll_ctl(uint32_t queue_num, + struct nfp_vdpa_dev *device) +{ + int ret; + uint32_t qid; + + for (qid = 0; qid < queue_num; qid++) { + struct epoll_event ev; + struct rte_vhost_vring vring; + + ev.events = EPOLLIN | EPOLLPRI; + rte_vhost_get_vhost_vring(device->vid, qid, &vring); + ev.data.u64 = qid | (uint64_t)vring.kickfd << 32; + ret = epoll_ctl(device->epoll_fd, EPOLL_CTL_ADD, vring.kickfd, &ev); + if (ret < 0) { + DRV_VDPA_LOG(ERR, "Epoll add error for queue %d", qid); + return ret; + } + } + + return 0; +} + +static int +nfp_vdpa_notify_epoll_wait(uint32_t queue_num, + struct nfp_vdpa_dev *device) +{ + int i; + int fds; + int kickfd; + uint32_t qid; + struct epoll_event events[NFP_VDPA_MAX_QUEUES * 2]; + + for (;;) { + fds = epoll_wait(device->epoll_fd, events, queue_num, -1); + if (fds < 0) { + if (errno == EINTR) + continue; + + DRV_VDPA_LOG(ERR, "Epoll wait fail"); + return -EACCES; + } + + for (i = 0; i < fds; i++) { + qid = events[i].data.u32; + kickfd = (uint32_t)(events[i].data.u64 >> 32); + + nfp_vdpa_read_kickfd(kickfd); + nfp_vdpa_notify_queue(&device->hw, qid); + } + } + + return 0; +} + +static uint32_t +nfp_vdpa_notify_relay(void *arg) +{ + int ret; + int epoll_fd; + uint32_t queue_num; + struct nfp_vdpa_dev *device = arg; + + epoll_fd = epoll_create(NFP_VDPA_MAX_QUEUES * 2); + if (epoll_fd < 0) { + DRV_VDPA_LOG(ERR, "failed to create epoll instance."); + return 1; + } + + device->epoll_fd = epoll_fd; + + queue_num = rte_vhost_get_vring_num(device->vid); + + ret = nfp_vdpa_notify_epoll_ctl(queue_num, device); + if (ret != 0) + goto notify_exit; + + ret = nfp_vdpa_notify_epoll_wait(queue_num, device); + if (ret != 0) + goto notify_exit; + + return 0; + +notify_exit: + close(device->epoll_fd); + device->epoll_fd = -1; + + return 1; +} + +static int +nfp_vdpa_setup_notify_relay(struct nfp_vdpa_dev *device) +{ + int ret; + char name[RTE_THREAD_INTERNAL_NAME_SIZE]; + + snprintf(name, sizeof(name), "nfp-noti%d", device->vid); + ret = rte_thread_create_internal_control(&device->tid, name, + nfp_vdpa_notify_relay, (void *)device); + if (ret != 0) { + DRV_VDPA_LOG(ERR, "Failed to create notify relay pthread."); + return -1; + } + + return 0; +} + +static void +nfp_vdpa_unset_notify_relay(struct nfp_vdpa_dev *device) +{ + if (device->tid.opaque_id != 0) { + pthread_cancel((pthread_t)device->tid.opaque_id); + rte_thread_join(device->tid, NULL); + device->tid.opaque_id = 0; + } + + if (device->epoll_fd >= 0) { + close(device->epoll_fd); + device->epoll_fd = -1; + } +} + static int update_datapath(struct nfp_vdpa_dev *device) { @@ -392,12 +539,18 @@ update_datapath(struct nfp_vdpa_dev *device) if (ret != 0) goto disable_vfio_intr; + ret = nfp_vdpa_setup_notify_relay(device); + if (ret != 0) + goto vdpa_stop; + rte_atomic_store_explicit(&device->running, 1, rte_memory_order_relaxed); } else if ((rte_atomic_load_explicit(&device->running, rte_memory_order_relaxed) != 0) && ((rte_atomic_load_explicit(&device->started, rte_memory_order_relaxed) != 0) || (rte_atomic_load_explicit(&device->dev_attached, rte_memory_order_relaxed) != 0))) { + nfp_vdpa_unset_notify_relay(device); + nfp_vdpa_stop(device); ret = nfp_vdpa_disable_vfio_intr(device); @@ -414,6 +567,8 @@ update_datapath(struct nfp_vdpa_dev *device) rte_spinlock_unlock(&device->lock); return 0; +vdpa_stop: + nfp_vdpa_stop(device); disable_vfio_intr: nfp_vdpa_disable_vfio_intr(device); dma_map_rollback: diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.c b/drivers/vdpa/nfp/nfp_vdpa_core.c index db9b8462b4..e2a6253ae5 100644 --- a/drivers/vdpa/nfp/nfp_vdpa_core.c +++ b/drivers/vdpa/nfp/nfp_vdpa_core.c @@ -15,6 +15,41 @@ #define VIRTIO_F_IN_ORDER 35 #endif +#define NFP_QCP_NOTIFY_MAX_ADD 0x7f + +enum nfp_qcp_notify_ptr { + NFP_QCP_NOTIFY_WRITE_PTR = 0, + NFP_QCP_NOTIFY_READ_PTR +}; + +/** + * Add the value to the selected pointer of a queue + * + * @param queue + * Base address for queue structure + * @param ptr + * Add to the Read or Write pointer + * @param val + * Value to add to the queue pointer + */ +static inline void +nfp_qcp_notify_ptr_add(uint8_t *q, + enum nfp_qcp_notify_ptr ptr, + uint32_t val) +{ + uint32_t off; + + if (ptr == NFP_QCP_NOTIFY_WRITE_PTR) + off = NFP_QCP_QUEUE_ADD_RPTR; + else + off = NFP_QCP_QUEUE_ADD_WPTR; + + for (; val > NFP_QCP_NOTIFY_MAX_ADD; val -= NFP_QCP_NOTIFY_MAX_ADD) + nn_writel(rte_cpu_to_le_32(NFP_QCP_NOTIFY_MAX_ADD), q + off); + + nn_writel(rte_cpu_to_le_32(val), q + off); +} + int nfp_vdpa_hw_init(struct nfp_vdpa_hw *vdpa_hw, struct rte_pci_device *pci_dev) @@ -130,3 +165,29 @@ nfp_vdpa_hw_stop(struct nfp_vdpa_hw *vdpa_hw) { nfp_disable_queues(&vdpa_hw->super); } + +/* + * This offset is used for mmaping the notify area. It implies it needs + * to be a multiple of PAGE_SIZE. + * For debugging, using notify region 0 with an offset of 4K. This should + * point to the conf bar. + */ +uint64_t +nfp_vdpa_get_queue_notify_offset(struct nfp_vdpa_hw *vdpa_hw __rte_unused, + int qid) +{ + return NFP_VDPA_NOTIFY_ADDR_BASE + (qid * NFP_VDPA_NOTIFY_ADDR_INTERVAL); +} + +/* + * With just one queue the increment is 0, which does not + * incremente the counter but will raise a queue event due + * to queue configured for watermark events. + */ +void +nfp_vdpa_notify_queue(struct nfp_vdpa_hw *vdpa_hw, + uint16_t qid) +{ + nfp_qcp_notify_ptr_add(vdpa_hw->notify_addr[qid], + NFP_QCP_NOTIFY_WRITE_PTR, qid); +} diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.h b/drivers/vdpa/nfp/nfp_vdpa_core.h index a88de768dd..a8e0d6dd70 100644 --- a/drivers/vdpa/nfp/nfp_vdpa_core.h +++ b/drivers/vdpa/nfp/nfp_vdpa_core.h @@ -44,4 +44,8 @@ int nfp_vdpa_hw_start(struct nfp_vdpa_hw *vdpa_hw, int vid); void nfp_vdpa_hw_stop(struct nfp_vdpa_hw *vdpa_hw); +void nfp_vdpa_notify_queue(struct nfp_vdpa_hw *vdpa_hw, uint16_t qid); + +uint64_t nfp_vdpa_get_queue_notify_offset(struct nfp_vdpa_hw *vdpa_hw, int qid); + #endif /* __NFP_VDPA_CORE_H__ */