From patchwork Thu Apr 1 09:46:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 90293 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8001CA0548; Thu, 1 Apr 2021 11:48:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 487AF140E9E; Thu, 1 Apr 2021 11:48:20 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 8F461140E93 for ; Thu, 1 Apr 2021 11:48:19 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1319fSDj014991 for ; Thu, 1 Apr 2021 02:48:18 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=vC0t+WMQtmkr6j4/NWx3UL1913ys4E2cagUWwW4OMSI=; b=Qvm8E85hlG6SCwBfV1MypiWmEPGMcBLotSf76+yCk2k6gdMWzj1ZNgUY2PCHf5WTtt// Sz8dlGMn56YGaYzpqDAD9OvhWZ28B5O2s79CFMPHedDAJ/D2+oh48mgQSTDSZBxIOTfm jH1qIUolcndKeCrYkK6RCoB/pJt0v6nQJ1vBomaIQD38o1dlJfQMmOCq4enDBYL79Yma zjr/xJxT6eDIMEyySBvgOp+C0RRSEX3LWVPyLxkg8n3bhzRTtxFpJuTJi4wN6po1nzDa QzGWs5UXwgsogaiEYZH/w30y0TppCYfpkSlsTxFYVNdm4PNJqh8M4LejqU9NopZseju2 nA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 37n28j1wmd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 01 Apr 2021 02:48:18 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Apr 2021 02:48:17 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 1 Apr 2021 02:48:17 -0700 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id B5D5D3F7044; Thu, 1 Apr 2021 02:48:14 -0700 (PDT) From: Nithin Dabilpuram To: CC: , , , , , , Date: Thu, 1 Apr 2021 15:16:51 +0530 Message-ID: <20210401094739.22714-5-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210401094739.22714-1-ndabilpuram@marvell.com> References: <20210305133918.8005-1-ndabilpuram@marvell.com> <20210401094739.22714-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: FQB00qLICit79HkEhHylyPwKkp_LEDvq X-Proofpoint-GUID: FQB00qLICit79HkEhHylyPwKkp_LEDvq X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-01_04:2021-03-31, 2021-04-01 signatures=0 Subject: [dpdk-dev] [PATCH v2 04/52] common/cnxk: add interrupt helper API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add interrupt helper API's in common code to register and unregister for specific interrupt vectors. These API's will be used by all cnxk drivers. Signed-off-by: Jerin Jacob --- drivers/common/cnxk/meson.build | 3 +- drivers/common/cnxk/roc_dev_priv.h | 14 +++ drivers/common/cnxk/roc_irq.c | 249 +++++++++++++++++++++++++++++++++++++ drivers/common/cnxk/roc_platform.c | 4 + drivers/common/cnxk/roc_platform.h | 11 ++ drivers/common/cnxk/roc_priv.h | 3 + drivers/common/cnxk/version.map | 1 + 7 files changed, 284 insertions(+), 1 deletion(-) create mode 100644 drivers/common/cnxk/roc_dev_priv.h create mode 100644 drivers/common/cnxk/roc_irq.c diff --git a/drivers/common/cnxk/meson.build b/drivers/common/cnxk/meson.build index 0f8e409..e90b848 100644 --- a/drivers/common/cnxk/meson.build +++ b/drivers/common/cnxk/meson.build @@ -16,7 +16,8 @@ endif config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON' deps = ['eal', 'pci', 'bus_pci', 'mbuf'] -sources = files('roc_model.c', +sources = files('roc_irq.c', + 'roc_model.c', 'roc_platform.c', 'roc_utils.c') includes += include_directories('../../bus/pci') diff --git a/drivers/common/cnxk/roc_dev_priv.h b/drivers/common/cnxk/roc_dev_priv.h new file mode 100644 index 0000000..2254677 --- /dev/null +++ b/drivers/common/cnxk/roc_dev_priv.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef _ROC_DEV_PRIV_H +#define _ROC_DEV_PRIV_H + +int dev_irq_register(struct plt_intr_handle *intr_handle, + plt_intr_callback_fn cb, void *data, unsigned int vec); +void dev_irq_unregister(struct plt_intr_handle *intr_handle, + plt_intr_callback_fn cb, void *data, unsigned int vec); +int dev_irqs_disable(struct plt_intr_handle *intr_handle); + +#endif /* _ROC_DEV_PRIV_H */ diff --git a/drivers/common/cnxk/roc_irq.c b/drivers/common/cnxk/roc_irq.c new file mode 100644 index 0000000..4c2b4c3 --- /dev/null +++ b/drivers/common/cnxk/roc_irq.c @@ -0,0 +1,249 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "roc_api.h" +#include "roc_priv.h" + +#if defined(__linux__) + +#include +#include +#include +#include +#include + +#define MSIX_IRQ_SET_BUF_LEN \ + (sizeof(struct vfio_irq_set) + sizeof(int) * (PLT_MAX_RXTX_INTR_VEC_ID)) + +static int +irq_get_info(struct plt_intr_handle *intr_handle) +{ + struct vfio_irq_info irq = {.argsz = sizeof(irq)}; + int rc; + + irq.index = VFIO_PCI_MSIX_IRQ_INDEX; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_GET_IRQ_INFO, &irq); + if (rc < 0) { + plt_err("Failed to get IRQ info rc=%d errno=%d", rc, errno); + return rc; + } + + plt_base_dbg("Flags=0x%x index=0x%x count=0x%x max_intr_vec_id=0x%x", + irq.flags, irq.index, irq.count, PLT_MAX_RXTX_INTR_VEC_ID); + + if (irq.count > PLT_MAX_RXTX_INTR_VEC_ID) { + plt_err("HW max=%d > PLT_MAX_RXTX_INTR_VEC_ID: %d", irq.count, + PLT_MAX_RXTX_INTR_VEC_ID); + intr_handle->max_intr = PLT_MAX_RXTX_INTR_VEC_ID; + } else { + intr_handle->max_intr = irq.count; + } + + return 0; +} + +static int +irq_config(struct plt_intr_handle *intr_handle, unsigned int vec) +{ + char irq_set_buf[MSIX_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int32_t *fd_ptr; + int len, rc; + + if (vec > intr_handle->max_intr) { + plt_err("vector=%d greater than max_intr=%d", vec, + intr_handle->max_intr); + return -EINVAL; + } + + len = sizeof(struct vfio_irq_set) + sizeof(int32_t); + + irq_set = (struct vfio_irq_set *)irq_set_buf; + irq_set->argsz = len; + + irq_set->start = vec; + irq_set->count = 1; + irq_set->flags = + VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX; + + /* Use vec fd to set interrupt vectors */ + fd_ptr = (int32_t *)&irq_set->data[0]; + fd_ptr[0] = intr_handle->efds[vec]; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (rc) + plt_err("Failed to set_irqs vector=0x%x rc=%d", vec, rc); + + return rc; +} + +static int +irq_init(struct plt_intr_handle *intr_handle) +{ + char irq_set_buf[MSIX_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int32_t *fd_ptr; + int len, rc; + uint32_t i; + + if (intr_handle->max_intr > PLT_MAX_RXTX_INTR_VEC_ID) { + plt_err("Max_intr=%d greater than PLT_MAX_RXTX_INTR_VEC_ID=%d", + intr_handle->max_intr, PLT_MAX_RXTX_INTR_VEC_ID); + return -ERANGE; + } + + len = sizeof(struct vfio_irq_set) + + sizeof(int32_t) * intr_handle->max_intr; + + irq_set = (struct vfio_irq_set *)irq_set_buf; + irq_set->argsz = len; + irq_set->start = 0; + irq_set->count = intr_handle->max_intr; + irq_set->flags = + VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX; + + fd_ptr = (int32_t *)&irq_set->data[0]; + for (i = 0; i < irq_set->count; i++) + fd_ptr[i] = -1; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (rc) + plt_err("Failed to set irqs vector rc=%d", rc); + + return rc; +} + +int +dev_irqs_disable(struct plt_intr_handle *intr_handle) +{ + /* Clear max_intr to indicate re-init next time */ + intr_handle->max_intr = 0; + return plt_intr_disable(intr_handle); +} + +int +dev_irq_register(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + struct plt_intr_handle tmp_handle; + int rc; + + /* If no max_intr read from VFIO */ + if (intr_handle->max_intr == 0) { + irq_get_info(intr_handle); + irq_init(intr_handle); + } + + if (vec > intr_handle->max_intr) { + plt_err("Vector=%d greater than max_intr=%d", vec, + intr_handle->max_intr); + return -EINVAL; + } + + tmp_handle = *intr_handle; + /* Create new eventfd for interrupt vector */ + tmp_handle.fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (tmp_handle.fd == -1) + return -ENODEV; + + /* Register vector interrupt callback */ + rc = plt_intr_callback_register(&tmp_handle, cb, data); + if (rc) { + plt_err("Failed to register vector:0x%x irq callback.", vec); + return rc; + } + + intr_handle->efds[vec] = tmp_handle.fd; + intr_handle->nb_efd = + (vec > intr_handle->nb_efd) ? vec : intr_handle->nb_efd; + if ((intr_handle->nb_efd + 1) > intr_handle->max_intr) + intr_handle->max_intr = intr_handle->nb_efd + 1; + + plt_base_dbg("Enable vector:0x%x for vfio (efds: %d, max:%d)", vec, + intr_handle->nb_efd, intr_handle->max_intr); + + /* Enable MSIX vectors to VFIO */ + return irq_config(intr_handle, vec); +} + +void +dev_irq_unregister(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + struct plt_intr_handle tmp_handle; + uint8_t retries = 5; /* 5 ms */ + int rc; + + if (vec > intr_handle->max_intr) { + plt_err("Error unregistering MSI-X interrupts vec:%d > %d", vec, + intr_handle->max_intr); + return; + } + + tmp_handle = *intr_handle; + tmp_handle.fd = intr_handle->efds[vec]; + if (tmp_handle.fd == -1) + return; + + do { + /* Un-register callback func from platform lib */ + rc = plt_intr_callback_unregister(&tmp_handle, cb, data); + /* Retry only if -EAGAIN */ + if (rc != -EAGAIN) + break; + plt_delay_ms(1); + retries--; + } while (retries); + + if (rc < 0) { + plt_err("Error unregistering MSI-X vec %d cb, rc=%d", vec, rc); + return; + } + + plt_base_dbg("Disable vector:0x%x for vfio (efds: %d, max:%d)", vec, + intr_handle->nb_efd, intr_handle->max_intr); + + if (intr_handle->efds[vec] != -1) + close(intr_handle->efds[vec]); + /* Disable MSIX vectors from VFIO */ + intr_handle->efds[vec] = -1; + irq_config(intr_handle, vec); +} + +#else + +int +dev_irq_register(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + PLT_SET_USED(intr_handle); + PLT_SET_USED(cb); + PLT_SET_USED(data); + PLT_SET_USED(vec); + + return -ENOTSUP; +} + +void +dev_irq_unregister(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + PLT_SET_USED(intr_handle); + PLT_SET_USED(cb); + PLT_SET_USED(data); + PLT_SET_USED(vec); +} + +int +dev_irqs_disable(struct plt_intr_handle *intr_handle) +{ + PLT_SET_USED(intr_handle); + + return -ENOTSUP; +} + +#endif /* __linux__ */ diff --git a/drivers/common/cnxk/roc_platform.c b/drivers/common/cnxk/roc_platform.c index 9f0c53e..ee1a28b 100644 --- a/drivers/common/cnxk/roc_platform.c +++ b/drivers/common/cnxk/roc_platform.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include + #include "roc_api.h" int @@ -24,3 +26,5 @@ roc_plt_init(void) roc_model_init(mz->addr); return 0; } + +RTE_LOG_REGISTER(cnxk_logtype_base, pmd.cnxk.base, NOTICE); diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 1d06435..0fe0c18 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -132,12 +132,23 @@ #define plt_strlcpy rte_strlcpy /* Log */ +extern int cnxk_logtype_base; #define plt_err(fmt, args...) \ RTE_LOG(ERR, PMD, "%s():%u " fmt "\n", __func__, __LINE__, ##args) #define plt_info(fmt, args...) RTE_LOG(INFO, PMD, fmt "\n", ##args) #define plt_warn(fmt, args...) RTE_LOG(WARNING, PMD, fmt "\n", ##args) #define plt_print(fmt, args...) RTE_LOG(INFO, PMD, fmt "\n", ##args) +/** + * Log debug message if given subsystem logging is enabled. + */ +#define plt_dbg(subsystem, fmt, args...) \ + rte_log(RTE_LOG_DEBUG, cnxk_logtype_##subsystem, \ + "[%s] %s():%u " fmt "\n", #subsystem, __func__, __LINE__, \ + ##args) + +#define plt_base_dbg(fmt, ...) plt_dbg(base, fmt, ##__VA_ARGS__) + #ifdef __cplusplus #define CNXK_PCI_ID(subsystem_dev, dev) \ { \ diff --git a/drivers/common/cnxk/roc_priv.h b/drivers/common/cnxk/roc_priv.h index 9c905d4..cd87035 100644 --- a/drivers/common/cnxk/roc_priv.h +++ b/drivers/common/cnxk/roc_priv.h @@ -8,4 +8,7 @@ /* Utils */ #include "roc_util_priv.h" +/* Dev */ +#include "roc_dev_priv.h" + #endif /* _ROC_PRIV_H_ */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 1798b48..7102704 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -1,6 +1,7 @@ INTERNAL { global: + cnxk_logtype_base; roc_error_msg_get; roc_model; roc_plt_init;