From patchwork Thu Apr 1 12:37:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 90377 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 69D86A0548; Thu, 1 Apr 2021 14:39:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ED461141192; Thu, 1 Apr 2021 14:38:51 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id B4D0114119C for ; Thu, 1 Apr 2021 14:38:49 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 131CPLcL019084 for ; Thu, 1 Apr 2021 05:38:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=HpTttIeOZ4ArukuR9JxbCPDlK/vH3JNR65/zZIX465w=; b=k7Yqo0a/ns4odLlArc01o+VujEZmndnP1DGb2ziZSP4I0V1Eq7B8XhLL//JwgnPrXMwq 4dTO6YD/G8CvSiv0x9qidWFFGdg2QLW4FpACLQ05Fcs5VnaIgjtB08MUewsIqfwvW35c NzReRsgK8ivqH1GL+5DN8yeXb5KnlgVELe08ttLg1icnSm02rStxcOHlNgCwWaMmYP1f 22QsKu5aNO/Bdfqi8Z1zcYWKAtgVoMumruF1LLhRN2VvDLI4Bw/d9KQNcTTD1oVWd3vg Xf3rFzOP2+jszR3iyBhuVRMopmcPVA6yAg9Dl3Mfbd6ipSEWeqqFNOwBA2Op9I29yhwM 7A== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37n28jjdy0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 01 Apr 2021 05:38:49 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 1 Apr 2021 05:38:47 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 1 Apr 2021 05:38:46 -0700 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 8C5B63F703F; Thu, 1 Apr 2021 05:38:44 -0700 (PDT) From: Nithin Dabilpuram To: CC: , , , , , , Date: Thu, 1 Apr 2021 18:07:29 +0530 Message-ID: <20210401123817.14348-5-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210401123817.14348-1-ndabilpuram@marvell.com> References: <20210305133918.8005-1-ndabilpuram@marvell.com> <20210401123817.14348-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: uANM1uj4KhPaOVnHkUvzforzaNG3haid X-Proofpoint-ORIG-GUID: uANM1uj4KhPaOVnHkUvzforzaNG3haid X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-01_05:2021-03-31, 2021-04-01 signatures=0 Subject: [dpdk-dev] [PATCH v3 04/52] common/cnxk: add interrupt helper API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add interrupt helper API's in common code to register and unregister for specific interrupt vectors. These API's will be used by all cnxk drivers. Signed-off-by: Jerin Jacob --- drivers/common/cnxk/meson.build | 3 +- drivers/common/cnxk/roc_dev_priv.h | 14 +++ drivers/common/cnxk/roc_irq.c | 249 +++++++++++++++++++++++++++++++++++++ drivers/common/cnxk/roc_platform.c | 4 + drivers/common/cnxk/roc_platform.h | 11 ++ drivers/common/cnxk/roc_priv.h | 3 + drivers/common/cnxk/version.map | 1 + 7 files changed, 284 insertions(+), 1 deletion(-) create mode 100644 drivers/common/cnxk/roc_dev_priv.h create mode 100644 drivers/common/cnxk/roc_irq.c diff --git a/drivers/common/cnxk/meson.build b/drivers/common/cnxk/meson.build index b0c02ce..3e0678d 100644 --- a/drivers/common/cnxk/meson.build +++ b/drivers/common/cnxk/meson.build @@ -10,7 +10,8 @@ endif config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON' deps = ['eal', 'pci', 'bus_pci', 'mbuf'] -sources = files('roc_model.c', +sources = files('roc_irq.c', + 'roc_model.c', 'roc_platform.c', 'roc_utils.c') includes += include_directories('../../bus/pci') diff --git a/drivers/common/cnxk/roc_dev_priv.h b/drivers/common/cnxk/roc_dev_priv.h new file mode 100644 index 0000000..2254677 --- /dev/null +++ b/drivers/common/cnxk/roc_dev_priv.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef _ROC_DEV_PRIV_H +#define _ROC_DEV_PRIV_H + +int dev_irq_register(struct plt_intr_handle *intr_handle, + plt_intr_callback_fn cb, void *data, unsigned int vec); +void dev_irq_unregister(struct plt_intr_handle *intr_handle, + plt_intr_callback_fn cb, void *data, unsigned int vec); +int dev_irqs_disable(struct plt_intr_handle *intr_handle); + +#endif /* _ROC_DEV_PRIV_H */ diff --git a/drivers/common/cnxk/roc_irq.c b/drivers/common/cnxk/roc_irq.c new file mode 100644 index 0000000..4c2b4c3 --- /dev/null +++ b/drivers/common/cnxk/roc_irq.c @@ -0,0 +1,249 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "roc_api.h" +#include "roc_priv.h" + +#if defined(__linux__) + +#include +#include +#include +#include +#include + +#define MSIX_IRQ_SET_BUF_LEN \ + (sizeof(struct vfio_irq_set) + sizeof(int) * (PLT_MAX_RXTX_INTR_VEC_ID)) + +static int +irq_get_info(struct plt_intr_handle *intr_handle) +{ + struct vfio_irq_info irq = {.argsz = sizeof(irq)}; + int rc; + + irq.index = VFIO_PCI_MSIX_IRQ_INDEX; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_GET_IRQ_INFO, &irq); + if (rc < 0) { + plt_err("Failed to get IRQ info rc=%d errno=%d", rc, errno); + return rc; + } + + plt_base_dbg("Flags=0x%x index=0x%x count=0x%x max_intr_vec_id=0x%x", + irq.flags, irq.index, irq.count, PLT_MAX_RXTX_INTR_VEC_ID); + + if (irq.count > PLT_MAX_RXTX_INTR_VEC_ID) { + plt_err("HW max=%d > PLT_MAX_RXTX_INTR_VEC_ID: %d", irq.count, + PLT_MAX_RXTX_INTR_VEC_ID); + intr_handle->max_intr = PLT_MAX_RXTX_INTR_VEC_ID; + } else { + intr_handle->max_intr = irq.count; + } + + return 0; +} + +static int +irq_config(struct plt_intr_handle *intr_handle, unsigned int vec) +{ + char irq_set_buf[MSIX_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int32_t *fd_ptr; + int len, rc; + + if (vec > intr_handle->max_intr) { + plt_err("vector=%d greater than max_intr=%d", vec, + intr_handle->max_intr); + return -EINVAL; + } + + len = sizeof(struct vfio_irq_set) + sizeof(int32_t); + + irq_set = (struct vfio_irq_set *)irq_set_buf; + irq_set->argsz = len; + + irq_set->start = vec; + irq_set->count = 1; + irq_set->flags = + VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX; + + /* Use vec fd to set interrupt vectors */ + fd_ptr = (int32_t *)&irq_set->data[0]; + fd_ptr[0] = intr_handle->efds[vec]; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (rc) + plt_err("Failed to set_irqs vector=0x%x rc=%d", vec, rc); + + return rc; +} + +static int +irq_init(struct plt_intr_handle *intr_handle) +{ + char irq_set_buf[MSIX_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int32_t *fd_ptr; + int len, rc; + uint32_t i; + + if (intr_handle->max_intr > PLT_MAX_RXTX_INTR_VEC_ID) { + plt_err("Max_intr=%d greater than PLT_MAX_RXTX_INTR_VEC_ID=%d", + intr_handle->max_intr, PLT_MAX_RXTX_INTR_VEC_ID); + return -ERANGE; + } + + len = sizeof(struct vfio_irq_set) + + sizeof(int32_t) * intr_handle->max_intr; + + irq_set = (struct vfio_irq_set *)irq_set_buf; + irq_set->argsz = len; + irq_set->start = 0; + irq_set->count = intr_handle->max_intr; + irq_set->flags = + VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX; + + fd_ptr = (int32_t *)&irq_set->data[0]; + for (i = 0; i < irq_set->count; i++) + fd_ptr[i] = -1; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (rc) + plt_err("Failed to set irqs vector rc=%d", rc); + + return rc; +} + +int +dev_irqs_disable(struct plt_intr_handle *intr_handle) +{ + /* Clear max_intr to indicate re-init next time */ + intr_handle->max_intr = 0; + return plt_intr_disable(intr_handle); +} + +int +dev_irq_register(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + struct plt_intr_handle tmp_handle; + int rc; + + /* If no max_intr read from VFIO */ + if (intr_handle->max_intr == 0) { + irq_get_info(intr_handle); + irq_init(intr_handle); + } + + if (vec > intr_handle->max_intr) { + plt_err("Vector=%d greater than max_intr=%d", vec, + intr_handle->max_intr); + return -EINVAL; + } + + tmp_handle = *intr_handle; + /* Create new eventfd for interrupt vector */ + tmp_handle.fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (tmp_handle.fd == -1) + return -ENODEV; + + /* Register vector interrupt callback */ + rc = plt_intr_callback_register(&tmp_handle, cb, data); + if (rc) { + plt_err("Failed to register vector:0x%x irq callback.", vec); + return rc; + } + + intr_handle->efds[vec] = tmp_handle.fd; + intr_handle->nb_efd = + (vec > intr_handle->nb_efd) ? vec : intr_handle->nb_efd; + if ((intr_handle->nb_efd + 1) > intr_handle->max_intr) + intr_handle->max_intr = intr_handle->nb_efd + 1; + + plt_base_dbg("Enable vector:0x%x for vfio (efds: %d, max:%d)", vec, + intr_handle->nb_efd, intr_handle->max_intr); + + /* Enable MSIX vectors to VFIO */ + return irq_config(intr_handle, vec); +} + +void +dev_irq_unregister(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + struct plt_intr_handle tmp_handle; + uint8_t retries = 5; /* 5 ms */ + int rc; + + if (vec > intr_handle->max_intr) { + plt_err("Error unregistering MSI-X interrupts vec:%d > %d", vec, + intr_handle->max_intr); + return; + } + + tmp_handle = *intr_handle; + tmp_handle.fd = intr_handle->efds[vec]; + if (tmp_handle.fd == -1) + return; + + do { + /* Un-register callback func from platform lib */ + rc = plt_intr_callback_unregister(&tmp_handle, cb, data); + /* Retry only if -EAGAIN */ + if (rc != -EAGAIN) + break; + plt_delay_ms(1); + retries--; + } while (retries); + + if (rc < 0) { + plt_err("Error unregistering MSI-X vec %d cb, rc=%d", vec, rc); + return; + } + + plt_base_dbg("Disable vector:0x%x for vfio (efds: %d, max:%d)", vec, + intr_handle->nb_efd, intr_handle->max_intr); + + if (intr_handle->efds[vec] != -1) + close(intr_handle->efds[vec]); + /* Disable MSIX vectors from VFIO */ + intr_handle->efds[vec] = -1; + irq_config(intr_handle, vec); +} + +#else + +int +dev_irq_register(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + PLT_SET_USED(intr_handle); + PLT_SET_USED(cb); + PLT_SET_USED(data); + PLT_SET_USED(vec); + + return -ENOTSUP; +} + +void +dev_irq_unregister(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + PLT_SET_USED(intr_handle); + PLT_SET_USED(cb); + PLT_SET_USED(data); + PLT_SET_USED(vec); +} + +int +dev_irqs_disable(struct plt_intr_handle *intr_handle) +{ + PLT_SET_USED(intr_handle); + + return -ENOTSUP; +} + +#endif /* __linux__ */ diff --git a/drivers/common/cnxk/roc_platform.c b/drivers/common/cnxk/roc_platform.c index 9f0c53e..ee1a28b 100644 --- a/drivers/common/cnxk/roc_platform.c +++ b/drivers/common/cnxk/roc_platform.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include + #include "roc_api.h" int @@ -24,3 +26,5 @@ roc_plt_init(void) roc_model_init(mz->addr); return 0; } + +RTE_LOG_REGISTER(cnxk_logtype_base, pmd.cnxk.base, NOTICE); diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 1d06435..0fe0c18 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -132,12 +132,23 @@ #define plt_strlcpy rte_strlcpy /* Log */ +extern int cnxk_logtype_base; #define plt_err(fmt, args...) \ RTE_LOG(ERR, PMD, "%s():%u " fmt "\n", __func__, __LINE__, ##args) #define plt_info(fmt, args...) RTE_LOG(INFO, PMD, fmt "\n", ##args) #define plt_warn(fmt, args...) RTE_LOG(WARNING, PMD, fmt "\n", ##args) #define plt_print(fmt, args...) RTE_LOG(INFO, PMD, fmt "\n", ##args) +/** + * Log debug message if given subsystem logging is enabled. + */ +#define plt_dbg(subsystem, fmt, args...) \ + rte_log(RTE_LOG_DEBUG, cnxk_logtype_##subsystem, \ + "[%s] %s():%u " fmt "\n", #subsystem, __func__, __LINE__, \ + ##args) + +#define plt_base_dbg(fmt, ...) plt_dbg(base, fmt, ##__VA_ARGS__) + #ifdef __cplusplus #define CNXK_PCI_ID(subsystem_dev, dev) \ { \ diff --git a/drivers/common/cnxk/roc_priv.h b/drivers/common/cnxk/roc_priv.h index 9c905d4..cd87035 100644 --- a/drivers/common/cnxk/roc_priv.h +++ b/drivers/common/cnxk/roc_priv.h @@ -8,4 +8,7 @@ /* Utils */ #include "roc_util_priv.h" +/* Dev */ +#include "roc_dev_priv.h" + #endif /* _ROC_PRIV_H_ */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 1798b48..7102704 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -1,6 +1,7 @@ INTERNAL { global: + cnxk_logtype_base; roc_error_msg_get; roc_model; roc_plt_init;