From patchwork Fri Sep 17 15:42:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99229 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DF0C1A0C43; Fri, 17 Sep 2021 17:42:39 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0B50C41169; Fri, 17 Sep 2021 17:42:37 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 3DDA2410E9 for ; Fri, 17 Sep 2021 17:42:34 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872663" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872663" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992053" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:32 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:17 +0000 Message-Id: <20210917154227.737554-2-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 01/11] dma/ioat: add device probe and removal functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the basic device probe/remove skeleton code and initial documentation for new IOAT DMA driver. Maintainers update is also included in this patch. Signed-off-by: Conor Walsh Reviewed-by: Kevin Laatz Reviewed-by: Chengwen Feng --- MAINTAINERS | 6 +++ doc/guides/dmadevs/index.rst | 2 + doc/guides/dmadevs/ioat.rst | 64 ++++++++++++++++++++++++ doc/guides/rel_notes/release_21_11.rst | 7 +-- drivers/dma/ioat/ioat_dmadev.c | 69 ++++++++++++++++++++++++++ drivers/dma/ioat/ioat_hw_defs.h | 35 +++++++++++++ drivers/dma/ioat/ioat_internal.h | 20 ++++++++ drivers/dma/ioat/meson.build | 7 +++ drivers/dma/ioat/version.map | 3 ++ drivers/dma/meson.build | 1 + 10 files changed, 211 insertions(+), 3 deletions(-) create mode 100644 doc/guides/dmadevs/ioat.rst create mode 100644 drivers/dma/ioat/ioat_dmadev.c create mode 100644 drivers/dma/ioat/ioat_hw_defs.h create mode 100644 drivers/dma/ioat/ioat_internal.h create mode 100644 drivers/dma/ioat/meson.build create mode 100644 drivers/dma/ioat/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 9cb59b831d..70993d23e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1209,6 +1209,12 @@ M: Kevin Laatz F: drivers/dma/idxd/ F: doc/guides/dmadevs/idxd.rst +Intel IOAT - EXPERIMENTAL +M: Bruce Richardson +M: Conor Walsh +F: drivers/dma/ioat/ +F: doc/guides/dmadevs/ioat.rst + RegEx Drivers ------------- diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst index 5d4abf880e..c59f4b5c92 100644 --- a/doc/guides/dmadevs/index.rst +++ b/doc/guides/dmadevs/index.rst @@ -12,3 +12,5 @@ an application through DMA API. :numbered: idxd + ioat + diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst new file mode 100644 index 0000000000..45a2e65d70 --- /dev/null +++ b/doc/guides/dmadevs/ioat.rst @@ -0,0 +1,64 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2021 Intel Corporation. + +.. include:: + +IOAT DMA Device Driver +======================= + +The ``ioat`` dmadev driver provides a poll-mode driver (PMD) for Intel\ +|reg| QuickData Technology which is part of part of Intel\ |reg| I/O +Acceleration Technology (`Intel I/OAT +`_). +This PMD, when used on supported hardware, allows data copies, for example, +cloning packet data, to be accelerated by IOAT hardware rather than having to +be done by software, freeing up CPU cycles for other tasks. + +Hardware Requirements +---------------------- + +The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the +presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma`` +will show all the DMA devices on the system, IOAT devices are included in this +list. For Intel\ |reg| IOAT devices, the hardware will often be listed as +"Crystal Beach DMA", or "CBDMA" or on some newer systems '0b00' due to the +absence of pci-id database entries for them at this point. + +Compilation +------------ + +For builds using ``meson`` and ``ninja``, the driver will be built when the +target platform is x86-based. No additional compilation steps are necessary. + +Device Setup +------------- + +Intel\ |reg| IOAT devices will need to be bound to a suitable DPDK-supported +user-space IO driver such as ``vfio-pci`` in order to be used by DPDK. + +The ``dpdk-devbind.py`` script can be used to view the state of the devices using:: + + $ dpdk-devbind.py --status-dev dma + +The ``dpdk-devbind.py`` script can also be used to bind devices to a suitable driver. +For example:: + + $ dpdk-devbind.py -b vfio-pci 00:01.0 00:01.1 + +Device Probing and Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For devices bound to a suitable DPDK-supported driver (``vfio-pci``), the HW +devices will be found as part of the device scan done at application +initialization time without the need to pass parameters to the application. + +If the application does not require all the devices available an allowlist can +be used in the same way that other DPDK devices use them. + +For example:: + + $ dpdk-test -a + +Once probed successfully, the device will appear as a ``dmadev``, that is a +"DMA device type" inside DPDK, and can be accessed using APIs from the +``rte_dmadev`` library. diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index c0bfd9c1ba..4d2b7bde1b 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -92,10 +92,11 @@ New Features * Device allocation and it's multi-process support. * Control and data plane functions. -* **Added IDXD dmadev driver implementation.** +* **Added Intel dmadev driver implementations.** - The IDXD dmadev driver provide device drivers for the Intel DSA devices. - This device driver can be used through the generic dmadev API. + The IDXD and IOAT dmadev drivers provide device drivers for Intel DSA + and IOAT devices. These device drivers can be used through the generic + dmadev API. Removed Items diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c new file mode 100644 index 0000000000..f3491d45b1 --- /dev/null +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#include +#include + +#include "ioat_internal.h" + +static struct rte_pci_driver ioat_pmd_drv; + +RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO); + +#define IOAT_PMD_NAME dmadev_ioat +#define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME) + +/* Probe DMA device. */ +static int +ioat_dmadev_probe(struct rte_pci_driver *drv, struct rte_pci_device *dev) +{ + char name[32]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + IOAT_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); + + dev->device.driver = &drv->driver; + return 0; +} + +/* Remove DMA device. */ +static int +ioat_dmadev_remove(struct rte_pci_device *dev) +{ + char name[32]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + + IOAT_PMD_INFO("Closing %s on NUMA node %d", + name, dev->device.numa_node); + + return 0; +} + +static const struct rte_pci_id pci_id_ioat_map[] = { + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_SKX) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX0) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX1) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX2) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX3) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX4) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX5) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX6) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX7) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXE) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXF) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_ICX) }, + { .vendor_id = 0, /* sentinel */ }, +}; + +static struct rte_pci_driver ioat_pmd_drv = { + .id_table = pci_id_ioat_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, + .probe = ioat_dmadev_probe, + .remove = ioat_dmadev_remove, +}; + +RTE_PMD_REGISTER_PCI(IOAT_PMD_NAME, ioat_pmd_drv); +RTE_PMD_REGISTER_PCI_TABLE(IOAT_PMD_NAME, pci_id_ioat_map); +RTE_PMD_REGISTER_KMOD_DEP(IOAT_PMD_NAME, "* igb_uio | uio_pci_generic | vfio-pci"); diff --git a/drivers/dma/ioat/ioat_hw_defs.h b/drivers/dma/ioat/ioat_hw_defs.h new file mode 100644 index 0000000000..eeabba41ef --- /dev/null +++ b/drivers/dma/ioat/ioat_hw_defs.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#ifndef IOAT_HW_DEFS_H +#define IOAT_HW_DEFS_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +#define IOAT_VER_3_0 0x30 +#define IOAT_VER_3_3 0x33 + +#define IOAT_VENDOR_ID 0x8086 +#define IOAT_DEVICE_ID_SKX 0x2021 +#define IOAT_DEVICE_ID_BDX0 0x6f20 +#define IOAT_DEVICE_ID_BDX1 0x6f21 +#define IOAT_DEVICE_ID_BDX2 0x6f22 +#define IOAT_DEVICE_ID_BDX3 0x6f23 +#define IOAT_DEVICE_ID_BDX4 0x6f24 +#define IOAT_DEVICE_ID_BDX5 0x6f25 +#define IOAT_DEVICE_ID_BDX6 0x6f26 +#define IOAT_DEVICE_ID_BDX7 0x6f27 +#define IOAT_DEVICE_ID_BDXE 0x6f2E +#define IOAT_DEVICE_ID_BDXF 0x6f2F +#define IOAT_DEVICE_ID_ICX 0x0b00 + +#ifdef __cplusplus +} +#endif + +#endif /* IOAT_HW_DEFS_H */ diff --git a/drivers/dma/ioat/ioat_internal.h b/drivers/dma/ioat/ioat_internal.h new file mode 100644 index 0000000000..f1ec12a919 --- /dev/null +++ b/drivers/dma/ioat/ioat_internal.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Intel Corporation + */ + +#ifndef _IOAT_INTERNAL_H_ +#define _IOAT_INTERNAL_H_ + +#include "ioat_hw_defs.h" + +extern int ioat_pmd_logtype; + +#define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ + ioat_pmd_logtype, "IOAT: %s(): " fmt "\n", __func__, ##args) + +#define IOAT_PMD_DEBUG(fmt, args...) IOAT_PMD_LOG(DEBUG, fmt, ## args) +#define IOAT_PMD_INFO(fmt, args...) IOAT_PMD_LOG(INFO, fmt, ## args) +#define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) +#define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) + +#endif /* _IOAT_INTERNAL_H_ */ diff --git a/drivers/dma/ioat/meson.build b/drivers/dma/ioat/meson.build new file mode 100644 index 0000000000..d67fac96fb --- /dev/null +++ b/drivers/dma/ioat/meson.build @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2021 Intel Corporation + +build = dpdk_conf.has('RTE_ARCH_X86') +reason = 'only supported on x86' +sources = files('ioat_dmadev.c') +deps += ['bus_pci', 'dmadev'] diff --git a/drivers/dma/ioat/version.map b/drivers/dma/ioat/version.map new file mode 100644 index 0000000000..c2e0723b4c --- /dev/null +++ b/drivers/dma/ioat/version.map @@ -0,0 +1,3 @@ +DPDK_22 { + local: *; +}; diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build index 411be7a240..a69418ce9b 100644 --- a/drivers/dma/meson.build +++ b/drivers/dma/meson.build @@ -3,6 +3,7 @@ drivers = [ 'idxd', + 'ioat', 'skeleton', ] std_deps = ['dmadev'] From patchwork Fri Sep 17 15:42:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99230 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 67C3CA0C43; Fri, 17 Sep 2021 17:42:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 20E5841175; Fri, 17 Sep 2021 17:42:39 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 085A241168 for ; Fri, 17 Sep 2021 17:42:35 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872671" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872671" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992059" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:34 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:18 +0000 Message-Id: <20210917154227.737554-3-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 02/11] dma/ioat: create dmadev instances on PCI probe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When a suitable device is found during the PCI probe, create a dmadev instance for each channel. Internal structures and HW definitions required for device creation are also included. Signed-off-by: Conor Walsh Reviewed-by: Kevin Laatz --- drivers/dma/ioat/ioat_dmadev.c | 119 ++++++++++++++++++++++++++++++- drivers/dma/ioat/ioat_hw_defs.h | 45 ++++++++++++ drivers/dma/ioat/ioat_internal.h | 24 +++++++ 3 files changed, 186 insertions(+), 2 deletions(-) diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index f3491d45b1..b815d30bcf 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -4,6 +4,7 @@ #include #include +#include #include "ioat_internal.h" @@ -14,6 +15,120 @@ RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO); #define IOAT_PMD_NAME dmadev_ioat #define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME) +/* Create a DMA device. */ +static int +ioat_dmadev_create(const char *name, struct rte_pci_device *dev) +{ + static const struct rte_dma_dev_ops ioat_dmadev_ops = { }; + + struct rte_dma_dev *dmadev = NULL; + struct ioat_dmadev *ioat = NULL; + int retry = 0; + + if (!name) { + IOAT_PMD_ERR("Invalid name of the device!"); + return -EINVAL; + } + + /* Allocate device structure. */ + dmadev = rte_dma_pmd_allocate(name, dev->device.numa_node, + sizeof(dmadev->dev_private)); + if (dmadev == NULL) { + IOAT_PMD_ERR("Unable to allocate dma device"); + return -ENOMEM; + } + + dmadev->device = &dev->device; + + dmadev->data->dev_private = rte_malloc_socket(NULL, sizeof(*ioat), + 0, dmadev->device->numa_node); + dmadev->dev_private = dmadev->data->dev_private; + + dmadev->dev_ops = &ioat_dmadev_ops; + + ioat = dmadev->data->dev_private; + ioat->dmadev = dmadev; + ioat->regs = dev->mem_resource[0].addr; + ioat->doorbell = &ioat->regs->dmacount; + ioat->qcfg.nb_desc = 0; + ioat->desc_ring = NULL; + + /* Do device initialization - reset and set error behaviour. */ + if (ioat->regs->chancnt != 1) + IOAT_PMD_WARN("%s: Channel count == %d\n", __func__, + ioat->regs->chancnt); + + /* Locked by someone else. */ + if (ioat->regs->chanctrl & IOAT_CHANCTRL_CHANNEL_IN_USE) { + IOAT_PMD_WARN("%s: Channel appears locked\n", __func__); + ioat->regs->chanctrl = 0; + } + + /* clear any previous errors */ + if (ioat->regs->chanerr != 0) { + uint32_t val = ioat->regs->chanerr; + ioat->regs->chanerr = val; + } + + ioat->regs->chancmd = IOAT_CHANCMD_SUSPEND; + rte_delay_ms(1); + ioat->regs->chancmd = IOAT_CHANCMD_RESET; + rte_delay_ms(1); + while (ioat->regs->chancmd & IOAT_CHANCMD_RESET) { + ioat->regs->chainaddr = 0; + rte_delay_ms(1); + if (++retry >= 200) { + IOAT_PMD_ERR("%s: cannot reset device. CHANCMD=%#"PRIx8 + ", CHANSTS=%#"PRIx64", CHANERR=%#"PRIx32"\n", + __func__, + ioat->regs->chancmd, + ioat->regs->chansts, + ioat->regs->chanerr); + return -EIO; + } + } + ioat->regs->chanctrl = IOAT_CHANCTRL_ANY_ERR_ABORT_EN | + IOAT_CHANCTRL_ERR_COMPLETION_EN; + + return 0; + +} + +/* Destroy a DMA device. */ +static int +ioat_dmadev_destroy(const char *name) +{ + struct rte_dma_dev *dev; + struct ioat_dmadev *ioat; + int ret; + + if (!name) { + IOAT_PMD_ERR("Invalid device name"); + return -EINVAL; + } + + dev = &rte_dma_devices[rte_dma_get_dev_id(name)]; + if (!dev) { + IOAT_PMD_ERR("Invalid device name (%s)", name); + return -EINVAL; + } + + ioat = dev->dev_private; + if (!ioat) { + IOAT_PMD_ERR("Error getting dev_private"); + return -EINVAL; + } + + dev->dev_private = NULL; + rte_free(ioat->desc_ring); + + ret = rte_dma_pmd_release(name); + if (ret) + IOAT_PMD_DEBUG("Device cleanup failed"); + + return 0; +} + /* Probe DMA device. */ static int ioat_dmadev_probe(struct rte_pci_driver *drv, struct rte_pci_device *dev) @@ -24,7 +139,7 @@ ioat_dmadev_probe(struct rte_pci_driver *drv, struct rte_pci_device *dev) IOAT_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); dev->device.driver = &drv->driver; - return 0; + return ioat_dmadev_create(name, dev); } /* Remove DMA device. */ @@ -38,7 +153,7 @@ ioat_dmadev_remove(struct rte_pci_device *dev) IOAT_PMD_INFO("Closing %s on NUMA node %d", name, dev->device.numa_node); - return 0; + return ioat_dmadev_destroy(name); } static const struct rte_pci_id pci_id_ioat_map[] = { diff --git a/drivers/dma/ioat/ioat_hw_defs.h b/drivers/dma/ioat/ioat_hw_defs.h index eeabba41ef..73bdf548b3 100644 --- a/drivers/dma/ioat/ioat_hw_defs.h +++ b/drivers/dma/ioat/ioat_hw_defs.h @@ -11,6 +11,8 @@ extern "C" { #include +#define IOAT_PCI_CHANERR_INT_OFFSET 0x180 + #define IOAT_VER_3_0 0x30 #define IOAT_VER_3_3 0x33 @@ -28,6 +30,49 @@ extern "C" { #define IOAT_DEVICE_ID_BDXF 0x6f2F #define IOAT_DEVICE_ID_ICX 0x0b00 +#define IOAT_COMP_UPDATE_SHIFT 3 +#define IOAT_CMD_OP_SHIFT 24 + +/* DMA Channel Registers */ +#define IOAT_CHANCTRL_CHANNEL_PRIORITY_MASK 0xF000 +#define IOAT_CHANCTRL_COMPL_DCA_EN 0x0200 +#define IOAT_CHANCTRL_CHANNEL_IN_USE 0x0100 +#define IOAT_CHANCTRL_DESCRIPTOR_ADDR_SNOOP_CONTROL 0x0020 +#define IOAT_CHANCTRL_ERR_INT_EN 0x0010 +#define IOAT_CHANCTRL_ANY_ERR_ABORT_EN 0x0008 +#define IOAT_CHANCTRL_ERR_COMPLETION_EN 0x0004 +#define IOAT_CHANCTRL_INT_REARM 0x0001 + +struct ioat_registers { + uint8_t chancnt; + uint8_t xfercap; + uint8_t genctrl; + uint8_t intrctrl; + uint32_t attnstatus; + uint8_t cbver; /* 0x08 */ + uint8_t reserved4[0x3]; /* 0x09 */ + uint16_t intrdelay; /* 0x0C */ + uint16_t cs_status; /* 0x0E */ + uint32_t dmacapability; /* 0x10 */ + uint8_t reserved5[0x6C]; /* 0x14 */ + uint16_t chanctrl; /* 0x80 */ + uint8_t reserved6[0x2]; /* 0x82 */ + uint8_t chancmd; /* 0x84 */ + uint8_t reserved3[1]; /* 0x85 */ + uint16_t dmacount; /* 0x86 */ + uint64_t chansts; /* 0x88 */ + uint64_t chainaddr; /* 0x90 */ + uint64_t chancmp; /* 0x98 */ + uint8_t reserved2[0x8]; /* 0xA0 */ + uint32_t chanerr; /* 0xA8 */ + uint32_t chanerrmask; /* 0xAC */ +} __rte_packed; + +#define IOAT_CHANCMD_RESET 0x20 +#define IOAT_CHANCMD_SUSPEND 0x04 + +#define IOAT_CHANCMP_ALIGN 8 /* CHANCMP address must be 64-bit aligned */ + #ifdef __cplusplus } #endif diff --git a/drivers/dma/ioat/ioat_internal.h b/drivers/dma/ioat/ioat_internal.h index f1ec12a919..a4e323f360 100644 --- a/drivers/dma/ioat/ioat_internal.h +++ b/drivers/dma/ioat/ioat_internal.h @@ -7,6 +7,30 @@ #include "ioat_hw_defs.h" +struct ioat_dmadev { + struct rte_dma_dev *dmadev; + struct rte_dma_vchan_conf qcfg; + struct rte_dma_stats stats; + + volatile uint16_t *doorbell __rte_cache_aligned; + phys_addr_t status_addr; + phys_addr_t ring_addr; + + struct ioat_dma_hw_desc *desc_ring; + + unsigned short next_read; + unsigned short next_write; + unsigned short last_write; /* Used to compute submitted count. */ + unsigned short offset; /* Used after a device recovery when counts -> 0. */ + unsigned int failure; /* Used to store chanerr for error handling. */ + + /* To report completions, the device will write status back here. */ + volatile uint64_t status __rte_cache_aligned; + + /* Pointer to the register bar. */ + volatile struct ioat_registers *regs; +}; + extern int ioat_pmd_logtype; #define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ From patchwork Fri Sep 17 15:42:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99231 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E448FA0C43; Fri, 17 Sep 2021 17:42:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 23A2F41181; Fri, 17 Sep 2021 17:42:40 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 0D87941173 for ; Fri, 17 Sep 2021 17:42:37 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872682" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872682" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992067" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:35 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:19 +0000 Message-Id: <20210917154227.737554-4-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 03/11] dma/ioat: add datapath structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add data structures required for the data path of IOAT devices. Signed-off-by: Conor Walsh Signed-off-by: Bruce Richardson Reviewed-by: Kevin Laatz --- drivers/dma/ioat/ioat_dmadev.c | 61 ++++++++- drivers/dma/ioat/ioat_hw_defs.h | 214 ++++++++++++++++++++++++++++++++ 2 files changed, 274 insertions(+), 1 deletion(-) diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index b815d30bcf..370be3ec72 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -15,11 +15,70 @@ RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO); #define IOAT_PMD_NAME dmadev_ioat #define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME) +/* Dump DMA device info. */ +static int +ioat_dev_dump(const struct rte_dma_dev *dev, FILE *f) +{ + struct ioat_dmadev *ioat = dev->dev_private; + uint64_t chansts_masked = ioat->regs->chansts & IOAT_CHANSTS_STATUS; + uint32_t chanerr = ioat->regs->chanerr; + uint64_t mask = (ioat->qcfg.nb_desc - 1); + fprintf(f, "========= IOAT =========\n"); + fprintf(f, " Channel status: %s [0x%"PRIx64"]\n", + chansts_readable[chansts_masked], chansts_masked); + fprintf(f, " ChainADDR: 0x%"PRIu64"\n", ioat->regs->chainaddr); + if (chanerr == 0) { + fprintf(f, " No Channel Errors\n"); + } else { + fprintf(f, " ChanERR: 0x%"PRIu32"\n", chanerr); + if (chanerr & IOAT_CHANERR_INVALID_SRC_ADDR_MASK) + fprintf(f, " Invalid Source Address\n"); + if (chanerr & IOAT_CHANERR_INVALID_DST_ADDR_MASK) + fprintf(f, " Invalid Destination Address\n"); + if (chanerr & IOAT_CHANERR_INVALID_LENGTH_MASK) + fprintf(f, " Invalid Descriptor Length\n"); + if (chanerr & IOAT_CHANERR_DESCRIPTOR_READ_ERROR_MASK) + fprintf(f, " Descriptor Read Error\n"); + if ((chanerr & ~(IOAT_CHANERR_INVALID_SRC_ADDR_MASK | + IOAT_CHANERR_INVALID_DST_ADDR_MASK | + IOAT_CHANERR_INVALID_LENGTH_MASK | + IOAT_CHANERR_DESCRIPTOR_READ_ERROR_MASK)) != 0) + fprintf(f, " Unknown Error(s)\n"); + } + fprintf(f, "== Private Data ==\n"); + fprintf(f, " Config: { ring_size: %u }\n", ioat->qcfg.nb_desc); + fprintf(f, " Status: 0x%"PRIx64"\n", ioat->status); + fprintf(f, " Status IOVA: 0x%"PRIx64"\n", ioat->status_addr); + fprintf(f, " Status ADDR: %p\n", &ioat->status); + fprintf(f, " Ring IOVA: 0x%"PRIx64"\n", ioat->ring_addr); + fprintf(f, " Ring ADDR: 0x%"PRIx64"\n", ioat->desc_ring[0].next-64); + fprintf(f, " Next write: %"PRIu16"\n", ioat->next_write); + fprintf(f, " Next read: %"PRIu16"\n", ioat->next_read); + struct ioat_dma_hw_desc *desc_ring = &ioat->desc_ring[(ioat->next_write - 1) & mask]; + fprintf(f, " Last Descriptor Written {\n"); + fprintf(f, " Size: %"PRIu32"\n", desc_ring->size); + fprintf(f, " Control: 0x%"PRIx32"\n", desc_ring->u.control_raw); + fprintf(f, " Src: 0x%"PRIx64"\n", desc_ring->src_addr); + fprintf(f, " Dest: 0x%"PRIx64"\n", desc_ring->dest_addr); + fprintf(f, " Next: 0x%"PRIx64"\n", desc_ring->next); + fprintf(f, " }\n"); + fprintf(f, " Next Descriptor {\n"); + fprintf(f, " Size: %"PRIu32"\n", ioat->desc_ring[ioat->next_read & mask].size); + fprintf(f, " Src: 0x%"PRIx64"\n", ioat->desc_ring[ioat->next_read & mask].src_addr); + fprintf(f, " Dest: 0x%"PRIx64"\n", ioat->desc_ring[ioat->next_read & mask].dest_addr); + fprintf(f, " Next: 0x%"PRIx64"\n", ioat->desc_ring[ioat->next_read & mask].next); + fprintf(f, " }\n"); + + return 0; +} + /* Create a DMA device. */ static int ioat_dmadev_create(const char *name, struct rte_pci_device *dev) { - static const struct rte_dma_dev_ops ioat_dmadev_ops = { }; + static const struct rte_dma_dev_ops ioat_dmadev_ops = { + .dev_dump = ioat_dev_dump, + }; struct rte_dma_dev *dmadev = NULL; struct ioat_dmadev *ioat = NULL; diff --git a/drivers/dma/ioat/ioat_hw_defs.h b/drivers/dma/ioat/ioat_hw_defs.h index 73bdf548b3..1f47209e60 100644 --- a/drivers/dma/ioat/ioat_hw_defs.h +++ b/drivers/dma/ioat/ioat_hw_defs.h @@ -43,6 +43,14 @@ extern "C" { #define IOAT_CHANCTRL_ERR_COMPLETION_EN 0x0004 #define IOAT_CHANCTRL_INT_REARM 0x0001 +/* DMA Channel Capabilities */ +#define IOAT_DMACAP_PB (1 << 0) +#define IOAT_DMACAP_DCA (1 << 4) +#define IOAT_DMACAP_BFILL (1 << 6) +#define IOAT_DMACAP_XOR (1 << 8) +#define IOAT_DMACAP_PQ (1 << 9) +#define IOAT_DMACAP_DMA_DIF (1 << 10) + struct ioat_registers { uint8_t chancnt; uint8_t xfercap; @@ -71,8 +79,214 @@ struct ioat_registers { #define IOAT_CHANCMD_RESET 0x20 #define IOAT_CHANCMD_SUSPEND 0x04 +#define IOAT_CHANSTS_STATUS 0x7ULL +#define IOAT_CHANSTS_ACTIVE 0x0 +#define IOAT_CHANSTS_IDLE 0x1 +#define IOAT_CHANSTS_SUSPENDED 0x2 +#define IOAT_CHANSTS_HALTED 0x3 +#define IOAT_CHANSTS_ARMED 0x4 + +#define IOAT_CHANERR_INVALID_SRC_ADDR_MASK (1 << 0) +#define IOAT_CHANERR_INVALID_DST_ADDR_MASK (1 << 1) +#define IOAT_CHANERR_DESCRIPTOR_READ_ERROR_MASK (1 << 8) +#define IOAT_CHANERR_INVALID_LENGTH_MASK (1 << 10) + +const char *chansts_readable[] = { + "ACTIVE", /* 0x0 */ + "IDLE", /* 0x1 */ + "SUSPENDED", /* 0x2 */ + "HALTED", /* 0x3 */ + "ARMED" /* 0x4 */ +}; + +#define IOAT_CHANSTS_UNAFFILIATED_ERROR 0x8ULL +#define IOAT_CHANSTS_SOFT_ERROR 0x10ULL + +#define IOAT_CHANSTS_COMPLETED_DESCRIPTOR_MASK (~0x3FULL) + #define IOAT_CHANCMP_ALIGN 8 /* CHANCMP address must be 64-bit aligned */ +struct ioat_dma_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t null: 1; + uint32_t src_page_break: 1; + uint32_t dest_page_break: 1; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t reserved: 13; +#define IOAT_OP_COPY 0x00 + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t dest_addr; + uint64_t next; + uint64_t reserved; + uint64_t reserved2; + uint64_t user1; + uint64_t user2; +}; + +struct ioat_fill_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t reserved: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t reserved2: 2; + uint32_t dest_page_break: 1; + uint32_t bundle: 1; + uint32_t reserved3: 15; +#define IOAT_OP_FILL 0x01 + uint32_t op: 8; + } control; + } u; + uint64_t src_data; + uint64_t dest_addr; + uint64_t next; + uint64_t reserved; + uint64_t next_dest_addr; + uint64_t user1; + uint64_t user2; +}; + +struct ioat_xor_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t src_count: 3; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t reserved: 13; +#define IOAT_OP_XOR 0x87 +#define IOAT_OP_XOR_VAL 0x88 + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t dest_addr; + uint64_t next; + uint64_t src_addr2; + uint64_t src_addr3; + uint64_t src_addr4; + uint64_t src_addr5; +}; + +struct ioat_xor_ext_hw_desc { + uint64_t src_addr6; + uint64_t src_addr7; + uint64_t src_addr8; + uint64_t next; + uint64_t reserved[4]; +}; + +struct ioat_pq_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t src_count: 3; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t p_disable: 1; + uint32_t q_disable: 1; + uint32_t reserved: 11; +#define IOAT_OP_PQ 0x89 +#define IOAT_OP_PQ_VAL 0x8a + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t p_addr; + uint64_t next; + uint64_t src_addr2; + uint64_t src_addr3; + uint8_t coef[8]; + uint64_t q_addr; +}; + +struct ioat_pq_ext_hw_desc { + uint64_t src_addr4; + uint64_t src_addr5; + uint64_t src_addr6; + uint64_t next; + uint64_t src_addr7; + uint64_t src_addr8; + uint64_t reserved[2]; +}; + +struct ioat_pq_update_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t src_cnt: 3; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t p_disable: 1; + uint32_t q_disable: 1; + uint32_t reserved: 3; + uint32_t coef: 8; +#define IOAT_OP_PQ_UP 0x8b + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t p_addr; + uint64_t next; + uint64_t src_addr2; + uint64_t p_src; + uint64_t q_src; + uint64_t q_addr; +}; + +union ioat_hw_desc { + struct ioat_dma_hw_desc dma; + struct ioat_fill_hw_desc fill; + struct ioat_xor_hw_desc xor_desc; + struct ioat_xor_ext_hw_desc xor_ext; + struct ioat_pq_hw_desc pq; + struct ioat_pq_ext_hw_desc pq_ext; + struct ioat_pq_update_hw_desc pq_update; +}; + +#define GENSTS_DEV_STATE_MASK 0x03 +#define CMDSTATUS_ACTIVE_SHIFT 31 +#define CMDSTATUS_ACTIVE_MASK (1 << 31) +#define CMDSTATUS_ERR_MASK 0xFF + #ifdef __cplusplus } #endif From patchwork Fri Sep 17 15:42:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99232 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 21C57A0C43; Fri, 17 Sep 2021 17:42:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 96CA541180; Fri, 17 Sep 2021 17:42:44 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id C325E4117E for ; Fri, 17 Sep 2021 17:42:39 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872694" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872694" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992081" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:37 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:20 +0000 Message-Id: <20210917154227.737554-5-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 04/11] dma/ioat: add configuration functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add functions for device configuration. The info_get and close functions are included here also. info_get can be useful for checking successful configuration and close is used by the dmadev api when releasing a configured device. Signed-off-by: Conor Walsh Reviewed-by: Kevin Laatz --- doc/guides/dmadevs/ioat.rst | 24 +++++++++ drivers/dma/ioat/ioat_dmadev.c | 93 ++++++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+) diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst index 45a2e65d70..6092599fe4 100644 --- a/doc/guides/dmadevs/ioat.rst +++ b/doc/guides/dmadevs/ioat.rst @@ -62,3 +62,27 @@ For example:: Once probed successfully, the device will appear as a ``dmadev``, that is a "DMA device type" inside DPDK, and can be accessed using APIs from the ``rte_dmadev`` library. + +Device Configuration +~~~~~~~~~~~~~~~~~~~~~ + +Configuring an IOAT dmadev device is done using the ``rte_dma_configure()`` +and ``rte_dma_vchan_setup()`` APIs. ``rte_dma_configure()`` uses the structure +``rte_dma_conf`` to configure an IOAT device, within this struct the number +of virtual DMA channels for the device is set and silent mode can be enabled. +Each IOAT device can only use 1 virtual DMA channel and silent mode is not +supported so these will always be set to ``1`` and ``false``. +``rte_dma_vchan_setup()`` uses the structure ``rte_dma_vchan_conf`` to setup +a virtual DMA channel within this struct the transfer direction and ring size +are set. Generally for an onboard Intel\ |reg| IOAT device the transfer direction +will be set to ``RTE_DMA_DIR_MEM_TO_MEM`` to copy from memory to memory (more info +available in the dmadev API guide). The ring size must be a power of two, between +64 and 4096. + +The following code shows how the device is configured in ``test_dmadev.c``: + +.. literalinclude:: ../../../app/test/test_dmadev.c + :language: c + :start-after: Setup of the dmadev device. 8< + :end-before: >8 End of setup of the dmadev device. + :dedent: 1 diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index 370be3ec72..d58022e258 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -12,9 +12,98 @@ static struct rte_pci_driver ioat_pmd_drv; RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO); +#define DESC_SZ sizeof(struct ioat_dma_hw_desc) + #define IOAT_PMD_NAME dmadev_ioat #define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME) +/* Configure a device. */ +static int +ioat_dev_configure(struct rte_dma_dev *dev __rte_unused, const struct rte_dma_conf *dev_conf, + uint32_t conf_sz) +{ + if (sizeof(struct rte_dma_conf) != conf_sz) + return -EINVAL; + + if (dev_conf->nb_vchans != 1) + return -EINVAL; + + return 0; +} + +/* Setup a virtual channel for IOAT, only 1 vchan is supported. */ +static int +ioat_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan __rte_unused, + const struct rte_dma_vchan_conf *qconf, uint32_t qconf_sz) +{ + struct ioat_dmadev *ioat = dev->dev_private; + uint16_t max_desc = qconf->nb_desc; + int i; + + if (sizeof(struct rte_dma_vchan_conf) != qconf_sz) + return -EINVAL; + + ioat->qcfg = *qconf; + + if (!rte_is_power_of_2(max_desc)) { + max_desc = rte_align32pow2(max_desc); + IOAT_PMD_DEBUG("DMA dev %u using %u descriptors", dev->data->dev_id, max_desc); + ioat->qcfg.nb_desc = max_desc; + } + + /* In case we are reconfiguring a device, free any existing memory. */ + rte_free(ioat->desc_ring); + + ioat->desc_ring = rte_zmalloc(NULL, sizeof(*ioat->desc_ring) * max_desc, 0); + if (ioat->desc_ring == NULL) + return -ENOMEM; + + ioat->ring_addr = rte_mem_virt2iova(ioat->desc_ring); + + ioat->status_addr = rte_mem_virt2iova(ioat) + offsetof(struct ioat_dmadev, status); + + /* Ensure all counters are reset, if reconfiguring/restarting device. */ + ioat->next_read = 0; + ioat->next_write = 0; + ioat->last_write = 0; + ioat->offset = 0; + ioat->failure = 0; + + /* Configure descriptor ring - each one points to next. */ + for (i = 0; i < ioat->qcfg.nb_desc; i++) { + ioat->desc_ring[i].next = ioat->ring_addr + + (((i + 1) % ioat->qcfg.nb_desc) * DESC_SZ); + } + + return 0; +} + +/* Get device information of a device. */ +static int +ioat_dev_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *info, uint32_t size) +{ + struct ioat_dmadev *ioat = dev->dev_private; + + if (size < sizeof(*info)) + return -EINVAL; + info->dev_capa = RTE_DMA_CAPA_MEM_TO_MEM | + RTE_DMA_CAPA_OPS_COPY | + RTE_DMA_CAPA_OPS_FILL; + info->max_vchans = 1; + info->min_desc = 32; + info->max_desc = 4096; + info->nb_vchans = (ioat->desc_ring == NULL); + return 0; +} + +/* Close a configured device. */ +static int +ioat_dev_close(struct rte_dma_dev *dev) +{ + RTE_SET_USED(dev); + return 0; +} + /* Dump DMA device info. */ static int ioat_dev_dump(const struct rte_dma_dev *dev, FILE *f) @@ -77,7 +166,11 @@ static int ioat_dmadev_create(const char *name, struct rte_pci_device *dev) { static const struct rte_dma_dev_ops ioat_dmadev_ops = { + .dev_close = ioat_dev_close, + .dev_configure = ioat_dev_configure, .dev_dump = ioat_dev_dump, + .dev_info_get = ioat_dev_info_get, + .vchan_setup = ioat_vchan_setup, }; struct rte_dma_dev *dmadev = NULL; From patchwork Fri Sep 17 15:42:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99233 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D8C14A0C43; Fri, 17 Sep 2021 17:43:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A87414116A; Fri, 17 Sep 2021 17:42:45 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 972E741173 for ; Fri, 17 Sep 2021 17:42:41 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872701" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872701" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992094" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:39 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:21 +0000 Message-Id: <20210917154227.737554-6-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 05/11] dma/ioat: add start and stop functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add start, stop and recover functions for IOAT devices. Signed-off-by: Conor Walsh Signed-off-by: Bruce Richardson Reviewed-by: Kevin Laatz --- doc/guides/dmadevs/ioat.rst | 3 ++ drivers/dma/ioat/ioat_dmadev.c | 92 ++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+) diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst index 6092599fe4..a64d67bf89 100644 --- a/doc/guides/dmadevs/ioat.rst +++ b/doc/guides/dmadevs/ioat.rst @@ -86,3 +86,6 @@ The following code shows how the device is configured in ``test_dmadev.c``: :start-after: Setup of the dmadev device. 8< :end-before: >8 End of setup of the dmadev device. :dedent: 1 + +Once configured, the device can then be made ready for use by calling the +``rte_dma_start()`` API. diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index d58022e258..a47567ca66 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -78,6 +78,96 @@ ioat_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan __rte_unused, return 0; } +/* Recover IOAT device. */ +static inline int +__ioat_recover(struct ioat_dmadev *ioat) +{ + uint32_t chanerr, retry = 0; + uint16_t mask = ioat->qcfg.nb_desc - 1; + + /* Clear any channel errors. Reading and writing to chanerr does this. */ + chanerr = ioat->regs->chanerr; + ioat->regs->chanerr = chanerr; + + /* Reset Channel. */ + ioat->regs->chancmd = IOAT_CHANCMD_RESET; + + /* Write new chain address to trigger state change. */ + ioat->regs->chainaddr = ioat->desc_ring[(ioat->next_read - 1) & mask].next; + /* Ensure channel control and status addr are correct. */ + ioat->regs->chanctrl = IOAT_CHANCTRL_ANY_ERR_ABORT_EN | + IOAT_CHANCTRL_ERR_COMPLETION_EN; + ioat->regs->chancmp = ioat->status_addr; + + /* Allow HW time to move to the ARMED state. */ + do { + rte_pause(); + retry++; + } while (ioat->regs->chansts != IOAT_CHANSTS_ARMED && retry < 200); + + /* Exit as failure if device is still HALTED. */ + if (ioat->regs->chansts != IOAT_CHANSTS_ARMED) + return -1; + + /* Store next write as offset as recover will move HW and SW ring out of sync. */ + ioat->offset = ioat->next_read; + + /* Prime status register with previous address. */ + ioat->status = ioat->desc_ring[(ioat->next_read - 2) & mask].next; + + return 0; +} + +/* Start a configured device. */ +static int +ioat_dev_start(struct rte_dma_dev *dev) +{ + struct ioat_dmadev *ioat = dev->dev_private; + + if (ioat->qcfg.nb_desc == 0 || ioat->desc_ring == NULL) + return -EBUSY; + + /* Inform hardware of where the descriptor ring is. */ + ioat->regs->chainaddr = ioat->ring_addr; + /* Inform hardware of where to write the status/completions. */ + ioat->regs->chancmp = ioat->status_addr; + + /* Prime the status register to be set to the last element. */ + ioat->status = ioat->ring_addr + ((ioat->qcfg.nb_desc - 1) * DESC_SZ); + + printf("IOAT.status: %s [0x%"PRIx64"]\n", + chansts_readable[ioat->status & IOAT_CHANSTS_STATUS], + ioat->status); + + if ((ioat->regs->chansts & IOAT_CHANSTS_STATUS) == IOAT_CHANSTS_HALTED) { + IOAT_PMD_WARN("Device HALTED on start, attempting to recover\n"); + if (__ioat_recover(ioat) != 0) { + IOAT_PMD_ERR("Device couldn't be recovered"); + return -1; + } + } + + return 0; +} + +/* Stop a configured device. */ +static int +ioat_dev_stop(struct rte_dma_dev *dev) +{ + struct ioat_dmadev *ioat = dev->dev_private; + uint32_t retry = 0; + + ioat->regs->chancmd = IOAT_CHANCMD_SUSPEND; + + do { + rte_pause(); + retry++; + } while ((ioat->regs->chansts & IOAT_CHANSTS_STATUS) != IOAT_CHANSTS_SUSPENDED + && retry < 200); + + return ((ioat->regs->chansts & IOAT_CHANSTS_STATUS) == IOAT_CHANSTS_SUSPENDED) ? 0 : -1; +} + /* Get device information of a device. */ static int ioat_dev_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *info, uint32_t size) @@ -170,6 +260,8 @@ ioat_dmadev_create(const char *name, struct rte_pci_device *dev) .dev_configure = ioat_dev_configure, .dev_dump = ioat_dev_dump, .dev_info_get = ioat_dev_info_get, + .dev_start = ioat_dev_start, + .dev_stop = ioat_dev_stop, .vchan_setup = ioat_vchan_setup, }; From patchwork Fri Sep 17 15:42:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99234 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A369DA0C43; Fri, 17 Sep 2021 17:43:07 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BEC6D411A3; Fri, 17 Sep 2021 17:42:46 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id BE74741101 for ; Fri, 17 Sep 2021 17:42:43 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872711" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872711" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992116" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:41 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:22 +0000 Message-Id: <20210917154227.737554-7-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 06/11] dma/ioat: add data path job submission functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add data path functions for enqueuing and submitting operations to IOAT devices. Signed-off-by: Conor Walsh Reviewed-by: Kevin Laatz Reviewed-by: Chengwen Feng --- doc/guides/dmadevs/ioat.rst | 54 ++++++++++++++++++++ drivers/dma/ioat/ioat_dmadev.c | 92 ++++++++++++++++++++++++++++++++++ 2 files changed, 146 insertions(+) diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst index a64d67bf89..2464207e20 100644 --- a/doc/guides/dmadevs/ioat.rst +++ b/doc/guides/dmadevs/ioat.rst @@ -89,3 +89,57 @@ The following code shows how the device is configured in ``test_dmadev.c``: Once configured, the device can then be made ready for use by calling the ``rte_dma_start()`` API. + +Performing Data Copies +~~~~~~~~~~~~~~~~~~~~~~~ + +To perform data copies using IOAT dmadev devices, the functions +``rte_dma_copy()`` and ``rte_dma_submit()`` should be used. Alternatively +``rte_dma_copy()`` can be called with the ``RTE_DMA_OP_FLAG_SUBMIT`` flag +set. + +The ``rte_dma_copy()`` function enqueues a single copy to the +device ring for copying at a later point. The parameters to the function +include the device ID of the desired device, the virtual DMA channel required +(always 0 for IOAT), the IOVA addresses of both the source and destination +buffers, the length of the data to be copied and any operation flags. The +function will return the index of the enqueued job which can be use to +track that operation. + +While the ``rte_dma_copy()`` function enqueues a copy operation on the device +ring, the copy will not actually be performed until after the application calls +the ``rte_dma_submit()`` function. This function informs the device hardware +of the elements enqueued on the ring, and the device will begin to process them. +It is expected that, for efficiency reasons, a burst of operations will be +enqueued to the device via multiple enqueue calls between calls to the +``rte_dma_submit()`` function. If desired you can pass the +``RTE_DMA_OP_FLAG_SUBMIT`` flag when calling ``rte_dma_copy()`` and this will +tell the device to perform the enqueued operation and any unperformed operations +before it. The ``RTE_DMA_OP_FLAG_SUBMIT`` flag can be passed instead of calling +the ``rte_dma_submit()`` function for example on the last enqueue of the burst. + +The following code from demonstrates how to enqueue a burst of copies to the +device and start the hardware processing of them: + +.. code-block:: C + + for (i = 0; i < BURST_SIZE; i++) { + if (rte_dma_copy(dev_id, vchan, rte_mbuf_data_iova(srcs[i]), + rte_mbuf_data_iova(dsts[i]), COPY_LEN, 0) < 0) { + PRINT_ERR("Error with rte_dma_copy for buffer %u\n", i); + return -1; + } + } + if (rte_dma_submit(dev_id, vchan) < 0) { + PRINT_ERR("Error with performing operations\n", i); + return -1; + } + +Filling an Area of Memory +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The driver also has support for the ``fill`` operation, where an area +of memory is overwritten, or filled, with a short pattern of data. +Fill operations can be performed in much the same was as copy operations +described above, just using the ``rte_dma_fill()`` function rather +than the ``rte_dma_copy()`` function. diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index a47567ca66..edcc882d63 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "ioat_internal.h" @@ -17,6 +18,12 @@ RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO); #define IOAT_PMD_NAME dmadev_ioat #define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME) +/* IOAT operations. */ +enum rte_ioat_ops { + ioat_op_copy = 0, /* Standard DMA Operation */ + ioat_op_fill /* Block Fill */ +}; + /* Configure a device. */ static int ioat_dev_configure(struct rte_dma_dev *dev __rte_unused, const struct rte_dma_conf *dev_conf, @@ -194,6 +201,87 @@ ioat_dev_close(struct rte_dma_dev *dev) return 0; } +/* Trigger hardware to begin performing enqueued operations. */ +static inline void +__submit(struct ioat_dmadev *ioat) +{ + *ioat->doorbell = ioat->next_write - ioat->offset; + + ioat->last_write = ioat->next_write; +} + +/* External submit function wrapper. */ +static int +ioat_submit(struct rte_dma_dev *dev, uint16_t qid __rte_unused) +{ + struct ioat_dmadev *ioat = (struct ioat_dmadev *)dev->dev_private; + + __submit(ioat); + + return 0; +} + +/* Write descriptor for enqueue. */ +static inline int +__write_desc(struct rte_dma_dev *dev, uint32_t op, uint64_t src, phys_addr_t dst, + unsigned int length, uint64_t flags) +{ + struct ioat_dmadev *ioat = dev->dev_private; + uint16_t ret; + const unsigned short mask = ioat->qcfg.nb_desc - 1; + const unsigned short read = ioat->next_read; + unsigned short write = ioat->next_write; + const unsigned short space = mask + read - write; + struct ioat_dma_hw_desc *desc; + + if (space == 0) + return -ENOSPC; + + ioat->next_write = write + 1; + write &= mask; + + desc = &ioat->desc_ring[write]; + desc->size = length; + desc->u.control_raw = (uint32_t)((op << IOAT_CMD_OP_SHIFT) | + (1 << IOAT_COMP_UPDATE_SHIFT)); + + /* In IOAT the fence ensures that all operations including the current one + * are completed before moving on, DMAdev assumes that the fence ensures + * all operations before the current one are completed before starting + * the current one, so in IOAT we set the fence for the previous descriptor. + */ + if (flags & RTE_DMA_OP_FLAG_FENCE) + ioat->desc_ring[(write - 1) & mask].u.control.fence = 1; + + desc->src_addr = src; + desc->dest_addr = dst; + + rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]); + + ret = (uint16_t)(ioat->next_write - 1); + + if (flags & RTE_DMA_OP_FLAG_SUBMIT) + __submit(ioat); + + return ret; +} + +/* Enqueue a fill operation onto the ioat device. */ +static int +ioat_enqueue_fill(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint64_t pattern, + rte_iova_t dst, unsigned int length, uint64_t flags) +{ + return __write_desc(dev, ioat_op_fill, pattern, dst, length, flags); +} + +/* Enqueue a copy operation onto the ioat device. */ +static int +ioat_enqueue_copy(struct rte_dma_dev *dev, uint16_t qid __rte_unused, rte_iova_t src, + rte_iova_t dst, unsigned int length, uint64_t flags) +{ + return __write_desc(dev, ioat_op_copy, src, dst, length, flags); +} + /* Dump DMA device info. */ static int ioat_dev_dump(const struct rte_dma_dev *dev, FILE *f) @@ -290,6 +378,10 @@ ioat_dmadev_create(const char *name, struct rte_pci_device *dev) dmadev->dev_ops = &ioat_dmadev_ops; + dmadev->copy = ioat_enqueue_copy; + dmadev->fill = ioat_enqueue_fill; + dmadev->submit = ioat_submit; + ioat = dmadev->data->dev_private; ioat->dmadev = dmadev; ioat->regs = dev->mem_resource[0].addr; From patchwork Fri Sep 17 15:42:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99235 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 48BF9A0C43; Fri, 17 Sep 2021 17:43:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D46FF411AC; Fri, 17 Sep 2021 17:42:48 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 86DDB41196 for ; Fri, 17 Sep 2021 17:42:45 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872716" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872716" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992130" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:43 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:23 +0000 Message-Id: <20210917154227.737554-8-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 07/11] dma/ioat: add data path completion functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the data path functions for gathering completed operations from IOAT devices. Signed-off-by: Conor Walsh Signed-off-by: Kevin Laatz Acked-by: Bruce Richardson --- doc/guides/dmadevs/ioat.rst | 48 ++++++++++- drivers/dma/ioat/ioat_dmadev.c | 141 +++++++++++++++++++++++++++++++++ 2 files changed, 188 insertions(+), 1 deletion(-) diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst index 2464207e20..42ecf1fb57 100644 --- a/doc/guides/dmadevs/ioat.rst +++ b/doc/guides/dmadevs/ioat.rst @@ -96,7 +96,9 @@ Performing Data Copies To perform data copies using IOAT dmadev devices, the functions ``rte_dma_copy()`` and ``rte_dma_submit()`` should be used. Alternatively ``rte_dma_copy()`` can be called with the ``RTE_DMA_OP_FLAG_SUBMIT`` flag -set. +set. Once copies have been completed, the completion will be reported back when +the application calls ``rte_dma_completed()`` or +``rte_dma_completed_status()``. The ``rte_dma_copy()`` function enqueues a single copy to the device ring for copying at a later point. The parameters to the function @@ -135,6 +137,50 @@ device and start the hardware processing of them: return -1; } +To retrieve information about completed copies, the API +``rte_dma_completed()`` or ``rte_dma_completed_status ()``should be used. +``rte_dma_completed()`` will return to the application how many operations +have successfully completed up to a supplied maximum, it can also pass back the +index of the last completed operation and if a completion within the requested +batch has failed. If ``rte_dma_completed()`` passes back ``has_error`` as +``true`` then ``rte_dma_completed_status()`` should be called to retrieve +the detailed status of each completion in that batch. Once an error has occurred +``rte_dma_completed()`` will not process any more completions until +``rte_dma_completed_status()`` has been called to get the error information. +The supplied ``status`` array will contain either ``RTE_DMA_STATUS_SUCCESSFUL`` +if the operation was successful or a DMA error code if the operation failed. +The status codes supported by IOAT are: + +* ``RTE_DMA_STATUS_SUCCESSFUL``: The operation was successful. +* ``RTE_DMA_STATUS_INVALID_SRC_ADDR``: The operation failed due to an invalid source address. +* ``RTE_DMA_STATUS_INVALID_DST_ADDR``: The operation failed due to an invalid destination address. +* ``RTE_DMA_STATUS_INVALID_LENGTH``: The operation failed due to an invalid descriptor length. +* ``RTE_DMA_STATUS_DESCRIPTOR_READ_ERROR``: The device could not read the descriptor. +* ``RTE_DMA_STATUS_ERROR_UNKNOWN``: The operation failed due to an unspecified error. + +``rte_dma_completed_status()`` can be used on it's own instead of with +``rte_dma_completed()`` but calling ``rte_dma_completed_status()`` may be +slower than ``rte_dma_completed()``. + +The following code shows how to retrieve the number of successfully completed +copies within a burst and then using ``rte_dma_completed_status()`` to check +which operation failed and reset the device: + +.. code-block:: C + + enum rte_dma_status_code status[COMP_BURST_SZ]; + uint16_t count, idx, status_count; + bool error = 0; + + count = rte_dma_completed(dev_id, vchan, COMP_BURST_SZ, &idx, &error); + + if (error){ + status_count = rte_dma_completed_status(dev_id, vchan, COMP_BURST_SZ, &idx, status); + } + +After this snippet the application would use the status array to find which copy +failed and handle it appropriately. + Filling an Area of Memory ~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index edcc882d63..fc89babc4a 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -6,6 +6,7 @@ #include #include #include +#include #include "ioat_internal.h" @@ -339,6 +340,144 @@ ioat_dev_dump(const struct rte_dma_dev *dev, FILE *f) return 0; } +/* Returns the index of the last completed operation. */ +static inline uint16_t +__get_last_completed(const struct ioat_dmadev *ioat, int *state) +{ + /* Status register contains the address of the completed operation */ + uint64_t status = ioat->status; + + /* lower 3 bits indicate "transfer status" : active, idle, halted. + * We can ignore bit 0. + */ + *state = status & IOAT_CHANSTS_STATUS; + + /* If we are just after recovering from an error the address returned by + * status will be 0, in this case we return the offset - 1 as the last + * completed. If not return the status value minus the chainaddr which + * gives us an offset into the ring. Right shifting by 6 (divide by 64) + * gives the index of the completion from the HW point of view and adding + * the offset translates the ring index from HW to SW point of view. + */ + if ((status & ~IOAT_CHANSTS_STATUS) == 0) + return ioat->offset - 1; + + return (status - ioat->ring_addr) >> 6; +} + +/* Translates IOAT ChanERRs to DMA error codes. */ +static inline enum rte_dma_status_code +__translate_status_ioat_to_dma(uint32_t chanerr) +{ + if (chanerr & IOAT_CHANERR_INVALID_SRC_ADDR_MASK) + return RTE_DMA_STATUS_INVALID_SRC_ADDR; + else if (chanerr & IOAT_CHANERR_INVALID_DST_ADDR_MASK) + return RTE_DMA_STATUS_INVALID_DST_ADDR; + else if (chanerr & IOAT_CHANERR_INVALID_LENGTH_MASK) + return RTE_DMA_STATUS_INVALID_LENGTH; + else if (chanerr & IOAT_CHANERR_DESCRIPTOR_READ_ERROR_MASK) + return RTE_DMA_STATUS_DESCRIPTOR_READ_ERROR; + else + return RTE_DMA_STATUS_ERROR_UNKNOWN; +} + +/* Returns details of operations that have been completed. */ +static uint16_t +ioat_completed(struct rte_dma_dev *dev, uint16_t qid __rte_unused, const uint16_t max_ops, + uint16_t *last_idx, bool *has_error) +{ + struct ioat_dmadev *ioat = dev->dev_private; + + const unsigned short mask = (ioat->qcfg.nb_desc - 1); + const unsigned short read = ioat->next_read; + unsigned short last_completed, count; + int state, fails = 0; + + /* Do not do any work if there is an uncleared error. */ + if (ioat->failure != 0) { + *has_error = true; + *last_idx = ioat->next_read - 2; + return 0; + } + + last_completed = __get_last_completed(ioat, &state); + count = (last_completed + 1 - read) & mask; + + /* Cap count at max_ops or set as last run in batch. */ + if (count > max_ops) + count = max_ops; + + if (count == max_ops || state != IOAT_CHANSTS_HALTED) { + ioat->next_read = read + count; + *last_idx = ioat->next_read - 1; + } else { + *has_error = true; + rte_errno = EIO; + ioat->failure = ioat->regs->chanerr; + ioat->next_read = read + count + 1; + if (__ioat_recover(ioat) != 0) { + IOAT_PMD_ERR("Device HALTED and could not be recovered\n"); + ioat_dev_dump(dev, stdout); + return 0; + } + __submit(ioat); + fails++; + *last_idx = ioat->next_read - 2; + } + + return count; +} + +/* Returns detailed status information about operations that have been completed. */ +static uint16_t +ioat_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, + uint16_t max_ops, uint16_t *last_idx, enum rte_dma_status_code *status) +{ + struct ioat_dmadev *ioat = dev->dev_private; + + const unsigned short mask = (ioat->qcfg.nb_desc - 1); + const unsigned short read = ioat->next_read; + unsigned short count, last_completed; + uint64_t fails = 0; + int state, i; + + last_completed = __get_last_completed(ioat, &state); + count = (last_completed + 1 - read) & mask; + + for (i = 0; i < RTE_MIN(count + 1, max_ops); i++) + status[i] = RTE_DMA_STATUS_SUCCESSFUL; + + /* Cap count at max_ops or set as last run in batch. */ + if (count > max_ops) + count = max_ops; + + if (count == max_ops || state != IOAT_CHANSTS_HALTED) + ioat->next_read = read + count; + else { + rte_errno = EIO; + status[count] = __translate_status_ioat_to_dma(ioat->regs->chanerr); + count++; + ioat->next_read = read + count; + if (__ioat_recover(ioat) != 0) { + IOAT_PMD_ERR("Device HALTED and could not be recovered\n"); + ioat_dev_dump(dev, stdout); + return 0; + } + __submit(ioat); + fails++; + } + + if (ioat->failure > 0) { + status[0] = __translate_status_ioat_to_dma(ioat->failure); + count = RTE_MIN(count + 1, max_ops); + ioat->failure = 0; + } + + *last_idx = ioat->next_read - 1; + + return count; +} + /* Create a DMA device. */ static int ioat_dmadev_create(const char *name, struct rte_pci_device *dev) @@ -378,6 +517,8 @@ ioat_dmadev_create(const char *name, struct rte_pci_device *dev) dmadev->dev_ops = &ioat_dmadev_ops; + dmadev->completed = ioat_completed; + dmadev->completed_status = ioat_completed_status; dmadev->copy = ioat_enqueue_copy; dmadev->fill = ioat_enqueue_fill; dmadev->submit = ioat_submit; From patchwork Fri Sep 17 15:42:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99236 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9B921A0C43; Fri, 17 Sep 2021 17:43:18 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ED3AE411B5; Fri, 17 Sep 2021 17:42:49 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 5A868411AC for ; Fri, 17 Sep 2021 17:42:47 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872721" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872721" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992136" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:45 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:24 +0000 Message-Id: <20210917154227.737554-9-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 08/11] dma/ioat: add statistics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add statistic tracking for operations in IOAT. Signed-off-by: Conor Walsh Reviewed-by: Kevin Laatz Acked-by: Bruce Richardson --- doc/guides/dmadevs/ioat.rst | 23 ++++++++++++++++++ drivers/dma/ioat/ioat_dmadev.c | 43 ++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+) diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst index 42ecf1fb57..69f3c56a7a 100644 --- a/doc/guides/dmadevs/ioat.rst +++ b/doc/guides/dmadevs/ioat.rst @@ -189,3 +189,26 @@ of memory is overwritten, or filled, with a short pattern of data. Fill operations can be performed in much the same was as copy operations described above, just using the ``rte_dma_fill()`` function rather than the ``rte_dma_copy()`` function. + +Querying Device Statistics +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The statistics from the IOAT dmadev device can be got via the +``rte_dma_stats_get()`` API in the ``rte_dmadev`` library. + +The statistics returned for each IOAT device are: + +* ``submitted``: The number of operations submitted to the device. +* ``completed``: The number of operations completed by the device (both successful and failed). +* ``errors``: The number of operations that failed. + +The stats function can be used as follows: + +.. code-block:: C + + struct rte_dma_stats stats; + if (rte_dma_stats_get(dmadev_id, vchan, &stats) >= 0) { + printf("Total submitted ops: %lu", stats.submitted); + printf("Total completed ops: %lu", stats.completed); + printf("Total failed ops: %lu", stats.errors); + } diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index fc89babc4a..f596637da4 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -77,6 +77,9 @@ ioat_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan __rte_unused, ioat->offset = 0; ioat->failure = 0; + /* Reset Stats. */ + ioat->stats = (struct rte_dma_stats){0}; + /* Configure descriptor ring - each one points to next. */ for (i = 0; i < ioat->qcfg.nb_desc; i++) { ioat->desc_ring[i].next = ioat->ring_addr + @@ -208,6 +211,8 @@ __submit(struct ioat_dmadev *ioat) { *ioat->doorbell = ioat->next_write - ioat->offset; + ioat->stats.submitted += (uint16_t)(ioat->next_write - ioat->last_write); + ioat->last_write = ioat->next_write; } @@ -336,6 +341,10 @@ ioat_dev_dump(const struct rte_dma_dev *dev, FILE *f) fprintf(f, " Dest: 0x%"PRIx64"\n", ioat->desc_ring[ioat->next_read & mask].dest_addr); fprintf(f, " Next: 0x%"PRIx64"\n", ioat->desc_ring[ioat->next_read & mask].next); fprintf(f, " }\n"); + fprintf(f, " Key Stats { submitted: %"PRIu64", comp: %"PRIu64", failed: %"PRIu64" }\n", + ioat->stats.submitted, + ioat->stats.completed, + ioat->stats.errors); return 0; } @@ -425,6 +434,9 @@ ioat_completed(struct rte_dma_dev *dev, uint16_t qid __rte_unused, const uint16_ *last_idx = ioat->next_read - 2; } + ioat->stats.completed += count; + ioat->stats.errors += fails; + return count; } @@ -475,9 +487,38 @@ ioat_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, *last_idx = ioat->next_read - 1; + ioat->stats.completed += count; + ioat->stats.errors += fails; + return count; } +/* Retrieve the generic stats of a DMA device. */ +static int +ioat_stats_get(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused, + struct rte_dma_stats *rte_stats, uint32_t size) +{ + struct rte_dma_stats *stats = (&((struct ioat_dmadev *)dev->dev_private)->stats); + + if (size < sizeof(rte_stats)) + return -EINVAL; + if (rte_stats == NULL) + return -EINVAL; + + *rte_stats = *stats; + return 0; +} + +/* Reset the generic stat counters for the DMA device. */ +static int +ioat_stats_reset(struct rte_dma_dev *dev, uint16_t vchan __rte_unused) +{ + struct ioat_dmadev *ioat = dev->dev_private; + + ioat->stats = (struct rte_dma_stats){0}; + return 0; +} + /* Create a DMA device. */ static int ioat_dmadev_create(const char *name, struct rte_pci_device *dev) @@ -489,6 +530,8 @@ ioat_dmadev_create(const char *name, struct rte_pci_device *dev) .dev_info_get = ioat_dev_info_get, .dev_start = ioat_dev_start, .dev_stop = ioat_dev_stop, + .stats_get = ioat_stats_get, + .stats_reset = ioat_stats_reset, .vchan_setup = ioat_vchan_setup, }; From patchwork Fri Sep 17 15:42:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99237 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E21D4A0C43; Fri, 17 Sep 2021 17:43:23 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0EE7E411BA; Fri, 17 Sep 2021 17:42:51 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 14B94411B2 for ; Fri, 17 Sep 2021 17:42:48 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872728" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872728" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992145" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:47 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:25 +0000 Message-Id: <20210917154227.737554-10-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 09/11] dma/ioat: add support for vchan status function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support for the rte_dmadev_vchan_status API call. Signed-off-by: Conor Walsh Reviewed-by: Kevin Laatz Acked-by: Bruce Richardson --- drivers/dma/ioat/ioat_dmadev.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index f596637da4..b93289516e 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -519,6 +519,26 @@ ioat_stats_reset(struct rte_dma_dev *dev, uint16_t vchan __rte_unused) return 0; } +/* Check if the IOAT device is idle. */ +static int +ioat_vchan_status(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused, + enum rte_dma_vchan_status *status) +{ + int state = 0; + const struct ioat_dmadev *ioat = dev->dev_private; + const uint16_t mask = ioat->qcfg.nb_desc - 1; + const uint16_t last = __get_last_completed(ioat, &state); + + if (state == IOAT_CHANSTS_HALTED || state == IOAT_CHANSTS_SUSPENDED) + *status = RTE_DMA_VCHAN_HALTED_ERROR; + else if (last == ((ioat->next_write - 1) & mask)) + *status = RTE_DMA_VCHAN_IDLE; + else + *status = RTE_DMA_VCHAN_ACTIVE; + + return 0; +} + /* Create a DMA device. */ static int ioat_dmadev_create(const char *name, struct rte_pci_device *dev) @@ -532,6 +552,7 @@ ioat_dmadev_create(const char *name, struct rte_pci_device *dev) .dev_stop = ioat_dev_stop, .stats_get = ioat_stats_get, .stats_reset = ioat_stats_reset, + .vchan_status = ioat_vchan_status, .vchan_setup = ioat_vchan_setup, }; From patchwork Fri Sep 17 15:42:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99238 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D7DC9A0C43; Fri, 17 Sep 2021 17:43:29 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 93947411CA; Fri, 17 Sep 2021 17:42:53 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id A758A41182 for ; Fri, 17 Sep 2021 17:42:50 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872733" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872733" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992160" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:48 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:26 +0000 Message-Id: <20210917154227.737554-11-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 10/11] dma/ioat: add burst capacity function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adds the ability to find the remaining space in the IOAT ring. Signed-off-by: Conor Walsh Signed-off-by: Kevin Laatz Acked-by: Bruce Richardson --- drivers/dma/ioat/ioat_dmadev.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c index b93289516e..e82a7424e5 100644 --- a/drivers/dma/ioat/ioat_dmadev.c +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -493,6 +493,19 @@ ioat_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, return count; } +/* Get the remaining capacity of the ring. */ +static uint16_t +ioat_burst_capacity(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused) +{ + struct ioat_dmadev *ioat = dev->data->dev_private; + unsigned short size = ioat->qcfg.nb_desc - 1; + unsigned short read = ioat->next_read; + unsigned short write = ioat->next_write; + unsigned short space = size - (write - read); + + return space; +} + /* Retrieve the generic stats of a DMA device. */ static int ioat_stats_get(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused, @@ -544,6 +557,7 @@ static int ioat_dmadev_create(const char *name, struct rte_pci_device *dev) { static const struct rte_dma_dev_ops ioat_dmadev_ops = { + .burst_capacity = ioat_burst_capacity, .dev_close = ioat_dev_close, .dev_configure = ioat_dev_configure, .dev_dump = ioat_dev_dump, From patchwork Fri Sep 17 15:42:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Conor Walsh X-Patchwork-Id: 99239 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 50B27A0C43; Fri, 17 Sep 2021 17:43:34 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A7F91411D0; Fri, 17 Sep 2021 17:42:55 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 813C7411C5 for ; Fri, 17 Sep 2021 17:42:52 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222872736" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222872736" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:42:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="509992179" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 17 Sep 2021 08:42:50 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 17 Sep 2021 15:42:27 +0000 Message-Id: <20210917154227.737554-12-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917154227.737554-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 11/11] devbind: move ioat device ID for ICX to dmadev category X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Move Intel IOAT devices on Ice Lake systems from Misc to DMA devices. Signed-off-by: Conor Walsh Reviewed-by: Kevin Laatz Reviewed-by: Bruce Richardson --- usertools/dpdk-devbind.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index 98b698ccc0..afebc8cb62 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -69,14 +69,13 @@ network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class] baseband_devices = [acceleration_class] crypto_devices = [encryption_class, intel_processor_class] -dma_devices = [intel_idxd_spr] +dma_devices = [intel_idxd_spr, intel_ioat_icx] eventdev_devices = [cavium_sso, cavium_tim, intel_dlb, octeontx2_sso] mempool_devices = [cavium_fpa, octeontx2_npa] compress_devices = [cavium_zip] regex_devices = [octeontx2_ree] misc_devices = [cnxk_bphy, cnxk_bphy_cgx, intel_ioat_bdw, intel_ioat_skx, - intel_ioat_icx, intel_ntb_skx, intel_ntb_icx, - octeontx2_dma] + intel_ntb_skx, intel_ntb_icx, octeontx2_dma] # global dict ethernet devices present. Dictionary indexed by PCI address. # Each device within this is itself a dictionary of device properties