From patchwork Thu May 30 21:25:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 53928 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8AAB35680; Thu, 30 May 2019 23:25:46 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 384B42B9A for ; Thu, 30 May 2019 23:25:41 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 14:25:40 -0700 X-ExtLoop1: 1 Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.2]) by orsmga001.jf.intel.com with ESMTP; 30 May 2019 14:25:39 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson Date: Thu, 30 May 2019 22:25:18 +0100 Message-Id: <20190530212525.40370-2-bruce.richardson@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530212525.40370-1-bruce.richardson@intel.com> References: <20190530212525.40370-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 1/8] raw/ioat: add initial support for ioat rawdev driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add stubs for ioat rawdev driver support in DPDK, specifically: * makefile and meson build hooks * initial public header file * rawdev main C file, with probe and release functions * release note update announcing the driver * initial documentation for the new section in the rawdev doc * unit test stubs for device unit tests Signed-off-by: Bruce Richardson --- MAINTAINERS | 7 +- app/test/Makefile | 1 + app/test/meson.build | 1 + app/test/test_ioat_rawdev.c | 22 +++++ config/common_armv8a_linux | 1 + config/common_base | 5 ++ config/defconfig_arm-armv7a-linuxapp-gcc | 1 + config/defconfig_ppc_64-power8-linuxapp-gcc | 1 + doc/guides/rawdevs/index.rst | 1 + doc/guides/rawdevs/ioat_rawdev.rst | 25 ++++++ doc/guides/rel_notes/release_19_08.rst | 11 +++ drivers/raw/Makefile | 1 + drivers/raw/ioat/Makefile | 28 +++++++ drivers/raw/ioat/ioat_rawdev.c | 93 +++++++++++++++++++++ drivers/raw/ioat/meson.build | 8 ++ drivers/raw/ioat/rte_ioat_rawdev.h | 24 ++++++ drivers/raw/ioat/rte_pmd_ioat_version.map | 4 + drivers/raw/meson.build | 3 +- mk/rte.app.mk | 1 + 19 files changed, 236 insertions(+), 2 deletions(-) create mode 100644 app/test/test_ioat_rawdev.c create mode 100644 doc/guides/rawdevs/ioat_rawdev.rst create mode 100644 drivers/raw/ioat/Makefile create mode 100644 drivers/raw/ioat/ioat_rawdev.c create mode 100644 drivers/raw/ioat/meson.build create mode 100644 drivers/raw/ioat/rte_ioat_rawdev.h create mode 100644 drivers/raw/ioat/rte_pmd_ioat_version.map diff --git a/MAINTAINERS b/MAINTAINERS index 15d0829c5..b613a1e74 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1042,6 +1042,12 @@ M: Tianfei zhang F: drivers/raw/ifpga_rawdev/ F: doc/guides/rawdevs/ifpga_rawdev.rst +IOAT Rawdev +M: Bruce Richardson +F: drivers/raw/ioat/ +F: doc/guides/rawdevs/ioat_rawdev.rst +F: app/test/test_ioat_rawdev.c + NXP DPAA2 QDMA M: Nipun Gupta F: drivers/raw/dpaa2_qdma/ @@ -1052,7 +1058,6 @@ M: Nipun Gupta F: drivers/raw/dpaa2_cmdif/ F: doc/guides/rawdevs/dpaa2_cmdif.rst - Packet processing ----------------- diff --git a/app/test/Makefile b/app/test/Makefile index 68d6b4fbc..7fbdd0755 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -212,6 +212,7 @@ endif ifeq ($(CONFIG_RTE_LIBRTE_RAWDEV),y) SRCS-y += test_rawdev.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += test_ioat_rawdev.c endif SRCS-$(CONFIG_RTE_LIBRTE_KVARGS) += test_kvargs.c diff --git a/app/test/meson.build b/app/test/meson.build index 83391cef0..9867619d3 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -52,6 +52,7 @@ test_sources = files('commands.c', 'test_hash_perf.c', 'test_hash_readwrite_lf.c', 'test_interrupts.c', + 'test_ioat_rawdev.c', 'test_ipsec.c', 'test_kni.c', 'test_kvargs.c', diff --git a/app/test/test_ioat_rawdev.c b/app/test/test_ioat_rawdev.c new file mode 100644 index 000000000..bd1bb2827 --- /dev/null +++ b/app/test/test_ioat_rawdev.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include "test.h" + +#ifndef RTE_LIBRTE_PMD_IOAT_RAWDEV + +static int +test_ioat_rawdev(void) { return TEST_SKIPPED; } + +#else + +static int +test_ioat_rawdev(void) +{ + return 0; +} + +#endif /* RTE_LIBRTE_PMD_IOAT_RAWDEV */ + +REGISTER_TEST_COMMAND(ioat_rawdev_autotest, test_ioat_rawdev); diff --git a/config/common_armv8a_linux b/config/common_armv8a_linux index 72091de1c..481712ebc 100644 --- a/config/common_armv8a_linux +++ b/config/common_armv8a_linux @@ -34,5 +34,6 @@ CONFIG_RTE_ARCH_ARM64_MEMCPY=n CONFIG_RTE_LIBRTE_FM10K_PMD=n CONFIG_RTE_LIBRTE_SFC_EFX_PMD=n CONFIG_RTE_LIBRTE_AVP_PMD=n +CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV=n CONFIG_RTE_SCHED_VECTOR=n diff --git a/config/common_base b/config/common_base index 6f19ad5d2..2b8db4880 100644 --- a/config/common_base +++ b/config/common_base @@ -741,6 +741,11 @@ CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV=n # CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV=y +# +# Compile PMD for Intel IOAT raw device +# +CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV=y + # # Compile librte_ring # diff --git a/config/defconfig_arm-armv7a-linuxapp-gcc b/config/defconfig_arm-armv7a-linuxapp-gcc index c9509b274..ee158ef9d 100644 --- a/config/defconfig_arm-armv7a-linuxapp-gcc +++ b/config/defconfig_arm-armv7a-linuxapp-gcc @@ -54,3 +54,4 @@ CONFIG_RTE_LIBRTE_QEDE_PMD=n CONFIG_RTE_LIBRTE_SFC_EFX_PMD=n CONFIG_RTE_LIBRTE_AVP_PMD=n CONFIG_RTE_LIBRTE_NFP_PMD=n +CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV=n diff --git a/config/defconfig_ppc_64-power8-linuxapp-gcc b/config/defconfig_ppc_64-power8-linuxapp-gcc index 7e248b755..9f3670ec0 100644 --- a/config/defconfig_ppc_64-power8-linuxapp-gcc +++ b/config/defconfig_ppc_64-power8-linuxapp-gcc @@ -56,3 +56,4 @@ CONFIG_RTE_LIBRTE_ENIC_PMD=n CONFIG_RTE_LIBRTE_FM10K_PMD=n CONFIG_RTE_LIBRTE_SFC_EFX_PMD=n CONFIG_RTE_LIBRTE_AVP_PMD=n +CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV=n diff --git a/doc/guides/rawdevs/index.rst b/doc/guides/rawdevs/index.rst index 7c3bd9586..0a21989e4 100644 --- a/doc/guides/rawdevs/index.rst +++ b/doc/guides/rawdevs/index.rst @@ -14,3 +14,4 @@ application through rawdev API. dpaa2_cmdif dpaa2_qdma ifpga_rawdev + ioat_rawdev diff --git a/doc/guides/rawdevs/ioat_rawdev.rst b/doc/guides/rawdevs/ioat_rawdev.rst new file mode 100644 index 000000000..40ab1b466 --- /dev/null +++ b/doc/guides/rawdevs/ioat_rawdev.rst @@ -0,0 +1,25 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +.. include:: + +IOAT Rawdev Driver for Intel\ |reg| QuickData Technology +====================================================================== + +The ``ioat`` rawdev driver provides a poll-mode driver (PMD) for Intel\ |reg| +QuickData Technology, part of Intel\ |reg| I/O Acceleration Technology +`(Intel I/OAT) `_. +This PMD allows data copies, for example, cloning packet data, to be +accelerated by hardware rather than having to be done by software, freeing +up CPU cycles for other tasks. + +Compilation +------------ + +For builds done with ``make``, the driver compilation is enabled by the +``CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV`` build configuration option. This is +enabled by default in builds for x86 platforms, and disabled in other +configurations. + +For builds using ``meson`` and ``ninja``, the driver will be built when the +target platform is x86-based. diff --git a/doc/guides/rel_notes/release_19_08.rst b/doc/guides/rel_notes/release_19_08.rst index a17e7dea5..a65f3514a 100644 --- a/doc/guides/rel_notes/release_19_08.rst +++ b/doc/guides/rel_notes/release_19_08.rst @@ -1,6 +1,8 @@ .. SPDX-License-Identifier: BSD-3-Clause Copyright 2019 The DPDK contributors +.. include:: + DPDK Release 19.08 ================== @@ -54,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **Added Intel QuickData Technology PMD** + + The PMD for Intel\ |reg| QuickData Technology, part of + Intel\ |reg| I/O Acceleration Technology `(Intel I/OAT) + `_, + allows data copies to be done by hardware instead + of via software, reducing cycles spent copying large blocks of data in + applications. + Removed Items ------------- diff --git a/drivers/raw/Makefile b/drivers/raw/Makefile index 8e29b4a56..c1b85c8c7 100644 --- a/drivers/raw/Makefile +++ b/drivers/raw/Makefile @@ -10,5 +10,6 @@ DIRS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV) += dpaa2_cmdif DIRS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV) += dpaa2_qdma endif DIRS-$(CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV) += ifpga_rawdev +DIRS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat include $(RTE_SDK)/mk/rte.subdir.mk diff --git a/drivers/raw/ioat/Makefile b/drivers/raw/ioat/Makefile new file mode 100644 index 000000000..7726e310a --- /dev/null +++ b/drivers/raw/ioat/Makefile @@ -0,0 +1,28 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# library name +LIB = librte_pmd_ioat_rawdev.a + +# build flags +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) + +LDLIBS += -lrte_eal -lrte_rawdev +LDLIBS += -lrte_pci -lrte_bus_pci + +# library version +LIBABIVER := 1 + +# versioning export map +EXPORT_MAP := rte_pmd_ioat_version.map + +# library source files +SRCS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat_rawdev.c + +# export include files +SYMLINK-y-include += rte_ioat_rawdev.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c new file mode 100644 index 000000000..d9fc3091a --- /dev/null +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -0,0 +1,93 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include +#include + +#include "rte_ioat_rawdev.h" + +/* Dynamic log type identifier */ +int ioat_pmd_logtype; + +static struct rte_pci_driver ioat_pmd_drv; + +#define IOAT_VENDOR_ID 0x8086 +#define IOAT_DEVICE_ID 0x2021 + +#define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ + ioat_pmd_logtype, "%s(): " fmt "\n", __func__, ##args) + +#define IOAT_PMD_DEBUG(fmt, args...) IOAT_PMD_LOG(DEBUG, fmt, ## args) +#define IOAT_PMD_INFO(fmt, args...) IOAT_PMD_LOG(INFO, fmt, ## args) +#define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) +#define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) + +static int +ioat_rawdev_create(const char *name, struct rte_pci_device *dev) +{ + RTE_SET_USED(name); + RTE_SET_USED(dev); + return 0; +} + +static int +ioat_rawdev_destroy(const char *name) +{ + RTE_SET_USED(name); + return 0; +} + +static int +ioat_rawdev_probe(struct rte_pci_driver *drv, struct rte_pci_device *dev) +{ + char name[32]; + int ret = 0; + + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + IOAT_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); + + dev->device.driver = &drv->driver; + ret = ioat_rawdev_create(name, dev); + return ret; +} + +static int +ioat_rawdev_remove(struct rte_pci_device *dev) +{ + char name[32]; + int ret; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + + IOAT_PMD_INFO("Closing %s on NUMA node %d", + name, dev->device.numa_node); + + ret = ioat_rawdev_destroy(name); + return ret; +} + +static const struct rte_pci_id pci_id_ioat_map[] = { + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID) }, + { .vendor_id = 0, /* sentinel */ }, +}; + +static struct rte_pci_driver ioat_pmd_drv = { + .id_table = pci_id_ioat_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | + RTE_PCI_DRV_IOVA_AS_VA, + .probe = ioat_rawdev_probe, + .remove = ioat_rawdev_remove, +}; + +RTE_PMD_REGISTER_PCI(IOAT_PMD_RAWDEV_NAME, ioat_pmd_drv); +RTE_PMD_REGISTER_PCI_TABLE(IOAT_PMD_RAWDEV_NAME, pci_id_ioat_map); +RTE_PMD_REGISTER_KMOD_DEP(IOAT_PMD_RAWDEV_NAME, "* igb_uio | uio_pci_generic"); + +RTE_INIT(ioat_pmd_init_log) +{ + ioat_pmd_logtype = rte_log_register(IOAT_PMD_LOG_NAME); + if (ioat_pmd_logtype >= 0) + rte_log_set_level(ioat_pmd_logtype, RTE_LOG_INFO); +} diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build new file mode 100644 index 000000000..ba7620a68 --- /dev/null +++ b/drivers/raw/ioat/meson.build @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2019 Intel Corporation + +build = dpdk_conf.has('RTE_ARCH_X86') +sources = files('ioat_rawdev.c') +deps += ['rawdev', 'bus_pci'] + +install_headers('rte_ioat_rawdev.h') diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h b/drivers/raw/ioat/rte_ioat_rawdev.h new file mode 100644 index 000000000..e77406403 --- /dev/null +++ b/drivers/raw/ioat/rte_ioat_rawdev.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef _RTE_IOAT_RAWDEV_H_ +#define _RTE_IOAT_RAWDEV_H_ + +/** + * @file rte_ioat_rawdev.h + * + * Definitions for using the ioat rawdev device driver + * + * @warning + * @b EXPERIMENTAL: these structures and APIs may change without prior notice + */ + +/** Name of the device driver */ +#define IOAT_PMD_RAWDEV_NAME rawdev_ioat +/** String reported as the device driver name by rte_rawdev_info_get() */ +#define IOAT_PMD_RAWDEV_NAME_STR "rawdev_ioat" +/** Name used to adjust the log level for this driver */ +#define IOAT_PMD_LOG_NAME "rawdev.ioat" + +#endif diff --git a/drivers/raw/ioat/rte_pmd_ioat_version.map b/drivers/raw/ioat/rte_pmd_ioat_version.map new file mode 100644 index 000000000..9a61188cd --- /dev/null +++ b/drivers/raw/ioat/rte_pmd_ioat_version.map @@ -0,0 +1,4 @@ +DPDK_19.08 { + + local: *; +}; diff --git a/drivers/raw/meson.build b/drivers/raw/meson.build index a61cdccef..39eb54a90 100644 --- a/drivers/raw/meson.build +++ b/drivers/raw/meson.build @@ -1,7 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright 2018 NXP -drivers = ['skeleton_rawdev', 'dpaa2_cmdif', 'dpaa2_qdma', 'ifpga_rawdev'] +drivers = ['skeleton_rawdev', 'dpaa2_cmdif', 'dpaa2_qdma', + 'ifpga_rawdev', 'ioat'] std_deps = ['rawdev'] config_flag_fmt = 'RTE_LIBRTE_PMD_@0@_RAWDEV' driver_name_fmt = 'rte_pmd_@0@' diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 7c9b4b538..9232877b0 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -303,6 +303,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_IFPGA_BUS),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV) += -lrte_pmd_ifpga_rawdev _LDLIBS-$(CONFIG_RTE_LIBRTE_IPN3KE_PMD) += -lrte_pmd_ipn3ke endif # CONFIG_RTE_LIBRTE_IFPGA_BUS +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += -lrte_pmd_ioat_rawdev endif # CONFIG_RTE_LIBRTE_RAWDEV endif # !CONFIG_RTE_BUILD_SHARED_LIBS From patchwork Thu May 30 21:25:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 53929 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 417AD1B947; Thu, 30 May 2019 23:25:50 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id F225737A2 for ; Thu, 30 May 2019 23:25:43 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 14:25:42 -0700 X-ExtLoop1: 1 Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.2]) by orsmga001.jf.intel.com with ESMTP; 30 May 2019 14:25:42 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson Date: Thu, 30 May 2019 22:25:19 +0100 Message-Id: <20190530212525.40370-3-bruce.richardson@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530212525.40370-1-bruce.richardson@intel.com> References: <20190530212525.40370-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 2/8] usertools/dpdk-devbind.py: add support for IOAT devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In order to allow binding/unbinding of devices for use by the ioat_rawdev, we need to update the devbind script to add a new class of device, and add device ids for the specific HW instances. Signed-off-by: Bruce Richardson --- doc/guides/rawdevs/ioat_rawdev.rst | 11 +++++++++++ usertools/dpdk-devbind.py | 10 ++++++++++ 2 files changed, 21 insertions(+) diff --git a/doc/guides/rawdevs/ioat_rawdev.rst b/doc/guides/rawdevs/ioat_rawdev.rst index 40ab1b466..99e757498 100644 --- a/doc/guides/rawdevs/ioat_rawdev.rst +++ b/doc/guides/rawdevs/ioat_rawdev.rst @@ -23,3 +23,14 @@ configurations. For builds using ``meson`` and ``ninja``, the driver will be built when the target platform is x86-based. + +Device Setup +------------- + +The Intel\ |reg| QuickData Technology HW devices will need to be bound to a +user-space IO driver for use. The script ``dpdk-devbind.py`` script +included with DPDK can be used to view the state of the devices and to bind +them to a suitable DPDK-supported kernel driver. When querying the +status of the devices, they will appear under the category of "dma +devices", i.e. the command ``dpdk-devbind.py --status-dev dma`` can be used +to see the state of those devices alone. diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index 9e79f0d28..bd0d97df3 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -36,11 +36,17 @@ octeontx2_npa = {'Class': '08', 'Vendor': '177d', 'Device': 'a0fb,a0fc', 'SVendor': None, 'SDevice': None} +intel_ioat_bdw = {'Class': '08', 'Vendor': '8086', 'Device': '6f20,6f21,6f22,6f23,6f24,6f25,6f26,6f27,6f2e,6f2f', + 'SVendor': None, 'SDevice': None} +intel_ioat_skx = {'Class': '08', 'Vendor': '8086', 'Device': '2021', + 'SVendor': None, 'SDevice': None} + network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class] crypto_devices = [encryption_class, intel_processor_class] eventdev_devices = [cavium_sso, cavium_tim, octeontx2_sso] mempool_devices = [cavium_fpa, octeontx2_npa] compress_devices = [cavium_zip] +dma_devices = [intel_ioat_bdw, intel_ioat_skx] # global dict ethernet devices present. Dictionary indexed by PCI address. # Each device within this is itself a dictionary of device properties @@ -595,6 +601,8 @@ def show_status(): if status_dev == "compress" or status_dev == "all": show_device_status(compress_devices , "Compress") + if status_dev == "dma" or status_dev == "all": + show_device_status(dma_devices, "DMA") def parse_args(): '''Parses the command-line arguments given by the user and takes the @@ -670,6 +678,7 @@ def do_arg_actions(): get_device_details(eventdev_devices) get_device_details(mempool_devices) get_device_details(compress_devices) + get_device_details(dma_devices) show_status() @@ -690,6 +699,7 @@ def main(): get_device_details(eventdev_devices) get_device_details(mempool_devices) get_device_details(compress_devices) + get_device_details(dma_devices) do_arg_actions() if __name__ == "__main__": From patchwork Thu May 30 21:25:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 53930 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A08651B94D; Thu, 30 May 2019 23:25:53 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id C70614CA6 for ; Thu, 30 May 2019 23:25:45 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 14:25:44 -0700 X-ExtLoop1: 1 Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.2]) by orsmga001.jf.intel.com with ESMTP; 30 May 2019 14:25:44 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson Date: Thu, 30 May 2019 22:25:20 +0100 Message-Id: <20190530212525.40370-4-bruce.richardson@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530212525.40370-1-bruce.richardson@intel.com> References: <20190530212525.40370-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 3/8] raw/ioat: add register definition file X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add in the list of registers for the device. File is taken from the SPDK project: https://github.com/spdk/spdk/blob/master/include/spdk/ioat_spec.h Signed-off-by: Bruce Richardson --- drivers/raw/ioat/Makefile | 1 + drivers/raw/ioat/meson.build | 3 +- drivers/raw/ioat/rte_ioat_spec.h | 301 +++++++++++++++++++++++++++++++ 3 files changed, 304 insertions(+), 1 deletion(-) create mode 100644 drivers/raw/ioat/rte_ioat_spec.h diff --git a/drivers/raw/ioat/Makefile b/drivers/raw/ioat/Makefile index 7726e310a..1e10938f3 100644 --- a/drivers/raw/ioat/Makefile +++ b/drivers/raw/ioat/Makefile @@ -24,5 +24,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV) += ioat_rawdev.c # export include files SYMLINK-y-include += rte_ioat_rawdev.h +SYMLINK-y-include += rte_ioat_spec.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build index ba7620a68..ca23e23fc 100644 --- a/drivers/raw/ioat/meson.build +++ b/drivers/raw/ioat/meson.build @@ -5,4 +5,5 @@ build = dpdk_conf.has('RTE_ARCH_X86') sources = files('ioat_rawdev.c') deps += ['rawdev', 'bus_pci'] -install_headers('rte_ioat_rawdev.h') +install_headers('rte_ioat_rawdev.h', + 'rte_ioat_spec.h') diff --git a/drivers/raw/ioat/rte_ioat_spec.h b/drivers/raw/ioat/rte_ioat_spec.h new file mode 100644 index 000000000..305e36ded --- /dev/null +++ b/drivers/raw/ioat/rte_ioat_spec.h @@ -0,0 +1,301 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) Intel Corporation + */ + +/** + * \file + * I/OAT specification definitions + * + * Taken from ioat_spec.h from SPDK project, with prefix renames and + * other minor changes. + */ + +#ifndef RTE_IOAT_SPEC_H +#define RTE_IOAT_SPEC_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +#define RTE_IOAT_PCI_CHANERR_INT_OFFSET 0x180 + +#define RTE_IOAT_INTRCTRL_MASTER_INT_EN 0x01 + +#define RTE_IOAT_VER_3_0 0x30 +#define RTE_IOAT_VER_3_3 0x33 + +/* DMA Channel Registers */ +#define RTE_IOAT_CHANCTRL_CHANNEL_PRIORITY_MASK 0xF000 +#define RTE_IOAT_CHANCTRL_COMPL_DCA_EN 0x0200 +#define RTE_IOAT_CHANCTRL_CHANNEL_IN_USE 0x0100 +#define RTE_IOAT_CHANCTRL_DESCRIPTOR_ADDR_SNOOP_CONTROL 0x0020 +#define RTE_IOAT_CHANCTRL_ERR_INT_EN 0x0010 +#define RTE_IOAT_CHANCTRL_ANY_ERR_ABORT_EN 0x0008 +#define RTE_IOAT_CHANCTRL_ERR_COMPLETION_EN 0x0004 +#define RTE_IOAT_CHANCTRL_INT_REARM 0x0001 + +/* DMA Channel Capabilities */ +#define RTE_IOAT_DMACAP_PB (1 << 0) +#define RTE_IOAT_DMACAP_DCA (1 << 4) +#define RTE_IOAT_DMACAP_BFILL (1 << 6) +#define RTE_IOAT_DMACAP_XOR (1 << 8) +#define RTE_IOAT_DMACAP_PQ (1 << 9) +#define RTE_IOAT_DMACAP_DMA_DIF (1 << 10) + +struct rte_ioat_registers { + uint8_t chancnt; + uint8_t xfercap; + uint8_t genctrl; + uint8_t intrctrl; + uint32_t attnstatus; + uint8_t cbver; /* 0x08 */ + uint8_t reserved4[0x3]; /* 0x09 */ + uint16_t intrdelay; /* 0x0C */ + uint16_t cs_status; /* 0x0E */ + uint32_t dmacapability; /* 0x10 */ + uint8_t reserved5[0x6C]; /* 0x14 */ + uint16_t chanctrl; /* 0x80 */ + uint8_t reserved6[0x2]; /* 0x82 */ + uint8_t chancmd; /* 0x84 */ + uint8_t reserved3[1]; /* 0x85 */ + uint16_t dmacount; /* 0x86 */ + uint64_t chansts; /* 0x88 */ + uint64_t chainaddr; /* 0x90 */ + uint64_t chancmp; /* 0x98 */ + uint8_t reserved2[0x8]; /* 0xA0 */ + uint32_t chanerr; /* 0xA8 */ + uint32_t chanerrmask; /* 0xAC */ +} __attribute__((packed)); + +#define RTE_IOAT_CHANCMD_RESET 0x20 +#define RTE_IOAT_CHANCMD_SUSPEND 0x04 + +#define RTE_IOAT_CHANSTS_STATUS 0x7ULL +#define RTE_IOAT_CHANSTS_ACTIVE 0x0 +#define RTE_IOAT_CHANSTS_IDLE 0x1 +#define RTE_IOAT_CHANSTS_SUSPENDED 0x2 +#define RTE_IOAT_CHANSTS_HALTED 0x3 +#define RTE_IOAT_CHANSTS_ARMED 0x4 + +#define RTE_IOAT_CHANSTS_UNAFFILIATED_ERROR 0x8ULL +#define RTE_IOAT_CHANSTS_SOFT_ERROR 0x10ULL + +#define RTE_IOAT_CHANSTS_COMPLETED_DESCRIPTOR_MASK (~0x3FULL) + +#define RTE_IOAT_CHANCMP_ALIGN 8 /* CHANCMP address must be 64-bit aligned */ + +struct rte_ioat_generic_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t reserved2: 1; + uint32_t src_page_break: 1; + uint32_t dest_page_break: 1; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t reserved: 13; + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t dest_addr; + uint64_t next; + uint64_t op_specific[4]; +}; + +struct rte_ioat_dma_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t null: 1; + uint32_t src_page_break: 1; + uint32_t dest_page_break: 1; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t reserved: 13; +#define RTE_IOAT_OP_COPY 0x00 + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t dest_addr; + uint64_t next; + uint64_t reserved; + uint64_t reserved2; + uint64_t user1; + uint64_t user2; +}; + +struct rte_ioat_fill_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t reserved: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t reserved2: 2; + uint32_t dest_page_break: 1; + uint32_t bundle: 1; + uint32_t reserved3: 15; +#define RTE_IOAT_OP_FILL 0x01 + uint32_t op: 8; + } control; + } u; + uint64_t src_data; + uint64_t dest_addr; + uint64_t next; + uint64_t reserved; + uint64_t next_dest_addr; + uint64_t user1; + uint64_t user2; +}; + +struct rte_ioat_xor_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t src_count: 3; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t reserved: 13; +#define RTE_IOAT_OP_XOR 0x87 +#define RTE_IOAT_OP_XOR_VAL 0x88 + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t dest_addr; + uint64_t next; + uint64_t src_addr2; + uint64_t src_addr3; + uint64_t src_addr4; + uint64_t src_addr5; +}; + +struct rte_ioat_xor_ext_hw_desc { + uint64_t src_addr6; + uint64_t src_addr7; + uint64_t src_addr8; + uint64_t next; + uint64_t reserved[4]; +}; + +struct rte_ioat_pq_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t src_count: 3; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t p_disable: 1; + uint32_t q_disable: 1; + uint32_t reserved: 11; +#define RTE_IOAT_OP_PQ 0x89 +#define RTE_IOAT_OP_PQ_VAL 0x8a + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t p_addr; + uint64_t next; + uint64_t src_addr2; + uint64_t src_addr3; + uint8_t coef[8]; + uint64_t q_addr; +}; + +struct rte_ioat_pq_ext_hw_desc { + uint64_t src_addr4; + uint64_t src_addr5; + uint64_t src_addr6; + uint64_t next; + uint64_t src_addr7; + uint64_t src_addr8; + uint64_t reserved[2]; +}; + +struct rte_ioat_pq_update_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t src_cnt: 3; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t p_disable: 1; + uint32_t q_disable: 1; + uint32_t reserved: 3; + uint32_t coef: 8; +#define RTE_IOAT_OP_PQ_UP 0x8b + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t p_addr; + uint64_t next; + uint64_t src_addr2; + uint64_t p_src; + uint64_t q_src; + uint64_t q_addr; +}; + +struct rte_ioat_raw_hw_desc { + uint64_t field[8]; +}; + +union rte_ioat_hw_desc { + struct rte_ioat_raw_hw_desc raw; + struct rte_ioat_generic_hw_desc generic; + struct rte_ioat_dma_hw_desc dma; + struct rte_ioat_fill_hw_desc fill; + struct rte_ioat_xor_hw_desc xor_desc; + struct rte_ioat_xor_ext_hw_desc xor_ext; + struct rte_ioat_pq_hw_desc pq; + struct rte_ioat_pq_ext_hw_desc pq_ext; + struct rte_ioat_pq_update_hw_desc pq_update; +}; + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_IOAT_SPEC_H */ From patchwork Thu May 30 21:25:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 53931 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D47C01B952; Thu, 30 May 2019 23:25:56 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 41C122C55 for ; Thu, 30 May 2019 23:25:48 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 14:25:47 -0700 X-ExtLoop1: 1 Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.2]) by orsmga001.jf.intel.com with ESMTP; 30 May 2019 14:25:47 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson Date: Thu, 30 May 2019 22:25:21 +0100 Message-Id: <20190530212525.40370-5-bruce.richardson@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530212525.40370-1-bruce.richardson@intel.com> References: <20190530212525.40370-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 4/8] raw/ioat: create device on probe and destroy on release X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the create/destroy driver functions so that we can actually allocate a rawdev and destroy it when done. No rawdev API functions are actually implemented at this point. Signed-off-by: Bruce Richardson --- doc/guides/rawdevs/ioat_rawdev.rst | 11 ++++ drivers/raw/ioat/ioat_rawdev.c | 93 +++++++++++++++++++++++++++++- drivers/raw/ioat/rte_ioat_rawdev.h | 20 +++++++ 3 files changed, 121 insertions(+), 3 deletions(-) diff --git a/doc/guides/rawdevs/ioat_rawdev.rst b/doc/guides/rawdevs/ioat_rawdev.rst index 99e757498..476b0503f 100644 --- a/doc/guides/rawdevs/ioat_rawdev.rst +++ b/doc/guides/rawdevs/ioat_rawdev.rst @@ -34,3 +34,14 @@ them to a suitable DPDK-supported kernel driver. When querying the status of the devices, they will appear under the category of "dma devices", i.e. the command ``dpdk-devbind.py --status-dev dma`` can be used to see the state of those devices alone. + +Device Probing and Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once bound to a suitable kernel device driver, the HW devices will be found +as part of the PCI scan done at application initialization time. No vdev +parameters need to be passed to create or initialize the device. + +Once probed successfully, the device will appear as a ``rawdev``, that is a +"raw device type" inside DPDK, and can be accessed using APIs from the +``rte_rawdev`` library. diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index d9fc3091a..b6964bccd 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -2,6 +2,7 @@ * Copyright(c) 2019 Intel Corporation */ +#include #include #include @@ -26,15 +27,101 @@ static struct rte_pci_driver ioat_pmd_drv; static int ioat_rawdev_create(const char *name, struct rte_pci_device *dev) { - RTE_SET_USED(name); - RTE_SET_USED(dev); + static const struct rte_rawdev_ops ioat_rawdev_ops = { + }; + + struct rte_rawdev *rawdev = NULL; + struct rte_ioat_rawdev *ioat = NULL; + int ret = 0; + int retry = 0; + + if (!name) { + IOAT_PMD_ERR("Invalid name of the device!"); + ret = -EINVAL; + goto cleanup; + } + + /* Allocate device structure */ + rawdev = rte_rawdev_pmd_allocate(name, sizeof(struct rte_ioat_rawdev), + dev->device.numa_node); + if (rawdev == NULL) { + IOAT_PMD_ERR("Unable to allocate raw device"); + ret = -EINVAL; + goto cleanup; + } + + rawdev->dev_ops = &ioat_rawdev_ops; + rawdev->device = &dev->device; + rawdev->driver_name = dev->device.driver->name; + + ioat = rawdev->dev_private; + ioat->rawdev = rawdev; + ioat->regs = dev->mem_resource[0].addr; + ioat->ring_size = 0; + ioat->desc_ring = NULL; + ioat->status_addr = rte_malloc_virt2iova(ioat) + + offsetof(struct rte_ioat_rawdev, status); + + /* do device initialization - reset and set error behaviour */ + if (ioat->regs->chancnt != 1) + IOAT_PMD_ERR("%s: Channel count == %d\n", __func__, + ioat->regs->chancnt); + + if (ioat->regs->chanctrl & 0x100) { /* locked by someone else */ + IOAT_PMD_WARN("%s: Channel appears locked\n", __func__); + ioat->regs->chanctrl = 0; + } + + ioat->regs->chancmd = RTE_IOAT_CHANCMD_SUSPEND; + rte_delay_ms(1); + ioat->regs->chancmd = RTE_IOAT_CHANCMD_RESET; + rte_delay_ms(1); + while (ioat->regs->chancmd & RTE_IOAT_CHANCMD_RESET) { + ioat->regs->chainaddr = 0; + rte_delay_ms(1); + if (++retry >= 200) { + IOAT_PMD_ERR("%s: cannot reset device. CHANCMD=0x%llx, CHANSTS=0x%llx, CHANERR=0x%llx\n", + __func__, + (unsigned long long)ioat->regs->chancmd, + (unsigned long long)ioat->regs->chansts, + (unsigned long long)ioat->regs->chanerr); + ret = -EIO; + } + } + ioat->regs->chanctrl = RTE_IOAT_CHANCTRL_ANY_ERR_ABORT_EN | + RTE_IOAT_CHANCTRL_ERR_COMPLETION_EN; + return 0; + +cleanup: + if (rawdev) + rte_rawdev_pmd_release(rawdev); + + return ret; } static int ioat_rawdev_destroy(const char *name) { - RTE_SET_USED(name); + int ret; + struct rte_rawdev *rdev; + + if (!name) { + IOAT_PMD_ERR("Invalid device name"); + return -EINVAL; + } + + rdev = rte_rawdev_pmd_get_named_dev(name); + if (!rdev) { + IOAT_PMD_ERR("Invalid device name (%s)", name); + return -EINVAL; + } + + /* rte_rawdev_close is called by pmd_release */ + ret = rte_rawdev_pmd_release(rdev); + if (ret) + IOAT_PMD_DEBUG("Device cleanup failed"); + return 0; } diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h b/drivers/raw/ioat/rte_ioat_rawdev.h index e77406403..c3216a174 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev.h +++ b/drivers/raw/ioat/rte_ioat_rawdev.h @@ -14,6 +14,9 @@ * @b EXPERIMENTAL: these structures and APIs may change without prior notice */ +#include +#include + /** Name of the device driver */ #define IOAT_PMD_RAWDEV_NAME rawdev_ioat /** String reported as the device driver name by rte_rawdev_info_get() */ @@ -21,4 +24,21 @@ /** Name used to adjust the log level for this driver */ #define IOAT_PMD_LOG_NAME "rawdev.ioat" +/** + * @internal + * Structure representing a device instance + */ +struct rte_ioat_rawdev { + struct rte_rawdev *rawdev; + volatile struct rte_ioat_registers *regs; + phys_addr_t status_addr; + phys_addr_t ring_addr; + + unsigned short ring_size; + struct rte_ioat_desc *desc_ring; + + /* to report completions, the device will write status back here */ + volatile uint64_t status __rte_cache_aligned; +}; + #endif From patchwork Thu May 30 21:25:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 53932 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 421341B95C; Thu, 30 May 2019 23:26:01 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id BD9B11B94A for ; Thu, 30 May 2019 23:25:50 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 14:25:49 -0700 X-ExtLoop1: 1 Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.2]) by orsmga001.jf.intel.com with ESMTP; 30 May 2019 14:25:49 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson Date: Thu, 30 May 2019 22:25:22 +0100 Message-Id: <20190530212525.40370-6-bruce.richardson@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530212525.40370-1-bruce.richardson@intel.com> References: <20190530212525.40370-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 5/8] raw/ioat: add device info function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add in the "info_get" function to the driver, to allow us to query the device. This allows us to have the unit test pick up the presence of supported hardware or not. Signed-off-by: Bruce Richardson --- app/test/meson.build | 3 +++ app/test/test_ioat_rawdev.c | 23 ++++++++++++++++++++ doc/guides/rawdevs/ioat_rawdev.rst | 34 ++++++++++++++++++++++++++++++ drivers/raw/ioat/ioat_rawdev.c | 11 ++++++++++ drivers/raw/ioat/rte_ioat_rawdev.h | 11 ++++++++++ 5 files changed, 82 insertions(+) diff --git a/app/test/meson.build b/app/test/meson.build index 9867619d3..9fe3ddc89 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -305,6 +305,9 @@ endif if dpdk_conf.has('RTE_LIBRTE_KNI') test_deps += 'kni' endif +if dpdk_conf.has('RTE_LIBRTE_PMD_IOAT_RAWDEV') + test_deps += 'pmd_ioat' +endif cflags = machine_args if cc.has_argument('-Wno-format-truncation') diff --git a/app/test/test_ioat_rawdev.c b/app/test/test_ioat_rawdev.c index bd1bb2827..ac1389f6e 100644 --- a/app/test/test_ioat_rawdev.c +++ b/app/test/test_ioat_rawdev.c @@ -11,9 +11,32 @@ test_ioat_rawdev(void) { return TEST_SKIPPED; } #else +#include +#include + +#include +#include +#include + static int test_ioat_rawdev(void) { + const int count = rte_rawdev_count(); + int i, found = 0; + + printf("Checking %d rawdevs\n", count); + for (i = 0; i < count && !found; i++) { + struct rte_rawdev_info info = { .dev_private = NULL }; + found = (rte_rawdev_info_get(i, &info) == 0 && + strcmp(info.driver_name, + IOAT_PMD_RAWDEV_NAME_STR) == 0); + } + + if (!found) { + printf("No IOAT rawdev found, skipping tests\n"); + return TEST_SKIPPED; + } + return 0; } diff --git a/doc/guides/rawdevs/ioat_rawdev.rst b/doc/guides/rawdevs/ioat_rawdev.rst index 476b0503f..b68cdffc3 100644 --- a/doc/guides/rawdevs/ioat_rawdev.rst +++ b/doc/guides/rawdevs/ioat_rawdev.rst @@ -45,3 +45,37 @@ parameters need to be passed to create or initialize the device. Once probed successfully, the device will appear as a ``rawdev``, that is a "raw device type" inside DPDK, and can be accessed using APIs from the ``rte_rawdev`` library. + +Using IOAT Rawdev Devices +-------------------------- + +To use the devices from an application, the rawdev API can be used, along +with definitions taken from the device-specific header file +``rte_ioat_rawdev.h``. This header is needed to get the definition of +structure parameters used by some of the rawdev APIs for IOAT rawdev +devices, as well as providing key functions for using the device for memory +copies. + +Getting Device Information +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Basic information about each rawdev device can be got using the +``rte_rawdev_info_get()`` API. For most applications, this API will be +needed to verify that the rawdev in question is of the expected type. For +example, the following code in ``test_ioat_rawdev.c`` is used to identify +the IOAT rawdev device for use for the tests: + +.. code-block:: C + + for (i = 0; i < count && !found; i++) { + struct rte_rawdev_info info = { .dev_private = NULL }; + found = (rte_rawdev_info_get(i, &info) == 0 && + strcmp(info.driver_name, + IOAT_PMD_RAWDEV_NAME_STR) == 0); + } + +When calling the ``rte_rawdev_info_get()`` API for an IOAT rawdev device, +the ``dev_private`` field in the ``rte_rawdev_info`` struct should either +be NULL, or else be set to point to a structure of type +``rte_ioat_rawdev_config``, in which case the size of the configured device +input ring will be returned in that structure. diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index b6964bccd..90bed2810 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -24,10 +24,21 @@ static struct rte_pci_driver ioat_pmd_drv; #define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) #define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) +static void +ioat_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info) +{ + struct rte_ioat_rawdev_config *cfg = dev_info; + struct rte_ioat_rawdev *ioat = dev->dev_private; + + if (cfg != NULL) + cfg->ring_size = ioat->ring_size; +} + static int ioat_rawdev_create(const char *name, struct rte_pci_device *dev) { static const struct rte_rawdev_ops ioat_rawdev_ops = { + .dev_info_get = ioat_dev_info_get, }; struct rte_rawdev *rawdev = NULL; diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h b/drivers/raw/ioat/rte_ioat_rawdev.h index c3216a174..7e0d72ca3 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev.h +++ b/drivers/raw/ioat/rte_ioat_rawdev.h @@ -24,6 +24,17 @@ /** Name used to adjust the log level for this driver */ #define IOAT_PMD_LOG_NAME "rawdev.ioat" +/** + * Configuration structure for an ioat rawdev instance + * + * This structure is to be passed as the ".dev_private" parameter when + * calling the rte_rawdev_get_info() and rte_rawdev_configure() APIs on + * an ioat rawdev instance. + */ +struct rte_ioat_rawdev_config { + unsigned short ring_size; +}; + /** * @internal * Structure representing a device instance From patchwork Thu May 30 21:25:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 53933 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DE4541B96E; Thu, 30 May 2019 23:26:03 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id B0DF84F91 for ; Thu, 30 May 2019 23:25:52 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 14:25:51 -0700 X-ExtLoop1: 1 Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.2]) by orsmga001.jf.intel.com with ESMTP; 30 May 2019 14:25:51 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson Date: Thu, 30 May 2019 22:25:23 +0100 Message-Id: <20190530212525.40370-7-bruce.richardson@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530212525.40370-1-bruce.richardson@intel.com> References: <20190530212525.40370-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 6/8] raw/ioat: add configure, start and stop functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Allow initializing a driver instance. Signed-off-by: Bruce Richardson --- app/test/test_ioat_rawdev.c | 35 +++++++++++++- doc/guides/rawdevs/ioat_rawdev.rst | 32 +++++++++++++ drivers/raw/ioat/ioat_rawdev.c | 75 ++++++++++++++++++++++++++++++ drivers/raw/ioat/rte_ioat_rawdev.h | 14 ++++++ 4 files changed, 155 insertions(+), 1 deletion(-) diff --git a/app/test/test_ioat_rawdev.c b/app/test/test_ioat_rawdev.c index ac1389f6e..36e97347c 100644 --- a/app/test/test_ioat_rawdev.c +++ b/app/test/test_ioat_rawdev.c @@ -18,6 +18,39 @@ test_ioat_rawdev(void) { return TEST_SKIPPED; } #include #include +static int +run_ioat_tests(int dev_id) +{ +#define IOAT_TEST_RINGSIZE 512 + struct rte_ioat_rawdev_config p = { .ring_size = -1 }; + struct rte_rawdev_info info = { .dev_private = &p }; + + rte_rawdev_info_get(dev_id, &info); + if (p.ring_size != 0) { + printf("Error, initial ring size is non-zero (%d)\n", + (int)p.ring_size); + return -1; + } + + p.ring_size = IOAT_TEST_RINGSIZE; + if (rte_rawdev_configure(dev_id, &info) != 0) { + printf("Error with rte_rawdev_configure()\n"); + return -1; + } + rte_rawdev_info_get(dev_id, &info); + if (p.ring_size != IOAT_TEST_RINGSIZE) { + printf("Error, ring size is not %d (%d)\n", + IOAT_TEST_RINGSIZE, (int)p.ring_size); + return -1; + } + + if (rte_rawdev_start(dev_id) != 0) { + printf("Error with rte_rawdev_start()\n"); + return -1; + } + return 0; +} + static int test_ioat_rawdev(void) { @@ -37,7 +70,7 @@ test_ioat_rawdev(void) return TEST_SKIPPED; } - return 0; + return run_ioat_tests(i); } #endif /* RTE_LIBRTE_PMD_IOAT_RAWDEV */ diff --git a/doc/guides/rawdevs/ioat_rawdev.rst b/doc/guides/rawdevs/ioat_rawdev.rst index b68cdffc3..b3fe79033 100644 --- a/doc/guides/rawdevs/ioat_rawdev.rst +++ b/doc/guides/rawdevs/ioat_rawdev.rst @@ -79,3 +79,35 @@ the ``dev_private`` field in the ``rte_rawdev_info`` struct should either be NULL, or else be set to point to a structure of type ``rte_ioat_rawdev_config``, in which case the size of the configured device input ring will be returned in that structure. + +Device Configuration +~~~~~~~~~~~~~~~~~~~~~ + +Configuring an IOAT rawdev device is done using the +``rte_rawdev_configure()`` API, which takes the same structure parameters +as the, previously referenced, ``rte_rawdev_info_get()`` API. The main +difference is that, because the parameter is used as input rather than +output, the ``dev_private`` structure element cannot be NULL, and must +point to a valid ``rte_ioat_rawdev_config`` structure, containing the ring +size to be used by the device. The ring size must be a power of two, +between 64 and 4096. + +The following code shows how the device is configured in +``test_ioat_rawdev.c``: + +.. code-block:: C + + #define IOAT_TEST_RINGSIZE 512 + struct rte_ioat_rawdev_config p = { .ring_size = -1 }; + struct rte_rawdev_info info = { .dev_private = &p }; + + /* ... */ + + p.ring_size = IOAT_TEST_RINGSIZE; + if (rte_rawdev_configure(dev_id, &info) != 0) { + printf("Error with rte_rawdev_configure()\n"); + return -1; + } + +Once configured, the device can then be made ready for use by calling the +``rte_rawdev_start()`` API. diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index 90bed2810..b4b70a1e6 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -24,6 +24,78 @@ static struct rte_pci_driver ioat_pmd_drv; #define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) #define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) +#define DESC_SZ sizeof(struct rte_ioat_desc) +#define COMPLETION_SZ sizeof(__m128i) + +static int +ioat_dev_configure(const struct rte_rawdev *dev, rte_rawdev_obj_t config) +{ + struct rte_ioat_rawdev_config *params = config; + struct rte_ioat_rawdev *ioat = dev->dev_private; + unsigned short i; + + if (dev->started) + return -EBUSY; + + if (params == NULL) + return -EINVAL; + + if (params->ring_size > 4096 || params->ring_size < 64 || + !rte_is_power_of_2(params->ring_size)) + return -EINVAL; + + ioat->ring_size = params->ring_size; + if (ioat->desc_ring != NULL) { + rte_free(ioat->desc_ring); + ioat->desc_ring = NULL; + } + + /* allocate one block of memory for both descriptors + * and completion handles. + */ + ioat->desc_ring = rte_zmalloc_socket(NULL, + (DESC_SZ + COMPLETION_SZ) * ioat->ring_size, + 0, /* alignment, default to 64Byte */ + dev->device->numa_node); + if (ioat->desc_ring == NULL) + return -ENOMEM; + ioat->hdls = (void *)&ioat->desc_ring[ioat->ring_size]; + + ioat->ring_addr = rte_malloc_virt2iova(ioat->desc_ring); + + /* configure descriptor ring - each one points to next */ + for (i = 0; i < ioat->ring_size; i++) { + ioat->desc_ring[i].next_desc_addr = ioat->ring_addr + + (((i + 1) % ioat->ring_size) * DESC_SZ); + } + + return 0; +} + +static int +ioat_dev_start(struct rte_rawdev *dev) +{ + struct rte_ioat_rawdev *ioat = dev->dev_private; + + if (ioat->ring_size == 0 || ioat->desc_ring == NULL) + return -EBUSY; + + /* inform hardware of where the descriptor ring is */ + ioat->regs->chainaddr = ioat->ring_addr; + /* inform hardware of where to write the status/completions */ + ioat->regs->chancmp = ioat->status_addr; + + /* prime the status register to be set to the last element */ + ioat->status = ioat->ring_addr + ((ioat->ring_size - 1) * DESC_SZ); + return 0; +} + +static void +ioat_dev_stop(struct rte_rawdev *dev) +{ + RTE_SET_USED(dev); +} + static void ioat_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info) { @@ -38,6 +110,9 @@ static int ioat_rawdev_create(const char *name, struct rte_pci_device *dev) { static const struct rte_rawdev_ops ioat_rawdev_ops = { + .dev_configure = ioat_dev_configure, + .dev_start = ioat_dev_start, + .dev_stop = ioat_dev_stop, .dev_info_get = ioat_dev_info_get, }; diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h b/drivers/raw/ioat/rte_ioat_rawdev.h index 7e0d72ca3..7eab216c8 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev.h +++ b/drivers/raw/ioat/rte_ioat_rawdev.h @@ -47,9 +47,23 @@ struct rte_ioat_rawdev { unsigned short ring_size; struct rte_ioat_desc *desc_ring; + __m128i *hdls; /* completion handles for returning to user */ /* to report completions, the device will write status back here */ volatile uint64_t status __rte_cache_aligned; }; +/** + * @internal + * Structure representing a descriptor for a copy operation + */ +struct rte_ioat_desc { + uint32_t xfer_size __rte_cache_aligned; + uint32_t desc_control; + phys_addr_t src_addr; /* 64 bits */ + phys_addr_t dest_addr; /* 64 bits */ + phys_addr_t next_desc_addr; /* 64 bits */ + uint64_t op_type_specific[4]; +}; + #endif From patchwork Thu May 30 21:25:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 53934 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 07A1B1B995; Thu, 30 May 2019 23:26:06 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 488781B952 for ; Thu, 30 May 2019 23:25:55 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 14:25:54 -0700 X-ExtLoop1: 1 Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.2]) by orsmga001.jf.intel.com with ESMTP; 30 May 2019 14:25:54 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson Date: Thu, 30 May 2019 22:25:24 +0100 Message-Id: <20190530212525.40370-8-bruce.richardson@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530212525.40370-1-bruce.richardson@intel.com> References: <20190530212525.40370-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 7/8] raw/ioat: add statistics functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add stats functions to track what is happening in the driver, and put unit tests to check those. Signed-off-by: Bruce Richardson --- app/test/test_ioat_rawdev.c | 38 ++++++++++++++++++++++++++ doc/guides/rawdevs/ioat_rawdev.rst | 14 ++++++++++ drivers/raw/ioat/ioat_rawdev.c | 44 ++++++++++++++++++++++++++++++ drivers/raw/ioat/rte_ioat_rawdev.h | 6 ++++ 4 files changed, 102 insertions(+) diff --git a/app/test/test_ioat_rawdev.c b/app/test/test_ioat_rawdev.c index 36e97347c..7081f3365 100644 --- a/app/test/test_ioat_rawdev.c +++ b/app/test/test_ioat_rawdev.c @@ -24,6 +24,11 @@ run_ioat_tests(int dev_id) #define IOAT_TEST_RINGSIZE 512 struct rte_ioat_rawdev_config p = { .ring_size = -1 }; struct rte_rawdev_info info = { .dev_private = &p }; + struct rte_rawdev_xstats_name *snames = NULL; + uint64_t *stats = NULL; + unsigned int *ids = NULL; + unsigned int nb_xstats; + unsigned int i; rte_rawdev_info_get(dev_id, &info); if (p.ring_size != 0) { @@ -48,6 +53,39 @@ run_ioat_tests(int dev_id) printf("Error with rte_rawdev_start()\n"); return -1; } + + /* allocate memory for xstats names and values */ + nb_xstats = rte_rawdev_xstats_names_get(dev_id, NULL, 0); + + snames = malloc(sizeof(*snames) * nb_xstats); + if (snames == NULL) { + printf("Error allocating xstat names memory\n"); + return -1; + } + rte_rawdev_xstats_names_get(dev_id, snames, nb_xstats); + + ids = malloc(sizeof(*ids) * nb_xstats); + if (ids == NULL) { + printf("Error allocating xstat ids memory\n"); + return -1; + } + for (i = 0; i < nb_xstats; i++) + ids[i] = i; + + stats = malloc(sizeof(*stats) * nb_xstats); + if (stats == NULL) { + printf("Error allocating xstat memory\n"); + return -1; + } + + rte_rawdev_xstats_get(dev_id, ids, stats, nb_xstats); + for (i = 0; i < nb_xstats; i++) + printf("%s: %"PRIu64" ", snames[i].name, stats[i]); + printf("\n"); + + free(snames); + free(stats); + free(ids); return 0; } diff --git a/doc/guides/rawdevs/ioat_rawdev.rst b/doc/guides/rawdevs/ioat_rawdev.rst index b3fe79033..47f12e95c 100644 --- a/doc/guides/rawdevs/ioat_rawdev.rst +++ b/doc/guides/rawdevs/ioat_rawdev.rst @@ -111,3 +111,17 @@ The following code shows how the device is configured in Once configured, the device can then be made ready for use by calling the ``rte_rawdev_start()`` API. + +Querying Device Statistics +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The statistics from the IOAT rawdev device can be got via the xstats +functions in the ``rte_rawdev`` library, i.e. +``rte_rawdev_xstats_names_get()``, ``rte_rawdev_xstats_get()`` and +``rte_rawdev_xstats_by_name_get``. The statistics returned for each device +instance are: + +* ``failed_enqueues`` +* ``successful_enqueues`` +* ``copies_started`` +* ``copies_completed`` diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index b4b70a1e6..09fbdbf9c 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -4,6 +4,7 @@ #include #include +#include #include #include "rte_ioat_rawdev.h" @@ -106,6 +107,47 @@ ioat_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info) cfg->ring_size = ioat->ring_size; } +static const char *xstat_names[] = { + "failed_enqueues", "successful_enqueues", + "copies_started", "copies_completed" +}; + +static int +ioat_xstats_get(const struct rte_rawdev *dev, const unsigned int ids[], + uint64_t values[], unsigned int n) +{ + const struct rte_ioat_rawdev *ioat = dev->dev_private; + unsigned int i; + + for (i = 0; i < n; i++) { + switch (ids[i]){ + case 0: values[i] = ioat->enqueue_failed; break; + case 1: values[i] = ioat->enqueued; break; + case 2: values[i] = ioat->started; break; + case 3: values[i] = ioat->completed; break; + default: values[i] = 0; break; + } + } + return n; +} + +static int +ioat_xstats_get_names(const struct rte_rawdev *dev, + struct rte_rawdev_xstats_name *names, + unsigned int size) +{ + unsigned int i; + + RTE_SET_USED(dev); + if (size < RTE_DIM(xstat_names)) + return RTE_DIM(xstat_names); + + for (i = 0; i < RTE_DIM(xstat_names); i++) + strlcpy(names[i].name, xstat_names[i], sizeof(names[i])); + + return RTE_DIM(xstat_names); +} + static int ioat_rawdev_create(const char *name, struct rte_pci_device *dev) { @@ -114,6 +156,8 @@ ioat_rawdev_create(const char *name, struct rte_pci_device *dev) .dev_start = ioat_dev_start, .dev_stop = ioat_dev_stop, .dev_info_get = ioat_dev_info_get, + .xstats_get = ioat_xstats_get, + .xstats_get_names = ioat_xstats_get_names, }; struct rte_rawdev *rawdev = NULL; diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h b/drivers/raw/ioat/rte_ioat_rawdev.h index 7eab216c8..abe5ba298 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev.h +++ b/drivers/raw/ioat/rte_ioat_rawdev.h @@ -49,6 +49,12 @@ struct rte_ioat_rawdev { struct rte_ioat_desc *desc_ring; __m128i *hdls; /* completion handles for returning to user */ + /* some statistics for tracking, if added/changed update xstats fns*/ + uint64_t enqueue_failed __rte_cache_aligned; + uint64_t enqueued; + uint64_t started; + uint64_t completed; + /* to report completions, the device will write status back here */ volatile uint64_t status __rte_cache_aligned; }; From patchwork Thu May 30 21:25:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 53935 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0DF9A1B99A; Thu, 30 May 2019 23:26:08 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id B2C7B1B955 for ; Thu, 30 May 2019 23:25:57 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 May 2019 14:25:56 -0700 X-ExtLoop1: 1 Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.2]) by orsmga001.jf.intel.com with ESMTP; 30 May 2019 14:25:56 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson Date: Thu, 30 May 2019 22:25:25 +0100 Message-Id: <20190530212525.40370-9-bruce.richardson@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190530212525.40370-1-bruce.richardson@intel.com> References: <20190530212525.40370-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 8/8] raw/ioat: add local API to perform copies X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add local APIs to trigger data copies, and retrieve handle values once those copies are completed. Included are unit tests to validate the data is copies correctly. Signed-off-by: Bruce Richardson --- app/test/test_ioat_rawdev.c | 159 ++++++++++++++++++++++++++++- doc/guides/rawdevs/ioat_rawdev.rst | 100 ++++++++++++++++++ drivers/raw/ioat/rte_ioat_rawdev.h | 155 +++++++++++++++++++++++++++- 3 files changed, 410 insertions(+), 4 deletions(-) diff --git a/app/test/test_ioat_rawdev.c b/app/test/test_ioat_rawdev.c index 7081f3365..f2240adec 100644 --- a/app/test/test_ioat_rawdev.c +++ b/app/test/test_ioat_rawdev.c @@ -18,6 +18,131 @@ test_ioat_rawdev(void) { return TEST_SKIPPED; } #include #include +static struct rte_mempool *pool; + +static int +test_enqueue_copies(int dev_id) +{ + const unsigned int length = 1024; + unsigned int i; + + do { + struct rte_mbuf *src, *dst; + char *src_data, *dst_data; + struct rte_mbuf *completed[2] = {0}; + + /* test doing a single copy */ + src = rte_pktmbuf_alloc(pool); + dst = rte_pktmbuf_alloc(pool); + src->data_len = src->pkt_len = length; + dst->data_len = dst->pkt_len = length; + src_data = rte_pktmbuf_mtod(src, char *); + dst_data = rte_pktmbuf_mtod(dst, char *); + + for (i = 0; i < length; i++) + src_data[i] = rand() & 0xFF; + + if (rte_ioat_enqueue_copy(dev_id, + src->buf_iova + src->data_off, + dst->buf_iova + dst->data_off, + length, + (uintptr_t)src, + (uintptr_t)dst, + 0 /* no fence */) != 1) { + printf("Error with rte_ioat_enqueue_copy\n"); + return -1; + } + rte_ioat_do_copies(dev_id); + usleep(10); + + if (rte_ioat_completed_copies(dev_id, 1, (void *)&completed[0], + (void *)&completed[1]) != 1) { + printf("Error with rte_ioat_completed_copies\n"); + return -1; + } + if (completed[0] != src || completed[1] != dst) { + printf("Error with completions: got (%p, %p), not (%p,%p)\n", + completed[0], completed[1], src, dst); + return -1; + } + + for (i = 0; i < length; i++) + if (dst_data[i] != src_data[i]) { + printf("Data mismatch at char %u\n", i); + return -1; + } + rte_pktmbuf_free(src); + rte_pktmbuf_free(dst); + } while(0); + + /* test doing multiple copies */ + do { + struct rte_mbuf *srcs[32], *dsts[32]; + struct rte_mbuf *completed_src[64]; + struct rte_mbuf *completed_dst[64]; + unsigned int j; + + for (i = 0; i < RTE_DIM(srcs); i++) { + char *src_data; + + srcs[i] = rte_pktmbuf_alloc(pool); + dsts[i] = rte_pktmbuf_alloc(pool); + srcs[i]->data_len = srcs[i]->pkt_len = length; + dsts[i]->data_len = dsts[i]->pkt_len = length; + src_data = rte_pktmbuf_mtod(srcs[i], char *); + + for (j = 0; j < length; j++) + src_data[j] = rand() & 0xFF; + + if (rte_ioat_enqueue_copy(dev_id, + srcs[i]->buf_iova + srcs[i]->data_off, + dsts[i]->buf_iova + dsts[i]->data_off, + length, + (uintptr_t)srcs[i], + (uintptr_t)dsts[i], + 0 /* nofence */) != 1) { + printf("Error with rte_ioat_enqueue_copy for buffer %u\n", + i); + return -1; + } + } + rte_ioat_do_copies(dev_id); + usleep(100); + + if (rte_ioat_completed_copies(dev_id, 64, (void *)completed_src, + (void *)completed_dst) != RTE_DIM(srcs)) { + printf("Error with rte_ioat_completed_copies\n"); + return -1; + } + for (i = 0; i < RTE_DIM(srcs); i++) { + char *src_data, *dst_data; + + if (completed_src[i] != srcs[i]) { + printf("Error with source pointer %u\n", i); + return -1; + } + if (completed_dst[i] != dsts[i]) { + printf("Error with dest pointer %u\n", i); + return -1; + } + + src_data = rte_pktmbuf_mtod(srcs[i], char *); + dst_data = rte_pktmbuf_mtod(dsts[i], char *); + for (j = 0; j < length; j++) + if (src_data[j] != dst_data[j]) { + printf("Error with copy of packet %u, byte %u\n", + i, j); + return -1; + } + rte_pktmbuf_free(srcs[i]); + rte_pktmbuf_free(dsts[i]); + } + + } while(0); + + return 0; +} + static int run_ioat_tests(int dev_id) { @@ -54,6 +179,17 @@ run_ioat_tests(int dev_id) return -1; } + pool = rte_pktmbuf_pool_create("TEST_IOAT_POOL", + 256, /* n == num elements */ + 32, /* cache size */ + 0, /* priv size */ + 2048, /* data room size */ + info.socket_id); + if (pool == NULL) { + printf("Error with mempool creation\n"); + return -1; + } + /* allocate memory for xstats names and values */ nb_xstats = rte_rawdev_xstats_names_get(dev_id, NULL, 0); @@ -78,15 +214,32 @@ run_ioat_tests(int dev_id) return -1; } - rte_rawdev_xstats_get(dev_id, ids, stats, nb_xstats); - for (i = 0; i < nb_xstats; i++) - printf("%s: %"PRIu64" ", snames[i].name, stats[i]); + /* run the test cases */ + for (i = 0; i < 100; i++) { + unsigned int j; + + if (test_enqueue_copies(dev_id) != 0) + goto err; + + rte_rawdev_xstats_get(dev_id, ids, stats, nb_xstats); + for (j = 0; j < nb_xstats; j++) + printf("%s: %"PRIu64" ", snames[j].name, stats[j]); + printf("\r"); + } printf("\n"); + rte_mempool_free(pool); free(snames); free(stats); free(ids); return 0; + +err: + rte_mempool_free(pool); + free(snames); + free(stats); + free(ids); + return -1; } static int diff --git a/doc/guides/rawdevs/ioat_rawdev.rst b/doc/guides/rawdevs/ioat_rawdev.rst index 47f12e95c..3ed12c964 100644 --- a/doc/guides/rawdevs/ioat_rawdev.rst +++ b/doc/guides/rawdevs/ioat_rawdev.rst @@ -112,6 +112,106 @@ The following code shows how the device is configured in Once configured, the device can then be made ready for use by calling the ``rte_rawdev_start()`` API. +Performing Data Copies +~~~~~~~~~~~~~~~~~~~~~~~ + +To perform data copies using IOAT rawdev devices, the functions +``rte_ioat_enqueue_copy()`` and ``rte_ioat_do_copies()`` should be used. +Once copies have been completed, the completion will be reported back when +the application calls ``rte_ioat_completed_copies()``. + +The ``rte_ioat_enqueue_copy()`` function enqueues a single copy to the +device ring for copying at a later point. The parameters to that function +include the physical addresses of both the source and destination buffers, +as well as two "handles" to be returned to the user when the copy is +completed. These handles can be arbitrary values, but two are provided so +that the library can track handles for both source and destination on +behalf of the user, e.g. virtual addresses for the buffers, or mbuf +pointers if packet data is being copied. + +While the ``rte_ioat_enqueue_copy()`` function enqueues a copy operation on +the device ring, the copy will not actually be performed until after the +application calls the ``rte_ioat_do_copies()`` function. This function +informs the device hardware of the elements enqueued on the ring, and the +device will begin to process them. It is expected that, for efficiency +reasons, a burst of operations will be enqueued to the device via multiple +enqueue calls between calls to the ``rte_ioat_do_copies()`` function. + +The following code from ``test_ioat_rawdev.c`` demonstrates how to enqueue +a burst of copies to the device and start the hardware processing of them: + +.. code-block:: C + + struct rte_mbuf *srcs[32], *dsts[32]; + unsigned int j; + + for (i = 0; i < RTE_DIM(srcs); i++) { + char *src_data; + + srcs[i] = rte_pktmbuf_alloc(pool); + dsts[i] = rte_pktmbuf_alloc(pool); + srcs[i]->data_len = srcs[i]->pkt_len = length; + dsts[i]->data_len = dsts[i]->pkt_len = length; + src_data = rte_pktmbuf_mtod(srcs[i], char *); + + for (j = 0; j < length; j++) + src_data[j] = rand() & 0xFF; + + if (rte_ioat_enqueue_copy(dev_id, + srcs[i]->buf_iova + srcs[i]->data_off, + dsts[i]->buf_iova + dsts[i]->data_off, + length, + (uintptr_t)srcs[i], + (uintptr_t)dsts[i], + 0 /* nofence */) != 1) { + printf("Error with rte_ioat_enqueue_copy for buffer %u\n", + i); + return -1; + } + } + rte_ioat_do_copies(dev_id); + +To retrieve information about completed copies, the API +``rte_ioat_completed_copies()`` should be used. This API will return to the +application a set of completion handles passed in when the relevant copies +were enqueued. + +The following code from ``test_ioat_rawdev.c`` shows the test code +retrieving information about the completed copies and validating the data +is correct before freeing the data buffers using the returned handles: + +.. code-block:: C + + if (rte_ioat_completed_copies(dev_id, 64, (void *)completed_src, + (void *)completed_dst) != RTE_DIM(srcs)) { + printf("Error with rte_ioat_completed_copies\n"); + return -1; + } + for (i = 0; i < RTE_DIM(srcs); i++) { + char *src_data, *dst_data; + + if (completed_src[i] != srcs[i]) { + printf("Error with source pointer %u\n", i); + return -1; + } + if (completed_dst[i] != dsts[i]) { + printf("Error with dest pointer %u\n", i); + return -1; + } + + src_data = rte_pktmbuf_mtod(srcs[i], char *); + dst_data = rte_pktmbuf_mtod(dsts[i], char *); + for (j = 0; j < length; j++) + if (src_data[j] != dst_data[j]) { + printf("Error with copy of packet %u, byte %u\n", + i, j); + return -1; + } + rte_pktmbuf_free(srcs[i]); + rte_pktmbuf_free(dsts[i]); + } + + Querying Device Statistics ~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h b/drivers/raw/ioat/rte_ioat_rawdev.h index abe5ba298..7750edba7 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev.h +++ b/drivers/raw/ioat/rte_ioat_rawdev.h @@ -14,7 +14,10 @@ * @b EXPERIMENTAL: these structures and APIs may change without prior notice */ +#include + #include +#include #include /** Name of the device driver */ @@ -49,6 +52,10 @@ struct rte_ioat_rawdev { struct rte_ioat_desc *desc_ring; __m128i *hdls; /* completion handles for returning to user */ + + unsigned short next_read; + unsigned short next_write; + /* some statistics for tracking, if added/changed update xstats fns*/ uint64_t enqueue_failed __rte_cache_aligned; uint64_t enqueued; @@ -72,4 +79,150 @@ struct rte_ioat_desc { uint64_t op_type_specific[4]; }; -#endif +/** + * Enqueue a copy operation onto the ioat device + * + * This queues up a copy operation to be performed by hardware, but does not + * trigger hardware to begin that operation. + * + * @param dev_id + * The rawdev device id of the ioat instance + * @param src + * The physical address of the source buffer + * @param dst + * The physical address of the destination buffer + * @param length + * The length of the data to be copied + * @param src_hdl + * An opaque handle for the source data, to be returned when this operation + * has been completed and the user polls for the completion details + * @param dst_hdl + * An opaque handle for the destination data, to be returned when this + * operation has been completed and the user polls for the completion details + * @param fence + * A flag parameter indicating that hardware should not begin to perform any + * subsequently enqueued copy operations until after this operation has + * completed + * @return + * Number of operations enqueued, either 0 or 1 + */ +static inline int +rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, + unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl, + int fence) +{ + struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; + unsigned short read = ioat->next_read; + unsigned short write = ioat->next_write; + unsigned short mask = ioat->ring_size - 1; + unsigned short space = mask + read - write; + + if (space == 0) { + ioat->enqueue_failed++; + return 0; + } + + ioat->next_write = write + 1; + write &= mask; + /* write transfer size and the descriptor control bits */ + *((uint64_t *)&ioat->desc_ring[write]) = + length | (uint64_t)!!fence << 36 | + (uint64_t)(!(write & 0xF)) << 35; + ioat->desc_ring[write].src_addr = src; + ioat->desc_ring[write].dest_addr = dst; + ioat->hdls[write] = _mm_set_epi64((__m64)dst_hdl, (__m64)src_hdl); + rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]); + + ioat->enqueued++; + return 1; +} + +/** + * Trigger hardware to begin performing enqueued copy operations + * + * This API is used to write the "doorbell" to the hardware to trigger it + * to begin the copy operations previously enqueued by rte_ioat_enqueue_copy() + * + * @param dev_id + * The rawdev device id of the ioat instance + */ +static inline void +rte_ioat_do_copies(int dev_id) +{ + struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; + ioat->desc_ring[(ioat->next_write - 1) & (ioat->ring_size - 1)].desc_control = 8; + rte_compiler_barrier(); + ioat->regs->dmacount = ioat->next_write; + ioat->started = ioat->enqueued; +} + +/** + * @internal + * Returns the index of the last completed operation. + */ +static inline int +rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat) +{ + uint64_t status = ioat->status; + /* lower 3 bits indicate "transfer status" : active, idle, halted.. */ + if (status & 0x6) + rte_panic("Error with cbdma: %u\n", (unsigned)status & 0x7); + + return (status - ioat->ring_addr) >> 6; +} + +/** + * Returns details of copy operations that have been completed + * + * Returns to the caller the user-provided "handles" for the copy operations + * which have been completed by the hardware, and not already returned by + * a previous call to this API. + * + * @param dev_id + * The rawdev device id of the ioat instance + * @param max_copies + * The number of entries which can fit in the src_hdls and dst_hdls + * arrays, i.e. max number of completed operations to report + * @param src_hdls + * Array to hold the source handle parameters of the completed copies + * @param dst_hdls + * Array to hold the destination handle parameters of the completed copies + * @return + * Number of completed operations i.e. number of entries written to the + * src_hdls and dst_hdls array parameters. + */ +static inline int +rte_ioat_completed_copies(int dev_id, uint8_t max_copies, + uintptr_t *src_hdls, uintptr_t *dst_hdls) +{ + struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; + unsigned short mask = (ioat->ring_size - 1); + unsigned short read = ioat->next_read; + unsigned short end_read = (rte_ioat_get_last_completed(ioat) + 1) & mask; + unsigned short count = (end_read - (read & mask)) & mask; + int i = 0; + + if (count > max_copies) + count = max_copies; + + for (; i < count - 1; i += 2, read += 2) { + __m128i hdls0 = _mm_load_si128(&ioat->hdls[read & mask]); + __m128i hdls1 = _mm_load_si128(&ioat->hdls[(read + 1) & mask]); + + _mm_storeu_si128((void *)&src_hdls[i], + _mm_unpacklo_epi64(hdls0, hdls1)); + _mm_storeu_si128((void *)&dst_hdls[i], + _mm_unpackhi_epi64(hdls0, hdls1)); + } + for (; i < count; i++, read++) { + uintptr_t *hdls = (void *)&ioat->hdls[read & mask]; + src_hdls[i] = hdls[0]; + dst_hdls[i] = hdls[1]; + } + + ioat->next_read = read; + ioat->completed += count; + return count; +} + +#endif /* _RTE_IOAT_RAWDEV_H_ */