From patchwork Tue Jul 21 09:51:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74540 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3689BA0526; Tue, 21 Jul 2020 11:52:07 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 18A121BFFD; Tue, 21 Jul 2020 11:52:07 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 48B8D1BFD1 for ; Tue, 21 Jul 2020 11:52:05 +0200 (CEST) IronPort-SDR: 2lhDHgqnIPrNPMfsUczKaKazbOuC7j/ytJxateUGtk+4LxVl5tyxK6quEVaQob1jbV+BxVWNbg BOpYCuGu8wug== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="234952334" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="234952334" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:52:04 -0700 IronPort-SDR: 2C2L1o25rkvWPz7C4fDdwEbt2Rvdw92ua0NpWrQ/qJmekwySZFXq5HYSjPfTDLbvjAUP016Rtp 2EwmbqXJWqkg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488023629" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:52:02 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Cheng Jiang , Bruce Richardson Date: Tue, 21 Jul 2020 10:51:21 +0100 Message-Id: <20200721095140.719297-2-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 01/20] raw/ioat: add a flag to control copying handle parameters X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Cheng Jiang Add a flag which controls whether rte_ioat_enqueue_copy and rte_ioat_completed_copies function should process handle parameters. Not doing so can improve the performance when handle parameters are not necessary. Signed-off-by: Cheng Jiang Signed-off-by: Bruce Richardson --- drivers/raw/ioat/ioat_rawdev.c | 1 + drivers/raw/ioat/rte_ioat_rawdev.h | 45 ++++++++++++++++++++++-------- 2 files changed, 35 insertions(+), 11 deletions(-) diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index 7f1a15436..53b33c1a7 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -58,6 +58,7 @@ ioat_dev_configure(const struct rte_rawdev *dev, rte_rawdev_obj_t config, return -EINVAL; ioat->ring_size = params->ring_size; + ioat->hdls_disable = params->hdls_disable; if (ioat->desc_ring != NULL) { rte_memzone_free(ioat->desc_mz); ioat->desc_ring = NULL; diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h b/drivers/raw/ioat/rte_ioat_rawdev.h index f765a6557..fd3a8fe14 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev.h +++ b/drivers/raw/ioat/rte_ioat_rawdev.h @@ -34,7 +34,8 @@ * an ioat rawdev instance. */ struct rte_ioat_rawdev_config { - unsigned short ring_size; + unsigned short ring_size; /**< size of job submission descriptor ring */ + bool hdls_disable; /**< when set, ignore user-supplied handle parameters */ }; /** @@ -52,6 +53,7 @@ struct rte_ioat_rawdev { unsigned short ring_size; struct rte_ioat_generic_hw_desc *desc_ring; + bool hdls_disable; __m128i *hdls; /* completion handles for returning to user */ @@ -84,10 +86,14 @@ struct rte_ioat_rawdev { * The length of the data to be copied * @param src_hdl * An opaque handle for the source data, to be returned when this operation - * has been completed and the user polls for the completion details + * has been completed and the user polls for the completion details. + * NOTE: If hdls_disable configuration option for the device is set, this + * parameter is ignored. * @param dst_hdl * An opaque handle for the destination data, to be returned when this - * operation has been completed and the user polls for the completion details + * operation has been completed and the user polls for the completion details. + * NOTE: If hdls_disable configuration option for the device is set, this + * parameter is ignored. * @param fence * A flag parameter indicating that hardware should not begin to perform any * subsequently enqueued copy operations until after this operation has @@ -121,8 +127,10 @@ rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, desc->u.control_raw = (uint32_t)((!!fence << 4) | (!(write & 0xF)) << 3); desc->src_addr = src; desc->dest_addr = dst; + if (!ioat->hdls_disable) + ioat->hdls[write] = _mm_set_epi64x((int64_t)dst_hdl, + (int64_t)src_hdl); - ioat->hdls[write] = _mm_set_epi64x((int64_t)dst_hdl, (int64_t)src_hdl); rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]); ioat->enqueued++; @@ -168,19 +176,29 @@ rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error) /** * Returns details of copy operations that have been completed * - * Returns to the caller the user-provided "handles" for the copy operations - * which have been completed by the hardware, and not already returned by - * a previous call to this API. + * If the hdls_disable option was not set when the device was configured, + * the function will return to the caller the user-provided "handles" for + * the copy operations which have been completed by the hardware, and not + * already returned by a previous call to this API. + * If the hdls_disable option for the device was set on configure, the + * max_copies, src_hdls and dst_hdls parameters will be ignored, and the + * function returns the number of newly-completed operations. * * @param dev_id * The rawdev device id of the ioat instance * @param max_copies * The number of entries which can fit in the src_hdls and dst_hdls - * arrays, i.e. max number of completed operations to report + * arrays, i.e. max number of completed operations to report. + * NOTE: If hdls_disable configuration option for the device is set, this + * parameter is ignored. * @param src_hdls - * Array to hold the source handle parameters of the completed copies + * Array to hold the source handle parameters of the completed copies. + * NOTE: If hdls_disable configuration option for the device is set, this + * parameter is ignored. * @param dst_hdls - * Array to hold the destination handle parameters of the completed copies + * Array to hold the destination handle parameters of the completed copies. + * NOTE: If hdls_disable configuration option for the device is set, this + * parameter is ignored. * @return * -1 on error, with rte_errno set appropriately. * Otherwise number of completed operations i.e. number of entries written @@ -205,6 +223,11 @@ rte_ioat_completed_copies(int dev_id, uint8_t max_copies, return -1; } + if (ioat->hdls_disable) { + read += count; + goto end; + } + if (count > max_copies) count = max_copies; @@ -222,7 +245,7 @@ rte_ioat_completed_copies(int dev_id, uint8_t max_copies, src_hdls[i] = hdls[0]; dst_hdls[i] = hdls[1]; } - +end: ioat->next_read = read; ioat->completed += count; return count; From patchwork Tue Jul 21 09:51:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74541 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C3C4AA0526; Tue, 21 Jul 2020 11:52:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 96B631C00F; Tue, 21 Jul 2020 11:52:10 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 8115D1C00D for ; Tue, 21 Jul 2020 11:52:08 +0200 (CEST) IronPort-SDR: 0DIMkA+2KgVEEJNUErXURGyjv1PhFNqKsy+LcvxNZifRPTOVn5gPvoG+p01MiV3HGm64yHyGCH VXDGPqONrY0w== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="234952340" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="234952340" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:52:08 -0700 IronPort-SDR: KSlq8q+9oU2tXpCWdT7hTmTSE24U+SeaMl7ObZskICErjL6lBARMUGSCI1T5jcYHGHtRkmiwrU 9ZHj0LlBd8ag== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488023648" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:52:06 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:22 +0100 Message-Id: <20200721095140.719297-3-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 02/20] raw/ioat: support multiple devices being tested X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The current selftest function uses a single global variable to track state which implies that only a single instance can have the selftest function called on it. Change this to an array to allow multiple instances to be tested. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/ioat_rawdev_test.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/raw/ioat/ioat_rawdev_test.c b/drivers/raw/ioat/ioat_rawdev_test.c index c463a82ad..e5b50ae9f 100644 --- a/drivers/raw/ioat/ioat_rawdev_test.c +++ b/drivers/raw/ioat/ioat_rawdev_test.c @@ -8,10 +8,13 @@ #include "rte_rawdev.h" #include "rte_ioat_rawdev.h" +#define MAX_SUPPORTED_RAWDEVS 64 +#define TEST_SKIPPED 77 + int ioat_rawdev_test(uint16_t dev_id); /* pre-define to keep compiler happy */ static struct rte_mempool *pool; -static unsigned short expected_ring_size; +static unsigned short expected_ring_size[MAX_SUPPORTED_RAWDEVS]; static int test_enqueue_copies(int dev_id) @@ -148,10 +151,16 @@ ioat_rawdev_test(uint16_t dev_id) unsigned int nb_xstats; unsigned int i; + if (dev_id >= MAX_SUPPORTED_RAWDEVS) { + printf("Skipping test. Cannot test rawdevs with id's greater than %d\n", + MAX_SUPPORTED_RAWDEVS); + return TEST_SKIPPED; + } + rte_rawdev_info_get(dev_id, &info, sizeof(p)); - if (p.ring_size != expected_ring_size) { + if (p.ring_size != expected_ring_size[dev_id]) { printf("Error, initial ring size is not as expected (Actual: %d, Expected: %d)\n", - (int)p.ring_size, expected_ring_size); + (int)p.ring_size, expected_ring_size[dev_id]); return -1; } @@ -166,7 +175,7 @@ ioat_rawdev_test(uint16_t dev_id) IOAT_TEST_RINGSIZE, (int)p.ring_size); return -1; } - expected_ring_size = p.ring_size; + expected_ring_size[dev_id] = p.ring_size; if (rte_rawdev_start(dev_id) != 0) { printf("Error with rte_rawdev_start()\n"); From patchwork Tue Jul 21 09:51:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74542 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1FF5DA0526; Tue, 21 Jul 2020 11:52:24 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 09F431C025; Tue, 21 Jul 2020 11:52:15 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 548871BFFE for ; Tue, 21 Jul 2020 11:52:14 +0200 (CEST) IronPort-SDR: r6RaznEtgAUMoWuVDejTEmWubfT7FLoghxlLq9tR56WbA9EwKOtrbf8z8/F+Lo9El2ajbyiLtt h0H3206DfATg== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="151440752" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="151440752" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:52:13 -0700 IronPort-SDR: Z9g8QCGj7OKbHu4xMo6q/jlARR/WE1mJxmVKQ7DxeBgHfVpoK/tuFf+lbe1F2RERSf8zqYDPRT MR4RZLmxImJQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488023683" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:52:11 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson , hemant.agrawal@nxp.com, nipun.gupta@nxp.com Date: Tue, 21 Jul 2020 10:51:23 +0100 Message-Id: <20200721095140.719297-4-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 03/20] app/test: change rawdev autotest to run selftest on all devs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Rather than having each rawdev provide its own autotest command, we can instead just use the generic rawdev_autotest to test any and all available rawdevs. Signed-off-by: Bruce Richardson Cc: hemant.agrawal@nxp.com Cc: nipun.gupta@nxp.com --- app/test/test_rawdev.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/app/test/test_rawdev.c b/app/test/test_rawdev.c index d8d9595be..a62f719d6 100644 --- a/app/test/test_rawdev.c +++ b/app/test/test_rawdev.c @@ -14,14 +14,37 @@ static int test_rawdev_selftest_impl(const char *pmd, const char *opts) { + int ret; + + printf("\n### Test rawdev infrastructure using skeleton driver\n"); rte_vdev_init(pmd, opts); - return rte_rawdev_selftest(rte_rawdev_get_dev_id(pmd)); + ret = rte_rawdev_selftest(rte_rawdev_get_dev_id(pmd)); + rte_vdev_uninit(pmd); + return ret; } static int test_rawdev_selftest_skeleton(void) { - return test_rawdev_selftest_impl("rawdev_skeleton", ""); + const int count = rte_rawdev_count(); + int ret = 0; + int i; + + /* basic sanity on rawdev infrastructure */ + if (test_rawdev_selftest_impl("rawdev_skeleton", "") < 0) + return -1; + + /* now run self-test on all rawdevs */ + if (count > 0) + printf("\n### Run selftest on each available rawdev\n"); + for (i = 0; i < count; i++) { + int result = rte_rawdev_selftest(i); + printf("Rawdev %u selftest: %s\n", i, + result == 0 ? "Passed" : "Failed"); + ret |= result; + } + + return ret; } REGISTER_TEST_COMMAND(rawdev_autotest, test_rawdev_selftest_skeleton); From patchwork Tue Jul 21 09:51:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74543 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5FAC5A0526; Tue, 21 Jul 2020 11:52:33 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5F9891C02D; Tue, 21 Jul 2020 11:52:18 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 63D321C02D for ; Tue, 21 Jul 2020 11:52:16 +0200 (CEST) IronPort-SDR: feHVELw6pIfcQffhg1C15eGdKG5RGhbvuF6BQ6atEPc6Ktm/XMlDh/UJ5psfujSwtrcB0Yp1Jw mlh3cWl5MgsQ== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="151440753" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="151440753" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:52:16 -0700 IronPort-SDR: mca1UxpqiCvPMysOUC7qAHCAvc/t6sK7jwUY42MKYQtTS3/ts7NU+Cm9bQVLpLItI+hkuobrxL guGkK+JvZkqQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488023701" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:52:14 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:24 +0100 Message-Id: <20200721095140.719297-5-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 04/20] app/test: remove ioat-specific autotest X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since the rawdev autotest can now be used to test all rawdevs on the system, there is no need for a dedicated ioat autotest command. Signed-off-by: Bruce Richardson --- app/test/test_rawdev.c | 20 -------------------- 1 file changed, 20 deletions(-) diff --git a/app/test/test_rawdev.c b/app/test/test_rawdev.c index a62f719d6..2944b3a27 100644 --- a/app/test/test_rawdev.c +++ b/app/test/test_rawdev.c @@ -48,23 +48,3 @@ test_rawdev_selftest_skeleton(void) } REGISTER_TEST_COMMAND(rawdev_autotest, test_rawdev_selftest_skeleton); - -static int -test_rawdev_selftest_ioat(void) -{ - const int count = rte_rawdev_count(); - int i; - - for (i = 0; i < count; i++) { - struct rte_rawdev_info info = { .dev_private = NULL }; - if (rte_rawdev_info_get(i, &info, 0) == 0 && - strstr(info.driver_name, "ioat") != NULL) - return rte_rawdev_selftest(i) == 0 ? - TEST_SUCCESS : TEST_FAILED; - } - - printf("No IOAT rawdev found, skipping tests\n"); - return TEST_SKIPPED; -} - -REGISTER_TEST_COMMAND(ioat_rawdev_autotest, test_rawdev_selftest_ioat); From patchwork Tue Jul 21 09:51:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74544 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 38027A0526; Tue, 21 Jul 2020 11:52:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8B2ED1C033; Tue, 21 Jul 2020 11:52:22 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 659531C00E for ; Tue, 21 Jul 2020 11:52:20 +0200 (CEST) IronPort-SDR: hKpn2SAEX9upDRQoWOX6V2YDPOfDinLi/jQnwlWreuL0p55VmMR1CoNEUv3AsOfSGzEQgiKesy zaZJ3YV8twzQ== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="151440760" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="151440760" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:52:20 -0700 IronPort-SDR: FRTVT6C6Vu9wilWYJazzVIFdfi8P7azos2N7njRJlBnnxhLnbcvQa8QmWOJYJkarX5h0/VPy7k /51NY4BrL6aA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488023718" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:52:18 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:25 +0100 Message-Id: <20200721095140.719297-6-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 05/20] raw/ioat: split header for readability X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Rather than having a single long complicated header file for general use we can split things so that there is one header with all the publically needed information - data structs and function prototypes - while the rest of the internal details are put separately. This makes it easier to read, understand and use the APIs. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/meson.build | 1 + drivers/raw/ioat/rte_ioat_rawdev.h | 144 +--------------------- drivers/raw/ioat/rte_ioat_rawdev_fns.h | 164 +++++++++++++++++++++++++ 3 files changed, 171 insertions(+), 138 deletions(-) create mode 100644 drivers/raw/ioat/rte_ioat_rawdev_fns.h diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build index 0878418ae..f66e9b605 100644 --- a/drivers/raw/ioat/meson.build +++ b/drivers/raw/ioat/meson.build @@ -8,4 +8,5 @@ sources = files('ioat_rawdev.c', deps += ['rawdev', 'bus_pci', 'mbuf'] install_headers('rte_ioat_rawdev.h', + 'rte_ioat_rawdev_fns.h', 'rte_ioat_spec.h') diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h b/drivers/raw/ioat/rte_ioat_rawdev.h index fd3a8fe14..6d338f50c 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev.h +++ b/drivers/raw/ioat/rte_ioat_rawdev.h @@ -14,12 +14,7 @@ * @b EXPERIMENTAL: these structures and APIs may change without prior notice */ -#include -#include -#include -#include -#include -#include "rte_ioat_spec.h" +#include /** Name of the device driver */ #define IOAT_PMD_RAWDEV_NAME rawdev_ioat @@ -38,38 +33,6 @@ struct rte_ioat_rawdev_config { bool hdls_disable; /**< when set, ignore user-supplied handle parameters */ }; -/** - * @internal - * Structure representing a device instance - */ -struct rte_ioat_rawdev { - struct rte_rawdev *rawdev; - const struct rte_memzone *mz; - const struct rte_memzone *desc_mz; - - volatile struct rte_ioat_registers *regs; - phys_addr_t status_addr; - phys_addr_t ring_addr; - - unsigned short ring_size; - struct rte_ioat_generic_hw_desc *desc_ring; - bool hdls_disable; - __m128i *hdls; /* completion handles for returning to user */ - - - unsigned short next_read; - unsigned short next_write; - - /* some statistics for tracking, if added/changed update xstats fns*/ - uint64_t enqueue_failed __rte_cache_aligned; - uint64_t enqueued; - uint64_t started; - uint64_t completed; - - /* to report completions, the device will write status back here */ - volatile uint64_t status __rte_cache_aligned; -}; - /** * Enqueue a copy operation onto the ioat device * @@ -104,38 +67,7 @@ struct rte_ioat_rawdev { static inline int rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl, - int fence) -{ - struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; - unsigned short read = ioat->next_read; - unsigned short write = ioat->next_write; - unsigned short mask = ioat->ring_size - 1; - unsigned short space = mask + read - write; - struct rte_ioat_generic_hw_desc *desc; - - if (space == 0) { - ioat->enqueue_failed++; - return 0; - } - - ioat->next_write = write + 1; - write &= mask; - - desc = &ioat->desc_ring[write]; - desc->size = length; - /* set descriptor write-back every 16th descriptor */ - desc->u.control_raw = (uint32_t)((!!fence << 4) | (!(write & 0xF)) << 3); - desc->src_addr = src; - desc->dest_addr = dst; - if (!ioat->hdls_disable) - ioat->hdls[write] = _mm_set_epi64x((int64_t)dst_hdl, - (int64_t)src_hdl); - - rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]); - - ioat->enqueued++; - return 1; -} + int fence); /** * Trigger hardware to begin performing enqueued copy operations @@ -147,31 +79,7 @@ rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, * The rawdev device id of the ioat instance */ static inline void -rte_ioat_do_copies(int dev_id) -{ - struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; - ioat->desc_ring[(ioat->next_write - 1) & (ioat->ring_size - 1)].u - .control.completion_update = 1; - rte_compiler_barrier(); - ioat->regs->dmacount = ioat->next_write; - ioat->started = ioat->enqueued; -} - -/** - * @internal - * Returns the index of the last completed operation. - */ -static inline int -rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error) -{ - uint64_t status = ioat->status; - - /* lower 3 bits indicate "transfer status" : active, idle, halted. - * We can ignore bit 0. - */ - *error = status & (RTE_IOAT_CHANSTS_SUSPENDED | RTE_IOAT_CHANSTS_ARMED); - return (status - ioat->ring_addr) >> 6; -} +rte_ioat_do_copies(int dev_id); /** * Returns details of copy operations that have been completed @@ -206,49 +114,9 @@ rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error) */ static inline int rte_ioat_completed_copies(int dev_id, uint8_t max_copies, - uintptr_t *src_hdls, uintptr_t *dst_hdls) -{ - struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; - unsigned short mask = (ioat->ring_size - 1); - unsigned short read = ioat->next_read; - unsigned short end_read, count; - int error; - int i = 0; - - end_read = (rte_ioat_get_last_completed(ioat, &error) + 1) & mask; - count = (end_read - (read & mask)) & mask; - - if (error) { - rte_errno = EIO; - return -1; - } - - if (ioat->hdls_disable) { - read += count; - goto end; - } - - if (count > max_copies) - count = max_copies; - - for (; i < count - 1; i += 2, read += 2) { - __m128i hdls0 = _mm_load_si128(&ioat->hdls[read & mask]); - __m128i hdls1 = _mm_load_si128(&ioat->hdls[(read + 1) & mask]); + uintptr_t *src_hdls, uintptr_t *dst_hdls); - _mm_storeu_si128((void *)&src_hdls[i], - _mm_unpacklo_epi64(hdls0, hdls1)); - _mm_storeu_si128((void *)&dst_hdls[i], - _mm_unpackhi_epi64(hdls0, hdls1)); - } - for (; i < count; i++, read++) { - uintptr_t *hdls = (void *)&ioat->hdls[read & mask]; - src_hdls[i] = hdls[0]; - dst_hdls[i] = hdls[1]; - } -end: - ioat->next_read = read; - ioat->completed += count; - return count; -} +/* include the implementation details from a separate file */ +#include "rte_ioat_rawdev_fns.h" #endif /* _RTE_IOAT_RAWDEV_H_ */ diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h b/drivers/raw/ioat/rte_ioat_rawdev_fns.h new file mode 100644 index 000000000..06b4edcbb --- /dev/null +++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h @@ -0,0 +1,164 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019-2020 Intel Corporation + */ +#ifndef _RTE_IOAT_RAWDEV_FNS_H_ +#define _RTE_IOAT_RAWDEV_FNS_H_ + +#include +#include +#include +#include "rte_ioat_spec.h" + +/** + * @internal + * Structure representing a device instance + */ +struct rte_ioat_rawdev { + struct rte_rawdev *rawdev; + const struct rte_memzone *mz; + const struct rte_memzone *desc_mz; + + volatile struct rte_ioat_registers *regs; + phys_addr_t status_addr; + phys_addr_t ring_addr; + + unsigned short ring_size; + bool hdls_disable; + struct rte_ioat_generic_hw_desc *desc_ring; + __m128i *hdls; /* completion handles for returning to user */ + + + unsigned short next_read; + unsigned short next_write; + + /* some statistics for tracking, if added/changed update xstats fns*/ + uint64_t enqueue_failed __rte_cache_aligned; + uint64_t enqueued; + uint64_t started; + uint64_t completed; + + /* to report completions, the device will write status back here */ + volatile uint64_t status __rte_cache_aligned; +}; + +/** + * Enqueue a copy operation onto the ioat device + */ +static inline int +rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, + unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl, + int fence) +{ + struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; + unsigned short read = ioat->next_read; + unsigned short write = ioat->next_write; + unsigned short mask = ioat->ring_size - 1; + unsigned short space = mask + read - write; + struct rte_ioat_generic_hw_desc *desc; + + if (space == 0) { + ioat->enqueue_failed++; + return 0; + } + + ioat->next_write = write + 1; + write &= mask; + + desc = &ioat->desc_ring[write]; + desc->size = length; + /* set descriptor write-back every 16th descriptor */ + desc->u.control_raw = (uint32_t)((!!fence << 4) | (!(write & 0xF)) << 3); + desc->src_addr = src; + desc->dest_addr = dst; + + if (!ioat->hdls_disable) + ioat->hdls[write] = _mm_set_epi64x((int64_t)dst_hdl, + (int64_t)src_hdl); + rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]); + + ioat->enqueued++; + return 1; +} + +/** + * Trigger hardware to begin performing enqueued copy operations + */ +static inline void +rte_ioat_do_copies(int dev_id) +{ + struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; + ioat->desc_ring[(ioat->next_write - 1) & (ioat->ring_size - 1)].u + .control.completion_update = 1; + rte_compiler_barrier(); + ioat->regs->dmacount = ioat->next_write; + ioat->started = ioat->enqueued; +} + +/** + * @internal + * Returns the index of the last completed operation. + */ +static inline int +rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error) +{ + uint64_t status = ioat->status; + + /* lower 3 bits indicate "transfer status" : active, idle, halted. + * We can ignore bit 0. + */ + *error = status & (RTE_IOAT_CHANSTS_SUSPENDED | RTE_IOAT_CHANSTS_ARMED); + return (status - ioat->ring_addr) >> 6; +} + +/** + * Returns details of copy operations that have been completed + */ +static inline int +rte_ioat_completed_copies(int dev_id, uint8_t max_copies, + uintptr_t *src_hdls, uintptr_t *dst_hdls) +{ + struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; + unsigned short mask = (ioat->ring_size - 1); + unsigned short read = ioat->next_read; + unsigned short end_read, count; + int error; + int i = 0; + + end_read = (rte_ioat_get_last_completed(ioat, &error) + 1) & mask; + count = (end_read - (read & mask)) & mask; + + if (error) { + rte_errno = EIO; + return -1; + } + + if (ioat->hdls_disable) { + read += count; + goto end; + } + + if (count > max_copies) + count = max_copies; + + for (; i < count - 1; i += 2, read += 2) { + __m128i hdls0 = _mm_load_si128(&ioat->hdls[read & mask]); + __m128i hdls1 = _mm_load_si128(&ioat->hdls[(read + 1) & mask]); + + _mm_storeu_si128((void *)&src_hdls[i], + _mm_unpacklo_epi64(hdls0, hdls1)); + _mm_storeu_si128((void *)&dst_hdls[i], + _mm_unpackhi_epi64(hdls0, hdls1)); + } + for (; i < count; i++, read++) { + uintptr_t *hdls = (void *)&ioat->hdls[read & mask]; + src_hdls[i] = hdls[0]; + dst_hdls[i] = hdls[1]; + } + +end: + ioat->next_read = read; + ioat->completed += count; + return count; +} + +#endif /* _RTE_IOAT_RAWDEV_FNS_H_ */ From patchwork Tue Jul 21 09:51:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74545 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1F7BBA0526; Tue, 21 Jul 2020 11:52:57 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3BEB01C066; Tue, 21 Jul 2020 11:52:25 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id D7DFC1C044 for ; Tue, 21 Jul 2020 11:52:23 +0200 (CEST) IronPort-SDR: kB2/v9jJ1lKV5irEg9i2IiBJ9+oSus7GFihXJuJTADzIkqgM6V0herGEeN+rBVAW96z4lIvCfI lelBnNPj9w5Q== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="151440767" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="151440767" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:52:23 -0700 IronPort-SDR: Y0OTZ8C8oA2vuU6Rn9am92D6EKEWnunN3pywOZBcaPO4hYu7B6ioyCF86AwFIDcsCoAnKLjKHE YX6uSorsDmZg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488023731" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:52:21 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:26 +0100 Message-Id: <20200721095140.719297-7-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 06/20] raw/ioat: make the HW register spec private X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Only a few definitions from the hardware spec are actually used in the driver runtime, so we can copy over those few and make the rest of the spec a private header in the driver. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/ioat_rawdev.c | 3 ++ .../raw/ioat/{rte_ioat_spec.h => ioat_spec.h} | 26 ----------- drivers/raw/ioat/meson.build | 3 +- drivers/raw/ioat/rte_ioat_rawdev_fns.h | 44 +++++++++++++++++-- 4 files changed, 44 insertions(+), 32 deletions(-) rename drivers/raw/ioat/{rte_ioat_spec.h => ioat_spec.h} (91%) diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index 53b33c1a7..11c6a57e1 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -4,10 +4,12 @@ #include #include +#include #include #include #include "rte_ioat_rawdev.h" +#include "ioat_spec.h" static struct rte_pci_driver ioat_pmd_drv; @@ -260,6 +262,7 @@ ioat_rawdev_create(const char *name, struct rte_pci_device *dev) ioat->rawdev = rawdev; ioat->mz = mz; ioat->regs = dev->mem_resource[0].addr; + ioat->doorbell = &ioat->regs->dmacount; ioat->ring_size = 0; ioat->desc_ring = NULL; ioat->status_addr = ioat->mz->iova + diff --git a/drivers/raw/ioat/rte_ioat_spec.h b/drivers/raw/ioat/ioat_spec.h similarity index 91% rename from drivers/raw/ioat/rte_ioat_spec.h rename to drivers/raw/ioat/ioat_spec.h index c6e7929b2..9645e16d4 100644 --- a/drivers/raw/ioat/rte_ioat_spec.h +++ b/drivers/raw/ioat/ioat_spec.h @@ -86,32 +86,6 @@ struct rte_ioat_registers { #define RTE_IOAT_CHANCMP_ALIGN 8 /* CHANCMP address must be 64-bit aligned */ -struct rte_ioat_generic_hw_desc { - uint32_t size; - union { - uint32_t control_raw; - struct { - uint32_t int_enable: 1; - uint32_t src_snoop_disable: 1; - uint32_t dest_snoop_disable: 1; - uint32_t completion_update: 1; - uint32_t fence: 1; - uint32_t reserved2: 1; - uint32_t src_page_break: 1; - uint32_t dest_page_break: 1; - uint32_t bundle: 1; - uint32_t dest_dca: 1; - uint32_t hint: 1; - uint32_t reserved: 13; - uint32_t op: 8; - } control; - } u; - uint64_t src_addr; - uint64_t dest_addr; - uint64_t next; - uint64_t op_specific[4]; -}; - struct rte_ioat_dma_hw_desc { uint32_t size; union { diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build index f66e9b605..06636f8a9 100644 --- a/drivers/raw/ioat/meson.build +++ b/drivers/raw/ioat/meson.build @@ -8,5 +8,4 @@ sources = files('ioat_rawdev.c', deps += ['rawdev', 'bus_pci', 'mbuf'] install_headers('rte_ioat_rawdev.h', - 'rte_ioat_rawdev_fns.h', - 'rte_ioat_spec.h') + 'rte_ioat_rawdev_fns.h') diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h b/drivers/raw/ioat/rte_ioat_rawdev_fns.h index 06b4edcbb..0cee6b1b0 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h +++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h @@ -5,9 +5,37 @@ #define _RTE_IOAT_RAWDEV_FNS_H_ #include -#include #include -#include "rte_ioat_spec.h" + +/** + * @internal + * Structure representing a device descriptor + */ +struct rte_ioat_generic_hw_desc { + uint32_t size; + union { + uint32_t control_raw; + struct { + uint32_t int_enable: 1; + uint32_t src_snoop_disable: 1; + uint32_t dest_snoop_disable: 1; + uint32_t completion_update: 1; + uint32_t fence: 1; + uint32_t reserved2: 1; + uint32_t src_page_break: 1; + uint32_t dest_page_break: 1; + uint32_t bundle: 1; + uint32_t dest_dca: 1; + uint32_t hint: 1; + uint32_t reserved: 13; + uint32_t op: 8; + } control; + } u; + uint64_t src_addr; + uint64_t dest_addr; + uint64_t next; + uint64_t op_specific[4]; +}; /** * @internal @@ -18,7 +46,7 @@ struct rte_ioat_rawdev { const struct rte_memzone *mz; const struct rte_memzone *desc_mz; - volatile struct rte_ioat_registers *regs; + volatile uint16_t *doorbell; phys_addr_t status_addr; phys_addr_t ring_addr; @@ -39,8 +67,16 @@ struct rte_ioat_rawdev { /* to report completions, the device will write status back here */ volatile uint64_t status __rte_cache_aligned; + + /* pointer to the register bar */ + volatile struct rte_ioat_registers *regs; }; +#define RTE_IOAT_CHANSTS_IDLE 0x1 +#define RTE_IOAT_CHANSTS_SUSPENDED 0x2 +#define RTE_IOAT_CHANSTS_HALTED 0x3 +#define RTE_IOAT_CHANSTS_ARMED 0x4 + /** * Enqueue a copy operation onto the ioat device */ @@ -90,7 +126,7 @@ rte_ioat_do_copies(int dev_id) ioat->desc_ring[(ioat->next_write - 1) & (ioat->ring_size - 1)].u .control.completion_update = 1; rte_compiler_barrier(); - ioat->regs->dmacount = ioat->next_write; + *ioat->doorbell = ioat->next_write; ioat->started = ioat->enqueued; } From patchwork Tue Jul 21 09:51:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74546 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BDA5DA0526; Tue, 21 Jul 2020 11:53:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A6D661BFF9; Tue, 21 Jul 2020 11:53:09 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id A3BA61BFF9 for ; Tue, 21 Jul 2020 11:53:07 +0200 (CEST) IronPort-SDR: Z7viS/LJdrCo3Tfj8jJ+ZCauojKZ99cPgRMIKg31lLYfzM/k/KFFjpxParpjR9UJTP8+EoS6OQ 7p4JdtGeXHMw== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191205" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191205" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:53:06 -0700 IronPort-SDR: BeUzfNdWZgLMZwJJml4YBze08VGtz7NoXWGq0qpF3vLZhTD/QNizPttGCUNb1aDwVqwar0nzZt /OQKemeGzZug== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488023857" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:53:04 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:27 +0100 Message-Id: <20200721095140.719297-8-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 07/20] usertools/dpdk-devbind.py: add support for DSA HW X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Kevin Laatz Intel Data Streaming Accelerator (Intel DSA) is a high-performance data copy and transformation accelerator which will be integrated in future Intel processors [1]. Add DSA device support to dpdk-devbind.py script. [1] https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz --- usertools/dpdk-devbind.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index dc008823f..39743abb2 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -47,6 +47,8 @@ 'SVendor': None, 'SDevice': None} intel_ioat_icx = {'Class': '08', 'Vendor': '8086', 'Device': '0b00', 'SVendor': None, 'SDevice': None} +intel_idxd_spr = {'Class': '08', 'Vendor': '8086', 'Device': '0b25', + 'SVendor': None, 'SDevice': None} intel_ntb_skx = {'Class': '06', 'Vendor': '8086', 'Device': '201c', 'SVendor': None, 'SDevice': None} @@ -56,7 +58,7 @@ eventdev_devices = [cavium_sso, cavium_tim, octeontx2_sso] mempool_devices = [cavium_fpa, octeontx2_npa] compress_devices = [cavium_zip] -misc_devices = [intel_ioat_bdw, intel_ioat_skx, intel_ioat_icx, intel_ntb_skx, octeontx2_dma] +misc_devices = [intel_ioat_bdw, intel_ioat_skx, intel_ioat_icx, intel_idxd_spr, intel_ntb_skx, octeontx2_dma] # global dict ethernet devices present. Dictionary indexed by PCI address. # Each device within this is itself a dictionary of device properties From patchwork Tue Jul 21 09:51:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74547 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EB57DA0526; Tue, 21 Jul 2020 11:53:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EB30B1C002; Tue, 21 Jul 2020 11:53:12 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 07D2F1BFE9 for ; Tue, 21 Jul 2020 11:53:08 +0200 (CEST) IronPort-SDR: Ng9Iur42Dk1i196WtAQYgDl8PYbcWmweidMTYhL6e+Tdpm06BzHsvQgmoPYOeRl2tGUfbiG+St 2871tu7SXgCg== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191210" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191210" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:53:08 -0700 IronPort-SDR: mWwOIrN/IsnwIsZP4WewmBzOhhda1BeK/OLWkFI8tWjMdM45KAs5mP0mrOcai1e5+3xHQuDgjH Lswv/s47LBAg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488023863" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:53:07 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:28 +0100 Message-Id: <20200721095140.719297-9-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 08/20] raw/ioat: add skeleton for vfio/uio based DSA device X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add in the basic probe/remove skeleton code for DSA devices which are bound directly to vfio or uio driver. The kernel module for supporting these uses the "idxd" name, so that name is used as function and file prefix to avoid conflict with existing "ioat" prefixed functions. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/idxd_pci.c | 56 +++++++++++++++++++++++++++++++++ drivers/raw/ioat/ioat_private.h | 27 ++++++++++++++++ drivers/raw/ioat/ioat_rawdev.c | 9 +----- drivers/raw/ioat/meson.build | 6 ++-- 4 files changed, 88 insertions(+), 10 deletions(-) create mode 100644 drivers/raw/ioat/idxd_pci.c create mode 100644 drivers/raw/ioat/ioat_private.h diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c new file mode 100644 index 000000000..f6af9d33a --- /dev/null +++ b/drivers/raw/ioat/idxd_pci.c @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include + +#include "ioat_private.h" + +#define IDXD_VENDOR_ID 0x8086 +#define IDXD_DEVICE_ID_SPR 0x0B25 + +#define IDXD_PMD_RAWDEV_NAME_PCI rawdev_idxd_pci + +const struct rte_pci_id pci_id_idxd_map[] = { + { RTE_PCI_DEVICE(IDXD_VENDOR_ID, IDXD_DEVICE_ID_SPR) }, + { .vendor_id = 0, /* sentinel */ }, +}; + +static int +idxd_rawdev_probe_pci(struct rte_pci_driver *drv, struct rte_pci_device *dev) +{ + int ret = 0; + char name[32]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + IOAT_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); + dev->device.driver = &drv->driver; + + return ret; +} + +static int +idxd_rawdev_remove_pci(struct rte_pci_device *dev) +{ + char name[32]; + int ret = 0; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + + IOAT_PMD_INFO("Closing %s on NUMA node %d", name, dev->device.numa_node); + + return ret; +} + +struct rte_pci_driver idxd_pmd_drv_pci = { + .id_table = pci_id_idxd_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING, + .probe = idxd_rawdev_probe_pci, + .remove = idxd_rawdev_remove_pci, +}; + +RTE_PMD_REGISTER_PCI(IDXD_PMD_RAWDEV_NAME_PCI, idxd_pmd_drv_pci); +RTE_PMD_REGISTER_PCI_TABLE(IDXD_PMD_RAWDEV_NAME_PCI, pci_id_idxd_map); +RTE_PMD_REGISTER_KMOD_DEP(IDXD_PMD_RAWDEV_NAME_PCI, + "* igb_uio | uio_pci_generic | vfio-pci"); + diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h new file mode 100644 index 000000000..d87d4d055 --- /dev/null +++ b/drivers/raw/ioat/ioat_private.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _IOAT_PRIVATE_H_ +#define _IOAT_PRIVATE_H_ + +/** + * @file idxd_private.h + * + * Private data structures for the idxd/DSA part of ioat device driver + * + * @warning + * @b EXPERIMENTAL: these structures and APIs may change without prior notice + */ + +extern int ioat_pmd_logtype; + +#define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ + ioat_pmd_logtype, "%s(): " fmt "\n", __func__, ##args) + +#define IOAT_PMD_DEBUG(fmt, args...) IOAT_PMD_LOG(DEBUG, fmt, ## args) +#define IOAT_PMD_INFO(fmt, args...) IOAT_PMD_LOG(INFO, fmt, ## args) +#define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) +#define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) + +#endif /* _IOAT_PRIVATE_H_ */ diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index 11c6a57e1..8f9c8b56f 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -10,6 +10,7 @@ #include "rte_ioat_rawdev.h" #include "ioat_spec.h" +#include "ioat_private.h" static struct rte_pci_driver ioat_pmd_drv; @@ -29,14 +30,6 @@ static struct rte_pci_driver ioat_pmd_drv; RTE_LOG_REGISTER(ioat_pmd_logtype, rawdev.ioat, INFO); -#define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ - ioat_pmd_logtype, "%s(): " fmt "\n", __func__, ##args) - -#define IOAT_PMD_DEBUG(fmt, args...) IOAT_PMD_LOG(DEBUG, fmt, ## args) -#define IOAT_PMD_INFO(fmt, args...) IOAT_PMD_LOG(INFO, fmt, ## args) -#define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) -#define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) - #define DESC_SZ sizeof(struct rte_ioat_generic_hw_desc) #define COMPLETION_SZ sizeof(__m128i) diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build index 06636f8a9..3529635e9 100644 --- a/drivers/raw/ioat/meson.build +++ b/drivers/raw/ioat/meson.build @@ -3,8 +3,10 @@ build = dpdk_conf.has('RTE_ARCH_X86') reason = 'only supported on x86' -sources = files('ioat_rawdev.c', - 'ioat_rawdev_test.c') +sources = files( + 'idxd_pci.c', + 'ioat_rawdev.c', + 'ioat_rawdev_test.c') deps += ['rawdev', 'bus_pci', 'mbuf'] install_headers('rte_ioat_rawdev.h', From patchwork Tue Jul 21 09:51:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74548 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CF523A0526; Tue, 21 Jul 2020 11:53:29 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 626031C02A; Tue, 21 Jul 2020 11:53:17 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 91C4B1C02A for ; Tue, 21 Jul 2020 11:53:13 +0200 (CEST) IronPort-SDR: Vv9yHgdfCJnqwLa+8LfCtpFz0udLw0ohK2ZBapNwAIIoWh1HWl64PIz7J3WV6Ga5m97GoV/QJm wj0XFdhMLV0Q== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191214" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191214" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:53:13 -0700 IronPort-SDR: BXkmIKEHRNcLmhO/VJmPs9y62VOzEbDgJ9rYN+YVvcsvjMVJ+vhgHxTlSpRlaEG5WPrAI7Ft1D obnWSVlvSxIw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488023880" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:53:11 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:29 +0100 Message-Id: <20200721095140.719297-10-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 09/20] raw/ioat: add vdev probe for DSA/idxd devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Kevin Laatz The Intel DSA devices can be exposed to userspace via kernel driver, so can be used without having to bind them to vfio/uio. Therefore we add support for using those kernel-configured devices as vdevs, taking as parameter the individual HW work queue to be used by the vdev. Signed-off-by: Kevin Laatz Signed-off-by: Bruce Richardson --- drivers/raw/ioat/idxd_vdev.c | 127 +++++++++++++++++++++++++++++++++++ drivers/raw/ioat/meson.build | 6 +- 2 files changed, 132 insertions(+), 1 deletion(-) create mode 100644 drivers/raw/ioat/idxd_vdev.c diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c new file mode 100644 index 000000000..73fce6d87 --- /dev/null +++ b/drivers/raw/ioat/idxd_vdev.c @@ -0,0 +1,127 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include +#include +#include +#include + +#include "ioat_private.h" + +/** Name of the device driver */ +#define IDXD_PMD_RAWDEV_NAME rawdev_idxd +/* takes a work queue(WQ) as parameter */ +#define IDXD_ARG_WQ "wq" + +extern struct rte_vdev_driver idxd_rawdev_drv_vdev; + +static const char * const valid_args[] = { + IDXD_ARG_WQ, + NULL +}; + +struct idxd_vdev_args { + uint8_t device_id; + uint8_t wq_id; +}; + +static int +idxd_rawdev_parse_wq(const char *key __rte_unused, const char *value, + void *extra_args) +{ + struct idxd_vdev_args *args = (struct idxd_vdev_args *)extra_args; + int dev, wq, bytes = -1; + int read = sscanf(value, "%d.%d%n", &dev, &wq, &bytes); + + if (read != 2 || bytes != (int)strlen(value)) { + IOAT_PMD_ERR("Error parsing work-queue id. Must be in . format"); + return -EINVAL; + } + + if (dev >= UINT8_MAX || wq >= UINT8_MAX) { + IOAT_PMD_ERR("Device or work queue id out of range"); + return -EINVAL; + } + + args->device_id = dev; + args->wq_id = wq; + + return 0; +} + +static int +idxd_vdev_parse_params(struct rte_kvargs *kvlist, struct idxd_vdev_args *args) +{ + if (rte_kvargs_count(kvlist, IDXD_ARG_WQ) == 1) { + if (rte_kvargs_process(kvlist, IDXD_ARG_WQ, + &idxd_rawdev_parse_wq, args) < 0) { + IOAT_PMD_ERR("Error parsing %s", IDXD_ARG_WQ); + goto free; + } + } else { + IOAT_PMD_ERR("%s is a mandatory arg", IDXD_ARG_WQ); + return -EINVAL; + } + + return 0; + +free: + if (kvlist) + rte_kvargs_free(kvlist); + return -EINVAL; +} + +static int +idxd_rawdev_probe_vdev(struct rte_vdev_device *vdev) +{ + struct rte_kvargs *kvlist; + struct idxd_vdev_args vdev_args; + const char *name; + int ret = 0; + + name = rte_vdev_device_name(vdev); + if (name == NULL) + return -EINVAL; + + IOAT_PMD_INFO("Initializing pmd_idxd for %s", name); + + kvlist = rte_kvargs_parse(rte_vdev_device_args(vdev), valid_args); + if (kvlist == NULL) { + IOAT_PMD_ERR("Invalid kvargs key"); + return -EINVAL; + } + + ret = idxd_vdev_parse_params(kvlist, &vdev_args); + if (ret) { + IOAT_PMD_ERR("Failed to parse kvargs"); + return -EINVAL; + } + + vdev->device.driver = &idxd_rawdev_drv_vdev.driver; + + return 0; +} + +static int +idxd_rawdev_remove_vdev(struct rte_vdev_device *vdev) +{ + const char *name; + + name = rte_vdev_device_name(vdev); + if (name == NULL) + return -EINVAL; + + IOAT_PMD_INFO("Remove DSA vdev %p", name); + + return 0; +} + +struct rte_vdev_driver idxd_rawdev_drv_vdev = { + .probe = idxd_rawdev_probe_vdev, + .remove = idxd_rawdev_remove_vdev, +}; + +RTE_PMD_REGISTER_VDEV(IDXD_PMD_RAWDEV_NAME, idxd_rawdev_drv_vdev); +RTE_PMD_REGISTER_PARAM_STRING(IDXD_PMD_RAWDEV_NAME, + "wq="); diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build index 3529635e9..b343b7367 100644 --- a/drivers/raw/ioat/meson.build +++ b/drivers/raw/ioat/meson.build @@ -5,9 +5,13 @@ build = dpdk_conf.has('RTE_ARCH_X86') reason = 'only supported on x86' sources = files( 'idxd_pci.c', + 'idxd_vdev.c', 'ioat_rawdev.c', 'ioat_rawdev_test.c') -deps += ['rawdev', 'bus_pci', 'mbuf'] +deps += ['bus_pci', + 'bus_vdev', + 'mbuf', + 'rawdev'] install_headers('rte_ioat_rawdev.h', 'rte_ioat_rawdev_fns.h') From patchwork Tue Jul 21 09:51:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74549 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4FE88A0526; Tue, 21 Jul 2020 11:56:06 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 365731BFFE; Tue, 21 Jul 2020 11:56:06 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id B8B881BFF9 for ; Tue, 21 Jul 2020 11:56:04 +0200 (CEST) IronPort-SDR: HyjjM6CFnWr+hbGA8MoK5mBGU1iPpxMHQK/D080f4sjgrNhDxRqoUtZ+P40FfYa2qCE5J9eBOn uEySGq0ux2yg== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191433" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191433" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:03 -0700 IronPort-SDR: IdakRBhEXu6LccmYX/9GdluEdcP92f2MnDB3YnJ39NBs5GEpPa0ai2cn+uEwtF4nZZ7mGsAMK/ oo5H5bcXz+Yw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024418" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:02 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:30 +0100 Message-Id: <20200721095140.719297-11-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 10/20] raw/ioat: create rawdev instances on idxd PCI probe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When a matching device is found via PCI probe create a rawdev instance for each queue on the hardware. Use empty self-test function for these devices so that the overall rawdev_autotest does not report failures. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/idxd_pci.c | 235 ++++++++++++++++++++++++- drivers/raw/ioat/ioat_common.c | 62 +++++++ drivers/raw/ioat/ioat_private.h | 37 ++++ drivers/raw/ioat/ioat_rawdev_test.c | 7 + drivers/raw/ioat/ioat_spec.h | 52 ++++++ drivers/raw/ioat/meson.build | 1 + drivers/raw/ioat/rte_ioat_rawdev_fns.h | 35 +++- 7 files changed, 426 insertions(+), 3 deletions(-) create mode 100644 drivers/raw/ioat/ioat_common.c diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c index f6af9d33a..11c07efaa 100644 --- a/drivers/raw/ioat/idxd_pci.c +++ b/drivers/raw/ioat/idxd_pci.c @@ -3,8 +3,10 @@ */ #include +#include #include "ioat_private.h" +#include "ioat_spec.h" #define IDXD_VENDOR_ID 0x8086 #define IDXD_DEVICE_ID_SPR 0x0B25 @@ -16,17 +18,244 @@ const struct rte_pci_id pci_id_idxd_map[] = { { .vendor_id = 0, /* sentinel */ }, }; +static inline int +idxd_pci_dev_command(struct idxd_rawdev *idxd, enum rte_idxd_cmds command) +{ + uint8_t err_code; + uint16_t qid = idxd->qid; + int i = 0; + + if (command >= idxd_disable_wq && command <= idxd_reset_wq) + qid = (1 << qid); + rte_spinlock_lock(&idxd->u.pci->lk); + idxd->u.pci->regs->cmd = (command << IDXD_CMD_SHIFT) | qid; + + do { + rte_pause(); + err_code = idxd->u.pci->regs->cmdstatus; + if (++i >= 1000) { + IOAT_PMD_ERR("Timeout waiting for command response from HW"); + rte_spinlock_unlock(&idxd->u.pci->lk); + return err_code; + } + } while (idxd->u.pci->regs->cmdstatus & CMDSTATUS_ACTIVE_MASK); + rte_spinlock_unlock(&idxd->u.pci->lk); + + return err_code & CMDSTATUS_ERR_MASK; +} + +static int +idxd_is_wq_enabled(struct idxd_rawdev *idxd) +{ + uint32_t state = (idxd->u.pci->wq_regs[idxd->qid].wqcfg[WQ_STATE_IDX] >> WQ_STATE_SHIFT); + return (state & WQ_STATE_MASK) == 0x1; +} + +static const struct rte_rawdev_ops idxd_pci_ops = { + .dev_selftest = idxd_rawdev_test +}; + +/* each portal uses 4 x 4k pages */ +#define IDXD_PORTAL_SIZE (4096 * 4) + +static int +init_pci_device(struct rte_pci_device *dev, struct idxd_rawdev *idxd) +{ + struct idxd_pci_common *pci; + uint8_t nb_groups, nb_engines, nb_wqs; + uint16_t grp_offset, wq_offset; /* how far into bar0 the regs are */ + uint16_t wq_size, total_wq_size; + uint8_t lg2_max_batch, lg2_max_copy_size; + unsigned int i, err_code; + + pci = malloc(sizeof(*pci)); + if (pci == NULL) { + IOAT_PMD_ERR("%s: Can't allocate memory", __func__); + goto err; + } + rte_spinlock_init(&pci->lk); + + /* assign the bar registers, and then configure device */ + pci->regs = dev->mem_resource[0].addr; + grp_offset = (uint16_t)pci->regs->offsets[0]; + pci->grp_regs = RTE_PTR_ADD(pci->regs, grp_offset * 0x100); + wq_offset = (uint16_t)(pci->regs->offsets[0] >> 16); + pci->wq_regs = RTE_PTR_ADD(pci->regs, wq_offset * 0x100); + pci->portals = dev->mem_resource[2].addr; + + /* sanity check device status */ + if (pci->regs->gensts & GENSTS_DEV_STATE_MASK) { + /* need function-level-reset (FLR) or is enabled */ + IOAT_PMD_ERR("Device status is not disabled, cannot init"); + goto err; + } + if (pci->regs->cmdstatus & CMDSTATUS_ACTIVE_MASK) { + /* command in progress */ + IOAT_PMD_ERR("Device has a command in progress, cannot init"); + goto err; + } + + /* read basic info about the hardware for use when configuring */ + nb_groups = (uint8_t)pci->regs->grpcap; + nb_engines = (uint8_t)pci->regs->engcap; + nb_wqs = (uint8_t)(pci->regs->wqcap >> 16); + total_wq_size = (uint16_t)pci->regs->wqcap; + lg2_max_copy_size = (uint8_t)(pci->regs->gencap >> 16) & 0x1F; + lg2_max_batch = (uint8_t)(pci->regs->gencap >> 21) & 0x0F; + + IOAT_PMD_DEBUG("nb_groups = %u, nb_engines = %u, nb_wqs = %u", + nb_groups, nb_engines, nb_wqs); + + /* zero out any old config */ + for (i = 0; i < nb_groups; i++) { + pci->grp_regs[i].grpengcfg = 0; + pci->grp_regs[i].grpwqcfg[0] = 0; + } + for (i = 0; i < nb_wqs; i++) + pci->wq_regs[i].wqcfg[0] = 0; + + /* put each engine into a separate group to avoid reordering */ + if (nb_groups > nb_engines) + nb_groups = nb_engines; + if (nb_groups < nb_engines) + nb_engines = nb_groups; + + /* assign engines to groups, round-robin style */ + for (i = 0; i < nb_engines; i++) { + IOAT_PMD_DEBUG("Assigning engine %u to group %u", + i, i % nb_groups); + pci->grp_regs[i % nb_groups].grpengcfg |= (1ULL << i); + } + + /* now do the same for queues and give work slots to each queue */ + wq_size = total_wq_size / nb_wqs; + IOAT_PMD_DEBUG("Work queue size = %u, max batch = 2^%u, max copy = 2^%u", + wq_size, lg2_max_batch, lg2_max_copy_size); + for (i = 0; i < nb_wqs; i++) { + /* add engine "i" to a group */ + IOAT_PMD_DEBUG("Assigning work queue %u to group %u", + i, i % nb_groups); + pci->grp_regs[i % nb_groups].grpwqcfg[0] |= (1ULL << i); + /* now configure it, in terms of size, max batch, mode */ + pci->wq_regs[i].wqcfg[0] = wq_size; + pci->wq_regs[i].wqcfg[2] = 0x11; /* TODO: use defines - dedicated mode, priority 1 */ + pci->wq_regs[i].wqcfg[3] = (lg2_max_batch << 5) | + lg2_max_copy_size; + } + + /* dump the group configuration to output */ + for (i = 0; i < nb_groups; i++) { + IOAT_PMD_DEBUG("## Group %d", i); + IOAT_PMD_DEBUG(" GRPWQCFG: %"PRIx64, pci->grp_regs[i].grpwqcfg[0]); + IOAT_PMD_DEBUG(" GRPENGCFG: %"PRIx64, pci->grp_regs[i].grpengcfg); + IOAT_PMD_DEBUG(" GRPFLAGS: %"PRIx32, pci->grp_regs[i].grpflags); + } + + idxd->u.pci = pci; + idxd->max_batches = wq_size; + + /* enable the device itself */ + err_code = idxd_pci_dev_command(idxd, idxd_enable_dev); + if (err_code) { + IOAT_PMD_ERR("Error enabling device: code %#x", err_code); + return err_code; + } + IOAT_PMD_DEBUG("IDXD Device enabled OK"); + + return nb_wqs; + +err: + free(pci); + return -1; +} + static int idxd_rawdev_probe_pci(struct rte_pci_driver *drv, struct rte_pci_device *dev) { - int ret = 0; + struct idxd_rawdev idxd = {0}; + uint8_t nb_wqs; + int qid, ret = 0; char name[32]; rte_pci_device_name(&dev->addr, name, sizeof(name)); IOAT_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); dev->device.driver = &drv->driver; - return ret; + ret = init_pci_device(dev, &idxd); + if (ret < 0) { + IOAT_PMD_ERR("Error initializing PCI hardware"); + return ret; + } + nb_wqs = (uint8_t)ret; + + /* set up one device for each queue */ + for (qid = 0; qid < nb_wqs; qid++) { + char qname[32]; + + /* add the queue number to each device name */ + snprintf(qname, sizeof(qname), "%s-q%d", name, qid); + idxd.qid = qid; + idxd.public.portal = RTE_PTR_ADD(idxd.u.pci->portals, + qid * IDXD_PORTAL_SIZE); + if (idxd_is_wq_enabled(&idxd)) + IOAT_PMD_ERR("Error, WQ %u seems enabled", qid); + ret = idxd_rawdev_create(qname, &dev->device, + &idxd, &idxd_pci_ops); + if (ret != 0){ + IOAT_PMD_ERR("Failed to create rawdev %s", name); + if (qid == 0) /* if no devices using this, free pci */ + free(idxd.u.pci); + return ret; + } + } + + return 0; +} + +static int +idxd_rawdev_destroy(const char *name) +{ + int ret; + uint8_t err_code; + struct rte_rawdev *rdev; + struct idxd_rawdev *idxd; + + if (!name) { + IOAT_PMD_ERR("Invalid device name"); + return -EINVAL; + } + + rdev = rte_rawdev_pmd_get_named_dev(name); + if (!rdev) { + IOAT_PMD_ERR("Invalid device name (%s)", name); + return -EINVAL; + } + + idxd = rdev->dev_private; + + /* disable the device */ + err_code = idxd_pci_dev_command(idxd, idxd_disable_dev); + if (err_code) { + IOAT_PMD_ERR("Error disabling device: code %#x", err_code); + return err_code; + } + IOAT_PMD_DEBUG("IDXD Device disabled OK"); + + /* free device memory */ + if (rdev->dev_private != NULL) { + IOAT_PMD_DEBUG("Freeing device driver memory"); + rdev->dev_private = NULL; + rte_free(idxd->public.batch_ring); + rte_free(idxd->public.hdl_ring); + rte_memzone_free(idxd->mz); + } + + /* rte_rawdev_close is called by pmd_release */ + ret = rte_rawdev_pmd_release(rdev); + if (ret) + IOAT_PMD_DEBUG("Device cleanup failed"); + + return 0; } static int @@ -39,6 +268,8 @@ idxd_rawdev_remove_pci(struct rte_pci_device *dev) IOAT_PMD_INFO("Closing %s on NUMA node %d", name, dev->device.numa_node); + ret = idxd_rawdev_destroy(name); + return ret; } diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c new file mode 100644 index 000000000..4397be886 --- /dev/null +++ b/drivers/raw/ioat/ioat_common.c @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include +#include + +#include "ioat_private.h" + +int +idxd_rawdev_create(const char *name, struct rte_device *dev, + const struct idxd_rawdev *base_idxd, + const struct rte_rawdev_ops *ops) +{ + struct idxd_rawdev *idxd; + struct rte_rawdev *rawdev = NULL; + const struct rte_memzone *mz = NULL; + char mz_name[RTE_MEMZONE_NAMESIZE]; + int ret = 0; + + if (!name) { + IOAT_PMD_ERR("Invalid name of the device!"); + ret = -EINVAL; + goto cleanup; + } + + /* Allocate device structure */ + rawdev = rte_rawdev_pmd_allocate(name, sizeof(struct idxd_rawdev), + dev->numa_node); + if (rawdev == NULL) { + IOAT_PMD_ERR("Unable to allocate raw device"); + ret = -ENOMEM; + goto cleanup; + } + + snprintf(mz_name, sizeof(mz_name), "rawdev%u_private", rawdev->dev_id); + mz = rte_memzone_reserve(mz_name, sizeof(struct idxd_rawdev), + dev->numa_node, RTE_MEMZONE_IOVA_CONTIG); + if (mz == NULL) { + IOAT_PMD_ERR("Unable to reserve memzone for private data\n"); + ret = -ENOMEM; + goto cleanup; + } + rawdev->dev_private = mz->addr; + rawdev->dev_ops = ops; + rawdev->device = dev; + rawdev->driver_name = RTE_STR(IOAT_PMD_RAWDEV_NAME); + + idxd = rawdev->dev_private; + *idxd = *base_idxd; /* copy over the main fields already passed in */ + idxd->rawdev = rawdev; + idxd->mz = mz; + + return 0; + +cleanup: + if (rawdev) + rte_rawdev_pmd_release(rawdev); + + return ret; +} + diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h index d87d4d055..2ddaddc37 100644 --- a/drivers/raw/ioat/ioat_private.h +++ b/drivers/raw/ioat/ioat_private.h @@ -14,6 +14,10 @@ * @b EXPERIMENTAL: these structures and APIs may change without prior notice */ +#include +#include +#include "rte_ioat_rawdev.h" + extern int ioat_pmd_logtype; #define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ @@ -24,4 +28,37 @@ extern int ioat_pmd_logtype; #define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) #define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) +struct idxd_pci_common { + rte_spinlock_t lk; + volatile struct rte_idxd_bar0 *regs; + volatile struct rte_idxd_wqcfg *wq_regs; + volatile struct rte_idxd_grpcfg *grp_regs; + volatile void *portals; +}; + +struct idxd_rawdev { + struct rte_idxd_rawdev public; /* the public members, must be first */ + + struct rte_rawdev *rawdev; + const struct rte_memzone *mz; + uint8_t qid; + uint16_t max_batches; + + union { + struct { + struct accfg_ctx *ctx; + struct accfg_device *device; + struct accfg_wq *wq; + } vdev; + + struct idxd_pci_common *pci; + } u; +}; + +extern int idxd_rawdev_create(const char *name, struct rte_device *dev, + const struct idxd_rawdev *idxd, + const struct rte_rawdev_ops *ops); + +extern int idxd_rawdev_test(uint16_t dev_id); + #endif /* _IOAT_PRIVATE_H_ */ diff --git a/drivers/raw/ioat/ioat_rawdev_test.c b/drivers/raw/ioat/ioat_rawdev_test.c index e5b50ae9f..b208b8c19 100644 --- a/drivers/raw/ioat/ioat_rawdev_test.c +++ b/drivers/raw/ioat/ioat_rawdev_test.c @@ -7,6 +7,7 @@ #include #include "rte_rawdev.h" #include "rte_ioat_rawdev.h" +#include "ioat_private.h" #define MAX_SUPPORTED_RAWDEVS 64 #define TEST_SKIPPED 77 @@ -252,3 +253,9 @@ ioat_rawdev_test(uint16_t dev_id) free(ids); return -1; } + +int +idxd_rawdev_test(uint16_t dev_id __rte_unused) +{ + return 0; +} diff --git a/drivers/raw/ioat/ioat_spec.h b/drivers/raw/ioat/ioat_spec.h index 9645e16d4..b865fc0ec 100644 --- a/drivers/raw/ioat/ioat_spec.h +++ b/drivers/raw/ioat/ioat_spec.h @@ -268,6 +268,58 @@ union rte_ioat_hw_desc { struct rte_ioat_pq_update_hw_desc pq_update; }; +/*** Definitions for Intel(R) Data Streaming Accelerator Follow ***/ + +#define IDXD_CMD_SHIFT 20 +enum rte_idxd_cmds { + idxd_enable_dev = 1, + idxd_disable_dev, + idxd_drain_all, + idxd_abort_all, + idxd_reset_device, + idxd_enable_wq, + idxd_disable_wq, + idxd_drain_wq, + idxd_abort_wq, + idxd_reset_wq, +}; + +/* General bar0 registers */ +struct rte_idxd_bar0 { + uint32_t version __rte_cache_aligned; /* offset 0x00 */ + uint64_t gencap __rte_aligned(0x10); /* offset 0x10 */ + uint64_t wqcap __rte_aligned(0x10); /* offset 0x20 */ + uint64_t grpcap __rte_aligned(0x10); /* offset 0x30 */ + uint64_t engcap __rte_aligned(0x08); /* offset 0x38 */ + uint64_t opcap __rte_aligned(0x10); /* offset 0x40 */ + uint64_t offsets[2] __rte_aligned(0x20); /* offset 0x60 */ + uint32_t gencfg __rte_aligned(0x20); /* offset 0x80 */ + uint32_t genctrl __rte_aligned(0x08); /* offset 0x88 */ + uint32_t gensts __rte_aligned(0x10); /* offset 0x90 */ + uint32_t intcause __rte_aligned(0x08); /* offset 0x98 */ + uint32_t cmd __rte_aligned(0x10); /* offset 0xA0 */ + uint32_t cmdstatus __rte_aligned(0x08); /* offset 0xA8 */ + uint64_t swerror[4] __rte_aligned(0x20); /* offset 0xC0 */ +}; + +struct rte_idxd_wqcfg { + uint32_t wqcfg[8] __rte_aligned(32); /* 32-byte register */ +}; +#define WQ_STATE_IDX 6 /* 7th 32-bit value in array */ +#define WQ_STATE_SHIFT 30 +#define WQ_STATE_MASK 0x3 + +struct rte_idxd_grpcfg { + uint64_t grpwqcfg[4] __rte_cache_aligned; /* 64-byte register set */ + uint64_t grpengcfg; /* offset 32 */ + uint32_t grpflags; /* offset 40 */ +}; + +#define GENSTS_DEV_STATE_MASK 0x03 +#define CMDSTATUS_ACTIVE_SHIFT 31 +#define CMDSTATUS_ACTIVE_MASK (1 << 31) +#define CMDSTATUS_ERR_MASK 0xFF + #ifdef __cplusplus } #endif diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build index b343b7367..5eff76a1a 100644 --- a/drivers/raw/ioat/meson.build +++ b/drivers/raw/ioat/meson.build @@ -6,6 +6,7 @@ reason = 'only supported on x86' sources = files( 'idxd_pci.c', 'idxd_vdev.c', + 'ioat_common.c', 'ioat_rawdev.c', 'ioat_rawdev_test.c') deps += ['bus_pci', diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h b/drivers/raw/ioat/rte_ioat_rawdev_fns.h index 0cee6b1b0..aca91dd4f 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h +++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h @@ -39,9 +39,20 @@ struct rte_ioat_generic_hw_desc { /** * @internal - * Structure representing a device instance + * Identify the data path to use. + * Must be first field of rte_ioat_rawdev and rte_idxd_rawdev structs + */ +enum rte_ioat_dev_type { + RTE_IOAT_DEV, + RTE_IDXD_DEV, +}; + +/** + * @internal + * Structure representing an IOAT device instance * Structure representing a device instance */ struct rte_ioat_rawdev { + enum rte_ioat_dev_type type; struct rte_rawdev *rawdev; const struct rte_memzone *mz; const struct rte_memzone *desc_mz; @@ -77,6 +88,28 @@ struct rte_ioat_rawdev { #define RTE_IOAT_CHANSTS_HALTED 0x3 #define RTE_IOAT_CHANSTS_ARMED 0x4 +/** + * @internal + * Structure representing an IDXD device instance + */ +struct rte_idxd_rawdev { + enum rte_ioat_dev_type type; + void *portal; /* address to write the batch descriptor */ + + /* counters to track the batches and the individual op handles */ + uint16_t batch_ring_sz; /* size of batch ring */ + uint16_t hdl_ring_sz; /* size of the user hdl ring */ + + uint16_t next_batch; /* where we write descriptor ops */ + uint16_t next_completed; /* batch where we read completions */ + uint16_t next_ret_hdl; /* the next user hdl to return */ + uint16_t last_completed_hdl; /* the last user hdl that has completed */ + uint16_t next_free_hdl; /* where the handle for next op will go */ + + struct rte_idxd_user_hdl *hdl_ring; + struct rte_idxd_desc_batch *batch_ring; +}; + /** * Enqueue a copy operation onto the ioat device */ From patchwork Tue Jul 21 09:51:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74550 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 22E53A0526; Tue, 21 Jul 2020 11:56:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0ABCD1C025; Tue, 21 Jul 2020 11:56:10 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 8C7BB1C002 for ; Tue, 21 Jul 2020 11:56:06 +0200 (CEST) IronPort-SDR: +sfUcKbeBjIueg1Xd66reC+7QfoKzZsYCa1+zsGxZiHrzacVzLWf2HMhn76dwopKtJvZcel8YG DFFoWeD19qhw== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191436" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191436" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:06 -0700 IronPort-SDR: 67GkL+qgTxOD6/AikW0Rt5iwyPa+1i8SjM3NVAjFcIB3NWX1M23ZMUNmotFQCU0JpzaCQYDokU nWZHfAKVMIvg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024427" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:04 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:31 +0100 Message-Id: <20200721095140.719297-12-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 11/20] raw/ioat: create rawdev instances for idxd vdevs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Kevin Laatz For each vdev (DSA work queue) instance, create a rawdev instance. Since the vdev support depends upon the accel-config libraries, make the vdev support conditional upon that in meson.build. Signed-off-by: Kevin Laatz Signed-off-by: Bruce Richardson --- drivers/raw/ioat/idxd_vdev.c | 163 ++++++++++++++++++++++++++++++++++- drivers/raw/ioat/meson.build | 9 +- 2 files changed, 169 insertions(+), 3 deletions(-) diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c index 73fce6d87..e81bd7326 100644 --- a/drivers/raw/ioat/idxd_vdev.c +++ b/drivers/raw/ioat/idxd_vdev.c @@ -2,6 +2,12 @@ * Copyright(c) 2020 Intel Corporation */ +#include +#include +#include +#include + +#include #include #include #include @@ -26,6 +32,79 @@ struct idxd_vdev_args { uint8_t wq_id; }; +static const struct rte_rawdev_ops idxd_vdev_ops = { + .dev_selftest = idxd_rawdev_test, +}; + +static void * +idxd_vdev_mmap_wq(struct accfg_device *dsa_dev, struct accfg_wq *wq) +{ + void *addr; + int major, minor; + char path[PATH_MAX]; + int fd; + + major = accfg_device_get_cdev_major(dsa_dev); + if (major < 0) { + IOAT_PMD_ERR("Invalid major version %d", major); + return NULL; + } + + minor = accfg_wq_get_cdev_minor(wq); + if (minor < 0) { + IOAT_PMD_ERR("Invalid minor version %d", minor); + return NULL; + } + + snprintf(path, sizeof(path), "/dev/char/%u:%u", major, minor); + fd = open(path, O_RDWR); + if (fd < 0) { + IOAT_PMD_ERR("Failed to open device path"); + return NULL; + } + + addr = mmap(NULL, 0x1000, PROT_WRITE, MAP_SHARED | MAP_POPULATE, fd, 0); + if (addr == MAP_FAILED) { + IOAT_PMD_ERR("Failed to mmap device"); + return NULL; + } + + return addr; +} + +static int +idxd_rawdev_vdev_config(struct idxd_rawdev *idxd, struct idxd_vdev_args *args) +{ + struct accfg_ctx *dsa_ctx; + struct accfg_device *dsa_dev; + struct accfg_wq *dsa_wq; + int ret; + + ret = accfg_new(&dsa_ctx); + if (ret < 0) { + IOAT_PMD_ERR("Failed to create device context"); + ret = -ENOMEM; + } + + dsa_dev = accfg_ctx_device_get_by_id(dsa_ctx, args->device_id); + if (dsa_dev == NULL) { + IOAT_PMD_ERR("device not found: %u", args->device_id); + return -1; + } + dsa_wq = accfg_device_wq_get_by_id(dsa_dev, args->wq_id); + if (dsa_wq == NULL) { + IOAT_PMD_ERR("queue not found: %u", args->wq_id); + return -1; + } + + idxd->u.vdev.ctx = dsa_ctx; + idxd->u.vdev.device = dsa_dev; + idxd->u.vdev.wq = dsa_wq; + idxd->max_batches = accfg_wq_get_size(dsa_wq); + + return ret; +} + static int idxd_rawdev_parse_wq(const char *key __rte_unused, const char *value, void *extra_args) @@ -76,6 +155,7 @@ static int idxd_rawdev_probe_vdev(struct rte_vdev_device *vdev) { struct rte_kvargs *kvlist; + struct idxd_rawdev idxd = {0}; struct idxd_vdev_args vdev_args; const char *name; int ret = 0; @@ -98,15 +178,52 @@ idxd_rawdev_probe_vdev(struct rte_vdev_device *vdev) return -EINVAL; } + ret = idxd_rawdev_vdev_config(&idxd, &vdev_args); + if (ret) { + IOAT_PMD_ERR("Failed to init vdev context"); + return ret; + } + + idxd.qid = vdev_args.wq_id; + idxd.public.portal = idxd_vdev_mmap_wq(idxd.u.vdev.device, idxd.u.vdev.wq); + if (idxd.public.portal == NULL) { + IOAT_PMD_ERR("WQ mmap failed"); + return -ENOMEM; + } + vdev->device.driver = &idxd_rawdev_drv_vdev.driver; + ret = idxd_rawdev_create(name, &vdev->device, &idxd, &idxd_vdev_ops); + if (ret) { + IOAT_PMD_ERR("Failed to create rawdev %s", name); + return ret; + } + + /* enable the device itself */ + if (accfg_device_is_active(idxd.u.vdev.device)) { + IOAT_PMD_INFO("Device %s already enabled", + accfg_device_get_devname(idxd.u.vdev.device)); + } else { + ret = accfg_device_enable(idxd.u.vdev.device); + if (ret) { + IOAT_PMD_ERR("Error enabling device %s", + accfg_device_get_devname(idxd.u.vdev.device)); + return -1; + } + IOAT_PMD_DEBUG("Enabling device %s OK", + accfg_device_get_devname(idxd.u.vdev.device)); + } + return 0; } static int idxd_rawdev_remove_vdev(struct rte_vdev_device *vdev) { + struct idxd_rawdev *idxd; const char *name; + struct rte_rawdev *rdev; + int ret = 0; name = rte_vdev_device_name(vdev); if (name == NULL) @@ -114,7 +231,51 @@ idxd_rawdev_remove_vdev(struct rte_vdev_device *vdev) IOAT_PMD_INFO("Remove DSA vdev %p", name); - return 0; + rdev = rte_rawdev_pmd_get_named_dev(name); + if (!rdev) { + IOAT_PMD_ERR("Invalid device name (%s)", name); + return -EINVAL; + } + + idxd = rdev->dev_private; + + /* disable the device */ + if (!accfg_device_is_active(idxd->u.vdev.device)) { + IOAT_PMD_ERR("Device %s already disabled", + accfg_device_get_devname(idxd->u.vdev.device)); + } + + ret = accfg_device_disable(idxd->u.vdev.device); + if (ret) { + IOAT_PMD_ERR("Not able to disable device %s", + accfg_device_get_devname(idxd->u.vdev.device)); + return ret; + } + IOAT_PMD_DEBUG("Disabling device %s OK", + accfg_device_get_devname(idxd->u.vdev.device)); + + /* free context and memory */ + if (rdev->dev_private != NULL) { + IOAT_PMD_DEBUG("Freeing device driver memory"); + rdev->dev_private = NULL; + accfg_unref(idxd->u.vdev.ctx); + + if (munmap(idxd->public.portal, 0x1000) < 0) { + IOAT_PMD_ERR("Error unmapping %s", + accfg_wq_get_devname(idxd->u.vdev.wq)); + ret = -errno; + } + + rte_free(idxd->public.batch_ring); + rte_free(idxd->public.hdl_ring); + + rte_memzone_free(idxd->mz); + } + + if (rte_rawdev_pmd_release(rdev)) + IOAT_PMD_ERR("Device cleanup failed"); + + return ret; } struct rte_vdev_driver idxd_rawdev_drv_vdev = { diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build index 5eff76a1a..a730953f8 100644 --- a/drivers/raw/ioat/meson.build +++ b/drivers/raw/ioat/meson.build @@ -5,14 +5,19 @@ build = dpdk_conf.has('RTE_ARCH_X86') reason = 'only supported on x86' sources = files( 'idxd_pci.c', - 'idxd_vdev.c', 'ioat_common.c', 'ioat_rawdev.c', 'ioat_rawdev_test.c') deps += ['bus_pci', - 'bus_vdev', 'mbuf', 'rawdev'] install_headers('rte_ioat_rawdev.h', 'rte_ioat_rawdev_fns.h') + +accfg_dep = dependency('libaccel-config', required: false) +if accfg_dep.found() + sources += files('idxd_vdev.c') + deps += ['bus_vdev'] + ext_deps += accfg_dep +endif From patchwork Tue Jul 21 09:51:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74551 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F339FA0526; Tue, 21 Jul 2020 11:56:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A26231C045; Tue, 21 Jul 2020 11:56:12 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 033C31C012 for ; Tue, 21 Jul 2020 11:56:08 +0200 (CEST) IronPort-SDR: 7uvBHcYb1sk4Dh2srfjorMi0pEgAn27PuCBVK9gUJ0nMn10llDbJMRrApVqSv1wPeAFbZc1iYd EocgH0WcmmHA== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191441" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191441" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:08 -0700 IronPort-SDR: xKpMF5u66dEj90w4nTvPYG39h9OK6TNlQNa/E3n2zl98cmqANXwPtQlTYEgdf+AWOll9Kfezvz Ozi9UiDVfaDw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024433" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:06 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:32 +0100 Message-Id: <20200721095140.719297-13-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 12/20] raw/ioat: add datapath data structures for idxd devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add in the relevant data structures for the data path for DSA devices. Also include a device dump function to output the status of each device. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/idxd_pci.c | 3 +- drivers/raw/ioat/idxd_vdev.c | 1 + drivers/raw/ioat/ioat_common.c | 34 +++++++++++ drivers/raw/ioat/ioat_private.h | 2 + drivers/raw/ioat/ioat_rawdev_test.c | 3 +- drivers/raw/ioat/rte_ioat_rawdev_fns.h | 80 ++++++++++++++++++++++++++ 6 files changed, 121 insertions(+), 2 deletions(-) diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c index 11c07efaa..78c443703 100644 --- a/drivers/raw/ioat/idxd_pci.c +++ b/drivers/raw/ioat/idxd_pci.c @@ -52,7 +52,8 @@ idxd_is_wq_enabled(struct idxd_rawdev *idxd) } static const struct rte_rawdev_ops idxd_pci_ops = { - .dev_selftest = idxd_rawdev_test + .dev_selftest = idxd_rawdev_test, + .dump = idxd_dev_dump, }; /* each portal uses 4 x 4k pages */ diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c index e81bd7326..2b8122cbc 100644 --- a/drivers/raw/ioat/idxd_vdev.c +++ b/drivers/raw/ioat/idxd_vdev.c @@ -34,6 +34,7 @@ struct idxd_vdev_args { static const struct rte_rawdev_ops idxd_vdev_ops = { .dev_selftest = idxd_rawdev_test, + .dump = idxd_dev_dump, }; static void * diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c index 4397be886..3dda4ab8e 100644 --- a/drivers/raw/ioat/ioat_common.c +++ b/drivers/raw/ioat/ioat_common.c @@ -7,6 +7,36 @@ #include "ioat_private.h" +int +idxd_dev_dump(struct rte_rawdev *dev, FILE *f) +{ + struct idxd_rawdev *idxd = dev->dev_private; + struct rte_idxd_rawdev *rte_idxd = &idxd->public; + int i; + + fprintf(f, "Raw Device #%d\n", dev->dev_id); + fprintf(f, "Driver: %s\n\n", dev->driver_name); + + fprintf(f, "Portal: %p\n", rte_idxd->portal); + fprintf(f, "Batch Ring size: %u\n", rte_idxd->batch_ring_sz); + fprintf(f, "Comp Handle Ring size: %u\n\n", rte_idxd->hdl_ring_sz); + + fprintf(f, "Next batch: %u\n", rte_idxd->next_batch); + fprintf(f, "Next batch to be completed: %u\n", rte_idxd->next_completed); + for (i = 0; i < rte_idxd->batch_ring_sz; i++) { + struct rte_idxd_desc_batch *b = &rte_idxd->batch_ring[i]; + fprintf(f, "Batch %u @%p: submitted=%u, op_count=%u, hdl_end=%u\n", + i, b, b->submitted, b->op_count, b->hdl_end); + } + + fprintf(f, "\n"); + fprintf(f, "Next free hdl: %u\n", rte_idxd->next_free_hdl); + fprintf(f, "Last completed hdl: %u\n", rte_idxd->last_completed_hdl); + fprintf(f, "Next returned hdl: %u\n", rte_idxd->next_ret_hdl); + + return 0; +} + int idxd_rawdev_create(const char *name, struct rte_device *dev, const struct idxd_rawdev *base_idxd, @@ -18,6 +48,10 @@ idxd_rawdev_create(const char *name, struct rte_device *dev, char mz_name[RTE_MEMZONE_NAMESIZE]; int ret = 0; + RTE_BUILD_BUG_ON(sizeof(struct rte_idxd_hw_desc) != 64); + RTE_BUILD_BUG_ON(offsetof(struct rte_idxd_hw_desc, size ) != 32); + RTE_BUILD_BUG_ON(sizeof(struct rte_idxd_completion) != 32); + if (!name) { IOAT_PMD_ERR("Invalid name of the device!"); ret = -EINVAL; diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h index 2ddaddc37..a873f3f2c 100644 --- a/drivers/raw/ioat/ioat_private.h +++ b/drivers/raw/ioat/ioat_private.h @@ -61,4 +61,6 @@ extern int idxd_rawdev_create(const char *name, struct rte_device *dev, extern int idxd_rawdev_test(uint16_t dev_id); +extern int idxd_dev_dump(struct rte_rawdev *dev, FILE *f); + #endif /* _IOAT_PRIVATE_H_ */ diff --git a/drivers/raw/ioat/ioat_rawdev_test.c b/drivers/raw/ioat/ioat_rawdev_test.c index b208b8c19..7864138fb 100644 --- a/drivers/raw/ioat/ioat_rawdev_test.c +++ b/drivers/raw/ioat/ioat_rawdev_test.c @@ -255,7 +255,8 @@ ioat_rawdev_test(uint16_t dev_id) } int -idxd_rawdev_test(uint16_t dev_id __rte_unused) +idxd_rawdev_test(uint16_t dev_id) { + rte_rawdev_dump(dev_id, stdout); return 0; } diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h b/drivers/raw/ioat/rte_ioat_rawdev_fns.h index aca91dd4f..abd90514b 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h +++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h @@ -88,6 +88,86 @@ struct rte_ioat_rawdev { #define RTE_IOAT_CHANSTS_HALTED 0x3 #define RTE_IOAT_CHANSTS_ARMED 0x4 +/* + * Defines used in the data path for interacting with hardware. + */ +#define IDXD_CMD_OP_SHIFT 24 +enum rte_idxd_ops { + idxd_op_nop = 0, + idxd_op_batch, + idxd_op_drain, + idxd_op_memmove, + idxd_op_fill +}; + +#define IDXD_FLAG_FENCE (1 << 0) +#define IDXD_FLAG_COMPLETION_ADDR_VALID (1 << 2) +#define IDXD_FLAG_REQUEST_COMPLETION (1 << 3) +#define IDXD_FLAG_CACHE_CONTROL (1 << 8) + +/** + * Hardware descriptor used by DSA hardware, for both bursts and + * for individual operations. + */ +struct rte_idxd_hw_desc { + uint32_t pasid; + uint32_t op_flags; + rte_iova_t completion; + + RTE_STD_C11 + union { + rte_iova_t src; /* source address for copy ops etc. */ + rte_iova_t desc_addr; /* descriptor pointer for batch */ + }; + rte_iova_t dst; + + uint32_t size; /* length of data for op, or batch size */ + + /* 28 bytes of padding here */ +} __rte_aligned(64); + +/** + * Completion record structure written back by DSA + */ +struct rte_idxd_completion { + uint8_t status; + uint8_t result; + /* 16-bits pad here */ + uint32_t completed_size; /* data length, or descriptors for batch */ + + rte_iova_t fault_address; + uint32_t invalid_flags; +} __rte_aligned(32); + +#define BATCH_SIZE 64 + +/** + * Structure used inside the driver for building up and submitting + * a batch of operations to the DSA hardware. + */ +struct rte_idxd_desc_batch { + struct rte_idxd_completion comp; /* the completion record for batch */ + + uint16_t submitted; + uint16_t op_count; + uint16_t hdl_end; + + struct rte_idxd_hw_desc batch_desc; + + /* batches must always have 2 descriptors, so put a null at the start */ + struct rte_idxd_hw_desc null_desc; + struct rte_idxd_hw_desc ops[BATCH_SIZE]; +}; + +/** + * structure used to save the "handles" provided by the user to be + * returned to the user on job completion. + */ +struct rte_idxd_user_hdl { + uint64_t src; + uint64_t dst; +}; + /** * @internal * Structure representing an IDXD device instance From patchwork Tue Jul 21 09:51:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74552 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5D784A0526; Tue, 21 Jul 2020 11:56:37 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 131DB1C06C; Tue, 21 Jul 2020 11:56:14 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id DAA611C02A for ; Tue, 21 Jul 2020 11:56:10 +0200 (CEST) IronPort-SDR: Eb1wQDJ8cnzdfR/00AiBJh6FtMKdU8gwpeEbsdDyQf+hntsyCqt+3dLFoPZ8mKeAgkEgouQihT L4oCFb3N3Mnw== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191447" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191447" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:10 -0700 IronPort-SDR: Eu3VvohGhybdZ/dW/SOOEpHQxfppyQOXLySzTOiCvR6zj9nnFCrY5yeGleoY5+XvEDEEpy8lpq gIQNaAzx9LCA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024453" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:09 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:33 +0100 Message-Id: <20200721095140.719297-14-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 13/20] raw/ioat: add configure function for idxd devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add configure function for idxd devices, taking the same parameters as the existing configure function for ioat. The ring_size parameter is used to compute the maximum number of bursts to be supported by the driver, given that the hardware works on individual bursts of descriptors at a time. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/idxd_pci.c | 1 + drivers/raw/ioat/idxd_vdev.c | 1 + drivers/raw/ioat/ioat_common.c | 61 ++++++++++++++++++++++++++ drivers/raw/ioat/ioat_private.h | 3 ++ drivers/raw/ioat/rte_ioat_rawdev_fns.h | 1 + 5 files changed, 67 insertions(+) diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c index 78c443703..762efd5ac 100644 --- a/drivers/raw/ioat/idxd_pci.c +++ b/drivers/raw/ioat/idxd_pci.c @@ -54,6 +54,7 @@ idxd_is_wq_enabled(struct idxd_rawdev *idxd) static const struct rte_rawdev_ops idxd_pci_ops = { .dev_selftest = idxd_rawdev_test, .dump = idxd_dev_dump, + .dev_configure = idxd_dev_configure, }; /* each portal uses 4 x 4k pages */ diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c index 2b8122cbc..90ad11006 100644 --- a/drivers/raw/ioat/idxd_vdev.c +++ b/drivers/raw/ioat/idxd_vdev.c @@ -35,6 +35,7 @@ struct idxd_vdev_args { static const struct rte_rawdev_ops idxd_vdev_ops = { .dev_selftest = idxd_rawdev_test, .dump = idxd_dev_dump, + .dev_configure = idxd_dev_configure, }; static void * diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c index 3dda4ab8e..699661e27 100644 --- a/drivers/raw/ioat/ioat_common.c +++ b/drivers/raw/ioat/ioat_common.c @@ -37,6 +37,67 @@ idxd_dev_dump(struct rte_rawdev *dev, FILE *f) return 0; } +int +idxd_dev_configure(const struct rte_rawdev *dev, + rte_rawdev_obj_t config, size_t config_size) +{ + struct idxd_rawdev *idxd = dev->dev_private; + struct rte_idxd_rawdev *rte_idxd = &idxd->public; + struct rte_ioat_rawdev_config *cfg = config; + uint16_t max_desc = cfg->ring_size; + uint16_t max_batches = max_desc / BATCH_SIZE; + uint16_t i; + + if (config_size != sizeof(*cfg)) + return -EINVAL; + + if (dev->started) { + IOAT_PMD_ERR("%s: Error, device is started.", __func__); + return -EAGAIN; + } + + rte_idxd->hdls_disable = cfg->hdls_disable; + + /* limit the batches to what can be stored in hardware */ + if (max_batches > idxd->max_batches) { + IOAT_PMD_DEBUG("Ring size of %u is too large for this device, need to limit to %u batches of %u", + max_desc, idxd->max_batches, BATCH_SIZE); + max_batches = idxd->max_batches; + max_desc = max_batches * BATCH_SIZE; + } + if (!rte_is_power_of_2(max_desc)) + max_desc = rte_align32pow2(max_desc); + IOAT_PMD_DEBUG("Rawdev %u using %u descriptors in %u batches", + dev->dev_id, max_desc, max_batches); + + /* TODO clean up if reconfiguring an existing device */ + rte_idxd->batch_ring = rte_zmalloc(NULL, + sizeof(*rte_idxd->batch_ring) * max_batches, 0); + if (rte_idxd->batch_ring == NULL) + return -ENOMEM; + + rte_idxd->hdl_ring = rte_zmalloc(NULL, + sizeof(*rte_idxd->hdl_ring) * max_desc, 0); + if (rte_idxd->hdl_ring == NULL) { + rte_free(rte_idxd->batch_ring); + rte_idxd->batch_ring = NULL; + return -ENOMEM; + } + rte_idxd->batch_ring_sz = max_batches; + rte_idxd->hdl_ring_sz = max_desc; + + for (i = 0; i < rte_idxd->batch_ring_sz; i++) { + struct rte_idxd_desc_batch *b = &rte_idxd->batch_ring[i]; + b->batch_desc.completion = rte_mem_virt2iova(&b->comp); + b->batch_desc.desc_addr = rte_mem_virt2iova(&b->null_desc); + b->batch_desc.op_flags = (idxd_op_batch << IDXD_CMD_OP_SHIFT) | + IDXD_FLAG_COMPLETION_ADDR_VALID | + IDXD_FLAG_REQUEST_COMPLETION; + } + + return 0; +} + int idxd_rawdev_create(const char *name, struct rte_device *dev, const struct idxd_rawdev *base_idxd, diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h index a873f3f2c..91ded444b 100644 --- a/drivers/raw/ioat/ioat_private.h +++ b/drivers/raw/ioat/ioat_private.h @@ -59,6 +59,9 @@ extern int idxd_rawdev_create(const char *name, struct rte_device *dev, const struct idxd_rawdev *idxd, const struct rte_rawdev_ops *ops); +extern int idxd_dev_configure(const struct rte_rawdev *dev, + rte_rawdev_obj_t config, size_t config_size); + extern int idxd_rawdev_test(uint16_t dev_id); extern int idxd_dev_dump(struct rte_rawdev *dev, FILE *f); diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h b/drivers/raw/ioat/rte_ioat_rawdev_fns.h index abd90514b..7090ac0a1 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h +++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h @@ -185,6 +185,7 @@ struct rte_idxd_rawdev { uint16_t next_ret_hdl; /* the next user hdl to return */ uint16_t last_completed_hdl; /* the last user hdl that has completed */ uint16_t next_free_hdl; /* where the handle for next op will go */ + uint16_t hdls_disable; /* disable tracking completion handles */ struct rte_idxd_user_hdl *hdl_ring; struct rte_idxd_desc_batch *batch_ring; From patchwork Tue Jul 21 09:51:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74553 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EEEC7A0526; Tue, 21 Jul 2020 11:56:47 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 41FF61C07E; Tue, 21 Jul 2020 11:56:16 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 39D3C1C02D for ; Tue, 21 Jul 2020 11:56:13 +0200 (CEST) IronPort-SDR: EysIruXBYgVZwuR/yzCJPEwxQJstlvPSTz1RjgL5b0KGR+dgjMvzYnBAAU6CfKSrAekx2H1tm5 qAM4Wa3jYVHg== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191453" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191453" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:12 -0700 IronPort-SDR: 5wWzcCB+mdpSWH3toItmKTYsk0u5zbKRk8fK0KAjX2WUIMJmpbCAIc4jDzlzp4ktyO1G6ntyAB xkHQQf4ggIeQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024464" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:11 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:34 +0100 Message-Id: <20200721095140.719297-15-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 14/20] raw/ioat: add start and stop functions for idxd devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the start and stop functions for DSA hardware devices. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz --- drivers/raw/ioat/idxd_pci.c | 52 ++++++++++++++++++++++++++++++++++++ drivers/raw/ioat/idxd_vdev.c | 50 ++++++++++++++++++++++++++++++++++ 2 files changed, 102 insertions(+) diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c index 762efd5ac..6655cf9b7 100644 --- a/drivers/raw/ioat/idxd_pci.c +++ b/drivers/raw/ioat/idxd_pci.c @@ -51,10 +51,62 @@ idxd_is_wq_enabled(struct idxd_rawdev *idxd) return (state & WQ_STATE_MASK) == 0x1; } +static void +idxd_pci_dev_stop(struct rte_rawdev *dev) +{ + struct idxd_rawdev *idxd = dev->dev_private; + uint8_t err_code; + + if (!idxd_is_wq_enabled(idxd)) { + IOAT_PMD_ERR("Work queue %d already disabled", idxd->qid); + return; + } + + err_code = idxd_pci_dev_command(idxd, idxd_disable_wq); + if (err_code || idxd_is_wq_enabled(idxd)) { + IOAT_PMD_ERR("Failed disabling work queue %d, error code: %#x", + idxd->qid, err_code); + return; + } + IOAT_PMD_DEBUG("Work queue %d disabled OK", idxd->qid); + + return; +} + +static int +idxd_pci_dev_start(struct rte_rawdev *dev) +{ + struct idxd_rawdev *idxd = dev->dev_private; + uint8_t err_code; + + if (idxd_is_wq_enabled(idxd)) { + IOAT_PMD_WARN("WQ %d already enabled", idxd->qid); + return 0; + } + + if (idxd->public.batch_ring == NULL) { + IOAT_PMD_ERR("WQ %d has not been fully configured", idxd->qid); + return -EINVAL; + } + + err_code = idxd_pci_dev_command(idxd, idxd_enable_wq); + if (err_code || !idxd_is_wq_enabled(idxd)) { + IOAT_PMD_ERR("Failed enabling work queue %d, error code: %#x", + idxd->qid, err_code); + return err_code == 0 ? -1 : err_code; + } + + IOAT_PMD_DEBUG("Work queue %d enabled OK", idxd->qid); + + return 0; +} + static const struct rte_rawdev_ops idxd_pci_ops = { .dev_selftest = idxd_rawdev_test, .dump = idxd_dev_dump, .dev_configure = idxd_dev_configure, + .dev_start = idxd_pci_dev_start, + .dev_stop = idxd_pci_dev_stop, }; /* each portal uses 4 x 4k pages */ diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c index 90ad11006..ab7efd216 100644 --- a/drivers/raw/ioat/idxd_vdev.c +++ b/drivers/raw/ioat/idxd_vdev.c @@ -32,10 +32,60 @@ struct idxd_vdev_args { uint8_t wq_id; }; +static void +idxd_vdev_stop(struct rte_rawdev *dev) +{ + struct idxd_rawdev *idxd = dev->dev_private; + int ret; + + if (!accfg_wq_is_enabled(idxd->u.vdev.wq)) { + IOAT_PMD_ERR("Work queue %s already disabled", + accfg_wq_get_devname(idxd->u.vdev.wq)); + return; + } + + ret = accfg_wq_disable(idxd->u.vdev.wq); + if (ret) { + IOAT_PMD_INFO("Work queue %s not disabled, continuing...", + accfg_wq_get_devname(idxd->u.vdev.wq)); + return; + } + IOAT_PMD_DEBUG("Disabling work queue %s OK", + accfg_wq_get_devname(idxd->u.vdev.wq)); + + return; +} + +static int +idxd_vdev_start(struct rte_rawdev *dev) +{ + struct idxd_rawdev *idxd = dev->dev_private; + int ret; + + if (accfg_wq_is_enabled(idxd->u.vdev.wq)) { + IOAT_PMD_ERR("Work queue %s already enabled", + accfg_wq_get_devname(idxd->u.vdev.wq)); + return 0; + } + + ret = accfg_wq_enable(idxd->u.vdev.wq); + if (ret) { + IOAT_PMD_ERR("Error enabling work queue %s", + accfg_wq_get_devname(idxd->u.vdev.wq)); + return -1; + } + IOAT_PMD_DEBUG("Enabling work queue %s OK", + accfg_wq_get_devname(idxd->u.vdev.wq)); + + return 0; +} + static const struct rte_rawdev_ops idxd_vdev_ops = { .dev_selftest = idxd_rawdev_test, .dump = idxd_dev_dump, .dev_configure = idxd_dev_configure, + .dev_start = idxd_vdev_start, + .dev_stop = idxd_vdev_stop, }; static void * From patchwork Tue Jul 21 09:51:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74554 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 90F11A0526; Tue, 21 Jul 2020 11:56:58 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 92FC31C0AF; Tue, 21 Jul 2020 11:56:19 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 6EE141C07B for ; Tue, 21 Jul 2020 11:56:15 +0200 (CEST) IronPort-SDR: qx5TBZSZn1CzRNBvKJERaD2qaCUwZ5E8bpwEkIEtN9j3Yk87fQc5AgpI2k3LN3Wyufau9C86/o QG+4V1BDoqyQ== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191455" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191455" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:15 -0700 IronPort-SDR: GXF3SHJf4N0cRqzGr8APC6bhA/WXTe0wEQlKryJkZ5fRHYD4GmYtkE0XQPeJiU7TtJLjcUY31O set+99VP9xCw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024483" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:13 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:35 +0100 Message-Id: <20200721095140.719297-16-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 15/20] raw/ioat: add data path support for idxd devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support for doing copies using DSA hardware. This is implemented by just switching on the device type field at the start of the inline functions. Since there is no hardware which will have both device types present this branch will always be predictable after the first call, meaning it has little to no perf penalty. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/ioat_common.c | 1 + drivers/raw/ioat/ioat_rawdev.c | 1 + drivers/raw/ioat/rte_ioat_rawdev_fns.h | 165 +++++++++++++++++++++++-- 3 files changed, 158 insertions(+), 9 deletions(-) diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c index 699661e27..33e5bb4a6 100644 --- a/drivers/raw/ioat/ioat_common.c +++ b/drivers/raw/ioat/ioat_common.c @@ -143,6 +143,7 @@ idxd_rawdev_create(const char *name, struct rte_device *dev, idxd = rawdev->dev_private; *idxd = *base_idxd; /* copy over the main fields already passed in */ + idxd->public.type = RTE_IDXD_DEV; idxd->rawdev = rawdev; idxd->mz = mz; diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index 8f9c8b56f..48fe32d0a 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -252,6 +252,7 @@ ioat_rawdev_create(const char *name, struct rte_pci_device *dev) rawdev->driver_name = dev->device.driver->name; ioat = rawdev->dev_private; + ioat->type = RTE_IOAT_DEV; ioat->rawdev = rawdev; ioat->mz = mz; ioat->regs = dev->mem_resource[0].addr; diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h b/drivers/raw/ioat/rte_ioat_rawdev_fns.h index 7090ac0a1..98af40894 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h +++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h @@ -194,8 +194,8 @@ struct rte_idxd_rawdev { /** * Enqueue a copy operation onto the ioat device */ -static inline int -rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, +static __rte_always_inline int +__ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl, int fence) { @@ -233,8 +233,8 @@ rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, /** * Trigger hardware to begin performing enqueued copy operations */ -static inline void -rte_ioat_do_copies(int dev_id) +static __rte_always_inline void +__ioat_perform_ops(int dev_id) { struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; ioat->desc_ring[(ioat->next_write - 1) & (ioat->ring_size - 1)].u @@ -248,8 +248,8 @@ rte_ioat_do_copies(int dev_id) * @internal * Returns the index of the last completed operation. */ -static inline int -rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error) +static __rte_always_inline int +__ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error) { uint64_t status = ioat->status; @@ -263,8 +263,8 @@ rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error) /** * Returns details of copy operations that have been completed */ -static inline int -rte_ioat_completed_copies(int dev_id, uint8_t max_copies, +static __rte_always_inline int +__ioat_completed_ops(int dev_id, uint8_t max_copies, uintptr_t *src_hdls, uintptr_t *dst_hdls) { struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private; @@ -274,7 +274,7 @@ rte_ioat_completed_copies(int dev_id, uint8_t max_copies, int error; int i = 0; - end_read = (rte_ioat_get_last_completed(ioat, &error) + 1) & mask; + end_read = (__ioat_get_last_completed(ioat, &error) + 1) & mask; count = (end_read - (read & mask)) & mask; if (error) { @@ -311,4 +311,151 @@ rte_ioat_completed_copies(int dev_id, uint8_t max_copies, return count; } +static __rte_always_inline int +__idxd_enqueue_copy(int dev_id, rte_iova_t src, rte_iova_t dst, + unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl, + int fence __rte_unused) +{ + struct rte_idxd_rawdev *idxd = rte_rawdevs[dev_id].dev_private; + struct rte_idxd_desc_batch *b = &idxd->batch_ring[idxd->next_batch]; + uint32_t op_flags = (idxd_op_memmove << IDXD_CMD_OP_SHIFT) | + IDXD_FLAG_CACHE_CONTROL; + + /* check for room in the handle ring */ + if (((idxd->next_free_hdl + 1) & (idxd->hdl_ring_sz - 1)) == idxd->next_ret_hdl) { + rte_errno = ENOSPC; + return 0; + } + if (b->op_count >= BATCH_SIZE) { + /* TODO change to submit batch and move on */ + rte_errno = ENOSPC; + return 0; + } + /* check that we can actually use the current batch */ + if (b->submitted) { + rte_errno = ENOSPC; + return 0; + } + + /* write the descriptor */ + b->ops[b->op_count++] = (struct rte_idxd_hw_desc){ + .op_flags = op_flags, + .src = src, + .dst = dst, + .size = length + }; + + /* store the completion details */ + if (!idxd->hdls_disable) + idxd->hdl_ring[idxd->next_free_hdl] = (struct rte_idxd_user_hdl) { + .src = src_hdl, + .dst =dst_hdl + }; + if (++idxd->next_free_hdl == idxd->hdl_ring_sz) + idxd->next_free_hdl = 0; + + return 1; +} + +static __rte_always_inline void +__idxd_movdir64b(volatile void *dst, const void *src) +{ + asm volatile (".byte 0x66, 0x0f, 0x38, 0xf8, 0x02" + : + : "a" (dst), "d" (src)); +} + +static __rte_always_inline void +__idxd_perform_ops(int dev_id) +{ + struct rte_idxd_rawdev *idxd = rte_rawdevs[dev_id].dev_private; + struct rte_idxd_desc_batch *b = &idxd->batch_ring[idxd->next_batch]; + + if (b->submitted || b->op_count == 0) + return; + b->hdl_end = idxd->next_free_hdl; + b->comp.status = 0; + b->submitted = 1; + b->batch_desc.size = b->op_count + 1; + __idxd_movdir64b(idxd->portal, &b->batch_desc); + + if (++idxd->next_batch == idxd->batch_ring_sz) + idxd->next_batch = 0; +} + +static __rte_always_inline int +__idxd_completed_ops(int dev_id, uint8_t max_ops, + uintptr_t *src_hdls, uintptr_t *dst_hdls) +{ + struct rte_idxd_rawdev *idxd = rte_rawdevs[dev_id].dev_private; + struct rte_idxd_desc_batch *b = &idxd->batch_ring[idxd->next_completed]; + uint16_t h_idx = idxd->next_ret_hdl; + int n = 0; + + while (b->submitted && b->comp.status != 0) { + idxd->last_completed_hdl = b->hdl_end; + b->submitted = 0; + b->op_count = 0; + if (++idxd->next_completed == idxd->batch_ring_sz) + idxd->next_completed = 0; + b = &idxd->batch_ring[idxd->next_completed]; + } + + if (!idxd->hdls_disable) + for (n = 0; n < max_ops && h_idx != idxd->last_completed_hdl; n++) { + src_hdls[n] = idxd->hdl_ring[h_idx].src; + dst_hdls[n] = idxd->hdl_ring[h_idx].dst; + if (++h_idx == idxd->hdl_ring_sz) + h_idx = 0; + } + else + while (h_idx != idxd->last_completed_hdl) { + n++; + if (++h_idx == idxd->hdl_ring_sz) + h_idx = 0; + } + + idxd->next_ret_hdl = h_idx; + + return n; +} + +static inline int +rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, + unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl, + int fence) +{ + enum rte_ioat_dev_type *type = rte_rawdevs[dev_id].dev_private; + if (*type == RTE_IDXD_DEV) + return __idxd_enqueue_copy(dev_id, src, dst, length, + src_hdl, dst_hdl, fence); + else + return __ioat_enqueue_copy(dev_id, src, dst, length, + src_hdl, dst_hdl, fence); +} + +static inline void +rte_ioat_do_copies(int dev_id) +{ + enum rte_ioat_dev_type *type = rte_rawdevs[dev_id].dev_private; + if (*type == RTE_IDXD_DEV) + return __idxd_perform_ops(dev_id); + else + return __ioat_perform_ops(dev_id); +} + +static inline int +rte_ioat_completed_copies(int dev_id, uint8_t max_copies, + uintptr_t *src_hdls, uintptr_t *dst_hdls) +{ + enum rte_ioat_dev_type *type = rte_rawdevs[dev_id].dev_private; + if (*type == RTE_IDXD_DEV) + return __idxd_completed_ops(dev_id, max_copies, + src_hdls, dst_hdls); + else + return __ioat_completed_ops(dev_id, max_copies, + src_hdls, dst_hdls); +} + + #endif /* _RTE_IOAT_RAWDEV_FNS_H_ */ From patchwork Tue Jul 21 09:51:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74555 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A67DDA0526; Tue, 21 Jul 2020 11:57:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6EEFE1C0B6; Tue, 21 Jul 2020 11:56:21 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id AEEB41C0AF for ; Tue, 21 Jul 2020 11:56:17 +0200 (CEST) IronPort-SDR: AabLZVozYU2CiAd+9SztCmgY82LBJhVp5HXkKi77YfgYZPrRaUyHwb997zjP12jtEeld568/gU 6eXGD6HKjnZw== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191458" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191458" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:17 -0700 IronPort-SDR: +WkxNGsLHZjW9uBwV9enZ/xVFHfN4A6xwCaTqfHlIUyBJKs9f+r+QAMFbug9KyCdv6tuwt4g4/ UfemiqAuW7Mw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024501" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:15 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:36 +0100 Message-Id: <20200721095140.719297-17-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 16/20] raw/ioat: add info function for idxd devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the info get function for DSA devices, returning just the ring size info about the device, same as is returned for existing IOAT/CBDMA devices. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/idxd_pci.c | 1 + drivers/raw/ioat/idxd_vdev.c | 1 + drivers/raw/ioat/ioat_common.c | 16 ++++++++++++++++ drivers/raw/ioat/ioat_private.h | 3 +++ 4 files changed, 21 insertions(+) diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c index 6655cf9b7..2b5a0e1c8 100644 --- a/drivers/raw/ioat/idxd_pci.c +++ b/drivers/raw/ioat/idxd_pci.c @@ -107,6 +107,7 @@ static const struct rte_rawdev_ops idxd_pci_ops = { .dev_configure = idxd_dev_configure, .dev_start = idxd_pci_dev_start, .dev_stop = idxd_pci_dev_stop, + .dev_info_get = idxd_dev_info_get, }; /* each portal uses 4 x 4k pages */ diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c index ab7efd216..8b7c2cacd 100644 --- a/drivers/raw/ioat/idxd_vdev.c +++ b/drivers/raw/ioat/idxd_vdev.c @@ -86,6 +86,7 @@ static const struct rte_rawdev_ops idxd_vdev_ops = { .dev_configure = idxd_dev_configure, .dev_start = idxd_vdev_start, .dev_stop = idxd_vdev_stop, + .dev_info_get = idxd_dev_info_get, }; static void * diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c index 33e5bb4a6..fe293aebf 100644 --- a/drivers/raw/ioat/ioat_common.c +++ b/drivers/raw/ioat/ioat_common.c @@ -37,6 +37,22 @@ idxd_dev_dump(struct rte_rawdev *dev, FILE *f) return 0; } +int +idxd_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info, + size_t info_size) +{ + struct rte_ioat_rawdev_config *cfg = dev_info; + struct idxd_rawdev *idxd = dev->dev_private; + struct rte_idxd_rawdev *rte_idxd = &idxd->public; + + if (info_size != sizeof(*cfg)) + return -EINVAL; + + if (cfg != NULL) + cfg->ring_size = rte_idxd->hdl_ring_sz; + return 0; +} + int idxd_dev_configure(const struct rte_rawdev *dev, rte_rawdev_obj_t config, size_t config_size) diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h index 91ded444b..35189b15f 100644 --- a/drivers/raw/ioat/ioat_private.h +++ b/drivers/raw/ioat/ioat_private.h @@ -62,6 +62,9 @@ extern int idxd_rawdev_create(const char *name, struct rte_device *dev, extern int idxd_dev_configure(const struct rte_rawdev *dev, rte_rawdev_obj_t config, size_t config_size); +extern int idxd_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info, + size_t info_size); + extern int idxd_rawdev_test(uint16_t dev_id); extern int idxd_dev_dump(struct rte_rawdev *dev, FILE *f); From patchwork Tue Jul 21 09:51:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74556 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24E27A0526; Tue, 21 Jul 2020 11:57:23 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E0F741C0C3; Tue, 21 Jul 2020 11:56:25 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id B638F1C0B5 for ; Tue, 21 Jul 2020 11:56:20 +0200 (CEST) IronPort-SDR: NAVZh+hGKctJYYHKtyDsHwl2KY4CnFqohfk18Re77S4ucYJ4XW7jokvHlof80NTpWvm4eDy9zj rURYHpOVeH7g== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191463" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191463" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:20 -0700 IronPort-SDR: OHBg/NmTvaqtx4NQpHbIScJF4KPtFUMfmpu9P8ovEIEmtEkehnkd/jCfmc9q5S5tBoKHqzc0BN HqJnZhjWX76w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024509" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:18 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:37 +0100 Message-Id: <20200721095140.719297-18-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 17/20] raw/ioat: create separate statistics structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Rather than having the xstats as fields inside the main driver structure, create a separate structure type for them. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/ioat_rawdev.c | 21 ++++++++---------- drivers/raw/ioat/rte_ioat_rawdev_fns.h | 30 ++++++++++++++++---------- 2 files changed, 28 insertions(+), 23 deletions(-) diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index 48fe32d0a..e4d39a2ee 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -135,10 +135,10 @@ ioat_xstats_get(const struct rte_rawdev *dev, const unsigned int ids[], for (i = 0; i < n; i++) { switch (ids[i]) { - case 0: values[i] = ioat->enqueue_failed; break; - case 1: values[i] = ioat->enqueued; break; - case 2: values[i] = ioat->started; break; - case 3: values[i] = ioat->completed; break; + case 0: values[i] = ioat->xstats.enqueue_failed; break; + case 1: values[i] = ioat->xstats.enqueued; break; + case 2: values[i] = ioat->xstats.started; break; + case 3: values[i] = ioat->xstats.completed; break; default: values[i] = 0; break; } } @@ -169,26 +169,23 @@ ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids, uint32_t nb_ids) unsigned int i; if (!ids) { - ioat->enqueue_failed = 0; - ioat->enqueued = 0; - ioat->started = 0; - ioat->completed = 0; + memset(&ioat->xstats, 0, sizeof(ioat->xstats)); return 0; } for (i = 0; i < nb_ids; i++) { switch (ids[i]) { case 0: - ioat->enqueue_failed = 0; + ioat->xstats.enqueue_failed = 0; break; case 1: - ioat->enqueued = 0; + ioat->xstats.enqueued = 0; break; case 2: - ioat->started = 0; + ioat->xstats.started = 0; break; case 3: - ioat->completed = 0; + ioat->xstats.completed = 0; break; default: IOAT_PMD_WARN("Invalid xstat id - cannot reset value"); diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h b/drivers/raw/ioat/rte_ioat_rawdev_fns.h index 98af40894..813c1a157 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h +++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h @@ -47,17 +47,31 @@ enum rte_ioat_dev_type { RTE_IDXD_DEV, }; +/** + * @internal + * some statistics for tracking, if added/changed update xstats fns + */ +struct rte_ioat_xstats { + uint64_t enqueue_failed; + uint64_t enqueued; + uint64_t started; + uint64_t completed; +}; + /** * @internal * Structure representing an IOAT device instance * Structure representing a device instance */ struct rte_ioat_rawdev { + /* common fields at the top - match those in rte_idxd_rawdev */ enum rte_ioat_dev_type type; + struct rte_ioat_xstats xstats; + struct rte_rawdev *rawdev; const struct rte_memzone *mz; const struct rte_memzone *desc_mz; - volatile uint16_t *doorbell; + volatile uint16_t *doorbell __rte_cache_aligned; phys_addr_t status_addr; phys_addr_t ring_addr; @@ -70,12 +84,6 @@ struct rte_ioat_rawdev { unsigned short next_read; unsigned short next_write; - /* some statistics for tracking, if added/changed update xstats fns*/ - uint64_t enqueue_failed __rte_cache_aligned; - uint64_t enqueued; - uint64_t started; - uint64_t completed; - /* to report completions, the device will write status back here */ volatile uint64_t status __rte_cache_aligned; @@ -207,7 +215,7 @@ __ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, struct rte_ioat_generic_hw_desc *desc; if (space == 0) { - ioat->enqueue_failed++; + ioat->xstats.enqueue_failed++; return 0; } @@ -226,7 +234,7 @@ __ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, (int64_t)src_hdl); rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]); - ioat->enqueued++; + ioat->xstats.enqueued++; return 1; } @@ -241,7 +249,7 @@ __ioat_perform_ops(int dev_id) .control.completion_update = 1; rte_compiler_barrier(); *ioat->doorbell = ioat->next_write; - ioat->started = ioat->enqueued; + ioat->xstats.started = ioat->xstats.enqueued; } /** @@ -307,7 +315,7 @@ __ioat_completed_ops(int dev_id, uint8_t max_copies, end: ioat->next_read = read; - ioat->completed += count; + ioat->xstats.completed += count; return count; } From patchwork Tue Jul 21 09:51:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74557 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3B557A0526; Tue, 21 Jul 2020 11:57:34 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5FAD21C0D0; Tue, 21 Jul 2020 11:56:27 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id E0EA81C0BE for ; Tue, 21 Jul 2020 11:56:22 +0200 (CEST) IronPort-SDR: 3AB9oc+wAkyNuDtECSzqc5hhUPhLZ5LjhHaBthhvLJ9tHbR3pP9LjJaOaarUpWGeNd2u5oCtiP 1oJgB0XXGXzQ== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191467" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191467" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:22 -0700 IronPort-SDR: 3LCV12u1qzDUcSgbbEiYSnOrfVekq7L0pKTHJI3/1g98DKeUh5Kmp13ZEagRVDp9wQgCuwiglI Jyv9B94w5k7A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024526" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:20 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:38 +0100 Message-Id: <20200721095140.719297-19-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 18/20] raw/ioat: move xstats functions to common file X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The xstats functions can be used by all ioat devices so move them from the ioat_rawdev.c file to ioat_common.c, and add the function prototypes to the internal header file. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/ioat_common.c | 76 +++++++++++++++++++++++++++++++++ drivers/raw/ioat/ioat_private.h | 10 +++++ drivers/raw/ioat/ioat_rawdev.c | 75 -------------------------------- 3 files changed, 86 insertions(+), 75 deletions(-) diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c index fe293aebf..5f366f009 100644 --- a/drivers/raw/ioat/ioat_common.c +++ b/drivers/raw/ioat/ioat_common.c @@ -4,9 +4,85 @@ #include #include +#include #include "ioat_private.h" +static const char * const xstat_names[] = { + "failed_enqueues", "successful_enqueues", + "copies_started", "copies_completed" +}; + +int +ioat_xstats_get(const struct rte_rawdev *dev, const unsigned int ids[], + uint64_t values[], unsigned int n) +{ + const struct rte_ioat_rawdev *ioat = dev->dev_private; + unsigned int i; + + for (i = 0; i < n; i++) { + switch (ids[i]) { + case 0: values[i] = ioat->xstats.enqueue_failed; break; + case 1: values[i] = ioat->xstats.enqueued; break; + case 2: values[i] = ioat->xstats.started; break; + case 3: values[i] = ioat->xstats.completed; break; + default: values[i] = 0; break; + } + } + return n; +} + +int +ioat_xstats_get_names(const struct rte_rawdev *dev, + struct rte_rawdev_xstats_name *names, + unsigned int size) +{ + unsigned int i; + + RTE_SET_USED(dev); + if (size < RTE_DIM(xstat_names)) + return RTE_DIM(xstat_names); + + for (i = 0; i < RTE_DIM(xstat_names); i++) + strlcpy(names[i].name, xstat_names[i], sizeof(names[i])); + + return RTE_DIM(xstat_names); +} + +int +ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids, uint32_t nb_ids) +{ + struct rte_ioat_rawdev *ioat = dev->dev_private; + unsigned int i; + + if (!ids) { + memset(&ioat->xstats, 0, sizeof(ioat->xstats)); + return 0; + } + + for (i = 0; i < nb_ids; i++) { + switch (ids[i]) { + case 0: + ioat->xstats.enqueue_failed = 0; + break; + case 1: + ioat->xstats.enqueued = 0; + break; + case 2: + ioat->xstats.started = 0; + break; + case 3: + ioat->xstats.completed = 0; + break; + default: + IOAT_PMD_WARN("Invalid xstat id - cannot reset value"); + break; + } + } + + return 0; +} + int idxd_dev_dump(struct rte_rawdev *dev, FILE *f) { diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h index 35189b15f..46465e675 100644 --- a/drivers/raw/ioat/ioat_private.h +++ b/drivers/raw/ioat/ioat_private.h @@ -55,6 +55,16 @@ struct idxd_rawdev { } u; }; +int ioat_xstats_get(const struct rte_rawdev *dev, const unsigned int ids[], + uint64_t values[], unsigned int n); + +int ioat_xstats_get_names(const struct rte_rawdev *dev, + struct rte_rawdev_xstats_name *names, + unsigned int size); + +int ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids, + uint32_t nb_ids); + extern int idxd_rawdev_create(const char *name, struct rte_device *dev, const struct idxd_rawdev *idxd, const struct rte_rawdev_ops *ops); diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index e4d39a2ee..e3c98a825 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -121,81 +121,6 @@ ioat_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info, return 0; } -static const char * const xstat_names[] = { - "failed_enqueues", "successful_enqueues", - "copies_started", "copies_completed" -}; - -static int -ioat_xstats_get(const struct rte_rawdev *dev, const unsigned int ids[], - uint64_t values[], unsigned int n) -{ - const struct rte_ioat_rawdev *ioat = dev->dev_private; - unsigned int i; - - for (i = 0; i < n; i++) { - switch (ids[i]) { - case 0: values[i] = ioat->xstats.enqueue_failed; break; - case 1: values[i] = ioat->xstats.enqueued; break; - case 2: values[i] = ioat->xstats.started; break; - case 3: values[i] = ioat->xstats.completed; break; - default: values[i] = 0; break; - } - } - return n; -} - -static int -ioat_xstats_get_names(const struct rte_rawdev *dev, - struct rte_rawdev_xstats_name *names, - unsigned int size) -{ - unsigned int i; - - RTE_SET_USED(dev); - if (size < RTE_DIM(xstat_names)) - return RTE_DIM(xstat_names); - - for (i = 0; i < RTE_DIM(xstat_names); i++) - strlcpy(names[i].name, xstat_names[i], sizeof(names[i])); - - return RTE_DIM(xstat_names); -} - -static int -ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids, uint32_t nb_ids) -{ - struct rte_ioat_rawdev *ioat = dev->dev_private; - unsigned int i; - - if (!ids) { - memset(&ioat->xstats, 0, sizeof(ioat->xstats)); - return 0; - } - - for (i = 0; i < nb_ids; i++) { - switch (ids[i]) { - case 0: - ioat->xstats.enqueue_failed = 0; - break; - case 1: - ioat->xstats.enqueued = 0; - break; - case 2: - ioat->xstats.started = 0; - break; - case 3: - ioat->xstats.completed = 0; - break; - default: - IOAT_PMD_WARN("Invalid xstat id - cannot reset value"); - break; - } - } - - return 0; -} - extern int ioat_rawdev_test(uint16_t dev_id); static int From patchwork Tue Jul 21 09:51:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74558 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0C33FA0526; Tue, 21 Jul 2020 11:57:45 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0FB3F1C0CC; Tue, 21 Jul 2020 11:56:30 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id A28961C0B0 for ; Tue, 21 Jul 2020 11:56:25 +0200 (CEST) IronPort-SDR: n6wwIJEVlj70dc9kmCRAa8Gdo44FlL2rwb/qFu06H8d/bI6MxxcA3qHDAT+htA+2PKYqmo0ds+ nBGA0uiewQlg== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191472" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191472" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:25 -0700 IronPort-SDR: Wu3j7PQl53vY4teH44yyUy4tUeswbyX0MbgrYtEzhhxu6yEXWuWKUv5CYXZapEhKMmzlhIN76g s9BMNHPkleEA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024538" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:23 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:39 +0100 Message-Id: <20200721095140.719297-20-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 19/20] raw/ioat: add xstats tracking for idxd devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add update of the relevant stats for the data path functions and point the overall device struct xstats function pointers to the existing ioat functions. At this point, all necessary hooks for supporting the existing unit tests are in place so call them for each device. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/idxd_pci.c | 3 +++ drivers/raw/ioat/idxd_vdev.c | 3 +++ drivers/raw/ioat/ioat_rawdev_test.c | 2 +- drivers/raw/ioat/rte_ioat_rawdev_fns.h | 26 ++++++++++++++++---------- 4 files changed, 23 insertions(+), 11 deletions(-) diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c index 2b5a0e1c8..a426897b2 100644 --- a/drivers/raw/ioat/idxd_pci.c +++ b/drivers/raw/ioat/idxd_pci.c @@ -108,6 +108,9 @@ static const struct rte_rawdev_ops idxd_pci_ops = { .dev_start = idxd_pci_dev_start, .dev_stop = idxd_pci_dev_stop, .dev_info_get = idxd_dev_info_get, + .xstats_get = ioat_xstats_get, + .xstats_get_names = ioat_xstats_get_names, + .xstats_reset = ioat_xstats_reset, }; /* each portal uses 4 x 4k pages */ diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c index 8b7c2cacd..aa2693368 100644 --- a/drivers/raw/ioat/idxd_vdev.c +++ b/drivers/raw/ioat/idxd_vdev.c @@ -87,6 +87,9 @@ static const struct rte_rawdev_ops idxd_vdev_ops = { .dev_start = idxd_vdev_start, .dev_stop = idxd_vdev_stop, .dev_info_get = idxd_dev_info_get, + .xstats_get = ioat_xstats_get, + .xstats_get_names = ioat_xstats_get_names, + .xstats_reset = ioat_xstats_reset, }; static void * diff --git a/drivers/raw/ioat/ioat_rawdev_test.c b/drivers/raw/ioat/ioat_rawdev_test.c index 7864138fb..678d135c4 100644 --- a/drivers/raw/ioat/ioat_rawdev_test.c +++ b/drivers/raw/ioat/ioat_rawdev_test.c @@ -258,5 +258,5 @@ int idxd_rawdev_test(uint16_t dev_id) { rte_rawdev_dump(dev_id, stdout); - return 0; + return ioat_rawdev_test(dev_id); } diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h b/drivers/raw/ioat/rte_ioat_rawdev_fns.h index 813c1a157..2ddd0a024 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h +++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h @@ -182,6 +182,8 @@ struct rte_idxd_user_hdl { */ struct rte_idxd_rawdev { enum rte_ioat_dev_type type; + struct rte_ioat_xstats xstats; + void *portal; /* address to write the batch descriptor */ /* counters to track the batches and the individual op handles */ @@ -330,20 +332,16 @@ __idxd_enqueue_copy(int dev_id, rte_iova_t src, rte_iova_t dst, IDXD_FLAG_CACHE_CONTROL; /* check for room in the handle ring */ - if (((idxd->next_free_hdl + 1) & (idxd->hdl_ring_sz - 1)) == idxd->next_ret_hdl) { - rte_errno = ENOSPC; - return 0; - } + if (((idxd->next_free_hdl + 1) & (idxd->hdl_ring_sz - 1)) == idxd->next_ret_hdl) + goto failed; + if (b->op_count >= BATCH_SIZE) { /* TODO change to submit batch and move on */ - rte_errno = ENOSPC; - return 0; + goto failed; } /* check that we can actually use the current batch */ - if (b->submitted) { - rte_errno = ENOSPC; - return 0; - } + if (b->submitted) + goto failed; /* write the descriptor */ b->ops[b->op_count++] = (struct rte_idxd_hw_desc){ @@ -362,7 +360,13 @@ __idxd_enqueue_copy(int dev_id, rte_iova_t src, rte_iova_t dst, if (++idxd->next_free_hdl == idxd->hdl_ring_sz) idxd->next_free_hdl = 0; + idxd->xstats.enqueued++; return 1; + +failed: + idxd->xstats.enqueue_failed++; + rte_errno = ENOSPC; + return 0; } static __rte_always_inline void @@ -389,6 +393,7 @@ __idxd_perform_ops(int dev_id) if (++idxd->next_batch == idxd->batch_ring_sz) idxd->next_batch = 0; + idxd->xstats.started = idxd->xstats.enqueued; } static __rte_always_inline int @@ -425,6 +430,7 @@ __idxd_completed_ops(int dev_id, uint8_t max_ops, idxd->next_ret_hdl = h_idx; + idxd->xstats.completed += n; return n; } From patchwork Tue Jul 21 09:51:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 74559 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3B8F6A0526; Tue, 21 Jul 2020 11:57:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6138F1C0DB; Tue, 21 Jul 2020 11:56:31 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id D15561C0CC for ; Tue, 21 Jul 2020 11:56:28 +0200 (CEST) IronPort-SDR: He7l/yhF76amX28Mu1B47+sTyIHQtlxgTQHVZLa7adxU0w/9e6JnnUdi90n28+FlRnb0Imf+9w +IsXhhiTuLIA== X-IronPort-AV: E=McAfee;i="6000,8403,9688"; a="138191475" X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="138191475" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2020 02:56:28 -0700 IronPort-SDR: xuH9gWtzOJ0vhiB3STLmCFEBj+pEqjf75J8xYQ83BdgpwDBaTns1T8+bFpLe3rN1P80WC81qAd BW0SYfnB9l6w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,378,1589266800"; d="scan'208";a="488024552" Received: from silpixa00399126.ir.intel.com ([10.237.222.36]) by fmsmga005.fm.intel.com with ESMTP; 21 Jul 2020 02:56:26 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: cheng1.jiang@intel.com, patrick.fu@intel.com, kevin.laatz@intel.com, Bruce Richardson Date: Tue, 21 Jul 2020 10:51:40 +0100 Message-Id: <20200721095140.719297-21-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200721095140.719297-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 20.11 20/20] raw/ioat: clean up use of common test function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Now that all devices can pass the same set of unit tests, elminate the temporary idxd_rawdev_test function and move the prototype for ioat_rawdev_test to the proper internal header file, to be used by all device instances. Signed-off-by: Bruce Richardson --- drivers/raw/ioat/idxd_pci.c | 2 +- drivers/raw/ioat/idxd_vdev.c | 2 +- drivers/raw/ioat/ioat_private.h | 4 ++-- drivers/raw/ioat/ioat_rawdev.c | 2 -- drivers/raw/ioat/ioat_rawdev_test.c | 7 ------- 5 files changed, 4 insertions(+), 13 deletions(-) diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c index a426897b2..d5e87d2e4 100644 --- a/drivers/raw/ioat/idxd_pci.c +++ b/drivers/raw/ioat/idxd_pci.c @@ -102,7 +102,7 @@ idxd_pci_dev_start(struct rte_rawdev *dev) } static const struct rte_rawdev_ops idxd_pci_ops = { - .dev_selftest = idxd_rawdev_test, + .dev_selftest = ioat_rawdev_test, .dump = idxd_dev_dump, .dev_configure = idxd_dev_configure, .dev_start = idxd_pci_dev_start, diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c index aa2693368..7c3c791ce 100644 --- a/drivers/raw/ioat/idxd_vdev.c +++ b/drivers/raw/ioat/idxd_vdev.c @@ -81,7 +81,7 @@ idxd_vdev_start(struct rte_rawdev *dev) } static const struct rte_rawdev_ops idxd_vdev_ops = { - .dev_selftest = idxd_rawdev_test, + .dev_selftest = ioat_rawdev_test, .dump = idxd_dev_dump, .dev_configure = idxd_dev_configure, .dev_start = idxd_vdev_start, diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h index 46465e675..0c626e6bf 100644 --- a/drivers/raw/ioat/ioat_private.h +++ b/drivers/raw/ioat/ioat_private.h @@ -65,6 +65,8 @@ int ioat_xstats_get_names(const struct rte_rawdev *dev, int ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids, uint32_t nb_ids); +extern int ioat_rawdev_test(uint16_t dev_id); + extern int idxd_rawdev_create(const char *name, struct rte_device *dev, const struct idxd_rawdev *idxd, const struct rte_rawdev_ops *ops); @@ -75,8 +77,6 @@ extern int idxd_dev_configure(const struct rte_rawdev *dev, extern int idxd_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info, size_t info_size); -extern int idxd_rawdev_test(uint16_t dev_id); - extern int idxd_dev_dump(struct rte_rawdev *dev, FILE *f); #endif /* _IOAT_PRIVATE_H_ */ diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index e3c98a825..1b6a58073 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -121,8 +121,6 @@ ioat_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info, return 0; } -extern int ioat_rawdev_test(uint16_t dev_id); - static int ioat_rawdev_create(const char *name, struct rte_pci_device *dev) { diff --git a/drivers/raw/ioat/ioat_rawdev_test.c b/drivers/raw/ioat/ioat_rawdev_test.c index 678d135c4..626299525 100644 --- a/drivers/raw/ioat/ioat_rawdev_test.c +++ b/drivers/raw/ioat/ioat_rawdev_test.c @@ -253,10 +253,3 @@ ioat_rawdev_test(uint16_t dev_id) free(ids); return -1; } - -int -idxd_rawdev_test(uint16_t dev_id) -{ - rte_rawdev_dump(dev_id, stdout); - return ioat_rawdev_test(dev_id); -}