Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/73387/?format=api
{ "id": 73387, "url": "https://patches.dpdk.org/api/patches/73387/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/20200707092244.12791-8-hemant.agrawal@nxp.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20200707092244.12791-8-hemant.agrawal@nxp.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20200707092244.12791-8-hemant.agrawal@nxp.com", "date": "2020-07-07T09:22:22", "name": "[v2,07/29] bus/fslmc: support portal migration", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": true, "hash": "7b895de5a6ed143eb63ea230f4a3ff06bacad05a", "submitter": { "id": 477, "url": "https://patches.dpdk.org/api/people/477/?format=api", "name": "Hemant Agrawal", "email": "hemant.agrawal@nxp.com" }, "delegate": { "id": 319, "url": "https://patches.dpdk.org/api/users/319/?format=api", "username": "fyigit", "first_name": "Ferruh", "last_name": "Yigit", "email": "ferruh.yigit@amd.com" }, "mbox": "https://patches.dpdk.org/project/dpdk/patch/20200707092244.12791-8-hemant.agrawal@nxp.com/mbox/", "series": [ { "id": 10842, "url": "https://patches.dpdk.org/api/series/10842/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/list/?series=10842", "date": "2020-07-07T09:22:15", "name": "NXP DPAAx enhancements", "version": 2, "mbox": "https://patches.dpdk.org/series/10842/mbox/" } ], "comments": "https://patches.dpdk.org/api/patches/73387/comments/", "check": "warning", "checks": "https://patches.dpdk.org/api/patches/73387/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from dpdk.org (dpdk.org [92.243.14.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id A7870A00BE;\n\tTue, 7 Jul 2020 11:28:13 +0200 (CEST)", "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id A6BFE1DD12;\n\tTue, 7 Jul 2020 11:27:09 +0200 (CEST)", "from inva020.nxp.com (inva020.nxp.com [92.121.34.13])\n by dpdk.org (Postfix) with ESMTP id 3D9A81DC86\n for <dev@dpdk.org>; Tue, 7 Jul 2020 11:27:03 +0200 (CEST)", "from inva020.nxp.com (localhost [127.0.0.1])\n by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 1FC5A1A0A58;\n Tue, 7 Jul 2020 11:27:03 +0200 (CEST)", "from invc005.ap-rdc01.nxp.com (invc005.ap-rdc01.nxp.com\n [165.114.16.14])\n by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 088021A0A30;\n Tue, 7 Jul 2020 11:27:01 +0200 (CEST)", "from bf-netperf1.ap.freescale.net (bf-netperf1.ap.freescale.net\n [10.232.133.63])\n by invc005.ap-rdc01.nxp.com (Postfix) with ESMTP id BEE2640305;\n Tue, 7 Jul 2020 17:26:57 +0800 (SGT)" ], "From": "Hemant Agrawal <hemant.agrawal@nxp.com>", "To": "dev@dpdk.org", "Cc": "ferruh.yigit@intel.com,\n\tNipun Gupta <nipun.gupta@nxp.com>", "Date": "Tue, 7 Jul 2020 14:52:22 +0530", "Message-Id": "<20200707092244.12791-8-hemant.agrawal@nxp.com>", "X-Mailer": "git-send-email 2.17.1", "In-Reply-To": "<20200707092244.12791-1-hemant.agrawal@nxp.com>", "References": "<20200527132326.1382-1-hemant.agrawal@nxp.com>\n <20200707092244.12791-1-hemant.agrawal@nxp.com>", "X-Virus-Scanned": "ClamAV using ClamSMTP", "Subject": "[dpdk-dev] [PATCH v2 07/29] bus/fslmc: support portal migration", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "From: Nipun Gupta <nipun.gupta@nxp.com>\n\nThe patch adds support for portal migration by disabling stashing\nfor the portals which is used in the non-affined threads, or on\nthreads affined to multiple cores\n\nSigned-off-by: Nipun Gupta <nipun.gupta@nxp.com>\n---\n doc/guides/rel_notes/release_20_08.rst | 5 +\n drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 83 +----\n .../bus/fslmc/qbman/include/fsl_qbman_debug.h | 1 +\n .../fslmc/qbman/include/fsl_qbman_portal.h | 8 +-\n drivers/bus/fslmc/qbman/qbman_portal.c | 340 +++++++++++++++++-\n drivers/bus/fslmc/qbman/qbman_portal.h | 19 +-\n drivers/bus/fslmc/qbman/qbman_sys.h | 135 ++++++-\n 7 files changed, 508 insertions(+), 83 deletions(-)", "diff": "diff --git a/doc/guides/rel_notes/release_20_08.rst b/doc/guides/rel_notes/release_20_08.rst\nindex ffae463f4..d915fce12 100644\n--- a/doc/guides/rel_notes/release_20_08.rst\n+++ b/doc/guides/rel_notes/release_20_08.rst\n@@ -119,6 +119,11 @@ New Features\n See the :doc:`../sample_app_ug/l2_forward_real_virtual` for more\n details of this parameter usage.\n \n+* **Updated NXP dpaa2 ethdev PMD.**\n+\n+ Updated the NXP dpaa2 ethdev with new features and improvements, including:\n+\n+ * Added support to use datapath APIs from non-EAL pthread\n \n Removed Items\n -------------\ndiff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c\nindex 5a12ff35d..97be76116 100644\n--- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c\n+++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c\n@@ -53,10 +53,6 @@ static uint32_t io_space_count;\n /* Variable to store DPAA2 platform type */\n uint32_t dpaa2_svr_family;\n \n-/* Physical core id for lcores running on dpaa2. */\n-/* DPAA2 only support 1 lcore to 1 phy cpu mapping */\n-static unsigned int dpaa2_cpu[RTE_MAX_LCORE];\n-\n /* Variable to store DPAA2 DQRR size */\n uint8_t dpaa2_dqrr_size;\n /* Variable to store DPAA2 EQCR size */\n@@ -159,7 +155,7 @@ dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id, int cpu_id)\n \t\treturn;\n \t}\n \n-\tcpu_mask = cpu_mask << dpaa2_cpu[cpu_id];\n+\tcpu_mask = cpu_mask << cpu_id;\n \tsnprintf(command, COMMAND_LEN, \"echo %X > /proc/irq/%s/smp_affinity\",\n \t\t cpu_mask, token);\n \tret = system(command);\n@@ -228,17 +224,9 @@ static void dpaa2_dpio_intr_deinit(struct dpaa2_dpio_dev *dpio_dev)\n #endif\n \n static int\n-dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev)\n+dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id)\n {\n \tint sdest, ret;\n-\tint cpu_id;\n-\n-\t/* Set the Stashing Destination */\n-\tcpu_id = dpaa2_get_core_id();\n-\tif (cpu_id < 0) {\n-\t\tDPAA2_BUS_ERR(\"Thread not affined to a single core\");\n-\t\treturn -1;\n-\t}\n \n \t/* Set the STASH Destination depending on Current CPU ID.\n \t * Valid values of SDEST are 4,5,6,7. Where,\n@@ -277,6 +265,7 @@ static void dpaa2_put_qbman_swp(struct dpaa2_dpio_dev *dpio_dev)\n static struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(void)\n {\n \tstruct dpaa2_dpio_dev *dpio_dev = NULL;\n+\tint cpu_id;\n \tint ret;\n \n \t/* Get DPIO dev handle from list using index */\n@@ -292,11 +281,19 @@ static struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(void)\n \tDPAA2_BUS_DEBUG(\"New Portal %p (%d) affined thread - %lu\",\n \t\t\tdpio_dev, dpio_dev->index, syscall(SYS_gettid));\n \n-\tret = dpaa2_configure_stashing(dpio_dev);\n-\tif (ret) {\n-\t\tDPAA2_BUS_ERR(\"dpaa2_configure_stashing failed\");\n-\t\trte_atomic16_clear(&dpio_dev->ref_count);\n-\t\treturn NULL;\n+\t/* Set the Stashing Destination */\n+\tcpu_id = dpaa2_get_core_id();\n+\tif (cpu_id < 0) {\n+\t\tDPAA2_BUS_WARN(\"Thread not affined to a single core\");\n+\t\tif (dpaa2_svr_family != SVR_LX2160A)\n+\t\t\tqbman_swp_update(dpio_dev->sw_portal, 1);\n+\t} else {\n+\t\tret = dpaa2_configure_stashing(dpio_dev, cpu_id);\n+\t\tif (ret) {\n+\t\t\tDPAA2_BUS_ERR(\"dpaa2_configure_stashing failed\");\n+\t\t\trte_atomic16_clear(&dpio_dev->ref_count);\n+\t\t\treturn NULL;\n+\t\t}\n \t}\n \n \tret = pthread_setspecific(dpaa2_portal_key, (void *)dpio_dev);\n@@ -363,46 +360,6 @@ static void dpaa2_portal_finish(void *arg)\n \tpthread_setspecific(dpaa2_portal_key, NULL);\n }\n \n-/*\n- * This checks for not supported lcore mappings as well as get the physical\n- * cpuid for the lcore.\n- * one lcore can only map to 1 cpu i.e. 1@10-14 not supported.\n- * one cpu can be mapped to more than one lcores.\n- */\n-static int\n-dpaa2_check_lcore_cpuset(void)\n-{\n-\tunsigned int lcore_id, i;\n-\tint ret = 0;\n-\n-\tfor (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++)\n-\t\tdpaa2_cpu[lcore_id] = 0xffffffff;\n-\n-\tfor (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {\n-\t\trte_cpuset_t cpuset = rte_lcore_cpuset(lcore_id);\n-\n-\t\tfor (i = 0; i < CPU_SETSIZE; i++) {\n-\t\t\tif (!CPU_ISSET(i, &cpuset))\n-\t\t\t\tcontinue;\n-\t\t\tif (i >= RTE_MAX_LCORE) {\n-\t\t\t\tDPAA2_BUS_ERR(\"ERR:lcore map to core %u (>= %u) not supported\",\n-\t\t\t\t\ti, RTE_MAX_LCORE);\n-\t\t\t\tret = -1;\n-\t\t\t\tcontinue;\n-\t\t\t}\n-\t\t\tRTE_LOG(DEBUG, EAL, \"lcore id = %u cpu=%u\\n\",\n-\t\t\t\tlcore_id, i);\n-\t\t\tif (dpaa2_cpu[lcore_id] != 0xffffffff) {\n-\t\t\t\tDPAA2_BUS_ERR(\"ERR:lcore map to multi-cpu not supported\");\n-\t\t\t\tret = -1;\n-\t\t\t\tcontinue;\n-\t\t\t}\n-\t\t\tdpaa2_cpu[lcore_id] = i;\n-\t\t}\n-\t}\n-\treturn ret;\n-}\n-\n static int\n dpaa2_create_dpio_device(int vdev_fd,\n \t\t\t struct vfio_device_info *obj_info,\n@@ -413,7 +370,6 @@ dpaa2_create_dpio_device(int vdev_fd,\n \tstruct qbman_swp_desc p_des;\n \tstruct dpio_attr attr;\n \tint ret;\n-\tstatic int check_lcore_cpuset;\n \n \tif (obj_info->num_regions < NUM_DPIO_REGIONS) {\n \t\tDPAA2_BUS_ERR(\"Not sufficient number of DPIO regions\");\n@@ -433,13 +389,6 @@ dpaa2_create_dpio_device(int vdev_fd,\n \t/* Using single portal for all devices */\n \tdpio_dev->mc_portal = dpaa2_get_mcp_ptr(MC_PORTAL_INDEX);\n \n-\tif (!check_lcore_cpuset) {\n-\t\tcheck_lcore_cpuset = 1;\n-\n-\t\tif (dpaa2_check_lcore_cpuset() < 0)\n-\t\t\tgoto err;\n-\t}\n-\n \tdpio_dev->dpio = rte_zmalloc(NULL, sizeof(struct fsl_mc_io),\n \t\t\t\t RTE_CACHE_LINE_SIZE);\n \tif (!dpio_dev->dpio) {\ndiff --git a/drivers/bus/fslmc/qbman/include/fsl_qbman_debug.h b/drivers/bus/fslmc/qbman/include/fsl_qbman_debug.h\nindex 11267d439..54096e877 100644\n--- a/drivers/bus/fslmc/qbman/include/fsl_qbman_debug.h\n+++ b/drivers/bus/fslmc/qbman/include/fsl_qbman_debug.h\n@@ -1,5 +1,6 @@\n /* SPDX-License-Identifier: BSD-3-Clause\n * Copyright (C) 2015 Freescale Semiconductor, Inc.\n+ * Copyright 2020 NXP\n */\n #ifndef _FSL_QBMAN_DEBUG_H\n #define _FSL_QBMAN_DEBUG_H\ndiff --git a/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h b/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h\nindex f820077d2..eb68c9cab 100644\n--- a/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h\n+++ b/drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h\n@@ -1,7 +1,7 @@\n /* SPDX-License-Identifier: BSD-3-Clause\n *\n * Copyright (C) 2014 Freescale Semiconductor, Inc.\n- * Copyright 2015-2019 NXP\n+ * Copyright 2015-2020 NXP\n *\n */\n #ifndef _FSL_QBMAN_PORTAL_H\n@@ -44,6 +44,12 @@ extern uint32_t dpaa2_svr_family;\n */\n struct qbman_swp *qbman_swp_init(const struct qbman_swp_desc *d);\n \n+/**\n+ * qbman_swp_update() - Update portal cacheability attributes.\n+ * @p: the given qbman swp portal\n+ */\n+int qbman_swp_update(struct qbman_swp *p, int stash_off);\n+\n /**\n * qbman_swp_finish() - Create and destroy a functional object representing\n * the given QBMan portal descriptor.\ndiff --git a/drivers/bus/fslmc/qbman/qbman_portal.c b/drivers/bus/fslmc/qbman/qbman_portal.c\nindex d7ff74c7a..57f50b0d8 100644\n--- a/drivers/bus/fslmc/qbman/qbman_portal.c\n+++ b/drivers/bus/fslmc/qbman/qbman_portal.c\n@@ -1,7 +1,7 @@\n /* SPDX-License-Identifier: BSD-3-Clause\n *\n * Copyright (C) 2014-2016 Freescale Semiconductor, Inc.\n- * Copyright 2018-2019 NXP\n+ * Copyright 2018-2020 NXP\n *\n */\n \n@@ -82,6 +82,10 @@ qbman_swp_enqueue_ring_mode_cinh_direct(struct qbman_swp *s,\n \t\tconst struct qbman_eq_desc *d,\n \t\tconst struct qbman_fd *fd);\n static int\n+qbman_swp_enqueue_ring_mode_cinh_direct(struct qbman_swp *s,\n+\t\tconst struct qbman_eq_desc *d,\n+\t\tconst struct qbman_fd *fd);\n+static int\n qbman_swp_enqueue_ring_mode_mem_back(struct qbman_swp *s,\n \t\tconst struct qbman_eq_desc *d,\n \t\tconst struct qbman_fd *fd);\n@@ -99,6 +103,12 @@ qbman_swp_enqueue_multiple_cinh_direct(struct qbman_swp *s,\n \t\tuint32_t *flags,\n \t\tint num_frames);\n static int\n+qbman_swp_enqueue_multiple_cinh_direct(struct qbman_swp *s,\n+\t\tconst struct qbman_eq_desc *d,\n+\t\tconst struct qbman_fd *fd,\n+\t\tuint32_t *flags,\n+\t\tint num_frames);\n+static int\n qbman_swp_enqueue_multiple_mem_back(struct qbman_swp *s,\n \t\tconst struct qbman_eq_desc *d,\n \t\tconst struct qbman_fd *fd,\n@@ -118,6 +128,12 @@ qbman_swp_enqueue_multiple_fd_cinh_direct(struct qbman_swp *s,\n \t\tuint32_t *flags,\n \t\tint num_frames);\n static int\n+qbman_swp_enqueue_multiple_fd_cinh_direct(struct qbman_swp *s,\n+\t\tconst struct qbman_eq_desc *d,\n+\t\tstruct qbman_fd **fd,\n+\t\tuint32_t *flags,\n+\t\tint num_frames);\n+static int\n qbman_swp_enqueue_multiple_fd_mem_back(struct qbman_swp *s,\n \t\tconst struct qbman_eq_desc *d,\n \t\tstruct qbman_fd **fd,\n@@ -135,6 +151,11 @@ qbman_swp_enqueue_multiple_desc_cinh_direct(struct qbman_swp *s,\n \t\tconst struct qbman_fd *fd,\n \t\tint num_frames);\n static int\n+qbman_swp_enqueue_multiple_desc_cinh_direct(struct qbman_swp *s,\n+\t\tconst struct qbman_eq_desc *d,\n+\t\tconst struct qbman_fd *fd,\n+\t\tint num_frames);\n+static int\n qbman_swp_enqueue_multiple_desc_mem_back(struct qbman_swp *s,\n \t\tconst struct qbman_eq_desc *d,\n \t\tconst struct qbman_fd *fd,\n@@ -143,9 +164,12 @@ qbman_swp_enqueue_multiple_desc_mem_back(struct qbman_swp *s,\n static int\n qbman_swp_pull_direct(struct qbman_swp *s, struct qbman_pull_desc *d);\n static int\n+qbman_swp_pull_cinh_direct(struct qbman_swp *s, struct qbman_pull_desc *d);\n+static int\n qbman_swp_pull_mem_back(struct qbman_swp *s, struct qbman_pull_desc *d);\n \n const struct qbman_result *qbman_swp_dqrr_next_direct(struct qbman_swp *s);\n+const struct qbman_result *qbman_swp_dqrr_next_cinh_direct(struct qbman_swp *s);\n const struct qbman_result *qbman_swp_dqrr_next_mem_back(struct qbman_swp *s);\n \n static int\n@@ -153,6 +177,10 @@ qbman_swp_release_direct(struct qbman_swp *s,\n \t\tconst struct qbman_release_desc *d,\n \t\tconst uint64_t *buffers, unsigned int num_buffers);\n static int\n+qbman_swp_release_cinh_direct(struct qbman_swp *s,\n+\t\tconst struct qbman_release_desc *d,\n+\t\tconst uint64_t *buffers, unsigned int num_buffers);\n+static int\n qbman_swp_release_mem_back(struct qbman_swp *s,\n \t\tconst struct qbman_release_desc *d,\n \t\tconst uint64_t *buffers, unsigned int num_buffers);\n@@ -327,6 +355,28 @@ struct qbman_swp *qbman_swp_init(const struct qbman_swp_desc *d)\n \treturn p;\n }\n \n+int qbman_swp_update(struct qbman_swp *p, int stash_off)\n+{\n+\tconst struct qbman_swp_desc *d = &p->desc;\n+\tstruct qbman_swp_sys *s = &p->sys;\n+\tint ret;\n+\n+\t/* Nothing needs to be done for QBMAN rev > 5000 with fast access */\n+\tif ((qman_version & QMAN_REV_MASK) >= QMAN_REV_5000\n+\t\t\t&& (d->cena_access_mode == qman_cena_fastest_access))\n+\t\treturn 0;\n+\n+\tret = qbman_swp_sys_update(s, d, p->dqrr.dqrr_size, stash_off);\n+\tif (ret) {\n+\t\tpr_err(\"qbman_swp_sys_init() failed %d\\n\", ret);\n+\t\treturn ret;\n+\t}\n+\n+\tp->stash_off = stash_off;\n+\n+\treturn 0;\n+}\n+\n void qbman_swp_finish(struct qbman_swp *p)\n {\n #ifdef QBMAN_CHECKING\n@@ -462,6 +512,27 @@ void qbman_swp_mc_submit(struct qbman_swp *p, void *cmd, uint8_t cmd_verb)\n #endif\n }\n \n+void qbman_swp_mc_submit_cinh(struct qbman_swp *p, void *cmd, uint8_t cmd_verb)\n+{\n+\tuint8_t *v = cmd;\n+#ifdef QBMAN_CHECKING\n+\tQBMAN_BUG_ON(!(p->mc.check != swp_mc_can_submit));\n+#endif\n+\t/* TBD: \"|=\" is going to hurt performance. Need to move as many fields\n+\t * out of word zero, and for those that remain, the \"OR\" needs to occur\n+\t * at the caller side. This debug check helps to catch cases where the\n+\t * caller wants to OR but has forgotten to do so.\n+\t */\n+\tQBMAN_BUG_ON((*v & cmd_verb) != *v);\n+\tdma_wmb();\n+\t*v = cmd_verb | p->mc.valid_bit;\n+\tqbman_cinh_write_complete(&p->sys, QBMAN_CENA_SWP_CR, cmd);\n+\tclean(cmd);\n+#ifdef QBMAN_CHECKING\n+\tp->mc.check = swp_mc_can_poll;\n+#endif\n+}\n+\n void *qbman_swp_mc_result(struct qbman_swp *p)\n {\n \tuint32_t *ret, verb;\n@@ -500,6 +571,27 @@ void *qbman_swp_mc_result(struct qbman_swp *p)\n \treturn ret;\n }\n \n+void *qbman_swp_mc_result_cinh(struct qbman_swp *p)\n+{\n+\tuint32_t *ret, verb;\n+#ifdef QBMAN_CHECKING\n+\tQBMAN_BUG_ON(p->mc.check != swp_mc_can_poll);\n+#endif\n+\tret = qbman_cinh_read_shadow(&p->sys,\n+\t\t\t QBMAN_CENA_SWP_RR(p->mc.valid_bit));\n+\t/* Remove the valid-bit -\n+\t * command completed iff the rest is non-zero\n+\t */\n+\tverb = ret[0] & ~QB_VALID_BIT;\n+\tif (!verb)\n+\t\treturn NULL;\n+\tp->mc.valid_bit ^= QB_VALID_BIT;\n+#ifdef QBMAN_CHECKING\n+\tp->mc.check = swp_mc_can_start;\n+#endif\n+\treturn ret;\n+}\n+\n /***********/\n /* Enqueue */\n /***********/\n@@ -640,6 +732,16 @@ static inline void qbman_write_eqcr_am_rt_register(struct qbman_swp *p,\n \t\t\t\t QMAN_RT_MODE);\n }\n \n+static void memcpy_byte_by_byte(void *to, const void *from, size_t n)\n+{\n+\tconst uint8_t *src = from;\n+\tvolatile uint8_t *dest = to;\n+\tsize_t i;\n+\n+\tfor (i = 0; i < n; i++)\n+\t\tdest[i] = src[i];\n+}\n+\n \n static int qbman_swp_enqueue_array_mode_direct(struct qbman_swp *s,\n \t\t\t\t\t const struct qbman_eq_desc *d,\n@@ -754,7 +856,7 @@ static int qbman_swp_enqueue_ring_mode_cinh_direct(\n \t\t\treturn -EBUSY;\n \t}\n \n-\tp = qbman_cena_write_start_wo_shadow(&s->sys,\n+\tp = qbman_cinh_write_start_wo_shadow(&s->sys,\n \t\t\tQBMAN_CENA_SWP_EQCR(s->eqcr.pi & half_mask));\n \tmemcpy(&p[1], &cl[1], 28);\n \tmemcpy(&p[8], fd, sizeof(*fd));\n@@ -762,8 +864,6 @@ static int qbman_swp_enqueue_ring_mode_cinh_direct(\n \n \t/* Set the verb byte, have to substitute in the valid-bit */\n \tp[0] = cl[0] | s->eqcr.pi_vb;\n-\tqbman_cena_write_complete_wo_shadow(&s->sys,\n-\t\t\tQBMAN_CENA_SWP_EQCR(s->eqcr.pi & half_mask));\n \ts->eqcr.pi++;\n \ts->eqcr.pi &= full_mask;\n \ts->eqcr.available--;\n@@ -815,7 +915,10 @@ static int qbman_swp_enqueue_ring_mode(struct qbman_swp *s,\n \t\t\t\t const struct qbman_eq_desc *d,\n \t\t\t\t const struct qbman_fd *fd)\n {\n-\treturn qbman_swp_enqueue_ring_mode_ptr(s, d, fd);\n+\tif (!s->stash_off)\n+\t\treturn qbman_swp_enqueue_ring_mode_ptr(s, d, fd);\n+\telse\n+\t\treturn qbman_swp_enqueue_ring_mode_cinh_direct(s, d, fd);\n }\n \n int qbman_swp_enqueue(struct qbman_swp *s, const struct qbman_eq_desc *d,\n@@ -1025,7 +1128,12 @@ int qbman_swp_enqueue_multiple(struct qbman_swp *s,\n \t\t\t\t uint32_t *flags,\n \t\t\t\t int num_frames)\n {\n-\treturn qbman_swp_enqueue_multiple_ptr(s, d, fd, flags, num_frames);\n+\tif (!s->stash_off)\n+\t\treturn qbman_swp_enqueue_multiple_ptr(s, d, fd, flags,\n+\t\t\t\t\t\tnum_frames);\n+\telse\n+\t\treturn qbman_swp_enqueue_multiple_cinh_direct(s, d, fd, flags,\n+\t\t\t\t\t\tnum_frames);\n }\n \n static int qbman_swp_enqueue_multiple_fd_direct(struct qbman_swp *s,\n@@ -1233,7 +1341,12 @@ int qbman_swp_enqueue_multiple_fd(struct qbman_swp *s,\n \t\t\t\t\t uint32_t *flags,\n \t\t\t\t\t int num_frames)\n {\n-\treturn qbman_swp_enqueue_multiple_fd_ptr(s, d, fd, flags, num_frames);\n+\tif (!s->stash_off)\n+\t\treturn qbman_swp_enqueue_multiple_fd_ptr(s, d, fd, flags,\n+\t\t\t\t\tnum_frames);\n+\telse\n+\t\treturn qbman_swp_enqueue_multiple_fd_cinh_direct(s, d, fd,\n+\t\t\t\t\tflags, num_frames);\n }\n \n static int qbman_swp_enqueue_multiple_desc_direct(struct qbman_swp *s,\n@@ -1426,7 +1539,13 @@ int qbman_swp_enqueue_multiple_desc(struct qbman_swp *s,\n \t\t\t\t\t const struct qbman_fd *fd,\n \t\t\t\t\t int num_frames)\n {\n-\treturn qbman_swp_enqueue_multiple_desc_ptr(s, d, fd, num_frames);\n+\tif (!s->stash_off)\n+\t\treturn qbman_swp_enqueue_multiple_desc_ptr(s, d, fd,\n+\t\t\t\t\tnum_frames);\n+\telse\n+\t\treturn qbman_swp_enqueue_multiple_desc_cinh_direct(s, d, fd,\n+\t\t\t\t\tnum_frames);\n+\n }\n \n /*************************/\n@@ -1574,6 +1693,30 @@ static int qbman_swp_pull_direct(struct qbman_swp *s,\n \treturn 0;\n }\n \n+static int qbman_swp_pull_cinh_direct(struct qbman_swp *s,\n+\t\t\t\t struct qbman_pull_desc *d)\n+{\n+\tuint32_t *p;\n+\tuint32_t *cl = qb_cl(d);\n+\n+\tif (!atomic_dec_and_test(&s->vdq.busy)) {\n+\t\tatomic_inc(&s->vdq.busy);\n+\t\treturn -EBUSY;\n+\t}\n+\n+\td->pull.tok = s->sys.idx + 1;\n+\ts->vdq.storage = (void *)(size_t)d->pull.rsp_addr_virt;\n+\tp = qbman_cinh_write_start_wo_shadow(&s->sys, QBMAN_CENA_SWP_VDQCR);\n+\tmemcpy_byte_by_byte(&p[1], &cl[1], 12);\n+\n+\t/* Set the verb byte, have to substitute in the valid-bit */\n+\tlwsync();\n+\tp[0] = cl[0] | s->vdq.valid_bit;\n+\ts->vdq.valid_bit ^= QB_VALID_BIT;\n+\n+\treturn 0;\n+}\n+\n static int qbman_swp_pull_mem_back(struct qbman_swp *s,\n \t\t\t\t struct qbman_pull_desc *d)\n {\n@@ -1601,7 +1744,10 @@ static int qbman_swp_pull_mem_back(struct qbman_swp *s,\n \n int qbman_swp_pull(struct qbman_swp *s, struct qbman_pull_desc *d)\n {\n-\treturn qbman_swp_pull_ptr(s, d);\n+\tif (!s->stash_off)\n+\t\treturn qbman_swp_pull_ptr(s, d);\n+\telse\n+\t\treturn qbman_swp_pull_cinh_direct(s, d);\n }\n \n /****************/\n@@ -1638,7 +1784,10 @@ void qbman_swp_prefetch_dqrr_next(struct qbman_swp *s)\n */\n const struct qbman_result *qbman_swp_dqrr_next(struct qbman_swp *s)\n {\n-\treturn qbman_swp_dqrr_next_ptr(s);\n+\tif (!s->stash_off)\n+\t\treturn qbman_swp_dqrr_next_ptr(s);\n+\telse\n+\t\treturn qbman_swp_dqrr_next_cinh_direct(s);\n }\n \n const struct qbman_result *qbman_swp_dqrr_next_direct(struct qbman_swp *s)\n@@ -1718,6 +1867,81 @@ const struct qbman_result *qbman_swp_dqrr_next_direct(struct qbman_swp *s)\n \treturn p;\n }\n \n+const struct qbman_result *qbman_swp_dqrr_next_cinh_direct(struct qbman_swp *s)\n+{\n+\tuint32_t verb;\n+\tuint32_t response_verb;\n+\tuint32_t flags;\n+\tconst struct qbman_result *p;\n+\n+\t/* Before using valid-bit to detect if something is there, we have to\n+\t * handle the case of the DQRR reset bug...\n+\t */\n+\tif (s->dqrr.reset_bug) {\n+\t\t/* We pick up new entries by cache-inhibited producer index,\n+\t\t * which means that a non-coherent mapping would require us to\n+\t\t * invalidate and read *only* once that PI has indicated that\n+\t\t * there's an entry here. The first trip around the DQRR ring\n+\t\t * will be much less efficient than all subsequent trips around\n+\t\t * it...\n+\t\t */\n+\t\tuint8_t pi = qbman_cinh_read(&s->sys, QBMAN_CINH_SWP_DQPI) &\n+\t\t\t QMAN_DQRR_PI_MASK;\n+\n+\t\t/* there are new entries if pi != next_idx */\n+\t\tif (pi == s->dqrr.next_idx)\n+\t\t\treturn NULL;\n+\n+\t\t/* if next_idx is/was the last ring index, and 'pi' is\n+\t\t * different, we can disable the workaround as all the ring\n+\t\t * entries have now been DMA'd to so valid-bit checking is\n+\t\t * repaired. Note: this logic needs to be based on next_idx\n+\t\t * (which increments one at a time), rather than on pi (which\n+\t\t * can burst and wrap-around between our snapshots of it).\n+\t\t */\n+\t\tQBMAN_BUG_ON((s->dqrr.dqrr_size - 1) < 0);\n+\t\tif (s->dqrr.next_idx == (s->dqrr.dqrr_size - 1u)) {\n+\t\t\tpr_debug(\"DEBUG: next_idx=%d, pi=%d, clear reset bug\\n\",\n+\t\t\t\t s->dqrr.next_idx, pi);\n+\t\t\ts->dqrr.reset_bug = 0;\n+\t\t}\n+\t}\n+\tp = qbman_cinh_read_wo_shadow(&s->sys,\n+\t\t\tQBMAN_CENA_SWP_DQRR(s->dqrr.next_idx));\n+\n+\tverb = p->dq.verb;\n+\n+\t/* If the valid-bit isn't of the expected polarity, nothing there. Note,\n+\t * in the DQRR reset bug workaround, we shouldn't need to skip these\n+\t * check, because we've already determined that a new entry is available\n+\t * and we've invalidated the cacheline before reading it, so the\n+\t * valid-bit behaviour is repaired and should tell us what we already\n+\t * knew from reading PI.\n+\t */\n+\tif ((verb & QB_VALID_BIT) != s->dqrr.valid_bit)\n+\t\treturn NULL;\n+\n+\t/* There's something there. Move \"next_idx\" attention to the next ring\n+\t * entry (and prefetch it) before returning what we found.\n+\t */\n+\ts->dqrr.next_idx++;\n+\tif (s->dqrr.next_idx == s->dqrr.dqrr_size) {\n+\t\ts->dqrr.next_idx = 0;\n+\t\ts->dqrr.valid_bit ^= QB_VALID_BIT;\n+\t}\n+\t/* If this is the final response to a volatile dequeue command\n+\t * indicate that the vdq is no longer busy\n+\t */\n+\tflags = p->dq.stat;\n+\tresponse_verb = verb & QBMAN_RESPONSE_VERB_MASK;\n+\tif ((response_verb == QBMAN_RESULT_DQ) &&\n+\t (flags & QBMAN_DQ_STAT_VOLATILE) &&\n+\t (flags & QBMAN_DQ_STAT_EXPIRED))\n+\t\tatomic_inc(&s->vdq.busy);\n+\n+\treturn p;\n+}\n+\n const struct qbman_result *qbman_swp_dqrr_next_mem_back(struct qbman_swp *s)\n {\n \tuint32_t verb;\n@@ -2096,6 +2320,37 @@ static int qbman_swp_release_direct(struct qbman_swp *s,\n \treturn 0;\n }\n \n+static int qbman_swp_release_cinh_direct(struct qbman_swp *s,\n+\t\t\t\t const struct qbman_release_desc *d,\n+\t\t\t\t const uint64_t *buffers,\n+\t\t\t\t unsigned int num_buffers)\n+{\n+\tuint32_t *p;\n+\tconst uint32_t *cl = qb_cl(d);\n+\tuint32_t rar = qbman_cinh_read(&s->sys, QBMAN_CINH_SWP_RAR);\n+\n+\tpr_debug(\"RAR=%08x\\n\", rar);\n+\tif (!RAR_SUCCESS(rar))\n+\t\treturn -EBUSY;\n+\n+\tQBMAN_BUG_ON(!num_buffers || (num_buffers > 7));\n+\n+\t/* Start the release command */\n+\tp = qbman_cinh_write_start_wo_shadow(&s->sys,\n+\t\t\t\t QBMAN_CENA_SWP_RCR(RAR_IDX(rar)));\n+\n+\t/* Copy the caller's buffer pointers to the command */\n+\tmemcpy_byte_by_byte(&p[2], buffers, num_buffers * sizeof(uint64_t));\n+\n+\t/* Set the verb byte, have to substitute in the valid-bit and the\n+\t * number of buffers.\n+\t */\n+\tlwsync();\n+\tp[0] = cl[0] | RAR_VB(rar) | num_buffers;\n+\n+\treturn 0;\n+}\n+\n static int qbman_swp_release_mem_back(struct qbman_swp *s,\n \t\t\t\t const struct qbman_release_desc *d,\n \t\t\t\t const uint64_t *buffers,\n@@ -2134,7 +2389,11 @@ int qbman_swp_release(struct qbman_swp *s,\n \t\t\t const uint64_t *buffers,\n \t\t\t unsigned int num_buffers)\n {\n-\treturn qbman_swp_release_ptr(s, d, buffers, num_buffers);\n+\tif (!s->stash_off)\n+\t\treturn qbman_swp_release_ptr(s, d, buffers, num_buffers);\n+\telse\n+\t\treturn qbman_swp_release_cinh_direct(s, d, buffers,\n+\t\t\t\t\t\tnum_buffers);\n }\n \n /*******************/\n@@ -2157,8 +2416,8 @@ struct qbman_acquire_rslt {\n \tuint64_t buf[7];\n };\n \n-int qbman_swp_acquire(struct qbman_swp *s, uint16_t bpid, uint64_t *buffers,\n-\t\t unsigned int num_buffers)\n+static int qbman_swp_acquire_direct(struct qbman_swp *s, uint16_t bpid,\n+\t\t\t\tuint64_t *buffers, unsigned int num_buffers)\n {\n \tstruct qbman_acquire_desc *p;\n \tstruct qbman_acquire_rslt *r;\n@@ -2202,6 +2461,61 @@ int qbman_swp_acquire(struct qbman_swp *s, uint16_t bpid, uint64_t *buffers,\n \treturn (int)r->num;\n }\n \n+static int qbman_swp_acquire_cinh_direct(struct qbman_swp *s, uint16_t bpid,\n+\t\t\tuint64_t *buffers, unsigned int num_buffers)\n+{\n+\tstruct qbman_acquire_desc *p;\n+\tstruct qbman_acquire_rslt *r;\n+\n+\tif (!num_buffers || (num_buffers > 7))\n+\t\treturn -EINVAL;\n+\n+\t/* Start the management command */\n+\tp = qbman_swp_mc_start(s);\n+\n+\tif (!p)\n+\t\treturn -EBUSY;\n+\n+\t/* Encode the caller-provided attributes */\n+\tp->bpid = bpid;\n+\tp->num = num_buffers;\n+\n+\t/* Complete the management command */\n+\tr = qbman_swp_mc_complete_cinh(s, p, QBMAN_MC_ACQUIRE);\n+\tif (!r) {\n+\t\tpr_err(\"qbman: acquire from BPID %d failed, no response\\n\",\n+\t\t bpid);\n+\t\treturn -EIO;\n+\t}\n+\n+\t/* Decode the outcome */\n+\tQBMAN_BUG_ON((r->verb & QBMAN_RESPONSE_VERB_MASK) != QBMAN_MC_ACQUIRE);\n+\n+\t/* Determine success or failure */\n+\tif (r->rslt != QBMAN_MC_RSLT_OK) {\n+\t\tpr_err(\"Acquire buffers from BPID 0x%x failed, code=0x%02x\\n\",\n+\t\t bpid, r->rslt);\n+\t\treturn -EIO;\n+\t}\n+\n+\tQBMAN_BUG_ON(r->num > num_buffers);\n+\n+\t/* Copy the acquired buffers to the caller's array */\n+\tu64_from_le32_copy(buffers, &r->buf[0], r->num);\n+\n+\treturn (int)r->num;\n+}\n+\n+int qbman_swp_acquire(struct qbman_swp *s, uint16_t bpid, uint64_t *buffers,\n+\t\t unsigned int num_buffers)\n+{\n+\tif (!s->stash_off)\n+\t\treturn qbman_swp_acquire_direct(s, bpid, buffers, num_buffers);\n+\telse\n+\t\treturn qbman_swp_acquire_cinh_direct(s, bpid, buffers,\n+\t\t\t\t\tnum_buffers);\n+}\n+\n /*****************/\n /* FQ management */\n /*****************/\ndiff --git a/drivers/bus/fslmc/qbman/qbman_portal.h b/drivers/bus/fslmc/qbman/qbman_portal.h\nindex 3aaacae52..1cf791830 100644\n--- a/drivers/bus/fslmc/qbman/qbman_portal.h\n+++ b/drivers/bus/fslmc/qbman/qbman_portal.h\n@@ -1,7 +1,7 @@\n /* SPDX-License-Identifier: BSD-3-Clause\n *\n * Copyright (C) 2014-2016 Freescale Semiconductor, Inc.\n- * Copyright 2018-2019 NXP\n+ * Copyright 2018-2020 NXP\n *\n */\n \n@@ -102,6 +102,7 @@ struct qbman_swp {\n \t\tuint32_t ci;\n \t\tint available;\n \t} eqcr;\n+\tuint8_t stash_off;\n };\n \n /* -------------------------- */\n@@ -118,7 +119,9 @@ struct qbman_swp {\n */\n void *qbman_swp_mc_start(struct qbman_swp *p);\n void qbman_swp_mc_submit(struct qbman_swp *p, void *cmd, uint8_t cmd_verb);\n+void qbman_swp_mc_submit_cinh(struct qbman_swp *p, void *cmd, uint8_t cmd_verb);\n void *qbman_swp_mc_result(struct qbman_swp *p);\n+void *qbman_swp_mc_result_cinh(struct qbman_swp *p);\n \n /* Wraps up submit + poll-for-result */\n static inline void *qbman_swp_mc_complete(struct qbman_swp *swp, void *cmd,\n@@ -135,6 +138,20 @@ static inline void *qbman_swp_mc_complete(struct qbman_swp *swp, void *cmd,\n \treturn cmd;\n }\n \n+static inline void *qbman_swp_mc_complete_cinh(struct qbman_swp *swp, void *cmd,\n+\t\t\t\t\t uint8_t cmd_verb)\n+{\n+\tint loopvar = 1000;\n+\n+\tqbman_swp_mc_submit_cinh(swp, cmd, cmd_verb);\n+\tdo {\n+\t\tcmd = qbman_swp_mc_result_cinh(swp);\n+\t} while (!cmd && loopvar--);\n+\tQBMAN_BUG_ON(!loopvar);\n+\n+\treturn cmd;\n+}\n+\n /* ---------------------- */\n /* Descriptors/cachelines */\n /* ---------------------- */\ndiff --git a/drivers/bus/fslmc/qbman/qbman_sys.h b/drivers/bus/fslmc/qbman/qbman_sys.h\nindex 55449edf3..61f817c47 100644\n--- a/drivers/bus/fslmc/qbman/qbman_sys.h\n+++ b/drivers/bus/fslmc/qbman/qbman_sys.h\n@@ -1,7 +1,7 @@\n /* SPDX-License-Identifier: BSD-3-Clause\n *\n * Copyright (C) 2014-2016 Freescale Semiconductor, Inc.\n- * Copyright 2019 NXP\n+ * Copyright 2019-2020 NXP\n */\n /* qbman_sys_decl.h and qbman_sys.h are the two platform-specific files in the\n * driver. They are only included via qbman_private.h, which is itself a\n@@ -190,6 +190,34 @@ static inline void qbman_cinh_write(struct qbman_swp_sys *s, uint32_t offset,\n #endif\n }\n \n+static inline void *qbman_cinh_write_start_wo_shadow(struct qbman_swp_sys *s,\n+\t\t\t\t\t\t uint32_t offset)\n+{\n+#ifdef QBMAN_CINH_TRACE\n+\tpr_info(\"qbman_cinh_write_start(%p:%d:0x%03x)\\n\",\n+\t\ts->addr_cinh, s->idx, offset);\n+#endif\n+\tQBMAN_BUG_ON(offset & 63);\n+\treturn (s->addr_cinh + offset);\n+}\n+\n+static inline void qbman_cinh_write_complete(struct qbman_swp_sys *s,\n+\t\t\t\t\t uint32_t offset, void *cmd)\n+{\n+\tconst uint32_t *shadow = cmd;\n+\tint loop;\n+#ifdef QBMAN_CINH_TRACE\n+\tpr_info(\"qbman_cinh_write_complete(%p:%d:0x%03x) %p\\n\",\n+\t\ts->addr_cinh, s->idx, offset, shadow);\n+\thexdump(cmd, 64);\n+#endif\n+\tfor (loop = 15; loop >= 1; loop--)\n+\t\t__raw_writel(shadow[loop], s->addr_cinh +\n+\t\t\t\t\t offset + loop * 4);\n+\tlwsync();\n+\t__raw_writel(shadow[0], s->addr_cinh + offset);\n+}\n+\n static inline uint32_t qbman_cinh_read(struct qbman_swp_sys *s, uint32_t offset)\n {\n \tuint32_t reg = __raw_readl(s->addr_cinh + offset);\n@@ -200,6 +228,35 @@ static inline uint32_t qbman_cinh_read(struct qbman_swp_sys *s, uint32_t offset)\n \treturn reg;\n }\n \n+static inline void *qbman_cinh_read_shadow(struct qbman_swp_sys *s,\n+\t\t\t\t\t uint32_t offset)\n+{\n+\tuint32_t *shadow = (uint32_t *)(s->cena + offset);\n+\tunsigned int loop;\n+#ifdef QBMAN_CINH_TRACE\n+\tpr_info(\" %s (%p:%d:0x%03x) %p\\n\", __func__,\n+\t\ts->addr_cinh, s->idx, offset, shadow);\n+#endif\n+\n+\tfor (loop = 0; loop < 16; loop++)\n+\t\tshadow[loop] = __raw_readl(s->addr_cinh + offset\n+\t\t\t\t\t+ loop * 4);\n+#ifdef QBMAN_CINH_TRACE\n+\thexdump(shadow, 64);\n+#endif\n+\treturn shadow;\n+}\n+\n+static inline void *qbman_cinh_read_wo_shadow(struct qbman_swp_sys *s,\n+\t\t\t\t\t uint32_t offset)\n+{\n+#ifdef QBMAN_CINH_TRACE\n+\tpr_info(\"qbman_cinh_read(%p:%d:0x%03x)\\n\",\n+\t\ts->addr_cinh, s->idx, offset);\n+#endif\n+\treturn s->addr_cinh + offset;\n+}\n+\n static inline void *qbman_cena_write_start(struct qbman_swp_sys *s,\n \t\t\t\t\t uint32_t offset)\n {\n@@ -476,6 +533,82 @@ static inline int qbman_swp_sys_init(struct qbman_swp_sys *s,\n \treturn 0;\n }\n \n+static inline int qbman_swp_sys_update(struct qbman_swp_sys *s,\n+\t\t\t\t const struct qbman_swp_desc *d,\n+\t\t\t\t uint8_t dqrr_size,\n+\t\t\t\t int stash_off)\n+{\n+\tuint32_t reg;\n+\tint i;\n+\tint cena_region_size = 4*1024;\n+\tuint8_t est = 1;\n+#ifdef RTE_ARCH_64\n+\tuint8_t wn = CENA_WRITE_ENABLE;\n+#else\n+\tuint8_t wn = CINH_WRITE_ENABLE;\n+#endif\n+\n+\tif (stash_off)\n+\t\twn = CINH_WRITE_ENABLE;\n+\n+\tQBMAN_BUG_ON(d->idx < 0);\n+#ifdef QBMAN_CHECKING\n+\t/* We should never be asked to initialise for a portal that isn't in\n+\t * the power-on state. (Ie. don't forget to reset portals when they are\n+\t * decommissioned!)\n+\t */\n+\treg = qbman_cinh_read(s, QBMAN_CINH_SWP_CFG);\n+\tQBMAN_BUG_ON(reg);\n+#endif\n+\tif ((d->qman_version & QMAN_REV_MASK) >= QMAN_REV_5000\n+\t\t\t&& (d->cena_access_mode == qman_cena_fastest_access))\n+\t\tmemset(s->addr_cena, 0, cena_region_size);\n+\telse {\n+\t\t/* Invalidate the portal memory.\n+\t\t * This ensures no stale cache lines\n+\t\t */\n+\t\tfor (i = 0; i < cena_region_size; i += 64)\n+\t\t\tdccivac(s->addr_cena + i);\n+\t}\n+\n+\tif (dpaa2_svr_family == SVR_LS1080A)\n+\t\test = 0;\n+\n+\tif (s->eqcr_mode == qman_eqcr_vb_array) {\n+\t\treg = qbman_set_swp_cfg(dqrr_size, wn,\n+\t\t\t\t\t0, 3, 2, 3, 1, 1, 1, 1, 1, 1);\n+\t} else {\n+\t\tif ((d->qman_version & QMAN_REV_MASK) >= QMAN_REV_5000 &&\n+\t\t\t (d->cena_access_mode == qman_cena_fastest_access))\n+\t\t\treg = qbman_set_swp_cfg(dqrr_size, wn,\n+\t\t\t\t\t\t1, 3, 2, 0, 1, 1, 1, 1, 1, 1);\n+\t\telse\n+\t\t\treg = qbman_set_swp_cfg(dqrr_size, wn,\n+\t\t\t\t\t\test, 3, 2, 2, 1, 1, 1, 1, 1, 1);\n+\t}\n+\n+\tif ((d->qman_version & QMAN_REV_MASK) >= QMAN_REV_5000\n+\t\t\t&& (d->cena_access_mode == qman_cena_fastest_access))\n+\t\treg |= 1 << SWP_CFG_CPBS_SHIFT | /* memory-backed mode */\n+\t\t 1 << SWP_CFG_VPM_SHIFT | /* VDQCR read triggered mode */\n+\t\t 1 << SWP_CFG_CPM_SHIFT; /* CR read triggered mode */\n+\n+\tqbman_cinh_write(s, QBMAN_CINH_SWP_CFG, reg);\n+\treg = qbman_cinh_read(s, QBMAN_CINH_SWP_CFG);\n+\tif (!reg) {\n+\t\tpr_err(\"The portal %d is not enabled!\\n\", s->idx);\n+\t\treturn -1;\n+\t}\n+\n+\tif ((d->qman_version & QMAN_REV_MASK) >= QMAN_REV_5000\n+\t\t\t&& (d->cena_access_mode == qman_cena_fastest_access)) {\n+\t\tqbman_cinh_write(s, QBMAN_CINH_SWP_EQCR_PI, QMAN_RT_MODE);\n+\t\tqbman_cinh_write(s, QBMAN_CINH_SWP_RCR_PI, QMAN_RT_MODE);\n+\t}\n+\n+\treturn 0;\n+}\n+\n static inline void qbman_swp_sys_finish(struct qbman_swp_sys *s)\n {\n \tfree(s->cena);\n", "prefixes": [ "v2", "07/29" ] }