From patchwork Thu Nov 19 08:46:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shijith Thotton X-Patchwork-Id: 84353 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 74C19A04DD; Thu, 19 Nov 2020 09:47:02 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 594A1F90; Thu, 19 Nov 2020 09:47:01 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 9CEBFDED; Thu, 19 Nov 2020 09:46:58 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0AJ8ipdt014749; Thu, 19 Nov 2020 00:46:57 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=tL7rT34y7KU6pvTkiETzoEh2B3z4F0d0yjh2VbdEX7E=; b=H/2sfg9LSBjA1LbzzuJ44AIuaXkRliQO8HyzK9tyBjdtCo/8tRS1JBdbhcwQpqDtzLa+ jGDIxJdgNVvN3ttuBlwxkUm9JULthgaeW7KYeUPSGX8O1c8h2ju2YQtswklpmHGVj/Ry uPg6zoScor/besW6umIOunGQyJnya5/0218GEQj09XswXMguodoK5D9I0CZHHvc8pyIS WDUn/ISWFej3MAX1JsiSnw2Kg/zdwr/K4HzS2XzjmDsivv0jb57zWtMudmIY4nr9gRVu mYzZ/8Lnh0/A3sOjc8IYaY0AHm1j0dFLROX58rBn4ILAl3NeGj9xwG6CFVCBlzQVqJPw 3w== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 34vd2sh3r6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 19 Nov 2020 00:46:57 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 19 Nov 2020 00:46:56 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 19 Nov 2020 00:46:56 -0800 Received: from localhost.localdomain (unknown [10.28.34.29]) by maili.marvell.com (Postfix) with ESMTP id 8548F3F7040; Thu, 19 Nov 2020 00:46:54 -0800 (PST) From: Shijith Thotton To: CC: , Shijith Thotton , , Pavan Nikhilesh Date: Thu, 19 Nov 2020 14:16:11 +0530 Message-ID: <20201119084611.1415215-1-sthotton@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201118085411.1367090-1-sthotton@marvell.com> References: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-11-19_05:2020-11-17, 2020-11-19 signatures=0 Subject: [dpdk-dev] [PATCH v2] event/octeontx2: unlink queues during port release X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Unlinking queues from port should be done during port release. Doing it during device re-configuration could result in segfault as ports array is re-allocated based on new number of ports. Fixes: f7ac8b66b23c ("event/octeontx2: support linking queues to ports") Cc: stable@dpdk.org Signed-off-by: Shijith Thotton Signed-off-by: Pavan Nikhilesh --- v2: * Took care of cookie during single workslot free and re-allocation. drivers/event/octeontx2/otx2_evdev.c | 100 ++++++++++++--------- drivers/event/octeontx2/otx2_evdev.h | 12 +++ drivers/event/octeontx2/otx2_evdev_adptr.c | 24 +++-- 3 files changed, 89 insertions(+), 47 deletions(-) diff --git a/drivers/event/octeontx2/otx2_evdev.c b/drivers/event/octeontx2/otx2_evdev.c index b31c26e95..de9e394a4 100644 --- a/drivers/event/octeontx2/otx2_evdev.c +++ b/drivers/event/octeontx2/otx2_evdev.c @@ -689,7 +689,36 @@ sso_lf_cfg(struct otx2_sso_evdev *dev, struct otx2_mbox *mbox, static void otx2_sso_port_release(void *port) { - rte_free(port); + struct otx2_ssogws_cookie *gws_cookie = ssogws_get_cookie(port); + struct otx2_sso_evdev *dev; + int i; + + if (!gws_cookie->configured) + goto free; + + dev = sso_pmd_priv(gws_cookie->event_dev); + if (dev->dual_ws) { + struct otx2_ssogws_dual *ws = port; + + for (i = 0; i < dev->nb_event_queues; i++) { + sso_port_link_modify((struct otx2_ssogws *) + &ws->ws_state[0], i, false); + sso_port_link_modify((struct otx2_ssogws *) + &ws->ws_state[1], i, false); + } + memset(ws, 0, sizeof(*ws)); + } else { + struct otx2_ssogws *ws = port; + + for (i = 0; i < dev->nb_event_queues; i++) + sso_port_link_modify(ws, i, false); + memset(ws, 0, sizeof(*ws)); + } + + memset(gws_cookie, 0, sizeof(*gws_cookie)); + +free: + rte_free(gws_cookie); } static void @@ -699,33 +728,6 @@ otx2_sso_queue_release(struct rte_eventdev *event_dev, uint8_t queue_id) RTE_SET_USED(queue_id); } -static void -sso_clr_links(const struct rte_eventdev *event_dev) -{ - struct otx2_sso_evdev *dev = sso_pmd_priv(event_dev); - int i, j; - - for (i = 0; i < dev->nb_event_ports; i++) { - if (dev->dual_ws) { - struct otx2_ssogws_dual *ws; - - ws = event_dev->data->ports[i]; - for (j = 0; j < dev->nb_event_queues; j++) { - sso_port_link_modify((struct otx2_ssogws *) - &ws->ws_state[0], j, false); - sso_port_link_modify((struct otx2_ssogws *) - &ws->ws_state[1], j, false); - } - } else { - struct otx2_ssogws *ws; - - ws = event_dev->data->ports[i]; - for (j = 0; j < dev->nb_event_queues; j++) - sso_port_link_modify(ws, j, false); - } - } -} - static void sso_restore_links(const struct rte_eventdev *event_dev) { @@ -803,6 +805,7 @@ sso_configure_dual_ports(const struct rte_eventdev *event_dev) } for (i = 0; i < dev->nb_event_ports; i++) { + struct otx2_ssogws_cookie *gws_cookie; struct otx2_ssogws_dual *ws; uintptr_t base; @@ -811,14 +814,20 @@ sso_configure_dual_ports(const struct rte_eventdev *event_dev) } else { /* Allocate event port memory */ ws = rte_zmalloc_socket("otx2_sso_ws", - sizeof(struct otx2_ssogws_dual), + sizeof(struct otx2_ssogws_dual) + + RTE_CACHE_LINE_SIZE, RTE_CACHE_LINE_SIZE, event_dev->data->socket_id); - } - if (ws == NULL) { - otx2_err("Failed to alloc memory for port=%d", i); - rc = -ENOMEM; - break; + if (ws == NULL) { + otx2_err("Failed to alloc memory for port=%d", + i); + rc = -ENOMEM; + break; + } + + /* First cache line is reserved for cookie */ + ws = (struct otx2_ssogws_dual *) + ((uint8_t *)ws + RTE_CACHE_LINE_SIZE); } ws->port = i; @@ -830,6 +839,10 @@ sso_configure_dual_ports(const struct rte_eventdev *event_dev) sso_set_port_ops((struct otx2_ssogws *)&ws->ws_state[1], base); vws++; + gws_cookie = ssogws_get_cookie(ws); + gws_cookie->event_dev = event_dev; + gws_cookie->configured = 1; + event_dev->data->ports[i] = ws; } @@ -866,19 +879,21 @@ sso_configure_ports(const struct rte_eventdev *event_dev) } for (i = 0; i < nb_lf; i++) { + struct otx2_ssogws_cookie *gws_cookie; struct otx2_ssogws *ws; uintptr_t base; /* Free memory prior to re-allocation if needed */ if (event_dev->data->ports[i] != NULL) { ws = event_dev->data->ports[i]; - rte_free(ws); + rte_free(ssogws_get_cookie(ws)); ws = NULL; } /* Allocate event port memory */ ws = rte_zmalloc_socket("otx2_sso_ws", - sizeof(struct otx2_ssogws), + sizeof(struct otx2_ssogws) + + RTE_CACHE_LINE_SIZE, RTE_CACHE_LINE_SIZE, event_dev->data->socket_id); if (ws == NULL) { @@ -887,10 +902,18 @@ sso_configure_ports(const struct rte_eventdev *event_dev) break; } + /* First cache line is reserved for cookie */ + ws = (struct otx2_ssogws *) + ((uint8_t *)ws + RTE_CACHE_LINE_SIZE); + ws->port = i; base = dev->bar2 + (RVU_BLOCK_ADDR_SSOW << 20 | i << 12); sso_set_port_ops(ws, base); + gws_cookie = ssogws_get_cookie(ws); + gws_cookie->event_dev = event_dev; + gws_cookie->configured = 1; + event_dev->data->ports[i] = ws; } @@ -1099,11 +1122,8 @@ otx2_sso_configure(const struct rte_eventdev *event_dev) return -EINVAL; } - if (dev->configured) { + if (dev->configured) sso_unregister_irqs(event_dev); - /* Clear any prior port-queue mapping. */ - sso_clr_links(event_dev); - } if (dev->nb_event_queues) { /* Finit any previous queues. */ diff --git a/drivers/event/octeontx2/otx2_evdev.h b/drivers/event/octeontx2/otx2_evdev.h index 547e29d4a..116f953f5 100644 --- a/drivers/event/octeontx2/otx2_evdev.h +++ b/drivers/event/octeontx2/otx2_evdev.h @@ -217,6 +217,18 @@ sso_pmd_priv(const struct rte_eventdev *event_dev) return event_dev->data->dev_private; } +struct otx2_ssogws_cookie { + const struct rte_eventdev *event_dev; + bool configured; +}; + +static inline struct otx2_ssogws_cookie * +ssogws_get_cookie(void *ws) +{ + return (struct otx2_ssogws_cookie *) + ((uint8_t *)ws - RTE_CACHE_LINE_SIZE); +} + static const union mbuf_initializer mbuf_init = { .fields = { .data_off = RTE_PKTMBUF_HEADROOM, diff --git a/drivers/event/octeontx2/otx2_evdev_adptr.c b/drivers/event/octeontx2/otx2_evdev_adptr.c index 0a5d7924a..d69f269df 100644 --- a/drivers/event/octeontx2/otx2_evdev_adptr.c +++ b/drivers/event/octeontx2/otx2_evdev_adptr.c @@ -453,9 +453,10 @@ sso_add_tx_queue_data(const struct rte_eventdev *event_dev, struct otx2_ssogws_dual *dws; old_dws = event_dev->data->ports[i]; - dws = rte_realloc_socket(old_dws, + dws = rte_realloc_socket(ssogws_get_cookie(old_dws), sizeof(struct otx2_ssogws_dual) - + (sizeof(uint64_t) * + + RTE_CACHE_LINE_SIZE + + (sizeof(uint64_t) * (dev->max_port_id + 1) * RTE_MAX_QUEUES_PER_PORT), RTE_CACHE_LINE_SIZE, @@ -463,6 +464,10 @@ sso_add_tx_queue_data(const struct rte_eventdev *event_dev, if (dws == NULL) return -ENOMEM; + /* First cache line is reserved for cookie */ + dws = (struct otx2_ssogws_dual *) + ((uint8_t *)dws + RTE_CACHE_LINE_SIZE); + ((uint64_t (*)[RTE_MAX_QUEUES_PER_PORT] )&dws->tx_adptr_data)[eth_port_id][tx_queue_id] = (uint64_t)txq; @@ -472,16 +477,21 @@ sso_add_tx_queue_data(const struct rte_eventdev *event_dev, struct otx2_ssogws *ws; old_ws = event_dev->data->ports[i]; - ws = rte_realloc_socket(old_ws, - sizeof(struct otx2_ssogws_dual) - + (sizeof(uint64_t) * - (dev->max_port_id + 1) * - RTE_MAX_QUEUES_PER_PORT), + ws = rte_realloc_socket(ssogws_get_cookie(old_ws), + sizeof(struct otx2_ssogws) + + RTE_CACHE_LINE_SIZE + + (sizeof(uint64_t) * + (dev->max_port_id + 1) * + RTE_MAX_QUEUES_PER_PORT), RTE_CACHE_LINE_SIZE, event_dev->data->socket_id); if (ws == NULL) return -ENOMEM; + /* First cache line is reserved for cookie */ + ws = (struct otx2_ssogws *) + ((uint8_t *)ws + RTE_CACHE_LINE_SIZE); + ((uint64_t (*)[RTE_MAX_QUEUES_PER_PORT] )&ws->tx_adptr_data)[eth_port_id][tx_queue_id] = (uint64_t)txq;