From patchwork Sat Jun 1 18:53:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 54027 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EA57D1B9C4; Sat, 1 Jun 2019 20:56:16 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 6C599378B for ; Sat, 1 Jun 2019 20:55:41 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x51ItbNR029526 for ; Sat, 1 Jun 2019 11:55:40 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=c6xFSeS1uBRJejKcoNleL/Oi5NZRHK6w4WGcZYaz05Q=; b=RvTOLXKoQ+Ud/6o4WoAz3AkN8DPkOMJul7r1N03ZZju7WZ5ioWsleQGDCOZpGdn/fp5M 79EIbg2IzD6XdLDB/6hjy0NK2xfOAsH6XYYmyrsRXXoXnH+cmjvq9hWAXcm50lPwFw6A WiXfIiscdkgQkfRGcbXEt+UVPRIuuH1yLo8vbcvIFFk9UN38VZUteJmNJ3f46pjOGEZc lE8mKHMeLLPAWjC0G2tTNcaBoKbv3okfXT8dT3k/Jf2nLTaMJ9A5Oft+7tTDO7jz6KKp 7HEo/FugxIkXh/gOVTXVM3+nwiCqe+91Jd4fHGWJVozqE9LdcuzocY2L19ujrXM1s4t7 wA== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2supqksfw6-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 01 Jun 2019 11:55:40 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sat, 1 Jun 2019 11:55:38 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sat, 1 Jun 2019 11:55:38 -0700 Received: from BG-LT7430.marvell.com (unknown [10.28.17.28]) by maili.marvell.com (Postfix) with ESMTP id E04BC3F7052; Sat, 1 Jun 2019 11:55:30 -0700 (PDT) From: To: , Pavan Nikhilesh CC: Date: Sun, 2 Jun 2019 00:23:24 +0530 Message-ID: <20190601185355.370-15-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190601185355.370-1-pbhagavatula@marvell.com> References: <20190601185355.370-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-01_13:, , signatures=0 Subject: [dpdk-dev] [PATCH 14/44] event/octeontx2: add SSO HW device operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Add SSO HW device operations used for enqueue/dequeue. Signed-off-by: Pavan Nikhilesh Signed-off-by: Jerin Jacob --- drivers/event/octeontx2/Makefile | 1 + drivers/event/octeontx2/meson.build | 3 +- drivers/event/octeontx2/otx2_evdev.h | 7 + drivers/event/octeontx2/otx2_worker.c | 5 + drivers/event/octeontx2/otx2_worker.h | 186 ++++++++++++++++++++++++++ 5 files changed, 201 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx2/otx2_worker.c create mode 100644 drivers/event/octeontx2/otx2_worker.h diff --git a/drivers/event/octeontx2/Makefile b/drivers/event/octeontx2/Makefile index 4f09c1fc8..a3de5ca23 100644 --- a/drivers/event/octeontx2/Makefile +++ b/drivers/event/octeontx2/Makefile @@ -30,6 +30,7 @@ LIBABIVER := 1 # all source are stored in SRCS-y # +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EVENTDEV) += otx2_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EVENTDEV) += otx2_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EVENTDEV) += otx2_evdev_irq.c diff --git a/drivers/event/octeontx2/meson.build b/drivers/event/octeontx2/meson.build index 5aa8113bd..1d2080b6d 100644 --- a/drivers/event/octeontx2/meson.build +++ b/drivers/event/octeontx2/meson.build @@ -2,7 +2,8 @@ # Copyright(C) 2019 Marvell International Ltd. # -sources = files('otx2_evdev.c', +sources = files('otx2_worker.c', + 'otx2_evdev.c', 'otx2_evdev_irq.c', ) diff --git a/drivers/event/octeontx2/otx2_evdev.h b/drivers/event/octeontx2/otx2_evdev.h index e1d2dcc69..6bb1c664c 100644 --- a/drivers/event/octeontx2/otx2_evdev.h +++ b/drivers/event/octeontx2/otx2_evdev.h @@ -82,6 +82,13 @@ enum otx2_sso_lf_type { SSO_LF_GWS }; +enum { + SSO_SYNC_ORDERED, + SSO_SYNC_ATOMIC, + SSO_SYNC_UNTAGGED, + SSO_SYNC_EMPTY +}; + struct otx2_sso_evdev { OTX2_DEV; /* Base class */ uint8_t max_event_queues; diff --git a/drivers/event/octeontx2/otx2_worker.c b/drivers/event/octeontx2/otx2_worker.c new file mode 100644 index 000000000..83f535d05 --- /dev/null +++ b/drivers/event/octeontx2/otx2_worker.c @@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#include "otx2_worker.h" diff --git a/drivers/event/octeontx2/otx2_worker.h b/drivers/event/octeontx2/otx2_worker.h new file mode 100644 index 000000000..e8705e53c --- /dev/null +++ b/drivers/event/octeontx2/otx2_worker.h @@ -0,0 +1,186 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#ifndef __OTX2_WORKER_H__ +#define __OTX2_WORKER_H__ + +#include +#include + +#include +#include "otx2_evdev.h" + +/* SSO Operations */ + +static __rte_always_inline uint16_t +otx2_ssogws_get_work(struct otx2_ssogws *ws, struct rte_event *ev) +{ + uint64_t get_work0; + uint64_t get_work1; + + otx2_write64(BIT_ULL(16) | /* wait for work. */ + 1, /* Use Mask set 0. */ + ws->getwrk_op); + +#ifdef RTE_ARCH_ARM64 + asm volatile( + " ldr %[tag], [%[tag_loc]] \n" + " ldr %[wqp], [%[wqp_loc]] \n" + " tbz %[tag], 63, done%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldr %[tag], [%[tag_loc]] \n" + " ldr %[wqp], [%[wqp_loc]] \n" + " tbnz %[tag], 63, rty%= \n" + "done%=: prfm pldl1strm, [%[wqp]] \n" + " dmb ld \n" + : [tag] "=&r" (get_work0), [wqp] "=&r" (get_work1) + : [tag_loc] "r" (ws->tag_op), + [wqp_loc] "r" (ws->wqp_op) + ); +#else + get_work0 = otx2_read64(ws->tag_op); + while ((BIT_ULL(63)) & get_work0) + get_work0 = otx2_read64(ws->tag_op); + + get_work1 = otx2_read64(ws->wqp_op); + rte_prefetch_non_temporal((const void *)get_work1); +#endif + + ws->cur_tt = (get_work0 >> 32) & 0x3; + ws->cur_grp = (get_work0 >> 36) & 0x3FF; + + get_work0 = (get_work0 & (0x3ull << 32)) << 6 | + (get_work0 & (0x3FFull << 36)) << 4 | + (get_work0 & 0xffffffff); + + ev->event = get_work0; + ev->u64 = get_work1; + + return !!get_work1; +} + +/* Used in cleaning up workslot. */ +static __rte_always_inline uint16_t +otx2_ssogws_get_work_empty(struct otx2_ssogws *ws, struct rte_event *ev) +{ + uint64_t get_work0; + uint64_t get_work1; + +#ifdef RTE_ARCH_ARM64 + asm volatile( + " ldr %[tag], [%[tag_loc]] \n" + " ldr %[wqp], [%[wqp_loc]] \n" + " tbz %[tag], 63, done%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldr %[tag], [%[tag_loc]] \n" + " ldr %[wqp], [%[wqp_loc]] \n" + " tbnz %[tag], 63, rty%= \n" + "done%=: prfm pldl1strm, [%[wqp]] \n" + " dmb ld \n" + : [tag] "=&r" (get_work0), [wqp] "=&r" (get_work1) + : [tag_loc] "r" (ws->tag_op), + [wqp_loc] "r" (ws->wqp_op) + ); +#else + get_work0 = otx2_read64(ws->tag_op); + while ((BIT_ULL(63)) & get_work0) + get_work0 = otx2_read64(ws->tag_op); + + get_work1 = otx2_read64(ws->wqp_op); + rte_prefetch_non_temporal((const void *)get_work1); +#endif + + ws->cur_tt = (get_work0 >> 32) & 0x3; + ws->cur_grp = (get_work0 >> 36) & 0x3FF; + + get_work0 = (get_work0 & (0x3ull << 32)) << 6 | + (get_work0 & (0x3FFull << 36)) << 4 | + (get_work0 & 0xffffffff); + + ev->event = get_work0; + ev->u64 = get_work1; + + return !!get_work1; +} + +static __rte_always_inline void +otx2_ssogws_add_work(struct otx2_ssogws *ws, const uint64_t event_ptr, + const uint32_t tag, const uint8_t new_tt, + const uint16_t grp) +{ + uint64_t add_work0; + + add_work0 = tag | ((uint64_t)(new_tt) << 32); + otx2_store_pair(add_work0, event_ptr, ws->grps_base[grp]); +} + +static __rte_always_inline void +otx2_ssogws_swtag_desched(struct otx2_ssogws *ws, uint32_t tag, uint8_t new_tt, + uint16_t grp) +{ + uint64_t val; + + val = tag | ((uint64_t)(new_tt & 0x3) << 32) | ((uint64_t)grp << 34); + otx2_write64(val, ws->swtag_desched_op); +} + +static __rte_always_inline void +otx2_ssogws_swtag_norm(struct otx2_ssogws *ws, uint32_t tag, uint8_t new_tt) +{ + uint64_t val; + + val = tag | ((uint64_t)(new_tt & 0x3) << 32); + otx2_write64(val, ws->swtag_norm_op); +} + +static __rte_always_inline void +otx2_ssogws_swtag_untag(struct otx2_ssogws *ws) +{ + otx2_write64(0, OTX2_SSOW_GET_BASE_ADDR(ws->getwrk_op) + + SSOW_LF_GWS_OP_SWTAG_UNTAG); + ws->cur_tt = SSO_SYNC_UNTAGGED; +} + +static __rte_always_inline void +otx2_ssogws_swtag_flush(struct otx2_ssogws *ws) +{ + otx2_write64(0, OTX2_SSOW_GET_BASE_ADDR(ws->getwrk_op) + + SSOW_LF_GWS_OP_SWTAG_FLUSH); + ws->cur_tt = SSO_SYNC_EMPTY; +} + +static __rte_always_inline void +otx2_ssogws_desched(struct otx2_ssogws *ws) +{ + otx2_write64(0, OTX2_SSOW_GET_BASE_ADDR(ws->getwrk_op) + + SSOW_LF_GWS_OP_DESCHED); +} + +static __rte_always_inline void +otx2_ssogws_swtag_wait(struct otx2_ssogws *ws) +{ +#ifdef RTE_ARCH_ARM64 + uint64_t swtp; + + asm volatile ( + " ldr %[swtb], [%[swtp_loc]] \n" + " cbz %[swtb], done%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldr %[swtb], [%[swtp_loc]] \n" + " cbnz %[swtb], rty%= \n" + "done%=: \n" + : [swtb] "=&r" (swtp) + : [swtp_loc] "r" (ws->swtp_op) + ); +#else + /* Wait for the SWTAG/SWTAG_FULL operation */ + while (otx2_read64(ws->swtp_op)) + ; +#endif +} + +#endif