From patchwork Thu Feb 2 08:10:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 122885 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5D91341BAA; Thu, 2 Feb 2023 09:10:35 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EB1DC42B7E; Thu, 2 Feb 2023 09:10:34 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id DF5A8406A2 for ; Thu, 2 Feb 2023 09:10:32 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3127abg3007907 for ; Thu, 2 Feb 2023 00:10:32 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=W9+fn8ypbkuOKKorsGQvp76n2Cv5gBMq3/8TkPoJMag=; b=ayDFEf2KQFOfxFC7gNPz1j+oEKaOFCgjfjz5EKQIASKSLrsVmsipy0/+SKWo9IXeH6Jo 3gwA783z5twM/+5RlZapJ71TNdzFHj0nH38tRBD4L1oRHMrRngwQsfP+jG8K7rXnceEm Bu5G5+tlKLKUnJAdokDw0ZKXSpg4G112POZgycEPYDRSYSfzZC5m6RmdRf8KEVWduuBb 2jYfNfizWXxDPvbEkENV6m0YH0QZumVsNBssgANH9XV+iTuKPK4GMHrLZc1/vn4CvweR fgcEltm6sXsnjrAk8lTP+Hf8nOYfO/Zb5WTaa4SJ6Y003IeUSfliOHH5JSz8ii+ZohoL Vg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3nfjrj6x71-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 02 Feb 2023 00:10:32 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Thu, 2 Feb 2023 00:10:29 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Thu, 2 Feb 2023 00:10:29 -0800 Received: from MININT-80QBFE8.corp.innovium.com (unknown [10.28.164.122]) by maili.marvell.com (Postfix) with ESMTP id 2A7315B6931; Thu, 2 Feb 2023 00:10:27 -0800 (PST) From: To: , Pavan Nikhilesh , "Shijith Thotton" CC: Subject: [PATCH 1/2] event/cnxk: fix burst timer arm routine Date: Thu, 2 Feb 2023 13:40:24 +0530 Message-ID: <20230202081025.4176-1-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: KdJn4Vba707P-fnNIMiuir7qdmpYU1ta X-Proofpoint-GUID: KdJn4Vba707P-fnNIMiuir7qdmpYU1ta X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_15,2023-01-31_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Pavan Nikhilesh Fix timer burst arm routine writing improper updates to the bucket. Fixes: 5f644e1bd14c ("event/cnxk: add timer arm timeout burst") Signed-off-by: Pavan Nikhilesh --- drivers/event/cnxk/cnxk_tim_worker.h | 81 ++++++++++++++-------------- 1 file changed, 42 insertions(+), 39 deletions(-) diff --git a/drivers/event/cnxk/cnxk_tim_worker.h b/drivers/event/cnxk/cnxk_tim_worker.h index eda84c6f31..6be31f6f9d 100644 --- a/drivers/event/cnxk/cnxk_tim_worker.h +++ b/drivers/event/cnxk/cnxk_tim_worker.h @@ -270,7 +270,8 @@ cnxk_tim_add_entry_sp(struct cnxk_tim_ring *const tim_ring, } while (hbt_state & BIT_ULL(33)); #endif - if (!(hbt_state & BIT_ULL(34))) { + if (!(hbt_state & BIT_ULL(34)) || + !(hbt_state & GENMASK(31, 0))) { cnxk_tim_bkt_dec_lock(bkt); goto __retry; } @@ -352,7 +353,8 @@ cnxk_tim_add_entry_mp(struct cnxk_tim_ring *const tim_ring, } while (hbt_state & BIT_ULL(33)); #endif - if (!(hbt_state & BIT_ULL(34))) { + if (!(hbt_state & BIT_ULL(34)) || + !(hbt_state & GENMASK(31, 0))) { cnxk_tim_bkt_dec_lock(bkt); goto __retry; } @@ -449,10 +451,10 @@ cnxk_tim_add_entry_brst(struct cnxk_tim_ring *const tim_ring, struct cnxk_tim_ent *chunk = NULL; struct cnxk_tim_bkt *mirr_bkt; struct cnxk_tim_bkt *bkt; - uint16_t chunk_remainder; + int16_t chunk_remainder; uint16_t index = 0; uint64_t lock_sema; - int16_t rem, crem; + int16_t rem; uint8_t lock_cnt; __retry: @@ -460,31 +462,6 @@ cnxk_tim_add_entry_brst(struct cnxk_tim_ring *const tim_ring, /* Only one thread beyond this. */ lock_sema = cnxk_tim_bkt_inc_lock(bkt); - lock_cnt = (uint8_t)((lock_sema >> TIM_BUCKET_W1_S_LOCK) & - TIM_BUCKET_W1_M_LOCK); - - if (lock_cnt) { - cnxk_tim_bkt_dec_lock(bkt); -#ifdef RTE_ARCH_ARM64 - asm volatile(PLT_CPU_FEATURE_PREAMBLE - " ldxrb %w[lock_cnt], [%[lock]] \n" - " tst %w[lock_cnt], 255 \n" - " beq dne%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldxrb %w[lock_cnt], [%[lock]] \n" - " tst %w[lock_cnt], 255 \n" - " bne rty%= \n" - "dne%=: \n" - : [lock_cnt] "=&r"(lock_cnt) - : [lock] "r"(&bkt->lock) - : "memory"); -#else - while (__atomic_load_n(&bkt->lock, __ATOMIC_RELAXED)) - ; -#endif - goto __retry; - } /* Bucket related checks. */ if (unlikely(cnxk_tim_bkt_get_hbt(lock_sema))) { @@ -509,21 +486,46 @@ cnxk_tim_add_entry_brst(struct cnxk_tim_ring *const tim_ring, } while (hbt_state & BIT_ULL(33)); #endif - if (!(hbt_state & BIT_ULL(34))) { + if (!(hbt_state & BIT_ULL(34)) || + !(hbt_state & GENMASK(31, 0))) { cnxk_tim_bkt_dec_lock(bkt); goto __retry; } } } + lock_cnt = (uint8_t)((lock_sema >> TIM_BUCKET_W1_S_LOCK) & + TIM_BUCKET_W1_M_LOCK); + if (lock_cnt) { + cnxk_tim_bkt_dec_lock(bkt); +#ifdef RTE_ARCH_ARM64 + asm volatile(PLT_CPU_FEATURE_PREAMBLE + " ldxrb %w[lock_cnt], [%[lock]] \n" + " tst %w[lock_cnt], 255 \n" + " beq dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxrb %w[lock_cnt], [%[lock]] \n" + " tst %w[lock_cnt], 255 \n" + " bne rty%= \n" + "dne%=: \n" + : [lock_cnt] "=&r"(lock_cnt) + : [lock] "r"(&bkt->lock) + : "memory"); +#else + while (__atomic_load_n(&bkt->lock, __ATOMIC_RELAXED)) + ; +#endif + goto __retry; + } + chunk_remainder = cnxk_tim_bkt_fetch_rem(lock_sema); rem = chunk_remainder - nb_timers; if (rem < 0) { - crem = tim_ring->nb_chunk_slots - chunk_remainder; - if (chunk_remainder && crem) { + if (chunk_remainder > 0) { chunk = ((struct cnxk_tim_ent *) mirr_bkt->current_chunk) + - crem; + tim_ring->nb_chunk_slots - chunk_remainder; index = cnxk_tim_cpy_wrk(index, chunk_remainder, chunk, tim, ents, bkt); @@ -537,18 +539,19 @@ cnxk_tim_add_entry_brst(struct cnxk_tim_ring *const tim_ring, chunk = cnxk_tim_insert_chunk(bkt, mirr_bkt, tim_ring); if (unlikely(chunk == NULL)) { - cnxk_tim_bkt_dec_lock(bkt); + cnxk_tim_bkt_dec_lock_relaxed(bkt); rte_errno = ENOMEM; tim[index]->state = RTE_EVENT_TIMER_ERROR; - return crem; + return index; } *(uint64_t *)(chunk + tim_ring->nb_chunk_slots) = 0; mirr_bkt->current_chunk = (uintptr_t)chunk; - cnxk_tim_cpy_wrk(index, nb_timers, chunk, tim, ents, bkt); + index = cnxk_tim_cpy_wrk(index, nb_timers, chunk, tim, ents, + bkt) - + index; - rem = nb_timers - chunk_remainder; - cnxk_tim_bkt_set_rem(bkt, tim_ring->nb_chunk_slots - rem); - cnxk_tim_bkt_add_nent(bkt, rem); + cnxk_tim_bkt_set_rem(bkt, tim_ring->nb_chunk_slots - index); + cnxk_tim_bkt_add_nent(bkt, index); } else { chunk = (struct cnxk_tim_ent *)mirr_bkt->current_chunk; chunk += (tim_ring->nb_chunk_slots - chunk_remainder); From patchwork Thu Feb 2 08:10:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 122886 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1BAF341BAA; Thu, 2 Feb 2023 09:10:42 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3F86C42D41; Thu, 2 Feb 2023 09:10:36 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9906542D2C for ; Thu, 2 Feb 2023 09:10:35 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3127abg5007907 for ; Thu, 2 Feb 2023 00:10:35 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=k7qmk+dhbVxHjNBzULR2N0KIT65nqWUMVkmefVeMpjA=; b=lPrZobLswVbWgo1F9DXsslo5oHmT63OqVsFpSVKW+KU8jMU3Taof11+jzqCGbLzO1DGi RKiy9j0rYe7uKLB6JsJgoDTDRPOfLTiOtI5IMZbV9L4gAnn9OKqQ7623Ie7jBJ+2tvDT 8Bnw8XUNx3j1b2cxA2VnnUS41EfjM3ABhW30bHEgC0GAxtazt/0tvfE/IONAFv6HMSo4 z/1lHinMZ6KpFbLHdlLIY3lsqkV4eJTihjlBrdHYZsZNYX1HrnQTQDRoqA+tdyqSjukf ZOafe9JLXHzzv1a7/bUX9U03u1dNil6bfPSVkQDCSO7cxXVHDZjx1RQpcksZHU0tkVji 6w== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3nfjrj6x7b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 02 Feb 2023 00:10:34 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Thu, 2 Feb 2023 00:10:32 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Thu, 2 Feb 2023 00:10:32 -0800 Received: from MININT-80QBFE8.corp.innovium.com (unknown [10.28.164.122]) by maili.marvell.com (Postfix) with ESMTP id 66FD35B6927; Thu, 2 Feb 2023 00:10:30 -0800 (PST) From: To: , Pavan Nikhilesh , "Shijith Thotton" CC: Subject: [PATCH 2/2] event/cnxk: update timer arm burst parameters Date: Thu, 2 Feb 2023 13:40:25 +0530 Message-ID: <20230202081025.4176-2-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202081025.4176-1-pbhagavatula@marvell.com> References: <20230202081025.4176-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: cH7dfMLW861bTG3XIv7RCx-3MoHNWoCD X-Proofpoint-GUID: cH7dfMLW861bTG3XIv7RCx-3MoHNWoCD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_15,2023-01-31_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Pavan Nikhilesh Increase the timer arm burst size to 16 and chunk size for optimum performance. Use fixed size chunk pool cache to avoid high alloc cycles. Signed-off-by: Pavan Nikhilesh --- drivers/event/cnxk/cnxk_tim_evdev.c | 7 ++----- drivers/event/cnxk/cnxk_tim_evdev.h | 5 ++--- drivers/event/cnxk/cnxk_tim_worker.h | 12 +++++++++--- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/drivers/event/cnxk/cnxk_tim_evdev.c b/drivers/event/cnxk/cnxk_tim_evdev.c index 5dd79cbd47..6ff3ca72f7 100644 --- a/drivers/event/cnxk/cnxk_tim_evdev.c +++ b/drivers/event/cnxk/cnxk_tim_evdev.c @@ -14,12 +14,11 @@ static int cnxk_tim_chnk_pool_create(struct cnxk_tim_ring *tim_ring, struct rte_event_timer_adapter_conf *rcfg) { - unsigned int cache_sz = (tim_ring->nb_chunks / 1.5); unsigned int mp_flags = 0; + unsigned int cache_sz; char pool_name[25]; int rc; - cache_sz /= rte_lcore_count(); /* Create chunk pool. */ if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { mp_flags = RTE_MEMPOOL_F_SP_PUT | RTE_MEMPOOL_F_SC_GET; @@ -30,9 +29,7 @@ cnxk_tim_chnk_pool_create(struct cnxk_tim_ring *tim_ring, snprintf(pool_name, sizeof(pool_name), "cnxk_tim_chunk_pool%d", tim_ring->ring_id); - if (cache_sz > CNXK_TIM_MAX_POOL_CACHE_SZ) - cache_sz = CNXK_TIM_MAX_POOL_CACHE_SZ; - cache_sz = cache_sz != 0 ? cache_sz : 2; + cache_sz = CNXK_TIM_MAX_POOL_CACHE_SZ; tim_ring->nb_chunks += (cache_sz * rte_lcore_count()); if (!tim_ring->disable_npa) { tim_ring->chunk_pool = rte_mempool_create_empty( diff --git a/drivers/event/cnxk/cnxk_tim_evdev.h b/drivers/event/cnxk/cnxk_tim_evdev.h index 0c192346c7..8c69d15c80 100644 --- a/drivers/event/cnxk/cnxk_tim_evdev.h +++ b/drivers/event/cnxk/cnxk_tim_evdev.h @@ -24,10 +24,9 @@ #define CNXK_TIM_EVDEV_NAME cnxk_tim_eventdev #define CNXK_TIM_MAX_BUCKETS (0xFFFFF) -#define CNXK_TIM_RING_DEF_CHUNK_SZ (256) +#define CNXK_TIM_RING_DEF_CHUNK_SZ (1024) #define CNXK_TIM_CHUNK_ALIGNMENT (16) -#define CNXK_TIM_MAX_BURST \ - (RTE_CACHE_LINE_SIZE / CNXK_TIM_CHUNK_ALIGNMENT) +#define CNXK_TIM_MAX_BURST (16) #define CNXK_TIM_NB_CHUNK_SLOTS(sz) (((sz) / CNXK_TIM_CHUNK_ALIGNMENT) - 1) #define CNXK_TIM_MIN_CHUNK_SLOTS (0x1) #define CNXK_TIM_MAX_CHUNK_SLOTS (0x1FFE) diff --git a/drivers/event/cnxk/cnxk_tim_worker.h b/drivers/event/cnxk/cnxk_tim_worker.h index 6be31f6f9d..87ac91f387 100644 --- a/drivers/event/cnxk/cnxk_tim_worker.h +++ b/drivers/event/cnxk/cnxk_tim_worker.h @@ -106,11 +106,17 @@ cnxk_tim_bkt_inc_nent(struct cnxk_tim_bkt *bktp) } static inline void -cnxk_tim_bkt_add_nent(struct cnxk_tim_bkt *bktp, uint32_t v) +cnxk_tim_bkt_add_nent_relaxed(struct cnxk_tim_bkt *bktp, uint32_t v) { __atomic_add_fetch(&bktp->nb_entry, v, __ATOMIC_RELAXED); } +static inline void +cnxk_tim_bkt_add_nent(struct cnxk_tim_bkt *bktp, uint32_t v) +{ + __atomic_add_fetch(&bktp->nb_entry, v, __ATOMIC_RELEASE); +} + static inline uint64_t cnxk_tim_bkt_clr_nent(struct cnxk_tim_bkt *bktp) { @@ -530,7 +536,7 @@ cnxk_tim_add_entry_brst(struct cnxk_tim_ring *const tim_ring, index = cnxk_tim_cpy_wrk(index, chunk_remainder, chunk, tim, ents, bkt); cnxk_tim_bkt_sub_rem(bkt, chunk_remainder); - cnxk_tim_bkt_add_nent(bkt, chunk_remainder); + cnxk_tim_bkt_add_nent_relaxed(bkt, chunk_remainder); } if (flags & CNXK_TIM_ENA_FB) @@ -561,7 +567,7 @@ cnxk_tim_add_entry_brst(struct cnxk_tim_ring *const tim_ring, cnxk_tim_bkt_add_nent(bkt, nb_timers); } - cnxk_tim_bkt_dec_lock(bkt); + cnxk_tim_bkt_dec_lock_relaxed(bkt); return nb_timers; }