From patchwork Tue Mar 23 08:44:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 89667 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8D377A0562; Tue, 23 Mar 2021 09:44:49 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 70E3D4014D; Tue, 23 Mar 2021 09:44:49 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id D909740143 for ; Tue, 23 Mar 2021 09:44:47 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12N8ZMc5021413 for ; Tue, 23 Mar 2021 01:44:47 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=C52OmCg6grAxxs5Vi0RzzS9ISycOvqDHZSiuRXAGlyg=; b=TTdNOviUCi8j99BiHNAyTFgCiFu9Hh0S3EgaeQKyttyb56YbAa5p2odaSpq4+bxeTOiB grg/upr+0CzKpRS0enJO8cUxfGhFx5/RpWYgjyXyqzna/LzAz/Ej35dL8Be9mESYLaOo 2znkxe7lG+kp1kESA98Reo70wWmoZRplVKYLNi1d+T2x5MLovdNTN42Cm0shplRcttuS ulQqLTlLMZeVxqP5GH/sgjnN877v+rV31RfXkNdwrjkUohjhradH1DtGvhLYzNkZCjC/ Qrnb2+COawqbzyUGX/fDQ3MPR0YPoluDLNVt57P+EMOpa9uNLdgtVMm+2eTo+A6mnIeo lA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 37dedrgdn9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 23 Mar 2021 01:44:47 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 23 Mar 2021 01:44:45 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 23 Mar 2021 01:44:45 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id 4CBCA3F703F; Tue, 23 Mar 2021 01:44:44 -0700 (PDT) From: To: , Pavan Nikhilesh CC: Date: Tue, 23 Mar 2021 14:14:35 +0530 Message-ID: <20210323084439.3898-1-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210321084915.2649-1-pbhagavatula@marvell.com> References: <20210321084915.2649-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-23_02:2021-03-22, 2021-03-23 signatures=0 Subject: [dpdk-dev] [PATCH v3 1/4] event/octeontx2: simplify timer bucket estimation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Simplify timer bucket estimation we need not align buckets to power of 2 instead use reciprocal division to compute mod. Signed-off-by: Pavan Nikhilesh --- drivers/event/octeontx2/otx2_tim_evdev.c | 78 ++++----------------- drivers/event/octeontx2/otx2_tim_evdev.h | 84 ++++++++--------------- drivers/event/octeontx2/otx2_tim_worker.c | 4 +- drivers/event/octeontx2/otx2_tim_worker.h | 40 +++++------ 4 files changed, 64 insertions(+), 142 deletions(-) diff --git a/drivers/event/octeontx2/otx2_tim_evdev.c b/drivers/event/octeontx2/otx2_tim_evdev.c index 4c24cc8a6..d1e967eb7 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.c +++ b/drivers/event/octeontx2/otx2_tim_evdev.c @@ -34,27 +34,25 @@ tim_set_fp_ops(struct otx2_tim_ring *tim_ring) { uint8_t prod_flag = !tim_ring->prod_type_sp; - /* [MOD/AND] [DFB/FB] [SP][MP]*/ - const rte_event_timer_arm_burst_t arm_burst[2][2][2][2] = { -#define FP(_name, _f4, _f3, _f2, _f1, flags) \ - [_f4][_f3][_f2][_f1] = otx2_tim_arm_burst_ ## _name, -TIM_ARM_FASTPATH_MODES + /* [DFB/FB] [SP][MP]*/ + const rte_event_timer_arm_burst_t arm_burst[2][2][2] = { +#define FP(_name, _f3, _f2, _f1, flags) \ + [_f3][_f2][_f1] = otx2_tim_arm_burst_##_name, + TIM_ARM_FASTPATH_MODES #undef FP }; - const rte_event_timer_arm_tmo_tick_burst_t arm_tmo_burst[2][2][2] = { -#define FP(_name, _f3, _f2, _f1, flags) \ - [_f3][_f2][_f1] = otx2_tim_arm_tmo_tick_burst_ ## _name, -TIM_ARM_TMO_FASTPATH_MODES + const rte_event_timer_arm_tmo_tick_burst_t arm_tmo_burst[2][2] = { +#define FP(_name, _f2, _f1, flags) \ + [_f2][_f1] = otx2_tim_arm_tmo_tick_burst_##_name, + TIM_ARM_TMO_FASTPATH_MODES #undef FP }; otx2_tim_ops.arm_burst = - arm_burst[tim_ring->enable_stats][tim_ring->optimized] - [tim_ring->ena_dfb][prod_flag]; + arm_burst[tim_ring->enable_stats][tim_ring->ena_dfb][prod_flag]; otx2_tim_ops.arm_tmo_tick_burst = - arm_tmo_burst[tim_ring->enable_stats][tim_ring->optimized] - [tim_ring->ena_dfb]; + arm_tmo_burst[tim_ring->enable_stats][tim_ring->ena_dfb]; otx2_tim_ops.cancel_burst = otx2_tim_timer_cancel_burst; } @@ -70,51 +68,6 @@ otx2_tim_ring_info_get(const struct rte_event_timer_adapter *adptr, sizeof(struct rte_event_timer_adapter_conf)); } -static void -tim_optimze_bkt_param(struct otx2_tim_ring *tim_ring) -{ - uint64_t tck_nsec; - uint32_t hbkts; - uint32_t lbkts; - - hbkts = rte_align32pow2(tim_ring->nb_bkts); - tck_nsec = RTE_ALIGN_MUL_CEIL(tim_ring->max_tout / (hbkts - 1), 10); - - if ((tck_nsec < TICK2NSEC(OTX2_TIM_MIN_TMO_TKS, - tim_ring->tenns_clk_freq) || - hbkts > OTX2_TIM_MAX_BUCKETS)) - hbkts = 0; - - lbkts = rte_align32prevpow2(tim_ring->nb_bkts); - tck_nsec = RTE_ALIGN_MUL_CEIL((tim_ring->max_tout / (lbkts - 1)), 10); - - if ((tck_nsec < TICK2NSEC(OTX2_TIM_MIN_TMO_TKS, - tim_ring->tenns_clk_freq) || - lbkts > OTX2_TIM_MAX_BUCKETS)) - lbkts = 0; - - if (!hbkts && !lbkts) - return; - - if (!hbkts) { - tim_ring->nb_bkts = lbkts; - goto end; - } else if (!lbkts) { - tim_ring->nb_bkts = hbkts; - goto end; - } - - tim_ring->nb_bkts = (hbkts - tim_ring->nb_bkts) < - (tim_ring->nb_bkts - lbkts) ? hbkts : lbkts; -end: - tim_ring->optimized = true; - tim_ring->tck_nsec = RTE_ALIGN_MUL_CEIL((tim_ring->max_tout / - (tim_ring->nb_bkts - 1)), 10); - otx2_tim_dbg("Optimized configured values"); - otx2_tim_dbg("Nb_bkts : %" PRIu32 "", tim_ring->nb_bkts); - otx2_tim_dbg("Tck_nsec : %" PRIu64 "", tim_ring->tck_nsec); -} - static int tim_chnk_pool_create(struct otx2_tim_ring *tim_ring, struct rte_event_timer_adapter_conf *rcfg) @@ -319,14 +272,6 @@ otx2_tim_ring_create(struct rte_event_timer_adapter *adptr) tim_ring->chunk_sz); tim_ring->nb_chunk_slots = OTX2_TIM_NB_CHUNK_SLOTS(tim_ring->chunk_sz); - /* Try to optimize the bucket parameters. */ - if ((rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES)) { - if (rte_is_power_of_2(tim_ring->nb_bkts)) - tim_ring->optimized = true; - else - tim_optimze_bkt_param(tim_ring); - } - if (tim_ring->disable_npa) tim_ring->nb_chunks = tim_ring->nb_chunks * tim_ring->nb_bkts; else @@ -459,6 +404,7 @@ otx2_tim_ring_start(const struct rte_event_timer_adapter *adptr) tim_ring->tck_int = NSEC2TICK(tim_ring->tck_nsec, rte_get_timer_hz()); tim_ring->tot_int = tim_ring->tck_int * tim_ring->nb_bkts; tim_ring->fast_div = rte_reciprocal_value_u64(tim_ring->tck_int); + tim_ring->fast_bkt = rte_reciprocal_value_u64(tim_ring->nb_bkts); otx2_tim_calibrate_start_tsc(tim_ring); diff --git a/drivers/event/octeontx2/otx2_tim_evdev.h b/drivers/event/octeontx2/otx2_tim_evdev.h index 44e3c7b51..bf89b85b0 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.h +++ b/drivers/event/octeontx2/otx2_tim_evdev.h @@ -76,8 +76,6 @@ #define OTX2_TIM_SP 0x1 #define OTX2_TIM_MP 0x2 -#define OTX2_TIM_BKT_AND 0x4 -#define OTX2_TIM_BKT_MOD 0x8 #define OTX2_TIM_ENA_FB 0x10 #define OTX2_TIM_ENA_DFB 0x20 #define OTX2_TIM_ENA_STATS 0x40 @@ -149,11 +147,11 @@ struct otx2_tim_ring { struct otx2_tim_bkt *bkt; struct rte_mempool *chunk_pool; struct rte_reciprocal_u64 fast_div; + struct rte_reciprocal_u64 fast_bkt; uint64_t arm_cnt; uint8_t prod_type_sp; uint8_t enable_stats; uint8_t disable_npa; - uint8_t optimized; uint8_t ena_dfb; uint16_t ring_id; uint32_t aura; @@ -178,60 +176,38 @@ tim_priv_get(void) return mz->addr; } -#define TIM_ARM_FASTPATH_MODES \ -FP(mod_sp, 0, 0, 0, 0, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ -FP(mod_mp, 0, 0, 0, 1, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ -FP(mod_fb_sp, 0, 0, 1, 0, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ -FP(mod_fb_mp, 0, 0, 1, 1, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_FB | OTX2_TIM_MP) \ -FP(and_sp, 0, 1, 0, 0, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ -FP(and_mp, 0, 1, 0, 1, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ -FP(and_fb_sp, 0, 1, 1, 0, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ -FP(and_fb_mp, 0, 1, 1, 1, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_FB | OTX2_TIM_MP) \ -FP(stats_mod_sp, 1, 0, 0, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ -FP(stats_mod_mp, 1, 0, 0, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ -FP(stats_mod_fb_sp, 1, 0, 1, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ -FP(stats_mod_fb_mp, 1, 0, 1, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_FB | OTX2_TIM_MP) \ -FP(stats_and_sp, 1, 1, 0, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ -FP(stats_and_mp, 1, 1, 0, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ -FP(stats_and_fb_sp, 1, 1, 1, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ -FP(stats_and_fb_mp, 1, 1, 1, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_FB | OTX2_TIM_MP) - -#define TIM_ARM_TMO_FASTPATH_MODES \ -FP(mod, 0, 0, 0, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_DFB) \ -FP(mod_fb, 0, 0, 1, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_FB) \ -FP(and, 0, 1, 0, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_DFB) \ -FP(and_fb, 0, 1, 1, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_FB) \ -FP(stats_mod, 1, 0, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_DFB) \ -FP(stats_mod_fb, 1, 0, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_FB) \ -FP(stats_and, 1, 1, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_DFB) \ -FP(stats_and_fb, 1, 1, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_FB) - -#define FP(_name, _f4, _f3, _f2, _f1, flags) \ -uint16_t \ -otx2_tim_arm_burst_ ## _name(const struct rte_event_timer_adapter *adptr, \ - struct rte_event_timer **tim, \ - const uint16_t nb_timers); +#define TIM_ARM_FASTPATH_MODES \ + FP(sp, 0, 0, 0, OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ + FP(mp, 0, 0, 1, OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ + FP(fb_sp, 0, 1, 0, OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ + FP(fb_mp, 0, 1, 1, OTX2_TIM_ENA_FB | OTX2_TIM_MP) \ + FP(stats_mod_sp, 1, 0, 0, \ + OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ + FP(stats_mod_mp, 1, 0, 1, \ + OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ + FP(stats_mod_fb_sp, 1, 1, 0, \ + OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ + FP(stats_mod_fb_mp, 1, 1, 1, \ + OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_FB | OTX2_TIM_MP) + +#define TIM_ARM_TMO_FASTPATH_MODES \ + FP(dfb, 0, 0, OTX2_TIM_ENA_DFB) \ + FP(fb, 0, 1, OTX2_TIM_ENA_FB) \ + FP(stats_dfb, 1, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_DFB) \ + FP(stats_fb, 1, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_FB) + +#define FP(_name, _f3, _f2, _f1, flags) \ + uint16_t otx2_tim_arm_burst_##_name( \ + const struct rte_event_timer_adapter *adptr, \ + struct rte_event_timer **tim, const uint16_t nb_timers); TIM_ARM_FASTPATH_MODES #undef FP -#define FP(_name, _f3, _f2, _f1, flags) \ -uint16_t \ -otx2_tim_arm_tmo_tick_burst_ ## _name( \ - const struct rte_event_timer_adapter *adptr, \ - struct rte_event_timer **tim, \ - const uint64_t timeout_tick, const uint16_t nb_timers); +#define FP(_name, _f2, _f1, flags) \ + uint16_t otx2_tim_arm_tmo_tick_burst_##_name( \ + const struct rte_event_timer_adapter *adptr, \ + struct rte_event_timer **tim, const uint64_t timeout_tick, \ + const uint16_t nb_timers); TIM_ARM_TMO_FASTPATH_MODES #undef FP diff --git a/drivers/event/octeontx2/otx2_tim_worker.c b/drivers/event/octeontx2/otx2_tim_worker.c index 4b5cfdc72..eb901844d 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.c +++ b/drivers/event/octeontx2/otx2_tim_worker.c @@ -136,7 +136,7 @@ tim_timer_arm_tmo_brst(const struct rte_event_timer_adapter *adptr, return set_timers; } -#define FP(_name, _f4, _f3, _f2, _f1, _flags) \ +#define FP(_name, _f3, _f2, _f1, _flags) \ uint16_t __rte_noinline \ otx2_tim_arm_burst_ ## _name(const struct rte_event_timer_adapter *adptr, \ struct rte_event_timer **tim, \ @@ -147,7 +147,7 @@ otx2_tim_arm_burst_ ## _name(const struct rte_event_timer_adapter *adptr, \ TIM_ARM_FASTPATH_MODES #undef FP -#define FP(_name, _f3, _f2, _f1, _flags) \ +#define FP(_name, _f2, _f1, _flags) \ uint16_t __rte_noinline \ otx2_tim_arm_tmo_tick_burst_ ## _name( \ const struct rte_event_timer_adapter *adptr, \ diff --git a/drivers/event/octeontx2/otx2_tim_worker.h b/drivers/event/octeontx2/otx2_tim_worker.h index af2f864d7..f03912b81 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.h +++ b/drivers/event/octeontx2/otx2_tim_worker.h @@ -115,27 +115,27 @@ tim_bkt_clr_nent(struct otx2_tim_bkt *bktp) return __atomic_and_fetch(&bktp->w1, v, __ATOMIC_ACQ_REL); } +static inline uint64_t +tim_bkt_fast_mod(uint64_t n, uint64_t d, struct rte_reciprocal_u64 R) +{ + return (n - (d * rte_reciprocal_divide_u64(n, &R))); +} + static __rte_always_inline void -tim_get_target_bucket(struct otx2_tim_ring * const tim_ring, +tim_get_target_bucket(struct otx2_tim_ring *const tim_ring, const uint32_t rel_bkt, struct otx2_tim_bkt **bkt, - struct otx2_tim_bkt **mirr_bkt, const uint8_t flag) + struct otx2_tim_bkt **mirr_bkt) { const uint64_t bkt_cyc = rte_rdtsc() - tim_ring->ring_start_cyc; - uint32_t bucket = rte_reciprocal_divide_u64(bkt_cyc, - &tim_ring->fast_div) + rel_bkt; - uint32_t mirr_bucket = 0; - - if (flag & OTX2_TIM_BKT_MOD) { - bucket = bucket % tim_ring->nb_bkts; - mirr_bucket = (bucket + (tim_ring->nb_bkts >> 1)) % - tim_ring->nb_bkts; - } - if (flag & OTX2_TIM_BKT_AND) { - bucket = bucket & (tim_ring->nb_bkts - 1); - mirr_bucket = (bucket + (tim_ring->nb_bkts >> 1)) & - (tim_ring->nb_bkts - 1); - } - + uint64_t bucket = + rte_reciprocal_divide_u64(bkt_cyc, &tim_ring->fast_div) + + rel_bkt; + uint64_t mirr_bucket = 0; + + bucket = + tim_bkt_fast_mod(bucket, tim_ring->nb_bkts, tim_ring->fast_bkt); + mirr_bucket = tim_bkt_fast_mod(bucket + (tim_ring->nb_bkts >> 1), + tim_ring->nb_bkts, tim_ring->fast_bkt); *bkt = &tim_ring->bkt[bucket]; *mirr_bkt = &tim_ring->bkt[mirr_bucket]; } @@ -236,7 +236,7 @@ tim_add_entry_sp(struct otx2_tim_ring * const tim_ring, int16_t rem; __retry: - tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt, flags); + tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt); /* Get Bucket sema*/ lock_sema = tim_bkt_fetch_sema_lock(bkt); @@ -322,7 +322,7 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, int16_t rem; __retry: - tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt, flags); + tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt); /* Get Bucket sema*/ lock_sema = tim_bkt_fetch_sema_lock(bkt); @@ -454,7 +454,7 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring, uint8_t lock_cnt; __retry: - tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt, flags); + tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt); /* Only one thread beyond this. */ lock_sema = tim_bkt_inc_lock(bkt); From patchwork Tue Mar 23 08:44:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 89668 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1B927A0562; Tue, 23 Mar 2021 09:44:57 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0F1A3140D04; Tue, 23 Mar 2021 09:44:52 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 66ACE40687 for ; Tue, 23 Mar 2021 09:44:51 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12N8ZFWS031024 for ; Tue, 23 Mar 2021 01:44:50 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=mYIrgGfbNE0lvwVgDGhy9VI6QI97ss6KjFS6XnlsYic=; b=g80uuyB9wf5I85t8iSP1b+1zPoX9gi+a5cLICKcc6jaL8b+qQ6ntojUTAdnCTFn61vw+ 9pPK7VYF2KolIWVFYEwVzoCEoCg+lCjqtECN6LlaUoWIlvPCETQv41Zskb+mGuJBRcND p32K2xgGH+RAUx0OYf4j1DYRmyHlTl/j2hOwMnsI2FkNEyVtbGRuIpBMgP/2uhmjKpYU zgW9F/kzE78eKt1CZr9LmZNd4Gwf0iweBoG9ZB2tGXi0s+oDLfTi7a/uF8rDCdCnio0s JlRg5HK6FwhdyDXYpM+wrLJIFFpFr8c7awOFARb6BUlK6Tfd5cT2uZjiiGn7u2bnh/9F +A== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37dgjp05um-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 23 Mar 2021 01:44:50 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 23 Mar 2021 01:44:48 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 23 Mar 2021 01:44:48 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id 686933F703F; Tue, 23 Mar 2021 01:44:47 -0700 (PDT) From: To: , Pavan Nikhilesh CC: Date: Tue, 23 Mar 2021 14:14:36 +0530 Message-ID: <20210323084439.3898-2-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210323084439.3898-1-pbhagavatula@marvell.com> References: <20210321084915.2649-1-pbhagavatula@marvell.com> <20210323084439.3898-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-23_02:2021-03-22, 2021-03-23 signatures=0 Subject: [dpdk-dev] [PATCH v3 2/4] event/octeontx2: optimize timer arm routine X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Use relaxed load exclusive when polling for other threads or hardware to complete. Signed-off-by: Pavan Nikhilesh --- v3 Changes: - Fix incorrect asm register usage detected by clang. drivers/event/octeontx2/otx2_tim_worker.c | 1 + drivers/event/octeontx2/otx2_tim_worker.h | 163 ++++++++++++---------- 2 files changed, 90 insertions(+), 74 deletions(-) -- 2.17.1 diff --git a/drivers/event/octeontx2/otx2_tim_worker.c b/drivers/event/octeontx2/otx2_tim_worker.c index eb901844d..6a3511ec0 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.c +++ b/drivers/event/octeontx2/otx2_tim_worker.c @@ -170,6 +170,7 @@ otx2_tim_timer_cancel_burst(const struct rte_event_timer_adapter *adptr, int ret; RTE_SET_USED(adptr); + rte_atomic_thread_fence(__ATOMIC_ACQUIRE); for (index = 0; index < nb_timers; index++) { if (tim[index]->state == RTE_EVENT_TIMER_CANCELED) { rte_errno = EALREADY; diff --git a/drivers/event/octeontx2/otx2_tim_worker.h b/drivers/event/octeontx2/otx2_tim_worker.h index f03912b81..5ece8fd05 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.h +++ b/drivers/event/octeontx2/otx2_tim_worker.h @@ -84,7 +84,13 @@ tim_bkt_inc_lock(struct otx2_tim_bkt *bktp) static inline void tim_bkt_dec_lock(struct otx2_tim_bkt *bktp) { - __atomic_add_fetch(&bktp->lock, 0xff, __ATOMIC_RELEASE); + __atomic_fetch_sub(&bktp->lock, 1, __ATOMIC_RELEASE); +} + +static inline void +tim_bkt_dec_lock_relaxed(struct otx2_tim_bkt *bktp) +{ + __atomic_fetch_sub(&bktp->lock, 1, __ATOMIC_RELAXED); } static inline uint32_t @@ -246,22 +252,20 @@ tim_add_entry_sp(struct otx2_tim_ring * const tim_ring, if (tim_bkt_get_nent(lock_sema) != 0) { uint64_t hbt_state; #ifdef RTE_ARCH_ARM64 - asm volatile( - " ldaxr %[hbt], [%[w1]] \n" - " tbz %[hbt], 33, dne%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldaxr %[hbt], [%[w1]] \n" - " tbnz %[hbt], 33, rty%= \n" - "dne%=: \n" - : [hbt] "=&r" (hbt_state) - : [w1] "r" ((&bkt->w1)) - : "memory" - ); + asm volatile(" ldxr %[hbt], [%[w1]] \n" + " tbz %[hbt], 33, dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxr %[hbt], [%[w1]] \n" + " tbnz %[hbt], 33, rty%= \n" + "dne%=: \n" + : [hbt] "=&r"(hbt_state) + : [w1] "r"((&bkt->w1)) + : "memory"); #else do { hbt_state = __atomic_load_n(&bkt->w1, - __ATOMIC_ACQUIRE); + __ATOMIC_RELAXED); } while (hbt_state & BIT_ULL(33)); #endif @@ -282,10 +286,10 @@ tim_add_entry_sp(struct otx2_tim_ring * const tim_ring, if (unlikely(chunk == NULL)) { bkt->chunk_remainder = 0; - tim_bkt_dec_lock(bkt); tim->impl_opaque[0] = 0; tim->impl_opaque[1] = 0; tim->state = RTE_EVENT_TIMER_ERROR; + tim_bkt_dec_lock(bkt); return -ENOMEM; } mirr_bkt->current_chunk = (uintptr_t)chunk; @@ -298,12 +302,11 @@ tim_add_entry_sp(struct otx2_tim_ring * const tim_ring, /* Copy work entry. */ *chunk = *pent; - tim_bkt_inc_nent(bkt); - tim_bkt_dec_lock(bkt); - tim->impl_opaque[0] = (uintptr_t)chunk; tim->impl_opaque[1] = (uintptr_t)bkt; - tim->state = RTE_EVENT_TIMER_ARMED; + __atomic_store_n(&tim->state, RTE_EVENT_TIMER_ARMED, __ATOMIC_RELEASE); + tim_bkt_inc_nent(bkt); + tim_bkt_dec_lock_relaxed(bkt); return 0; } @@ -331,22 +334,20 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, if (tim_bkt_get_nent(lock_sema) != 0) { uint64_t hbt_state; #ifdef RTE_ARCH_ARM64 - asm volatile( - " ldaxr %[hbt], [%[w1]] \n" - " tbz %[hbt], 33, dne%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldaxr %[hbt], [%[w1]] \n" - " tbnz %[hbt], 33, rty%= \n" - "dne%=: \n" - : [hbt] "=&r" (hbt_state) - : [w1] "r" ((&bkt->w1)) - : "memory" - ); + asm volatile(" ldxr %[hbt], [%[w1]] \n" + " tbz %[hbt], 33, dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxr %[hbt], [%[w1]] \n" + " tbnz %[hbt], 33, rty%= \n" + "dne%=: \n" + : [hbt] "=&r"(hbt_state) + : [w1] "r"((&bkt->w1)) + : "memory"); #else do { hbt_state = __atomic_load_n(&bkt->w1, - __ATOMIC_ACQUIRE); + __ATOMIC_RELAXED); } while (hbt_state & BIT_ULL(33)); #endif @@ -359,26 +360,24 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, rem = tim_bkt_fetch_rem(lock_sema); if (rem < 0) { + tim_bkt_dec_lock(bkt); #ifdef RTE_ARCH_ARM64 - asm volatile( - " ldaxrh %w[rem], [%[crem]] \n" - " tbz %w[rem], 15, dne%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldaxrh %w[rem], [%[crem]] \n" - " tbnz %w[rem], 15, rty%= \n" - "dne%=: \n" - : [rem] "=&r" (rem) - : [crem] "r" (&bkt->chunk_remainder) - : "memory" - ); + uint64_t w1; + asm volatile(" ldxr %[w1], [%[crem]] \n" + " tbz %[w1], 63, dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxr %[w1], [%[crem]] \n" + " tbnz %[w1], 63, rty%= \n" + "dne%=: \n" + : [w1] "=&r"(w1) + : [crem] "r"(&bkt->w1) + : "memory"); #else - while (__atomic_load_n(&bkt->chunk_remainder, - __ATOMIC_ACQUIRE) < 0) + while (__atomic_load_n((int64_t *)&bkt->w1, __ATOMIC_RELAXED) < + 0) ; #endif - /* Goto diff bucket. */ - tim_bkt_dec_lock(bkt); goto __retry; } else if (!rem) { /* Only one thread can be here*/ @@ -388,18 +387,21 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, chunk = tim_insert_chunk(bkt, mirr_bkt, tim_ring); if (unlikely(chunk == NULL)) { - tim_bkt_set_rem(bkt, 0); - tim_bkt_dec_lock(bkt); tim->impl_opaque[0] = 0; tim->impl_opaque[1] = 0; tim->state = RTE_EVENT_TIMER_ERROR; + tim_bkt_set_rem(bkt, 0); + tim_bkt_dec_lock(bkt); return -ENOMEM; } *chunk = *pent; - while (tim_bkt_fetch_lock(lock_sema) != - (-tim_bkt_fetch_rem(lock_sema))) - lock_sema = __atomic_load_n(&bkt->w1, __ATOMIC_ACQUIRE); - + if (tim_bkt_fetch_lock(lock_sema)) { + do { + lock_sema = __atomic_load_n(&bkt->w1, + __ATOMIC_RELAXED); + } while (tim_bkt_fetch_lock(lock_sema) - 1); + rte_atomic_thread_fence(__ATOMIC_ACQUIRE); + } mirr_bkt->current_chunk = (uintptr_t)chunk; __atomic_store_n(&bkt->chunk_remainder, tim_ring->nb_chunk_slots - 1, __ATOMIC_RELEASE); @@ -409,12 +411,11 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, *chunk = *pent; } - /* Copy work entry. */ - tim_bkt_inc_nent(bkt); - tim_bkt_dec_lock(bkt); tim->impl_opaque[0] = (uintptr_t)chunk; tim->impl_opaque[1] = (uintptr_t)bkt; - tim->state = RTE_EVENT_TIMER_ARMED; + __atomic_store_n(&tim->state, RTE_EVENT_TIMER_ARMED, __ATOMIC_RELEASE); + tim_bkt_inc_nent(bkt); + tim_bkt_dec_lock_relaxed(bkt); return 0; } @@ -463,6 +464,23 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring, if (lock_cnt) { tim_bkt_dec_lock(bkt); +#ifdef RTE_ARCH_ARM64 + asm volatile(" ldxrb %w[lock_cnt], [%[lock]] \n" + " tst %w[lock_cnt], 255 \n" + " beq dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxrb %w[lock_cnt], [%[lock]] \n" + " tst %w[lock_cnt], 255 \n" + " bne rty%= \n" + "dne%=: \n" + : [lock_cnt] "=&r"(lock_cnt) + : [lock] "r"(&bkt->lock) + : "memory"); +#else + while (__atomic_load_n(&bkt->lock, __ATOMIC_RELAXED)) + ; +#endif goto __retry; } @@ -471,22 +489,20 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring, if (tim_bkt_get_nent(lock_sema) != 0) { uint64_t hbt_state; #ifdef RTE_ARCH_ARM64 - asm volatile( - " ldaxr %[hbt], [%[w1]] \n" - " tbz %[hbt], 33, dne%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldaxr %[hbt], [%[w1]] \n" - " tbnz %[hbt], 33, rty%= \n" - "dne%=: \n" - : [hbt] "=&r" (hbt_state) - : [w1] "r" ((&bkt->w1)) - : "memory" - ); + asm volatile(" ldxr %[hbt], [%[w1]] \n" + " tbz %[hbt], 33, dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxr %[hbt], [%[w1]] \n" + " tbnz %[hbt], 33, rty%= \n" + "dne%=: \n" + : [hbt] "=&r"(hbt_state) + : [w1] "r"((&bkt->w1)) + : "memory"); #else do { hbt_state = __atomic_load_n(&bkt->w1, - __ATOMIC_ACQUIRE); + __ATOMIC_RELAXED); } while (hbt_state & BIT_ULL(33)); #endif @@ -563,19 +579,18 @@ tim_rm_entry(struct rte_event_timer *tim) bkt = (struct otx2_tim_bkt *)(uintptr_t)tim->impl_opaque[1]; lock_sema = tim_bkt_inc_lock(bkt); if (tim_bkt_get_hbt(lock_sema) || !tim_bkt_get_nent(lock_sema)) { - tim_bkt_dec_lock(bkt); tim->impl_opaque[0] = 0; tim->impl_opaque[1] = 0; + tim_bkt_dec_lock(bkt); return -ENOENT; } entry->w0 = 0; entry->wqe = 0; - tim_bkt_dec_lock(bkt); - tim->state = RTE_EVENT_TIMER_CANCELED; tim->impl_opaque[0] = 0; tim->impl_opaque[1] = 0; + tim_bkt_dec_lock(bkt); return 0; } From patchwork Tue Mar 23 08:44:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 89669 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F36C3A0562; Tue, 23 Mar 2021 09:45:03 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4518F4069E; Tue, 23 Mar 2021 09:44:55 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 8BFB540689 for ; Tue, 23 Mar 2021 09:44:53 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12N8ZMNt021414 for ; Tue, 23 Mar 2021 01:44:52 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=bLcf9pyxyhy+bR71LxYy/vkf0fnbdT0zm7EI28hpLvs=; b=HXN4NuQg2yXrdIvMs+e+DWx0QlQCdIGpaKdgST9dunX8PXrjZkaqJVZ9qZQrGmB0L7ha IiqcowTaB75d4BiruY0+Zckp+vFQJAp3tqtB/LfKnn9MsxO+yMu+O4eV6dKH4P43bpkq bcnRj+QBpJWkDkEjRpmvz8S2kwGUHtJoC1q0G0Oii14ofcQZQtRbR7diryO0alEilOE+ /hNonYnT2pnLeZQSZkmUUy0adGLuyGWcDtmi7k8orIjm9j9hwY9oTXlJD28sCbuWuSMy OB32iXy7Na0TNJg8WKUDF93OMoWZmOcShbw04pHVAFtQC5C6fBjM+//dKU9zQw19jLE2 vg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 37dedrgdnk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 23 Mar 2021 01:44:52 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 23 Mar 2021 01:44:51 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 23 Mar 2021 01:44:51 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id 3D1953F703F; Tue, 23 Mar 2021 01:44:49 -0700 (PDT) From: To: , Pavan Nikhilesh CC: Date: Tue, 23 Mar 2021 14:14:37 +0530 Message-ID: <20210323084439.3898-3-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210323084439.3898-1-pbhagavatula@marvell.com> References: <20210321084915.2649-1-pbhagavatula@marvell.com> <20210323084439.3898-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-23_02:2021-03-22, 2021-03-23 signatures=0 Subject: [dpdk-dev] [PATCH v3 3/4] event/octeontx2: reduce chunk pool memory usage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Reduce amount of memory used by chunk pool when the mempool used is OCTEONTX2 NPA. Previously, the number of chunks configured when NPA is used is equal to the number of timers requested plus the number of buckets and if the max timeout is long enough w.r.t. resolution requested there will a large number of buckets which would cause high memory usage. Reduce the number of chunks when NPA is used to the number of timers requested as buckets that are processed chunk lists are automatically freed. Signed-off-by: Pavan Nikhilesh --- v2 Changes: - Describe memory allocation changes. drivers/event/octeontx2/otx2_tim_evdev.c | 19 ++++++++++--------- drivers/event/octeontx2/otx2_tim_evdev.h | 2 +- 2 files changed, 11 insertions(+), 10 deletions(-) -- 2.17.1 diff --git a/drivers/event/octeontx2/otx2_tim_evdev.c b/drivers/event/octeontx2/otx2_tim_evdev.c index d1e967eb7..4fb002ddb 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.c +++ b/drivers/event/octeontx2/otx2_tim_evdev.c @@ -91,6 +91,8 @@ tim_chnk_pool_create(struct otx2_tim_ring *tim_ring, if (cache_sz > RTE_MEMPOOL_CACHE_MAX_SIZE) cache_sz = RTE_MEMPOOL_CACHE_MAX_SIZE; + cache_sz = cache_sz != 0 ? cache_sz : 2; + tim_ring->nb_chunks += (cache_sz * rte_lcore_count()); if (!tim_ring->disable_npa) { tim_ring->chunk_pool = rte_mempool_create_empty(pool_name, tim_ring->nb_chunks, tim_ring->chunk_sz, @@ -268,16 +270,15 @@ otx2_tim_ring_create(struct rte_event_timer_adapter *adptr) } } - tim_ring->nb_chunks = tim_ring->nb_timers / OTX2_TIM_NB_CHUNK_SLOTS( - tim_ring->chunk_sz); - tim_ring->nb_chunk_slots = OTX2_TIM_NB_CHUNK_SLOTS(tim_ring->chunk_sz); - - if (tim_ring->disable_npa) + if (tim_ring->disable_npa) { + tim_ring->nb_chunks = + tim_ring->nb_timers / + OTX2_TIM_NB_CHUNK_SLOTS(tim_ring->chunk_sz); tim_ring->nb_chunks = tim_ring->nb_chunks * tim_ring->nb_bkts; - else - tim_ring->nb_chunks = tim_ring->nb_chunks + tim_ring->nb_bkts; - - /* Create buckets. */ + } else { + tim_ring->nb_chunks = tim_ring->nb_timers; + } + tim_ring->nb_chunk_slots = OTX2_TIM_NB_CHUNK_SLOTS(tim_ring->chunk_sz); tim_ring->bkt = rte_zmalloc("otx2_tim_bucket", (tim_ring->nb_bkts) * sizeof(struct otx2_tim_bkt), RTE_CACHE_LINE_SIZE); diff --git a/drivers/event/octeontx2/otx2_tim_evdev.h b/drivers/event/octeontx2/otx2_tim_evdev.h index bf89b85b0..0667d4576 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.h +++ b/drivers/event/octeontx2/otx2_tim_evdev.h @@ -70,7 +70,7 @@ #define OTX2_TIM_MAX_BURST (RTE_CACHE_LINE_SIZE / \ OTX2_TIM_CHUNK_ALIGNMENT) #define OTX2_TIM_NB_CHUNK_SLOTS(sz) (((sz) / OTX2_TIM_CHUNK_ALIGNMENT) - 1) -#define OTX2_TIM_MIN_CHUNK_SLOTS (0x1) +#define OTX2_TIM_MIN_CHUNK_SLOTS (0x8) #define OTX2_TIM_MAX_CHUNK_SLOTS (0x1FFE) #define OTX2_TIM_MIN_TMO_TKS (256) From patchwork Tue Mar 23 08:44:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 89670 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E8A61A0562; Tue, 23 Mar 2021 09:45:10 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 75545140D1E; Tue, 23 Mar 2021 09:44:58 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 73E56140D1C for ; Tue, 23 Mar 2021 09:44:56 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12N8ZLjk021392 for ; Tue, 23 Mar 2021 01:44:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=/UaWKgL5JKsdVzuFK5WgfUsmVEDTI05gNnSsWKSxvSI=; b=Lt466RdWHVKTqjFarLXhwjUvcjGVb1EWOWGoM8HsDuFvTuJ0E9aiuJZjRpPvxPkD7yCj MNFGlSAxp/2O494sdOLOXsgMKe7FDqMKqEZ8kTVHfBlgr9p43LDty4cTcdeR1deyIlnV t8Zf6oFp6xsYxxvMCtn8ypSyIww+b6IC+djge+LUXO5Us1IgNwMTjhzg3kAC94gE+m/+ I3LsA/SShafRqJbD6NMI31zGnoFfXb2boSnRH29euFPuBN7dig8pzI7p+uYEBrIEQlGO sSka7hF3yc1Ku97EDKk7cDHzYmrXsBrRebmgZp4pcfnE96urNrv0L+RLHfOA9a91pJ7s ag== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 37dedrgdnn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 23 Mar 2021 01:44:55 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 23 Mar 2021 01:44:54 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 23 Mar 2021 01:44:54 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id 1E7913F703F; Tue, 23 Mar 2021 01:44:52 -0700 (PDT) From: To: , Pavan Nikhilesh CC: Date: Tue, 23 Mar 2021 14:14:38 +0530 Message-ID: <20210323084439.3898-4-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210323084439.3898-1-pbhagavatula@marvell.com> References: <20210321084915.2649-1-pbhagavatula@marvell.com> <20210323084439.3898-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-23_02:2021-03-22, 2021-03-23 signatures=0 Subject: [dpdk-dev] [PATCH v3 4/4] event/octeontx2: timer always use virtual counter X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Use virtual counter for estimating current bucket as PMU cannot be reliably used to estimate time. Signed-off-by: Pavan Nikhilesh --- v2 Changes: - Use __rte_arm64_cntvct and __rte_arm64_cntfrq instead of reimplementing asm. drivers/event/octeontx2/otx2_tim_evdev.c | 19 +++-------------- drivers/event/octeontx2/otx2_tim_evdev.h | 26 +++++++++++++++++++++++ drivers/event/octeontx2/otx2_tim_worker.c | 4 ++-- drivers/event/octeontx2/otx2_tim_worker.h | 2 +- 4 files changed, 32 insertions(+), 19 deletions(-) -- 2.17.1 diff --git a/drivers/event/octeontx2/otx2_tim_evdev.c b/drivers/event/octeontx2/otx2_tim_evdev.c index 4fb002ddb..926c2dce6 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.c +++ b/drivers/event/octeontx2/otx2_tim_evdev.c @@ -354,7 +354,7 @@ otx2_tim_calibrate_start_tsc(struct otx2_tim_ring *tim_ring) for (icount = 0; icount < OTX2_TIM_CALIB_ITER; icount++) { real_bkt = otx2_read64(tim_ring->base + TIM_LF_RING_REL) >> 44; - bkt_cyc = rte_rdtsc(); + bkt_cyc = tim_cntvct(); bucket = (bkt_cyc - tim_ring->ring_start_cyc) / tim_ring->tck_int; bucket = bucket % (tim_ring->nb_bkts); @@ -389,20 +389,8 @@ otx2_tim_ring_start(const struct rte_event_timer_adapter *adptr) tim_err_desc(rc); goto fail; } -#ifdef RTE_ARM_EAL_RDTSC_USE_PMU - uint64_t tenns_stmp, tenns_diff; - uint64_t pmu_stmp; - - pmu_stmp = rte_rdtsc(); - asm volatile("mrs %0, cntvct_el0" : "=r" (tenns_stmp)); - - tenns_diff = tenns_stmp - rsp->timestarted; - pmu_stmp = pmu_stmp - (NSEC2TICK(tenns_diff * 10, rte_get_timer_hz())); - tim_ring->ring_start_cyc = pmu_stmp; -#else tim_ring->ring_start_cyc = rsp->timestarted; -#endif - tim_ring->tck_int = NSEC2TICK(tim_ring->tck_nsec, rte_get_timer_hz()); + tim_ring->tck_int = NSEC2TICK(tim_ring->tck_nsec, tim_cntfrq()); tim_ring->tot_int = tim_ring->tck_int * tim_ring->nb_bkts; tim_ring->fast_div = rte_reciprocal_value_u64(tim_ring->tck_int); tim_ring->fast_bkt = rte_reciprocal_value_u64(tim_ring->nb_bkts); @@ -470,8 +458,7 @@ otx2_tim_stats_get(const struct rte_event_timer_adapter *adapter, struct rte_event_timer_adapter_stats *stats) { struct otx2_tim_ring *tim_ring = adapter->data->adapter_priv; - uint64_t bkt_cyc = rte_rdtsc() - tim_ring->ring_start_cyc; - + uint64_t bkt_cyc = tim_cntvct() - tim_ring->ring_start_cyc; stats->evtim_exp_count = __atomic_load_n(&tim_ring->arm_cnt, __ATOMIC_RELAXED); diff --git a/drivers/event/octeontx2/otx2_tim_evdev.h b/drivers/event/octeontx2/otx2_tim_evdev.h index 0667d4576..410880e14 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.h +++ b/drivers/event/octeontx2/otx2_tim_evdev.h @@ -176,6 +176,32 @@ tim_priv_get(void) return mz->addr; } +#ifdef RTE_ARCH_ARM64 +static inline uint64_t +tim_cntvct(void) +{ + return __rte_arm64_cntvct(); +} + +static inline uint64_t +tim_cntfrq(void) +{ + return __rte_arm64_cntfrq(); +} +#else +static inline uint64_t +tim_cntvct(void) +{ + return 0; +} + +static inline uint64_t +tim_cntfrq(void) +{ + return 0; +} +#endif + #define TIM_ARM_FASTPATH_MODES \ FP(sp, 0, 0, 0, OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ FP(mp, 0, 0, 1, OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ diff --git a/drivers/event/octeontx2/otx2_tim_worker.c b/drivers/event/octeontx2/otx2_tim_worker.c index 6a3511ec0..9ee07958f 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.c +++ b/drivers/event/octeontx2/otx2_tim_worker.c @@ -41,12 +41,12 @@ tim_format_event(const struct rte_event_timer * const tim, static inline void tim_sync_start_cyc(struct otx2_tim_ring *tim_ring) { - uint64_t cur_cyc = rte_rdtsc(); + uint64_t cur_cyc = tim_cntvct(); uint32_t real_bkt; if (cur_cyc - tim_ring->last_updt_cyc > tim_ring->tot_int) { real_bkt = otx2_read64(tim_ring->base + TIM_LF_RING_REL) >> 44; - cur_cyc = rte_rdtsc(); + cur_cyc = tim_cntvct(); tim_ring->ring_start_cyc = cur_cyc - (real_bkt * tim_ring->tck_int); diff --git a/drivers/event/octeontx2/otx2_tim_worker.h b/drivers/event/octeontx2/otx2_tim_worker.h index 5ece8fd05..efe88a869 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.h +++ b/drivers/event/octeontx2/otx2_tim_worker.h @@ -132,7 +132,7 @@ tim_get_target_bucket(struct otx2_tim_ring *const tim_ring, const uint32_t rel_bkt, struct otx2_tim_bkt **bkt, struct otx2_tim_bkt **mirr_bkt) { - const uint64_t bkt_cyc = rte_rdtsc() - tim_ring->ring_start_cyc; + const uint64_t bkt_cyc = tim_cntvct() - tim_ring->ring_start_cyc; uint64_t bucket = rte_reciprocal_divide_u64(bkt_cyc, &tim_ring->fast_div) + rel_bkt;