From patchwork Sun Mar 21 08:49:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 89598 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 28BD2A034F; Sun, 21 Mar 2021 09:49:57 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0949E40042; Sun, 21 Mar 2021 09:49:56 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9107940041 for ; Sun, 21 Mar 2021 09:49:54 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12L8kkft013684 for ; Sun, 21 Mar 2021 01:49:53 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=C52OmCg6grAxxs5Vi0RzzS9ISycOvqDHZSiuRXAGlyg=; b=QXQ4Y8xBrklQBXmhjqWmB4RKE45WVEz/9DGVde5FUPeyBVhcV1j9wyfGudGc9zEyHDQ2 rbi5VNv/1GGUpwdw6FRz6iJ+3YGB3Gf0/D++vfSDmVnvyfTGk96pbvL3WZakS02OwKbt gzt0CkHvtJJDCtOsSgcl5TrxSmbVEuMWW3GroiEYm2YorunFMp4A+86fP81N1dYcOdc3 LYNwNf1bVq/PeUnxowje9UTcA2T1aJjEr0EfmjBzSb1aWEppKmKl66X3pGjNX1kf9gjM aAnajjVs3dth711nsFjwHAxlSqUJU0WbdCDOTSJ8GMcC74adqpnfxfQ1doHgFzMDojzJ JQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37dgjnsrkm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sun, 21 Mar 2021 01:49:53 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 21 Mar 2021 01:49:51 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sun, 21 Mar 2021 01:49:51 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id 696543F7040; Sun, 21 Mar 2021 01:49:50 -0700 (PDT) From: To: , Pavan Nikhilesh CC: Date: Sun, 21 Mar 2021 14:19:11 +0530 Message-ID: <20210321084915.2649-1-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210225122315.6350-1-pbhagavatula@marvell.com> References: <20210225122315.6350-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-20_13:2021-03-19, 2021-03-20 signatures=0 Subject: [dpdk-dev] [PATCH v2 1/4] event/octeontx2: simplify timer bucket estimation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Simplify timer bucket estimation we need not align buckets to power of 2 instead use reciprocal division to compute mod. Signed-off-by: Pavan Nikhilesh --- drivers/event/octeontx2/otx2_tim_evdev.c | 78 ++++----------------- drivers/event/octeontx2/otx2_tim_evdev.h | 84 ++++++++--------------- drivers/event/octeontx2/otx2_tim_worker.c | 4 +- drivers/event/octeontx2/otx2_tim_worker.h | 40 +++++------ 4 files changed, 64 insertions(+), 142 deletions(-) diff --git a/drivers/event/octeontx2/otx2_tim_evdev.c b/drivers/event/octeontx2/otx2_tim_evdev.c index 4c24cc8a6..d1e967eb7 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.c +++ b/drivers/event/octeontx2/otx2_tim_evdev.c @@ -34,27 +34,25 @@ tim_set_fp_ops(struct otx2_tim_ring *tim_ring) { uint8_t prod_flag = !tim_ring->prod_type_sp; - /* [MOD/AND] [DFB/FB] [SP][MP]*/ - const rte_event_timer_arm_burst_t arm_burst[2][2][2][2] = { -#define FP(_name, _f4, _f3, _f2, _f1, flags) \ - [_f4][_f3][_f2][_f1] = otx2_tim_arm_burst_ ## _name, -TIM_ARM_FASTPATH_MODES + /* [DFB/FB] [SP][MP]*/ + const rte_event_timer_arm_burst_t arm_burst[2][2][2] = { +#define FP(_name, _f3, _f2, _f1, flags) \ + [_f3][_f2][_f1] = otx2_tim_arm_burst_##_name, + TIM_ARM_FASTPATH_MODES #undef FP }; - const rte_event_timer_arm_tmo_tick_burst_t arm_tmo_burst[2][2][2] = { -#define FP(_name, _f3, _f2, _f1, flags) \ - [_f3][_f2][_f1] = otx2_tim_arm_tmo_tick_burst_ ## _name, -TIM_ARM_TMO_FASTPATH_MODES + const rte_event_timer_arm_tmo_tick_burst_t arm_tmo_burst[2][2] = { +#define FP(_name, _f2, _f1, flags) \ + [_f2][_f1] = otx2_tim_arm_tmo_tick_burst_##_name, + TIM_ARM_TMO_FASTPATH_MODES #undef FP }; otx2_tim_ops.arm_burst = - arm_burst[tim_ring->enable_stats][tim_ring->optimized] - [tim_ring->ena_dfb][prod_flag]; + arm_burst[tim_ring->enable_stats][tim_ring->ena_dfb][prod_flag]; otx2_tim_ops.arm_tmo_tick_burst = - arm_tmo_burst[tim_ring->enable_stats][tim_ring->optimized] - [tim_ring->ena_dfb]; + arm_tmo_burst[tim_ring->enable_stats][tim_ring->ena_dfb]; otx2_tim_ops.cancel_burst = otx2_tim_timer_cancel_burst; } @@ -70,51 +68,6 @@ otx2_tim_ring_info_get(const struct rte_event_timer_adapter *adptr, sizeof(struct rte_event_timer_adapter_conf)); } -static void -tim_optimze_bkt_param(struct otx2_tim_ring *tim_ring) -{ - uint64_t tck_nsec; - uint32_t hbkts; - uint32_t lbkts; - - hbkts = rte_align32pow2(tim_ring->nb_bkts); - tck_nsec = RTE_ALIGN_MUL_CEIL(tim_ring->max_tout / (hbkts - 1), 10); - - if ((tck_nsec < TICK2NSEC(OTX2_TIM_MIN_TMO_TKS, - tim_ring->tenns_clk_freq) || - hbkts > OTX2_TIM_MAX_BUCKETS)) - hbkts = 0; - - lbkts = rte_align32prevpow2(tim_ring->nb_bkts); - tck_nsec = RTE_ALIGN_MUL_CEIL((tim_ring->max_tout / (lbkts - 1)), 10); - - if ((tck_nsec < TICK2NSEC(OTX2_TIM_MIN_TMO_TKS, - tim_ring->tenns_clk_freq) || - lbkts > OTX2_TIM_MAX_BUCKETS)) - lbkts = 0; - - if (!hbkts && !lbkts) - return; - - if (!hbkts) { - tim_ring->nb_bkts = lbkts; - goto end; - } else if (!lbkts) { - tim_ring->nb_bkts = hbkts; - goto end; - } - - tim_ring->nb_bkts = (hbkts - tim_ring->nb_bkts) < - (tim_ring->nb_bkts - lbkts) ? hbkts : lbkts; -end: - tim_ring->optimized = true; - tim_ring->tck_nsec = RTE_ALIGN_MUL_CEIL((tim_ring->max_tout / - (tim_ring->nb_bkts - 1)), 10); - otx2_tim_dbg("Optimized configured values"); - otx2_tim_dbg("Nb_bkts : %" PRIu32 "", tim_ring->nb_bkts); - otx2_tim_dbg("Tck_nsec : %" PRIu64 "", tim_ring->tck_nsec); -} - static int tim_chnk_pool_create(struct otx2_tim_ring *tim_ring, struct rte_event_timer_adapter_conf *rcfg) @@ -319,14 +272,6 @@ otx2_tim_ring_create(struct rte_event_timer_adapter *adptr) tim_ring->chunk_sz); tim_ring->nb_chunk_slots = OTX2_TIM_NB_CHUNK_SLOTS(tim_ring->chunk_sz); - /* Try to optimize the bucket parameters. */ - if ((rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES)) { - if (rte_is_power_of_2(tim_ring->nb_bkts)) - tim_ring->optimized = true; - else - tim_optimze_bkt_param(tim_ring); - } - if (tim_ring->disable_npa) tim_ring->nb_chunks = tim_ring->nb_chunks * tim_ring->nb_bkts; else @@ -459,6 +404,7 @@ otx2_tim_ring_start(const struct rte_event_timer_adapter *adptr) tim_ring->tck_int = NSEC2TICK(tim_ring->tck_nsec, rte_get_timer_hz()); tim_ring->tot_int = tim_ring->tck_int * tim_ring->nb_bkts; tim_ring->fast_div = rte_reciprocal_value_u64(tim_ring->tck_int); + tim_ring->fast_bkt = rte_reciprocal_value_u64(tim_ring->nb_bkts); otx2_tim_calibrate_start_tsc(tim_ring); diff --git a/drivers/event/octeontx2/otx2_tim_evdev.h b/drivers/event/octeontx2/otx2_tim_evdev.h index 44e3c7b51..bf89b85b0 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.h +++ b/drivers/event/octeontx2/otx2_tim_evdev.h @@ -76,8 +76,6 @@ #define OTX2_TIM_SP 0x1 #define OTX2_TIM_MP 0x2 -#define OTX2_TIM_BKT_AND 0x4 -#define OTX2_TIM_BKT_MOD 0x8 #define OTX2_TIM_ENA_FB 0x10 #define OTX2_TIM_ENA_DFB 0x20 #define OTX2_TIM_ENA_STATS 0x40 @@ -149,11 +147,11 @@ struct otx2_tim_ring { struct otx2_tim_bkt *bkt; struct rte_mempool *chunk_pool; struct rte_reciprocal_u64 fast_div; + struct rte_reciprocal_u64 fast_bkt; uint64_t arm_cnt; uint8_t prod_type_sp; uint8_t enable_stats; uint8_t disable_npa; - uint8_t optimized; uint8_t ena_dfb; uint16_t ring_id; uint32_t aura; @@ -178,60 +176,38 @@ tim_priv_get(void) return mz->addr; } -#define TIM_ARM_FASTPATH_MODES \ -FP(mod_sp, 0, 0, 0, 0, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ -FP(mod_mp, 0, 0, 0, 1, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ -FP(mod_fb_sp, 0, 0, 1, 0, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ -FP(mod_fb_mp, 0, 0, 1, 1, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_FB | OTX2_TIM_MP) \ -FP(and_sp, 0, 1, 0, 0, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ -FP(and_mp, 0, 1, 0, 1, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ -FP(and_fb_sp, 0, 1, 1, 0, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ -FP(and_fb_mp, 0, 1, 1, 1, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_FB | OTX2_TIM_MP) \ -FP(stats_mod_sp, 1, 0, 0, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ -FP(stats_mod_mp, 1, 0, 0, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ -FP(stats_mod_fb_sp, 1, 0, 1, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ -FP(stats_mod_fb_mp, 1, 0, 1, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_FB | OTX2_TIM_MP) \ -FP(stats_and_sp, 1, 1, 0, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ -FP(stats_and_mp, 1, 1, 0, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ -FP(stats_and_fb_sp, 1, 1, 1, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ -FP(stats_and_fb_mp, 1, 1, 1, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_FB | OTX2_TIM_MP) - -#define TIM_ARM_TMO_FASTPATH_MODES \ -FP(mod, 0, 0, 0, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_DFB) \ -FP(mod_fb, 0, 0, 1, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_FB) \ -FP(and, 0, 1, 0, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_DFB) \ -FP(and_fb, 0, 1, 1, OTX2_TIM_BKT_AND | OTX2_TIM_ENA_FB) \ -FP(stats_mod, 1, 0, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_DFB) \ -FP(stats_mod_fb, 1, 0, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_MOD | \ - OTX2_TIM_ENA_FB) \ -FP(stats_and, 1, 1, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_DFB) \ -FP(stats_and_fb, 1, 1, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_BKT_AND | \ - OTX2_TIM_ENA_FB) - -#define FP(_name, _f4, _f3, _f2, _f1, flags) \ -uint16_t \ -otx2_tim_arm_burst_ ## _name(const struct rte_event_timer_adapter *adptr, \ - struct rte_event_timer **tim, \ - const uint16_t nb_timers); +#define TIM_ARM_FASTPATH_MODES \ + FP(sp, 0, 0, 0, OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ + FP(mp, 0, 0, 1, OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ + FP(fb_sp, 0, 1, 0, OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ + FP(fb_mp, 0, 1, 1, OTX2_TIM_ENA_FB | OTX2_TIM_MP) \ + FP(stats_mod_sp, 1, 0, 0, \ + OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ + FP(stats_mod_mp, 1, 0, 1, \ + OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ + FP(stats_mod_fb_sp, 1, 1, 0, \ + OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_FB | OTX2_TIM_SP) \ + FP(stats_mod_fb_mp, 1, 1, 1, \ + OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_FB | OTX2_TIM_MP) + +#define TIM_ARM_TMO_FASTPATH_MODES \ + FP(dfb, 0, 0, OTX2_TIM_ENA_DFB) \ + FP(fb, 0, 1, OTX2_TIM_ENA_FB) \ + FP(stats_dfb, 1, 0, OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_DFB) \ + FP(stats_fb, 1, 1, OTX2_TIM_ENA_STATS | OTX2_TIM_ENA_FB) + +#define FP(_name, _f3, _f2, _f1, flags) \ + uint16_t otx2_tim_arm_burst_##_name( \ + const struct rte_event_timer_adapter *adptr, \ + struct rte_event_timer **tim, const uint16_t nb_timers); TIM_ARM_FASTPATH_MODES #undef FP -#define FP(_name, _f3, _f2, _f1, flags) \ -uint16_t \ -otx2_tim_arm_tmo_tick_burst_ ## _name( \ - const struct rte_event_timer_adapter *adptr, \ - struct rte_event_timer **tim, \ - const uint64_t timeout_tick, const uint16_t nb_timers); +#define FP(_name, _f2, _f1, flags) \ + uint16_t otx2_tim_arm_tmo_tick_burst_##_name( \ + const struct rte_event_timer_adapter *adptr, \ + struct rte_event_timer **tim, const uint64_t timeout_tick, \ + const uint16_t nb_timers); TIM_ARM_TMO_FASTPATH_MODES #undef FP diff --git a/drivers/event/octeontx2/otx2_tim_worker.c b/drivers/event/octeontx2/otx2_tim_worker.c index 4b5cfdc72..eb901844d 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.c +++ b/drivers/event/octeontx2/otx2_tim_worker.c @@ -136,7 +136,7 @@ tim_timer_arm_tmo_brst(const struct rte_event_timer_adapter *adptr, return set_timers; } -#define FP(_name, _f4, _f3, _f2, _f1, _flags) \ +#define FP(_name, _f3, _f2, _f1, _flags) \ uint16_t __rte_noinline \ otx2_tim_arm_burst_ ## _name(const struct rte_event_timer_adapter *adptr, \ struct rte_event_timer **tim, \ @@ -147,7 +147,7 @@ otx2_tim_arm_burst_ ## _name(const struct rte_event_timer_adapter *adptr, \ TIM_ARM_FASTPATH_MODES #undef FP -#define FP(_name, _f3, _f2, _f1, _flags) \ +#define FP(_name, _f2, _f1, _flags) \ uint16_t __rte_noinline \ otx2_tim_arm_tmo_tick_burst_ ## _name( \ const struct rte_event_timer_adapter *adptr, \ diff --git a/drivers/event/octeontx2/otx2_tim_worker.h b/drivers/event/octeontx2/otx2_tim_worker.h index af2f864d7..f03912b81 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.h +++ b/drivers/event/octeontx2/otx2_tim_worker.h @@ -115,27 +115,27 @@ tim_bkt_clr_nent(struct otx2_tim_bkt *bktp) return __atomic_and_fetch(&bktp->w1, v, __ATOMIC_ACQ_REL); } +static inline uint64_t +tim_bkt_fast_mod(uint64_t n, uint64_t d, struct rte_reciprocal_u64 R) +{ + return (n - (d * rte_reciprocal_divide_u64(n, &R))); +} + static __rte_always_inline void -tim_get_target_bucket(struct otx2_tim_ring * const tim_ring, +tim_get_target_bucket(struct otx2_tim_ring *const tim_ring, const uint32_t rel_bkt, struct otx2_tim_bkt **bkt, - struct otx2_tim_bkt **mirr_bkt, const uint8_t flag) + struct otx2_tim_bkt **mirr_bkt) { const uint64_t bkt_cyc = rte_rdtsc() - tim_ring->ring_start_cyc; - uint32_t bucket = rte_reciprocal_divide_u64(bkt_cyc, - &tim_ring->fast_div) + rel_bkt; - uint32_t mirr_bucket = 0; - - if (flag & OTX2_TIM_BKT_MOD) { - bucket = bucket % tim_ring->nb_bkts; - mirr_bucket = (bucket + (tim_ring->nb_bkts >> 1)) % - tim_ring->nb_bkts; - } - if (flag & OTX2_TIM_BKT_AND) { - bucket = bucket & (tim_ring->nb_bkts - 1); - mirr_bucket = (bucket + (tim_ring->nb_bkts >> 1)) & - (tim_ring->nb_bkts - 1); - } - + uint64_t bucket = + rte_reciprocal_divide_u64(bkt_cyc, &tim_ring->fast_div) + + rel_bkt; + uint64_t mirr_bucket = 0; + + bucket = + tim_bkt_fast_mod(bucket, tim_ring->nb_bkts, tim_ring->fast_bkt); + mirr_bucket = tim_bkt_fast_mod(bucket + (tim_ring->nb_bkts >> 1), + tim_ring->nb_bkts, tim_ring->fast_bkt); *bkt = &tim_ring->bkt[bucket]; *mirr_bkt = &tim_ring->bkt[mirr_bucket]; } @@ -236,7 +236,7 @@ tim_add_entry_sp(struct otx2_tim_ring * const tim_ring, int16_t rem; __retry: - tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt, flags); + tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt); /* Get Bucket sema*/ lock_sema = tim_bkt_fetch_sema_lock(bkt); @@ -322,7 +322,7 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, int16_t rem; __retry: - tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt, flags); + tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt); /* Get Bucket sema*/ lock_sema = tim_bkt_fetch_sema_lock(bkt); @@ -454,7 +454,7 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring, uint8_t lock_cnt; __retry: - tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt, flags); + tim_get_target_bucket(tim_ring, rel_bkt, &bkt, &mirr_bkt); /* Only one thread beyond this. */ lock_sema = tim_bkt_inc_lock(bkt); From patchwork Sun Mar 21 08:49:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 89599 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F2B94A034F; Sun, 21 Mar 2021 09:50:06 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7D536140DFC; Sun, 21 Mar 2021 09:49:58 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id D548F140DF2 for ; Sun, 21 Mar 2021 09:49:56 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12L8kfvl013206 for ; Sun, 21 Mar 2021 01:49:56 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=QQxPv00LWYfTHsh8j69rkiGEam6Ib99jlfCJNOs2RGM=; b=Mquf0l+/ox/pWb15DmUagvMNrbD4zpTK1QoVl3LPq3UCWQcvI3fwfGl/PrwNkJW8lbjw YAdqLNUspZbpxlgpGR+swI1LF35zqxhMBT8V+7Ab+y2GqyXIK4OLo5kY4L9FInNlgStT RwexAk1u/QOaOrs2olHbTP9Iem4zoHwekgZpOhakBMxP5WFc0qSAPUnp7oZjBvlb/HL0 ieKcCKOxeUqnS0Rw+iSQa0IVnOkF/aHD9BMzf5oRWbxiAoxw3G2TqoSYBAYD/gqZ0QbP gnhrLCdaP6BX2ZIwWfkb09tVJTwfPN7rfk6dUcz5y8s4qRT3YvIRkQCk0oMrXnEKLH0i AA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37dgjnsrkn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sun, 21 Mar 2021 01:49:56 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 21 Mar 2021 01:49:53 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sun, 21 Mar 2021 01:49:53 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id CCF4B3F703F; Sun, 21 Mar 2021 01:49:52 -0700 (PDT) From: To: , Pavan Nikhilesh CC: Date: Sun, 21 Mar 2021 14:19:12 +0530 Message-ID: <20210321084915.2649-2-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210321084915.2649-1-pbhagavatula@marvell.com> References: <20210225122315.6350-1-pbhagavatula@marvell.com> <20210321084915.2649-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-20_13:2021-03-19, 2021-03-20 signatures=0 Subject: [dpdk-dev] [PATCH v2 2/4] event/octeontx2: optimize timer arm routine X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Use relaxed load exclusive when polling for other threads or hardware to complete. Signed-off-by: Pavan Nikhilesh --- drivers/event/octeontx2/otx2_tim_worker.c | 1 + drivers/event/octeontx2/otx2_tim_worker.h | 162 ++++++++++++---------- 2 files changed, 89 insertions(+), 74 deletions(-) diff --git a/drivers/event/octeontx2/otx2_tim_worker.c b/drivers/event/octeontx2/otx2_tim_worker.c index eb901844d..6a3511ec0 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.c +++ b/drivers/event/octeontx2/otx2_tim_worker.c @@ -170,6 +170,7 @@ otx2_tim_timer_cancel_burst(const struct rte_event_timer_adapter *adptr, int ret; RTE_SET_USED(adptr); + rte_atomic_thread_fence(__ATOMIC_ACQUIRE); for (index = 0; index < nb_timers; index++) { if (tim[index]->state == RTE_EVENT_TIMER_CANCELED) { rte_errno = EALREADY; diff --git a/drivers/event/octeontx2/otx2_tim_worker.h b/drivers/event/octeontx2/otx2_tim_worker.h index f03912b81..aa69da96c 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.h +++ b/drivers/event/octeontx2/otx2_tim_worker.h @@ -84,7 +84,13 @@ tim_bkt_inc_lock(struct otx2_tim_bkt *bktp) static inline void tim_bkt_dec_lock(struct otx2_tim_bkt *bktp) { - __atomic_add_fetch(&bktp->lock, 0xff, __ATOMIC_RELEASE); + __atomic_fetch_sub(&bktp->lock, 1, __ATOMIC_RELEASE); +} + +static inline void +tim_bkt_dec_lock_relaxed(struct otx2_tim_bkt *bktp) +{ + __atomic_fetch_sub(&bktp->lock, 1, __ATOMIC_RELAXED); } static inline uint32_t @@ -246,22 +252,20 @@ tim_add_entry_sp(struct otx2_tim_ring * const tim_ring, if (tim_bkt_get_nent(lock_sema) != 0) { uint64_t hbt_state; #ifdef RTE_ARCH_ARM64 - asm volatile( - " ldaxr %[hbt], [%[w1]] \n" - " tbz %[hbt], 33, dne%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldaxr %[hbt], [%[w1]] \n" - " tbnz %[hbt], 33, rty%= \n" - "dne%=: \n" - : [hbt] "=&r" (hbt_state) - : [w1] "r" ((&bkt->w1)) - : "memory" - ); + asm volatile(" ldxr %[hbt], [%[w1]] \n" + " tbz %[hbt], 33, dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxr %[hbt], [%[w1]] \n" + " tbnz %[hbt], 33, rty%= \n" + "dne%=: \n" + : [hbt] "=&r"(hbt_state) + : [w1] "r"((&bkt->w1)) + : "memory"); #else do { hbt_state = __atomic_load_n(&bkt->w1, - __ATOMIC_ACQUIRE); + __ATOMIC_RELAXED); } while (hbt_state & BIT_ULL(33)); #endif @@ -282,10 +286,10 @@ tim_add_entry_sp(struct otx2_tim_ring * const tim_ring, if (unlikely(chunk == NULL)) { bkt->chunk_remainder = 0; - tim_bkt_dec_lock(bkt); tim->impl_opaque[0] = 0; tim->impl_opaque[1] = 0; tim->state = RTE_EVENT_TIMER_ERROR; + tim_bkt_dec_lock(bkt); return -ENOMEM; } mirr_bkt->current_chunk = (uintptr_t)chunk; @@ -298,12 +302,11 @@ tim_add_entry_sp(struct otx2_tim_ring * const tim_ring, /* Copy work entry. */ *chunk = *pent; - tim_bkt_inc_nent(bkt); - tim_bkt_dec_lock(bkt); - tim->impl_opaque[0] = (uintptr_t)chunk; tim->impl_opaque[1] = (uintptr_t)bkt; - tim->state = RTE_EVENT_TIMER_ARMED; + __atomic_store_n(&tim->state, RTE_EVENT_TIMER_ARMED, __ATOMIC_RELEASE); + tim_bkt_inc_nent(bkt); + tim_bkt_dec_lock_relaxed(bkt); return 0; } @@ -331,22 +334,20 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, if (tim_bkt_get_nent(lock_sema) != 0) { uint64_t hbt_state; #ifdef RTE_ARCH_ARM64 - asm volatile( - " ldaxr %[hbt], [%[w1]] \n" - " tbz %[hbt], 33, dne%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldaxr %[hbt], [%[w1]] \n" - " tbnz %[hbt], 33, rty%= \n" - "dne%=: \n" - : [hbt] "=&r" (hbt_state) - : [w1] "r" ((&bkt->w1)) - : "memory" - ); + asm volatile(" ldxr %[hbt], [%[w1]] \n" + " tbz %[hbt], 33, dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxr %[hbt], [%[w1]] \n" + " tbnz %[hbt], 33, rty%= \n" + "dne%=: \n" + : [hbt] "=&r"(hbt_state) + : [w1] "r"((&bkt->w1)) + : "memory"); #else do { hbt_state = __atomic_load_n(&bkt->w1, - __ATOMIC_ACQUIRE); + __ATOMIC_RELAXED); } while (hbt_state & BIT_ULL(33)); #endif @@ -359,26 +360,23 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, rem = tim_bkt_fetch_rem(lock_sema); if (rem < 0) { + tim_bkt_dec_lock(bkt); #ifdef RTE_ARCH_ARM64 - asm volatile( - " ldaxrh %w[rem], [%[crem]] \n" - " tbz %w[rem], 15, dne%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldaxrh %w[rem], [%[crem]] \n" - " tbnz %w[rem], 15, rty%= \n" - "dne%=: \n" - : [rem] "=&r" (rem) - : [crem] "r" (&bkt->chunk_remainder) - : "memory" - ); + asm volatile(" ldxr %[rem], [%[crem]] \n" + " tbz %[rem], 63, dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxr %[rem], [%[crem]] \n" + " tbnz %[rem], 63, rty%= \n" + "dne%=: \n" + : [rem] "=&r"(rem) + : [crem] "r"(&bkt->w1) + : "memory"); #else - while (__atomic_load_n(&bkt->chunk_remainder, - __ATOMIC_ACQUIRE) < 0) + while (__atomic_load_n((int64_t *)&bkt->w1, __ATOMIC_RELAXED) < + 0) ; #endif - /* Goto diff bucket. */ - tim_bkt_dec_lock(bkt); goto __retry; } else if (!rem) { /* Only one thread can be here*/ @@ -388,18 +386,21 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, chunk = tim_insert_chunk(bkt, mirr_bkt, tim_ring); if (unlikely(chunk == NULL)) { - tim_bkt_set_rem(bkt, 0); - tim_bkt_dec_lock(bkt); tim->impl_opaque[0] = 0; tim->impl_opaque[1] = 0; tim->state = RTE_EVENT_TIMER_ERROR; + tim_bkt_set_rem(bkt, 0); + tim_bkt_dec_lock(bkt); return -ENOMEM; } *chunk = *pent; - while (tim_bkt_fetch_lock(lock_sema) != - (-tim_bkt_fetch_rem(lock_sema))) - lock_sema = __atomic_load_n(&bkt->w1, __ATOMIC_ACQUIRE); - + if (tim_bkt_fetch_lock(lock_sema)) { + do { + lock_sema = __atomic_load_n(&bkt->w1, + __ATOMIC_RELAXED); + } while (tim_bkt_fetch_lock(lock_sema) - 1); + rte_atomic_thread_fence(__ATOMIC_ACQUIRE); + } mirr_bkt->current_chunk = (uintptr_t)chunk; __atomic_store_n(&bkt->chunk_remainder, tim_ring->nb_chunk_slots - 1, __ATOMIC_RELEASE); @@ -409,12 +410,11 @@ tim_add_entry_mp(struct otx2_tim_ring * const tim_ring, *chunk = *pent; } - /* Copy work entry. */ - tim_bkt_inc_nent(bkt); - tim_bkt_dec_lock(bkt); tim->impl_opaque[0] = (uintptr_t)chunk; tim->impl_opaque[1] = (uintptr_t)bkt; - tim->state = RTE_EVENT_TIMER_ARMED; + __atomic_store_n(&tim->state, RTE_EVENT_TIMER_ARMED, __ATOMIC_RELEASE); + tim_bkt_inc_nent(bkt); + tim_bkt_dec_lock_relaxed(bkt); return 0; } @@ -463,6 +463,23 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring, if (lock_cnt) { tim_bkt_dec_lock(bkt); +#ifdef RTE_ARCH_ARM64 + asm volatile(" ldxrb %w[lock_cnt], [%[lock]] \n" + " tst %w[lock_cnt], 255 \n" + " beq dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxrb %w[lock_cnt], [%[lock]] \n" + " tst %w[lock_cnt], 255 \n" + " bne rty%= \n" + "dne%=: \n" + : [lock_cnt] "=&r"(lock_cnt) + : [lock] "r"(&bkt->lock) + : "memory"); +#else + while (__atomic_load_n(&bkt->lock, __ATOMIC_RELAXED)) + ; +#endif goto __retry; } @@ -471,22 +488,20 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring, if (tim_bkt_get_nent(lock_sema) != 0) { uint64_t hbt_state; #ifdef RTE_ARCH_ARM64 - asm volatile( - " ldaxr %[hbt], [%[w1]] \n" - " tbz %[hbt], 33, dne%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldaxr %[hbt], [%[w1]] \n" - " tbnz %[hbt], 33, rty%= \n" - "dne%=: \n" - : [hbt] "=&r" (hbt_state) - : [w1] "r" ((&bkt->w1)) - : "memory" - ); + asm volatile(" ldxr %[hbt], [%[w1]] \n" + " tbz %[hbt], 33, dne%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldxr %[hbt], [%[w1]] \n" + " tbnz %[hbt], 33, rty%= \n" + "dne%=: \n" + : [hbt] "=&r"(hbt_state) + : [w1] "r"((&bkt->w1)) + : "memory"); #else do { hbt_state = __atomic_load_n(&bkt->w1, - __ATOMIC_ACQUIRE); + __ATOMIC_RELAXED); } while (hbt_state & BIT_ULL(33)); #endif @@ -563,19 +578,18 @@ tim_rm_entry(struct rte_event_timer *tim) bkt = (struct otx2_tim_bkt *)(uintptr_t)tim->impl_opaque[1]; lock_sema = tim_bkt_inc_lock(bkt); if (tim_bkt_get_hbt(lock_sema) || !tim_bkt_get_nent(lock_sema)) { - tim_bkt_dec_lock(bkt); tim->impl_opaque[0] = 0; tim->impl_opaque[1] = 0; + tim_bkt_dec_lock(bkt); return -ENOENT; } entry->w0 = 0; entry->wqe = 0; - tim_bkt_dec_lock(bkt); - tim->state = RTE_EVENT_TIMER_CANCELED; tim->impl_opaque[0] = 0; tim->impl_opaque[1] = 0; + tim_bkt_dec_lock(bkt); return 0; } From patchwork Sun Mar 21 08:49:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 89600 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 09D57A034F; Sun, 21 Mar 2021 09:50:15 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D9441140E25; Sun, 21 Mar 2021 09:49:59 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id BD5EC140E24 for ; Sun, 21 Mar 2021 09:49:58 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12L8l9de014497 for ; Sun, 21 Mar 2021 01:49:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=xexL9EgCec+AhX/xNk61JvLknszYN4ZPZbvjUq0MMXs=; b=dB1ofIoVPKY3anJ3RpNbrLmt4OTiLyN5n7FS9Y5UuPvwSqrhuYMitn7YXLo/l4Z4w7bQ lzTjTe5pEAnf46gdaQSbcl0CyOXj7bwTXc1bkZ3UjAAacgaaFdT5hbQdlWxPZXPMonMF D67wsrahVtvaSjaf10vZjTxQljYdU8rGbCY39/Mx4dZSPmVW1KjH7RoyrxOqc455O81Z eyQB+spWqAYB6WoAk8PV9ukVETDioGl2Ell9EZ5F33UIw310ff6bpK2gmXMusMb7Lq03 VQD86xJ1qAoI8sbfX+Y2Y1hRXLiyj90csCCxJcntMsosdQbswl/XWUPwwkMBi1JLTvfu TQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37dgjnsrkw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sun, 21 Mar 2021 01:49:58 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 21 Mar 2021 01:49:56 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sun, 21 Mar 2021 01:49:55 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id 150113F703F; Sun, 21 Mar 2021 01:49:54 -0700 (PDT) From: To: , Pavan Nikhilesh CC: Date: Sun, 21 Mar 2021 14:19:13 +0530 Message-ID: <20210321084915.2649-3-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210321084915.2649-1-pbhagavatula@marvell.com> References: <20210225122315.6350-1-pbhagavatula@marvell.com> <20210321084915.2649-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-20_13:2021-03-19, 2021-03-20 signatures=0 Subject: [dpdk-dev] [PATCH v2 3/4] event/octeontx2: reduce chunk pool memory usage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Reduce amount of memory used by chunk pool when the mempool used is OCTEONTX2 NPA. Previously, the number of chunks configured when NPA is used is equal to the number of timers requested plus the number of buckets and if the max timeout is long enough w.r.t. resolution requested there will a large number of buckets which would cause high memory usage. Reduce the number of chunks when NPA is used to the number of timers requested as buckets that are processed chunk lists are automatically freed. Signed-off-by: Pavan Nikhilesh --- drivers/event/octeontx2/otx2_tim_evdev.c | 19 ++++++++++--------- drivers/event/octeontx2/otx2_tim_evdev.h | 2 +- 2 files changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/event/octeontx2/otx2_tim_evdev.c b/drivers/event/octeontx2/otx2_tim_evdev.c index d1e967eb7..4fb002ddb 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.c +++ b/drivers/event/octeontx2/otx2_tim_evdev.c @@ -91,6 +91,8 @@ tim_chnk_pool_create(struct otx2_tim_ring *tim_ring, if (cache_sz > RTE_MEMPOOL_CACHE_MAX_SIZE) cache_sz = RTE_MEMPOOL_CACHE_MAX_SIZE; + cache_sz = cache_sz != 0 ? cache_sz : 2; + tim_ring->nb_chunks += (cache_sz * rte_lcore_count()); if (!tim_ring->disable_npa) { tim_ring->chunk_pool = rte_mempool_create_empty(pool_name, tim_ring->nb_chunks, tim_ring->chunk_sz, @@ -268,16 +270,15 @@ otx2_tim_ring_create(struct rte_event_timer_adapter *adptr) } } - tim_ring->nb_chunks = tim_ring->nb_timers / OTX2_TIM_NB_CHUNK_SLOTS( - tim_ring->chunk_sz); - tim_ring->nb_chunk_slots = OTX2_TIM_NB_CHUNK_SLOTS(tim_ring->chunk_sz); - - if (tim_ring->disable_npa) + if (tim_ring->disable_npa) { + tim_ring->nb_chunks = + tim_ring->nb_timers / + OTX2_TIM_NB_CHUNK_SLOTS(tim_ring->chunk_sz); tim_ring->nb_chunks = tim_ring->nb_chunks * tim_ring->nb_bkts; - else - tim_ring->nb_chunks = tim_ring->nb_chunks + tim_ring->nb_bkts; - - /* Create buckets. */ + } else { + tim_ring->nb_chunks = tim_ring->nb_timers; + } + tim_ring->nb_chunk_slots = OTX2_TIM_NB_CHUNK_SLOTS(tim_ring->chunk_sz); tim_ring->bkt = rte_zmalloc("otx2_tim_bucket", (tim_ring->nb_bkts) * sizeof(struct otx2_tim_bkt), RTE_CACHE_LINE_SIZE); diff --git a/drivers/event/octeontx2/otx2_tim_evdev.h b/drivers/event/octeontx2/otx2_tim_evdev.h index bf89b85b0..0667d4576 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.h +++ b/drivers/event/octeontx2/otx2_tim_evdev.h @@ -70,7 +70,7 @@ #define OTX2_TIM_MAX_BURST (RTE_CACHE_LINE_SIZE / \ OTX2_TIM_CHUNK_ALIGNMENT) #define OTX2_TIM_NB_CHUNK_SLOTS(sz) (((sz) / OTX2_TIM_CHUNK_ALIGNMENT) - 1) -#define OTX2_TIM_MIN_CHUNK_SLOTS (0x1) +#define OTX2_TIM_MIN_CHUNK_SLOTS (0x8) #define OTX2_TIM_MAX_CHUNK_SLOTS (0x1FFE) #define OTX2_TIM_MIN_TMO_TKS (256) From patchwork Sun Mar 21 08:49:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 89601 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 62605A034F; Sun, 21 Mar 2021 09:50:23 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1EA7C140E30; Sun, 21 Mar 2021 09:50:02 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id B1249140E2A for ; Sun, 21 Mar 2021 09:50:00 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12L8kYAG019208 for ; Sun, 21 Mar 2021 01:49:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=GkvfsTRgS0VQl2YZLmlof3MMi2mmVn2pQo51eGw7xOE=; b=ZyFhKZSJ75TQYI0QQR4EfvWqfwti2dPnwIEXw1Tl2zk5qUHtr5OmWsjGYDVJXHK6fFKp NYqEWiEj1LoCPJCcb+hqW+dCE6Q+MnxMMQXkmXnyBE8WCzsyWOiLZihYWIydxH1scLdY sEWHrKNpYZDjFk3hOGGC8XAH1g2vCYbszsnPZ7kQn245Z8bHZfDTKi6mmZhtXsZtrWO+ 9dAe6cOwCIwrPFE+1eNMPZJI6gdb+2EpbbqQauzJc/m4CCBAuZT+qjW4Zk5OROgPGY3Q pHZGMHhLjXr9INFShTudNpIRtWlqPs4srX3L5ZjBkvHTGle3tW401dNUjng+dcYJahVo jw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 37dedra09e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sun, 21 Mar 2021 01:49:59 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 21 Mar 2021 01:49:57 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sun, 21 Mar 2021 01:49:57 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id 559193F703F; Sun, 21 Mar 2021 01:49:57 -0700 (PDT) From: To: , Pavan Nikhilesh CC: Date: Sun, 21 Mar 2021 14:19:14 +0530 Message-ID: <20210321084915.2649-4-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210321084915.2649-1-pbhagavatula@marvell.com> References: <20210225122315.6350-1-pbhagavatula@marvell.com> <20210321084915.2649-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-20_13:2021-03-19, 2021-03-20 signatures=0 Subject: [dpdk-dev] [PATCH v2 4/4] event/octeontx2: timer always use virtual counter X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Use virtual counter for estimating current bucket as PMU cannot be reliably used to estimate time. Signed-off-by: Pavan Nikhilesh --- drivers/event/octeontx2/otx2_tim_evdev.c | 19 +++-------------- drivers/event/octeontx2/otx2_tim_evdev.h | 26 +++++++++++++++++++++++ drivers/event/octeontx2/otx2_tim_worker.c | 4 ++-- drivers/event/octeontx2/otx2_tim_worker.h | 2 +- 4 files changed, 32 insertions(+), 19 deletions(-) diff --git a/drivers/event/octeontx2/otx2_tim_evdev.c b/drivers/event/octeontx2/otx2_tim_evdev.c index 4fb002ddb..926c2dce6 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.c +++ b/drivers/event/octeontx2/otx2_tim_evdev.c @@ -354,7 +354,7 @@ otx2_tim_calibrate_start_tsc(struct otx2_tim_ring *tim_ring) for (icount = 0; icount < OTX2_TIM_CALIB_ITER; icount++) { real_bkt = otx2_read64(tim_ring->base + TIM_LF_RING_REL) >> 44; - bkt_cyc = rte_rdtsc(); + bkt_cyc = tim_cntvct(); bucket = (bkt_cyc - tim_ring->ring_start_cyc) / tim_ring->tck_int; bucket = bucket % (tim_ring->nb_bkts); @@ -389,20 +389,8 @@ otx2_tim_ring_start(const struct rte_event_timer_adapter *adptr) tim_err_desc(rc); goto fail; } -#ifdef RTE_ARM_EAL_RDTSC_USE_PMU - uint64_t tenns_stmp, tenns_diff; - uint64_t pmu_stmp; - - pmu_stmp = rte_rdtsc(); - asm volatile("mrs %0, cntvct_el0" : "=r" (tenns_stmp)); - - tenns_diff = tenns_stmp - rsp->timestarted; - pmu_stmp = pmu_stmp - (NSEC2TICK(tenns_diff * 10, rte_get_timer_hz())); - tim_ring->ring_start_cyc = pmu_stmp; -#else tim_ring->ring_start_cyc = rsp->timestarted; -#endif - tim_ring->tck_int = NSEC2TICK(tim_ring->tck_nsec, rte_get_timer_hz()); + tim_ring->tck_int = NSEC2TICK(tim_ring->tck_nsec, tim_cntfrq()); tim_ring->tot_int = tim_ring->tck_int * tim_ring->nb_bkts; tim_ring->fast_div = rte_reciprocal_value_u64(tim_ring->tck_int); tim_ring->fast_bkt = rte_reciprocal_value_u64(tim_ring->nb_bkts); @@ -470,8 +458,7 @@ otx2_tim_stats_get(const struct rte_event_timer_adapter *adapter, struct rte_event_timer_adapter_stats *stats) { struct otx2_tim_ring *tim_ring = adapter->data->adapter_priv; - uint64_t bkt_cyc = rte_rdtsc() - tim_ring->ring_start_cyc; - + uint64_t bkt_cyc = tim_cntvct() - tim_ring->ring_start_cyc; stats->evtim_exp_count = __atomic_load_n(&tim_ring->arm_cnt, __ATOMIC_RELAXED); diff --git a/drivers/event/octeontx2/otx2_tim_evdev.h b/drivers/event/octeontx2/otx2_tim_evdev.h index 0667d4576..410880e14 100644 --- a/drivers/event/octeontx2/otx2_tim_evdev.h +++ b/drivers/event/octeontx2/otx2_tim_evdev.h @@ -176,6 +176,32 @@ tim_priv_get(void) return mz->addr; } +#ifdef RTE_ARCH_ARM64 +static inline uint64_t +tim_cntvct(void) +{ + return __rte_arm64_cntvct(); +} + +static inline uint64_t +tim_cntfrq(void) +{ + return __rte_arm64_cntfrq(); +} +#else +static inline uint64_t +tim_cntvct(void) +{ + return 0; +} + +static inline uint64_t +tim_cntfrq(void) +{ + return 0; +} +#endif + #define TIM_ARM_FASTPATH_MODES \ FP(sp, 0, 0, 0, OTX2_TIM_ENA_DFB | OTX2_TIM_SP) \ FP(mp, 0, 0, 1, OTX2_TIM_ENA_DFB | OTX2_TIM_MP) \ diff --git a/drivers/event/octeontx2/otx2_tim_worker.c b/drivers/event/octeontx2/otx2_tim_worker.c index 6a3511ec0..9ee07958f 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.c +++ b/drivers/event/octeontx2/otx2_tim_worker.c @@ -41,12 +41,12 @@ tim_format_event(const struct rte_event_timer * const tim, static inline void tim_sync_start_cyc(struct otx2_tim_ring *tim_ring) { - uint64_t cur_cyc = rte_rdtsc(); + uint64_t cur_cyc = tim_cntvct(); uint32_t real_bkt; if (cur_cyc - tim_ring->last_updt_cyc > tim_ring->tot_int) { real_bkt = otx2_read64(tim_ring->base + TIM_LF_RING_REL) >> 44; - cur_cyc = rte_rdtsc(); + cur_cyc = tim_cntvct(); tim_ring->ring_start_cyc = cur_cyc - (real_bkt * tim_ring->tck_int); diff --git a/drivers/event/octeontx2/otx2_tim_worker.h b/drivers/event/octeontx2/otx2_tim_worker.h index aa69da96c..582b9c1b9 100644 --- a/drivers/event/octeontx2/otx2_tim_worker.h +++ b/drivers/event/octeontx2/otx2_tim_worker.h @@ -132,7 +132,7 @@ tim_get_target_bucket(struct otx2_tim_ring *const tim_ring, const uint32_t rel_bkt, struct otx2_tim_bkt **bkt, struct otx2_tim_bkt **mirr_bkt) { - const uint64_t bkt_cyc = rte_rdtsc() - tim_ring->ring_start_cyc; + const uint64_t bkt_cyc = tim_cntvct() - tim_ring->ring_start_cyc; uint64_t bucket = rte_reciprocal_divide_u64(bkt_cyc, &tim_ring->fast_div) + rel_bkt;