From patchwork Tue Mar 10 10:54:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linhaifeng X-Patchwork-Id: 66502 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E3A55A0565; Tue, 10 Mar 2020 11:54:55 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5085D1C002; Tue, 10 Mar 2020 11:54:55 +0100 (CET) Received: from huawei.com (szxga01-in.huawei.com [45.249.212.187]) by dpdk.org (Postfix) with ESMTP id AA9901C001 for ; Tue, 10 Mar 2020 11:54:53 +0100 (CET) Received: from dggeml406-hub.china.huawei.com (unknown [172.30.72.57]) by Forcepoint Email with ESMTP id 13E8BCFC7F36F599F3E5; Tue, 10 Mar 2020 18:54:52 +0800 (CST) Received: from DGGEML421-HUB.china.huawei.com (10.1.199.38) by dggeml406-hub.china.huawei.com (10.3.17.50) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 10 Mar 2020 18:54:51 +0800 Received: from DGGEML502-MBX.china.huawei.com ([169.254.2.18]) by dggeml421-hub.china.huawei.com ([10.1.199.38]) with mapi id 14.03.0439.000; Tue, 10 Mar 2020 18:54:43 +0800 From: Linhaifeng To: Jerin Jacob CC: Gavin Hu , "dev@dpdk.org" , "thomas@monjalon.net" , chenchanghu , xudingke , "Lilijun (Jerry)" , Honnappa Nagarahalli , Steve Capper , nd Thread-Topic: [PATCH v4] eal/arm64: fix rdtsc precise version Thread-Index: AdX2ykiGKBTdXVpVQzy8Gn4s50Uz2g== Date: Tue, 10 Mar 2020 10:54:42 +0000 Message-ID: <4099DE2E54AFAD489356C6C9161D53339729F63A@DGGEML502-MBX.china.huawei.com> Accept-Language: en-GB, zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.133.215.248] MIME-Version: 1.0 X-CFilter-Loop: Reflected Subject: [dpdk-dev] [PATCH v4] eal/arm64: fix rdtsc precise version X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In order to get more accurate the cntvct_el0 reading, SW must invoke isb and arch_counter_enforce_ordering. Reference of linux kernel: https://git.kernel.org/pub/scm/linux/kernel/git/ torvalds/linux.git/tree/arch/arm64/include/asm/arch_timer.h?h=v5.5#n220 Fixes: ccad39ea0712 ("eal/arm: add cpu cycle operations for ARMv8") Cc: stable@dpdk.org Signed-off-by: Haifeng Lin --- .../common/include/arch/arm/rte_atomic_64.h | 3 +++ .../common/include/arch/arm/rte_cycles_64.h | 20 +++++++++++++++++-- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h index 859ae129d..2587f98a2 100644 --- a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h +++ b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h @@ -21,6 +21,7 @@ extern "C" { #define dsb(opt) asm volatile("dsb " #opt : : : "memory") #define dmb(opt) asm volatile("dmb " #opt : : : "memory") +#define isb() (asm volatile("isb" : : : "memory")) #define rte_mb() dsb(sy) @@ -44,6 +45,8 @@ extern "C" { #define rte_cio_rmb() dmb(oshld) +#define rte_isb() isb() + /*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) || defined(RTE_ARM_FEATURE_ATOMICS) diff --git a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h index 68e7c7338..bc4e3f8e6 100644 --- a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h +++ b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h @@ -18,6 +18,7 @@ extern "C" { * The time base for this lcore. */ #ifndef RTE_ARM_EAL_RDTSC_USE_PMU + /** * This call is portable to any ARMv8 architecture, however, typically * cntvct_el0 runs at <= 100MHz and it may be imprecise for some tasks. @@ -59,11 +60,26 @@ rte_rdtsc(void) } #endif +#define arch_counter_enforce_ordering(val) do { \ + uint64_t tmp, _val = (val); \ + \ + asm volatile( \ + " eor %0, %1, %1\n" \ + " add %0, sp, %0\n" \ + " ldr xzr, [%0]" \ + : "=r" (tmp) : "r" (_val)); \ +} while (0) + + static inline uint64_t rte_rdtsc_precise(void) { - rte_mb(); - return rte_rdtsc(); + uint64_t tsc; + + rte_isb(); + tsc = rte_rdtsc(); + arch_counter_enforce_ordering(tsc); + return tsc; } static inline uint64_t