diff mbox series

cycles: add isb before read cntvct_el0

Message ID 4099DE2E54AFAD489356C6C9161D53339729EB7E@DGGEML502-MBX.china.huawei.com (mailing list archive)
State Superseded, archived
Delegated to: David Marchand
Headers show
Series cycles: add isb before read cntvct_el0 | expand

Checks

Context Check Description
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-testing success Testing PASS
ci/travis-robot success Travis build: passed
ci/iol-intel-Performance fail Performance Testing issues
ci/Intel-compilation success Compilation OK
ci/checkpatch warning coding style issues

Commit Message

Linhaifeng March 9, 2020, 9:13 a.m. UTC
We nead isb rather than dsb to sync system counter to cntvct_el0.

Signed-off-by: Haifeng Lin <haifeng.lin@huawei.com>
---
 lib/librte_eal/common/include/arch/arm/rte_atomic_64.h | 3 +++
 lib/librte_eal/common/include/arch/arm/rte_cycles_64.h | 2 ++
 2 files changed, 5 insertions(+)

Comments

David Marchand March 9, 2020, 9:19 a.m. UTC | #1
On Mon, Mar 9, 2020 at 10:14 AM Linhaifeng <haifeng.lin@huawei.com> wrote:
>
> We nead isb rather than dsb to sync system counter to cntvct_el0.

I'll leave the arm maintainers look at this, but I have a comment on the form.


>
> Signed-off-by: Haifeng Lin <haifeng.lin@huawei.com>
> ---
>  lib/librte_eal/common/include/arch/arm/rte_atomic_64.h | 3 +++
>  lib/librte_eal/common/include/arch/arm/rte_cycles_64.h | 2 ++
>  2 files changed, 5 insertions(+)
>
> diff --git a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> index 859ae129d..705351394 100644
> --- a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> +++ b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> @@ -21,6 +21,7 @@ extern "C" {
>
>  #define dsb(opt) asm volatile("dsb " #opt : : : "memory")
>  #define dmb(opt) asm volatile("dmb " #opt : : : "memory")
> +#define isb()    asm volatile("isb" : : : "memory")

dsb and dmb should not be exported as public macros in the first place
(I forgot to send the patch that drops those, will send later).
Please don't add more public macro that make no sense except for
aarch64: neither isb, nor rte_isb.


>
>  #define rte_mb() dsb(sy)
>
> @@ -186,6 +187,8 @@ rte_atomic128_cmp_exchange(rte_int128_t *dst, rte_int128_t *exp,
>         return (old.int128 == expected.int128);
>  }
>
> +#define rte_isb() isb()
> +
>  #ifdef __cplusplus
>  }
>  #endif
Jerin Jacob March 9, 2020, 3:43 p.m. UTC | #2
On Mon, Mar 9, 2020 at 2:43 PM Linhaifeng <haifeng.lin@huawei.com> wrote:
>
> We nead isb rather than dsb to sync system counter to cntvct_el0.

# Currently rte_rdtsc() does not have dsb. Right? or any barriers.
# Why do you need it? If it regarding, getting accurate value then use
rte_rdtsc_precise().

>
> Signed-off-by: Haifeng Lin <haifeng.lin@huawei.com>
Linhaifeng March 10, 2020, 2:39 a.m. UTC | #3
-----邮件原件-----
发件人: Jerin Jacob [mailto:jerinjacobk@gmail.com] 
发送时间: 2020年3月9日 23:43
收件人: Linhaifeng <haifeng.lin@huawei.com>
抄送: dev@dpdk.org; thomas@monjalon.net; Lilijun (Jerry) <jerry.lilijun@huawei.com>; chenchanghu <chenchanghu@huawei.com>; xudingke <xudingke@huawei.com>
主题: Re: [dpdk-dev] [PATCH] cycles: add isb before read cntvct_el0

On Mon, Mar 9, 2020 at 2:43 PM Linhaifeng <haifeng.lin@huawei.com> wrote:
>
> We nead isb rather than dsb to sync system counter to cntvct_el0.

# Currently rte_rdtsc() does not have dsb. Right? or any barriers.
# Why do you need it? If it regarding, getting accurate value then use rte_rdtsc_precise().

We use rte_get_tsc_cycles get start_value in pmd1 and end_value in pmd2 in our qos module, it works ok in x86 but not ok in arm64.

Then we use rte_mb() to sync instruction but it not work.Because rte_mb is dsb I think it only have affect on memory. cntvct_el0 and system counter is register so I think we should use isb.

It works well after we use isb in multi core scenes.

Use rte_rdtsc_precise is good idea. Maybe use isb replace of rte_mb(dsb) ?

>
> Signed-off-by: Haifeng Lin <haifeng.lin@huawei.com>
Linhaifeng March 10, 2020, 2:51 a.m. UTC | #4
-----邮件原件-----
发件人: David Marchand [mailto:david.marchand@redhat.com] 
发送时间: 2020年3月9日 17:19
收件人: Linhaifeng <haifeng.lin@huawei.com>
抄送: dev@dpdk.org; thomas@monjalon.net; Lilijun (Jerry) <jerry.lilijun@huawei.com>; chenchanghu <chenchanghu@huawei.com>; xudingke <xudingke@huawei.com>
主题: Re: [dpdk-dev] [PATCH] cycles: add isb before read cntvct_el0

On Mon, Mar 9, 2020 at 10:14 AM Linhaifeng <haifeng.lin@huawei.com> wrote:
>
> We nead isb rather than dsb to sync system counter to cntvct_el0.

I'll leave the arm maintainers look at this, but I have a comment on the form.

Thank you

>
> Signed-off-by: Haifeng Lin <haifeng.lin@huawei.com>
> ---
>  lib/librte_eal/common/include/arch/arm/rte_atomic_64.h | 3 +++  
> lib/librte_eal/common/include/arch/arm/rte_cycles_64.h | 2 ++
>  2 files changed, 5 insertions(+)
>
> diff --git a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h 
> b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> index 859ae129d..705351394 100644
> --- a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> +++ b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> @@ -21,6 +21,7 @@ extern "C" {
>
>  #define dsb(opt) asm volatile("dsb " #opt : : : "memory")  #define 
> dmb(opt) asm volatile("dmb " #opt : : : "memory")
> +#define isb()    asm volatile("isb" : : : "memory")

dsb and dmb should not be exported as public macros in the first place (I forgot to send the patch that drops those, will send later).
Please don't add more public macro that make no sense except for
aarch64: neither isb, nor rte_isb.


Ok.I will send a new patch after yours.


>
>  #define rte_mb() dsb(sy)
>
> @@ -186,6 +187,8 @@ rte_atomic128_cmp_exchange(rte_int128_t *dst, rte_int128_t *exp,
>         return (old.int128 == expected.int128);  }
>
> +#define rte_isb() isb()
> +
>  #ifdef __cplusplus
>  }
>  #endif


--
David Marchand
Jerin Jacob March 10, 2020, 7:53 a.m. UTC | #5
On Tue, Mar 10, 2020 at 8:09 AM Linhaifeng <haifeng.lin@huawei.com> wrote:
>
>
>
> -----邮件原件-----
> 发件人: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> 发送时间: 2020年3月9日 23:43
> 收件人: Linhaifeng <haifeng.lin@huawei.com>
> 抄送: dev@dpdk.org; thomas@monjalon.net; Lilijun (Jerry) <jerry.lilijun@huawei.com>; chenchanghu <chenchanghu@huawei.com>; xudingke <xudingke@huawei.com>
> 主题: Re: [dpdk-dev] [PATCH] cycles: add isb before read cntvct_el0
>
> On Mon, Mar 9, 2020 at 2:43 PM Linhaifeng <haifeng.lin@huawei.com> wrote:
> >
> > We nead isb rather than dsb to sync system counter to cntvct_el0.
>
> # Currently rte_rdtsc() does not have dsb. Right? or any barriers.
> # Why do you need it? If it regarding, getting accurate value then use rte_rdtsc_precise().
>
> We use rte_get_tsc_cycles get start_value in pmd1 and end_value in pmd2 in our qos module, it works ok in x86 but not ok in arm64.
>
> Then we use rte_mb() to sync instruction but it not work.Because rte_mb is dsb I think it only have affect on memory. cntvct_el0 and system counter is register so I think we should use isb.
>
> It works well after we use isb in multi core scenes.
>
> Use rte_rdtsc_precise is good idea. Maybe use isb replace of rte_mb(dsb) ?

Yes. Please.


>
> >
> > Signed-off-by: Haifeng Lin <haifeng.lin@huawei.com>
diff mbox series

Patch

diff --git a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
index 859ae129d..705351394 100644
--- a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
+++ b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
@@ -21,6 +21,7 @@  extern "C" {
 
 #define dsb(opt) asm volatile("dsb " #opt : : : "memory")
 #define dmb(opt) asm volatile("dmb " #opt : : : "memory")
+#define isb()    asm volatile("isb" : : : "memory")
 
 #define rte_mb() dsb(sy)
 
@@ -186,6 +187,8 @@  rte_atomic128_cmp_exchange(rte_int128_t *dst, rte_int128_t *exp,
 	return (old.int128 == expected.int128);
 }
 
+#define rte_isb() isb()
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
index 68e7c7338..29f524901 100644
--- a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
+++ b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
@@ -18,6 +18,7 @@  extern "C" {
  *   The time base for this lcore.
  */
 #ifndef RTE_ARM_EAL_RDTSC_USE_PMU
+
 /**
  * This call is portable to any ARMv8 architecture, however, typically
  * cntvct_el0 runs at <= 100MHz and it may be imprecise for some tasks.
@@ -27,6 +28,7 @@  rte_rdtsc(void)
 {
 	uint64_t tsc;
 
+	rte_isb();
 	asm volatile("mrs %0, cntvct_el0" : "=r" (tsc));
 	return tsc;
 }