From patchwork Thu Sep 12 11:24:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59151 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CAFF21E95D; Thu, 12 Sep 2019 13:25:10 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id D853B1E8BC for ; Thu, 12 Sep 2019 13:25:07 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3BFC21597; Thu, 12 Sep 2019 04:25:07 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 68E543F67D; Thu, 12 Sep 2019 04:25:05 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com Date: Thu, 12 Sep 2019 19:24:26 +0800 Message-Id: <1568287473-55306-2-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> References: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v5 1/8] config: add WFE config entry for aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the RTE_USE_WFE configuration entry for aarch64, disabled by default. It can be enabled selectively based on the performance benchmarking. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Steve Capper Reviewed-by: Honnappa Nagarahalli Reviewed-by: Phil Yang Acked-by: Pavan Nikhilesh --- config/arm/meson.build | 1 + config/common_base | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/config/arm/meson.build b/config/arm/meson.build index 979018e..18ecd53 100644 --- a/config/arm/meson.build +++ b/config/arm/meson.build @@ -116,6 +116,7 @@ impl_dpaa = ['NXP DPAA', flags_dpaa, machine_args_generic] impl_dpaa2 = ['NXP DPAA2', flags_dpaa2, machine_args_generic] dpdk_conf.set('RTE_FORCE_INTRINSICS', 1) +dpdk_conf.set('RTE_ARM_USE_WFE', 0) if not dpdk_conf.get('RTE_ARCH_64') dpdk_conf.set('RTE_CACHE_LINE_SIZE', 64) diff --git a/config/common_base b/config/common_base index 8ef75c2..d4cf974 100644 --- a/config/common_base +++ b/config/common_base @@ -570,6 +570,12 @@ CONFIG_RTE_CRYPTO_MAX_DEVS=64 CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO=n CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO_DEBUG=n +# Use WFE instructions to implement the rte_wait_for_equal_xxx APIs, +# calling these APIs put the cores in low power state while waiting +# for the memory address to become equal to the expected value. +# This is supported only by aarch64. +CONFIG_RTE_ARM_USE_WFE=n + # # Compile NXP CAAM JR crypto Driver # From patchwork Thu Sep 12 11:24:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59152 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 280821E965; Thu, 12 Sep 2019 13:25:13 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 720EA1E953; Thu, 12 Sep 2019 13:25:10 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 946E028; Thu, 12 Sep 2019 04:25:09 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 913B73F67D; Thu, 12 Sep 2019 04:25:07 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com, stable@dpdk.org Date: Thu, 12 Sep 2019 19:24:27 +0800 Message-Id: <1568287473-55306-3-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> References: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v5 2/8] bus/fslmc: fix the conflicting dmb function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" There are two definitions conflicting each other, for more details, refer to [1]. include/rte_atomic_64.h:19: error: "dmb" redefined [-Werror] drivers/bus/fslmc/mc/fsl_mc_sys.h:36: note: this is the location of the previous definition #define dmb() {__asm__ __volatile__("" : : : "memory"); } The fix is to include the spinlock.h file before the other header files, this is inline with the coding style[2] about the "header includes". The fix changes the function to take the argument for arm to be meaningful. [1] http://inbox.dpdk.org/users/VI1PR08MB537631AB25F41B8880DCCA988FDF0@i VI1PR08MB5376.eurprd08.prod.outlook.com/T/#u [2] https://doc.dpdk.org/guides/contributing/coding_style.html Fixes: 3af733ba8da8 ("bus/fslmc: introduce MC object functions") Cc: stable@dpdk.org Signed-off-by: Gavin Hu Reviewed-by: Phil Yang --- drivers/bus/fslmc/mc/fsl_mc_sys.h | 10 +++++++--- drivers/bus/fslmc/mc/mc_sys.c | 3 +-- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/drivers/bus/fslmc/mc/fsl_mc_sys.h b/drivers/bus/fslmc/mc/fsl_mc_sys.h index d0c7b39..fe9dc95 100644 --- a/drivers/bus/fslmc/mc/fsl_mc_sys.h +++ b/drivers/bus/fslmc/mc/fsl_mc_sys.h @@ -33,10 +33,14 @@ struct fsl_mc_io { #include #ifndef dmb -#define dmb() {__asm__ __volatile__("" : : : "memory"); } +#ifdef RTE_ARCH_ARM64 +#define dmb(opt) {asm volatile("dmb " #opt : : : "memory"); } +#else +#define dmb(opt) #endif -#define __iormb() dmb() -#define __iowmb() dmb() +#endif +#define __iormb() dmb(ld) +#define __iowmb() dmb(st) #define __arch_getq(a) (*(volatile uint64_t *)(a)) #define __arch_putq(v, a) (*(volatile uint64_t *)(a) = (v)) #define __arch_putq32(v, a) (*(volatile uint32_t *)(a) = (v)) diff --git a/drivers/bus/fslmc/mc/mc_sys.c b/drivers/bus/fslmc/mc/mc_sys.c index efafdc3..22143ef 100644 --- a/drivers/bus/fslmc/mc/mc_sys.c +++ b/drivers/bus/fslmc/mc/mc_sys.c @@ -4,11 +4,10 @@ * Copyright 2017 NXP * */ +#include #include #include -#include - /** User space framework uses MC Portal in shared mode. Following change * introduces lock in MC FLIB */ From patchwork Thu Sep 12 11:24:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59153 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AEEC21E97D; Thu, 12 Sep 2019 13:25:15 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 544481E960 for ; Thu, 12 Sep 2019 13:25:12 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BDD1E1597; Thu, 12 Sep 2019 04:25:11 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E99E03F67D; Thu, 12 Sep 2019 04:25:09 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com Date: Thu, 12 Sep 2019 19:24:28 +0800 Message-Id: <1568287473-55306-4-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> References: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v5 3/8] eal: add the APIs to wait until equal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The rte_wait_until_equalxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Steve Capper Reviewed-by: Ola Liljedahl Reviewed-by: Honnappa Nagarahalli Reviewed-by: Phil Yang Acked-by: Pavan Nikhilesh --- .../common/include/arch/arm/rte_pause_64.h | 30 +++++++ lib/librte_eal/common/include/generic/rte_pause.h | 98 +++++++++++++++++++++- 2 files changed, 127 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h index 93895d3..dabde17 100644 --- a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h +++ b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2017 Cavium, Inc + * Copyright(c) 2019 Arm Limited */ #ifndef _RTE_PAUSE_ARM64_H_ @@ -17,6 +18,35 @@ static inline void rte_pause(void) asm volatile("yield" ::: "memory"); } +#ifdef RTE_ARM_USE_WFE +#define __WAIT_UNTIL_EQUAL(name, asm_op, wide, type) \ +static __rte_always_inline void \ +rte_wait_until_equal_##name(volatile type * addr, type expected) \ +{ \ + type tmp; \ + asm volatile( \ + #asm_op " %" #wide "[tmp], %[addr]\n" \ + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ + "b.eq 2f\n" \ + "sevl\n" \ + "1: wfe\n" \ + #asm_op " %" #wide "[tmp], %[addr]\n" \ + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ + "bne 1b\n" \ + "2:\n" \ + : [tmp] "=&r" (tmp) \ + : [addr] "Q"(*addr), [expected] "r"(expected) \ + : "cc", "memory"); \ +} +/* Wait for *addr to be updated with expected value */ +__WAIT_UNTIL_EQUAL(relaxed_16, ldxrh, w, uint16_t) +__WAIT_UNTIL_EQUAL(acquire_16, ldaxrh, w, uint16_t) +__WAIT_UNTIL_EQUAL(relaxed_32, ldxr, w, uint32_t) +__WAIT_UNTIL_EQUAL(acquire_32, ldaxr, w, uint32_t) +__WAIT_UNTIL_EQUAL(relaxed_64, ldxr, x, uint64_t) +__WAIT_UNTIL_EQUAL(acquire_64, ldaxr, x, uint64_t) +#endif + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/common/include/generic/rte_pause.h b/lib/librte_eal/common/include/generic/rte_pause.h index 52bd4db..dfa6a53 100644 --- a/lib/librte_eal/common/include/generic/rte_pause.h +++ b/lib/librte_eal/common/include/generic/rte_pause.h @@ -1,10 +1,10 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2017 Cavium, Inc + * Copyright(c) 2019 Arm Limited */ #ifndef _RTE_PAUSE_H_ #define _RTE_PAUSE_H_ - /** * @file * @@ -12,6 +12,10 @@ * */ +#include +#include +#include + /** * Pause CPU execution for a short while * @@ -20,4 +24,96 @@ */ static inline void rte_pause(void); +/** + * Wait for *addr to be updated with expected value + * + * @param addr + * A pointer to the memory location. + * @param expected + * An expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_relaxed_16(volatile uint16_t *addr, uint16_t expected); + +/** + * Wait for *addr to be updated with expected value + * + * @param addr + * A pointer to the memory location. + * @param expected + * An expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_relaxed_32(volatile uint32_t *addr, uint32_t expected); + +/** + * Wait for *addr to be updated with expected value + * + * @param addr + * A pointer to the memory location. + * @param expected + * An expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_relaxed_64(volatile uint64_t *addr, uint64_t expected); + +/** + * Wait for *addr to be updated with expected value + * + * @param addr + * A pointer to the memory location. + * @param expected + * An expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_acquire_16(volatile uint16_t *addr, uint16_t expected); + +/** + * Wait for *addr to be updated with expected value + * + * @param addr + * A pointer to the memory location. + * @param expected + * An expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_acquire_32(volatile uint32_t *addr, uint32_t expected); + +/** + * Wait for *addr to be updated with expected value + * + * @param addr + * A pointer to the memory location. + * @param expected + * An expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_acquire_64(volatile uint64_t *addr, uint64_t expected); + +#if !defined(RTE_ARM_USE_WFE) +#define __WAIT_UNTIL_EQUAL(op_name, size, type, memorder) \ +__rte_always_inline \ +static void \ +rte_wait_until_equal_##op_name##_##size(volatile type *addr, \ + type expected) \ +{ \ + while (__atomic_load_n(addr, memorder) != expected) \ + rte_pause(); \ +} + +/* Wait for *addr to be updated with expected value */ +__WAIT_UNTIL_EQUAL(relaxed, 16, uint16_t, __ATOMIC_RELAXED) +__WAIT_UNTIL_EQUAL(acquire, 16, uint16_t, __ATOMIC_ACQUIRE) +__WAIT_UNTIL_EQUAL(relaxed, 32, uint32_t, __ATOMIC_RELAXED) +__WAIT_UNTIL_EQUAL(acquire, 32, uint32_t, __ATOMIC_ACQUIRE) +__WAIT_UNTIL_EQUAL(relaxed, 64, uint64_t, __ATOMIC_RELAXED) +__WAIT_UNTIL_EQUAL(acquire, 64, uint64_t, __ATOMIC_ACQUIRE) +#endif /* RTE_ARM_USE_WFE */ + #endif /* _RTE_PAUSE_H_ */ From patchwork Thu Sep 12 11:24:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59154 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 961511E99D; Thu, 12 Sep 2019 13:25:17 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 6A12E1E96C for ; Thu, 12 Sep 2019 13:25:14 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id ECB6128; Thu, 12 Sep 2019 04:25:13 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 1F1D53F67D; Thu, 12 Sep 2019 04:25:11 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com Date: Thu, 12 Sep 2019 19:24:29 +0800 Message-Id: <1568287473-55306-5-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> References: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v5 4/8] spinlock: use wfe to reduce contention on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In acquiring a spinlock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API. Running the micro benchmarking and the testpmd and l3fwd traffic tests on ThunderX2, Ampere eMAG80 and Arm N1SDP, everything went well and no notable performance gain nor degradation was measured. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Phil Yang Reviewed-by: Steve Capper Reviewed-by: Ola Liljedahl Reviewed-by: Honnappa Nagarahalli Tested-by: Pavan Nikhilesh --- .../common/include/arch/arm/rte_spinlock.h | 26 ++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/lib/librte_eal/common/include/arch/arm/rte_spinlock.h b/lib/librte_eal/common/include/arch/arm/rte_spinlock.h index 1a6916b..b61c055 100644 --- a/lib/librte_eal/common/include/arch/arm/rte_spinlock.h +++ b/lib/librte_eal/common/include/arch/arm/rte_spinlock.h @@ -16,6 +16,32 @@ extern "C" { #include #include "generic/rte_spinlock.h" +/* armv7a does support WFE, but an explicit wake-up signal using SEV is + * required (must be preceded by DSB to drain the store buffer) and + * this is less performant, so keep armv7a implementation unchanged. + */ +#ifndef RTE_FORCE_INTRINSICS +static inline void +rte_spinlock_lock(rte_spinlock_t *sl) +{ + unsigned int tmp; + /* http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc. + * faqs/ka16809.html + */ + asm volatile( + "1: ldaxr %w[tmp], %w[locked]\n" + "cbnz %w[tmp], 2f\n" + "stxr %w[tmp], %w[one], %w[locked]\n" + "cbnz %w[tmp], 1b\n" + "ret\n" + "2: sevl\n" + "wfe\n" + "jmp 1b\n" + : [tmp] "=&r" (tmp), [locked] "+Q"(sl->locked) + : [one] "r" (1) +} +#endif + static inline int rte_tm_supported(void) { return 0; From patchwork Thu Sep 12 11:24:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59155 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 95B261E9A3; Thu, 12 Sep 2019 13:25:19 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 9672E1E986 for ; Thu, 12 Sep 2019 13:25:16 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 24FFB1597; Thu, 12 Sep 2019 04:25:16 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4F1A03F67D; Thu, 12 Sep 2019 04:25:14 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com Date: Thu, 12 Sep 2019 19:24:30 +0800 Message-Id: <1568287473-55306-6-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> References: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v5 5/8] ticketlock: use new API to reduce contention on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" While using ticket lock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API. Running ticketlock_autotest on ThunderX2, Ampere eMAG80, and Arm N1SDP[1], there were variances between runs, but no notable performance gain or degradation were seen with and without this patch. [1] https://community.arm.com/developer/tools-software/oss-platforms/w/\ docs/440/neoverse-n1-sdp Signed-off-by: Gavin Hu Reviewed-by: Honnappa Nagarahalli Tested-by: Phil Yang Tested-by: Pavan Nikhilesh Reviewed-by: Jerin Jacob --- lib/librte_eal/common/include/generic/rte_ticketlock.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/generic/rte_ticketlock.h b/lib/librte_eal/common/include/generic/rte_ticketlock.h index d9bec87..232bbe9 100644 --- a/lib/librte_eal/common/include/generic/rte_ticketlock.h +++ b/lib/librte_eal/common/include/generic/rte_ticketlock.h @@ -66,8 +66,7 @@ static inline void rte_ticketlock_lock(rte_ticketlock_t *tl) { uint16_t me = __atomic_fetch_add(&tl->s.next, 1, __ATOMIC_RELAXED); - while (__atomic_load_n(&tl->s.current, __ATOMIC_ACQUIRE) != me) - rte_pause(); + rte_wait_until_equal_acquire_16(&tl->s.current, me); } /** From patchwork Thu Sep 12 11:24:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59156 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 875DC1E9AB; Thu, 12 Sep 2019 13:25:21 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id C79C91E9A1 for ; Thu, 12 Sep 2019 13:25:18 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5004828; Thu, 12 Sep 2019 04:25:18 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7CB763F67D; Thu, 12 Sep 2019 04:25:16 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com Date: Thu, 12 Sep 2019 19:24:31 +0800 Message-Id: <1568287473-55306-7-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> References: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v5 6/8] ring: use wfe to wait for ring tail update on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instead of polling for tail to be updated, use wfe instruction. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Steve Capper Reviewed-by: Ola Liljedahl Reviewed-by: Honnappa Nagarahalli --- lib/librte_ring/rte_ring_c11_mem.h | 4 ++-- lib/librte_ring/rte_ring_generic.h | 3 +-- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/lib/librte_ring/rte_ring_c11_mem.h b/lib/librte_ring/rte_ring_c11_mem.h index 0fb73a3..764d8f1 100644 --- a/lib/librte_ring/rte_ring_c11_mem.h +++ b/lib/librte_ring/rte_ring_c11_mem.h @@ -2,6 +2,7 @@ * * Copyright (c) 2017,2018 HXT-semitech Corporation. * Copyright (c) 2007-2009 Kip Macy kmacy@freebsd.org + * Copyright (c) 2019 Arm Limited * All rights reserved. * Derived from FreeBSD's bufring.h * Used as BSD-3 Licensed with permission from Kip Macy. @@ -21,8 +22,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val, * we need to wait for them to complete */ if (!single) - while (unlikely(ht->tail != old_val)) - rte_pause(); + rte_wait_until_equal_relaxed_32(&ht->tail, old_val); __atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE); } diff --git a/lib/librte_ring/rte_ring_generic.h b/lib/librte_ring/rte_ring_generic.h index 953cdbb..6828527 100644 --- a/lib/librte_ring/rte_ring_generic.h +++ b/lib/librte_ring/rte_ring_generic.h @@ -23,8 +23,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val, * we need to wait for them to complete */ if (!single) - while (unlikely(ht->tail != old_val)) - rte_pause(); + rte_wait_until_equal_relaxed_32(&ht->tail, old_val); ht->tail = new_val; } From patchwork Thu Sep 12 11:24:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59157 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 975801E9B7; Thu, 12 Sep 2019 13:25:23 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id EBCC61E9A8 for ; Thu, 12 Sep 2019 13:25:20 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 801D81597; Thu, 12 Sep 2019 04:25:20 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A53853F67D; Thu, 12 Sep 2019 04:25:18 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com Date: Thu, 12 Sep 2019 19:24:32 +0800 Message-Id: <1568287473-55306-8-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> References: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v5 7/8] net/thunderx: use new API to save cycles on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Use the new API to wait in low power state instead of continuous polling to save CPU cycles and power. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Acked-by: Jerin Jacob --- drivers/net/thunderx/nicvf_rxtx.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/net/thunderx/nicvf_rxtx.c b/drivers/net/thunderx/nicvf_rxtx.c index 1c42874..16f34c6 100644 --- a/drivers/net/thunderx/nicvf_rxtx.c +++ b/drivers/net/thunderx/nicvf_rxtx.c @@ -385,8 +385,7 @@ nicvf_fill_rbdr(struct nicvf_rxq *rxq, int to_fill) ltail++; } - while (__atomic_load_n(&rbdr->tail, __ATOMIC_RELAXED) != next_tail) - rte_pause(); + rte_wait_until_equal_relaxed_32(&rbdr->tail, next_tail); __atomic_store_n(&rbdr->tail, ltail, __ATOMIC_RELEASE); nicvf_addr_write(door, to_fill); From patchwork Thu Sep 12 11:24:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59158 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5F02C1E9C3; Thu, 12 Sep 2019 13:25:25 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 2E7A71E9B4 for ; Thu, 12 Sep 2019 13:25:23 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BD2FE28; Thu, 12 Sep 2019 04:25:22 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D78073F67D; Thu, 12 Sep 2019 04:25:20 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com Date: Thu, 12 Sep 2019 19:24:33 +0800 Message-Id: <1568287473-55306-9-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> References: <1568287473-55306-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v5 8/8] event/opdl: use new API to save cycles on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Use the new API to wait in low power state instead of continuous polling to save CPU cycles and power. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Jerin Jacob --- drivers/event/opdl/opdl_ring.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/event/opdl/opdl_ring.c b/drivers/event/opdl/opdl_ring.c index e8b29e2..f446fa3 100644 --- a/drivers/event/opdl/opdl_ring.c +++ b/drivers/event/opdl/opdl_ring.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "opdl_ring.h" #include "opdl_log.h" @@ -475,9 +476,7 @@ opdl_ring_input_multithread(struct opdl_ring *t, const void *entries, /* If another thread started inputting before this one, but hasn't * finished, we need to wait for it to complete to update the tail. */ - while (unlikely(__atomic_load_n(&s->shared.tail, __ATOMIC_ACQUIRE) != - old_head)) - rte_pause(); + rte_wait_until_equal_acquire_32(&s->shared.tail, old_head); __atomic_store_n(&s->shared.tail, old_head + num_entries, __ATOMIC_RELEASE);