From patchwork Thu Nov 7 21:35:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 62704 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E1BB1A04AB; Thu, 7 Nov 2019 22:35:52 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EE3041BFBD; Thu, 7 Nov 2019 22:35:45 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id C360D1BFB2 for ; Thu, 7 Nov 2019 22:35:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573162544; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0QTTLLULoPUD5uBF0D4jQoFiBTCejxKAeEAatZtLQ+w=; b=Pxhrt7IJBezenxRBCMneSE9UYZ//AHUnQCPwjvrUqD4/rM2Jj7d78rA7kGNa1d0oNZQRlW UW32ZWvBVoDBAS+nLAnPMHmt9jRllJ/gYbHEVNxAFBVBKGdMadad817cKAPgm3QGlyFElw G8makF0clu931aVh4LDeDsuBN+OLSY4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-53-6Eo6QFTnPN-V4T9cJCIpqw-1; Thu, 07 Nov 2019 16:35:40 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 04EE01800D6B; Thu, 7 Nov 2019 21:35:39 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-222.brq.redhat.com [10.40.204.222]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0EE54608B2; Thu, 7 Nov 2019 21:35:36 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: nd@arm.com, konstantin.ananyev@intel.com, Gavin Hu , stable@dpdk.org, Hemant Agrawal , Sachin Saxena Date: Thu, 7 Nov 2019 22:35:24 +0100 Message-Id: <1573162528-16230-2-git-send-email-david.marchand@redhat.com> In-Reply-To: <1573162528-16230-1-git-send-email-david.marchand@redhat.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1573162528-16230-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: 6Eo6QFTnPN-V4T9cJCIpqw-1 X-Mimecast-Spam-Score: 0 Subject: [dpdk-dev] [PATCH v13 1/5] bus/fslmc: fix the conflicting dmb function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Gavin Hu There are two definitions conflicting each other, for more details, refer to [1]. include/rte_atomic_64.h:19: error: "dmb" redefined [-Werror] drivers/bus/fslmc/mc/fsl_mc_sys.h:36: note: this is the location of the previous definition #define dmb() {__asm__ __volatile__("" : : : "memory"); } The fix is to reuse the EAL definition to avoid conflicts. [1] http://inbox.dpdk.org/users/VI1PR08MB537631AB25F41B8880DCCA988FDF0@ VI1PR08MB5376.eurprd08.prod.outlook.com/T/#u Fixes: 3af733ba8da8 ("bus/fslmc: introduce MC object functions") Cc: stable@dpdk.org Signed-off-by: Gavin Hu Reviewed-by: Phil Yang Acked-by: Hemant Agrawal --- Changelog since v12: - fixed Phil Yang mail address, --- drivers/bus/fslmc/mc/fsl_mc_sys.h | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/drivers/bus/fslmc/mc/fsl_mc_sys.h b/drivers/bus/fslmc/mc/fsl_mc_sys.h index d0c7b39..a310c56 100644 --- a/drivers/bus/fslmc/mc/fsl_mc_sys.h +++ b/drivers/bus/fslmc/mc/fsl_mc_sys.h @@ -32,11 +32,10 @@ struct fsl_mc_io { #include #include -#ifndef dmb -#define dmb() {__asm__ __volatile__("" : : : "memory"); } -#endif -#define __iormb() dmb() -#define __iowmb() dmb() +#include + +#define __iormb() rte_io_rmb() +#define __iowmb() rte_io_wmb() #define __arch_getq(a) (*(volatile uint64_t *)(a)) #define __arch_putq(v, a) (*(volatile uint64_t *)(a) = (v)) #define __arch_putq32(v, a) (*(volatile uint32_t *)(a) = (v)) From patchwork Thu Nov 7 21:35:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 62705 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4AD60A04AB; Thu, 7 Nov 2019 22:36:01 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9364C1BFD2; Thu, 7 Nov 2019 22:35:50 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by dpdk.org (Postfix) with ESMTP id 25AC81BFCD for ; Thu, 7 Nov 2019 22:35:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573162547; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+08drIEGy16nzp5c3Qu5t96IMzAq+cBKnVtgiLXQlkM=; b=UnheV/oPVMkBjx1pZ4KhI++TT5Ck+YJeLi2AVUPDx7ihwoIMeOrSLoRwgw9+MvRF4tflgz 5Jdah6Ovajcz6hjNqgQPja4MJ/IpdQ8ITRx472lvdC8MmwRaEHQd/OXGFE+mgo5FlWSRfS eaClKvbiiikmfO55X2UxVXp5q/NWDis= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-336-LwF417vvP8mZF1aYIjq1-A-1; Thu, 07 Nov 2019 16:35:44 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 99F87107ACC3; Thu, 7 Nov 2019 21:35:42 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-222.brq.redhat.com [10.40.204.222]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8AE74608B2; Thu, 7 Nov 2019 21:35:39 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: nd@arm.com, konstantin.ananyev@intel.com, Gavin Hu , Thomas Monjalon , John McNamara , Marko Kovacevic , Jerin Jacob , Jan Viktorin Date: Thu, 7 Nov 2019 22:35:25 +0100 Message-Id: <1573162528-16230-3-git-send-email-david.marchand@redhat.com> In-Reply-To: <1573162528-16230-1-git-send-email-david.marchand@redhat.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1573162528-16230-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: LwF417vvP8mZF1aYIjq1-A-1 X-Mimecast-Spam-Score: 0 Subject: [dpdk-dev] [PATCH v13 2/5] eal: add the APIs to wait until equal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Gavin Hu The rte_wait_until_equal_xx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'. Add the RTE_ARM_USE_WFE configuration entry for aarch64, disabled by default. When it is enabled, the above APIs will call WFE instruction to save CPU cycles and power. From a VM, when calling this API on aarch64, it may trap in and out to release vCPUs whereas cause high exit latency. Since kernel 4.18.20 an adaptive trapping mechanism is introduced to balance the latency and workload. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Steve Capper Reviewed-by: Ola Liljedahl Reviewed-by: Honnappa Nagarahalli Reviewed-by: Phil Yang Acked-by: Pavan Nikhilesh Acked-by: Jerin Jacob Acked-by: Konstantin Ananyev Signed-off-by: David Marchand --- Changelog since v12: - added release notes update, - fixed function prototypes indent, - reimplemented the arm implementation without exposing internal inline functions, - added asserts in generic implementation, --- config/arm/meson.build | 1 + config/common_base | 5 + doc/guides/rel_notes/release_19_11.rst | 5 + .../common/include/arch/arm/rte_pause_64.h | 133 +++++++++++++++++++++ lib/librte_eal/common/include/generic/rte_pause.h | 105 ++++++++++++++++ 5 files changed, 249 insertions(+) diff --git a/config/arm/meson.build b/config/arm/meson.build index 46dff3a..ea47425 100644 --- a/config/arm/meson.build +++ b/config/arm/meson.build @@ -26,6 +26,7 @@ flags_common_default = [ ['RTE_LIBRTE_AVP_PMD', false], ['RTE_SCHED_VECTOR', false], + ['RTE_ARM_USE_WFE', false], ] flags_generic = [ diff --git a/config/common_base b/config/common_base index 1858598..bb1b1ed 100644 --- a/config/common_base +++ b/config/common_base @@ -110,6 +110,11 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=64 CONFIG_RTE_MALLOC_DEBUG=n CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n CONFIG_RTE_USE_LIBBSD=n +# Use WFE instructions to implement the rte_wait_for_equal_xxx APIs, +# calling these APIs put the cores in low power state while waiting +# for the memory address to become equal to the expected value. +# This is supported only by aarch64. +CONFIG_RTE_ARM_USE_WFE=n # # Recognize/ignore the AVX/AVX512 CPU flags for performance/power testing. diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst index fe11b4b..af5f2c5 100644 --- a/doc/guides/rel_notes/release_19_11.rst +++ b/doc/guides/rel_notes/release_19_11.rst @@ -65,6 +65,11 @@ New Features The lock-free stack implementation is enabled for aarch64 platforms. +* **Added Wait Until Equal API.** + + A new API has been added to wait for a memory location to be updated with a + 16-bit, 32-bit, 64-bit value. + * **Changed mempool allocation behaviour.** Objects are no longer across pages by default. diff --git a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h index 93895d3..e87d10b 100644 --- a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h +++ b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2017 Cavium, Inc + * Copyright(c) 2019 Arm Limited */ #ifndef _RTE_PAUSE_ARM64_H_ @@ -10,6 +11,11 @@ extern "C" { #endif #include + +#ifdef RTE_ARM_USE_WFE +#define RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED +#endif + #include "generic/rte_pause.h" static inline void rte_pause(void) @@ -17,6 +23,133 @@ static inline void rte_pause(void) asm volatile("yield" ::: "memory"); } +#ifdef RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED + +/* Send an event to quit WFE. */ +#define __SEVL() { asm volatile("sevl" : : : "memory"); } + +/* Put processor into low power WFE(Wait For Event) state. */ +#define __WFE() { asm volatile("wfe" : : : "memory"); } + +static __rte_always_inline void +rte_wait_until_equal_16(volatile uint16_t *addr, uint16_t expected, + int memorder) +{ + uint16_t value; + + assert(memorder == __ATOMIC_ACQUIRE || memorder == __ATOMIC_RELAXED); + + /* + * Atomic exclusive load from addr, it returns the 16-bit content of + * *addr while making it 'monitored',when it is written by someone + * else, the 'monitored' state is cleared and a event is generated + * implicitly to exit WFE. + */ +#define __LOAD_EXC_16(src, dst, memorder) { \ + if (memorder == __ATOMIC_RELAXED) { \ + asm volatile("ldxrh %w[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r"(src) \ + : "memory"); \ + } else { \ + asm volatile("ldaxrh %w[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r"(src) \ + : "memory"); \ + } } + + __LOAD_EXC_16(addr, value, memorder) + if (value != expected) { + __SEVL() + do { + __WFE() + __LOAD_EXC_16(addr, value, memorder) + } while (value != expected); + } +#undef __LOAD_EXC_16 +} + +static __rte_always_inline void +rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, + int memorder) +{ + uint32_t value; + + assert(memorder == __ATOMIC_ACQUIRE || memorder == __ATOMIC_RELAXED); + + /* + * Atomic exclusive load from addr, it returns the 32-bit content of + * *addr while making it 'monitored',when it is written by someone + * else, the 'monitored' state is cleared and a event is generated + * implicitly to exit WFE. + */ +#define __LOAD_EXC_32(src, dst, memorder) { \ + if (memorder == __ATOMIC_RELAXED) { \ + asm volatile("ldxr %w[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r"(src) \ + : "memory"); \ + } else { \ + asm volatile("ldaxr %w[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r"(src) \ + : "memory"); \ + } } + + __LOAD_EXC_32(addr, value, memorder) + if (value != expected) { + __SEVL() + do { + __WFE() + __LOAD_EXC_32(addr, value, memorder) + } while (value != expected); + } +#undef __LOAD_EXC_32 +} + +static __rte_always_inline void +rte_wait_until_equal_64(volatile uint64_t *addr, uint64_t expected, + int memorder) +{ + uint64_t value; + + assert(memorder == __ATOMIC_ACQUIRE || memorder == __ATOMIC_RELAXED); + + /* + * Atomic exclusive load from addr, it returns the 64-bit content of + * *addr while making it 'monitored',when it is written by someone + * else, the 'monitored' state is cleared and a event is generated + * implicitly to exit WFE. + */ +#define __LOAD_EXC_64(src, dst, memorder) { \ + if (memorder == __ATOMIC_RELAXED) { \ + asm volatile("ldxr %x[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r"(src) \ + : "memory"); \ + } else { \ + asm volatile("ldaxr %x[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r"(src) \ + : "memory"); \ + } } + + __LOAD_EXC_64(addr, value, memorder) + if (value != expected) { + __SEVL() + do { + __WFE() + __LOAD_EXC_64(addr, value, memorder) + } while (value != expected); + } +} +#undef __LOAD_EXC_64 + +#undef __SEVL +#undef __WFE + +#endif + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/common/include/generic/rte_pause.h b/lib/librte_eal/common/include/generic/rte_pause.h index 52bd4db..7422785 100644 --- a/lib/librte_eal/common/include/generic/rte_pause.h +++ b/lib/librte_eal/common/include/generic/rte_pause.h @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2017 Cavium, Inc + * Copyright(c) 2019 Arm Limited */ #ifndef _RTE_PAUSE_H_ @@ -12,6 +13,12 @@ * */ +#include +#include +#include +#include +#include + /** * Pause CPU execution for a short while * @@ -20,4 +27,102 @@ */ static inline void rte_pause(void); +/** + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * Wait for *addr to be updated with a 16-bit expected value, with a relaxed + * memory ordering model meaning the loads around this API can be reordered. + * + * @param addr + * A pointer to the memory location. + * @param expected + * A 16-bit expected value to be in the memory location. + * @param memorder + * Two different memory orders that can be specified: + * __ATOMIC_ACQUIRE and __ATOMIC_RELAXED. These map to + * C++11 memory orders with the same names, see the C++11 standard or + * the GCC wiki on atomic synchronization for detailed definition. + */ +__rte_experimental +static __rte_always_inline void +rte_wait_until_equal_16(volatile uint16_t *addr, uint16_t expected, + int memorder); + +/** + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * Wait for *addr to be updated with a 32-bit expected value, with a relaxed + * memory ordering model meaning the loads around this API can be reordered. + * + * @param addr + * A pointer to the memory location. + * @param expected + * A 32-bit expected value to be in the memory location. + * @param memorder + * Two different memory orders that can be specified: + * __ATOMIC_ACQUIRE and __ATOMIC_RELAXED. These map to + * C++11 memory orders with the same names, see the C++11 standard or + * the GCC wiki on atomic synchronization for detailed definition. + */ +__rte_experimental +static __rte_always_inline void +rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, + int memorder); + +/** + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice + * + * Wait for *addr to be updated with a 64-bit expected value, with a relaxed + * memory ordering model meaning the loads around this API can be reordered. + * + * @param addr + * A pointer to the memory location. + * @param expected + * A 64-bit expected value to be in the memory location. + * @param memorder + * Two different memory orders that can be specified: + * __ATOMIC_ACQUIRE and __ATOMIC_RELAXED. These map to + * C++11 memory orders with the same names, see the C++11 standard or + * the GCC wiki on atomic synchronization for detailed definition. + */ +__rte_experimental +static __rte_always_inline void +rte_wait_until_equal_64(volatile uint64_t *addr, uint64_t expected, + int memorder); + +#ifndef RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED +static __rte_always_inline void +rte_wait_until_equal_16(volatile uint16_t *addr, uint16_t expected, + int memorder) +{ + assert(memorder == __ATOMIC_ACQUIRE || memorder == __ATOMIC_RELAXED); + + while (__atomic_load_n(addr, memorder) != expected) + rte_pause(); +} + +static __rte_always_inline void +rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, + int memorder) +{ + assert(memorder == __ATOMIC_ACQUIRE || memorder == __ATOMIC_RELAXED); + + while (__atomic_load_n(addr, memorder) != expected) + rte_pause(); +} + +static __rte_always_inline void +rte_wait_until_equal_64(volatile uint64_t *addr, uint64_t expected, + int memorder) +{ + assert(memorder == __ATOMIC_ACQUIRE || memorder == __ATOMIC_RELAXED); + + while (__atomic_load_n(addr, memorder) != expected) + rte_pause(); +} +#endif + #endif /* _RTE_PAUSE_H_ */ From patchwork Thu Nov 7 21:35:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 62706 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 62F28A04AB; Thu, 7 Nov 2019 22:36:11 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 969C11BFF4; Thu, 7 Nov 2019 22:35:53 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by dpdk.org (Postfix) with ESMTP id 184681BFE2 for ; Thu, 7 Nov 2019 22:35:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573162550; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7FcBahTFoLUWe7kYcAz2Bkf5qFTqB8bzGqgv9/zW4F4=; b=EkGVsCSJ9X5Atn52q0h3VkYd1VNYA6U5pyVNRpUE0YkJJuzh3p9txLBYXgv9ZYk22KZfk7 CHB1zaJXCHO/3UpI+QdEr2gigrVTWTpLLSjtyK0hn6/Od6JC2aA2HNd7rUy7LuEnStFs63 n0aAyvn8X/YDElKYqIu0h68+LlU4UAQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-150-YEu60hSLMMSohNpRQVyI2Q-1; Thu, 07 Nov 2019 16:35:46 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1CD731005500; Thu, 7 Nov 2019 21:35:45 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-222.brq.redhat.com [10.40.204.222]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2E008608B2; Thu, 7 Nov 2019 21:35:42 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: nd@arm.com, konstantin.ananyev@intel.com, Gavin Hu , Joyce Kong Date: Thu, 7 Nov 2019 22:35:26 +0100 Message-Id: <1573162528-16230-4-git-send-email-david.marchand@redhat.com> In-Reply-To: <1573162528-16230-1-git-send-email-david.marchand@redhat.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1573162528-16230-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: YEu60hSLMMSohNpRQVyI2Q-1 X-Mimecast-Spam-Score: 0 Subject: [dpdk-dev] [PATCH v13 3/5] ticketlock: use new API to reduce contention on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Gavin Hu While using ticket lock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API. Running ticketlock_autotest on ThunderX2, Ampere eMAG80, and Arm N1SDP[1], there were variances between runs, but no notable performance gain or degradation were seen with and without this patch. [1] https://community.arm.com/developer/tools-software/oss-platforms/w/\ docs/440/neoverse-n1-sdp Signed-off-by: Gavin Hu Reviewed-by: Honnappa Nagarahalli Tested-by: Phil Yang Tested-by: Pavan Nikhilesh Reviewed-by: Jerin Jacob --- lib/librte_eal/common/include/generic/rte_ticketlock.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/generic/rte_ticketlock.h b/lib/librte_eal/common/include/generic/rte_ticketlock.h index d9bec87..c295ae7 100644 --- a/lib/librte_eal/common/include/generic/rte_ticketlock.h +++ b/lib/librte_eal/common/include/generic/rte_ticketlock.h @@ -66,8 +66,7 @@ static inline void rte_ticketlock_lock(rte_ticketlock_t *tl) { uint16_t me = __atomic_fetch_add(&tl->s.next, 1, __ATOMIC_RELAXED); - while (__atomic_load_n(&tl->s.current, __ATOMIC_ACQUIRE) != me) - rte_pause(); + rte_wait_until_equal_16(&tl->s.current, me, __ATOMIC_ACQUIRE); } /** From patchwork Thu Nov 7 21:35:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 62707 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 84D67A04AB; Thu, 7 Nov 2019 22:36:18 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 26F9A1BFF6; Thu, 7 Nov 2019 22:35:56 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 3400E1BFF2 for ; Thu, 7 Nov 2019 22:35:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573162552; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KOAOZi4lMjuf+IP8ZYJgH8H6nN7b4EPrB/0FRiULjEU=; b=Lw6jXI3d62czkByeDyWOXAnXumWfiUo6YKnduxv6UhBOb+zvOmGuddUVhdcO7/LyrC+KYz HDQoOz6i3vI826Hk3b6QPJwHuVXGcR4eVRvKqdolyFvbUiY+e7YbZlauWpoKO+r+uIwPC6 BO4sqoYifHcvJIWt8hB8wBojVp3FZz0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-375-3YkZY8mlMKK9ZjaUMQKFrA-1; Thu, 07 Nov 2019 16:35:48 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B3A12107ACC3; Thu, 7 Nov 2019 21:35:47 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-222.brq.redhat.com [10.40.204.222]) by smtp.corp.redhat.com (Postfix) with ESMTP id 96DE1608B2; Thu, 7 Nov 2019 21:35:45 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: nd@arm.com, konstantin.ananyev@intel.com, Gavin Hu , Jerin Jacob , Maciej Czekaj Date: Thu, 7 Nov 2019 22:35:27 +0100 Message-Id: <1573162528-16230-5-git-send-email-david.marchand@redhat.com> In-Reply-To: <1573162528-16230-1-git-send-email-david.marchand@redhat.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1573162528-16230-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: 3YkZY8mlMKK9ZjaUMQKFrA-1 X-Mimecast-Spam-Score: 0 Subject: [dpdk-dev] [PATCH v13 4/5] net/thunderx: use new API to save cycles on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Gavin Hu Use the new API to wait in low power state instead of continuous polling to save CPU cycles and power. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Acked-by: Jerin Jacob --- drivers/net/thunderx/Makefile | 1 + drivers/net/thunderx/meson.build | 1 + drivers/net/thunderx/nicvf_rxtx.c | 3 +-- 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/net/thunderx/Makefile b/drivers/net/thunderx/Makefile index e6bf497..9e0de10 100644 --- a/drivers/net/thunderx/Makefile +++ b/drivers/net/thunderx/Makefile @@ -10,6 +10,7 @@ include $(RTE_SDK)/mk/rte.vars.mk LIB = librte_pmd_thunderx_nicvf.a CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API LDLIBS += -lm LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring diff --git a/drivers/net/thunderx/meson.build b/drivers/net/thunderx/meson.build index 69819a9..23d9458 100644 --- a/drivers/net/thunderx/meson.build +++ b/drivers/net/thunderx/meson.build @@ -4,6 +4,7 @@ subdir('base') objs = [base_objs] +allow_experimental_apis = true sources = files('nicvf_rxtx.c', 'nicvf_ethdev.c', 'nicvf_svf.c' diff --git a/drivers/net/thunderx/nicvf_rxtx.c b/drivers/net/thunderx/nicvf_rxtx.c index 1c42874..90a6098 100644 --- a/drivers/net/thunderx/nicvf_rxtx.c +++ b/drivers/net/thunderx/nicvf_rxtx.c @@ -385,8 +385,7 @@ nicvf_fill_rbdr(struct nicvf_rxq *rxq, int to_fill) ltail++; } - while (__atomic_load_n(&rbdr->tail, __ATOMIC_RELAXED) != next_tail) - rte_pause(); + rte_wait_until_equal_32(&rbdr->tail, next_tail, __ATOMIC_RELAXED); __atomic_store_n(&rbdr->tail, ltail, __ATOMIC_RELEASE); nicvf_addr_write(door, to_fill); From patchwork Thu Nov 7 21:35:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 62708 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3FADAA04AB; Thu, 7 Nov 2019 22:36:26 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5EB8D1BFFC; Thu, 7 Nov 2019 22:35:59 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id 398151BFBA for ; Thu, 7 Nov 2019 22:35:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573162556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D95qJCr/G32cXrBouwFbG4N5qt/9jq+el241nA/D474=; b=ATuT+TYFHApdVQS+W46OHsuMrOPtuCxeriTW8P/MEBC7+kpwgcT7mR4NVo5ozDZvpIr3e8 HRL/lvSj06TlVedr3z3tT97n4n3vVJvVFMnys6CPV5x3SE2BCaGQ2s5T/dO5e4/xQbanNi WYyVQH09rlJjZSuOHegFick4brNc6Ak= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-61-QrqafbPBPyC91kiuzNtuWA-1; Thu, 07 Nov 2019 16:35:52 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1D14C477; Thu, 7 Nov 2019 21:35:51 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-222.brq.redhat.com [10.40.204.222]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B685608B3; Thu, 7 Nov 2019 21:35:48 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: nd@arm.com, konstantin.ananyev@intel.com, Gavin Hu , Liang Ma , Peter Mccarthy Date: Thu, 7 Nov 2019 22:35:28 +0100 Message-Id: <1573162528-16230-6-git-send-email-david.marchand@redhat.com> In-Reply-To: <1573162528-16230-1-git-send-email-david.marchand@redhat.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1573162528-16230-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: QrqafbPBPyC91kiuzNtuWA-1 X-Mimecast-Spam-Score: 0 Subject: [dpdk-dev] [PATCH v13 5/5] event/opdl: use new API to save cycles on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Gavin Hu Use the new API to wait in low power state instead of continuous polling to save CPU cycles and power. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Jerin Jacob --- drivers/event/opdl/Makefile | 1 + drivers/event/opdl/meson.build | 1 + drivers/event/opdl/opdl_ring.c | 5 ++--- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/event/opdl/Makefile b/drivers/event/opdl/Makefile index bf50a60..72ef07d 100644 --- a/drivers/event/opdl/Makefile +++ b/drivers/event/opdl/Makefile @@ -9,6 +9,7 @@ LIB = librte_pmd_opdl_event.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API # for older GCC versions, allow us to initialize an event using # designated initializers. ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) diff --git a/drivers/event/opdl/meson.build b/drivers/event/opdl/meson.build index 1fe034e..e67b164 100644 --- a/drivers/event/opdl/meson.build +++ b/drivers/event/opdl/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi +allow_experimental_apis = true sources = files( 'opdl_evdev.c', 'opdl_evdev_init.c', diff --git a/drivers/event/opdl/opdl_ring.c b/drivers/event/opdl/opdl_ring.c index 06fb5b3..c8d19fe 100644 --- a/drivers/event/opdl/opdl_ring.c +++ b/drivers/event/opdl/opdl_ring.c @@ -16,6 +16,7 @@ #include #include #include +#include #include "opdl_ring.h" #include "opdl_log.h" @@ -474,9 +475,7 @@ opdl_ring_input_multithread(struct opdl_ring *t, const void *entries, /* If another thread started inputting before this one, but hasn't * finished, we need to wait for it to complete to update the tail. */ - while (unlikely(__atomic_load_n(&s->shared.tail, __ATOMIC_ACQUIRE) != - old_head)) - rte_pause(); + rte_wait_until_equal_32(&s->shared.tail, old_head, __ATOMIC_ACQUIRE); __atomic_store_n(&s->shared.tail, old_head + num_entries, __ATOMIC_RELEASE);