From patchwork Mon Nov 1 06:00:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 103288 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A168EA0C52; Mon, 1 Nov 2021 07:00:24 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F1154410FF; Mon, 1 Nov 2021 07:00:21 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 39594410FE for ; Mon, 1 Nov 2021 07:00:20 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 97B85D6E; Sun, 31 Oct 2021 23:00:19 -0700 (PDT) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.102]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7A3303F70D; Sun, 31 Oct 2021 23:00:16 -0700 (PDT) From: Feifei Wang To: Ruifeng Wang Cc: dev@dpdk.org, nd@arm.com, jerinjacobk@gmail.com, stephen@networkplumber.org, thomas@monjalon.net, david.marchand@redhat.com, Feifei Wang , Konstantin Ananyev , Jerin Jacob Date: Mon, 1 Nov 2021 14:00:03 +0800 Message-Id: <20211101060007.2632418-2-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211101060007.2632418-1-feifei.wang2@arm.com> References: <20210902053253.3017858-1-feifei.wang2@arm.com> <20211101060007.2632418-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v9 1/5] eal: add a new generic helper for wait scheme X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a new generic helper which is a macro for wait scheme. Furthermore, to prevent compilation warning in arm: ---------------------------------------------- 'warning: implicit declaration of function ...' ---------------------------------------------- Delete 'undef' constructions for '__LOAD_EXC_xx', '__SEVL' and '__WFE'. And add ‘__RTE_ARM’ for these macros to fix the namespace. This is because original macros are undefine at the end of the file. If new macro 'rte_wait_event' calls them in other files, they will be seen as 'not defined'. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Konstantin Ananyev Acked-by: Jerin Jacob --- lib/eal/arm/include/rte_pause_64.h | 202 +++++++++++++++++----------- lib/eal/include/generic/rte_pause.h | 29 ++++ 2 files changed, 155 insertions(+), 76 deletions(-) diff --git a/lib/eal/arm/include/rte_pause_64.h b/lib/eal/arm/include/rte_pause_64.h index e87d10b8cc..0ca03c6130 100644 --- a/lib/eal/arm/include/rte_pause_64.h +++ b/lib/eal/arm/include/rte_pause_64.h @@ -26,47 +26,120 @@ static inline void rte_pause(void) #ifdef RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED /* Send an event to quit WFE. */ -#define __SEVL() { asm volatile("sevl" : : : "memory"); } +#define __RTE_ARM_SEVL() { asm volatile("sevl" : : : "memory"); } /* Put processor into low power WFE(Wait For Event) state. */ -#define __WFE() { asm volatile("wfe" : : : "memory"); } +#define __RTE_ARM_WFE() { asm volatile("wfe" : : : "memory"); } -static __rte_always_inline void -rte_wait_until_equal_16(volatile uint16_t *addr, uint16_t expected, - int memorder) -{ - uint16_t value; - - assert(memorder == __ATOMIC_ACQUIRE || memorder == __ATOMIC_RELAXED); - - /* - * Atomic exclusive load from addr, it returns the 16-bit content of - * *addr while making it 'monitored',when it is written by someone - * else, the 'monitored' state is cleared and a event is generated - * implicitly to exit WFE. - */ -#define __LOAD_EXC_16(src, dst, memorder) { \ +/* + * Atomic exclusive load from addr, it returns the 16-bit content of + * *addr while making it 'monitored', when it is written by someone + * else, the 'monitored' state is cleared and an event is generated + * implicitly to exit WFE. + */ +#define __RTE_ARM_LOAD_EXC_16(src, dst, memorder) { \ if (memorder == __ATOMIC_RELAXED) { \ asm volatile("ldxrh %w[tmp], [%x[addr]]" \ : [tmp] "=&r" (dst) \ - : [addr] "r"(src) \ + : [addr] "r" (src) \ : "memory"); \ } else { \ asm volatile("ldaxrh %w[tmp], [%x[addr]]" \ : [tmp] "=&r" (dst) \ - : [addr] "r"(src) \ + : [addr] "r" (src) \ : "memory"); \ } } - __LOAD_EXC_16(addr, value, memorder) +/* + * Atomic exclusive load from addr, it returns the 32-bit content of + * *addr while making it 'monitored', when it is written by someone + * else, the 'monitored' state is cleared and an event is generated + * implicitly to exit WFE. + */ +#define __RTE_ARM_LOAD_EXC_32(src, dst, memorder) { \ + if (memorder == __ATOMIC_RELAXED) { \ + asm volatile("ldxr %w[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r" (src) \ + : "memory"); \ + } else { \ + asm volatile("ldaxr %w[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r" (src) \ + : "memory"); \ + } } + +/* + * Atomic exclusive load from addr, it returns the 64-bit content of + * *addr while making it 'monitored', when it is written by someone + * else, the 'monitored' state is cleared and an event is generated + * implicitly to exit WFE. + */ +#define __RTE_ARM_LOAD_EXC_64(src, dst, memorder) { \ + if (memorder == __ATOMIC_RELAXED) { \ + asm volatile("ldxr %x[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r" (src) \ + : "memory"); \ + } else { \ + asm volatile("ldaxr %x[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r" (src) \ + : "memory"); \ + } } + +/* + * Atomic exclusive load from addr, it returns the 128-bit content of + * *addr while making it 'monitored', when it is written by someone + * else, the 'monitored' state is cleared and an event is generated + * implicitly to exit WFE. + */ +#define __RTE_ARM_LOAD_EXC_128(src, dst, memorder) { \ + volatile rte_int128_t *dst_128 = (volatile rte_int128_t *)&dst; \ + if (memorder == __ATOMIC_RELAXED) { \ + asm volatile("ldxp %x[tmp0], %x[tmp1], [%x[addr]]" \ + : [tmp0] "=&r" (dst_128->val[0]), \ + [tmp1] "=&r" (dst_128->val[1]) \ + : [addr] "r" (src) \ + : "memory"); \ + } else { \ + asm volatile("ldaxp %x[tmp0], %x[tmp1], [%x[addr]]" \ + : [tmp0] "=&r" (dst_128->val[0]), \ + [tmp1] "=&r" (dst_128->val[1]) \ + : [addr] "r" (src) \ + : "memory"); \ + } } \ + +#define __RTE_ARM_LOAD_EXC(src, dst, memorder, size) { \ + RTE_BUILD_BUG_ON(size != 16 && size != 32 && size != 64 \ + && size != 128); \ + if (size == 16) \ + __RTE_ARM_LOAD_EXC_16(src, dst, memorder) \ + else if (size == 32) \ + __RTE_ARM_LOAD_EXC_32(src, dst, memorder) \ + else if (size == 64) \ + __RTE_ARM_LOAD_EXC_64(src, dst, memorder) \ + else if (size == 128) \ + __RTE_ARM_LOAD_EXC_128(src, dst, memorder) \ +} + +static __rte_always_inline void +rte_wait_until_equal_16(volatile uint16_t *addr, uint16_t expected, + int memorder) +{ + uint16_t value; + + RTE_BUILD_BUG_ON(memorder != __ATOMIC_ACQUIRE && + memorder != __ATOMIC_RELAXED); + + __RTE_ARM_LOAD_EXC_16(addr, value, memorder) if (value != expected) { - __SEVL() + __RTE_ARM_SEVL() do { - __WFE() - __LOAD_EXC_16(addr, value, memorder) + __RTE_ARM_WFE() + __RTE_ARM_LOAD_EXC_16(addr, value, memorder) } while (value != expected); } -#undef __LOAD_EXC_16 } static __rte_always_inline void @@ -75,36 +148,17 @@ rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, { uint32_t value; - assert(memorder == __ATOMIC_ACQUIRE || memorder == __ATOMIC_RELAXED); - - /* - * Atomic exclusive load from addr, it returns the 32-bit content of - * *addr while making it 'monitored',when it is written by someone - * else, the 'monitored' state is cleared and a event is generated - * implicitly to exit WFE. - */ -#define __LOAD_EXC_32(src, dst, memorder) { \ - if (memorder == __ATOMIC_RELAXED) { \ - asm volatile("ldxr %w[tmp], [%x[addr]]" \ - : [tmp] "=&r" (dst) \ - : [addr] "r"(src) \ - : "memory"); \ - } else { \ - asm volatile("ldaxr %w[tmp], [%x[addr]]" \ - : [tmp] "=&r" (dst) \ - : [addr] "r"(src) \ - : "memory"); \ - } } + RTE_BUILD_BUG_ON(memorder != __ATOMIC_ACQUIRE && + memorder != __ATOMIC_RELAXED); - __LOAD_EXC_32(addr, value, memorder) + __RTE_ARM_LOAD_EXC_32(addr, value, memorder) if (value != expected) { - __SEVL() + __RTE_ARM_SEVL() do { - __WFE() - __LOAD_EXC_32(addr, value, memorder) + __RTE_ARM_WFE() + __RTE_ARM_LOAD_EXC_32(addr, value, memorder) } while (value != expected); } -#undef __LOAD_EXC_32 } static __rte_always_inline void @@ -113,40 +167,36 @@ rte_wait_until_equal_64(volatile uint64_t *addr, uint64_t expected, { uint64_t value; - assert(memorder == __ATOMIC_ACQUIRE || memorder == __ATOMIC_RELAXED); - - /* - * Atomic exclusive load from addr, it returns the 64-bit content of - * *addr while making it 'monitored',when it is written by someone - * else, the 'monitored' state is cleared and a event is generated - * implicitly to exit WFE. - */ -#define __LOAD_EXC_64(src, dst, memorder) { \ - if (memorder == __ATOMIC_RELAXED) { \ - asm volatile("ldxr %x[tmp], [%x[addr]]" \ - : [tmp] "=&r" (dst) \ - : [addr] "r"(src) \ - : "memory"); \ - } else { \ - asm volatile("ldaxr %x[tmp], [%x[addr]]" \ - : [tmp] "=&r" (dst) \ - : [addr] "r"(src) \ - : "memory"); \ - } } + RTE_BUILD_BUG_ON(memorder != __ATOMIC_ACQUIRE && + memorder != __ATOMIC_RELAXED); - __LOAD_EXC_64(addr, value, memorder) + __RTE_ARM_LOAD_EXC_64(addr, value, memorder) if (value != expected) { - __SEVL() + __RTE_ARM_SEVL() do { - __WFE() - __LOAD_EXC_64(addr, value, memorder) + __RTE_ARM_WFE() + __RTE_ARM_LOAD_EXC_64(addr, value, memorder) } while (value != expected); } } -#undef __LOAD_EXC_64 -#undef __SEVL -#undef __WFE +#define RTE_WAIT_UNTIL_MASKED(addr, mask, cond, expected, memorder) \ +do { \ + RTE_BUILD_BUG_ON(!__builtin_constant_p(memorder)); \ + RTE_BUILD_BUG_ON(memorder != __ATOMIC_ACQUIRE && \ + memorder != __ATOMIC_RELAXED); \ + const uint32_t size = sizeof(*(addr)) << 3; \ + typeof(*(addr)) expected_value = (expected); \ + typeof(*(addr)) value; \ + __RTE_ARM_LOAD_EXC((addr), value, memorder, size) \ + if (!((value & (mask)) cond expected_value)) { \ + __RTE_ARM_SEVL() \ + do { \ + __RTE_ARM_WFE() \ + __RTE_ARM_LOAD_EXC((addr), value, memorder, size) \ + } while (!((value & (mask)) cond expected_value)); \ + } \ +} while (0) #endif diff --git a/lib/eal/include/generic/rte_pause.h b/lib/eal/include/generic/rte_pause.h index 668ee4a184..5894a0ad94 100644 --- a/lib/eal/include/generic/rte_pause.h +++ b/lib/eal/include/generic/rte_pause.h @@ -111,6 +111,35 @@ rte_wait_until_equal_64(volatile uint64_t *addr, uint64_t expected, while (__atomic_load_n(addr, memorder) != expected) rte_pause(); } + +/* + * Wait until *addr & mask makes the condition true. With a relaxed memory + * ordering model, the loads around this helper can be reordered. + * + * @param addr + * A pointer to the memory location. + * @param mask + * A mask of value bits in interest. + * @param cond + * A symbol representing the condition. + * @param expected + * An expected value to be in the memory location. + * @param memorder + * Two different memory orders that can be specified: + * __ATOMIC_ACQUIRE and __ATOMIC_RELAXED. These map to + * C++11 memory orders with the same names, see the C++11 standard or + * the GCC wiki on atomic synchronization for detailed definition. + */ +#define RTE_WAIT_UNTIL_MASKED(addr, mask, cond, expected, memorder) \ +do { \ + RTE_BUILD_BUG_ON(!__builtin_constant_p(memorder)); \ + RTE_BUILD_BUG_ON(memorder != __ATOMIC_ACQUIRE && \ + memorder != __ATOMIC_RELAXED); \ + typeof(*(addr)) expected_value = (expected); \ + while (!((__atomic_load_n((addr), (memorder)) & (mask)) cond \ + expected_value)) \ + rte_pause(); \ +} while (0) #endif #endif /* _RTE_PAUSE_H_ */ From patchwork Mon Nov 1 06:00:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 103289 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ED746A0C52; Mon, 1 Nov 2021 07:00:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7CA81410E8; Mon, 1 Nov 2021 07:00:25 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 994834111E for ; Mon, 1 Nov 2021 07:00:23 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1A923106F; Sun, 31 Oct 2021 23:00:23 -0700 (PDT) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.102]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 329083F70D; Sun, 31 Oct 2021 23:00:19 -0700 (PDT) From: Feifei Wang To: Cc: dev@dpdk.org, nd@arm.com, jerinjacobk@gmail.com, stephen@networkplumber.org, thomas@monjalon.net, david.marchand@redhat.com, Feifei Wang , Ruifeng Wang , Jerin Jacob Date: Mon, 1 Nov 2021 14:00:04 +0800 Message-Id: <20211101060007.2632418-3-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211101060007.2632418-1-feifei.wang2@arm.com> References: <20210902053253.3017858-1-feifei.wang2@arm.com> <20211101060007.2632418-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v9 2/5] pflock: use wait until scheme for read pflock X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instead of polling for read pflock update, use wait until scheme for this case. Jira: ENTNET-2903 Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Jerin Jacob --- lib/eal/include/generic/rte_pflock.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/lib/eal/include/generic/rte_pflock.h b/lib/eal/include/generic/rte_pflock.h index e57c179ef2..5d2864ee8e 100644 --- a/lib/eal/include/generic/rte_pflock.h +++ b/lib/eal/include/generic/rte_pflock.h @@ -121,9 +121,7 @@ rte_pflock_read_lock(rte_pflock_t *pf) return; /* Wait for current write phase to complete. */ - while ((__atomic_load_n(&pf->rd.in, __ATOMIC_ACQUIRE) - & RTE_PFLOCK_WBITS) == w) - rte_pause(); + RTE_WAIT_UNTIL_MASKED(&pf->rd.in, RTE_PFLOCK_WBITS, !=, w, __ATOMIC_ACQUIRE); } /** From patchwork Mon Nov 1 06:00:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 103290 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 590DBA0C52; Mon, 1 Nov 2021 07:00:41 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9129141121; Mon, 1 Nov 2021 07:00:28 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 4F80641120 for ; Mon, 1 Nov 2021 07:00:27 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D2F16D6E; Sun, 31 Oct 2021 23:00:26 -0700 (PDT) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.102]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id AA21D3F70D; Sun, 31 Oct 2021 23:00:23 -0700 (PDT) From: Feifei Wang To: Honnappa Nagarahalli Cc: dev@dpdk.org, nd@arm.com, jerinjacobk@gmail.com, stephen@networkplumber.org, thomas@monjalon.net, david.marchand@redhat.com, Feifei Wang , Ruifeng Wang , Jerin Jacob Date: Mon, 1 Nov 2021 14:00:05 +0800 Message-Id: <20211101060007.2632418-4-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211101060007.2632418-1-feifei.wang2@arm.com> References: <20210902053253.3017858-1-feifei.wang2@arm.com> <20211101060007.2632418-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v9 3/5] mcslock: use wait until scheme for mcslock X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instead of polling for mcslock to be updated, use wait until scheme for this case. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Jerin Jacob --- lib/eal/include/generic/rte_mcslock.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/eal/include/generic/rte_mcslock.h b/lib/eal/include/generic/rte_mcslock.h index 34f33c64a5..753836d23c 100644 --- a/lib/eal/include/generic/rte_mcslock.h +++ b/lib/eal/include/generic/rte_mcslock.h @@ -116,8 +116,9 @@ rte_mcslock_unlock(rte_mcslock_t **msl, rte_mcslock_t *me) /* More nodes added to the queue by other CPUs. * Wait until the next pointer is set. */ - while (__atomic_load_n(&me->next, __ATOMIC_RELAXED) == NULL) - rte_pause(); + uintptr_t *next; + next = (uintptr_t *)&me->next; + RTE_WAIT_UNTIL_MASKED(next, UINTPTR_MAX, !=, 0, __ATOMIC_RELAXED); } /* Pass lock to next waiter. */ From patchwork Mon Nov 1 06:00:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 103291 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9FAB6A0C52; Mon, 1 Nov 2021 07:00:47 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9CB45410FD; Mon, 1 Nov 2021 07:00:32 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id BEB7540DF6 for ; Mon, 1 Nov 2021 07:00:30 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5511DD6E; Sun, 31 Oct 2021 23:00:30 -0700 (PDT) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.102]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6E1263F70D; Sun, 31 Oct 2021 23:00:27 -0700 (PDT) From: Feifei Wang To: Konstantin Ananyev Cc: dev@dpdk.org, nd@arm.com, jerinjacobk@gmail.com, stephen@networkplumber.org, thomas@monjalon.net, david.marchand@redhat.com, Feifei Wang , Ruifeng Wang Date: Mon, 1 Nov 2021 14:00:06 +0800 Message-Id: <20211101060007.2632418-5-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211101060007.2632418-1-feifei.wang2@arm.com> References: <20210902053253.3017858-1-feifei.wang2@arm.com> <20211101060007.2632418-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v9 4/5] bpf: use wait until scheme for Rx/Tx iteration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instead of polling for cbi->use to be updated, use wait until scheme. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Konstantin Ananyev --- lib/bpf/bpf_pkt.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/lib/bpf/bpf_pkt.c b/lib/bpf/bpf_pkt.c index 6e8248f0d6..000b82d935 100644 --- a/lib/bpf/bpf_pkt.c +++ b/lib/bpf/bpf_pkt.c @@ -113,7 +113,7 @@ bpf_eth_cbi_unuse(struct bpf_eth_cbi *cbi) static void bpf_eth_cbi_wait(const struct bpf_eth_cbi *cbi) { - uint32_t nuse, puse; + uint32_t puse; /* make sure all previous loads and stores are completed */ rte_smp_mb(); @@ -122,11 +122,8 @@ bpf_eth_cbi_wait(const struct bpf_eth_cbi *cbi) /* in use, busy wait till current RX/TX iteration is finished */ if ((puse & BPF_ETH_CBI_INUSE) != 0) { - do { - rte_pause(); - rte_compiler_barrier(); - nuse = cbi->use; - } while (nuse == puse); + RTE_WAIT_UNTIL_MASKED((uint32_t *)(uintptr_t)&cbi->use, UINT32_MAX, + !=, puse, __ATOMIC_RELAXED); } } From patchwork Mon Nov 1 06:00:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 103292 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 469A2A0C52; Mon, 1 Nov 2021 07:00:54 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ACD5741130; Mon, 1 Nov 2021 07:00:35 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 8A15741130 for ; Mon, 1 Nov 2021 07:00:34 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1642ED6E; Sun, 31 Oct 2021 23:00:34 -0700 (PDT) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.102]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E49EC3F70D; Sun, 31 Oct 2021 23:00:30 -0700 (PDT) From: Feifei Wang To: David Hunt Cc: dev@dpdk.org, nd@arm.com, jerinjacobk@gmail.com, stephen@networkplumber.org, thomas@monjalon.net, david.marchand@redhat.com, Feifei Wang , Ruifeng Wang , Jerin Jacob Date: Mon, 1 Nov 2021 14:00:07 +0800 Message-Id: <20211101060007.2632418-6-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211101060007.2632418-1-feifei.wang2@arm.com> References: <20210902053253.3017858-1-feifei.wang2@arm.com> <20211101060007.2632418-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v9 5/5] distributor: use wait until scheme X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instead of polling for bufptr64 to be updated, use wait until scheme for this case. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: Jerin Jacob --- lib/distributor/rte_distributor_single.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/lib/distributor/rte_distributor_single.c b/lib/distributor/rte_distributor_single.c index f4725b1d0b..b653620688 100644 --- a/lib/distributor/rte_distributor_single.c +++ b/lib/distributor/rte_distributor_single.c @@ -33,9 +33,8 @@ rte_distributor_request_pkt_single(struct rte_distributor_single *d, union rte_distributor_buffer_single *buf = &d->bufs[worker_id]; int64_t req = (((int64_t)(uintptr_t)oldpkt) << RTE_DISTRIB_FLAG_BITS) | RTE_DISTRIB_GET_BUF; - while (unlikely(__atomic_load_n(&buf->bufptr64, __ATOMIC_RELAXED) - & RTE_DISTRIB_FLAGS_MASK)) - rte_pause(); + RTE_WAIT_UNTIL_MASKED(&buf->bufptr64, RTE_DISTRIB_FLAGS_MASK, + ==, 0, __ATOMIC_RELAXED); /* Sync with distributor on GET_BUF flag. */ __atomic_store_n(&(buf->bufptr64), req, __ATOMIC_RELEASE); @@ -74,9 +73,8 @@ rte_distributor_return_pkt_single(struct rte_distributor_single *d, union rte_distributor_buffer_single *buf = &d->bufs[worker_id]; uint64_t req = (((int64_t)(uintptr_t)oldpkt) << RTE_DISTRIB_FLAG_BITS) | RTE_DISTRIB_RETURN_BUF; - while (unlikely(__atomic_load_n(&buf->bufptr64, __ATOMIC_RELAXED) - & RTE_DISTRIB_FLAGS_MASK)) - rte_pause(); + RTE_WAIT_UNTIL_MASKED(&buf->bufptr64, RTE_DISTRIB_FLAGS_MASK, + ==, 0, __ATOMIC_RELAXED); /* Sync with distributor on RETURN_BUF flag. */ __atomic_store_n(&(buf->bufptr64), req, __ATOMIC_RELEASE);