From patchwork Thu Dec 20 10:42:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 49167 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A53241BA5A; Thu, 20 Dec 2018 11:43:09 +0100 (CET) Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id C0CF61B96E for ; Thu, 20 Dec 2018 11:43:07 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 15D8A15AB; Thu, 20 Dec 2018 02:43:07 -0800 (PST) Received: from net-debian.shanghai.arm.com (net-debian.shanghai.arm.com [10.169.36.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 938AD3F5C0; Thu, 20 Dec 2018 02:43:05 -0800 (PST) From: Gavin Hu To: dev@dpdk.org Cc: thomas@monjalon.net, jerinj@marvell.com, hemant.agrawal@nxp.com, bruce.richardson@intel.com, chaozhu@linux.vnet.ibm.com, nd@arm.com, Honnappa.Nagarahalli@arm.com, Gavin Hu Date: Thu, 20 Dec 2018 18:42:42 +0800 Message-Id: <20181220104246.5590-2-gavin.hu@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181220104246.5590-1-gavin.hu@arm.com> References: <20181220104246.5590-1-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v1 1/5] test/spinlock: remove 1us delay for correct spinlock benchmarking X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The test is to benchmark the performance of spinlock by counting the number of spinlock acquire and release operations within the specified time. A typical pair of lock and unlock operations costs tens or hundreds of nano seconds, in comparison to this, delaying 1 us outside of the locked region is too much, compromising the goal of benchmarking the lock and unlock performance. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Joyce Kong Reviewed-by: Phil Yang Reviewed-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedahl --- test/test/test_spinlock.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/test/test/test_spinlock.c b/test/test/test_spinlock.c index 73bff128e..6795195ae 100644 --- a/test/test/test_spinlock.c +++ b/test/test/test_spinlock.c @@ -120,8 +120,6 @@ load_loop_fn(void *func_param) lcount++; if (use_lock) rte_spinlock_unlock(&lk); - /* delay to make lock duty cycle slighlty realistic */ - rte_delay_us(1); time_diff = rte_get_timer_cycles() - begin; } lock_count[lcore] = lcount; From patchwork Thu Dec 20 10:42:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 49168 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 87C141BA90; Thu, 20 Dec 2018 11:43:11 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id 5DEEB1BA30 for ; Thu, 20 Dec 2018 11:43:09 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C403B15BE; Thu, 20 Dec 2018 02:43:08 -0800 (PST) Received: from net-debian.shanghai.arm.com (net-debian.shanghai.arm.com [10.169.36.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4E45B3F5C0; Thu, 20 Dec 2018 02:43:07 -0800 (PST) From: Gavin Hu To: dev@dpdk.org Cc: thomas@monjalon.net, jerinj@marvell.com, hemant.agrawal@nxp.com, bruce.richardson@intel.com, chaozhu@linux.vnet.ibm.com, nd@arm.com, Honnappa.Nagarahalli@arm.com, Gavin Hu Date: Thu, 20 Dec 2018 18:42:43 +0800 Message-Id: <20181220104246.5590-3-gavin.hu@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181220104246.5590-1-gavin.hu@arm.com> References: <20181220104246.5590-1-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v1 2/5] test/spinlock: get timestamp more precisely X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To precisely benchmark the spinlock performance, uses the precise version of getting timestamps, which enforces the timestamps are obtained at the expected places. Signed-off-by: Gavin Hu Reviewed-by: Phil Yang --- test/test/test_spinlock.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/test/test_spinlock.c b/test/test/test_spinlock.c index 6795195ae..648474833 100644 --- a/test/test/test_spinlock.c +++ b/test/test/test_spinlock.c @@ -113,14 +113,14 @@ load_loop_fn(void *func_param) if (lcore != rte_get_master_lcore()) while (rte_atomic32_read(&synchro) == 0); - begin = rte_get_timer_cycles(); + begin = rte_rdtsc_precise(); while (time_diff < hz * TIME_MS / 1000) { if (use_lock) rte_spinlock_lock(&lk); lcount++; if (use_lock) rte_spinlock_unlock(&lk); - time_diff = rte_get_timer_cycles() - begin; + time_diff = rte_rdtsc_precise() - begin; } lock_count[lcore] = lcount; return 0; From patchwork Thu Dec 20 10:42:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 49169 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6B3ED1BABD; Thu, 20 Dec 2018 11:43:13 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id 0C5621B9BB for ; Thu, 20 Dec 2018 11:43:11 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 79696A78; Thu, 20 Dec 2018 02:43:10 -0800 (PST) Received: from net-debian.shanghai.arm.com (net-debian.shanghai.arm.com [10.169.36.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 072AB3F5C0; Thu, 20 Dec 2018 02:43:08 -0800 (PST) From: Gavin Hu To: dev@dpdk.org Cc: thomas@monjalon.net, jerinj@marvell.com, hemant.agrawal@nxp.com, bruce.richardson@intel.com, chaozhu@linux.vnet.ibm.com, nd@arm.com, Honnappa.Nagarahalli@arm.com, Gavin Hu Date: Thu, 20 Dec 2018 18:42:44 +0800 Message-Id: <20181220104246.5590-4-gavin.hu@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181220104246.5590-1-gavin.hu@arm.com> References: <20181220104246.5590-1-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v1 3/5] test/spinlock: amortize the cost of getting time X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instead of getting timestamps per iteration, amortize its overhead can help getting more precise benchmarking results. Signed-off-by: Gavin Hu Reviewed-by: Joyce Kong --- test/test/test_spinlock.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/test/test/test_spinlock.c b/test/test/test_spinlock.c index 648474833..e9839b979 100644 --- a/test/test/test_spinlock.c +++ b/test/test/test_spinlock.c @@ -96,9 +96,9 @@ test_spinlock_recursive_per_core(__attribute__((unused)) void *arg) } static rte_spinlock_t lk = RTE_SPINLOCK_INITIALIZER; -static uint64_t lock_count[RTE_MAX_LCORE] = {0}; +static uint64_t time_count[RTE_MAX_LCORE] = {0}; -#define TIME_MS 100 +#define MAX_LOOP 10000 static int load_loop_fn(void *func_param) @@ -114,15 +114,14 @@ load_loop_fn(void *func_param) while (rte_atomic32_read(&synchro) == 0); begin = rte_rdtsc_precise(); - while (time_diff < hz * TIME_MS / 1000) { + while (lcount < MAX_LOOP) { if (use_lock) rte_spinlock_lock(&lk); - lcount++; if (use_lock) rte_spinlock_unlock(&lk); - time_diff = rte_rdtsc_precise() - begin; } - lock_count[lcore] = lcount; + time_diff = rte_rdtsc_precise() - begin; + time_count[lcore] = time_diff * 1000000 / hz; return 0; } @@ -136,14 +135,16 @@ test_spinlock_perf(void) printf("\nTest with no lock on single core...\n"); load_loop_fn(&lock); - printf("Core [%u] count = %"PRIu64"\n", lcore, lock_count[lcore]); - memset(lock_count, 0, sizeof(lock_count)); + printf("Core [%u] Cost Time = %"PRIu64" us\n", lcore, + time_count[lcore]); + memset(time_count, 0, sizeof(time_count)); printf("\nTest with lock on single core...\n"); lock = 1; load_loop_fn(&lock); - printf("Core [%u] count = %"PRIu64"\n", lcore, lock_count[lcore]); - memset(lock_count, 0, sizeof(lock_count)); + printf("Core [%u] Cost Time = %"PRIu64" us\n", lcore, + time_count[lcore]); + memset(time_count, 0, sizeof(time_count)); printf("\nTest with lock on %u cores...\n", rte_lcore_count()); @@ -158,11 +159,12 @@ test_spinlock_perf(void) rte_eal_mp_wait_lcore(); RTE_LCORE_FOREACH(i) { - printf("Core [%u] count = %"PRIu64"\n", i, lock_count[i]); - total += lock_count[i]; + printf("Core [%u] Cost Time = %"PRIu64" us\n", i, + time_count[i]); + total += time_count[i]; } - printf("Total count = %"PRIu64"\n", total); + printf("Total Cost Time = %"PRIu64" us\n", total); return 0; } From patchwork Thu Dec 20 10:42:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 49170 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 790BE1BB22; Thu, 20 Dec 2018 11:43:15 +0100 (CET) Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id C54B51BAA8 for ; Thu, 20 Dec 2018 11:43:12 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 314C715AB; Thu, 20 Dec 2018 02:43:12 -0800 (PST) Received: from net-debian.shanghai.arm.com (net-debian.shanghai.arm.com [10.169.36.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B0B4F3F5C0; Thu, 20 Dec 2018 02:43:10 -0800 (PST) From: Gavin Hu To: dev@dpdk.org Cc: thomas@monjalon.net, jerinj@marvell.com, hemant.agrawal@nxp.com, bruce.richardson@intel.com, chaozhu@linux.vnet.ibm.com, nd@arm.com, Honnappa.Nagarahalli@arm.com, Gavin Hu Date: Thu, 20 Dec 2018 18:42:45 +0800 Message-Id: <20181220104246.5590-5-gavin.hu@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181220104246.5590-1-gavin.hu@arm.com> References: <20181220104246.5590-1-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v1 4/5] spinlock: move the implementation to arm specific file X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" remove the hard code #ifdef RTE_FORCE_INTRINSICS, move the implementation to the arm specific file, x86 and POWER have their own implementations. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Phil Yang Reviewed-by: Honnappa Nagarahalli --- .../common/include/arch/arm/rte_spinlock.h | 20 +++++++++++++++++ .../common/include/generic/rte_spinlock.h | 26 ---------------------- 2 files changed, 20 insertions(+), 26 deletions(-) diff --git a/lib/librte_eal/common/include/arch/arm/rte_spinlock.h b/lib/librte_eal/common/include/arch/arm/rte_spinlock.h index 1a6916b6e..25d22fd1e 100644 --- a/lib/librte_eal/common/include/arch/arm/rte_spinlock.h +++ b/lib/librte_eal/common/include/arch/arm/rte_spinlock.h @@ -16,6 +16,26 @@ extern "C" { #include #include "generic/rte_spinlock.h" +static inline void +rte_spinlock_lock(rte_spinlock_t *sl) +{ + while (__sync_lock_test_and_set(&sl->locked, 1)) + while (sl->locked) + rte_pause(); +} + +static inline void +rte_spinlock_unlock(rte_spinlock_t *sl) +{ + __sync_lock_release(&sl->locked); +} + +static inline int +rte_spinlock_trylock(rte_spinlock_t *sl) +{ + return __sync_lock_test_and_set(&sl->locked, 1) == 0; +} + static inline int rte_tm_supported(void) { return 0; diff --git a/lib/librte_eal/common/include/generic/rte_spinlock.h b/lib/librte_eal/common/include/generic/rte_spinlock.h index c4c3fc31e..e555ecb95 100644 --- a/lib/librte_eal/common/include/generic/rte_spinlock.h +++ b/lib/librte_eal/common/include/generic/rte_spinlock.h @@ -57,16 +57,6 @@ rte_spinlock_init(rte_spinlock_t *sl) static inline void rte_spinlock_lock(rte_spinlock_t *sl); -#ifdef RTE_FORCE_INTRINSICS -static inline void -rte_spinlock_lock(rte_spinlock_t *sl) -{ - while (__sync_lock_test_and_set(&sl->locked, 1)) - while(sl->locked) - rte_pause(); -} -#endif - /** * Release the spinlock. * @@ -76,14 +66,6 @@ rte_spinlock_lock(rte_spinlock_t *sl) static inline void rte_spinlock_unlock (rte_spinlock_t *sl); -#ifdef RTE_FORCE_INTRINSICS -static inline void -rte_spinlock_unlock (rte_spinlock_t *sl) -{ - __sync_lock_release(&sl->locked); -} -#endif - /** * Try to take the lock. * @@ -95,14 +77,6 @@ rte_spinlock_unlock (rte_spinlock_t *sl) static inline int rte_spinlock_trylock (rte_spinlock_t *sl); -#ifdef RTE_FORCE_INTRINSICS -static inline int -rte_spinlock_trylock (rte_spinlock_t *sl) -{ - return __sync_lock_test_and_set(&sl->locked,1) == 0; -} -#endif - /** * Test if the lock is taken. * From patchwork Thu Dec 20 10:42:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 49171 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CF80E1BB4D; Thu, 20 Dec 2018 11:43:17 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id 6EB4B1BA57 for ; Thu, 20 Dec 2018 11:43:14 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DB4A515BE; Thu, 20 Dec 2018 02:43:13 -0800 (PST) Received: from net-debian.shanghai.arm.com (net-debian.shanghai.arm.com [10.169.36.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 69C343F5C0; Thu, 20 Dec 2018 02:43:12 -0800 (PST) From: Gavin Hu To: dev@dpdk.org Cc: thomas@monjalon.net, jerinj@marvell.com, hemant.agrawal@nxp.com, bruce.richardson@intel.com, chaozhu@linux.vnet.ibm.com, nd@arm.com, Honnappa.Nagarahalli@arm.com, Gavin Hu Date: Thu, 20 Dec 2018 18:42:46 +0800 Message-Id: <20181220104246.5590-6-gavin.hu@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181220104246.5590-1-gavin.hu@arm.com> References: <20181220104246.5590-1-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v1 5/5] spinlock: reimplement with atomic one-way barrier builtins X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The __sync builtin based implementation generates full memory barriers ('dmb ish') on Arm platforms. Using C11 atomic builtins to generate one way barriers. Here is the assembly code of __sync_compare_and_swap builtin. __sync_bool_compare_and_swap(dst, exp, src); 0x000000000090f1b0 <+16>: e0 07 40 f9 ldr x0, [sp, #8] 0x000000000090f1b4 <+20>: e1 0f 40 79 ldrh w1, [sp, #6] 0x000000000090f1b8 <+24>: e2 0b 40 79 ldrh w2, [sp, #4] 0x000000000090f1bc <+28>: 21 3c 00 12 and w1, w1, #0xffff 0x000000000090f1c0 <+32>: 03 7c 5f 48 ldxrh w3, [x0] 0x000000000090f1c4 <+36>: 7f 00 01 6b cmp w3, w1 0x000000000090f1c8 <+40>: 61 00 00 54 b.ne 0x90f1d4 // b.any 0x000000000090f1cc <+44>: 02 fc 04 48 stlxrh w4, w2, [x0] 0x000000000090f1d0 <+48>: 84 ff ff 35 cbnz w4, 0x90f1c0 0x000000000090f1d4 <+52>: bf 3b 03 d5 dmb ish 0x000000000090f1d8 <+56>: e0 17 9f 1a cset w0, eq // eq = none The benchmarking results showed 3X performance gain on Cavium ThunderX2 and 13% on Qualcomm Falmon and 3.7% on 4-A72 Marvell macchiatobin. Here is the example test result on TX2: *** spinlock_autotest without this patch *** Core [123] Cost Time = 639822 us Core [124] Cost Time = 633253 us Core [125] Cost Time = 646030 us Core [126] Cost Time = 643189 us Core [127] Cost Time = 647039 us Total Cost Time = 95433298 us *** spinlock_autotest with this patch *** Core [123] Cost Time = 163615 us Core [124] Cost Time = 166471 us Core [125] Cost Time = 189044 us Core [126] Cost Time = 195745 us Core [127] Cost Time = 78423 us Total Cost Time = 27339656 us Signed-off-by: Gavin Hu Reviewed-by: Phil Yang Reviewed-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedahl Reviewed-by: Steve Capper --- lib/librte_eal/common/include/arch/arm/rte_spinlock.h | 16 ++++++++++++---- lib/librte_eal/common/include/generic/rte_spinlock.h | 2 +- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/lib/librte_eal/common/include/arch/arm/rte_spinlock.h b/lib/librte_eal/common/include/arch/arm/rte_spinlock.h index 25d22fd1e..33a6d382f 100644 --- a/lib/librte_eal/common/include/arch/arm/rte_spinlock.h +++ b/lib/librte_eal/common/include/arch/arm/rte_spinlock.h @@ -19,21 +19,29 @@ extern "C" { static inline void rte_spinlock_lock(rte_spinlock_t *sl) { - while (__sync_lock_test_and_set(&sl->locked, 1)) - while (sl->locked) + int exp = 0; + + while (!__atomic_compare_exchange_n(&sl->locked, &exp, 1, 1, + __ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) { + while (__atomic_load_n(&sl->locked, __ATOMIC_RELAXED)) rte_pause(); + exp = 0; + } } static inline void rte_spinlock_unlock(rte_spinlock_t *sl) { - __sync_lock_release(&sl->locked); + __atomic_store_n(&sl->locked, 0, __ATOMIC_RELEASE); } static inline int rte_spinlock_trylock(rte_spinlock_t *sl) { - return __sync_lock_test_and_set(&sl->locked, 1) == 0; + int exp = 0; + return __atomic_compare_exchange_n(&sl->locked, &exp, 1, + false, /* disallow spurious failure */ + __ATOMIC_ACQUIRE, __ATOMIC_RELAXED); } static inline int rte_tm_supported(void) diff --git a/lib/librte_eal/common/include/generic/rte_spinlock.h b/lib/librte_eal/common/include/generic/rte_spinlock.h index e555ecb95..a8cc75adc 100644 --- a/lib/librte_eal/common/include/generic/rte_spinlock.h +++ b/lib/librte_eal/common/include/generic/rte_spinlock.h @@ -87,7 +87,7 @@ rte_spinlock_trylock (rte_spinlock_t *sl); */ static inline int rte_spinlock_is_locked (rte_spinlock_t *sl) { - return sl->locked; + return __atomic_load_n(&sl->locked, __ATOMIC_ACQUIRE); } /**