From patchwork Thu Apr 25 08:58:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 139675 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2202A43F04; Thu, 25 Apr 2024 11:10:22 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 443BE436DD; Thu, 25 Apr 2024 11:10:05 +0200 (CEST) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2067.outbound.protection.outlook.com [40.107.104.67]) by mails.dpdk.org (Postfix) with ESMTP id 0BAEE436C8 for ; Thu, 25 Apr 2024 11:10:02 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dwYCK1/aIjUjCBbv9sggvwNApATG6KAWBLGcUDlVqoN0dw0kcUBraRs2sgU+A/vC97FX5YmyiRP+fn7fvM83TAQRcWICJsg8Mx4d5qC0J7RpTwsyl6vuRk4CVDTsYN55vm+CJ7IuLF2cDz+wkujN7wlh0FuVq3/0UMPjrcKsyrDH+tYaI1e2hfh1S3W/xQIc+8wq844/KWkWZ/xa420brwJap2VemKKsnzLW9terlWJ7lX6B+oj+xmDdyZmcdvy1Ei3aDBjOXhUxytwzBjYSwmSUjMPOVrPgcR3nz1ZdJRzSFTkJtIyLxvYb9lvI1scdxa/KI0acFdO0OlQ6gw/TrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m127R3HU9pNG3bWTrW0IgoJ9+Gojo1qtktVZxd6EF04=; b=WqyyjqfglsJmRJPc3nQXD5vW9H6lpTQ+RlFiQtrCMpS3+ACKanXUH/HYG1hBjlR9VHK+QU+F/WmiUKG0liffLjQXOrdkuj2DLBarMN7mPHv6iTGsPR65qZubMnuTuM/qJ0drOlBOjiv1qWG2LalXuWcoqLsw+rhHsorQFm80wUTphqN7tDNVCkZWf00CF3Lpo2hJO7v4e9r7ykHw4zK+UmbcAZv6+3wTrwxdnp1596OBvkiMinMw5KkXsD9L1alMambUIg6/xtBgXRIkQbrjoM0snvZX3bAjSGLsh8rEns/pPFokgWMV/d5kiVuJ4LjxLL7gMrZaW0VbtH21baPjDA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m127R3HU9pNG3bWTrW0IgoJ9+Gojo1qtktVZxd6EF04=; b=EXcjhBGE39dmj6/PA/QAGws6O2jMxFVoUcR9xDqrPJ/Ez3PDZPRSlYBAF+zvWoqD4JTibpo+Ed3Stig0IwAVSed+1GfsEFjNCY32vITAxMr9UeofAu+ziv21mpjGtCONOYT9ZP3+pD+O67a9OWjkNjS9FjAAaeB+Tm13Pp7S7nHk8wHy705EyD2zY9kLO1IJ+jw9lI4Nh/psMa2p7X0nl+duEbeCM09Il5G5L1+BKi9Bfhv4w5mjj49X84YH5WT+mN+zFggMVM1nHlPaA3HvPEFvkz/1gyoEw1NhQm39Ru17bkAeXU/wLaKkq6aE8rEe6/gNP5G1CLilBSnKVCCkmA== Received: from DUZP191CA0028.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:4f8::24) by DBAPR07MB6807.eurprd07.prod.outlook.com (2603:10a6:10:19f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7472.44; Thu, 25 Apr 2024 09:09:59 +0000 Received: from DU2PEPF0001E9C2.eurprd03.prod.outlook.com (2603:10a6:10:4f8:cafe::52) by DUZP191CA0028.outlook.office365.com (2603:10a6:10:4f8::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.25 via Frontend Transport; Thu, 25 Apr 2024 09:09:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU2PEPF0001E9C2.mail.protection.outlook.com (10.167.8.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.19 via Frontend Transport; Thu, 25 Apr 2024 09:09:59 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.65) with Microsoft SMTP Server id 15.2.1258.12; Thu, 25 Apr 2024 11:09:59 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id E238F1C006A; Thu, 25 Apr 2024 11:09:58 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Tyler Retzlaff , =?utf-8?q?Mattias_R=C3=B6nnb?= =?utf-8?q?lom?= Subject: [RFC v2 5/6] eal: add atomic bit operations Date: Thu, 25 Apr 2024 10:58:52 +0200 Message-ID: <20240425085853.97888-6-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240425085853.97888-1-mattias.ronnblom@ericsson.com> References: <20240302135328.531940-2-mattias.ronnblom@ericsson.com> <20240425085853.97888-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF0001E9C2:EE_|DBAPR07MB6807:EE_ X-MS-Office365-Filtering-Correlation-Id: a656cc31-d8ad-4bb5-5e8f-08dc65077c01 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: =?utf-8?q?I+8TyHoetdndPMF8NL534sGUk3Y1jdX?= =?utf-8?q?H2C6RPhgWod0c6/DvDDV1v7bSrGmZTk51jN45Q0Eh2MXwHX8cEgQc6bna2r0QScmy?= =?utf-8?q?Xd/+ev2m9QZvJjmrOi22bo1wbBKM1k8yYaEDlfrm3B7JvVKHdyHi/RBDhIMBxxJ6h?= =?utf-8?q?JaEzOk4cUEHg93I5/6wVm4AppeiGFD/nqWh8qOPKiPvx96r3dGqqm7qEe3YCiVV/S?= =?utf-8?q?cwfliVnEp/fD/CkJhX+TaapRQ7a/REg7DpPP2Dw/evs/q2E3HIND92U3myJifmCcR?= =?utf-8?q?6QfRU3kaNC7pRcYr81IOTGuELn/uQg1Dd48tnPlzBWYnDBMHkL1yk/FVoNhL4QB3z?= =?utf-8?q?Dd+jndQ/+8UXNmpHCPDsYKvdauhF7lsY2wwp3X+Pc/06ALef7bFGhgNg9TYA5UlHH?= =?utf-8?q?Q9DO49Dzhi/rEcpftalIc2UHu+2CDlUQPMmgAJMOzf6TEQ48BgNTo4AdxJn+xjdQf?= =?utf-8?q?1iOv1n1/TkrWup+bgh1fDr2+QcNZWZTkJhL7tdWT93CcRIEkPdP1z+nJRiYM6iVeQ?= =?utf-8?q?4xTEohRpiOkI5eEuK4El3zowvn2OJU3GzQ3TKjtnh0AFiCXjuRpOWkoARX1uLYVbe?= =?utf-8?q?8ZM5QOsOtCuAlanzysqjS1Wg1s76RyOOPcG/clhKzMbXEm7hdmGAxOSkpVY8qQHJo?= =?utf-8?q?xRdb5flCI35x/FDDsyYMm0nf1/FsWjvHnUUpc+uUGMdsVaNWspBDz2F9pbtVm9/lT?= =?utf-8?q?vThGfqCtlfMK26lnIhbHIv122ppJwy86ldmNdYu1/7dAygPplBnWCwRL9QWFybgkp?= =?utf-8?q?l6R0efr4TRw2sNiFOJg58W1ecD7VeI7EnYv+pK8xJFGe1DRUxE70RAtQc8/w2J/kr?= =?utf-8?q?PpCamgJCw4UOlqEojaTsJZHNqbzcib+7M8a+hlaV+TYH6tJoBRYVQtRPxGk2LWADx?= =?utf-8?q?8a/VT/7qcos8dJx2+drZSyKms8fABUJVycuYS9Lu0Ry5/DjJ1Z5DKHehaEJCYt0su?= =?utf-8?q?7lZCTlg9B2n2MBMxDtvWcfbzpc32aGPZMxs5uSmJPjAdPvAEVtydabTZvbLiK2qd0?= =?utf-8?q?3edb3lt9MmhfMBia+SGuX5HHqC3NDULKTJHeF4oaNklxItdL8wsPe3sChlhlXiAcM?= =?utf-8?q?yDCHVa8pJVq+74d/LfudHvT9v6jPE2dh1la0GJBRhn43bDlaHZPkW/nvcQiwg/MHB?= =?utf-8?q?E8qBvtWsno5OBpNuuRy/4Okr1G9QADE/OChVJN2eDAfxBmT4+90/9chjWqs5PF9Z7?= =?utf-8?q?k871z+zsVhk2kXKhJ3LLnCX3GWav2Z3ZdlAOpzDzOtqgFhewBH6oo+7soaPkKbaeY?= =?utf-8?q?Icnx+Hu2/9yASSzKsx9BKhFhx4dEMvgnEERa5wipONyqyRNOpnsqsAVLoFcc0AyYH?= =?utf-8?q?RhJ7lU6B5Dw8?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(376005)(36860700004)(82310400014)(1800799015); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2024 09:09:59.8471 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a656cc31-d8ad-4bb5-5e8f-08dc65077c01 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF0001E9C2.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR07MB6807 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add atomic bit test/set/clear/assign and test-and-set/clear functions. All atomic bit functions allow (and indeed, require) the caller to specify a memory order. RFC v2: o Add rte_bit_atomic_test_and_assign() (for consistency). o Fix bugs in rte_bit_atomic_test_and_[set|clear](). o Use to support MSVC. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup --- lib/eal/include/rte_bitops.h | 297 +++++++++++++++++++++++++++++++++++ 1 file changed, 297 insertions(+) diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index a2746e657f..8c38a1ac03 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -21,6 +21,7 @@ #include #include +#include #ifdef __cplusplus extern "C" { @@ -342,6 +343,177 @@ extern "C" { uint32_t *: __rte_bit_once_assign32, \ uint64_t *: __rte_bit_once_assign64)(addr, nr, value) +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Test if a particular bit in a word is set with a particular memory + * order. + * + * Test a bit with the resulting memory load ordered as per the + * specified memory order. + * + * @param addr + * A pointer to the word to query. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. See for details. + * @return + * Returns true if the bit is set, and false otherwise. + */ +#define rte_bit_atomic_test(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test32, \ + uint64_t *: __rte_bit_atomic_test64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically set bit in word. + * + * Atomically set bit specified by @c nr in the word pointed to by @c + * addr to '1', with the memory ordering as specified by @c + * memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. See for details. + */ +#define rte_bit_atomic_set(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_set32, \ + uint64_t *: __rte_bit_atomic_set64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically clear bit in word. + * + * Atomically set bit specified by @c nr in the word pointed to by @c + * addr to '0', with the memory ordering as specified by @c + * memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. See for details. + */ +#define rte_bit_atomic_clear(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_clear32, \ + uint64_t *: __rte_bit_atomic_clear64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically assign a value to bit in word. + * + * Atomically set bit specified by @c nr in the word pointed to by @c + * addr to the value indicated by @c value, with the memory ordering + * as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + * @param memory_order + * The memory order to use. See for details. + */ +#define rte_bit_atomic_assign(addr, nr, value, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_assign32, \ + uint64_t *: __rte_bit_atomic_assign64)(addr, nr, value, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and set a bit in word. + * + * Atomically test and set bit specified by @c nr in the word pointed + * to by @c addr to '1', with the memory ordering as specified with @c + * memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. See for details. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_set(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_set32, \ + uint64_t *: __rte_bit_atomic_test_and_set64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and clear a bit in word. + * + * Atomically test and clear bit specified by @c nr in the word + * pointed to by @c addr to '0', with the memory ordering as specified + * with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. See for details. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_clear(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_clear32, \ + uint64_t *: __rte_bit_atomic_test_and_clear64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and assign a bit in word. + * + * Atomically test and assign bit specified by @c nr in the word + * pointed to by @c addr the value specified by @c value, with the + * memory ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + * @param memory_order + * The memory order to use. See for details. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_assign(addr, nr, value, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_assign32, \ + uint64_t *: __rte_bit_atomic_test_and_assign64)(addr, nr, \ + value, \ + memory_order) + #define __RTE_GEN_BIT_TEST(name, size, qualifier) \ static inline bool \ name(const qualifier uint ## size ## _t *addr, unsigned int nr) \ @@ -429,6 +601,131 @@ __rte_bit_once_assign64(volatile uint64_t *addr, unsigned int nr, bool value) __rte_bit_once_clear64(addr, nr); } +#define __RTE_GEN_BIT_ATOMIC_TEST(size) \ + static inline bool \ + __rte_bit_atomic_test ## size(const uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + const RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (const RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + return rte_atomic_load_explicit(a_addr, memory_order) & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_SET(size) \ + static inline void \ + __rte_bit_atomic_set ## size(uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_or_explicit(a_addr, mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ + static inline void \ + __rte_bit_atomic_clear ## size(uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_and_explicit(a_addr, ~mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ + static inline void \ + __rte_bit_atomic_assign ## size(uint ## size ## _t *addr, \ + unsigned int nr, bool value, \ + int memory_order) \ + { \ + if (value) \ + __rte_bit_atomic_set ## size(addr, nr, memory_order); \ + else \ + __rte_bit_atomic_clear ## size(addr, nr, \ + memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \ + static inline bool \ + __rte_bit_atomic_test_and_assign ## size(uint ## size ## _t *addr, \ + unsigned int nr, \ + bool value, \ + int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t before; \ + uint ## size ## _t target; \ + \ + before = rte_atomic_load_explicit(a_addr, \ + rte_memory_order_relaxed); \ + \ + do { \ + target = before; \ + __rte_bit_assign ## size(&target, nr, value); \ + } while (!rte_atomic_compare_exchange_weak_explicit( \ + a_addr, &before, target, \ + rte_memory_order_relaxed, \ + memory_order)); \ + return __rte_bit_test ## size(&before, nr); \ + } + +#define __RTE_GEN_BIT_ATOMIC_OPS(size) \ + __RTE_GEN_BIT_ATOMIC_TEST(size) \ + __RTE_GEN_BIT_ATOMIC_SET(size) \ + __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ + __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) + +__RTE_GEN_BIT_ATOMIC_OPS(32) +__RTE_GEN_BIT_ATOMIC_OPS(64) + +__rte_experimental +static inline bool +__rte_bit_atomic_test_and_set32(uint32_t *addr, unsigned int nr, + int memory_order) +{ + return __rte_bit_atomic_test_and_assign32(addr, nr, true, + memory_order); +} + +__rte_experimental +static inline bool +__rte_bit_atomic_test_and_clear32(uint32_t *addr, unsigned int nr, + int memory_order) +{ + return __rte_bit_atomic_test_and_assign32(addr, nr, false, + memory_order); +} + +__rte_experimental +static inline bool +__rte_bit_atomic_test_and_set64(uint64_t *addr, unsigned int nr, + int memory_order) +{ + return __rte_bit_atomic_test_and_assign64(addr, nr, true, + memory_order); +} + +__rte_experimental +static inline bool +__rte_bit_atomic_test_and_clear64(uint64_t *addr, unsigned int nr, + int memory_order) +{ + return __rte_bit_atomic_test_and_assign64(addr, nr, false, + memory_order); +} + /*------------------------ 32-bit relaxed operations ------------------------*/ /**