From patchwork Fri Aug 9 09:58:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143046 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B484245778; Fri, 9 Aug 2024 12:08:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7413042EF5; Fri, 9 Aug 2024 12:08:11 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2045.outbound.protection.outlook.com [40.107.20.45]) by mails.dpdk.org (Postfix) with ESMTP id B2E65402C4 for ; Fri, 9 Aug 2024 12:08:06 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Chyy4nfqLgnU6Vi9h4u6lMRDAvl2+ZnodXjUIPgngsQ9XGlZHzrc32f5w7pP4dID7MF3wIMSfqUC/Yi9c978TqR0ljlqod/h1wUaS9DAp8/RD1dwWptZT4vr3ZGcScrSdnCE1p1FWAqzWSjMbp73EWn2rM88p8i8LsSb4+lunFO0hNTwhFS2e7FmMz/0d+oYuxWyKm1L0UcVjusJTS2irPAyJLMXvFyETTT7IB0a/1Sh7Jx2BdV79GJVTYzc0Os9QfS/yYc/YK2msthUuI81dXUpZEgWVMjyOL4ASI0h59tbKirsdMrDyofGwKOUgTQcvuIW1r4rgeKeD0+cApEmeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yT9/InfrNKgLxrg+7IONvBCaDCUWOGZoVI+shNYwzrg=; b=F1yY2thUWDhVBnMScxJGxpVPHeSXVGwOBCjpJEeNTZbSqhaV9eTKr8CcU32++g/TDT3osSn5mIiTVIZateWBsPCpaCKOYfC0VDbOBOcieKofJamqupakH49t5TXe2wbskC1bmJO/M60MslYKyMzECumkFGUmSQzFRDuS5PWOYeJzCJMJv8hV46r2J/3wdP4o02KnRz4UGBQTzHIiC58nLGAJ3LOqlxjM2DPcrQ6123FT5krErW9pOHb2e9B+VAHIE3mI0OltNqtPEN7UbpYUc+0Ry98k8NZMQF+txrAsOEKf29Go9OhcWQ9EeApJo5fDfyQ2Ew7Dl3y72PWL5CQF/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yT9/InfrNKgLxrg+7IONvBCaDCUWOGZoVI+shNYwzrg=; b=gaiA1KRtlr49mcLvWxiw9jlxqYwXE+XcDT5UOFIkJGtshzv5Abay5uCRnZue5hXV+0jYQWVbs/1G4UfPoWs44bJu7fJ9AfOCgeeMIQDQdk3g1yH+Ou52EJpUBwhAezpUPOCs1miJVEwgERaZpB5ZuTCG1mIpwTQSp3DmFsf68Ev2vhOGxZtkwooqIHz/f2VcmcBd04XiWU3u3ee5mFCMpIeYGPAt1TWvqxQEoO4g6+Y92b5a9Dp3E1jAZQ8i74k/nQA/ErpAzjWv3DtC8Rtr2y+qb35dlNnr8ODrDIFF2Km9Egw84Be7UwXrl8nMsS3Ptm9AAFTfHMIUNNC0nXjQtQ== Received: from DB6PR0301CA0099.eurprd03.prod.outlook.com (2603:10a6:6:30::46) by VI1PR07MB10042.eurprd07.prod.outlook.com (2603:10a6:800:1d1::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15; Fri, 9 Aug 2024 10:08:04 +0000 Received: from DB1PEPF00039232.eurprd03.prod.outlook.com (2603:10a6:6:30:cafe::d6) by DB6PR0301CA0099.outlook.office365.com (2603:10a6:6:30::46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15 via Frontend Transport; Fri, 9 Aug 2024 10:08:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF00039232.mail.protection.outlook.com (10.167.8.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Fri, 9 Aug 2024 10:08:03 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.67) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 12:08:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id 8D73E1C006B; Fri, 9 Aug 2024 12:08:03 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH v2 1/5] eal: extend bit manipulation functionality Date: Fri, 9 Aug 2024 11:58:25 +0200 Message-ID: <20240809095829.589396-2-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809095829.589396-1-mattias.ronnblom@ericsson.com> References: <20240809090439.589295-2-mattias.ronnblom@ericsson.com> <20240809095829.589396-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF00039232:EE_|VI1PR07MB10042:EE_ X-MS-Office365-Filtering-Correlation-Id: de3dec6c-7911-48a8-0dfa-08dcb85b2877 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?q?qxH+54oXhZuUxb7USf+YjFYSR5Tvxg5?= =?utf-8?q?6ej6NXYiO9/paUaGQBOuvs5H1iG8IkJ5nqx+m4mSpZq5X3hzpC0I0nPxU6iCsz6BW?= =?utf-8?q?R4NlFJTTE9VxnrCzddwFpIByNBFnPKmatYXXnihEoo/I/vnWM0vbMMB0+zZFccKdG?= =?utf-8?q?MZT3vtUZK0WogGO8+qIJrXWFoOhrklEvJvLRjsX+nL/A67jMvJIOxaq0wuHjHFXVu?= =?utf-8?q?yrGQkSV9z7TjC++vV6Nh73PE2M+oEsScGRW0YkHwZW/jC6nhrhFsvaOmtcB2Xxm9u?= =?utf-8?q?s/Q76Kx0xBXV2SkD+ZZrk+xNzzm3ekr2kNNSV5osHs9OFuVOuiMxS5xnt0lR0lC3y?= =?utf-8?q?PqqnTJGlyqg8DyqrDPufx8uoVzedwfi03USdg7QVEJT3lTEt3sYZG7DoqNA3xQvRU?= =?utf-8?q?NB3KTK6EhnYrmxJaPUK4bphnxiTVDldXGk9sjxEyvhLNzJ5lsyk/7hGzvjYds2485?= =?utf-8?q?7OHA6k6Ko2IiipQzXclXwGY7MDOcMauxOKJt5szm/wlYk4e8Y/Fxxmfmp89OSSUAh?= =?utf-8?q?xjpyujEoEAYkWLW7VRGwwI4qVYauvVqqNR3xB26V/34QE9wsiRyDukff42kOvkIey?= =?utf-8?q?S5CEVZNztTnEVA8b1pQ2u1TQ7LhuoHGALQAcM66FPyNhVz+gmMy1FuhaP1QuiD+/f?= =?utf-8?q?HbHcf/LhutZ6ebgs4k4fvpNWUAzxGIsV/hgmMcFnuswEhwvaaeMFAAxISlVKNJMja?= =?utf-8?q?/DmoRI/tXjtnLxfeKMY3b/NjLnnTi8DDJOFy93I0/EOPLYkevBYkFLW8iQCJdqg9I?= =?utf-8?q?BjGwnotKmmFaQ1tkkVM2od7ZFScdioemMAp66RHGUmtXeaxiiV9UNZ73S+E+6gobc?= =?utf-8?q?+c1IWMDfc12pjAlL87cztdMGCQ4c3u6wYbMwoIGxO9Gt0r3+SHZLY8t1r5pmMslM6?= =?utf-8?q?7QxbjwI+uQUyj7rCvGBdUq81ue4hDxZgOqvGwahPts8SvtiMcjDCrwInmY1qhRIDT?= =?utf-8?q?xNavnXGfJBEK0X4PlRnXcOjp2hLwnEs1mjlEhzYcmxNcioRDaSg0sZiFfUWhi9l+3?= =?utf-8?q?pepAk9hcwThQQX3E9sDNZybIU7T2Tlty+zoEdPxC1EmeczdZyT3jp7AjhVONOPm7K?= =?utf-8?q?jMJz/+Zydqf4pwTR7FBcH3vFv17IWuUAV3ZZXPFOdZ3emihthc+9Al4bnSzFJv6lZ?= =?utf-8?q?/ZMp7bJIcFK91MoxlNNHBJQF4s3LOHIcwlL67e5YGl0VEVnUlRohejDef4UcFru1T?= =?utf-8?q?fEzu94vLOnXcGkAMINvZiAVxwLCCk2zLXrM3HEkpjfah2Toy8w1PtWpF1R+TXuq/o?= =?utf-8?q?n6cfwbaFiphCfBImwI7XXIQVkthKTvgn6XBcu4QoVhA+U6ZuHnONOL/IVKHGJzKgn?= =?utf-8?q?nSGnq2aCeKPWDejiA+c5mqBudrfhGqMDSA=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 10:08:03.9256 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: de3dec6c-7911-48a8-0dfa-08dcb85b2877 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF00039232.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR07MB10042 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add functionality to test and modify the value of individual bits in 32-bit or 64-bit words. These functions have no implications on memory ordering, atomicity and does not use volatile and thus does not prevent any compiler optimizations. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- RFC v6: * Have rte_bit_test() accept const-marked bitsets. RFC v4: * Add rte_bit_flip() which, believe it or not, flips the value of a bit. * Mark macro-generated private functions as experimental. * Use macros to generate *assign*() functions. RFC v3: * Work around lack of C++ support for _Generic (Tyler Retzlaff). * Fix ','-related checkpatch warnings. --- lib/eal/include/rte_bitops.h | 259 ++++++++++++++++++++++++++++++++++- 1 file changed, 257 insertions(+), 2 deletions(-) diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 449565eeae..3297133e22 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -2,6 +2,7 @@ * Copyright(c) 2020 Arm Limited * Copyright(c) 2010-2019 Intel Corporation * Copyright(c) 2023 Microsoft Corporation + * Copyright(c) 2024 Ericsson AB */ #ifndef _RTE_BITOPS_H_ @@ -11,12 +12,14 @@ * @file * Bit Operations * - * This file defines a family of APIs for bit operations - * without enforcing memory ordering. + * This file provides functionality for low-level, single-word + * arithmetic and bit-level operations, such as counting or + * setting individual bits. */ #include +#include #include #ifdef __cplusplus @@ -105,6 +108,196 @@ extern "C" { #define RTE_FIELD_GET64(mask, reg) \ ((typeof(mask))(((reg) & (mask)) >> rte_ctz64(mask))) +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Test bit in word. + * + * Generic selection macro to test the value of a bit in a 32-bit or + * 64-bit word. The type of operation depends on the type of the @c + * addr parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_test(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_test32, \ + const uint32_t *: __rte_bit_test32, \ + uint64_t *: __rte_bit_test64, \ + const uint64_t *: __rte_bit_test64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set bit in word. + * + * Generic selection macro to set a bit in a 32-bit or 64-bit + * word. The type of operation depends on the type of the @c addr + * parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_set(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_set32, \ + uint64_t *: __rte_bit_set64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Clear bit in word. + * + * Generic selection macro to clear a bit in a 32-bit or 64-bit + * word. The type of operation depends on the type of the @c addr + * parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_clear(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_clear32, \ + uint64_t *: __rte_bit_clear64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Assign a value to a bit in word. + * + * Generic selection macro to assign a value to a bit in a 32-bit or 64-bit + * word. The type of operation depends on the type of the @c addr parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + */ +#define rte_bit_assign(addr, nr, value) \ + _Generic((addr), \ + uint32_t *: __rte_bit_assign32, \ + uint64_t *: __rte_bit_assign64)(addr, nr, value) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Flip a bit in word. + * + * Generic selection macro to change the value of a bit to '0' if '1' + * or '1' if '0' in a 32-bit or 64-bit word. The type of operation + * depends on the type of the @c addr parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_flip(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_flip32, \ + uint64_t *: __rte_bit_flip64)(addr, nr) + +#define __RTE_GEN_BIT_TEST(family, fun, qualifier, size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_ ## family ## fun ## size(const qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + RTE_ASSERT(nr < size); \ + \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + return *addr & mask; \ + } + +#define __RTE_GEN_BIT_SET(family, fun, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## family ## fun ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + RTE_ASSERT(nr < size); \ + \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + *addr |= mask; \ + } \ + +#define __RTE_GEN_BIT_CLEAR(family, fun, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## family ## fun ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + RTE_ASSERT(nr < size); \ + \ + uint ## size ## _t mask = ~((uint ## size ## _t)1 << nr); \ + (*addr) &= mask; \ + } \ + +#define __RTE_GEN_BIT_ASSIGN(family, fun, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## family ## fun ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, bool value) \ + { \ + if (value) \ + __rte_bit_ ## family ## set ## size(addr, nr); \ + else \ + __rte_bit_ ## family ## clear ## size(addr, nr); \ + } + +#define __RTE_GEN_BIT_FLIP(family, fun, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## family ## fun ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + bool value; \ + \ + value = __rte_bit_ ## family ## test ## size(addr, nr); \ + __rte_bit_ ## family ## assign ## size(addr, nr, !value); \ + } + +__RTE_GEN_BIT_TEST(, test,, 32) +__RTE_GEN_BIT_SET(, set,, 32) +__RTE_GEN_BIT_CLEAR(, clear,, 32) +__RTE_GEN_BIT_ASSIGN(, assign,, 32) +__RTE_GEN_BIT_FLIP(, flip,, 32) + +__RTE_GEN_BIT_TEST(, test,, 64) +__RTE_GEN_BIT_SET(, set,, 64) +__RTE_GEN_BIT_CLEAR(, clear,, 64) +__RTE_GEN_BIT_ASSIGN(, assign,, 64) +__RTE_GEN_BIT_FLIP(, flip,, 64) + /*------------------------ 32-bit relaxed operations ------------------------*/ /** @@ -787,6 +980,68 @@ rte_log2_u64(uint64_t v) #ifdef __cplusplus } + +/* + * Since C++ doesn't support generic selection (i.e., _Generic), + * function overloading is used instead. Such functions must be + * defined outside 'extern "C"' to be accepted by the compiler. + */ + +#undef rte_bit_test +#undef rte_bit_set +#undef rte_bit_clear +#undef rte_bit_assign +#undef rte_bit_flip + +#define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ + static inline void \ + rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ + arg1_type arg1_name) \ + { \ + __rte_bit_ ## fun ## size(addr, arg1_name); \ + } + +#define __RTE_BIT_OVERLOAD_2(fun, qualifier, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 32, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 64, arg1_type, arg1_name) + +#define __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, size, ret_type, arg1_type, \ + arg1_name) \ + static inline ret_type \ + rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ + arg1_type arg1_name) \ + { \ + return __rte_bit_ ## fun ## size(addr, arg1_name); \ + } + +#define __RTE_BIT_OVERLOAD_2R(fun, qualifier, ret_type, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 32, ret_type, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 64, ret_type, arg1_type, \ + arg1_name) + +#define __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + static inline void \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name) \ + { \ + __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + } + +#define __RTE_BIT_OVERLOAD_3(fun, qualifier, arg1_type, arg1_name, arg2_type, \ + arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 32, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 64, arg1_type, arg1_name, \ + arg2_type, arg2_name) + +__RTE_BIT_OVERLOAD_2R(test, const, bool, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(set,, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) +__RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) +__RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) + #endif #endif /* _RTE_BITOPS_H_ */ From patchwork Fri Aug 9 09:58:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143044 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6200245778; Fri, 9 Aug 2024 12:08:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0A91842EE4; Fri, 9 Aug 2024 12:08:09 +0200 (CEST) Received: from EUR02-AM0-obe.outbound.protection.outlook.com (mail-am0eur02on2054.outbound.protection.outlook.com [40.107.247.54]) by mails.dpdk.org (Postfix) with ESMTP id 5A415402C4 for ; Fri, 9 Aug 2024 12:08:06 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fgry8m0RfzNBEYtyluKnsj+3Nm/3orOXDQ4kqhtiSeSG1bUz2R0HwH2342s9c38VFolj+HmdKLRroZrDONCEF6Yie6t0xaGihsjlKYeg4qNGUhh0QkJeW8HDa0S059+76sOvE3S5ptSzqAk0g/kmVGiaY/ZhfkZKF2i6qqGPphR0OpzI1VBPObHjblk3EKlhBknO8cQzCWCmcohOpO3o913/o1RG17VGc25WYlbeAkOecdQscx6BFLRcWBPeHTRVcnWxdVW6zfePvSB7Zu2SXgD98gtzlRrxRcwSI/Ol0oEiIpMfFRdiD2ezJRcME5qvR6IxIn3cEJoYWcbUjVSxkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rlj6TbwktGuphxsLT7E1iv4WZ/+LLJlVUku+K2BR9LA=; b=kMPosZDLecDhl+2N6fUW+L8YW/PbwQJsbV2qe+QeZARc8S2zFEbvaJYWmwoELN7E9ACg87NX6lMt8Y6DeD2PHs4sMJRmLQ2Lghk5ROS+wlbiARpp45/IrGg3D0cDQXEoSWcrICHKktqSV2GdbZzzMwvy7grjRMIuKZxYp1X0byVWWo4kQkhfVjdvet/Pz95NdT0/tR55MXAG81vXl/VDoeOsNyLs+ivmTLBT8Wzr51Dz0B+3i134vP5v/f2VNcKFLPLbRpAVHuMapNnpu24VzTWXIOasttIYZYhDb7PUxjww0hDK1/n+WMS5aZTXLb1gybjjQhseFqqP1w/RDfwYAA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlj6TbwktGuphxsLT7E1iv4WZ/+LLJlVUku+K2BR9LA=; b=rnlvAkQfSmjKyGx+mhWEOHyWF0O9kOItUet8LFFsJ/Jpe1m9gT++c8uDvFMKJX4NclHnQe064s2cNgkI0LM+jWlpnI9zhqlre1qaKSDwzefs34VTnjao2Ed8jcNQQrLMfOopop2yBfsOswF5f2c+OuZCwFHxjSnvFRYhpoVuogUK1aLdify9FIJy+QKTufsRD+J5Xx1ftEwb4t9Bg1ITjwnNEoXrrNWJYqHSG+/yD3Ow7WcnhVV2AflIHI5QYw1LvuHHlLwC07S4lvYFgitfv0WA2LJ/1p5VKpFHx60gGrUXfP3QWj9vNJkQyoeGR3t8gMSudTzTkRkjyyYBCDQn7Q== Received: from DB6PR0301CA0076.eurprd03.prod.outlook.com (2603:10a6:6:30::23) by DB9PR07MB7882.eurprd07.prod.outlook.com (2603:10a6:10:2ae::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15; Fri, 9 Aug 2024 10:08:04 +0000 Received: from DB1PEPF00039232.eurprd03.prod.outlook.com (2603:10a6:6:30:cafe::30) by DB6PR0301CA0076.outlook.office365.com (2603:10a6:6:30::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.14 via Frontend Transport; Fri, 9 Aug 2024 10:08:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF00039232.mail.protection.outlook.com (10.167.8.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Fri, 9 Aug 2024 10:08:04 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.67) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 12:08:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id 9E72C1C006D; Fri, 9 Aug 2024 12:08:03 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH v2 2/5] eal: add unit tests for bit operations Date: Fri, 9 Aug 2024 11:58:26 +0200 Message-ID: <20240809095829.589396-3-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809095829.589396-1-mattias.ronnblom@ericsson.com> References: <20240809090439.589295-2-mattias.ronnblom@ericsson.com> <20240809095829.589396-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF00039232:EE_|DB9PR07MB7882:EE_ X-MS-Office365-Filtering-Correlation-Id: 1c3722a7-186d-496d-a628-08dcb85b28f0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|82310400026|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?q?f2qrs/hIYfyeoWwrt8Fbr2Ym8BToEDx?= =?utf-8?q?YUbg9XmaXhMsMkdgHAEItIUrBXdUOqJtdjX1+x1UFP6GybStx1XU6t2KfQqRLXZHK?= =?utf-8?q?CG6F841aXbdkKKD5ipLCzr9lMMiG7mWVe8h99ZJ8nzgrKtClIjGjE7c8PqUZiRGIe?= =?utf-8?q?OVxI0zZbyqsr+RY63cVqTUBTZlKOZZ0FzyFvvnvY7jfxNJO47hOJ2ujFKPXlolC2r?= =?utf-8?q?WSpx5Q3NPgDCt3/k1WwCBooWgRUdGwfInKHc2L4KpqfAXD6pOgg0K+H/xzd4vNpb/?= =?utf-8?q?M1gkOieH4FlPNq/Y9qgtA5ykW5uq/T1iJRD5a+2Vz3j2gXSv6z7ofaVY2k2VWZfQC?= =?utf-8?q?eLTSifKSviY4m7R/E4grENn7ui98f/xAZfaxqpV9MWrcEYs6mgAwmsyHoap3Pug7J?= =?utf-8?q?5WNuh99vdrKqnJmuGMlxPTKVH/LdMGZ5q9jIgy8vPiGI4/DiF37BLtKmaiJVN5o6/?= =?utf-8?q?RoDVMYbOxA9mrc9ulXrFb2dxh1QHER7lCD4CwmwjTj3RZDxyXOoU41h+PCh+2XmKW?= =?utf-8?q?/q/9NOlsvLCskHS8Hd+nSEXCKMwfdA+DMOQBZoAfnYPt8MW13trHU1ySXy+74E+zn?= =?utf-8?q?u9XkvAD1RA7yDuWNjROjDjbpSkZTzWhM8huWMiYHGLi7Eff1JCCEUvsOfwLY7PgSN?= =?utf-8?q?ShhKQaFqmiTd5+0kUhr/W93bmtryOyzoqG9HyS7Gz8j3Oc68ycjsrIAr97CLvotox?= =?utf-8?q?zYIIoYLrCgoqMC3LKUCmcUtVUkqKS5DMURdXxCqseTIYZvc14THO7qVB7vJTJeDBZ?= =?utf-8?q?UFvjxJq9/wMymQ3xlPGacs3x8WC+aZK6b/OJbJm/ekJvrYrd5kqgnpXo0gkPnPbv+?= =?utf-8?q?/IuzEnTjQAXBPK6/VGuTNlyw44r6QhCgybVVIxqk9n/IHqj37u5nyhOcytjwP+xlF?= =?utf-8?q?0q8eMTmle+9164iv63pHLH1F+Vm4V2H19TwuBOz8BGVJrw0x6cXnj6Pr9WxHJ/4pU?= =?utf-8?q?Ypu5y3+0EsxJ3oASXxJmDW3gyjALw8Pp07x3EfwL/MVhbjZG/ytYdRN90CfckQBXW?= =?utf-8?q?a01U0/HoaU2cjxQ+0RBu2JaLE/el0apJHI/b7OzsnO9ls7LYUvgPnopikburidfYg?= =?utf-8?q?gBk/a9vWI+RBOa8twUfZFXBOKUY+9vyf8v3GAqfgZuz4wrrSYjQSk48j9G0eb0tzF?= =?utf-8?q?wfKIht3C7t2v5lyLPBAzyk1uqdC33nYTKELB5qktkgp5X5ddR6qCzadDTGNJVQgQE?= =?utf-8?q?M60oyeC31Ayt6qZOC3chNEaFrQ9+2ImNPW9yZfiKmk6QcM07ShMHsKYbKwumsma6c?= =?utf-8?q?XDPnjGdOqaa855E0atIYZhkpH/PmzrgGOqZQH74m/8TKEg+DfDNWG+lq6eWlag4dj?= =?utf-8?q?/ppNnA1MHz+b4GA7U4D9WGQ0wZd2klLtmw=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(36860700013)(82310400026)(376014)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 10:08:04.7381 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1c3722a7-186d-496d-a628-08dcb85b28f0 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF00039232.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR07MB7882 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Extend bitops tests to cover the rte_bit_[test|set|clear|assign|flip]() functions. The tests are converted to use the test suite runner framework. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- RFC v6: * Test rte_bit_*test() usage through const pointers. RFC v4: * Remove redundant line continuations. --- app/test/test_bitops.c | 85 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 70 insertions(+), 15 deletions(-) diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c index 0d4ccfb468..322f58c066 100644 --- a/app/test/test_bitops.c +++ b/app/test/test_bitops.c @@ -1,13 +1,68 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2019 Arm Limited + * Copyright(c) 2024 Ericsson AB */ +#include + #include #include +#include #include "test.h" -uint32_t val32; -uint64_t val64; +#define GEN_TEST_BIT_ACCESS(test_name, set_fun, clear_fun, assign_fun, \ + flip_fun, test_fun, size) \ + static int \ + test_name(void) \ + { \ + uint ## size ## _t reference = (uint ## size ## _t)rte_rand(); \ + unsigned int bit_nr; \ + uint ## size ## _t word = (uint ## size ## _t)rte_rand(); \ + \ + for (bit_nr = 0; bit_nr < size; bit_nr++) { \ + bool reference_bit = (reference >> bit_nr) & 1; \ + bool assign = rte_rand() & 1; \ + if (assign) \ + assign_fun(&word, bit_nr, reference_bit); \ + else { \ + if (reference_bit) \ + set_fun(&word, bit_nr); \ + else \ + clear_fun(&word, bit_nr); \ + \ + } \ + TEST_ASSERT(test_fun(&word, bit_nr) == reference_bit, \ + "Bit %d had unexpected value", bit_nr); \ + flip_fun(&word, bit_nr); \ + TEST_ASSERT(test_fun(&word, bit_nr) != reference_bit, \ + "Bit %d had unflipped value", bit_nr); \ + flip_fun(&word, bit_nr); \ + \ + const uint ## size ## _t *const_ptr = &word; \ + TEST_ASSERT(test_fun(const_ptr, bit_nr) == \ + reference_bit, \ + "Bit %d had unexpected value", bit_nr); \ + } \ + \ + for (bit_nr = 0; bit_nr < size; bit_nr++) { \ + bool reference_bit = (reference >> bit_nr) & 1; \ + TEST_ASSERT(test_fun(&word, bit_nr) == reference_bit, \ + "Bit %d had unexpected value", bit_nr); \ + } \ + \ + TEST_ASSERT(reference == word, "Word had unexpected value"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_ACCESS(test_bit_access32, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 32) + +GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 64) + +static uint32_t val32; +static uint64_t val64; #define MAX_BITS_32 32 #define MAX_BITS_64 64 @@ -117,22 +172,22 @@ test_bit_relaxed_test_set_clear(void) return TEST_SUCCESS; } +static struct unit_test_suite test_suite = { + .suite_name = "Bitops test suite", + .unit_test_cases = { + TEST_CASE(test_bit_access32), + TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_relaxed_set), + TEST_CASE(test_bit_relaxed_clear), + TEST_CASE(test_bit_relaxed_test_set_clear), + TEST_CASES_END() + } +}; + static int test_bitops(void) { - val32 = 0; - val64 = 0; - - if (test_bit_relaxed_set() < 0) - return TEST_FAILED; - - if (test_bit_relaxed_clear() < 0) - return TEST_FAILED; - - if (test_bit_relaxed_test_set_clear() < 0) - return TEST_FAILED; - - return TEST_SUCCESS; + return unit_test_suite_runner(&test_suite); } REGISTER_FAST_TEST(bitops_autotest, true, true, test_bitops); From patchwork Fri Aug 9 09:58:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143047 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E892E45778; Fri, 9 Aug 2024 12:08:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8729742F08; Fri, 9 Aug 2024 12:08:13 +0200 (CEST) Received: from EUR02-AM0-obe.outbound.protection.outlook.com (mail-am0eur02on2074.outbound.protection.outlook.com [40.107.247.74]) by mails.dpdk.org (Postfix) with ESMTP id D83D142EE1 for ; Fri, 9 Aug 2024 12:08:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=G+Y4QSuN70UYW5fcr5DbZmPisCyE11rJp7vc+b2bWKPAmbkjnIJNwkBm9zwHeT7Y+3OCifsjcyPCACSvr8LrE62JwEf7tiYYv2onkFLwikWLY3XExru3fJgfE7b7erGjcxmFC4n3ER1NzwZK9yX/RkoT0qDjofBpm5yrIWeinMnC7/Smq4/zoQL4Y8wgf42odt3SiOH7v+STFopJcGfJfUckdHw1JCZD3AiySOx6lz87EYS49Y7CMJYFaxpc7foppqgpMblaWOM05GxQdvHWrcBmVzaYEVlcxOXigINAsuCZsaT+fZ1cX7XmzTr3CV4/sqXlCsuK/0o+1Ld/hhhyew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tBtZmE6Fft2GxhfCFsHi3dKep9HGbo7iaXeN1b6hyi4=; b=Kq1AWA7V+PgZbVIKEovweGgOZQqy5KFMLN0ue9vTrv1HQnuepPaeH6L5phk0aqcTUjnsug15JaTymoOCPO80RuzGiyE0oPqg5k1lm8HSQg+jTN1EMJRutNdUGExHroZU2B4XBG4xBQY0k9FXIKR9KAbr6omFrlM7oCljbNCl3AnVYAMtl7mIbsYgaAExSB1iu1wUXY/m7ZU6V9ks7D+uEuokaBdQ3tZWZvc+lVl5S99nhxFSU7gSCt5L3X97VEcq7UnbKgie+SJbnncoEvBZe0bv9OBHrD722h3YyuFAVbMTHjN4SZfIS3/Pso3k6Al6rx6nXL1dM3IoPRhEJt19Vw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tBtZmE6Fft2GxhfCFsHi3dKep9HGbo7iaXeN1b6hyi4=; b=kh1/mBQEhHl5ZSv6xqNPSNgP+MuDaY/FMkLAGd2ByTLhQSEtRSoBD3bCHfWp9bbYghcgkH+dh6sjFcYuvqdboUqLZEYd1G9Uc4oCZb7VqtkdTHW1GS+A6n2XYzYJyJwSYFQY4MYFUwdK/XeQGMUKbcU/kqCtmRWt5DTORQbE40rIa27dhaZGeoSKOsYZNnmeFz9z4LH2tPOljn9djPQ5AxaFSwykq5pfY0A1a+HDNMEuNQKf9cQnbGEMpQJ2nifPS918sSLIP4Mc9BrKmL65uURCWzZUKiiIhPRTjYMEZKZcd2gq/6bMx6bU41ARAd7kHFW0sbQQeOACCWCnAjHm+g== Received: from DB6PR0301CA0068.eurprd03.prod.outlook.com (2603:10a6:6:30::15) by AS8PR07MB7830.eurprd07.prod.outlook.com (2603:10a6:20b:38f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15; Fri, 9 Aug 2024 10:08:05 +0000 Received: from DB1PEPF00039232.eurprd03.prod.outlook.com (2603:10a6:6:30:cafe::fc) by DB6PR0301CA0068.outlook.office365.com (2603:10a6:6:30::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.14 via Frontend Transport; Fri, 9 Aug 2024 10:08:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF00039232.mail.protection.outlook.com (10.167.8.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Fri, 9 Aug 2024 10:08:05 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.67) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 12:08:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id AE9EF1C0070; Fri, 9 Aug 2024 12:08:03 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH v2 3/5] eal: add atomic bit operations Date: Fri, 9 Aug 2024 11:58:27 +0200 Message-ID: <20240809095829.589396-4-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809095829.589396-1-mattias.ronnblom@ericsson.com> References: <20240809090439.589295-2-mattias.ronnblom@ericsson.com> <20240809095829.589396-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF00039232:EE_|AS8PR07MB7830:EE_ X-MS-Office365-Filtering-Correlation-Id: 218abe93-5df5-4be5-908f-08dcb85b291b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|1800799024|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?q?ar1J+wzimh94tN7lt1Bx7vA6iqxQXOX?= =?utf-8?q?SSs9sFbHmgUDPpLm55Z2oGZ0VamMTs0hxL4LJ5vn/asx3+aSdyOmMKfvsdOxM5fK8?= =?utf-8?q?D46R1vWJp4KNaTPyVMDOS9hx4y3K5PeJlhLx7MgrM8ueFLjIfhWsQnM3BL0LZkvbx?= =?utf-8?q?E19AjNnJfa76w2txjz4106t+9Dt6zuF0YPGgMttP/06hKkJEH8Hl9hVKgsjr6CLlo?= =?utf-8?q?qA3bRz5EjigQQVp5XFQ0IRUEp5RHHQv1BZSZStydyEY2bY0zFwiZy/JRbLCjZoTyR?= =?utf-8?q?h7wauGNJcsT4tMzL1pynYSevou5XjpZvK1Mf5iWpuNRyC8jSU52+v4P8TyDIXFWYf?= =?utf-8?q?QRJO7k9hYlD2gBwpnjvoVybl/xpfk+840kMAVla6+tjpcJeGglmzuLTivh9kOKz2v?= =?utf-8?q?5WQuu/jFmjecue2fs83Lz+m7ZNo4BuXtAZxf2JV5hZ4tOKpCzmJa1Mk/jHv/2E8nX?= =?utf-8?q?3LPOYxAwh6jdBXvdLWQe9XUiDsEMggPuUSotlwHZi/k8EeDo3T9eVSf1Z9hOFmxKW?= =?utf-8?q?UVytaMslpWJPD5VDb0B/3UGkxXBD9P10B2MBgvv8NIfwnqsLNmVOfNbCxon+dVgTl?= =?utf-8?q?dbR8Z3e5LXfWdWAtvgnFlR3hSO+W9K9ehjfE20IcLSGeojOV6n6HGJ+xzxH5UpXG9?= =?utf-8?q?lxvq2/GKJIiT8o73dqDcpKQz+xLP3rjTjNlpHVNv2D7HwiaQCJs+aGZ+OhRH/h2M/?= =?utf-8?q?tsrIq6qK+ShF18qNzkKNQT7XFQQqQFaNZbaQppxGyTCsO1nwmnO8NVaj5kPolPDrr?= =?utf-8?q?MjHIVcXxRJEr0yFFrANEi5lVvy+oQCSUSoqPy4KIzfrw79ZKMyUeY739YBiwbp1/E?= =?utf-8?q?1x9ImsyXyFGMceP9fGPYnGXm9GEJudLkmAI9mdGLb5B5wyEHlpaGicwNnhvMj8KhE?= =?utf-8?q?NbXRs/bNTiUW2eteEHwPy8mVgnVnxkAyxKRuVCK4yXwH/EAALOO7dU3rPtYRpSZT/?= =?utf-8?q?o8a1vXxiOAhYTysKlk9bKqtWPZnlWCzV/aye4I45jR4YYSPwjS2QjIKuRmiwzqlxr?= =?utf-8?q?zDFdYDEfoClGmSXtt/n5jHXkeuBUuWg4P91pdaIyL3FzWQ4RvajUCzbLVC4mAlqLq?= =?utf-8?q?//MRYGs5pOjoYgZ6aLcvRagieNSAnn9cOiIagkJdEP01FTF/iiWEUXdWTGQczyrEn?= =?utf-8?q?DPZMMCiiceEyt3JfcY5lsa1Sh6CR+AmyztrbwTUVJcZA8Yt6IfsORnjbqye+u4zJ2?= =?utf-8?q?SjTBARg7JSw4e7rxsvLK3RP9z/DpXw/A8v/7zVF0tZFUapNRYcYfjNKM/YoBo6cug?= =?utf-8?q?RGuyc+Ejp8yl7Aao42Td/aCvk6FkIcwYVqOJa/MLH79r3c2NkFi3qq27REtDr+HQ7?= =?utf-8?q?IZTttS6Mby0VMpgVGY16BmJnbcF1vtPOPw=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(36860700013)(1800799024)(376014)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 10:08:05.0193 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 218abe93-5df5-4be5-908f-08dcb85b291b X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF00039232.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR07MB7830 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add atomic bit test/set/clear/assign/flip and test-and-set/clear/assign/flip functions. All atomic bit functions allow (and indeed, require) the caller to specify a memory order. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- PATCH: * Add missing macro #undef for C++ version of atomic bit flip. RFC v7: * Replace compare-exchange-based rte_bitset_atomic_test_and_*() and flip() with implementations that use the previous value as returned by the atomic fetch function. * Reword documentation to match the non-atomic macro variants. * Remove pointer to for memory model documentation, since there is no documentation for that API. RFC v6: * Have rte_bit_atomic_test() accept const-marked bitsets. RFC v4: * Add atomic bit flip. * Mark macro-generated private functions experimental. RFC v3: * Work around lack of C++ support for _Generic (Tyler Retzlaff). RFC v2: o Add rte_bit_atomic_test_and_assign() (for consistency). o Fix bugs in rte_bit_atomic_test_and_[set|clear](). o Use to support MSVC. --- lib/eal/include/rte_bitops.h | 414 +++++++++++++++++++++++++++++++++++ 1 file changed, 414 insertions(+) diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 3297133e22..4d878099ed 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -21,6 +21,7 @@ #include #include +#include #ifdef __cplusplus extern "C" { @@ -226,6 +227,204 @@ extern "C" { uint32_t *: __rte_bit_flip32, \ uint64_t *: __rte_bit_flip64)(addr, nr) +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Test if a particular bit in a word is set with a particular memory + * order. + * + * Test a bit with the resulting memory load ordered as per the + * specified memory order. + * + * @param addr + * A pointer to the word to query. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit is set, and false otherwise. + */ +#define rte_bit_atomic_test(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test32, \ + const uint32_t *: __rte_bit_atomic_test32, \ + uint64_t *: __rte_bit_atomic_test64, \ + const uint64_t *: __rte_bit_atomic_test64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically set bit in word. + * + * Generic selection macro to atomically set bit specified by @c nr in + * the word pointed to by @c addr to '1', with the memory ordering as + * specified by @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_set(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_set32, \ + uint64_t *: __rte_bit_atomic_set64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically clear bit in word. + * + * Generic selection macro to atomically set bit specified by @c nr in + * the word pointed to by @c addr to '0', with the memory ordering as + * specified by @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_clear(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_clear32, \ + uint64_t *: __rte_bit_atomic_clear64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically assign a value to bit in word. + * + * Generic selection macro to atomically set bit specified by @c nr in the + * word pointed to by @c addr to the value indicated by @c value, with + * the memory ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_assign(addr, nr, value, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_assign32, \ + uint64_t *: __rte_bit_atomic_assign64)(addr, nr, value, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically flip bit in word. + * + * Generic selection macro to atomically negate the value of the bit + * specified by @c nr in the word pointed to by @c addr to the value + * indicated by @c value, with the memory ordering as specified with + * @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_flip(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_flip32, \ + uint64_t *: __rte_bit_atomic_flip64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and set a bit in word. + * + * Generic selection macro to atomically test and set bit specified by + * @c nr in the word pointed to by @c addr to '1', with the memory + * ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_set(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_set32, \ + uint64_t *: __rte_bit_atomic_test_and_set64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and clear a bit in word. + * + * Generic selection macro to atomically test and clear bit specified + * by @c nr in the word pointed to by @c addr to '0', with the memory + * ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_clear(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_clear32, \ + uint64_t *: __rte_bit_atomic_test_and_clear64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and assign a bit in word. + * + * Generic selection macro to atomically test and assign bit specified + * by @c nr in the word pointed to by @c addr the value specified by + * @c value, with the memory ordering as specified with @c + * memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_assign(addr, nr, value, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_assign32, \ + uint64_t *: __rte_bit_atomic_test_and_assign64)(addr, nr, \ + value, \ + memory_order) + #define __RTE_GEN_BIT_TEST(family, fun, qualifier, size) \ __rte_experimental \ static inline bool \ @@ -298,6 +497,145 @@ __RTE_GEN_BIT_CLEAR(, clear,, 64) __RTE_GEN_BIT_ASSIGN(, assign,, 64) __RTE_GEN_BIT_FLIP(, flip,, 64) +#define __RTE_GEN_BIT_ATOMIC_TEST(size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_test ## size(const uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + const RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (const RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + return rte_atomic_load_explicit(a_addr, memory_order) & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_SET(size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_set ## size(uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_or_explicit(a_addr, mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_clear ## size(uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_and_explicit(a_addr, ~mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_FLIP(size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_flip ## size(uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_xor_explicit(a_addr, mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_assign ## size(uint ## size ## _t *addr, \ + unsigned int nr, bool value, \ + int memory_order) \ + { \ + if (value) \ + __rte_bit_atomic_set ## size(addr, nr, memory_order); \ + else \ + __rte_bit_atomic_clear ## size(addr, nr, \ + memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_test_and_set ## size(uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + uint ## size ## _t prev; \ + \ + prev = rte_atomic_fetch_or_explicit(a_addr, mask, \ + memory_order); \ + \ + return prev & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_test_and_clear ## size(uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + uint ## size ## _t prev; \ + \ + prev = rte_atomic_fetch_and_explicit(a_addr, ~mask, \ + memory_order); \ + \ + return prev & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_test_and_assign ## size(uint ## size ## _t *addr, \ + unsigned int nr, \ + bool value, \ + int memory_order) \ + { \ + if (value) \ + return __rte_bit_atomic_test_and_set ## size(addr, nr, \ + memory_order); \ + else \ + return __rte_bit_atomic_test_and_clear ## size(addr, nr, \ + memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_OPS(size) \ + __RTE_GEN_BIT_ATOMIC_TEST(size) \ + __RTE_GEN_BIT_ATOMIC_SET(size) \ + __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ + __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \ + __RTE_GEN_BIT_ATOMIC_FLIP(size) + +__RTE_GEN_BIT_ATOMIC_OPS(32) +__RTE_GEN_BIT_ATOMIC_OPS(64) + /*------------------------ 32-bit relaxed operations ------------------------*/ /** @@ -993,6 +1331,15 @@ rte_log2_u64(uint64_t v) #undef rte_bit_assign #undef rte_bit_flip +#undef rte_bit_atomic_test +#undef rte_bit_atomic_set +#undef rte_bit_atomic_clear +#undef rte_bit_atomic_assign +#undef rte_bit_atomic_flip +#undef rte_bit_atomic_test_and_set +#undef rte_bit_atomic_test_and_clear +#undef rte_bit_atomic_test_and_assign + #define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ static inline void \ rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ @@ -1036,12 +1383,79 @@ rte_log2_u64(uint64_t v) __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 64, arg1_type, arg1_name, \ arg2_type, arg2_name) +#define __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) \ + static inline ret_type \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name) \ + { \ + return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + } + +#define __RTE_BIT_OVERLOAD_3R(fun, qualifier, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 32, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 64, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) + +#define __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, size, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + static inline void \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name, arg3_type arg3_name) \ + { \ + __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ + arg3_name); \ + } + +#define __RTE_BIT_OVERLOAD_4(fun, qualifier, arg1_type, arg1_name, arg2_type, \ + arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 32, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 64, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) + +#define __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + static inline ret_type \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name, arg3_type arg3_name) \ + { \ + return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ + arg3_name); \ + } + +#define __RTE_BIT_OVERLOAD_4R(fun, qualifier, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 32, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 64, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) + __RTE_BIT_OVERLOAD_2R(test, const, bool, unsigned int, nr) __RTE_BIT_OVERLOAD_2(set,, unsigned int, nr) __RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) __RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) __RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) +__RTE_BIT_OVERLOAD_3R(atomic_test, const, bool, unsigned int, nr, + int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_set,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_clear,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_4(atomic_assign,, unsigned int, nr, bool, value, + int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_flip,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3R(atomic_test_and_set,, bool, unsigned int, nr, + int, memory_order) +__RTE_BIT_OVERLOAD_3R(atomic_test_and_clear,, bool, unsigned int, nr, + int, memory_order) +__RTE_BIT_OVERLOAD_4R(atomic_test_and_assign,, bool, unsigned int, nr, + bool, value, int, memory_order) + #endif #endif /* _RTE_BITOPS_H_ */ From patchwork Fri Aug 9 09:58:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143045 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 74CC045778; Fri, 9 Aug 2024 12:08:19 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4684C42EEB; Fri, 9 Aug 2024 12:08:10 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2064.outbound.protection.outlook.com [40.107.20.64]) by mails.dpdk.org (Postfix) with ESMTP id 5062142EB8 for ; Fri, 9 Aug 2024 12:08:06 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tEo0OqYRSOZSQqh0KPDxRjdbg/HKWk0TS8Dy81m6ivf8HOSOW9lMRCzU7tIi3pG07awWyaaabhn2G+3CQ8T2xCl4ML83lPzD75/yyeINhtsITIb4fz+yvBgMFFYkwRy/DA6wCIUlcupez8SvlzgxMvuxWbh+CssCzkim/nI9e3RirMaTFmlYuYihRNCgb3gRF3An1mae9H/XS+rgNcbbRbaKqmIzEBMjdfT1V4MpN+yFTBtHrGqIHO547o2WL2lUUlcDHe+Lr2opBfJ4j3s3Zhu+AQ8xHNMX5AW1wlPI8hAN6U6l8gEx9UIOuuxXvFvxut/YkkYkXXxcz4xL/x/ZDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a5WMo7gTsm2Qr9ZTxjtFLLnl+2HwDjGCyFIPlASvPhI=; b=Kd3zEHhaF+ZcrgD48Kap4V6gpyY3xxAImHraK5iDkO5eY1PGt9ilLQcWhvwqeWDOjm1ftbyWWiupkbIItPNUq8jmpgintX3j49q44YyEC1QOa9ho5uhuiPddqcs0z9WkzRjybYKscHVUGDNFY/FD3A19JP7FhVlZ7FHdmVUFyj+KIbFaVOR+fuhHnFDs2MA3UCGC+xj7Ej6agFrq28r3Cx28X/UoX99EVm1n8AhR5QMci0HDT315XpPeHLgFFnLdbLq8Xaig/dqqPQbf37v85Wmo8lEMXradl1JMVZIggRUwzBE/duAmjYcE/UmHGHqz4T+nKgY4dg1GxxzjY/xZLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a5WMo7gTsm2Qr9ZTxjtFLLnl+2HwDjGCyFIPlASvPhI=; b=ukXbFbn0cJb8LYKpqrgWZi1keMQ1HSJfm4qpL3QYoI8gq9vafUNL8vMOVgLHwVEGACt1vZRxUgujYxM8DA8P39JMJBEtMxAlamlS2fdaUkzdvmfvnEzw0Qa4dg2DxECNGvHWsMRXAYj3wJvc3hzT9KNvi9OuGAuXjoTb7qGvdhDJ+IZfOwCVqtNuxuYIx1SIv4nfmUg/noxHdcBWne3PTbtXd/1LW8ia0o4BOVZjUfVua03C4DElW3GDD7AV+aYuJIGjgvmBgHeZxsYPzdH8Y9V8/uQPs1ZqpjWeA4MIk1ZkRufsOKATzB7ULp2AmGmgfzzb3TVBisFkSDbqW0EXMQ== Received: from DUZPR01CA0087.eurprd01.prod.exchangelabs.com (2603:10a6:10:46a::18) by PR3PR07MB7052.eurprd07.prod.outlook.com (2603:10a6:102:7d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.13; Fri, 9 Aug 2024 10:08:04 +0000 Received: from DU6PEPF0000A7E2.eurprd02.prod.outlook.com (2603:10a6:10:46a:cafe::8f) by DUZPR01CA0087.outlook.office365.com (2603:10a6:10:46a::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.14 via Frontend Transport; Fri, 9 Aug 2024 10:08:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU6PEPF0000A7E2.mail.protection.outlook.com (10.167.8.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Fri, 9 Aug 2024 10:08:04 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.63) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 12:08:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id C18821C006A; Fri, 9 Aug 2024 12:08:03 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH v2 4/5] eal: add unit tests for atomic bit access functions Date: Fri, 9 Aug 2024 11:58:28 +0200 Message-ID: <20240809095829.589396-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809095829.589396-1-mattias.ronnblom@ericsson.com> References: <20240809090439.589295-2-mattias.ronnblom@ericsson.com> <20240809095829.589396-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU6PEPF0000A7E2:EE_|PR3PR07MB7052:EE_ X-MS-Office365-Filtering-Correlation-Id: a73abc31-b097-4068-90bc-08dcb85b28b5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|36860700013|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?q?CwldXYLyXfGazYGs9WCkapJpYlH8OO8?= =?utf-8?q?QxYJnOB8ZHVnTmQNX/rLaObf40q82giQ15vemZrIdByDfGB7ipCydPvivDVUfF1Vk?= =?utf-8?q?YMKG9xFVIEjpKUHZMwg/1TOF6bCXlC/fr7czPeB3EKl71Gy/rb/0beho0IFWH75yc?= =?utf-8?q?jWBOckZzf0T46RqiDaHgpTFucFRWuhdZveKTMh1cy+Hij4PuH9z8Sar2cKfRJy0+C?= =?utf-8?q?rLDwpiYsN30f+OUPP/1nOmzTULOr+ElvoGQC/ZgRKVAPRNuSDTfn7x9QQamBwY4la?= =?utf-8?q?KtzRgnu/eu2GRHTkK2ewG35vATuM3/aCED32wlUPu9Tu5i7l7jk/oJzY9eXdGDcGn?= =?utf-8?q?pyNA/aQ+znd2neE1yf+DJ6NkXlvcM7hk0vd4JMjgXZBr9dpYJlFxnXoKX7WAG3G3b?= =?utf-8?q?EjkPdNZdCtU9MTbDS7DKMUdJ68/oXCaI9M6YI0wQS0DwBxCmmcRVTV9TB2qwsm77h?= =?utf-8?q?pxCFQuY55S6T7fXVEpi3l0kwpstmQHhA0UaoaUA3ni7gku15vZkexjuYxKl2TYSSX?= =?utf-8?q?WHbeU52aDBxoNP4tHxNahETP71APHANl8lVb/ZueoU6Hmx6vFbf2LJ6AhCzeSrUrT?= =?utf-8?q?7QvYAHCTDv/jFKgnHLv4k9Mm08O7yjF9A1GmSEmgWrmX36pmaRUX5oGurLjRjonJA?= =?utf-8?q?PY7bQMhK3BF/VtKvnC9Y8lD1yxjTilWb0JYak/rCCGDsZJaluYWtnsUdV58gz+tBc?= =?utf-8?q?DE+y51eSUE0OLnRrDazzD0wmYwQCqNtfKoKVP7TejjXZy5SePf9d9zalhGtmb8if7?= =?utf-8?q?k3dNa6LDrk1PeW2GswngTGbcesslg+KXvbnU13bKeV+V2U4MpA7+S3n2Dbtz6Dz2t?= =?utf-8?q?CFs1iIXM0UQyf1q9AqL1DZyvxr2frwhahd8/ixL5TxIBTLp1gbqtSrryo53YoAxrd?= =?utf-8?q?W8zDYUmrLodWOoEIYFpHfHG01Nn3pFCi3FhOAI/VbMDlG4pdgPXOuvyFqJgWd1ZFS?= =?utf-8?q?FX7WByzwgoo96Yee0SH4vrfquavjW+8GU5R8JM2cKc9t2mK+i/5CBk70Qw3a3etfP?= =?utf-8?q?Mn/ZKKXetmy64MjY1Hif9hLM1ON471RIumOg5/lOC7QOvve97WZSRuMZISjg2MUA6?= =?utf-8?q?nbdj2y0ueZjOJust6IrtyPbIb9rj/H/OscmljgDJfbzt1whzYRYlhdSBD/RiHSqMx?= =?utf-8?q?kC8zF53TpnsqfbA7EhE9jmdVEcbDKT/SF6v82bROjvm6DzQ7/ezkyLKklTttlCSN3?= =?utf-8?q?v2FeJg+3SPSOISH0qVkLD5Xa+yPBwFuMyAB07JXKYyOGLwsZA9ynzyJEcDpruk4Lj?= =?utf-8?q?UdWsK8fpeJ2c3ovYB172cWpsZb8j83vuzrvH12XKt8dbyqcUt2h01SuqafbMrpysd?= =?utf-8?q?VLrnhlnJCsx8uFeBJoq/SjOGZt4POf2c1D7spxTHscGe2BJnJVMKgtqU95JhkKVnc?= =?utf-8?q?KrODa1gbUKX?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(82310400026)(36860700013)(376014)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 10:08:04.3508 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a73abc31-b097-4068-90bc-08dcb85b28b5 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF0000A7E2.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR07MB7052 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Extend bitops tests to cover the rte_bit_atomic_*() family of functions. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- RFC v4: * Add atomicity test for atomic bit flip. RFC v3: * Rename variable 'main' to make ICC happy. --- app/test/test_bitops.c | 313 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 312 insertions(+), 1 deletion(-) diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c index 322f58c066..b80216a0a1 100644 --- a/app/test/test_bitops.c +++ b/app/test/test_bitops.c @@ -3,10 +3,13 @@ * Copyright(c) 2024 Ericsson AB */ +#include #include -#include #include +#include +#include +#include #include #include "test.h" @@ -61,6 +64,304 @@ GEN_TEST_BIT_ACCESS(test_bit_access32, rte_bit_set, rte_bit_clear, GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, rte_bit_assign, rte_bit_flip, rte_bit_test, 64) +#define bit_atomic_set(addr, nr) \ + rte_bit_atomic_set(addr, nr, rte_memory_order_relaxed) + +#define bit_atomic_clear(addr, nr) \ + rte_bit_atomic_clear(addr, nr, rte_memory_order_relaxed) + +#define bit_atomic_assign(addr, nr, value) \ + rte_bit_atomic_assign(addr, nr, value, rte_memory_order_relaxed) + +#define bit_atomic_flip(addr, nr) \ + rte_bit_atomic_flip(addr, nr, rte_memory_order_relaxed) + +#define bit_atomic_test(addr, nr) \ + rte_bit_atomic_test(addr, nr, rte_memory_order_relaxed) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_access32, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 32) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_access64, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 64) + +#define PARALLEL_TEST_RUNTIME 0.25 + +#define GEN_TEST_BIT_PARALLEL_ASSIGN(size) \ + \ + struct parallel_access_lcore ## size \ + { \ + unsigned int bit; \ + uint ## size ##_t *word; \ + bool failed; \ + }; \ + \ + static int \ + run_parallel_assign ## size(void *arg) \ + { \ + struct parallel_access_lcore ## size *lcore = arg; \ + uint64_t deadline = rte_get_timer_cycles() + \ + PARALLEL_TEST_RUNTIME * rte_get_timer_hz(); \ + bool value = false; \ + \ + do { \ + bool new_value = rte_rand() & 1; \ + bool use_test_and_modify = rte_rand() & 1; \ + bool use_assign = rte_rand() & 1; \ + \ + if (rte_bit_atomic_test(lcore->word, lcore->bit, \ + rte_memory_order_relaxed) != value) { \ + lcore->failed = true; \ + break; \ + } \ + \ + if (use_test_and_modify) { \ + bool old_value; \ + if (use_assign) \ + old_value = rte_bit_atomic_test_and_assign( \ + lcore->word, lcore->bit, new_value, \ + rte_memory_order_relaxed); \ + else { \ + old_value = new_value ? \ + rte_bit_atomic_test_and_set( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed) : \ + rte_bit_atomic_test_and_clear( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + } \ + if (old_value != value) { \ + lcore->failed = true; \ + break; \ + } \ + } else { \ + if (use_assign) \ + rte_bit_atomic_assign(lcore->word, lcore->bit, \ + new_value, \ + rte_memory_order_relaxed); \ + else { \ + if (new_value) \ + rte_bit_atomic_set( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + else \ + rte_bit_atomic_clear( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + } \ + } \ + \ + value = new_value; \ + } while (rte_get_timer_cycles() < deadline); \ + \ + return 0; \ + } \ + \ + static int \ + test_bit_atomic_parallel_assign ## size(void) \ + { \ + unsigned int worker_lcore_id; \ + uint ## size ## _t word = 0; \ + struct parallel_access_lcore ## size lmain = { \ + .word = &word \ + }; \ + struct parallel_access_lcore ## size lworker = { \ + .word = &word \ + }; \ + \ + if (rte_lcore_count() < 2) { \ + printf("Need multiple cores to run parallel test.\n"); \ + return TEST_SKIPPED; \ + } \ + \ + worker_lcore_id = rte_get_next_lcore(-1, 1, 0); \ + \ + lmain.bit = rte_rand_max(size); \ + do { \ + lworker.bit = rte_rand_max(size); \ + } while (lworker.bit == lmain.bit); \ + \ + int rc = rte_eal_remote_launch(run_parallel_assign ## size, \ + &lworker, worker_lcore_id); \ + TEST_ASSERT(rc == 0, "Worker thread launch failed"); \ + \ + run_parallel_assign ## size(&lmain); \ + \ + rte_eal_mp_wait_lcore(); \ + \ + TEST_ASSERT(!lmain.failed, "Main lcore atomic access failed"); \ + TEST_ASSERT(!lworker.failed, "Worker lcore atomic access " \ + "failed"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_PARALLEL_ASSIGN(32) +GEN_TEST_BIT_PARALLEL_ASSIGN(64) + +#define GEN_TEST_BIT_PARALLEL_TEST_AND_MODIFY(size) \ + \ + struct parallel_test_and_set_lcore ## size \ + { \ + uint ## size ##_t *word; \ + unsigned int bit; \ + uint64_t flips; \ + }; \ + \ + static int \ + run_parallel_test_and_modify ## size(void *arg) \ + { \ + struct parallel_test_and_set_lcore ## size *lcore = arg; \ + uint64_t deadline = rte_get_timer_cycles() + \ + PARALLEL_TEST_RUNTIME * rte_get_timer_hz(); \ + do { \ + bool old_value; \ + bool new_value = rte_rand() & 1; \ + bool use_assign = rte_rand() & 1; \ + \ + if (use_assign) \ + old_value = rte_bit_atomic_test_and_assign( \ + lcore->word, lcore->bit, new_value, \ + rte_memory_order_relaxed); \ + else \ + old_value = new_value ? \ + rte_bit_atomic_test_and_set( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed) : \ + rte_bit_atomic_test_and_clear( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + if (old_value != new_value) \ + lcore->flips++; \ + } while (rte_get_timer_cycles() < deadline); \ + \ + return 0; \ + } \ + \ + static int \ + test_bit_atomic_parallel_test_and_modify ## size(void) \ + { \ + unsigned int worker_lcore_id; \ + uint ## size ## _t word = 0; \ + unsigned int bit = rte_rand_max(size); \ + struct parallel_test_and_set_lcore ## size lmain = { \ + .word = &word, \ + .bit = bit \ + }; \ + struct parallel_test_and_set_lcore ## size lworker = { \ + .word = &word, \ + .bit = bit \ + }; \ + \ + if (rte_lcore_count() < 2) { \ + printf("Need multiple cores to run parallel test.\n"); \ + return TEST_SKIPPED; \ + } \ + \ + worker_lcore_id = rte_get_next_lcore(-1, 1, 0); \ + \ + int rc = rte_eal_remote_launch(run_parallel_test_and_modify ## size, \ + &lworker, worker_lcore_id); \ + TEST_ASSERT(rc == 0, "Worker thread launch failed"); \ + \ + run_parallel_test_and_modify ## size(&lmain); \ + \ + rte_eal_mp_wait_lcore(); \ + \ + uint64_t total_flips = lmain.flips + lworker.flips; \ + bool expected_value = total_flips % 2; \ + \ + TEST_ASSERT(expected_value == rte_bit_test(&word, bit), \ + "After %"PRId64" flips, the bit value " \ + "should be %d", total_flips, expected_value); \ + \ + uint64_t expected_word = 0; \ + rte_bit_assign(&expected_word, bit, expected_value); \ + \ + TEST_ASSERT(expected_word == word, "Untouched bits have " \ + "changed value"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_PARALLEL_TEST_AND_MODIFY(32) +GEN_TEST_BIT_PARALLEL_TEST_AND_MODIFY(64) + +#define GEN_TEST_BIT_PARALLEL_FLIP(size) \ + \ + struct parallel_flip_lcore ## size \ + { \ + uint ## size ##_t *word; \ + unsigned int bit; \ + uint64_t flips; \ + }; \ + \ + static int \ + run_parallel_flip ## size(void *arg) \ + { \ + struct parallel_flip_lcore ## size *lcore = arg; \ + uint64_t deadline = rte_get_timer_cycles() + \ + PARALLEL_TEST_RUNTIME * rte_get_timer_hz(); \ + do { \ + rte_bit_atomic_flip(lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + lcore->flips++; \ + } while (rte_get_timer_cycles() < deadline); \ + \ + return 0; \ + } \ + \ + static int \ + test_bit_atomic_parallel_flip ## size(void) \ + { \ + unsigned int worker_lcore_id; \ + uint ## size ## _t word = 0; \ + unsigned int bit = rte_rand_max(size); \ + struct parallel_flip_lcore ## size lmain = { \ + .word = &word, \ + .bit = bit \ + }; \ + struct parallel_flip_lcore ## size lworker = { \ + .word = &word, \ + .bit = bit \ + }; \ + \ + if (rte_lcore_count() < 2) { \ + printf("Need multiple cores to run parallel test.\n"); \ + return TEST_SKIPPED; \ + } \ + \ + worker_lcore_id = rte_get_next_lcore(-1, 1, 0); \ + \ + int rc = rte_eal_remote_launch(run_parallel_flip ## size, \ + &lworker, worker_lcore_id); \ + TEST_ASSERT(rc == 0, "Worker thread launch failed"); \ + \ + run_parallel_flip ## size(&lmain); \ + \ + rte_eal_mp_wait_lcore(); \ + \ + uint64_t total_flips = lmain.flips + lworker.flips; \ + bool expected_value = total_flips % 2; \ + \ + TEST_ASSERT(expected_value == rte_bit_test(&word, bit), \ + "After %"PRId64" flips, the bit value " \ + "should be %d", total_flips, expected_value); \ + \ + uint64_t expected_word = 0; \ + rte_bit_assign(&expected_word, bit, expected_value); \ + \ + TEST_ASSERT(expected_word == word, "Untouched bits have " \ + "changed value"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_PARALLEL_FLIP(32) +GEN_TEST_BIT_PARALLEL_FLIP(64) + static uint32_t val32; static uint64_t val64; @@ -177,6 +478,16 @@ static struct unit_test_suite test_suite = { .unit_test_cases = { TEST_CASE(test_bit_access32), TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_access32), + TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_atomic_access32), + TEST_CASE(test_bit_atomic_access64), + TEST_CASE(test_bit_atomic_parallel_assign32), + TEST_CASE(test_bit_atomic_parallel_assign64), + TEST_CASE(test_bit_atomic_parallel_test_and_modify32), + TEST_CASE(test_bit_atomic_parallel_test_and_modify64), + TEST_CASE(test_bit_atomic_parallel_flip32), + TEST_CASE(test_bit_atomic_parallel_flip64), TEST_CASE(test_bit_relaxed_set), TEST_CASE(test_bit_relaxed_clear), TEST_CASE(test_bit_relaxed_test_set_clear), From patchwork Fri Aug 9 09:58:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143048 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A6DB45778; Fri, 9 Aug 2024 12:08:42 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E211242F15; Fri, 9 Aug 2024 12:08:14 +0200 (CEST) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2041.outbound.protection.outlook.com [40.107.104.41]) by mails.dpdk.org (Postfix) with ESMTP id CBCB442EB8 for ; Fri, 9 Aug 2024 12:08:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MYiT6QQ7PjG7J2WHFr5/DgDRuiNYRR73FyP0dyOGWV9oC01loieCQw9QN+mzI+eAAO3XBeUZWYLo1pQQt7NnWOTQqO35OyZ7mR1LgXjdWjrYOUsrPw+alr5Kq/aA+zcv3KRYFs9bP76jxRjRoNSTpBAMHMlAWxYEQLoYKAHtExTg5VIJtGtCFbmNbOwshjgG9YVMCYWgZ/KWBMe4HSDJez3xqfAibW39OBZSqwUkRVxesc5ATfmA1mi7HZozDvHFdXvGUDJJMTTyAsB2beqz4eElU0dFsoxS+/lzF/g34LXbrwIsqMFaCbf+95B+kfJ1lNAncXPBXIrDrAPdrsQMVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1LGm30rZXPVAtL0AoWBQ5eyMwJ8Oxi4dgiomg5LBXFE=; b=vGV7I3XsMoyuLQ9CkyT08DVu6KFHka2y5OESvl68fEvrYhmyRWnM7HjBYfWbFoSEHwAp8hTNpmEY44Xm0m/h6AbH0Tj+eCRvAem6HTi3fxXOa1NoDNqzMG2vy3Ix2GzM8znp9+imXWJtr5rinG9xdKVKhsFeRNGdRY6MFEPxyT3YpmJYx/pRYmp83D1r5A6P2VzXWJa+pHAtJ8gQBOdH0Jaif/dr+L5iheGQ/BVvypgiM3+mJiTJba5gYewCOHLZzuG3yXubr5wihu0DkSbUfzaWI6fd8O+rzlzxMBdlOFCDM6gTatiybHH3s63IXdOR1VNuCheWkK2MtCdpw5L+TQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1LGm30rZXPVAtL0AoWBQ5eyMwJ8Oxi4dgiomg5LBXFE=; b=dBtPcUyDaM4vBPiJ5Ga7sAc/fxmm2VnCiQ1iktgbPyF0YQe6/Qh0cdfyMmCzQ8jGmii+WFm8ES73iw7RvCrXZ9XGnaqny/PeHqYz96Jd1zWZ2jSKHTXhoFBle1geFBvoNE2UGQVnYznBfJGAcMk2wOZx2i71PCxE9YmeoPEqqHItk+jgfQd48/WHjEVAsqhS0+kZv9FhYF2/0f4WlKuxLaGcMLA/Z9Hbjl2ZaKQaHv5r3JhNWm4XWl33OV7b4g5K62X7N5RH5L7F5ChouPU1OHpxaKkXUejP4iWRUamWqnFwdQug75UZA9tRJC45f+OJiMxdzYBiuDtQvXRfrVRWuw== Received: from DUZPR01CA0088.eurprd01.prod.exchangelabs.com (2603:10a6:10:46a::14) by AM8PR07MB7523.eurprd07.prod.outlook.com (2603:10a6:20b:244::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.14; Fri, 9 Aug 2024 10:08:05 +0000 Received: from DU6PEPF0000A7E2.eurprd02.prod.outlook.com (2603:10a6:10:46a:cafe::d8) by DUZPR01CA0088.outlook.office365.com (2603:10a6:10:46a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.14 via Frontend Transport; Fri, 9 Aug 2024 10:08:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU6PEPF0000A7E2.mail.protection.outlook.com (10.167.8.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Fri, 9 Aug 2024 10:08:04 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.63) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 12:08:04 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id D46301C006B; Fri, 9 Aug 2024 12:08:03 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH v2 5/5] eal: extend bitops to handle volatile pointers Date: Fri, 9 Aug 2024 11:58:29 +0200 Message-ID: <20240809095829.589396-6-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809095829.589396-1-mattias.ronnblom@ericsson.com> References: <20240809090439.589295-2-mattias.ronnblom@ericsson.com> <20240809095829.589396-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU6PEPF0000A7E2:EE_|AM8PR07MB7523:EE_ X-MS-Office365-Filtering-Correlation-Id: 099b8a4d-4f41-435a-0363-08dcb85b2909 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?q?vLxZulcel8++u+BItgglM1YOKQiA8I+?= =?utf-8?q?1tZgixWbu47FUe3fZJg+dd78EF3IgyQ9MV09DvKdh9lisXGWojciUe5xeR/+xaaRD?= =?utf-8?q?++T2//12MU5LAvjSgOFnXfVm2DqJ3jOGzAKQh/0FfCXzCp4QGroYupkuP1EYpKlJq?= =?utf-8?q?t+S6EuLEt0oC5DP4Bp82/w323pCK9QnLl7xwVCD2sapMGs0vIj9WQysOspqM3KzuL?= =?utf-8?q?uH40MjV0G6XIGQXcmZ9VIuZhljN1yyY3eUYDopqSrbReCgiezUbW4rcF7cGoy63aZ?= =?utf-8?q?7YCibXoJt5JiqW7lL9dQ3jb8xg14DiT1yYGZwU0DUnZH+W6hBdSn7NKYjF+BcY5GB?= =?utf-8?q?s+sMrgHqp+iKCK8K4tbZdi1pOGXTp85byTmdgW4vr5Ivlhg9heA7Ivk7Wwg65DaKh?= =?utf-8?q?afT92vAYYh7FXygtQJj5HLzcYzVBoYFE+32B7DfRNmmVNb37g/hcpX/j6pqc3VUer?= =?utf-8?q?2F0eWpy5IRX6vWbulYsXHvWu8G4mnwmsRcQ2cYcHGRlGMSEpSQnVQjvAbwARP3H25?= =?utf-8?q?XzkfpzAlCdHljhWG27RrPsMyVF0WNYsHQACocMrJ3D7Vst00leroAntDfKIh6TQoR?= =?utf-8?q?4FZ+2pLqVN2/mxd1uAussH0aeTCws6M5abGMo24G0AKmKgUqZBm4dWyKuiF4oauZr?= =?utf-8?q?yzvcrg5pSnDCrcqo3shD6XILE5ZMbVyXr6HSN5JM/mZA95BKCswNOasHoAPRFSUtl?= =?utf-8?q?x56IfbO6ApJie2RAXbjPYoY0GvvWOcQwnScU+lXvMJVriQ1cO1rlu7c3NTh5p6WJD?= =?utf-8?q?8UGxfp6Invhs3mL3PiZewUtQt1MacuDICtrCqal5n5Knzt9FvviqlRtRukvfYdGnM?= =?utf-8?q?N+qWkWC6mM4FsXrrzVHrUB+H9nn0eURONTjzelahrRY4z4ncS4IRx2u2e6HLB7CnS?= =?utf-8?q?LeuBaZSNbdWwP4Bk0QsfCsmTtD7L/6O1320A5DcKOcOtTlkc0MPRd1u3djhJtp706?= =?utf-8?q?K2tYrngNhchwJpoIXXclj7i6c/fnRSxJqhMOdA1vw2jPTkR8xpkJcunz4e6UnlkJF?= =?utf-8?q?b4UXpebhQuIXHp+KlYSfekjJgvlcFd5h3jjbq/8aIOL1g5qwJ7SCYRr/xoCXKgroy?= =?utf-8?q?ELXLqisMJTxlFRi5UZhw43PzWJaWcmFxKADcAII2uCFSLF2pFuZ8fgUenEWQRJD7k?= =?utf-8?q?krf5Gkun9uynV+FWZOG19Qnbs05Bwki9bhtHSNftDN9u1B3+hAQKnYmPuyvOEIpyq?= =?utf-8?q?rZaJJstjM2UacLN5AyndqxGiBfsz59r9CXq98Rb69Bkw3gUYlMz+JALE+SdjSLO6d?= =?utf-8?q?nlnbMAu9WwSj0Xkod9ptKQ7t0ejpa2jvdI7K9iRd5PK/pVmtM9Pq1SX7OR2p9AN3p?= =?utf-8?q?jlo+jXseIGV1VaoccCnNqVlyr/vK0UqVSA=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 10:08:04.8821 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 099b8a4d-4f41-435a-0363-08dcb85b2909 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF0000A7E2.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR07MB7523 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Have rte_bit_[test|set|clear|assign|flip]() and rte_bit_atomic_*() handle volatile-marked pointers. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup --- PATCH v2: * Actually run the est_bit_atomic_v_access*() test functions. --- app/test/test_bitops.c | 32 ++- lib/eal/include/rte_bitops.h | 427 ++++++++++++++++++++++------------- 2 files changed, 291 insertions(+), 168 deletions(-) diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c index b80216a0a1..10e87f6776 100644 --- a/app/test/test_bitops.c +++ b/app/test/test_bitops.c @@ -14,13 +14,13 @@ #include "test.h" #define GEN_TEST_BIT_ACCESS(test_name, set_fun, clear_fun, assign_fun, \ - flip_fun, test_fun, size) \ + flip_fun, test_fun, size, mod) \ static int \ test_name(void) \ { \ uint ## size ## _t reference = (uint ## size ## _t)rte_rand(); \ unsigned int bit_nr; \ - uint ## size ## _t word = (uint ## size ## _t)rte_rand(); \ + mod uint ## size ## _t word = (uint ## size ## _t)rte_rand(); \ \ for (bit_nr = 0; bit_nr < size; bit_nr++) { \ bool reference_bit = (reference >> bit_nr) & 1; \ @@ -41,7 +41,7 @@ "Bit %d had unflipped value", bit_nr); \ flip_fun(&word, bit_nr); \ \ - const uint ## size ## _t *const_ptr = &word; \ + const mod uint ## size ## _t *const_ptr = &word; \ TEST_ASSERT(test_fun(const_ptr, bit_nr) == \ reference_bit, \ "Bit %d had unexpected value", bit_nr); \ @@ -59,10 +59,16 @@ } GEN_TEST_BIT_ACCESS(test_bit_access32, rte_bit_set, rte_bit_clear, - rte_bit_assign, rte_bit_flip, rte_bit_test, 32) + rte_bit_assign, rte_bit_flip, rte_bit_test, 32,) GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, - rte_bit_assign, rte_bit_flip, rte_bit_test, 64) + rte_bit_assign, rte_bit_flip, rte_bit_test, 64,) + +GEN_TEST_BIT_ACCESS(test_bit_v_access32, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 32, volatile) + +GEN_TEST_BIT_ACCESS(test_bit_v_access64, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 64, volatile) #define bit_atomic_set(addr, nr) \ rte_bit_atomic_set(addr, nr, rte_memory_order_relaxed) @@ -81,11 +87,19 @@ GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, GEN_TEST_BIT_ACCESS(test_bit_atomic_access32, bit_atomic_set, bit_atomic_clear, bit_atomic_assign, - bit_atomic_flip, bit_atomic_test, 32) + bit_atomic_flip, bit_atomic_test, 32,) GEN_TEST_BIT_ACCESS(test_bit_atomic_access64, bit_atomic_set, bit_atomic_clear, bit_atomic_assign, - bit_atomic_flip, bit_atomic_test, 64) + bit_atomic_flip, bit_atomic_test, 64,) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_v_access32, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 32, volatile) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_v_access64, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 64, volatile) #define PARALLEL_TEST_RUNTIME 0.25 @@ -480,8 +494,12 @@ static struct unit_test_suite test_suite = { TEST_CASE(test_bit_access64), TEST_CASE(test_bit_access32), TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_v_access32), + TEST_CASE(test_bit_v_access64), TEST_CASE(test_bit_atomic_access32), TEST_CASE(test_bit_atomic_access64), + TEST_CASE(test_bit_atomic_v_access32), + TEST_CASE(test_bit_atomic_v_access64), TEST_CASE(test_bit_atomic_parallel_assign32), TEST_CASE(test_bit_atomic_parallel_assign64), TEST_CASE(test_bit_atomic_parallel_test_and_modify32), diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 4d878099ed..1355949fb6 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -127,12 +127,16 @@ extern "C" { * @param nr * The index of the bit. */ -#define rte_bit_test(addr, nr) \ - _Generic((addr), \ - uint32_t *: __rte_bit_test32, \ - const uint32_t *: __rte_bit_test32, \ - uint64_t *: __rte_bit_test64, \ - const uint64_t *: __rte_bit_test64)(addr, nr) +#define rte_bit_test(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_test32, \ + const uint32_t *: __rte_bit_test32, \ + volatile uint32_t *: __rte_bit_v_test32, \ + const volatile uint32_t *: __rte_bit_v_test32, \ + uint64_t *: __rte_bit_test64, \ + const uint64_t *: __rte_bit_test64, \ + volatile uint64_t *: __rte_bit_v_test64, \ + const volatile uint64_t *: __rte_bit_v_test64)(addr, nr) /** * @warning @@ -152,10 +156,12 @@ extern "C" { * @param nr * The index of the bit. */ -#define rte_bit_set(addr, nr) \ - _Generic((addr), \ - uint32_t *: __rte_bit_set32, \ - uint64_t *: __rte_bit_set64)(addr, nr) +#define rte_bit_set(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_set32, \ + volatile uint32_t *: __rte_bit_v_set32, \ + uint64_t *: __rte_bit_set64, \ + volatile uint64_t *: __rte_bit_v_set64)(addr, nr) /** * @warning @@ -175,10 +181,12 @@ extern "C" { * @param nr * The index of the bit. */ -#define rte_bit_clear(addr, nr) \ - _Generic((addr), \ - uint32_t *: __rte_bit_clear32, \ - uint64_t *: __rte_bit_clear64)(addr, nr) +#define rte_bit_clear(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_clear32, \ + volatile uint32_t *: __rte_bit_v_clear32, \ + uint64_t *: __rte_bit_clear64, \ + volatile uint64_t *: __rte_bit_v_clear64)(addr, nr) /** * @warning @@ -202,7 +210,9 @@ extern "C" { #define rte_bit_assign(addr, nr, value) \ _Generic((addr), \ uint32_t *: __rte_bit_assign32, \ - uint64_t *: __rte_bit_assign64)(addr, nr, value) + volatile uint32_t *: __rte_bit_v_assign32, \ + uint64_t *: __rte_bit_assign64, \ + volatile uint64_t *: __rte_bit_v_assign64)(addr, nr, value) /** * @warning @@ -225,7 +235,9 @@ extern "C" { #define rte_bit_flip(addr, nr) \ _Generic((addr), \ uint32_t *: __rte_bit_flip32, \ - uint64_t *: __rte_bit_flip64)(addr, nr) + volatile uint32_t *: __rte_bit_v_flip32, \ + uint64_t *: __rte_bit_flip64, \ + volatile uint64_t *: __rte_bit_v_flip64)(addr, nr) /** * @warning @@ -250,9 +262,13 @@ extern "C" { _Generic((addr), \ uint32_t *: __rte_bit_atomic_test32, \ const uint32_t *: __rte_bit_atomic_test32, \ + volatile uint32_t *: __rte_bit_atomic_v_test32, \ + const volatile uint32_t *: __rte_bit_atomic_v_test32, \ uint64_t *: __rte_bit_atomic_test64, \ - const uint64_t *: __rte_bit_atomic_test64)(addr, nr, \ - memory_order) + const uint64_t *: __rte_bit_atomic_test64, \ + volatile uint64_t *: __rte_bit_atomic_v_test64, \ + const volatile uint64_t *: __rte_bit_atomic_v_test64) \ + (addr, nr, memory_order) /** * @warning @@ -274,7 +290,10 @@ extern "C" { #define rte_bit_atomic_set(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_set32, \ - uint64_t *: __rte_bit_atomic_set64)(addr, nr, memory_order) + volatile uint32_t *: __rte_bit_atomic_v_set32, \ + uint64_t *: __rte_bit_atomic_set64, \ + volatile uint64_t *: __rte_bit_atomic_v_set64)(addr, nr, \ + memory_order) /** * @warning @@ -296,7 +315,10 @@ extern "C" { #define rte_bit_atomic_clear(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_clear32, \ - uint64_t *: __rte_bit_atomic_clear64)(addr, nr, memory_order) + volatile uint32_t *: __rte_bit_atomic_v_clear32, \ + uint64_t *: __rte_bit_atomic_clear64, \ + volatile uint64_t *: __rte_bit_atomic_v_clear64)(addr, nr, \ + memory_order) /** * @warning @@ -320,8 +342,11 @@ extern "C" { #define rte_bit_atomic_assign(addr, nr, value, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_assign32, \ - uint64_t *: __rte_bit_atomic_assign64)(addr, nr, value, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_assign32, \ + uint64_t *: __rte_bit_atomic_assign64, \ + volatile uint64_t *: __rte_bit_atomic_v_assign64)(addr, nr, \ + value, \ + memory_order) /** * @warning @@ -344,7 +369,10 @@ extern "C" { #define rte_bit_atomic_flip(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_flip32, \ - uint64_t *: __rte_bit_atomic_flip64)(addr, nr, memory_order) + volatile uint32_t *: __rte_bit_atomic_v_flip32, \ + uint64_t *: __rte_bit_atomic_flip64, \ + volatile uint64_t *: __rte_bit_atomic_v_flip64)(addr, nr, \ + memory_order) /** * @warning @@ -368,8 +396,10 @@ extern "C" { #define rte_bit_atomic_test_and_set(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_test_and_set32, \ - uint64_t *: __rte_bit_atomic_test_and_set64)(addr, nr, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_test_and_set32, \ + uint64_t *: __rte_bit_atomic_test_and_set64, \ + volatile uint64_t *: __rte_bit_atomic_v_test_and_set64) \ + (addr, nr, memory_order) /** * @warning @@ -393,8 +423,10 @@ extern "C" { #define rte_bit_atomic_test_and_clear(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_test_and_clear32, \ - uint64_t *: __rte_bit_atomic_test_and_clear64)(addr, nr, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_test_and_clear32, \ + uint64_t *: __rte_bit_atomic_test_and_clear64, \ + volatile uint64_t *: __rte_bit_atomic_v_test_and_clear64) \ + (addr, nr, memory_order) /** * @warning @@ -421,9 +453,10 @@ extern "C" { #define rte_bit_atomic_test_and_assign(addr, nr, value, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_test_and_assign32, \ - uint64_t *: __rte_bit_atomic_test_and_assign64)(addr, nr, \ - value, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_test_and_assign32, \ + uint64_t *: __rte_bit_atomic_test_and_assign64, \ + volatile uint64_t *: __rte_bit_atomic_v_test_and_assign64) \ + (addr, nr, value, memory_order) #define __RTE_GEN_BIT_TEST(family, fun, qualifier, size) \ __rte_experimental \ @@ -491,93 +524,105 @@ __RTE_GEN_BIT_CLEAR(, clear,, 32) __RTE_GEN_BIT_ASSIGN(, assign,, 32) __RTE_GEN_BIT_FLIP(, flip,, 32) +__RTE_GEN_BIT_TEST(v_, test, volatile, 32) +__RTE_GEN_BIT_SET(v_, set, volatile, 32) +__RTE_GEN_BIT_CLEAR(v_, clear, volatile, 32) +__RTE_GEN_BIT_ASSIGN(v_, assign, volatile, 32) +__RTE_GEN_BIT_FLIP(v_, flip, volatile, 32) + __RTE_GEN_BIT_TEST(, test,, 64) __RTE_GEN_BIT_SET(, set,, 64) __RTE_GEN_BIT_CLEAR(, clear,, 64) __RTE_GEN_BIT_ASSIGN(, assign,, 64) __RTE_GEN_BIT_FLIP(, flip,, 64) -#define __RTE_GEN_BIT_ATOMIC_TEST(size) \ +__RTE_GEN_BIT_TEST(v_, test, volatile, 64) +__RTE_GEN_BIT_SET(v_, set, volatile, 64) +__RTE_GEN_BIT_CLEAR(v_, clear, volatile, 64) +__RTE_GEN_BIT_ASSIGN(v_, assign, volatile, 64) +__RTE_GEN_BIT_FLIP(v_, flip, volatile, 64) + +#define __RTE_GEN_BIT_ATOMIC_TEST(v, qualifier, size) \ __rte_experimental \ static inline bool \ - __rte_bit_atomic_test ## size(const uint ## size ## _t *addr, \ - unsigned int nr, int memory_order) \ + __rte_bit_atomic_ ## v ## test ## size(const qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - const RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (const RTE_ATOMIC(uint ## size ## _t) *)addr; \ + const qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (const qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ return rte_atomic_load_explicit(a_addr, memory_order) & mask; \ } -#define __RTE_GEN_BIT_ATOMIC_SET(size) \ +#define __RTE_GEN_BIT_ATOMIC_SET(v, qualifier, size) \ __rte_experimental \ static inline void \ - __rte_bit_atomic_set ## size(uint ## size ## _t *addr, \ - unsigned int nr, int memory_order) \ + __rte_bit_atomic_ ## v ## set ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ rte_atomic_fetch_or_explicit(a_addr, mask, memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ +#define __RTE_GEN_BIT_ATOMIC_CLEAR(v, qualifier, size) \ __rte_experimental \ static inline void \ - __rte_bit_atomic_clear ## size(uint ## size ## _t *addr, \ - unsigned int nr, int memory_order) \ + __rte_bit_atomic_ ## v ## clear ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ rte_atomic_fetch_and_explicit(a_addr, ~mask, memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_FLIP(size) \ +#define __RTE_GEN_BIT_ATOMIC_FLIP(v, qualifier, size) \ __rte_experimental \ static inline void \ - __rte_bit_atomic_flip ## size(uint ## size ## _t *addr, \ - unsigned int nr, int memory_order) \ + __rte_bit_atomic_ ## v ## flip ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ rte_atomic_fetch_xor_explicit(a_addr, mask, memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ +#define __RTE_GEN_BIT_ATOMIC_ASSIGN(v, qualifier, size) \ __rte_experimental \ static inline void \ - __rte_bit_atomic_assign ## size(uint ## size ## _t *addr, \ - unsigned int nr, bool value, \ - int memory_order) \ + __rte_bit_atomic_## v ## assign ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, bool value, \ + int memory_order) \ { \ if (value) \ - __rte_bit_atomic_set ## size(addr, nr, memory_order); \ + __rte_bit_atomic_ ## v ## set ## size(addr, nr, memory_order); \ else \ - __rte_bit_atomic_clear ## size(addr, nr, \ - memory_order); \ + __rte_bit_atomic_ ## v ## clear ## size(addr, nr, \ + memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(size) \ +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(v, qualifier, size) \ __rte_experimental \ static inline bool \ - __rte_bit_atomic_test_and_set ## size(uint ## size ## _t *addr, \ - unsigned int nr, \ - int memory_order) \ + __rte_bit_atomic_ ## v ## test_and_set ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ uint ## size ## _t prev; \ \ @@ -587,17 +632,17 @@ __RTE_GEN_BIT_FLIP(, flip,, 64) return prev & mask; \ } -#define __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(size) \ +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(v, qualifier, size) \ __rte_experimental \ static inline bool \ - __rte_bit_atomic_test_and_clear ## size(uint ## size ## _t *addr, \ - unsigned int nr, \ - int memory_order) \ + __rte_bit_atomic_ ## v ## test_and_clear ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ uint ## size ## _t prev; \ \ @@ -607,34 +652,36 @@ __RTE_GEN_BIT_FLIP(, flip,, 64) return prev & mask; \ } -#define __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \ +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(v, qualifier, size) \ __rte_experimental \ static inline bool \ - __rte_bit_atomic_test_and_assign ## size(uint ## size ## _t *addr, \ - unsigned int nr, \ - bool value, \ - int memory_order) \ + __rte_bit_atomic_ ## v ## test_and_assign ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, \ + bool value, \ + int memory_order) \ { \ if (value) \ - return __rte_bit_atomic_test_and_set ## size(addr, nr, \ - memory_order); \ + return __rte_bit_atomic_ ## v ## test_and_set ## size(addr, nr, memory_order); \ else \ - return __rte_bit_atomic_test_and_clear ## size(addr, nr, \ - memory_order); \ + return __rte_bit_atomic_ ## v ## test_and_clear ## size(addr, nr, memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_OPS(size) \ - __RTE_GEN_BIT_ATOMIC_TEST(size) \ - __RTE_GEN_BIT_ATOMIC_SET(size) \ - __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ - __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ - __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(size) \ - __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(size) \ - __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \ - __RTE_GEN_BIT_ATOMIC_FLIP(size) +#define __RTE_GEN_BIT_ATOMIC_OPS(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_SET(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_CLEAR(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_ASSIGN(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_FLIP(v, qualifier, size) -__RTE_GEN_BIT_ATOMIC_OPS(32) -__RTE_GEN_BIT_ATOMIC_OPS(64) +#define __RTE_GEN_BIT_ATOMIC_OPS_SIZE(size) \ + __RTE_GEN_BIT_ATOMIC_OPS(,, size) \ + __RTE_GEN_BIT_ATOMIC_OPS(v_, volatile, size) + +__RTE_GEN_BIT_ATOMIC_OPS_SIZE(32) +__RTE_GEN_BIT_ATOMIC_OPS_SIZE(64) /*------------------------ 32-bit relaxed operations ------------------------*/ @@ -1340,120 +1387,178 @@ rte_log2_u64(uint64_t v) #undef rte_bit_atomic_test_and_clear #undef rte_bit_atomic_test_and_assign -#define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ +#define __RTE_BIT_OVERLOAD_V_2(family, v, fun, c, size, arg1_type, arg1_name) \ static inline void \ - rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ - arg1_type arg1_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name) \ { \ - __rte_bit_ ## fun ## size(addr, arg1_name); \ + __rte_bit_ ## family ## v ## fun ## size(addr, arg1_name); \ } -#define __RTE_BIT_OVERLOAD_2(fun, qualifier, arg1_type, arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 32, arg1_type, arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 64, arg1_type, arg1_name) +#define __RTE_BIT_OVERLOAD_SZ_2(family, fun, c, size, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_V_2(family,, fun, c, size, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_V_2(family, v_, fun, c volatile, size, \ + arg1_type, arg1_name) -#define __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, size, ret_type, arg1_type, \ - arg1_name) \ +#define __RTE_BIT_OVERLOAD_2(family, fun, c, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(family, fun, c, 32, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(family, fun, c, 64, arg1_type, arg1_name) + +#define __RTE_BIT_OVERLOAD_V_2R(family, v, fun, c, size, ret_type, arg1_type, \ + arg1_name) \ static inline ret_type \ - rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ arg1_type arg1_name) \ { \ - return __rte_bit_ ## fun ## size(addr, arg1_name); \ + return __rte_bit_ ## family ## v ## fun ## size(addr, \ + arg1_name); \ } -#define __RTE_BIT_OVERLOAD_2R(fun, qualifier, ret_type, arg1_type, arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 32, ret_type, arg1_type, \ +#define __RTE_BIT_OVERLOAD_SZ_2R(family, fun, c, size, ret_type, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_V_2R(family,, fun, c, size, ret_type, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_V_2R(family, v_, fun, c volatile, \ + size, ret_type, arg1_type, arg1_name) + +#define __RTE_BIT_OVERLOAD_2R(family, fun, c, ret_type, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2R(family, fun, c, 32, ret_type, arg1_type, \ arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 64, ret_type, arg1_type, \ + __RTE_BIT_OVERLOAD_SZ_2R(family, fun, c, 64, ret_type, arg1_type, \ arg1_name) -#define __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, size, arg1_type, arg1_name, \ - arg2_type, arg2_name) \ +#define __RTE_BIT_OVERLOAD_V_3(family, v, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ static inline void \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name) \ { \ - __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + __rte_bit_ ## family ## v ## fun ## size(addr, arg1_name, \ + arg2_name); \ } -#define __RTE_BIT_OVERLOAD_3(fun, qualifier, arg1_type, arg1_name, arg2_type, \ +#define __RTE_BIT_OVERLOAD_SZ_3(family, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_V_3(family,, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_V_3(family, v_, fun, c volatile, size, arg1_type, \ + arg1_name, arg2_type, arg2_name) + +#define __RTE_BIT_OVERLOAD_3(family, fun, c, arg1_type, arg1_name, arg2_type, \ arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 32, arg1_type, arg1_name, \ + __RTE_BIT_OVERLOAD_SZ_3(family, fun, c, 32, arg1_type, arg1_name, \ arg2_type, arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 64, arg1_type, arg1_name, \ + __RTE_BIT_OVERLOAD_SZ_3(family, fun, c, 64, arg1_type, arg1_name, \ arg2_type, arg2_name) -#define __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, size, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name) \ +#define __RTE_BIT_OVERLOAD_V_3R(family, v, fun, c, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) \ static inline ret_type \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name) \ { \ - return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + return __rte_bit_ ## family ## v ## fun ## size(addr, \ + arg1_name, \ + arg2_name); \ } -#define __RTE_BIT_OVERLOAD_3R(fun, qualifier, ret_type, arg1_type, arg1_name, \ - arg2_type, arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 32, ret_type, arg1_type, \ +#define __RTE_BIT_OVERLOAD_SZ_3R(family, fun, c, size, ret_type, arg1_type, \ arg1_name, arg2_type, arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 64, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name) + __RTE_BIT_OVERLOAD_V_3R(family,, fun, c, size, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_V_3R(family, v_, fun, c volatile, size, \ + ret_type, arg1_type, arg1_name, arg2_type, \ + arg2_name) -#define __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, size, arg1_type, arg1_name, \ - arg2_type, arg2_name, arg3_type, arg3_name) \ +#define __RTE_BIT_OVERLOAD_3R(family, fun, c, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(family, fun, c, 32, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(family, fun, c, 64, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name) + +#define __RTE_BIT_OVERLOAD_V_4(family, v, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ static inline void \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name, arg3_type arg3_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name, \ + arg3_type arg3_name) \ { \ - __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ - arg3_name); \ + __rte_bit_ ## family ## v ## fun ## size(addr, arg1_name, \ + arg2_name, \ + arg3_name); \ } -#define __RTE_BIT_OVERLOAD_4(fun, qualifier, arg1_type, arg1_name, arg2_type, \ - arg2_name, arg3_type, arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 32, arg1_type, arg1_name, \ +#define __RTE_BIT_OVERLOAD_SZ_4(family, fun, c, size, arg1_type, arg1_name, \ arg2_type, arg2_name, arg3_type, arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 64, arg1_type, arg1_name, \ - arg2_type, arg2_name, arg3_type, arg3_name) - -#define __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, size, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name, arg3_type, \ - arg3_name) \ + __RTE_BIT_OVERLOAD_V_4(family,, fun, c, size, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_V_4(family, v_, fun, c volatile, size, \ + arg1_type, arg1_name, arg2_type, arg2_name, \ + arg3_type, arg3_name) + +#define __RTE_BIT_OVERLOAD_4(family, fun, c, arg1_type, arg1_name, arg2_type, \ + arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(family, fun, c, 32, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(family, fun, c, 64, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) + +#define __RTE_BIT_OVERLOAD_V_4R(family, v, fun, c, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ static inline ret_type \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name, arg3_type arg3_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name, \ + arg3_type arg3_name) \ { \ - return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ - arg3_name); \ + return __rte_bit_ ## family ## v ## fun ## size(addr, \ + arg1_name, \ + arg2_name, \ + arg3_name); \ } -#define __RTE_BIT_OVERLOAD_4R(fun, qualifier, ret_type, arg1_type, arg1_name, \ - arg2_type, arg2_name, arg3_type, arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 32, ret_type, arg1_type, \ +#define __RTE_BIT_OVERLOAD_SZ_4R(family, fun, c, size, ret_type, arg1_type, \ arg1_name, arg2_type, arg2_name, arg3_type, \ arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 64, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name, arg3_type, \ - arg3_name) - -__RTE_BIT_OVERLOAD_2R(test, const, bool, unsigned int, nr) -__RTE_BIT_OVERLOAD_2(set,, unsigned int, nr) -__RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) -__RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) -__RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) - -__RTE_BIT_OVERLOAD_3R(atomic_test, const, bool, unsigned int, nr, + __RTE_BIT_OVERLOAD_V_4R(family,, fun, c, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_V_4R(family, v_, fun, c volatile, size, \ + ret_type, arg1_type, arg1_name, arg2_type, \ + arg2_name, arg3_type, arg3_name) + +#define __RTE_BIT_OVERLOAD_4R(family, fun, c, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(family, fun, c, 32, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name, \ + arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(family, fun, c, 64, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name, \ + arg3_type, arg3_name) + +__RTE_BIT_OVERLOAD_2R(, test, const, bool, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(, set,, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(, clear,, unsigned int, nr) +__RTE_BIT_OVERLOAD_3(, assign,, unsigned int, nr, bool, value) +__RTE_BIT_OVERLOAD_2(, flip,, unsigned int, nr) + +__RTE_BIT_OVERLOAD_3R(atomic_, test, const, bool, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3(atomic_set,, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3(atomic_clear,, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_4(atomic_assign,, unsigned int, nr, bool, value, +__RTE_BIT_OVERLOAD_3(atomic_, set,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_, clear,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_4(atomic_, assign,, unsigned int, nr, bool, value, int, memory_order) -__RTE_BIT_OVERLOAD_3(atomic_flip,, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3R(atomic_test_and_set,, bool, unsigned int, nr, +__RTE_BIT_OVERLOAD_3(atomic_, flip,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3R(atomic_, test_and_set,, bool, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3R(atomic_test_and_clear,, bool, unsigned int, nr, +__RTE_BIT_OVERLOAD_3R(atomic_, test_and_clear,, bool, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_4R(atomic_test_and_assign,, bool, unsigned int, nr, +__RTE_BIT_OVERLOAD_4R(atomic_, test_and_assign,, bool, unsigned int, nr, bool, value, int, memory_order) #endif