From patchwork Fri Aug 9 09:04:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143037 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8597745777; Fri, 9 Aug 2024 11:14:19 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 76ADA42DC9; Fri, 9 Aug 2024 11:14:19 +0200 (CEST) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2045.outbound.protection.outlook.com [40.107.104.45]) by mails.dpdk.org (Postfix) with ESMTP id 78D7C40274 for ; Fri, 9 Aug 2024 11:14:18 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=D6CvFyOCtZIq4YuPsUv98IOGtkv4sc6KGO4OdOsj1gAO119Etek0Qz4fdJZXKNkhc7NinSFGY64w3p8FejNkyV5WHbiJuw63gprE7b8LXEOMNdDtYrXxFgDLAOM6Ne5oRKVHfwg1DioUp4ydcGpEYfWQL+6fBE3onJMqWvBrjXwZqvSgxVGnTDJcBNWVpDNsKZZe7uQOmS1aM2NzgGnToNq656ob8kknB7iuoxPkl5wcS0z2BvbBQuYTiwpclsItPWc6P2w/xBt8rMLZHd1KZ3xSczsmgo09UyMIfmAMzojdJcQV2wFCjfBcl0oElFoOIPPAgZDOfLSuJdYLsGct7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uGaYNBWczZiaiy2F567USLR166LHBzCdDI8GvTpYwsg=; b=SL+HijfozMKAin7iDP2TIEtOnod416MLhLhXOCkakISukzvf/1hxjChjTZDaBqi+GHUizQ5ATOJf+UymR0YdbXuNr+A6+GBzcsnmK6D96e7l0BjnkniQouFrW9eF6XG/Pc7tQquoy3+hUCiCa0OUCUQdIf1l6ZUcHoT6kJo6gE2uwUugMSNAqR8xm0mQRSQi7lhy9a4MJdx4ANhZmUHlQv4uA7eE3bSkwvmc+DrEPjfe3S2cIH6UUWZsObDJmJTIJLvl/VrgP4l1E2YGXGGDt/nlhL3EcyhQtnzuCz8EHsRSgNkho0qhHFL22CJOvEpE+a6kC7F56zDIAAlM0Hm/AQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uGaYNBWczZiaiy2F567USLR166LHBzCdDI8GvTpYwsg=; b=nzGx8jaAsLtV4srtwPWa4O6SjpjrMxq4m3DusTGPkPf46Xm3qtj+zmO8j0olLB0cWjG3+X8gsO5N0d5FzvDl+YKs3KaPDKxhV4tgFkmM+kxTKgTrF95J2+/tR/ryDkAEn3zXli+/wBkVCAXOc5SPs0RLxQtGx00qZHkQJ0MQ7UiL/fSWzQvoPw3kyv/AZKIUkEdFZz3avWazp/vgz1bYJ9DFP6o43OB2EKyqw+lWqKOjT+/956ESxZN2+YFdCfcqsiqLWxoK8lHGsxvbUSOU/n7tHKHFZrXsOngeKOy4fvdI4qbLAoAlc5la+B6BdufP7sSvKDI6w9uzDU/5A5RJbg== Received: from PR2P264CA0003.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::15) by AM7PR07MB6849.eurprd07.prod.outlook.com (2603:10a6:20b:1c1::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15; Fri, 9 Aug 2024 09:14:16 +0000 Received: from AM4PEPF00025F98.EURPRD83.prod.outlook.com (2603:10a6:101:0:cafe::e2) by PR2P264CA0003.outlook.office365.com (2603:10a6:101::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00025F98.mail.protection.outlook.com (10.167.16.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.2 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.63) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 11:14:15 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id 8D5CE1C006B; Fri, 9 Aug 2024 11:14:15 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH 1/5] eal: extend bit manipulation functionality Date: Fri, 9 Aug 2024 11:04:35 +0200 Message-ID: <20240809090439.589295-2-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809090439.589295-1-mattias.ronnblom@ericsson.com> References: <20240505083737.118649-2-mattias.ronnblom@ericsson.com> <20240809090439.589295-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00025F98:EE_|AM7PR07MB6849:EE_ X-MS-Office365-Filtering-Correlation-Id: 66e43faf-4b08-4850-0f0a-08dcb853a476 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700013|82310400026|376014; X-Microsoft-Antispam-Message-Info: =?utf-8?q?f302fxhS+JcDkJsKbT2sgyf+jfNts2e?= =?utf-8?q?t39IhL2OjYn/vftQ2DKLQbk6V3KHWpMJpv9DvWvuRfWbX33G0ZF4XXuAtvKpyHoP/?= =?utf-8?q?Z3pHwYHN0h/fivloYrcEe15xQdS2nU4kLaPQlb2+v8hNSa89u2ZoU9BNg0NeQFlic?= =?utf-8?q?Gx0b44Um0zg0P54HGSbcHXewOBw2UxQ0fk33Rd9wo2LVvjj26HWHjaMLf+dT6m6um?= =?utf-8?q?KBm8dcI+hOiVMmQJJhYfToDo5lHvObowTCGdt4bM7tX23gl2/YZtTMUA5cRVsPGKR?= =?utf-8?q?UVgbrjsc2mECzVL7HPygt90LIEkEPdes3R7UpEGNADUFMlHM+MVJ7KZL3EGb9EcGg?= =?utf-8?q?0TWkBZtyY5XnUnV8xDd0W4kQ7EuGCibvyHRF5729YQPSeac01wtW/X8ULZW4cRst8?= =?utf-8?q?MAclj4FQdpa6a1NV9uaBuGyl/82f5IM+BabJDhbnTryUnaTqPnSga41kbxADP7K3B?= =?utf-8?q?fPdjtwv8UR04JN/9txiHPHIORknsgk+8FxhF1IYrYbtJtN+JPkOgKIVve8bencHrC?= =?utf-8?q?ISNdcnMzf6W3OMThdnGEuBZi177339PZs3cD59Yk69e3S0pKZX5kfkmNHoqAfnCZH?= =?utf-8?q?xZZVixnBhdMZouDR9NknFZce17KtVTMCqLRQESWqSJ/K/KSbAepnnKFz7FZV9owsw?= =?utf-8?q?KUMpHClVpGrXOzlMiwm6vuCrKFg6ht7USU8gY5UNO6PLcyEUEqFYum51cyDvQCE8N?= =?utf-8?q?DuctRHdG/ulwXKdTFtOyZm6CUiT/IzcureCFx8j6j1r/707XOPqYILS/i9tSctIAY?= =?utf-8?q?vYilec6MBHWd3kY+6tAl1bQBZTvYtsew+fduFNwBwN+K8HxwJFHNZXOciAchAheHl?= =?utf-8?q?tMSzHWdDF3Akx4w9DXwLS2fc0u7QEeeXdCojtWYT2ik84CvdvXupxF/zL4GxJKfb9?= =?utf-8?q?cNZowkpD95hIYJ7LfjW6SdM6WYgNnqIQRNgMxn4omFGWiAyBgjn8SQFYcpQZ3TbAn?= =?utf-8?q?mt5RlVMsoYpK0tGzhWWTG+2Vvy/g1TXtxZp7m6T34OhUb/oCSQUnPt///G8GPP44e?= =?utf-8?q?SXwB9ysDQlcDFUu0e9Oq4Po5AKdf2ruxEXGtdD2JoD+eAblIIuZE6OmH/i2Aj78Q7?= =?utf-8?q?HtuD1MTG/PtQ1118v8r/aWEO+eLUjVY/BZbl03UTdV4HvndovkhTK+fjaqmxCuI/3?= =?utf-8?q?RKFeWNOrXg0ngsaoYG8r+3VxnbxrU6bXJnTThCQs40bsiL7zdBmIcPLtYciJCbVCo?= =?utf-8?q?BLkz+KlguGtU9KxeyeK4KxkSDHoNFw6RIKQBdOywZ149aNGaqiaov0VegUm+v99AK?= =?utf-8?q?FuoulQGIx0Vjpcifw4X7Rjc2NfGBWx+YOTxt0M23osetg9+gUwKbx8uY/HhNn1jHr?= =?utf-8?q?CHjp4pMdAa6QeE0fCdRp+n4Wr7rN0Si0BA=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 09:14:16.0295 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 66e43faf-4b08-4850-0f0a-08dcb853a476 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00025F98.EURPRD83.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR07MB6849 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add functionality to test and modify the value of individual bits in 32-bit or 64-bit words. These functions have no implications on memory ordering, atomicity and does not use volatile and thus does not prevent any compiler optimizations. RFC v6: * Have rte_bit_test() accept const-marked bitsets. RFC v4: * Add rte_bit_flip() which, believe it or not, flips the value of a bit. * Mark macro-generated private functions as experimental. * Use macros to generate *assign*() functions. RFC v3: * Work around lack of C++ support for _Generic (Tyler Retzlaff). * Fix ','-related checkpatch warnings. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- lib/eal/include/rte_bitops.h | 259 ++++++++++++++++++++++++++++++++++- 1 file changed, 257 insertions(+), 2 deletions(-) diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 449565eeae..3297133e22 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -2,6 +2,7 @@ * Copyright(c) 2020 Arm Limited * Copyright(c) 2010-2019 Intel Corporation * Copyright(c) 2023 Microsoft Corporation + * Copyright(c) 2024 Ericsson AB */ #ifndef _RTE_BITOPS_H_ @@ -11,12 +12,14 @@ * @file * Bit Operations * - * This file defines a family of APIs for bit operations - * without enforcing memory ordering. + * This file provides functionality for low-level, single-word + * arithmetic and bit-level operations, such as counting or + * setting individual bits. */ #include +#include #include #ifdef __cplusplus @@ -105,6 +108,196 @@ extern "C" { #define RTE_FIELD_GET64(mask, reg) \ ((typeof(mask))(((reg) & (mask)) >> rte_ctz64(mask))) +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Test bit in word. + * + * Generic selection macro to test the value of a bit in a 32-bit or + * 64-bit word. The type of operation depends on the type of the @c + * addr parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_test(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_test32, \ + const uint32_t *: __rte_bit_test32, \ + uint64_t *: __rte_bit_test64, \ + const uint64_t *: __rte_bit_test64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set bit in word. + * + * Generic selection macro to set a bit in a 32-bit or 64-bit + * word. The type of operation depends on the type of the @c addr + * parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_set(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_set32, \ + uint64_t *: __rte_bit_set64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Clear bit in word. + * + * Generic selection macro to clear a bit in a 32-bit or 64-bit + * word. The type of operation depends on the type of the @c addr + * parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_clear(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_clear32, \ + uint64_t *: __rte_bit_clear64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Assign a value to a bit in word. + * + * Generic selection macro to assign a value to a bit in a 32-bit or 64-bit + * word. The type of operation depends on the type of the @c addr parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + */ +#define rte_bit_assign(addr, nr, value) \ + _Generic((addr), \ + uint32_t *: __rte_bit_assign32, \ + uint64_t *: __rte_bit_assign64)(addr, nr, value) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Flip a bit in word. + * + * Generic selection macro to change the value of a bit to '0' if '1' + * or '1' if '0' in a 32-bit or 64-bit word. The type of operation + * depends on the type of the @c addr parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_flip(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_flip32, \ + uint64_t *: __rte_bit_flip64)(addr, nr) + +#define __RTE_GEN_BIT_TEST(family, fun, qualifier, size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_ ## family ## fun ## size(const qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + RTE_ASSERT(nr < size); \ + \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + return *addr & mask; \ + } + +#define __RTE_GEN_BIT_SET(family, fun, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## family ## fun ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + RTE_ASSERT(nr < size); \ + \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + *addr |= mask; \ + } \ + +#define __RTE_GEN_BIT_CLEAR(family, fun, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## family ## fun ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + RTE_ASSERT(nr < size); \ + \ + uint ## size ## _t mask = ~((uint ## size ## _t)1 << nr); \ + (*addr) &= mask; \ + } \ + +#define __RTE_GEN_BIT_ASSIGN(family, fun, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## family ## fun ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, bool value) \ + { \ + if (value) \ + __rte_bit_ ## family ## set ## size(addr, nr); \ + else \ + __rte_bit_ ## family ## clear ## size(addr, nr); \ + } + +#define __RTE_GEN_BIT_FLIP(family, fun, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## family ## fun ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + bool value; \ + \ + value = __rte_bit_ ## family ## test ## size(addr, nr); \ + __rte_bit_ ## family ## assign ## size(addr, nr, !value); \ + } + +__RTE_GEN_BIT_TEST(, test,, 32) +__RTE_GEN_BIT_SET(, set,, 32) +__RTE_GEN_BIT_CLEAR(, clear,, 32) +__RTE_GEN_BIT_ASSIGN(, assign,, 32) +__RTE_GEN_BIT_FLIP(, flip,, 32) + +__RTE_GEN_BIT_TEST(, test,, 64) +__RTE_GEN_BIT_SET(, set,, 64) +__RTE_GEN_BIT_CLEAR(, clear,, 64) +__RTE_GEN_BIT_ASSIGN(, assign,, 64) +__RTE_GEN_BIT_FLIP(, flip,, 64) + /*------------------------ 32-bit relaxed operations ------------------------*/ /** @@ -787,6 +980,68 @@ rte_log2_u64(uint64_t v) #ifdef __cplusplus } + +/* + * Since C++ doesn't support generic selection (i.e., _Generic), + * function overloading is used instead. Such functions must be + * defined outside 'extern "C"' to be accepted by the compiler. + */ + +#undef rte_bit_test +#undef rte_bit_set +#undef rte_bit_clear +#undef rte_bit_assign +#undef rte_bit_flip + +#define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ + static inline void \ + rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ + arg1_type arg1_name) \ + { \ + __rte_bit_ ## fun ## size(addr, arg1_name); \ + } + +#define __RTE_BIT_OVERLOAD_2(fun, qualifier, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 32, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 64, arg1_type, arg1_name) + +#define __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, size, ret_type, arg1_type, \ + arg1_name) \ + static inline ret_type \ + rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ + arg1_type arg1_name) \ + { \ + return __rte_bit_ ## fun ## size(addr, arg1_name); \ + } + +#define __RTE_BIT_OVERLOAD_2R(fun, qualifier, ret_type, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 32, ret_type, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 64, ret_type, arg1_type, \ + arg1_name) + +#define __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + static inline void \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name) \ + { \ + __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + } + +#define __RTE_BIT_OVERLOAD_3(fun, qualifier, arg1_type, arg1_name, arg2_type, \ + arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 32, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 64, arg1_type, arg1_name, \ + arg2_type, arg2_name) + +__RTE_BIT_OVERLOAD_2R(test, const, bool, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(set,, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) +__RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) +__RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) + #endif #endif /* _RTE_BITOPS_H_ */ From patchwork Fri Aug 9 09:04:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143038 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BC42145777; Fri, 9 Aug 2024 11:14:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A90BC42E7D; Fri, 9 Aug 2024 11:14:20 +0200 (CEST) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2068.outbound.protection.outlook.com [40.107.104.68]) by mails.dpdk.org (Postfix) with ESMTP id AD966402A5 for ; Fri, 9 Aug 2024 11:14:18 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=EBjbsHEcWDMCy72gYMZHwGXiRe9nqPDu7fV777//8Fx7EtgdgHs4nK96XEo+jjfgbNSJouhQeAGeCQhZcb+LnEKT0A2wYcSI1NvTmqnC8UYsuvuQ3gKzXBrp9M8Z1Elrs3t8YheHC+HXDXbxyBGpex+hxwMXfP6exdMhirX9adJX18mAJyyDFvzlzIfqPmSlHxseRK9vNmctYKiU5SDeabwOZiXS0j3e/ty+i+lrEqleF3E8ZvsjGZ0JEUu4HOvnkw443MLCpsQ9HgPePBr0+iQzpcIa1L58FUWG69c+0B5bsL18CWg33kw8fBzztuRXU7T1BGfOfoROqxDCcGycYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=d8z68B8fzU8h0PRajH8DnhYus8GmVFVNaHI8wvhWqbw=; b=WaPxl2lxu8mTvfhk9OMcRcunQWufShqnUR70WzkiOzC3fCjFyYHKpJcP3FbJEdAvQoSNtWzW9CP7nHSCSozzv3TkY+J7QwJcPdGMlDIWRoEEn70UPu0LoNt27VRMAdDNZnYdPxJlDpt5cKt3VD3Q8ua4Qu6Jx5JfzrFt9VlJf7kaHJeQz/cWRx/ersINL2eUOMxkilZxzywRa1n0wEag+v/FpSihpdlInIt1n4ebTdCVtzInE2dtkAP5I2sS7M+xaIcmOA9+c0+Og6VBKk4cO05QOl9J+45iEqWbpudj66dCOQ5afMST2Fg1TaOokT8km7h40xLHbcDo01HhxhTWYg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d8z68B8fzU8h0PRajH8DnhYus8GmVFVNaHI8wvhWqbw=; b=t8Ilu3VX5Sf9yPQoQeMh86M1bFbuyj52pqJtVSiClxwcNHE0cSNkNSDdt9QW4PkZHLY8AUGuoTDDud+jwUSg8YeHhinmBNg8OUcAgsf/P1ngKLiikJscUIhmg7AaKVX9NZzZbagjw0qyBCFi92R7/WjSNH4N8C2QQb1FRwWtjstMUl59Xta/fNW0ap0yIYLYOPLn6bpjLhxJjYSuQvaCzWD7XZMK2/N56eheAP7UzaEF8luOD/yTyoiD/FIIOjQAXAXPXDRxZBAz3q6e/7l/KJsTrryZSx2EE8szQuY0/MZT9a5UQO/FGhTkaNweDnZ1rZtAszBCb+YsY+pQ4Un4kA== Received: from PR2P264CA0018.FRAP264.PROD.OUTLOOK.COM (2603:10a6:101::30) by PAVPR07MB9216.eurprd07.prod.outlook.com (2603:10a6:102:316::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.13; Fri, 9 Aug 2024 09:14:16 +0000 Received: from AM4PEPF00025F98.EURPRD83.prod.outlook.com (2603:10a6:101:0:cafe::bc) by PR2P264CA0018.outlook.office365.com (2603:10a6:101::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00025F98.mail.protection.outlook.com (10.167.16.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.2 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.63) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 11:14:15 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id 946D01C006D; Fri, 9 Aug 2024 11:14:15 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH 2/5] eal: add unit tests for bit operations Date: Fri, 9 Aug 2024 11:04:36 +0200 Message-ID: <20240809090439.589295-3-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809090439.589295-1-mattias.ronnblom@ericsson.com> References: <20240505083737.118649-2-mattias.ronnblom@ericsson.com> <20240809090439.589295-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00025F98:EE_|PAVPR07MB9216:EE_ X-MS-Office365-Filtering-Correlation-Id: 90829c63-7430-44d1-e3c2-08dcb853a4cb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|82310400026|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?q?J9vlRFzPwWpOihi0mgtDIdmuZ/w4nmM?= =?utf-8?q?zN/7lfBR918vlwSY162zm2m1T0ss7XH6+uwXN6Gez0DmVKiZkOaglailmTH15OnZa?= =?utf-8?q?9ox0YlGy1jqbWXq3VUklrjU2M7S3Un5kVkjs2Dd3vvRkY+0B+e2KOBoYkrtmDazxe?= =?utf-8?q?iKWRbZAwuHlEOUl6Py1/uCEnjhu5ZgS3kjmZRSbeAg5/jAJGrIz7+GUH+ju8UpEfE?= =?utf-8?q?Zr1yppkcvsW6dAk9gQrT7WzmVD1zcJ6evfqMjaZ66hffAApx3zsCQu+1QQhzOZFsO?= =?utf-8?q?u5PtMuO3teP7Ruc4vQ7p58vwVAnGTpr3CY3iTQPraEyyWnx7m1paXfaGN7x8fd6y8?= =?utf-8?q?WkIgv0IamSU94Po8OsVpN6qiWfHhrjoCjRONLVpEDWdE8ZaDMeZ9nKVevPNFJpFB3?= =?utf-8?q?QE5TLpvFJZlFG6f9TYfifhHGyFh96rGkqMR+7qNfqza1m3zQ9zGPFz2YvFGg5/jK3?= =?utf-8?q?7avIgqhwtiaTM4SOW7jFjZpT1kuWBgnd+0Kw5nQqJLIhVg+gnMMHj8cJIZClu3rdi?= =?utf-8?q?pf5GxU//PnlK++mJnSmF4UdW1BDdym8cIIw08+wJlDxVr1bfrWIAif5LPXhNkHlq1?= =?utf-8?q?OuNx7xwEX7pHUWkZ8LUANQ8P2NSTgCKpMwnQywfcPqXdLLIcoJqImzdEp/pDQUGhV?= =?utf-8?q?OCBAlFvc4gTI+k01vJnqHNhkC5u/7g/KNIC5vOjgAr0uhMs4yVEy5W7vz2m3VZv4E?= =?utf-8?q?jeFADF/mJlAWEc+3xyGzkyk3D+lmFqKRj/DdCDvdLBxm7HHi7PW+5nlqiGLjyt+iU?= =?utf-8?q?Qd/yxJbu6IyAVCHax6llmfLvtOdrRumFIijJiltvmSATjI4vvo8mDaRIkXHn9dAZG?= =?utf-8?q?8cCLYmL1Q26FYEci26S3HItC9EFMDHAMr1/MwNzy9+dmpxRXF7WkZPaCKPXHTNbY0?= =?utf-8?q?dzD/oyGuGmwvOWthsJfV2fdwQhv9eg0nplozPI/pO9ELELHdmdMfuoqmpwBk+eTHM?= =?utf-8?q?Sq2PduHnkD1QF00fczHpvhXdpONeb5PQv/aJxwRSBU3kys7qM7aa/ZKBECzYKQfj8?= =?utf-8?q?LBy6p2DSL2ocYewxrsyRC2vP8Zux3Np2CJ4GJwnoMgeF3QpH77BB08TRUcS2WTlPG?= =?utf-8?q?FD28X17219myvf+mdqDtppNmD3hEIb2V2QiFhdNQIpFT/64oFdzUckjCStVfK7ZhR?= =?utf-8?q?Vr/a+qLxmoX02cHIpnKON5vvY05t8u4ZB7yVWNsT8Rv37qIqoALafxAVErFR9lpfC?= =?utf-8?q?A4YvtV7czwOBsbH4FX5DzF7mzofFkvOYvjGNFXg5Wsu8/NqpmUJqTCMFwCa1x664o?= =?utf-8?q?TMoVgqm7a6ogSl6SwZHN1Ohormv5gxO431BP3Uy3LANUEuSVX8JgsLy9kreUcDFgq?= =?utf-8?q?T8v2fBwYEasPd4GxsN57nhwvkgDsxTn1Aw=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(376014)(82310400026)(36860700013)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 09:14:16.5920 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 90829c63-7430-44d1-e3c2-08dcb853a4cb X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00025F98.EURPRD83.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR07MB9216 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Extend bitops tests to cover the rte_bit_[test|set|clear|assign|flip]() functions. The tests are converted to use the test suite runner framework. RFC v6: * Test rte_bit_*test() usage through const pointers. RFC v4: * Remove redundant line continuations. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- app/test/test_bitops.c | 85 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 70 insertions(+), 15 deletions(-) diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c index 0d4ccfb468..322f58c066 100644 --- a/app/test/test_bitops.c +++ b/app/test/test_bitops.c @@ -1,13 +1,68 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2019 Arm Limited + * Copyright(c) 2024 Ericsson AB */ +#include + #include #include +#include #include "test.h" -uint32_t val32; -uint64_t val64; +#define GEN_TEST_BIT_ACCESS(test_name, set_fun, clear_fun, assign_fun, \ + flip_fun, test_fun, size) \ + static int \ + test_name(void) \ + { \ + uint ## size ## _t reference = (uint ## size ## _t)rte_rand(); \ + unsigned int bit_nr; \ + uint ## size ## _t word = (uint ## size ## _t)rte_rand(); \ + \ + for (bit_nr = 0; bit_nr < size; bit_nr++) { \ + bool reference_bit = (reference >> bit_nr) & 1; \ + bool assign = rte_rand() & 1; \ + if (assign) \ + assign_fun(&word, bit_nr, reference_bit); \ + else { \ + if (reference_bit) \ + set_fun(&word, bit_nr); \ + else \ + clear_fun(&word, bit_nr); \ + \ + } \ + TEST_ASSERT(test_fun(&word, bit_nr) == reference_bit, \ + "Bit %d had unexpected value", bit_nr); \ + flip_fun(&word, bit_nr); \ + TEST_ASSERT(test_fun(&word, bit_nr) != reference_bit, \ + "Bit %d had unflipped value", bit_nr); \ + flip_fun(&word, bit_nr); \ + \ + const uint ## size ## _t *const_ptr = &word; \ + TEST_ASSERT(test_fun(const_ptr, bit_nr) == \ + reference_bit, \ + "Bit %d had unexpected value", bit_nr); \ + } \ + \ + for (bit_nr = 0; bit_nr < size; bit_nr++) { \ + bool reference_bit = (reference >> bit_nr) & 1; \ + TEST_ASSERT(test_fun(&word, bit_nr) == reference_bit, \ + "Bit %d had unexpected value", bit_nr); \ + } \ + \ + TEST_ASSERT(reference == word, "Word had unexpected value"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_ACCESS(test_bit_access32, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 32) + +GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 64) + +static uint32_t val32; +static uint64_t val64; #define MAX_BITS_32 32 #define MAX_BITS_64 64 @@ -117,22 +172,22 @@ test_bit_relaxed_test_set_clear(void) return TEST_SUCCESS; } +static struct unit_test_suite test_suite = { + .suite_name = "Bitops test suite", + .unit_test_cases = { + TEST_CASE(test_bit_access32), + TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_relaxed_set), + TEST_CASE(test_bit_relaxed_clear), + TEST_CASE(test_bit_relaxed_test_set_clear), + TEST_CASES_END() + } +}; + static int test_bitops(void) { - val32 = 0; - val64 = 0; - - if (test_bit_relaxed_set() < 0) - return TEST_FAILED; - - if (test_bit_relaxed_clear() < 0) - return TEST_FAILED; - - if (test_bit_relaxed_test_set_clear() < 0) - return TEST_FAILED; - - return TEST_SUCCESS; + return unit_test_suite_runner(&test_suite); } REGISTER_FAST_TEST(bitops_autotest, true, true, test_bitops); From patchwork Fri Aug 9 09:04:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143040 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D3B2345777; Fri, 9 Aug 2024 11:14:50 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9B45B42E8F; Fri, 9 Aug 2024 11:14:24 +0200 (CEST) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2041.outbound.protection.outlook.com [40.107.104.41]) by mails.dpdk.org (Postfix) with ESMTP id 7CDB142E46 for ; Fri, 9 Aug 2024 11:14:19 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=g5IZxPIGBYUlJ3t3/KdK9pPKkLtzJF29JiWxu0OPan2G+KbQyJmZITigpKInxliGyBVTjbINZisLrkAuhkZ+0TG1NWGXl4bYTxhVoU1DXKdCpYcejQTB7J5yhJC1OtfiqLcLOOaHjslxloKU3qme32CKkFkp3kgmIRweyFODTi8C/MNnwfWQ3DF1VLx+xRKeEZNj5E6UyVEMDi8fJjPx93ujXL6fIKFusonJ8HLG0qG3Ld+caJZZnGFmG1OPr80RSsoqMB4Ht970PLU8N47grz4yYZH3g1TQhJPtuqs0Pl5bIZwR27Vo5HVblT07YhXK3/+nRC4eLEzeo90DQ5SdSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bpTu1BJa9LiTtAkdxe9Wxu7Fs/6r2nSAYg7gg++Hfp8=; b=FMd3QiWfiipeTyvg8om0rWEM5rXOINg+XeBFqjyF7yC0s8KYqq6Wb2G+VH8WF9NzDsYmdCmh+1BfQ71ybYXSN44VkZRI4sOqCobSx1IT2NK3d2xkT9O9WI0V5tgoVfGUbmw8cqZOsY+n+tDGvvncNFDKtFpTO4sUQby09x2cLyVYQlFWxR+gFg2Aqh4R5B3yxweo3rcTgdwFkOm2JaotcK78iPc+jxrY1G+oJHtalVwVfcqspUNjIznuO7DXybbfyS30kYWZXa2MGDdP6e3StKZB5ES5CAGmXEeWlDfFWwMM9EHhDri7m34npujnmRP8WgrLwBEw1mHJ5Hlun9xhHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bpTu1BJa9LiTtAkdxe9Wxu7Fs/6r2nSAYg7gg++Hfp8=; b=SwgbNq2JaGGBAblgkvUB3NnVLUdC9N1uZp51UmyeiMSqC8atBwYNLHE4gBRWPCkcQESp3br/IUWal/YYHkQzCgZR145pzBCFvH2rG6VEdZP7ng/R1aE+vAdoExmTVmbXH0uNWGD/BIahmHflbJ/6kjsSmukk7QtAaNtNgLkSSlaxQrzDVFdB2SwFuHVgV9Tyeq6Tx18FLegKewdjcmRPrej9UDg5kjn+McTFnWnZM9+OfAn0cf7gqJuYdzRNwfmlIPn40+O8eAh4AFDocv4nU8G9RbSwrY4NIs/+STahEb7CmNzUU6aElTPFPCNMpFyx8uEhoMDQMTkUFCoK9tHAFw== Received: from PR1P264CA0140.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:2ce::8) by AS8PR07MB7383.eurprd07.prod.outlook.com (2603:10a6:20b:2ab::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15; Fri, 9 Aug 2024 09:14:16 +0000 Received: from AM4PEPF00025F96.EURPRD83.prod.outlook.com (2603:10a6:102:2ce:cafe::8d) by PR1P264CA0140.outlook.office365.com (2603:10a6:102:2ce::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.14 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00025F96.mail.protection.outlook.com (10.167.16.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.2 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.67) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 11:14:15 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id A4F141C0070; Fri, 9 Aug 2024 11:14:15 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH 3/5] eal: add atomic bit operations Date: Fri, 9 Aug 2024 11:04:37 +0200 Message-ID: <20240809090439.589295-4-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809090439.589295-1-mattias.ronnblom@ericsson.com> References: <20240505083737.118649-2-mattias.ronnblom@ericsson.com> <20240809090439.589295-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00025F96:EE_|AS8PR07MB7383:EE_ X-MS-Office365-Filtering-Correlation-Id: c7dccf88-df58-4055-3761-08dcb853a4bf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?q?AJ0Vsrs0ynpnOIFXk2hbijZ9gz8n6rd?= =?utf-8?q?rkZfGGIoDCb+ubxFbSTKWHR59TLJKjRwUJbTAcYy8oxmv7Qs8ie8ZnoZ9K3jV1bKK?= =?utf-8?q?Jmbhg1isx+9ee2m0uyqiNdx10dS+67nTMcxGzh3SLKCDMYrpnt3z8TYWvuNjvhouG?= =?utf-8?q?E9whREhC0LhgFf0gTYHEU/y4IiLUw/Pvn0GjqHXnDi+oR9oKk0671CTs+bu2hf0vq?= =?utf-8?q?9xuOKedlDFGyID5+ySfB2Lsj7DJVglHcAqlIgU59RoZ05EQINNXWEJccFsLPuScA8?= =?utf-8?q?6rEUGsSY5lR3sGk7zg/Hgb/yoEfKRvSU7kW9vh/OcL3ta9AUqvbJIa1wbzLk0rs5O?= =?utf-8?q?e7H/uc+PwYrhB0f0ZaEHZEnZTQVE1Uomqy6PhlEoS1ZjjLibQEqVyNNqOv2u26UzA?= =?utf-8?q?/KpshthTOqrJabiY4l7lne8HLQU0nVFXZYI4XAImA4B6NMq6uLyhtoOr+lxrrMJOQ?= =?utf-8?q?uw2FS3OvjQBXlYdlB2NBO9JChPliVPaEZKoSOrQkUDItyRzwhBoOo8I4255qawxe9?= =?utf-8?q?Dk8ptBTN/EqmvrB15u5lNPKfRp3sr492dIJndqmMMQ8fmqA/5nc7Sl1QbPuhhiE6V?= =?utf-8?q?G2UrACNk2kGxOQyGNLTz2LwSHjisAVdCWcPaWjoOjnMgfBlRtzxb4rdStU3E25IeF?= =?utf-8?q?yOv/QArgjeoXb0uzfk0MCqvmw1gmKZT2EbNidaxqX//Lj2FTgCN2a4CknisxO9X/p?= =?utf-8?q?aY4iRExu8YpbxTI90nsSvY3uCutjHHKfHclI9uaL5iwsBhrUk0QTO73JptXwz4xUs?= =?utf-8?q?O/aLR2TNfL2B6DqKXeNPbAKgy0nc4fr1JgawX5Qm9IIE+cjzV1IXKC60Xe/ufYjOu?= =?utf-8?q?HHnskr3GpQfWVuKa7WSZRIndpDlQfRKIaNxt+zIbHkSr+T8rbNIV9wrrSOc31sqD2?= =?utf-8?q?Itaa9Bjndt5e40p0mZYGcIYZD+bPVLFei7Iu08Rwv3st+iVyJ06+SurdqdNkMOC5S?= =?utf-8?q?sijO+TcFkILIerGFJRd0ZgSNxXOohb8azHdQ5rh4YBNhDSwbOBcI+/DMbG+o31e+z?= =?utf-8?q?uJlRWKaImVgPAVxJuPmNCxvU6wW15/+vo0ODaVeWbzcdZDGWKhrfnd7lE8pLxNYQr?= =?utf-8?q?i3zy+JZs/JeI/INPngxxLtqZNIE9xCh/Nsw9ydjFEhykwDujBbVdJX4D6/Jz1i8dp?= =?utf-8?q?neV2UdoGKXKAj1DR+oljskFEZE9e3h5AlJIkWNHUiHed9Xnq2qh3MEKjZbQnqZH/t?= =?utf-8?q?bWW7uCmrUSDaFVnu+869mqXODV+bDlj1ACEnCsWcGcrl+uyQWofyZwxHrAU5IxQ9E?= =?utf-8?q?qo0hcHNOtGfpWb/Dtumcm4+hCstXOmgkMybAHKrMupGqa4r7gRLNblGEybnVpREjn?= =?utf-8?q?ovxjolmcphtEgC8FGBSC22LMRt+nQzj/bw=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 09:14:16.5090 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c7dccf88-df58-4055-3761-08dcb853a4bf X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00025F96.EURPRD83.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR07MB7383 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add atomic bit test/set/clear/assign/flip and test-and-set/clear/assign/flip functions. All atomic bit functions allow (and indeed, require) the caller to specify a memory order. RFC v8: * Add missing macro #undef for C++ version of atomic bit flip. RFC v7: * Replace compare-exchange-based rte_bitset_atomic_test_and_*() and flip() with implementations that use the previous value as returned by the atomic fetch function. * Reword documentation to match the non-atomic macro variants. * Remove pointer to for memory model documentation, since there is no documentation for that API. RFC v6: * Have rte_bit_atomic_test() accept const-marked bitsets. RFC v4: * Add atomic bit flip. * Mark macro-generated private functions experimental. RFC v3: * Work around lack of C++ support for _Generic (Tyler Retzlaff). RFC v2: o Add rte_bit_atomic_test_and_assign() (for consistency). o Fix bugs in rte_bit_atomic_test_and_[set|clear](). o Use to support MSVC. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- lib/eal/include/rte_bitops.h | 414 +++++++++++++++++++++++++++++++++++ 1 file changed, 414 insertions(+) diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 3297133e22..4d878099ed 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -21,6 +21,7 @@ #include #include +#include #ifdef __cplusplus extern "C" { @@ -226,6 +227,204 @@ extern "C" { uint32_t *: __rte_bit_flip32, \ uint64_t *: __rte_bit_flip64)(addr, nr) +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Test if a particular bit in a word is set with a particular memory + * order. + * + * Test a bit with the resulting memory load ordered as per the + * specified memory order. + * + * @param addr + * A pointer to the word to query. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit is set, and false otherwise. + */ +#define rte_bit_atomic_test(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test32, \ + const uint32_t *: __rte_bit_atomic_test32, \ + uint64_t *: __rte_bit_atomic_test64, \ + const uint64_t *: __rte_bit_atomic_test64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically set bit in word. + * + * Generic selection macro to atomically set bit specified by @c nr in + * the word pointed to by @c addr to '1', with the memory ordering as + * specified by @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_set(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_set32, \ + uint64_t *: __rte_bit_atomic_set64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically clear bit in word. + * + * Generic selection macro to atomically set bit specified by @c nr in + * the word pointed to by @c addr to '0', with the memory ordering as + * specified by @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_clear(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_clear32, \ + uint64_t *: __rte_bit_atomic_clear64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically assign a value to bit in word. + * + * Generic selection macro to atomically set bit specified by @c nr in the + * word pointed to by @c addr to the value indicated by @c value, with + * the memory ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_assign(addr, nr, value, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_assign32, \ + uint64_t *: __rte_bit_atomic_assign64)(addr, nr, value, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically flip bit in word. + * + * Generic selection macro to atomically negate the value of the bit + * specified by @c nr in the word pointed to by @c addr to the value + * indicated by @c value, with the memory ordering as specified with + * @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_flip(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_flip32, \ + uint64_t *: __rte_bit_atomic_flip64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and set a bit in word. + * + * Generic selection macro to atomically test and set bit specified by + * @c nr in the word pointed to by @c addr to '1', with the memory + * ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_set(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_set32, \ + uint64_t *: __rte_bit_atomic_test_and_set64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and clear a bit in word. + * + * Generic selection macro to atomically test and clear bit specified + * by @c nr in the word pointed to by @c addr to '0', with the memory + * ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_clear(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_clear32, \ + uint64_t *: __rte_bit_atomic_test_and_clear64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and assign a bit in word. + * + * Generic selection macro to atomically test and assign bit specified + * by @c nr in the word pointed to by @c addr the value specified by + * @c value, with the memory ordering as specified with @c + * memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_assign(addr, nr, value, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_assign32, \ + uint64_t *: __rte_bit_atomic_test_and_assign64)(addr, nr, \ + value, \ + memory_order) + #define __RTE_GEN_BIT_TEST(family, fun, qualifier, size) \ __rte_experimental \ static inline bool \ @@ -298,6 +497,145 @@ __RTE_GEN_BIT_CLEAR(, clear,, 64) __RTE_GEN_BIT_ASSIGN(, assign,, 64) __RTE_GEN_BIT_FLIP(, flip,, 64) +#define __RTE_GEN_BIT_ATOMIC_TEST(size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_test ## size(const uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + const RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (const RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + return rte_atomic_load_explicit(a_addr, memory_order) & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_SET(size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_set ## size(uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_or_explicit(a_addr, mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_clear ## size(uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_and_explicit(a_addr, ~mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_FLIP(size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_flip ## size(uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_xor_explicit(a_addr, mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_assign ## size(uint ## size ## _t *addr, \ + unsigned int nr, bool value, \ + int memory_order) \ + { \ + if (value) \ + __rte_bit_atomic_set ## size(addr, nr, memory_order); \ + else \ + __rte_bit_atomic_clear ## size(addr, nr, \ + memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_test_and_set ## size(uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + uint ## size ## _t prev; \ + \ + prev = rte_atomic_fetch_or_explicit(a_addr, mask, \ + memory_order); \ + \ + return prev & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_test_and_clear ## size(uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + uint ## size ## _t prev; \ + \ + prev = rte_atomic_fetch_and_explicit(a_addr, ~mask, \ + memory_order); \ + \ + return prev & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_test_and_assign ## size(uint ## size ## _t *addr, \ + unsigned int nr, \ + bool value, \ + int memory_order) \ + { \ + if (value) \ + return __rte_bit_atomic_test_and_set ## size(addr, nr, \ + memory_order); \ + else \ + return __rte_bit_atomic_test_and_clear ## size(addr, nr, \ + memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_OPS(size) \ + __RTE_GEN_BIT_ATOMIC_TEST(size) \ + __RTE_GEN_BIT_ATOMIC_SET(size) \ + __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ + __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \ + __RTE_GEN_BIT_ATOMIC_FLIP(size) + +__RTE_GEN_BIT_ATOMIC_OPS(32) +__RTE_GEN_BIT_ATOMIC_OPS(64) + /*------------------------ 32-bit relaxed operations ------------------------*/ /** @@ -993,6 +1331,15 @@ rte_log2_u64(uint64_t v) #undef rte_bit_assign #undef rte_bit_flip +#undef rte_bit_atomic_test +#undef rte_bit_atomic_set +#undef rte_bit_atomic_clear +#undef rte_bit_atomic_assign +#undef rte_bit_atomic_flip +#undef rte_bit_atomic_test_and_set +#undef rte_bit_atomic_test_and_clear +#undef rte_bit_atomic_test_and_assign + #define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ static inline void \ rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ @@ -1036,12 +1383,79 @@ rte_log2_u64(uint64_t v) __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 64, arg1_type, arg1_name, \ arg2_type, arg2_name) +#define __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) \ + static inline ret_type \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name) \ + { \ + return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + } + +#define __RTE_BIT_OVERLOAD_3R(fun, qualifier, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 32, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 64, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) + +#define __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, size, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + static inline void \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name, arg3_type arg3_name) \ + { \ + __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ + arg3_name); \ + } + +#define __RTE_BIT_OVERLOAD_4(fun, qualifier, arg1_type, arg1_name, arg2_type, \ + arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 32, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 64, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) + +#define __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + static inline ret_type \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name, arg3_type arg3_name) \ + { \ + return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ + arg3_name); \ + } + +#define __RTE_BIT_OVERLOAD_4R(fun, qualifier, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 32, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 64, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) + __RTE_BIT_OVERLOAD_2R(test, const, bool, unsigned int, nr) __RTE_BIT_OVERLOAD_2(set,, unsigned int, nr) __RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) __RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) __RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) +__RTE_BIT_OVERLOAD_3R(atomic_test, const, bool, unsigned int, nr, + int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_set,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_clear,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_4(atomic_assign,, unsigned int, nr, bool, value, + int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_flip,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3R(atomic_test_and_set,, bool, unsigned int, nr, + int, memory_order) +__RTE_BIT_OVERLOAD_3R(atomic_test_and_clear,, bool, unsigned int, nr, + int, memory_order) +__RTE_BIT_OVERLOAD_4R(atomic_test_and_assign,, bool, unsigned int, nr, + bool, value, int, memory_order) + #endif #endif /* _RTE_BITOPS_H_ */ From patchwork Fri Aug 9 09:04:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143039 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3DCFC45777; Fri, 9 Aug 2024 11:14:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 523D542E89; Fri, 9 Aug 2024 11:14:22 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2066.outbound.protection.outlook.com [40.107.22.66]) by mails.dpdk.org (Postfix) with ESMTP id 4C0F440274 for ; Fri, 9 Aug 2024 11:14:19 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ovy62gbcRcgx1cLJhP5Vq9a9FfSH8QzOepyd2lWzyEqbt84ThvDMN7sPR+c3HTxPOlx/uLcxt2+3INI9PrfTaUmmL4g0WLcdFnWmwWjD/pVAjR4mtU3mBQsSl8K+zwj5URptXgHw3Qcd8I4H7M6s8qQFawRWtZeX3eooixiIt6UFwV6l1zEQIPwTqMKaMHsBy23/C3Cuth9tzuJqyuIf4RJU7fTqtW99Z3Tx04YKa5QZc37A3BjdBi9aYlcMAC5RoN3fGBaaZyEH5MrUagepecFuh8PeAW80F3j1WTmB6EbfvVPE+5M66ytg1ymzf+ZHp4UlPN0UngSpsbhdXtnWQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jqGcpwfHVC/DYFyNL8+lquUhAol/PLB+PTIxYV+C6HY=; b=qp5AxhGPum1UluR6mdnVtkVrn+HprEr0rUQW6mLMnFgB4Da7ZO1BjTLJJaGKE3fsrfXXgf/eA84qSaENrp56MYY/N/XLcHNd1Wzjxop2BzvTYN3hphr5D75rVyllb80d+CXGAZ5hFIufUMSug6ek0P/MAwd0Os6PHygSzYXkGjL0m0es5bTJIT+Wn7XsVPQZVSVpqZZx4qBfOazMv/ZDyZ+Qwr3SmbNp2JWasmA9u5NzvSUTJVRIOvAz+j8bIBRkvJ/u2Dn63UwkWvhLPJtmeCA/FVexIYpuALo2sRrGkmx7i44a0gw3h12taZrSoJ9QZkWN+nkFcsZLMd3CsSicOw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jqGcpwfHVC/DYFyNL8+lquUhAol/PLB+PTIxYV+C6HY=; b=hIxWrlQmBH89IsAGDhsoD1maq+cL/R3dRSL2uthEEWLnA2yQztDNwTTWTSPFKsiCZd9Ue12OwfuM4tcBi2k6ek1XKa0uFQPszEPEPR7EdesY9ifI+OiBFFKogf+/OPEWJQIuiBO+bsSL7cCicTrj7v2dfnoRND0MMLd0Aqtj3wr8ucvqs+jsTtZTTzATK9xARnPQhdSFYPdiNERs0nYk3atCb2WA24JJkv9pMFhbJFG8xVSRDopPrw1KO6It1HFYdAQCt95CvZvZt39gsOe8cHHIQD9Pi1TnbgHQovx2LGs8+UJDRVi9ncXQjGWXeL38pQm7D+r/VrFDm3vXz/BWNw== Received: from DU2PR04CA0048.eurprd04.prod.outlook.com (2603:10a6:10:234::23) by AM9PR07MB7874.eurprd07.prod.outlook.com (2603:10a6:20b:2fc::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15; Fri, 9 Aug 2024 09:14:16 +0000 Received: from DB1PEPF000509FC.eurprd03.prod.outlook.com (2603:10a6:10:234:cafe::68) by DU2PR04CA0048.outlook.office365.com (2603:10a6:10:234::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF000509FC.mail.protection.outlook.com (10.167.242.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.60) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 11:14:15 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id ADE1D1C0073; Fri, 9 Aug 2024 11:14:15 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH 4/5] eal: add unit tests for atomic bit access functions Date: Fri, 9 Aug 2024 11:04:38 +0200 Message-ID: <20240809090439.589295-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809090439.589295-1-mattias.ronnblom@ericsson.com> References: <20240505083737.118649-2-mattias.ronnblom@ericsson.com> <20240809090439.589295-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509FC:EE_|AM9PR07MB7874:EE_ X-MS-Office365-Filtering-Correlation-Id: abeb5db1-45a7-40f2-bbc6-08dcb853a4a0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?q?ve53ql+jIR1VRssOmrmb0Cg4jisILsY?= =?utf-8?q?kg6ce38HKDhX0flDjJLH++BAxnZ2gl2LAslatZCVqulWoNry2C8S0gPbbO3TbgQkg?= =?utf-8?q?Gx2OX9BR75rEBJZAk/rKeu/56jL4rJrVs/KHTdKDFv7NyME3NKpXvcIPfUR0a9kc1?= =?utf-8?q?Eu5J3La2r+QuXtn47S0jfMoGtss0ego79WfyIEexFraA8XJ1bVSeE0UPdzZXIjRRd?= =?utf-8?q?tUq/LiPV9gph/PXXTbGHnZ6H9y+vzx/+8BxtehQnPFJdJtU1grrJSXDHsDxTd6OiM?= =?utf-8?q?Z6HZY6cE6Cl8UsRzZn0Lo9Rns9X3S/I0IiYTXPBtrOL8YQztU4xTJLOb3annWgbjn?= =?utf-8?q?ap2b+NG20Sdik3Gk0IahFd5EH8D9mDZz9ya86cOg99eKtgRkcCjYan6gU50G+KQPT?= =?utf-8?q?Ty0W+l7VwwZUVcTjs7fMPdwZZ6SE6MZaXtfWP9a4KMK1UyUG83x5ypAj21J45P9/Y?= =?utf-8?q?bYL948e11hRCl5FoDx7SjlyYXJfXTrXhe0hyihEzEaaUEEIUjflS1i8i3B4GDiuu/?= =?utf-8?q?uHxn/9KZMkygMe+TKavqU4wBkGlh8SUg5JVCivBUotseWQqyIP0ememLW07ZjsuAB?= =?utf-8?q?SK+QORJvNbI59Sf/k0UHjAOTR9gVYF/l/Y5NsXb5K528l9z8ztGtETW+2WQJUH3iN?= =?utf-8?q?yT9jQ+q+wTcsLdWypYO2ylMWt3XVnNuVdpBPFJyDto/X03Kcgn7OvFrH7TibWJ9F8?= =?utf-8?q?uzQs7pYIY1NG1XP+bV66i3WXJT/gnz9OWkd9Po4kZ4USeTnT6yl0hnq0iTAPLl9ad?= =?utf-8?q?dWBkxSp6NOuAS9XaMUK0W/eg/6Y4yBkQRR1e35JOPUI5eXrKqg08PyTWJVIMoBMlb?= =?utf-8?q?J1N8/BhjO/AJHFvAn0TsGeDMwZ3KEqEAOD9nG7CqO+jtQdNa6dzgMb3E6Ukz+V7RC?= =?utf-8?q?MiVv+pdmY1ma3JZKzxX/Bfc86P1cth/hZ6b/OrtmZVQOikFq6bg+rRf5SVU6U+qsD?= =?utf-8?q?AVSqknJEPCgo+vEVCXPZmPtDoNxEh7a6XeYOdqcTcyJyPEVRFi9IVeTJ4BvFcHles?= =?utf-8?q?dTpbxYGnhzPWPeiJaLQZixSv6tFeeNbJLI76GXU79HbJai7FcOCgRxYthOheqKJNr?= =?utf-8?q?NGj0LX3ljPuddNqWNwHAwWPq2auRLOgpAlfV6eOSm1G4EA1gR/2cZko8/T9iuEzE5?= =?utf-8?q?orbfq7unC1dWHRmNo4jFlnREYtx3fpSqNVldFIL8HQ+VrXcCRMnnzgPFTPuzrdnNd?= =?utf-8?q?L+Pchzop0k03t6W7gm0n3U3gz3+WkorlWWzwalst+OCRSZat0zGvBiZJtgMHsBZnh?= =?utf-8?q?W83Ip0yugrDhcqAUEGFXnK6iYq3icNhI7jwCwxf04BoYNH76n/EJoDKMKghLBkMus?= =?utf-8?q?kOBNGm0Rz9Oj1DwCTfGlZrNJ3J1WAAtz9w=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 09:14:16.2780 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: abeb5db1-45a7-40f2-bbc6-08dcb853a4a0 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509FC.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR07MB7874 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Extend bitops tests to cover the rte_bit_atomic_*() family of functions. RFC v4: * Add atomicity test for atomic bit flip. RFC v3: * Rename variable 'main' to make ICC happy. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- app/test/test_bitops.c | 313 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 312 insertions(+), 1 deletion(-) diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c index 322f58c066..b80216a0a1 100644 --- a/app/test/test_bitops.c +++ b/app/test/test_bitops.c @@ -3,10 +3,13 @@ * Copyright(c) 2024 Ericsson AB */ +#include #include -#include #include +#include +#include +#include #include #include "test.h" @@ -61,6 +64,304 @@ GEN_TEST_BIT_ACCESS(test_bit_access32, rte_bit_set, rte_bit_clear, GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, rte_bit_assign, rte_bit_flip, rte_bit_test, 64) +#define bit_atomic_set(addr, nr) \ + rte_bit_atomic_set(addr, nr, rte_memory_order_relaxed) + +#define bit_atomic_clear(addr, nr) \ + rte_bit_atomic_clear(addr, nr, rte_memory_order_relaxed) + +#define bit_atomic_assign(addr, nr, value) \ + rte_bit_atomic_assign(addr, nr, value, rte_memory_order_relaxed) + +#define bit_atomic_flip(addr, nr) \ + rte_bit_atomic_flip(addr, nr, rte_memory_order_relaxed) + +#define bit_atomic_test(addr, nr) \ + rte_bit_atomic_test(addr, nr, rte_memory_order_relaxed) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_access32, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 32) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_access64, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 64) + +#define PARALLEL_TEST_RUNTIME 0.25 + +#define GEN_TEST_BIT_PARALLEL_ASSIGN(size) \ + \ + struct parallel_access_lcore ## size \ + { \ + unsigned int bit; \ + uint ## size ##_t *word; \ + bool failed; \ + }; \ + \ + static int \ + run_parallel_assign ## size(void *arg) \ + { \ + struct parallel_access_lcore ## size *lcore = arg; \ + uint64_t deadline = rte_get_timer_cycles() + \ + PARALLEL_TEST_RUNTIME * rte_get_timer_hz(); \ + bool value = false; \ + \ + do { \ + bool new_value = rte_rand() & 1; \ + bool use_test_and_modify = rte_rand() & 1; \ + bool use_assign = rte_rand() & 1; \ + \ + if (rte_bit_atomic_test(lcore->word, lcore->bit, \ + rte_memory_order_relaxed) != value) { \ + lcore->failed = true; \ + break; \ + } \ + \ + if (use_test_and_modify) { \ + bool old_value; \ + if (use_assign) \ + old_value = rte_bit_atomic_test_and_assign( \ + lcore->word, lcore->bit, new_value, \ + rte_memory_order_relaxed); \ + else { \ + old_value = new_value ? \ + rte_bit_atomic_test_and_set( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed) : \ + rte_bit_atomic_test_and_clear( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + } \ + if (old_value != value) { \ + lcore->failed = true; \ + break; \ + } \ + } else { \ + if (use_assign) \ + rte_bit_atomic_assign(lcore->word, lcore->bit, \ + new_value, \ + rte_memory_order_relaxed); \ + else { \ + if (new_value) \ + rte_bit_atomic_set( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + else \ + rte_bit_atomic_clear( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + } \ + } \ + \ + value = new_value; \ + } while (rte_get_timer_cycles() < deadline); \ + \ + return 0; \ + } \ + \ + static int \ + test_bit_atomic_parallel_assign ## size(void) \ + { \ + unsigned int worker_lcore_id; \ + uint ## size ## _t word = 0; \ + struct parallel_access_lcore ## size lmain = { \ + .word = &word \ + }; \ + struct parallel_access_lcore ## size lworker = { \ + .word = &word \ + }; \ + \ + if (rte_lcore_count() < 2) { \ + printf("Need multiple cores to run parallel test.\n"); \ + return TEST_SKIPPED; \ + } \ + \ + worker_lcore_id = rte_get_next_lcore(-1, 1, 0); \ + \ + lmain.bit = rte_rand_max(size); \ + do { \ + lworker.bit = rte_rand_max(size); \ + } while (lworker.bit == lmain.bit); \ + \ + int rc = rte_eal_remote_launch(run_parallel_assign ## size, \ + &lworker, worker_lcore_id); \ + TEST_ASSERT(rc == 0, "Worker thread launch failed"); \ + \ + run_parallel_assign ## size(&lmain); \ + \ + rte_eal_mp_wait_lcore(); \ + \ + TEST_ASSERT(!lmain.failed, "Main lcore atomic access failed"); \ + TEST_ASSERT(!lworker.failed, "Worker lcore atomic access " \ + "failed"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_PARALLEL_ASSIGN(32) +GEN_TEST_BIT_PARALLEL_ASSIGN(64) + +#define GEN_TEST_BIT_PARALLEL_TEST_AND_MODIFY(size) \ + \ + struct parallel_test_and_set_lcore ## size \ + { \ + uint ## size ##_t *word; \ + unsigned int bit; \ + uint64_t flips; \ + }; \ + \ + static int \ + run_parallel_test_and_modify ## size(void *arg) \ + { \ + struct parallel_test_and_set_lcore ## size *lcore = arg; \ + uint64_t deadline = rte_get_timer_cycles() + \ + PARALLEL_TEST_RUNTIME * rte_get_timer_hz(); \ + do { \ + bool old_value; \ + bool new_value = rte_rand() & 1; \ + bool use_assign = rte_rand() & 1; \ + \ + if (use_assign) \ + old_value = rte_bit_atomic_test_and_assign( \ + lcore->word, lcore->bit, new_value, \ + rte_memory_order_relaxed); \ + else \ + old_value = new_value ? \ + rte_bit_atomic_test_and_set( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed) : \ + rte_bit_atomic_test_and_clear( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + if (old_value != new_value) \ + lcore->flips++; \ + } while (rte_get_timer_cycles() < deadline); \ + \ + return 0; \ + } \ + \ + static int \ + test_bit_atomic_parallel_test_and_modify ## size(void) \ + { \ + unsigned int worker_lcore_id; \ + uint ## size ## _t word = 0; \ + unsigned int bit = rte_rand_max(size); \ + struct parallel_test_and_set_lcore ## size lmain = { \ + .word = &word, \ + .bit = bit \ + }; \ + struct parallel_test_and_set_lcore ## size lworker = { \ + .word = &word, \ + .bit = bit \ + }; \ + \ + if (rte_lcore_count() < 2) { \ + printf("Need multiple cores to run parallel test.\n"); \ + return TEST_SKIPPED; \ + } \ + \ + worker_lcore_id = rte_get_next_lcore(-1, 1, 0); \ + \ + int rc = rte_eal_remote_launch(run_parallel_test_and_modify ## size, \ + &lworker, worker_lcore_id); \ + TEST_ASSERT(rc == 0, "Worker thread launch failed"); \ + \ + run_parallel_test_and_modify ## size(&lmain); \ + \ + rte_eal_mp_wait_lcore(); \ + \ + uint64_t total_flips = lmain.flips + lworker.flips; \ + bool expected_value = total_flips % 2; \ + \ + TEST_ASSERT(expected_value == rte_bit_test(&word, bit), \ + "After %"PRId64" flips, the bit value " \ + "should be %d", total_flips, expected_value); \ + \ + uint64_t expected_word = 0; \ + rte_bit_assign(&expected_word, bit, expected_value); \ + \ + TEST_ASSERT(expected_word == word, "Untouched bits have " \ + "changed value"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_PARALLEL_TEST_AND_MODIFY(32) +GEN_TEST_BIT_PARALLEL_TEST_AND_MODIFY(64) + +#define GEN_TEST_BIT_PARALLEL_FLIP(size) \ + \ + struct parallel_flip_lcore ## size \ + { \ + uint ## size ##_t *word; \ + unsigned int bit; \ + uint64_t flips; \ + }; \ + \ + static int \ + run_parallel_flip ## size(void *arg) \ + { \ + struct parallel_flip_lcore ## size *lcore = arg; \ + uint64_t deadline = rte_get_timer_cycles() + \ + PARALLEL_TEST_RUNTIME * rte_get_timer_hz(); \ + do { \ + rte_bit_atomic_flip(lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + lcore->flips++; \ + } while (rte_get_timer_cycles() < deadline); \ + \ + return 0; \ + } \ + \ + static int \ + test_bit_atomic_parallel_flip ## size(void) \ + { \ + unsigned int worker_lcore_id; \ + uint ## size ## _t word = 0; \ + unsigned int bit = rte_rand_max(size); \ + struct parallel_flip_lcore ## size lmain = { \ + .word = &word, \ + .bit = bit \ + }; \ + struct parallel_flip_lcore ## size lworker = { \ + .word = &word, \ + .bit = bit \ + }; \ + \ + if (rte_lcore_count() < 2) { \ + printf("Need multiple cores to run parallel test.\n"); \ + return TEST_SKIPPED; \ + } \ + \ + worker_lcore_id = rte_get_next_lcore(-1, 1, 0); \ + \ + int rc = rte_eal_remote_launch(run_parallel_flip ## size, \ + &lworker, worker_lcore_id); \ + TEST_ASSERT(rc == 0, "Worker thread launch failed"); \ + \ + run_parallel_flip ## size(&lmain); \ + \ + rte_eal_mp_wait_lcore(); \ + \ + uint64_t total_flips = lmain.flips + lworker.flips; \ + bool expected_value = total_flips % 2; \ + \ + TEST_ASSERT(expected_value == rte_bit_test(&word, bit), \ + "After %"PRId64" flips, the bit value " \ + "should be %d", total_flips, expected_value); \ + \ + uint64_t expected_word = 0; \ + rte_bit_assign(&expected_word, bit, expected_value); \ + \ + TEST_ASSERT(expected_word == word, "Untouched bits have " \ + "changed value"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_PARALLEL_FLIP(32) +GEN_TEST_BIT_PARALLEL_FLIP(64) + static uint32_t val32; static uint64_t val64; @@ -177,6 +478,16 @@ static struct unit_test_suite test_suite = { .unit_test_cases = { TEST_CASE(test_bit_access32), TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_access32), + TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_atomic_access32), + TEST_CASE(test_bit_atomic_access64), + TEST_CASE(test_bit_atomic_parallel_assign32), + TEST_CASE(test_bit_atomic_parallel_assign64), + TEST_CASE(test_bit_atomic_parallel_test_and_modify32), + TEST_CASE(test_bit_atomic_parallel_test_and_modify64), + TEST_CASE(test_bit_atomic_parallel_flip32), + TEST_CASE(test_bit_atomic_parallel_flip64), TEST_CASE(test_bit_relaxed_set), TEST_CASE(test_bit_relaxed_clear), TEST_CASE(test_bit_relaxed_test_set_clear), From patchwork Fri Aug 9 09:04:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143041 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B1CB945777; Fri, 9 Aug 2024 11:14:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BA1A942E93; Fri, 9 Aug 2024 11:14:25 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2081.outbound.protection.outlook.com [40.107.21.81]) by mails.dpdk.org (Postfix) with ESMTP id EB2D440274 for ; Fri, 9 Aug 2024 11:14:19 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jbdJqSWRQ2MRnqMC/TP2KCgv7OfVoVEOtqHYgFYR90jYDMxyUD/mZR5LDqz9xWFzoIRzsLoKk4tj+lQZvAvQ2e0StIi+4upe2jdjcbuA/uSkh07kKEjoMqpVEmpmV5XKWxyz373bRwTkPCoW1bcaovRWm6KgKSBrGzGAxVA7mfdiG7pFe7iQtiytUYI4a3MyXTSHX4YhKmfcO1spv/ENgUrZPjwiKx1rkK6evWe5eaxtezP5T/wZQzVW3ZQ7i1t3leMVRc4QJb6LNqoLAh2Wvdyl92Pgw5rmNfOu/qa7beZ9sIUNrX5hqr3CSlfQjOzCA5oZd90LSi9ZyZzfiX7GbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jwiGG0fQkLZvNDw6SkRjQhF852jGO3AvqFfPxnYlUEw=; b=qKTPjUCefvdEpywPS5fp+zn2WTacLwAtHN2278PTPLTHbT/uvKtaFOHipQI/4dh8q1Rgltpzlf3RZQ8JvmyZTNx6j1RjGens86uRP0autkZN5AI1rbWn1pyo2GImED1NnDwk50etzMyfQG+wy7JICcsktub8sbqTyVLJ7YI0dVX5hkIjFTfrSUJa02xVSIo0yKRsT2xzc39C0I78hnkeYAJsGK+29hF9Y88K5MGAXLJICXCPzWgZGR/dbbPa8jeQMfYQIpawIr4uMLU/tlZvNRlPhtKO8WmCnJ6gLQcYvDqUMBartqwWHkMZXNqLGH5WHc3FFv79YYdTDhoOlfeiBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jwiGG0fQkLZvNDw6SkRjQhF852jGO3AvqFfPxnYlUEw=; b=zJLpM4jpmW2oLxJn6pQmOQuAqk+ruHHp+8BXv9Q8/2uxdKr4FUv6TOIKK+OhFv02mhC/09DP5Bu66edJUBdWnXEEsL+h3dNdYgUiytKZ4j3nRygakDxcAjoaWFqxlrRZiot3yxoFvp+ilHY5PP4IOgCkXCwnSvvWcFT9frg6oGjKNoQa7hjff41j1S3zDnewtaFt5JiAcUqrICdBvltmK4xKwBzuzrS4vFmY3QpnhJa1evsCK286TPDJTgxAJPwjBZckE8Sdh0TcKPYvKsvXwIi5fDBN4MhTn1AToO7o4O6/Q7tAu265+UV1Ethbu9+nQ2ohKJ4KNDY13Figg33D1A== Received: from PR1P264CA0139.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:2ce::10) by AM7PR07MB6216.eurprd07.prod.outlook.com (2603:10a6:20b:132::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15; Fri, 9 Aug 2024 09:14:17 +0000 Received: from AM4PEPF00025F96.EURPRD83.prod.outlook.com (2603:10a6:102:2ce:cafe::e0) by PR1P264CA0139.outlook.office365.com (2603:10a6:102:2ce::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00025F96.mail.protection.outlook.com (10.167.16.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.2 via Frontend Transport; Fri, 9 Aug 2024 09:14:16 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.67) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 11:14:15 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id C0D331C006A; Fri, 9 Aug 2024 11:14:15 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Joyce Kong , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , =?utf-8?q?Mattia?= =?utf-8?q?s_R=C3=B6nnblom?= Subject: [PATCH 5/5] eal: extend bitops to handle volatile pointers Date: Fri, 9 Aug 2024 11:04:39 +0200 Message-ID: <20240809090439.589295-6-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809090439.589295-1-mattias.ronnblom@ericsson.com> References: <20240505083737.118649-2-mattias.ronnblom@ericsson.com> <20240809090439.589295-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00025F96:EE_|AM7PR07MB6216:EE_ X-MS-Office365-Filtering-Correlation-Id: 3c80aa5d-1aa8-4de0-e666-08dcb853a4ee X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|376014|1800799024|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?q?951irKbBqd12AXEgGQY+tsCmEqOe2si?= =?utf-8?q?KNDnRah9fhOxIdkoaUrEUq+jTDRdzYWjMmWT7KK3Ymt/KRU/fIurNDAraf9IzTgx0?= =?utf-8?q?btzTospKiMZeWYePVW96/TOQ8Ka6i4N9bWQx25n5CHCVCNkwpv8mMY/lwJxO3JA5j?= =?utf-8?q?ZCnuJVv80jufHwtTtTE09RfunZULq2naHANt1SvKy4NYiZNQiwZuE2IAyGz4U8D3C?= =?utf-8?q?G/d9Vb2C0h2RuAfzvzENqrOo7N5oPEKgF0bXodjeeYkayA+l0f99NISYEeE8kxsNk?= =?utf-8?q?W6vMRWNffpNf5qi2Hrc57J3y3ONqLJN5uT0NctDkX7qay4xsRYGSB3iNZvRnNhzK9?= =?utf-8?q?JQbxDw6bkGaKE2QT4l1oKVCfCeW6s9qcwn/qOWCw+imLC+UYBb7sxcyPsi2o4fhI7?= =?utf-8?q?EXN6sbS5zdFRBd/8T8VGx2matTatFBbo+ifoESTaaiLrNN6spML46iZP6KbUjRayL?= =?utf-8?q?MmwK16lm73NrsidWrdi1XA4y0qNbrCdFQqu6CYllhDNkb+uOa1ysxal7kcvBDNJnB?= =?utf-8?q?j+w8nCIbLRu63Pg/5op0fCCiwsWFmCENWexY6XdW4/3QZoQWP4Lifr0lyqBylXpxj?= =?utf-8?q?aZdSpHVfTf6fjLJ14xLCHx2jBSOpXQZHpbtPEzs7sE0V64xDnuddtdQaXxUChWg89?= =?utf-8?q?2c/4uBfDTkb5DBWqqqAMP9DGXRPwPPlow/Z+LopRkBehshXux+wLf/rG9C65VMEEo?= =?utf-8?q?KALrtj41u1GaiNgzxKrb79Eh76pMVJIay+36C1UHI/1cq/CE8XZ2F8ng4fFKgYdvp?= =?utf-8?q?sThbPrgWVcCt3r4WQEQgVvAyM9h94Y60+JzGqhbJJ7+zqhsFrq9cBCWFw6uLyhKD7?= =?utf-8?q?YKBpDFzFJ/BX+29uW+uXqnVFMiQ9ZxAx5skU0aheqFNdLvG4WwZv4wrD5y+RDrrPy?= =?utf-8?q?0shFbp0gf4iFtsR+7kOoBQ4KVw79XHPZFzIlIe8HmYoIvplPd0FfAAJu/2WSZMmeF?= =?utf-8?q?N8nhQPc2cw9yvboKdzMOXAHyFctSnTEZA6LOEvibj2ACgniSwqPh+MiaXur8nY1Sw?= =?utf-8?q?7txkftdoOw7Rlec9W4p7DnA3orGzbpwKRDZiv3S/+OUR3aeHH2kOt9I6ti7/v2HZL?= =?utf-8?q?rzvkSZTev+ivp9mTbmqC+36LhTPRzTK3iQ7xnB1Tdq8uMWQ9z6W+qubvF54o0l0o7?= =?utf-8?q?eokkuobBD0oGXiGPq8l2ErcyFfIys9JUW/XIs7eDUQwXXCaboMIF+VBsiNcSN8484?= =?utf-8?q?9ML9LvtFtu0Z3N0r2HrTs9qfLREQMYDtvBt2esoxJCImOfzRVABcOD9cgVl1dNCLf?= =?utf-8?q?DrwTU83jSKybeBijqYsIyBo0n1FRuCxuLvnyxAtpC4WhnXQcS83KUUyGXroUPx+lz?= =?utf-8?q?2jZbGJeu26byXTQPeuOMkZQYugK9F7eN5A=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(36860700013)(376014)(1800799024)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 09:14:16.8059 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3c80aa5d-1aa8-4de0-e666-08dcb853a4ee X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00025F96.EURPRD83.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR07MB6216 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Have rte_bit_[test|set|clear|assign|flip]() and rte_bit_atomic_*() handle volatile-marked pointers. Signed-off-by: Mattias Rönnblom --- app/test/test_bitops.c | 30 ++- lib/eal/include/rte_bitops.h | 427 ++++++++++++++++++++++------------- 2 files changed, 289 insertions(+), 168 deletions(-) diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c index b80216a0a1..e6e9f7ec44 100644 --- a/app/test/test_bitops.c +++ b/app/test/test_bitops.c @@ -14,13 +14,13 @@ #include "test.h" #define GEN_TEST_BIT_ACCESS(test_name, set_fun, clear_fun, assign_fun, \ - flip_fun, test_fun, size) \ + flip_fun, test_fun, size, mod) \ static int \ test_name(void) \ { \ uint ## size ## _t reference = (uint ## size ## _t)rte_rand(); \ unsigned int bit_nr; \ - uint ## size ## _t word = (uint ## size ## _t)rte_rand(); \ + mod uint ## size ## _t word = (uint ## size ## _t)rte_rand(); \ \ for (bit_nr = 0; bit_nr < size; bit_nr++) { \ bool reference_bit = (reference >> bit_nr) & 1; \ @@ -41,7 +41,7 @@ "Bit %d had unflipped value", bit_nr); \ flip_fun(&word, bit_nr); \ \ - const uint ## size ## _t *const_ptr = &word; \ + const mod uint ## size ## _t *const_ptr = &word; \ TEST_ASSERT(test_fun(const_ptr, bit_nr) == \ reference_bit, \ "Bit %d had unexpected value", bit_nr); \ @@ -59,10 +59,16 @@ } GEN_TEST_BIT_ACCESS(test_bit_access32, rte_bit_set, rte_bit_clear, - rte_bit_assign, rte_bit_flip, rte_bit_test, 32) + rte_bit_assign, rte_bit_flip, rte_bit_test, 32,) GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, - rte_bit_assign, rte_bit_flip, rte_bit_test, 64) + rte_bit_assign, rte_bit_flip, rte_bit_test, 64,) + +GEN_TEST_BIT_ACCESS(test_bit_v_access32, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 32, volatile) + +GEN_TEST_BIT_ACCESS(test_bit_v_access64, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 64, volatile) #define bit_atomic_set(addr, nr) \ rte_bit_atomic_set(addr, nr, rte_memory_order_relaxed) @@ -81,11 +87,19 @@ GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, GEN_TEST_BIT_ACCESS(test_bit_atomic_access32, bit_atomic_set, bit_atomic_clear, bit_atomic_assign, - bit_atomic_flip, bit_atomic_test, 32) + bit_atomic_flip, bit_atomic_test, 32,) GEN_TEST_BIT_ACCESS(test_bit_atomic_access64, bit_atomic_set, bit_atomic_clear, bit_atomic_assign, - bit_atomic_flip, bit_atomic_test, 64) + bit_atomic_flip, bit_atomic_test, 64,) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_v_access32, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 32, volatile) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_v_access64, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 64, volatile) #define PARALLEL_TEST_RUNTIME 0.25 @@ -480,6 +494,8 @@ static struct unit_test_suite test_suite = { TEST_CASE(test_bit_access64), TEST_CASE(test_bit_access32), TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_v_access32), + TEST_CASE(test_bit_v_access64), TEST_CASE(test_bit_atomic_access32), TEST_CASE(test_bit_atomic_access64), TEST_CASE(test_bit_atomic_parallel_assign32), diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 4d878099ed..1355949fb6 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -127,12 +127,16 @@ extern "C" { * @param nr * The index of the bit. */ -#define rte_bit_test(addr, nr) \ - _Generic((addr), \ - uint32_t *: __rte_bit_test32, \ - const uint32_t *: __rte_bit_test32, \ - uint64_t *: __rte_bit_test64, \ - const uint64_t *: __rte_bit_test64)(addr, nr) +#define rte_bit_test(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_test32, \ + const uint32_t *: __rte_bit_test32, \ + volatile uint32_t *: __rte_bit_v_test32, \ + const volatile uint32_t *: __rte_bit_v_test32, \ + uint64_t *: __rte_bit_test64, \ + const uint64_t *: __rte_bit_test64, \ + volatile uint64_t *: __rte_bit_v_test64, \ + const volatile uint64_t *: __rte_bit_v_test64)(addr, nr) /** * @warning @@ -152,10 +156,12 @@ extern "C" { * @param nr * The index of the bit. */ -#define rte_bit_set(addr, nr) \ - _Generic((addr), \ - uint32_t *: __rte_bit_set32, \ - uint64_t *: __rte_bit_set64)(addr, nr) +#define rte_bit_set(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_set32, \ + volatile uint32_t *: __rte_bit_v_set32, \ + uint64_t *: __rte_bit_set64, \ + volatile uint64_t *: __rte_bit_v_set64)(addr, nr) /** * @warning @@ -175,10 +181,12 @@ extern "C" { * @param nr * The index of the bit. */ -#define rte_bit_clear(addr, nr) \ - _Generic((addr), \ - uint32_t *: __rte_bit_clear32, \ - uint64_t *: __rte_bit_clear64)(addr, nr) +#define rte_bit_clear(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_clear32, \ + volatile uint32_t *: __rte_bit_v_clear32, \ + uint64_t *: __rte_bit_clear64, \ + volatile uint64_t *: __rte_bit_v_clear64)(addr, nr) /** * @warning @@ -202,7 +210,9 @@ extern "C" { #define rte_bit_assign(addr, nr, value) \ _Generic((addr), \ uint32_t *: __rte_bit_assign32, \ - uint64_t *: __rte_bit_assign64)(addr, nr, value) + volatile uint32_t *: __rte_bit_v_assign32, \ + uint64_t *: __rte_bit_assign64, \ + volatile uint64_t *: __rte_bit_v_assign64)(addr, nr, value) /** * @warning @@ -225,7 +235,9 @@ extern "C" { #define rte_bit_flip(addr, nr) \ _Generic((addr), \ uint32_t *: __rte_bit_flip32, \ - uint64_t *: __rte_bit_flip64)(addr, nr) + volatile uint32_t *: __rte_bit_v_flip32, \ + uint64_t *: __rte_bit_flip64, \ + volatile uint64_t *: __rte_bit_v_flip64)(addr, nr) /** * @warning @@ -250,9 +262,13 @@ extern "C" { _Generic((addr), \ uint32_t *: __rte_bit_atomic_test32, \ const uint32_t *: __rte_bit_atomic_test32, \ + volatile uint32_t *: __rte_bit_atomic_v_test32, \ + const volatile uint32_t *: __rte_bit_atomic_v_test32, \ uint64_t *: __rte_bit_atomic_test64, \ - const uint64_t *: __rte_bit_atomic_test64)(addr, nr, \ - memory_order) + const uint64_t *: __rte_bit_atomic_test64, \ + volatile uint64_t *: __rte_bit_atomic_v_test64, \ + const volatile uint64_t *: __rte_bit_atomic_v_test64) \ + (addr, nr, memory_order) /** * @warning @@ -274,7 +290,10 @@ extern "C" { #define rte_bit_atomic_set(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_set32, \ - uint64_t *: __rte_bit_atomic_set64)(addr, nr, memory_order) + volatile uint32_t *: __rte_bit_atomic_v_set32, \ + uint64_t *: __rte_bit_atomic_set64, \ + volatile uint64_t *: __rte_bit_atomic_v_set64)(addr, nr, \ + memory_order) /** * @warning @@ -296,7 +315,10 @@ extern "C" { #define rte_bit_atomic_clear(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_clear32, \ - uint64_t *: __rte_bit_atomic_clear64)(addr, nr, memory_order) + volatile uint32_t *: __rte_bit_atomic_v_clear32, \ + uint64_t *: __rte_bit_atomic_clear64, \ + volatile uint64_t *: __rte_bit_atomic_v_clear64)(addr, nr, \ + memory_order) /** * @warning @@ -320,8 +342,11 @@ extern "C" { #define rte_bit_atomic_assign(addr, nr, value, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_assign32, \ - uint64_t *: __rte_bit_atomic_assign64)(addr, nr, value, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_assign32, \ + uint64_t *: __rte_bit_atomic_assign64, \ + volatile uint64_t *: __rte_bit_atomic_v_assign64)(addr, nr, \ + value, \ + memory_order) /** * @warning @@ -344,7 +369,10 @@ extern "C" { #define rte_bit_atomic_flip(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_flip32, \ - uint64_t *: __rte_bit_atomic_flip64)(addr, nr, memory_order) + volatile uint32_t *: __rte_bit_atomic_v_flip32, \ + uint64_t *: __rte_bit_atomic_flip64, \ + volatile uint64_t *: __rte_bit_atomic_v_flip64)(addr, nr, \ + memory_order) /** * @warning @@ -368,8 +396,10 @@ extern "C" { #define rte_bit_atomic_test_and_set(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_test_and_set32, \ - uint64_t *: __rte_bit_atomic_test_and_set64)(addr, nr, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_test_and_set32, \ + uint64_t *: __rte_bit_atomic_test_and_set64, \ + volatile uint64_t *: __rte_bit_atomic_v_test_and_set64) \ + (addr, nr, memory_order) /** * @warning @@ -393,8 +423,10 @@ extern "C" { #define rte_bit_atomic_test_and_clear(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_test_and_clear32, \ - uint64_t *: __rte_bit_atomic_test_and_clear64)(addr, nr, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_test_and_clear32, \ + uint64_t *: __rte_bit_atomic_test_and_clear64, \ + volatile uint64_t *: __rte_bit_atomic_v_test_and_clear64) \ + (addr, nr, memory_order) /** * @warning @@ -421,9 +453,10 @@ extern "C" { #define rte_bit_atomic_test_and_assign(addr, nr, value, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_test_and_assign32, \ - uint64_t *: __rte_bit_atomic_test_and_assign64)(addr, nr, \ - value, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_test_and_assign32, \ + uint64_t *: __rte_bit_atomic_test_and_assign64, \ + volatile uint64_t *: __rte_bit_atomic_v_test_and_assign64) \ + (addr, nr, value, memory_order) #define __RTE_GEN_BIT_TEST(family, fun, qualifier, size) \ __rte_experimental \ @@ -491,93 +524,105 @@ __RTE_GEN_BIT_CLEAR(, clear,, 32) __RTE_GEN_BIT_ASSIGN(, assign,, 32) __RTE_GEN_BIT_FLIP(, flip,, 32) +__RTE_GEN_BIT_TEST(v_, test, volatile, 32) +__RTE_GEN_BIT_SET(v_, set, volatile, 32) +__RTE_GEN_BIT_CLEAR(v_, clear, volatile, 32) +__RTE_GEN_BIT_ASSIGN(v_, assign, volatile, 32) +__RTE_GEN_BIT_FLIP(v_, flip, volatile, 32) + __RTE_GEN_BIT_TEST(, test,, 64) __RTE_GEN_BIT_SET(, set,, 64) __RTE_GEN_BIT_CLEAR(, clear,, 64) __RTE_GEN_BIT_ASSIGN(, assign,, 64) __RTE_GEN_BIT_FLIP(, flip,, 64) -#define __RTE_GEN_BIT_ATOMIC_TEST(size) \ +__RTE_GEN_BIT_TEST(v_, test, volatile, 64) +__RTE_GEN_BIT_SET(v_, set, volatile, 64) +__RTE_GEN_BIT_CLEAR(v_, clear, volatile, 64) +__RTE_GEN_BIT_ASSIGN(v_, assign, volatile, 64) +__RTE_GEN_BIT_FLIP(v_, flip, volatile, 64) + +#define __RTE_GEN_BIT_ATOMIC_TEST(v, qualifier, size) \ __rte_experimental \ static inline bool \ - __rte_bit_atomic_test ## size(const uint ## size ## _t *addr, \ - unsigned int nr, int memory_order) \ + __rte_bit_atomic_ ## v ## test ## size(const qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - const RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (const RTE_ATOMIC(uint ## size ## _t) *)addr; \ + const qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (const qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ return rte_atomic_load_explicit(a_addr, memory_order) & mask; \ } -#define __RTE_GEN_BIT_ATOMIC_SET(size) \ +#define __RTE_GEN_BIT_ATOMIC_SET(v, qualifier, size) \ __rte_experimental \ static inline void \ - __rte_bit_atomic_set ## size(uint ## size ## _t *addr, \ - unsigned int nr, int memory_order) \ + __rte_bit_atomic_ ## v ## set ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ rte_atomic_fetch_or_explicit(a_addr, mask, memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ +#define __RTE_GEN_BIT_ATOMIC_CLEAR(v, qualifier, size) \ __rte_experimental \ static inline void \ - __rte_bit_atomic_clear ## size(uint ## size ## _t *addr, \ - unsigned int nr, int memory_order) \ + __rte_bit_atomic_ ## v ## clear ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ rte_atomic_fetch_and_explicit(a_addr, ~mask, memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_FLIP(size) \ +#define __RTE_GEN_BIT_ATOMIC_FLIP(v, qualifier, size) \ __rte_experimental \ static inline void \ - __rte_bit_atomic_flip ## size(uint ## size ## _t *addr, \ - unsigned int nr, int memory_order) \ + __rte_bit_atomic_ ## v ## flip ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ rte_atomic_fetch_xor_explicit(a_addr, mask, memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ +#define __RTE_GEN_BIT_ATOMIC_ASSIGN(v, qualifier, size) \ __rte_experimental \ static inline void \ - __rte_bit_atomic_assign ## size(uint ## size ## _t *addr, \ - unsigned int nr, bool value, \ - int memory_order) \ + __rte_bit_atomic_## v ## assign ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, bool value, \ + int memory_order) \ { \ if (value) \ - __rte_bit_atomic_set ## size(addr, nr, memory_order); \ + __rte_bit_atomic_ ## v ## set ## size(addr, nr, memory_order); \ else \ - __rte_bit_atomic_clear ## size(addr, nr, \ - memory_order); \ + __rte_bit_atomic_ ## v ## clear ## size(addr, nr, \ + memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(size) \ +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(v, qualifier, size) \ __rte_experimental \ static inline bool \ - __rte_bit_atomic_test_and_set ## size(uint ## size ## _t *addr, \ - unsigned int nr, \ - int memory_order) \ + __rte_bit_atomic_ ## v ## test_and_set ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ uint ## size ## _t prev; \ \ @@ -587,17 +632,17 @@ __RTE_GEN_BIT_FLIP(, flip,, 64) return prev & mask; \ } -#define __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(size) \ +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(v, qualifier, size) \ __rte_experimental \ static inline bool \ - __rte_bit_atomic_test_and_clear ## size(uint ## size ## _t *addr, \ - unsigned int nr, \ - int memory_order) \ + __rte_bit_atomic_ ## v ## test_and_clear ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ { \ RTE_ASSERT(nr < size); \ \ - RTE_ATOMIC(uint ## size ## _t) *a_addr = \ - (RTE_ATOMIC(uint ## size ## _t) *)addr; \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ uint ## size ## _t prev; \ \ @@ -607,34 +652,36 @@ __RTE_GEN_BIT_FLIP(, flip,, 64) return prev & mask; \ } -#define __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \ +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(v, qualifier, size) \ __rte_experimental \ static inline bool \ - __rte_bit_atomic_test_and_assign ## size(uint ## size ## _t *addr, \ - unsigned int nr, \ - bool value, \ - int memory_order) \ + __rte_bit_atomic_ ## v ## test_and_assign ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, \ + bool value, \ + int memory_order) \ { \ if (value) \ - return __rte_bit_atomic_test_and_set ## size(addr, nr, \ - memory_order); \ + return __rte_bit_atomic_ ## v ## test_and_set ## size(addr, nr, memory_order); \ else \ - return __rte_bit_atomic_test_and_clear ## size(addr, nr, \ - memory_order); \ + return __rte_bit_atomic_ ## v ## test_and_clear ## size(addr, nr, memory_order); \ } -#define __RTE_GEN_BIT_ATOMIC_OPS(size) \ - __RTE_GEN_BIT_ATOMIC_TEST(size) \ - __RTE_GEN_BIT_ATOMIC_SET(size) \ - __RTE_GEN_BIT_ATOMIC_CLEAR(size) \ - __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \ - __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(size) \ - __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(size) \ - __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \ - __RTE_GEN_BIT_ATOMIC_FLIP(size) +#define __RTE_GEN_BIT_ATOMIC_OPS(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_SET(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_CLEAR(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_ASSIGN(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(v, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_FLIP(v, qualifier, size) -__RTE_GEN_BIT_ATOMIC_OPS(32) -__RTE_GEN_BIT_ATOMIC_OPS(64) +#define __RTE_GEN_BIT_ATOMIC_OPS_SIZE(size) \ + __RTE_GEN_BIT_ATOMIC_OPS(,, size) \ + __RTE_GEN_BIT_ATOMIC_OPS(v_, volatile, size) + +__RTE_GEN_BIT_ATOMIC_OPS_SIZE(32) +__RTE_GEN_BIT_ATOMIC_OPS_SIZE(64) /*------------------------ 32-bit relaxed operations ------------------------*/ @@ -1340,120 +1387,178 @@ rte_log2_u64(uint64_t v) #undef rte_bit_atomic_test_and_clear #undef rte_bit_atomic_test_and_assign -#define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ +#define __RTE_BIT_OVERLOAD_V_2(family, v, fun, c, size, arg1_type, arg1_name) \ static inline void \ - rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ - arg1_type arg1_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name) \ { \ - __rte_bit_ ## fun ## size(addr, arg1_name); \ + __rte_bit_ ## family ## v ## fun ## size(addr, arg1_name); \ } -#define __RTE_BIT_OVERLOAD_2(fun, qualifier, arg1_type, arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 32, arg1_type, arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 64, arg1_type, arg1_name) +#define __RTE_BIT_OVERLOAD_SZ_2(family, fun, c, size, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_V_2(family,, fun, c, size, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_V_2(family, v_, fun, c volatile, size, \ + arg1_type, arg1_name) -#define __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, size, ret_type, arg1_type, \ - arg1_name) \ +#define __RTE_BIT_OVERLOAD_2(family, fun, c, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(family, fun, c, 32, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(family, fun, c, 64, arg1_type, arg1_name) + +#define __RTE_BIT_OVERLOAD_V_2R(family, v, fun, c, size, ret_type, arg1_type, \ + arg1_name) \ static inline ret_type \ - rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ arg1_type arg1_name) \ { \ - return __rte_bit_ ## fun ## size(addr, arg1_name); \ + return __rte_bit_ ## family ## v ## fun ## size(addr, \ + arg1_name); \ } -#define __RTE_BIT_OVERLOAD_2R(fun, qualifier, ret_type, arg1_type, arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 32, ret_type, arg1_type, \ +#define __RTE_BIT_OVERLOAD_SZ_2R(family, fun, c, size, ret_type, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_V_2R(family,, fun, c, size, ret_type, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_V_2R(family, v_, fun, c volatile, \ + size, ret_type, arg1_type, arg1_name) + +#define __RTE_BIT_OVERLOAD_2R(family, fun, c, ret_type, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2R(family, fun, c, 32, ret_type, arg1_type, \ arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 64, ret_type, arg1_type, \ + __RTE_BIT_OVERLOAD_SZ_2R(family, fun, c, 64, ret_type, arg1_type, \ arg1_name) -#define __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, size, arg1_type, arg1_name, \ - arg2_type, arg2_name) \ +#define __RTE_BIT_OVERLOAD_V_3(family, v, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ static inline void \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name) \ { \ - __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + __rte_bit_ ## family ## v ## fun ## size(addr, arg1_name, \ + arg2_name); \ } -#define __RTE_BIT_OVERLOAD_3(fun, qualifier, arg1_type, arg1_name, arg2_type, \ +#define __RTE_BIT_OVERLOAD_SZ_3(family, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_V_3(family,, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_V_3(family, v_, fun, c volatile, size, arg1_type, \ + arg1_name, arg2_type, arg2_name) + +#define __RTE_BIT_OVERLOAD_3(family, fun, c, arg1_type, arg1_name, arg2_type, \ arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 32, arg1_type, arg1_name, \ + __RTE_BIT_OVERLOAD_SZ_3(family, fun, c, 32, arg1_type, arg1_name, \ arg2_type, arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 64, arg1_type, arg1_name, \ + __RTE_BIT_OVERLOAD_SZ_3(family, fun, c, 64, arg1_type, arg1_name, \ arg2_type, arg2_name) -#define __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, size, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name) \ +#define __RTE_BIT_OVERLOAD_V_3R(family, v, fun, c, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) \ static inline ret_type \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name) \ { \ - return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + return __rte_bit_ ## family ## v ## fun ## size(addr, \ + arg1_name, \ + arg2_name); \ } -#define __RTE_BIT_OVERLOAD_3R(fun, qualifier, ret_type, arg1_type, arg1_name, \ - arg2_type, arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 32, ret_type, arg1_type, \ +#define __RTE_BIT_OVERLOAD_SZ_3R(family, fun, c, size, ret_type, arg1_type, \ arg1_name, arg2_type, arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 64, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name) + __RTE_BIT_OVERLOAD_V_3R(family,, fun, c, size, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_V_3R(family, v_, fun, c volatile, size, \ + ret_type, arg1_type, arg1_name, arg2_type, \ + arg2_name) -#define __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, size, arg1_type, arg1_name, \ - arg2_type, arg2_name, arg3_type, arg3_name) \ +#define __RTE_BIT_OVERLOAD_3R(family, fun, c, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(family, fun, c, 32, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(family, fun, c, 64, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name) + +#define __RTE_BIT_OVERLOAD_V_4(family, v, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ static inline void \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name, arg3_type arg3_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name, \ + arg3_type arg3_name) \ { \ - __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ - arg3_name); \ + __rte_bit_ ## family ## v ## fun ## size(addr, arg1_name, \ + arg2_name, \ + arg3_name); \ } -#define __RTE_BIT_OVERLOAD_4(fun, qualifier, arg1_type, arg1_name, arg2_type, \ - arg2_name, arg3_type, arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 32, arg1_type, arg1_name, \ +#define __RTE_BIT_OVERLOAD_SZ_4(family, fun, c, size, arg1_type, arg1_name, \ arg2_type, arg2_name, arg3_type, arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 64, arg1_type, arg1_name, \ - arg2_type, arg2_name, arg3_type, arg3_name) - -#define __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, size, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name, arg3_type, \ - arg3_name) \ + __RTE_BIT_OVERLOAD_V_4(family,, fun, c, size, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_V_4(family, v_, fun, c volatile, size, \ + arg1_type, arg1_name, arg2_type, arg2_name, \ + arg3_type, arg3_name) + +#define __RTE_BIT_OVERLOAD_4(family, fun, c, arg1_type, arg1_name, arg2_type, \ + arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(family, fun, c, 32, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(family, fun, c, 64, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) + +#define __RTE_BIT_OVERLOAD_V_4R(family, v, fun, c, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ static inline ret_type \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name, arg3_type arg3_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name, \ + arg3_type arg3_name) \ { \ - return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ - arg3_name); \ + return __rte_bit_ ## family ## v ## fun ## size(addr, \ + arg1_name, \ + arg2_name, \ + arg3_name); \ } -#define __RTE_BIT_OVERLOAD_4R(fun, qualifier, ret_type, arg1_type, arg1_name, \ - arg2_type, arg2_name, arg3_type, arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 32, ret_type, arg1_type, \ +#define __RTE_BIT_OVERLOAD_SZ_4R(family, fun, c, size, ret_type, arg1_type, \ arg1_name, arg2_type, arg2_name, arg3_type, \ arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 64, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name, arg3_type, \ - arg3_name) - -__RTE_BIT_OVERLOAD_2R(test, const, bool, unsigned int, nr) -__RTE_BIT_OVERLOAD_2(set,, unsigned int, nr) -__RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) -__RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) -__RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) - -__RTE_BIT_OVERLOAD_3R(atomic_test, const, bool, unsigned int, nr, + __RTE_BIT_OVERLOAD_V_4R(family,, fun, c, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_V_4R(family, v_, fun, c volatile, size, \ + ret_type, arg1_type, arg1_name, arg2_type, \ + arg2_name, arg3_type, arg3_name) + +#define __RTE_BIT_OVERLOAD_4R(family, fun, c, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(family, fun, c, 32, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name, \ + arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(family, fun, c, 64, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name, \ + arg3_type, arg3_name) + +__RTE_BIT_OVERLOAD_2R(, test, const, bool, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(, set,, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(, clear,, unsigned int, nr) +__RTE_BIT_OVERLOAD_3(, assign,, unsigned int, nr, bool, value) +__RTE_BIT_OVERLOAD_2(, flip,, unsigned int, nr) + +__RTE_BIT_OVERLOAD_3R(atomic_, test, const, bool, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3(atomic_set,, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3(atomic_clear,, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_4(atomic_assign,, unsigned int, nr, bool, value, +__RTE_BIT_OVERLOAD_3(atomic_, set,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_, clear,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_4(atomic_, assign,, unsigned int, nr, bool, value, int, memory_order) -__RTE_BIT_OVERLOAD_3(atomic_flip,, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3R(atomic_test_and_set,, bool, unsigned int, nr, +__RTE_BIT_OVERLOAD_3(atomic_, flip,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3R(atomic_, test_and_set,, bool, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3R(atomic_test_and_clear,, bool, unsigned int, nr, +__RTE_BIT_OVERLOAD_3R(atomic_, test_and_clear,, bool, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_4R(atomic_test_and_assign,, bool, unsigned int, nr, +__RTE_BIT_OVERLOAD_4R(atomic_, test_and_assign,, bool, unsigned int, nr, bool, value, int, memory_order) #endif