From patchwork Wed Feb 28 10:09:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 137415 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 61CC143C1F; Wed, 28 Feb 2024 11:17:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BF73540DDB; Wed, 28 Feb 2024 11:17:25 +0100 (CET) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2080.outbound.protection.outlook.com [40.107.22.80]) by mails.dpdk.org (Postfix) with ESMTP id 57B064003C for ; Wed, 28 Feb 2024 11:17:23 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qv6z8mqvFxsST/qgg/APtZzcyvK1XLD4TCOREGgnXSo5Th5iU2Gty09DnKwENsoqrakJMkgss/2SRVZiF1Son+Wu/q7SjRMysRATzf2DK5v/BuBNESZxzaNrFDFKtxM6MJ102kwZtyAVOm1czgJCbPWigl1T9R+QmaP9hW6OvE3srU/ENiCvVpcnLv5OYnPZfW5YLa57Ml+0cY2OYNgy/SKBMJ3ij6irYquRHtA0zbRv0cv3m+5dmHY4e0440/EA2OwUyYSqdGhBcrFsGa7kmaTeE8++9ilg/ZkkhOx2tRCvRdmiO1O8m1VMvBBcPCdrzONHYhXaT7vkV2dgBWE1Xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mkBbgt9rBNpPV9hx8l8X6BB2KyMeY6OM7TKqME0WLOI=; b=iXTVt4Hakmexej+XDE0VwM9iQ3cmnRw8bx22tcmkCij07XxK9yJlyJVuRfkJparwirvrlAY6NFR7ecfbvY7rfZbZ7QfK2N3akKJsWUL0rk2BNDiq0ePiMY+xe6RinT9KwyH8wLQR5g+/tzDbu2PbCH5vtuu6wtQjigPObvp3wQHfXoaBgOz46cwIMViMpH/+H0LWKaXriA2IWLhK5bbTo6bLkh7gGDMtMLko6+aacWUK4WZt9iqQcl3h3XvyqjbiiIiZyuxdxMCEwMS3iwGOcB//D1u+eQvWWp55GRDhhXmLoHs9WaPXzzkc7P2IOfikg8SLQQQimspdRvd8Ty9HEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mkBbgt9rBNpPV9hx8l8X6BB2KyMeY6OM7TKqME0WLOI=; b=C/IP+bdWBBwXSG8KhWJPVkibqK/HDcyOQ9Fvjw4RGeBhwA0ShtRJFnRYv7NxHYB0z0acu7weX3wM9RipQnUpm6hPKLfviC1tor4zaIIH06SHmdgAYfYayFzMvtuDnADRRnU/VZ83x2bEWfKbHHauEy6qpM6+7rYGny0WL7mlN/Mg8zZv5YfRHf7JDDwmLhnXAzS1cz68iTdFX2uBM2oBrNUgZoAmtDF02ap98MHRcISQOnvDOccRbO7dCwGzS8JvFRWfaWo9s54gux6WyA5w+eTXS/j4jz+mBXt2l08FtWD5RDb2o4cU4h39oWynfjI8o7kE7wz9xcNx3mK+/Sk/Gw== Received: from DUZPR01CA0078.eurprd01.prod.exchangelabs.com (2603:10a6:10:46a::20) by DBAPR07MB6757.eurprd07.prod.outlook.com (2603:10a6:10:19a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7202.24; Wed, 28 Feb 2024 10:17:21 +0000 Received: from DU6PEPF0000B61E.eurprd02.prod.outlook.com (2603:10a6:10:46a:cafe::1f) by DUZPR01CA0078.outlook.office365.com (2603:10a6:10:46a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend Transport; Wed, 28 Feb 2024 10:17:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU6PEPF0000B61E.mail.protection.outlook.com (10.167.8.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 10:17:21 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.68) with Microsoft SMTP Server id 15.2.1258.12; Wed, 28 Feb 2024 11:17:20 +0100 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id A32FF380061; Wed, 28 Feb 2024 11:17:20 +0100 (CET) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , =?utf-8?q?Morten_Br=C3=B8rup?= , Stephen Hemminger , =?utf-8?q?Mattias_R=C3=B6nn?= =?utf-8?q?blom?= Subject: [RFC v5 1/6] eal: add static per-lcore memory allocation facility Date: Wed, 28 Feb 2024 11:09:23 +0100 Message-ID: <20240228100928.524277-2-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240228100928.524277-1-mattias.ronnblom@ericsson.com> References: <20240225150330.517225-2-mattias.ronnblom@ericsson.com> <20240228100928.524277-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU6PEPF0000B61E:EE_|DBAPR07MB6757:EE_ X-MS-Office365-Filtering-Correlation-Id: ceee10c8-bb6b-4317-3ec3-08dc3846738c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rqqXMktokhOEUlSc6RBjwOrMLZaxWA5sXwCViPxGtWMo5BNWuvmr02TqvNTJ0OQIRX3iNggGIPtn4evC38oEcr9sRujXn+4l9H85E8xkmJz/QG+TmA6aFp265r3tgcypxYYk4Zqr1y4rCQBO3rhZcAIIF6DJeOecSE1h6diuI4VnD2cbEjD/Hju2/5KORPSMqY5eFNghK+HIWFkayrZxlnCZpfy79TDjwu73+CN7hVw7bIFNCNzVWTULuD16i5SfC9NWJUd2sHBVm2CBFf7KP7+qtxJrEXMoonZVuVSScyBOC33SBHpDvCcVde17jz4ucfS8OSpcGvBNrEkmJ3mkVzPDC07S5/jxigc8QXUtx27RuGZtydP83cSPi7TSqsbrKtEkzekFdUoDFTD5pNBaA7tMlnV7kGvvP9+pTZiBd+e0KYPwtlQmUBnjJbMVap3DLGgqkZIUJtu7dkepNlDO+pvlxEgmEsWc6dmHf3aqeLyFODwATxbxSpazzOvPH5LQBhHYKb8S9DVTjzzYO+ZpZlgZUYXOeqe9jIXkm54AKABr8z4C3/yuYQFVb+64UXemexQvB3Zr+V0pBl7XIHDwuVj+OOfFhtKqmmd/k1+AMUXmWXoqLufFCMfWkS2xBsqZaXfarsxntij+wpCfrPB/lFnUXciVWwp0p2y44xV/EplQhdmA1WzKeAyJudy8hZQMQWiyNylAbyi1z4HnVFzBxKGKNxFzXl9xzs87krse2cERl6+mWgExwm0wqHecep3t X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(36860700004)(82310400014); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 10:17:21.6423 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ceee10c8-bb6b-4317-3ec3-08dc3846738c X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF0000B61E.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR07MB6757 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Introduce DPDK per-lcore id variables, or lcore variables for short. An lcore variable has one value for every current and future lcore id-equipped thread. The primary use case is for statically allocating small chunks of often-used data, which is related logically, but where there are performance benefits to reap from having updates being local to an lcore. Lcore variables are similar to thread-local storage (TLS, e.g., C11 _Thread_local), but decoupling the values' life time with that of the threads. Lcore variables are also similar in terms of functionality provided by FreeBSD kernel's DPCPU_*() family of macros and the associated build-time machinery. DPCPU uses linker scripts, which effectively prevents the reuse of its, otherwise seemingly viable, approach. The currently-prevailing way to solve the same problem as lcore variables is to keep a module's per-lcore data as RTE_MAX_LCORE-sized array of cache-aligned, RTE_CACHE_GUARDed structs. The benefit of lcore variables over this approach is that data related to the same lcore now is close (spatially, in memory), rather than data used by the same module, which in turn avoid excessive use of padding, polluting caches with unused data. RFC v5: * In Doxygen, consistenly use @ (and not \). * The RTE_LCORE_VAR_GET() and SET() convience access macros covered an uncommon use case, where the lcore value is of a primitive type, rather than a struct, and is thus eliminated from the API. (Morten Brørup) * In the wake up GET()/SET() removeal, rename RTE_LCORE_VAR_PTR() RTE_LCORE_VAR_VALUE(). * The underscores are removed from __rte_lcore_var_lcore_ptr() to signal that this function is a part of the public API. * Macro arguments are documented. RFV v4: * Replace large static array with libc heap-allocated memory. One implication of this change is there no longer exists a fixed upper bound for the total amount of memory used by lcore variables. RTE_MAX_LCORE_VAR has changed meaning, and now represent the maximum size of any individual lcore variable value. * Fix issues in example. (Morten Brørup) * Improve access macro type checking. (Morten Brørup) * Refer to the lcore variable handle as "handle" and not "name" in various macros. * Document lack of thread safety in rte_lcore_var_alloc(). * Provide API-level assurance the lcore variable handle is always non-NULL, to all applications to use NULL to mean "not yet allocated". * Note zero-sized allocations are not allowed. * Give API-level guarantee the lcore variable values are zeroed. RFC v3: * Replace use of GCC-specific alignof() with alignof(). * Update example to reflect FOREACH macro name change (in RFC v2). RFC v2: * Use alignof to derive alignment requirements. (Morten Brørup) * Change name of FOREACH to make it distinct from 's *per-EAL-thread* RTE_LCORE_FOREACH(). (Morten Brørup) * Allow user-specified alignment, but limit max to cache line size. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup --- config/rte_config.h | 1 + doc/api/doxy-api-index.md | 1 + lib/eal/common/eal_common_lcore_var.c | 68 +++++ lib/eal/common/meson.build | 1 + lib/eal/include/meson.build | 1 + lib/eal/include/rte_lcore_var.h | 368 ++++++++++++++++++++++++++ lib/eal/version.map | 4 + 7 files changed, 444 insertions(+) create mode 100644 lib/eal/common/eal_common_lcore_var.c create mode 100644 lib/eal/include/rte_lcore_var.h diff --git a/config/rte_config.h b/config/rte_config.h index d743a5c3d3..0dac33d3b9 100644 --- a/config/rte_config.h +++ b/config/rte_config.h @@ -41,6 +41,7 @@ /* EAL defines */ #define RTE_CACHE_GUARD_LINES 1 #define RTE_MAX_HEAPS 32 +#define RTE_MAX_LCORE_VAR 1048576 #define RTE_MAX_MEMSEG_LISTS 128 #define RTE_MAX_MEMSEG_PER_LIST 8192 #define RTE_MAX_MEM_MB_PER_LIST 32768 diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 8c1eb8fafa..a3b8391570 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -99,6 +99,7 @@ The public API headers are grouped by topics: [interrupts](@ref rte_interrupts.h), [launch](@ref rte_launch.h), [lcore](@ref rte_lcore.h), + [lcore-varible](@ref rte_lcore_var.h), [per-lcore](@ref rte_per_lcore.h), [service cores](@ref rte_service.h), [keepalive](@ref rte_keepalive.h), diff --git a/lib/eal/common/eal_common_lcore_var.c b/lib/eal/common/eal_common_lcore_var.c new file mode 100644 index 0000000000..5c353ebd46 --- /dev/null +++ b/lib/eal/common/eal_common_lcore_var.c @@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Ericsson AB + */ + +#include + +#include +#include +#include + +#include + +#include "eal_private.h" + +#define LCORE_BUFFER_SIZE (RTE_MAX_LCORE_VAR * RTE_MAX_LCORE) + +static void *lcore_buffer; +static size_t offset = RTE_MAX_LCORE_VAR; + +static void * +lcore_var_alloc(size_t size, size_t align) +{ + void *handle; + void *value; + + offset = RTE_ALIGN_CEIL(offset, align); + + if (offset + size > RTE_MAX_LCORE_VAR) { + lcore_buffer = aligned_alloc(RTE_CACHE_LINE_SIZE, + LCORE_BUFFER_SIZE); + RTE_VERIFY(lcore_buffer != NULL); + + offset = 0; + } + + handle = RTE_PTR_ADD(lcore_buffer, offset); + + offset += size; + + RTE_LCORE_VAR_FOREACH_VALUE(value, handle) + memset(value, 0, size); + + EAL_LOG(DEBUG, "Allocated %"PRIuPTR" bytes of per-lcore data with a " + "%"PRIuPTR"-byte alignment", size, align); + + return handle; +} + +void * +rte_lcore_var_alloc(size_t size, size_t align) +{ + /* Having the per-lcore buffer size aligned on cache lines + * assures as well as having the base pointer aligned on cache + * size assures that aligned offsets also translate to alipgned + * pointers across all values. + */ + RTE_BUILD_BUG_ON(RTE_MAX_LCORE_VAR % RTE_CACHE_LINE_SIZE != 0); + RTE_ASSERT(align <= RTE_CACHE_LINE_SIZE); + RTE_ASSERT(size <= RTE_MAX_LCORE_VAR); + + /* '0' means asking for worst-case alignment requirements */ + if (align == 0) + align = alignof(max_align_t); + + RTE_ASSERT(rte_is_power_of_2(align)); + + return lcore_var_alloc(size, align); +} diff --git a/lib/eal/common/meson.build b/lib/eal/common/meson.build index 22a626ba6f..d41403680b 100644 --- a/lib/eal/common/meson.build +++ b/lib/eal/common/meson.build @@ -18,6 +18,7 @@ sources += files( 'eal_common_interrupts.c', 'eal_common_launch.c', 'eal_common_lcore.c', + 'eal_common_lcore_var.c', 'eal_common_mcfg.c', 'eal_common_memalloc.c', 'eal_common_memory.c', diff --git a/lib/eal/include/meson.build b/lib/eal/include/meson.build index e94b056d46..9449253e23 100644 --- a/lib/eal/include/meson.build +++ b/lib/eal/include/meson.build @@ -27,6 +27,7 @@ headers += files( 'rte_keepalive.h', 'rte_launch.h', 'rte_lcore.h', + 'rte_lcore_var.h', 'rte_lock_annotations.h', 'rte_malloc.h', 'rte_mcslock.h', diff --git a/lib/eal/include/rte_lcore_var.h b/lib/eal/include/rte_lcore_var.h new file mode 100644 index 0000000000..1db479253d --- /dev/null +++ b/lib/eal/include/rte_lcore_var.h @@ -0,0 +1,368 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Ericsson AB + */ + +#ifndef _RTE_LCORE_VAR_H_ +#define _RTE_LCORE_VAR_H_ + +/** + * @file + * + * RTE Per-lcore id variables + * + * This API provides a mechanism to create and access per-lcore id + * variables in a space- and cycle-efficient manner. + * + * A per-lcore id variable (or lcore variable for short) has one value + * for each EAL thread and registered non-EAL thread. In other words, + * there's one copy of its value for each and every current and future + * lcore id-equipped thread, with the total number of copies amounting + * to @c RTE_MAX_LCORE. + * + * In order to access the values of an lcore variable, a handle is + * used. The type of the handle is a pointer to the value's type + * (e.g., for @c uint32_t lcore variable, the handle is a + * uint32_t *. A handle may be passed between modules and + * threads just like any pointer, but its value is not the address of + * any particular object, but rather just an opaque identifier, stored + * in a typed pointer (to inform the access macro the type of values). + * + * @b Creation + * + * An lcore variable is created in two steps: + * 1. Define a lcore variable handle by using @ref RTE_LCORE_VAR_HANDLE. + * 2. Allocate lcore variable storage and initialize the handle with + * a unique identifier by @ref RTE_LCORE_VAR_ALLOC or + * @ref RTE_LCORE_VAR_INIT. Allocation generally occurs the time of + * module initialization, but may be done at any time. + * + * An lcore variable is not tied to the owning thread's lifetime. It's + * available for use by any thread immediately after having been + * allocated, and continues to be available throughout the lifetime of + * the EAL. + * + * Lcore variables cannot and need not be freed. + * + * @b Access + * + * The value of any lcore variable for any lcore id may be accessed + * from any thread (including unregistered threads), but is should + * generally only *frequently* read from or written to by the owner. + * + * Values of the same lcore variable but owned by to different lcore + * ids *may* be frequently read or written by the owners without the + * risk of false sharing. + * + * An appropriate synchronization mechanism (e.g., atomics) should + * employed to assure there are no data races between the owning + * thread and any non-owner threads accessing the same lcore variable + * instance. + * + * The value of the lcore variable for a particular lcore id is + * accessed using @ref RTE_LCORE_VAR_LCORE_VALUE. + * + * A common pattern is for an EAL thread or a registered non-EAL + * thread to access its own lcore variable value, for which a + * short-hand exists in the form of @ref RTE_LCORE_VAR_VALUE. + * + * Although the handle (as defined by @ref RTE_LCORE_VAR_HANDLE) is a + * pointer with the same type as the value, it may not be directly + * dereferenced and must be treated as an opaque identifier. + * + * Lcore variable handles and value pointers may be freely passed + * between different threads. + * + * @b Storage + * + * An lcore variable's values may by of a primitive type like @c int, + * but would more typically be a @c struct. An application may choose + * to define an lcore variable, which it then it goes on to never + * allocate. + * + * The lcore variable handle introduces a per-variable (not + * per-value/per-lcore id) overhead of @c sizeof(void *) bytes, so + * there are some memory footprint gains to be made by organizing all + * per-lcore id data for a particular module as one lcore variable + * (e.g., as a struct). + * + * The size of a lcore variable's value must be less than the DPDK + * build-time constant @c RTE_MAX_LCORE_VAR. + * + * The lcore variable are stored in a series of lcore buffers, which + * are allocated from the libc heap. Heap allocation failures are + * treated as fatal. + * + * Lcore variables should generally *not* be @ref __rte_cache_aligned + * and need *not* include a @ref RTE_CACHE_GUARD field, since the use + * of these constructs are designed to avoid false sharing. In the + * case of an lcore variable instance, all nearby data structures + * should almost-always be written to by a single thread (the lcore + * variable owner). Adding padding will increase the effective memory + * working set size, and potentially reducing performance. + * + * @b Example + * + * Below is an example of the use of an lcore variable: + * + * @code{.c} + * struct foo_lcore_state { + * int a; + * long b; + * }; + * + * static RTE_LCORE_VAR_HANDLE(struct foo_lcore_state, lcore_states); + * + * long foo_get_a_plus_b(void) + * { + * struct foo_lcore_state *state = RTE_LCORE_VAR_VALUE(lcore_states); + * + * return state->a + state->b; + * } + * + * RTE_INIT(rte_foo_init) + * { + * RTE_LCORE_VAR_ALLOC(lcore_states); + * + * struct foo_lcore_state *state; + * RTE_LCORE_VAR_FOREACH_VALUE(state, lcore_states) { + * (initialize 'state') + * } + * + * (other initialization) + * } + * @endcode + * + * + * @b Alternatives + * + * Lcore variables are designed to replace a pattern exemplified below: + * @code{.c} + * struct foo_lcore_state { + * int a; + * long b; + * RTE_CACHE_GUARD; + * } __rte_cache_aligned; + * + * static struct foo_lcore_state lcore_states[RTE_MAX_LCORE]; + * @endcode + * + * This scheme is simple and effective, but has one drawback: the data + * is organized so that objects related to all lcores for a particular + * module is kept close in memory. At a bare minimum, this forces the + * use of cache-line alignment to avoid false sharing. With CPU + * hardware prefetching and memory loads resulting from speculative + * execution (functions which seemingly are getting more eager faster + * than they are getting more intelligent), one or more "guard" cache + * lines may be required to separate one lcore's data from another's. + * + * Lcore variables has the upside of working with, not against, the + * CPU's assumptions and for example next-line prefetchers may well + * work the way its designers intended (i.e., to the benefit, not + * detriment, of system performance). + * + * Another alternative to @ref rte_lcore_var.h is the @ref + * rte_per_lcore.h API, which make use of thread-local storage (TLS, + * e.g., GCC __thread or C11 _Thread_local). The main differences + * between by using the various forms of TLS (e.g., @ref + * RTE_DEFINE_PER_LCORE or _Thread_local) and the use of lcore + * variables are: + * + * * The existence and non-existence of a thread-local variable + * instance follow that of particular thread's. The data cannot be + * accessed before the thread has been created, nor after it has + * exited. One effect of this is thread-local variables must + * initialized in a "lazy" manner (e.g., at the point of thread + * creation). Lcore variables may be accessed immediately after + * having been allocated (which is usually prior any thread beyond + * the main thread is running). + * * A thread-local variable is duplicated across all threads in the + * process, including unregistered non-EAL threads (i.e., + * "regular" threads). For DPDK applications heavily relying on + * multi-threading (in conjunction to DPDK's "one thread per core" + * pattern), either by having many concurrent threads or + * creating/destroying threads at a high rate, an excessive use of + * thread-local variables may cause inefficiencies (e.g., + * increased thread creation overhead due to thread-local storage + * initialization or increased total RAM footprint usage). Lcore + * variables *only* exist for threads with an lcore id, and thus + * not for such "regular" threads. + * * If data in thread-local storage may be shared between threads + * (i.e., can a pointer to a thread-local variable be passed to + * and successfully dereferenced by non-owning thread) depends on + * the details of the TLS implementation. With GCC __thread and + * GCC _Thread_local, such data sharing is supported. In the C11 + * standard, the result of accessing another thread's + * _Thread_local object is implementation-defined. Lcore variable + * instances may be accessed reliably by any thread. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include + +#include +#include +#include + +/** + * Given the lcore variable type, produces the type of the lcore + * variable handle. + */ +#define RTE_LCORE_VAR_HANDLE_TYPE(type) \ + type * + +/** + * Define a lcore variable handle. + * + * This macro defines a variable which is used as a handle to access + * the various per-lcore id instances of a per-lcore id variable. + * + * The aim with this macro is to make clear at the point of + * declaration that this is an lcore handler, rather than a regular + * pointer. + * + * Add @b static as a prefix in case the lcore variable are only to be + * accessed from a particular translation unit. + */ +#define RTE_LCORE_VAR_HANDLE(type, name) \ + RTE_LCORE_VAR_HANDLE_TYPE(type) name + +/** + * Allocate space for an lcore variable, and initialize its handle. + */ +#define RTE_LCORE_VAR_ALLOC_SIZE_ALIGN(handle, size, align) \ + handle = rte_lcore_var_alloc(size, align) + +/** + * Allocate space for an lcore variable, and initialize its handle, + * with values aligned for any type of object. + */ +#define RTE_LCORE_VAR_ALLOC_SIZE(handle, size) \ + RTE_LCORE_VAR_ALLOC_SIZE_ALIGN(handle, size, 0) + +/** + * Allocate space for an lcore variable of the size and alignment requirements + * suggested by the handler pointer type, and initialize its handle. + */ +#define RTE_LCORE_VAR_ALLOC(handle) \ + RTE_LCORE_VAR_ALLOC_SIZE_ALIGN(handle, sizeof(*(handle)), \ + alignof(typeof(*(handle)))) + +/** + * Allocate an explicitly-sized, explicitly-aligned lcore variable by + * means of a @ref RTE_INIT constructor. + */ +#define RTE_LCORE_VAR_INIT_SIZE_ALIGN(name, size, align) \ + RTE_INIT(rte_lcore_var_init_ ## name) \ + { \ + RTE_LCORE_VAR_ALLOC_SIZE_ALIGN(name, size, align); \ + } + +/** + * Allocate an explicitly-sized lcore variable by means of a @ref + * RTE_INIT constructor. + */ +#define RTE_LCORE_VAR_INIT_SIZE(name, size) \ + RTE_LCORE_VAR_INIT_SIZE_ALIGN(name, size, 0) + +/** + * Allocate an lcore variable by means of a @ref RTE_INIT constructor. + */ +#define RTE_LCORE_VAR_INIT(name) \ + RTE_INIT(rte_lcore_var_init_ ## name) \ + { \ + RTE_LCORE_VAR_ALLOC(name); \ + } + +/** + * Get void pointer to lcore variable instance with the specified + * lcore id. + * + * @param lcore_id + * The lcore id specifying which of the @c RTE_MAX_LCORE value + * instances should be accessed. The lcore id need not be valid + * (e.g., may be @ref LCORE_ID_ANY), but in such a case, the pointer + * is also not valid (and thus should not be dereferenced). + * @param handle + * The lcore variable handle. + */ +static inline void * +rte_lcore_var_lcore_ptr(unsigned int lcore_id, void *handle) +{ + return RTE_PTR_ADD(handle, lcore_id * RTE_MAX_LCORE_VAR); +} + +/** + * Get pointer to lcore variable instance with the specified lcore id. + * + * @param lcore_id + * The lcore id specifying which of the @c RTE_MAX_LCORE value + * instances should be accessed. The lcore id need not be valid + * (e.g., may be @ref LCORE_ID_ANY), but in such a case, the pointer + * is also not valid (and thus should not be dereferenced). + * @param handle + * The lcore variable handle. + */ +#define RTE_LCORE_VAR_LCORE_VALUE(lcore_id, handle) \ + ((typeof(handle))rte_lcore_var_lcore_ptr(lcore_id, handle)) + +/** + * Get pointer to lcore variable instance of the current thread. + * + * May only be used by EAL threads and registered non-EAL threads. + */ +#define RTE_LCORE_VAR_VALUE(handle) \ + RTE_LCORE_VAR_LCORE_VALUE(rte_lcore_id(), handle) + +/** + * Iterate over each lcore id's value for a lcore variable. + * + * @param value + * A pointer set successivly set to point to lcore variable value + * corresponding to every lcore id (up to @c RTE_MAX_LCORE). + * @param handle + * The lcore variable handle. + */ +#define RTE_LCORE_VAR_FOREACH_VALUE(value, handle) \ + for (unsigned int lcore_id = \ + (((value) = RTE_LCORE_VAR_LCORE_VALUE(0, handle)), 0); \ + lcore_id < RTE_MAX_LCORE; \ + lcore_id++, (value) = RTE_LCORE_VAR_LCORE_VALUE(lcore_id, handle)) + +/** + * Allocate space in the per-lcore id buffers for a lcore variable. + * + * The pointer returned is only an opaque identifer of the variable. To + * get an actual pointer to a particular instance of the variable use + * @ref RTE_LCORE_VAR_VALUE or @ref RTE_LCORE_VAR_LCORE_VALUE. + * + * The lcore variable values' memory is set to zero. + * + * The allocation is always successful, barring a fatal exhaustion of + * the per-lcore id buffer space. + * + * rte_lcore_var_alloc() is not multi-thread safe. + * + * @param size + * The size (in bytes) of the variable's per-lcore id value. Must be > 0. + * @param align + * If 0, the values will be suitably aligned for any kind of type + * (i.e., alignof(max_align_t)). Otherwise, the values will be aligned + * on a multiple of *align*, which must be a power of 2 and equal or + * less than @c RTE_CACHE_LINE_SIZE. + * @return + * The id of the variable, stored in a void pointer value. The value + * is always non-NULL. + */ +__rte_experimental +void * +rte_lcore_var_alloc(size_t size, size_t align); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_LCORE_VAR_H_ */ diff --git a/lib/eal/version.map b/lib/eal/version.map index 5e0cd47c82..e90b86115a 100644 --- a/lib/eal/version.map +++ b/lib/eal/version.map @@ -393,6 +393,10 @@ EXPERIMENTAL { # added in 23.07 rte_memzone_max_get; rte_memzone_max_set; + + # added in 24.03 + rte_lcore_var_alloc; + rte_lcore_var; }; INTERNAL { From patchwork Wed Feb 28 10:09:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 137416 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1C99A43C1F; Wed, 28 Feb 2024 11:17:37 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3A6B7410D3; Wed, 28 Feb 2024 11:17:27 +0100 (CET) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04on2052.outbound.protection.outlook.com [40.107.6.52]) by mails.dpdk.org (Postfix) with ESMTP id 6BB8D4027D for ; Wed, 28 Feb 2024 11:17:23 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LsizICzon+AvGVgG0uw34XZzpWgQAtG5t3YIryqWsz7stR3C0RrStLRJKaejjQrSnSbFIREsPSlyTgb+5BnXs7Bhzm4vsMik23lHRo6YtYeHjDt7RdAPbw/3tq3czqlw5OhI/Jtk5LD9c+HNou/RZBBxvqOAXqSbtsBqviVa4A9N0qRzzkL14EIsyXA/ry0IRJN+CftU/u3kZZKjMVNUbeUxzn1smI2PUl1Uq9h3le0r053w96qAsrbZiJgNBBhrcnqxUYhNlFLq4FhqGnDAaReOt1PmO1jdsbul5CekQFGY0GLjXJAvFaQoCzWvFLjtiFO8MQYgUE6yb89bNOINdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wy88gM49EvoC9uWorDMfxcXWJxsryZubX8VaSUoVskY=; b=IwM/E/bbpaz58Ll6/6z3/nJTT+8BQwJEk6VRrJUe3g0t5Mw1IcufkDw7vs1HTjegQ+JQRhhNn2Y/+WRe9kcwmoxm3tKhHRgT3tJ76mWdxO15kbyRchQyfBM5z64Ci/lqfnWFYuIFOakz1AfnLg5lJdBaEPwlPAEdbV/2xupCG+nRpxPyOQBMPFXxxkV3guJAGvo1/tUMTz61aBTXTCQCOZ5zrMw97P7zzcvYL57c60OSoNetybsKbssbK+1MN/WtDFBdymx1gMxv+TbW1xqywqefNAWUE8pWjuYmAEAHq4TkWIF0gB7tCfgL0GwOHr0VOKmBpNqg9pLRdwHqjTf8yg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wy88gM49EvoC9uWorDMfxcXWJxsryZubX8VaSUoVskY=; b=AAaWrfthm89TqxJpLLnXLgaeO3QkuPKGT08if7D5xt6VV7isu9qoZCK+UwYUzxq+JWl1YL2KG3TWKFWot3WMEGfjuxQnV/oloTMpZGIEEcwKV3Pxb1Km95T002hMvcVVZaVOMQv4tEpJ+e6E0bqJLsBoZJp/Pdqp2mPmMl7+6NTNEeS/IL5K6Lum5de5Ny3QZOkfNQkv6ibnPtCBFYGxx5ISkEzH0t3qVzIx7giBINLZbD/TZ2NSvOFVSTHRRGwLZgLcEvYnTWkRZ8RDEc5vKEM5piO7bMCgZfEEQb31Zkm7/Of+mWUvozMTTqP99K+EFoHJBn/kYW6Xr7wkeY+kzA== Received: from DUZPR01CA0083.eurprd01.prod.exchangelabs.com (2603:10a6:10:46a::19) by DBAPR07MB6949.eurprd07.prod.outlook.com (2603:10a6:10:194::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Wed, 28 Feb 2024 10:17:22 +0000 Received: from DU6PEPF0000B61E.eurprd02.prod.outlook.com (2603:10a6:10:46a:cafe::91) by DUZPR01CA0083.outlook.office365.com (2603:10a6:10:46a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend Transport; Wed, 28 Feb 2024 10:17:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU6PEPF0000B61E.mail.protection.outlook.com (10.167.8.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 10:17:21 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.68) with Microsoft SMTP Server id 15.2.1258.12; Wed, 28 Feb 2024 11:17:21 +0100 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 1F216380061; Wed, 28 Feb 2024 11:17:21 +0100 (CET) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , =?utf-8?q?Morten_Br=C3=B8rup?= , Stephen Hemminger , =?utf-8?q?Mattias_R=C3=B6nn?= =?utf-8?q?blom?= Subject: [RFC v5 2/6] eal: add lcore variable test suite Date: Wed, 28 Feb 2024 11:09:24 +0100 Message-ID: <20240228100928.524277-3-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240228100928.524277-1-mattias.ronnblom@ericsson.com> References: <20240225150330.517225-2-mattias.ronnblom@ericsson.com> <20240228100928.524277-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU6PEPF0000B61E:EE_|DBAPR07MB6949:EE_ X-MS-Office365-Filtering-Correlation-Id: 51cf2aeb-faa1-4877-edb6-08dc384673bb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /DrvNA+H7h1Tm+iss5yqpt24X9a3uRslLvQCsUl6CBBkfPwnFACRq2B//SdkGfEeKLwKfPYZcDFLFTsnVvaeW+/bdJ+WPFlc3CjLkfbPue7ifw4jQBAADk1q45+BQyD0j3ZdY8p9LYyZ7IysuATdPvxEIgzGHo8aAbLfthoREYHdJKKElOpG3M55+wjJUTQh2OP+fcNItejPNI7uxJcHnu8VYMXEggdboyKwkSdmnGaDeBMeZTTfo79A0vW4KUhZ3XBHJoWp1R5UULPqsO7P5QN+2dKq8GK6LCLkiHX/gAnKuKqTU7+vHvVhJdBAbF6MehjF6PgNK9/LIoWFhFSesFRpWp+7itg85tZa+rQiHHBD5a8bZ/4jVRrNTPeNarFp+EqwJvwEf3A7GEscQV30HItFOR479dTladY3ZR/V5HrC6pAlnzqmwxkvU6BLkZufkb0GE3YcGTRAKp/zm6EMyGAyuxjXPMVOiMzK/Kw2JlfoUASgA+IFYF4LVAP+FVJabZvj6kz+d0p4lS+ZysbtGeL7BRPk5L1YzLTidRdBG90V0YnEIE3HdIJCYr2LRizNEQJktedl/LjuVZpyl3DapdzyQlE0LRqbakO/nbDH8nARcPuwgC/5eIvj8ZdIv3y5GNroEdmN3SGmNtyBFpLwK1LSv93ewrSMzc/692OAooMuPr23+XGE5OI1DjugGd4OKrn/qNWdBltMQk1g1xISBQpzazhuhWeOc1U6rS8unixm+WYfWwaDEO0frjIeG1ki X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(82310400014)(36860700004); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 10:17:21.9548 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 51cf2aeb-faa1-4877-edb6-08dc384673bb X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF0000B61E.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR07MB6949 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add test suite to exercise the API. RFC v5: * Adapt tests to reflect the removal of the GET() and SET() macros. RFC v4: * Check all lcore id's values for all variables in the many variables test case. * Introduce test case for max-sized lcore variables. RFC v2: * Improve alignment-related test coverage. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup --- app/test/meson.build | 1 + app/test/test_lcore_var.c | 432 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 433 insertions(+) create mode 100644 app/test/test_lcore_var.c diff --git a/app/test/meson.build b/app/test/meson.build index 7d909039ae..846affa98c 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -103,6 +103,7 @@ source_file_deps = { 'test_ipsec_sad.c': ['ipsec'], 'test_kvargs.c': ['kvargs'], 'test_latencystats.c': ['ethdev', 'latencystats', 'metrics'] + sample_packet_forward_deps, + 'test_lcore_var.c': [], 'test_lcores.c': [], 'test_link_bonding.c': ['ethdev', 'net_bond', 'net'] + packet_burst_generator_deps + virtual_pmd_deps, diff --git a/app/test/test_lcore_var.c b/app/test/test_lcore_var.c new file mode 100644 index 0000000000..e07d13460f --- /dev/null +++ b/app/test/test_lcore_var.c @@ -0,0 +1,432 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Ericsson AB + */ + +#include +#include +#include + +#include +#include +#include + +#include "test.h" + +#define MIN_LCORES 2 + +RTE_LCORE_VAR_HANDLE(int, test_int); +RTE_LCORE_VAR_HANDLE(char, test_char); +RTE_LCORE_VAR_HANDLE(long, test_long_sized); +RTE_LCORE_VAR_HANDLE(short, test_short); +RTE_LCORE_VAR_HANDLE(long, test_long_sized_aligned); + +struct int_checker_state { + int old_value; + int new_value; + bool success; +}; + +static void +rand_blk(void *blk, size_t size) +{ + size_t i; + + for (i = 0; i < size; i++) + ((unsigned char *)blk)[i] = (unsigned char)rte_rand(); +} + +static bool +is_ptr_aligned(const void *ptr, size_t align) +{ + return ptr != NULL ? (uintptr_t)ptr % align == 0 : false; +} + +static int +check_int(void *arg) +{ + struct int_checker_state *state = arg; + + int *ptr = RTE_LCORE_VAR_VALUE(test_int); + + bool naturally_aligned = is_ptr_aligned(ptr, sizeof(int)); + + bool equal = *(RTE_LCORE_VAR_VALUE(test_int)) == state->old_value; + + state->success = equal && naturally_aligned; + + *ptr = state->new_value; + + return 0; +} + +RTE_LCORE_VAR_INIT(test_int); +RTE_LCORE_VAR_INIT(test_char); +RTE_LCORE_VAR_INIT_SIZE(test_long_sized, 32); +RTE_LCORE_VAR_INIT(test_short); +RTE_LCORE_VAR_INIT_SIZE_ALIGN(test_long_sized_aligned, sizeof(long), + RTE_CACHE_LINE_SIZE); + +static int +test_int_lvar(void) +{ + unsigned int lcore_id; + + struct int_checker_state states[RTE_MAX_LCORE] = {}; + + RTE_LCORE_FOREACH_WORKER(lcore_id) { + struct int_checker_state *state = &states[lcore_id]; + + state->old_value = (int)rte_rand(); + state->new_value = (int)rte_rand(); + + *RTE_LCORE_VAR_LCORE_VALUE(lcore_id, test_int) = + state->old_value; + } + + RTE_LCORE_FOREACH_WORKER(lcore_id) + rte_eal_remote_launch(check_int, &states[lcore_id], lcore_id); + + rte_eal_mp_wait_lcore(); + + RTE_LCORE_FOREACH_WORKER(lcore_id) { + struct int_checker_state *state = &states[lcore_id]; + int value; + + TEST_ASSERT(state->success, "Unexpected value " + "encountered on lcore %d", lcore_id); + + value = *RTE_LCORE_VAR_LCORE_VALUE(lcore_id, test_int); + TEST_ASSERT_EQUAL(state->new_value, value, + "Lcore %d failed to update int", lcore_id); + } + + /* take the opportunity to test the foreach macro */ + int *v; + lcore_id = 0; + RTE_LCORE_VAR_FOREACH_VALUE(v, test_int) { + TEST_ASSERT_EQUAL(states[lcore_id].new_value, *v, + "Unexpected value on lcore %d during " + "iteration", lcore_id); + lcore_id++; + } + + return TEST_SUCCESS; +} + +static int +test_sized_alignment(void) +{ + long *v; + + RTE_LCORE_VAR_FOREACH_VALUE(v, test_long_sized) { + TEST_ASSERT(is_ptr_aligned(v, alignof(long)), + "Type-derived alignment failed"); + } + + RTE_LCORE_VAR_FOREACH_VALUE(v, test_long_sized_aligned) { + TEST_ASSERT(is_ptr_aligned(v, RTE_CACHE_LINE_SIZE), + "Explicit alignment failed"); + } + + return TEST_SUCCESS; +} + +/* private, larger, struct */ +#define TEST_STRUCT_DATA_SIZE 1234 + +struct test_struct { + uint8_t data[TEST_STRUCT_DATA_SIZE]; +}; + +static RTE_LCORE_VAR_HANDLE(char, before_struct); +static RTE_LCORE_VAR_HANDLE(struct test_struct, test_struct); +static RTE_LCORE_VAR_HANDLE(char, after_struct); + +struct struct_checker_state { + struct test_struct old_value; + struct test_struct new_value; + bool success; +}; + +static int check_struct(void *arg) +{ + struct struct_checker_state *state = arg; + + struct test_struct *lcore_struct = RTE_LCORE_VAR_VALUE(test_struct); + + bool properly_aligned = + is_ptr_aligned(test_struct, alignof(struct test_struct)); + + bool equal = memcmp(lcore_struct->data, state->old_value.data, + TEST_STRUCT_DATA_SIZE) == 0; + + state->success = equal && properly_aligned; + + memcpy(lcore_struct->data, state->new_value.data, + TEST_STRUCT_DATA_SIZE); + + return 0; +} + +static int +test_struct_lvar(void) +{ + unsigned int lcore_id; + + RTE_LCORE_VAR_ALLOC(before_struct); + RTE_LCORE_VAR_ALLOC(test_struct); + RTE_LCORE_VAR_ALLOC(after_struct); + + struct struct_checker_state states[RTE_MAX_LCORE]; + + RTE_LCORE_FOREACH_WORKER(lcore_id) { + struct struct_checker_state *state = &states[lcore_id]; + + rand_blk(state->old_value.data, TEST_STRUCT_DATA_SIZE); + rand_blk(state->new_value.data, TEST_STRUCT_DATA_SIZE); + + memcpy(RTE_LCORE_VAR_LCORE_VALUE(lcore_id, test_struct)->data, + state->old_value.data, TEST_STRUCT_DATA_SIZE); + } + + RTE_LCORE_FOREACH_WORKER(lcore_id) + rte_eal_remote_launch(check_struct, &states[lcore_id], + lcore_id); + + rte_eal_mp_wait_lcore(); + + RTE_LCORE_FOREACH_WORKER(lcore_id) { + struct struct_checker_state *state = &states[lcore_id]; + struct test_struct *lstruct = + RTE_LCORE_VAR_LCORE_VALUE(lcore_id, test_struct); + + TEST_ASSERT(state->success, "Unexpected value encountered on " + "lcore %d", lcore_id); + + bool equal = memcmp(lstruct->data, state->new_value.data, + TEST_STRUCT_DATA_SIZE) == 0; + + TEST_ASSERT(equal, "Lcore %d failed to update struct", + lcore_id); + } + + RTE_LCORE_FOREACH_WORKER(lcore_id) { + char before = + *RTE_LCORE_VAR_LCORE_VALUE(lcore_id, before_struct); + char after = + *RTE_LCORE_VAR_LCORE_VALUE(lcore_id, after_struct); + + TEST_ASSERT_EQUAL(before, 0, "Lcore variable before test " + "struct was modified on lcore %d", lcore_id); + TEST_ASSERT_EQUAL(after, 0, "Lcore variable after test " + "struct was modified on lcore %d", lcore_id); + } + + return TEST_SUCCESS; +} + +#define TEST_ARRAY_SIZE 99 + +typedef uint16_t test_array_t[TEST_ARRAY_SIZE]; + +static void test_array_init_rand(test_array_t a) +{ + size_t i; + for (i = 0; i < TEST_ARRAY_SIZE; i++) + a[i] = (uint16_t)rte_rand(); +} + +static bool test_array_equal(test_array_t a, test_array_t b) +{ + size_t i; + for (i = 0; i < TEST_ARRAY_SIZE; i++) { + if (a[i] != b[i]) + return false; + } + return true; +} + +static void test_array_copy(test_array_t dst, const test_array_t src) +{ + size_t i; + for (i = 0; i < TEST_ARRAY_SIZE; i++) + dst[i] = src[i]; +} + +static RTE_LCORE_VAR_HANDLE(char, before_array); +static RTE_LCORE_VAR_HANDLE(test_array_t, test_array); +static RTE_LCORE_VAR_HANDLE(char, after_array); + +struct array_checker_state { + test_array_t old_value; + test_array_t new_value; + bool success; +}; + +static int check_array(void *arg) +{ + struct array_checker_state *state = arg; + + test_array_t *lcore_array = RTE_LCORE_VAR_VALUE(test_array); + + bool properly_aligned = + is_ptr_aligned(lcore_array, alignof(test_array_t)); + + bool equal = test_array_equal(*lcore_array, state->old_value); + + state->success = equal && properly_aligned; + + test_array_copy(*lcore_array, state->new_value); + + return 0; +} + +static int +test_array_lvar(void) +{ + unsigned int lcore_id; + + RTE_LCORE_VAR_ALLOC(before_array); + RTE_LCORE_VAR_ALLOC(test_array); + RTE_LCORE_VAR_ALLOC(after_array); + + struct array_checker_state states[RTE_MAX_LCORE]; + + RTE_LCORE_FOREACH_WORKER(lcore_id) { + struct array_checker_state *state = &states[lcore_id]; + + test_array_init_rand(state->new_value); + test_array_init_rand(state->old_value); + + test_array_copy(*RTE_LCORE_VAR_LCORE_VALUE(lcore_id, + test_array), + state->old_value); + } + + RTE_LCORE_FOREACH_WORKER(lcore_id) + rte_eal_remote_launch(check_array, &states[lcore_id], + lcore_id); + + rte_eal_mp_wait_lcore(); + + RTE_LCORE_FOREACH_WORKER(lcore_id) { + struct array_checker_state *state = &states[lcore_id]; + test_array_t *larray = + RTE_LCORE_VAR_LCORE_VALUE(lcore_id, test_array); + + TEST_ASSERT(state->success, "Unexpected value encountered on " + "lcore %d", lcore_id); + + bool equal = test_array_equal(*larray, state->new_value); + + TEST_ASSERT(equal, "Lcore %d failed to update array", + lcore_id); + } + + RTE_LCORE_FOREACH_WORKER(lcore_id) { + char before = + *RTE_LCORE_VAR_LCORE_VALUE(lcore_id, before_array); + char after = + *RTE_LCORE_VAR_LCORE_VALUE(lcore_id, after_array); + + TEST_ASSERT_EQUAL(before, 0, "Lcore variable before test " + "array was modified on lcore %d", lcore_id); + TEST_ASSERT_EQUAL(after, 0, "Lcore variable after test " + "array was modified on lcore %d", lcore_id); + } + + return TEST_SUCCESS; +} + +#define MANY_LVARS (2 * RTE_MAX_LCORE_VAR / sizeof(uint32_t)) + +static int +test_many_lvars(void) +{ + uint32_t **handlers = malloc(sizeof(uint32_t *) * MANY_LVARS); + unsigned int i; + + TEST_ASSERT(handlers != NULL, "Unable to allocate memory"); + + for (i = 0; i < MANY_LVARS; i++) { + unsigned int lcore_id; + + RTE_LCORE_VAR_ALLOC(handlers[i]); + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + uint32_t *v = + RTE_LCORE_VAR_LCORE_VALUE(lcore_id, handlers[i]); + *v = (uint32_t)(i * lcore_id); + } + } + + for (i = 0; i < MANY_LVARS; i++) { + unsigned int lcore_id; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + uint32_t v = *RTE_LCORE_VAR_LCORE_VALUE(lcore_id, + handlers[i]); + TEST_ASSERT_EQUAL((uint32_t)(i * lcore_id), v, + "Unexpected lcore variable value on " + "lcore %d", lcore_id); + } + } + + free(handlers); + + return TEST_SUCCESS; +} + +static int +test_large_lvar(void) +{ + RTE_LCORE_VAR_HANDLE(unsigned char, large); + unsigned int lcore_id; + + RTE_LCORE_VAR_ALLOC_SIZE(large, RTE_MAX_LCORE_VAR); + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + unsigned char *ptr = RTE_LCORE_VAR_LCORE_VALUE(lcore_id, large); + + memset(ptr, (unsigned char)lcore_id, RTE_MAX_LCORE_VAR); + } + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + unsigned char *ptr = RTE_LCORE_VAR_LCORE_VALUE(lcore_id, large); + size_t i; + + for (i = 0; i < RTE_MAX_LCORE_VAR; i++) + TEST_ASSERT_EQUAL(ptr[i], (unsigned char)lcore_id, + "Large lcore variable value is " + "corrupted on lcore %d.", + lcore_id); + } + + return TEST_SUCCESS; +} + +static struct unit_test_suite lcore_var_testsuite = { + .suite_name = "lcore variable autotest", + .unit_test_cases = { + TEST_CASE(test_int_lvar), + TEST_CASE(test_sized_alignment), + TEST_CASE(test_struct_lvar), + TEST_CASE(test_array_lvar), + TEST_CASE(test_many_lvars), + TEST_CASE(test_large_lvar), + TEST_CASES_END() + }, +}; + +static int test_lcore_var(void) +{ + if (rte_lcore_count() < MIN_LCORES) { + printf("Not enough cores for lcore_var_autotest; expecting at " + "least %d.\n", MIN_LCORES); + return TEST_SKIPPED; + } + + return unit_test_suite_runner(&lcore_var_testsuite); +} + +REGISTER_FAST_TEST(lcore_var_autotest, true, false, test_lcore_var); From patchwork Wed Feb 28 10:09:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 137417 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D70F843C1F; Wed, 28 Feb 2024 11:17:46 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0130341149; Wed, 28 Feb 2024 11:17:37 +0100 (CET) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on2043.outbound.protection.outlook.com [40.107.7.43]) by mails.dpdk.org (Postfix) with ESMTP id 7FE9A4027D for ; Wed, 28 Feb 2024 11:17:24 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FzNmO1wYZr+L39ByLtnmWpoQWpnR76kgzHr6DD921j/ctkdqH+8miXJIrXeWFVVgBcWE3JtAJQdGqXJAoe2LgMtUBwlYJoOVXDi9ixJ3z/c9t8kvWAfOauEtt7esc2ZRGZKCkotf1hImloP2i+Ts1X2bh/gAUQ46r2wQLVdbg5v7QWZP4uWKGRUVVgev/oBRt3hpOrXwH2JTOQBnDYdoYqlHxgLXWxp4meg7B+SwzJT9b4lIYMkh6gW4QVYUGuCP6dyS/AYoCGloOO/kOH1/j5x+9IZFcge/6QHonQieBTrSTUVK5044YAOHyTR/H3BAQqXVlMf5eDadQuyey/IhyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xjwMbnBnEUMqtIj3sDNKGiBopp35z1bhhwJOhJUYdZ8=; b=ls7eiujqvMNfuvkU1OijtvoFz9EQ45l8wYSnAH73nJA1gPJ6MPNT4ep0IK7wqmyx/vqV7mxGACcWyX3rv6rtgCUgLtceJmeY6+vJvKvxFt7OIlMYC3xKPw3YY/HhPUVaArMemQP8lGl7MLwSWxSlylWBM9iy6rrCJGtQH7afcwhRshEa8G9q1k/MhCUiBLDouY64i39dCQUBEemjhHGqGytoCMEFJ8qFTOWHraDVrsVDRLt/J5z7VRgLL7Hb0+RiekumdTWabsGdMCSS84JINHdBipnpXQIVMCdiAif8NGzH3tt5jaYS9LKjryzrWyrAFHwPAbJCKUpLs3pxRY9bKw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xjwMbnBnEUMqtIj3sDNKGiBopp35z1bhhwJOhJUYdZ8=; b=SgbfS8GuWOHcQUAv9pEDuq2AF0cpY8ZUHCACCVzR8VfV2xTD7aBRwacUqzW3dbtSalctNgBwe/COAIfZaSZm4yQiyRoUXK+9afOD5rj6XDPTqSY4AeY8VoVU4yoC0/qMiJoBjqtxDDBDD5RmUBw56jklIrdy5F3sicUh8vrBiIzbo/b/XleRcB/hU7TiyNd6kKcr6uV25AYRZihFMuZYusRkxAmFshhcNdcgWqoXvzAdVjgh3sSUjWL0aPdTAgh9EyU/plAt73cuNHOZSh4eWBjKCsgk65UW5GpDaxuhbAoyye4oT6EW0QQtfUu8ixNApDCuBOW+fxP/oirfwRUr0Q== Received: from AS8PR04CA0110.eurprd04.prod.outlook.com (2603:10a6:20b:31e::25) by DB9PR07MB7689.eurprd07.prod.outlook.com (2603:10a6:10:1f9::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Wed, 28 Feb 2024 10:17:22 +0000 Received: from AMS1EPF0000004A.eurprd04.prod.outlook.com (2603:10a6:20b:31e:cafe::6b) by AS8PR04CA0110.outlook.office365.com (2603:10a6:20b:31e::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.50 via Frontend Transport; Wed, 28 Feb 2024 10:17:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AMS1EPF0000004A.mail.protection.outlook.com (10.167.16.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 10:17:22 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.63) with Microsoft SMTP Server id 15.2.1258.12; Wed, 28 Feb 2024 11:17:21 +0100 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 9742B380061; Wed, 28 Feb 2024 11:17:21 +0100 (CET) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , =?utf-8?q?Morten_Br=C3=B8rup?= , Stephen Hemminger , =?utf-8?q?Mattias_R=C3=B6nn?= =?utf-8?q?blom?= Subject: [RFC v5 3/6] random: keep PRNG state in lcore variable Date: Wed, 28 Feb 2024 11:09:25 +0100 Message-ID: <20240228100928.524277-4-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240228100928.524277-1-mattias.ronnblom@ericsson.com> References: <20240225150330.517225-2-mattias.ronnblom@ericsson.com> <20240228100928.524277-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS1EPF0000004A:EE_|DB9PR07MB7689:EE_ X-MS-Office365-Filtering-Correlation-Id: c0f388c5-7a1c-44d6-f8d3-08dc384673d0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zzG9JoeEdmiycAm51IfqaS8Mn0oW1fkILy+cE2v9k+gfiseSTuA7RVPL72dVAnP2W7wzoPj4qTDg9yYUDzGI9ane9ya7VejD1gMOkTgX9jPpaxcjO89v+pIs2zaIg8bVLzwEYkHS30+BW4aEvPfRPPDMeZBsW5uE+6yDXM8xrZEctvltNgY3cxja55OBQms5pbqic/ABYygdD5xpGkSuy8sa6G0DljH1DVYi2Uk7ay7f9CLH2YkCfUlQBsk9a9bZz6KcNv5PrDrBoKnKg7p/0Hp+6NVT9GwplHiyGKkcqKQfDi7tljfnlUyQsymYJ41Sy0IHD6gFIGpSdNL/Xz5bi99grzKcNUzqnHRlkT/EbC4mc4dval+m4g61etYfN3WMZmg2q6Q2ti1WaQecY01u3yjCNR3Ufak30qLW447a+R5B9rBFih/Q9N+LO88+Whac9iV9RSwNhnmeQ+hoVgLZDvmjAsIKRk9/SxEyUd4hrMvF3zFALiVFrae/6vwVJ+PJ3BSUj1oWD05YpicC67yi/4bFFrQC0+VDqxfmjJUgjb8rO7mIf67hpDFVlSFCTJW8W3AZHF7CWZEDaaq6/e3bb9IhnYakmNqZGoCErhuCTZ3P+91A5s0SDRxd4h8cYNCgkd38OEFbYGNzjtma0kZ0QcpkX/DsDNlqBTWE+kzTwAHVk2CPTYsZk/2vbdRFFGj3OQZ8YYo3ln66uUOiPk7ZYjIgQsiUPr7y8HVuZBA+Jg45sDovruP32Cw7k6F9ooZt X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(82310400014)(36860700004); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 10:17:22.1084 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c0f388c5-7a1c-44d6-f8d3-08dc384673d0 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF0000004A.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR07MB7689 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Replace keeping PRNG state in a RTE_MAX_LCORE-sized static array of cache-aligned and RTE_CACHE_GUARDed struct instances with keeping the same state in a more cache-friendly lcore variable. RFC v3: * Remove cache alignment on unregistered threads' rte_rand_state. (Morten Brørup) Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup --- lib/eal/common/rte_random.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/lib/eal/common/rte_random.c b/lib/eal/common/rte_random.c index 7709b8f2c6..b265660283 100644 --- a/lib/eal/common/rte_random.c +++ b/lib/eal/common/rte_random.c @@ -11,6 +11,7 @@ #include #include #include +#include #include struct rte_rand_state { @@ -19,14 +20,12 @@ struct rte_rand_state { uint64_t z3; uint64_t z4; uint64_t z5; - RTE_CACHE_GUARD; -} __rte_cache_aligned; +}; -/* One instance each for every lcore id-equipped thread, and one - * additional instance to be shared by all others threads (i.e., all - * unregistered non-EAL threads). - */ -static struct rte_rand_state rand_states[RTE_MAX_LCORE + 1]; +RTE_LCORE_VAR_HANDLE(struct rte_rand_state, rand_state); + +/* instance to be shared by all unregistered non-EAL threads */ +static struct rte_rand_state unregistered_rand_state; static uint32_t __rte_rand_lcg32(uint32_t *seed) @@ -85,8 +84,14 @@ rte_srand(uint64_t seed) unsigned int lcore_id; /* add lcore_id to seed to avoid having the same sequence */ - for (lcore_id = 0; lcore_id < RTE_DIM(rand_states); lcore_id++) - __rte_srand_lfsr258(seed + lcore_id, &rand_states[lcore_id]); + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + struct rte_rand_state *lcore_state = + RTE_LCORE_VAR_LCORE_VALUE(lcore_id, rand_state); + + __rte_srand_lfsr258(seed + lcore_id, lcore_state); + } + + __rte_srand_lfsr258(seed + lcore_id, &unregistered_rand_state); } static __rte_always_inline uint64_t @@ -124,11 +129,10 @@ struct rte_rand_state *__rte_rand_get_state(void) idx = rte_lcore_id(); - /* last instance reserved for unregistered non-EAL threads */ if (unlikely(idx == LCORE_ID_ANY)) - idx = RTE_MAX_LCORE; + return &unregistered_rand_state; - return &rand_states[idx]; + return RTE_LCORE_VAR_VALUE(rand_state); } uint64_t @@ -228,6 +232,8 @@ RTE_INIT(rte_rand_init) { uint64_t seed; + RTE_LCORE_VAR_ALLOC(rand_state); + seed = __rte_random_initial_seed(); rte_srand(seed); From patchwork Wed Feb 28 10:09:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 137418 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 44C2143C1F; Wed, 28 Feb 2024 11:17:53 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3880A427E8; Wed, 28 Feb 2024 11:17:38 +0100 (CET) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on2040.outbound.protection.outlook.com [40.107.7.40]) by mails.dpdk.org (Postfix) with ESMTP id 86F3D40DDB for ; Wed, 28 Feb 2024 11:17:24 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QiNr7Eljkb8nM+MRdrsbgpfBShpIDiH+g51pHX8TbWYyDWM16QtA6JppLV7bgO/TOfzEryXNj55Zfe2S7QI4JziCWCm6eSnNKoVB2Qwr6ycJAvoI2hRnLLzQs3bkpwsLnhdLKxu+yKriBFmHU85EGWqk3TDTXgIzllulMKQlxAPjP2p+sBK9PEnZCCst4X8AYjpvCqFb0KPuri4DHxCI9hW/uo7erOYWj/MpQLB0sKocjpIJNBmt2jZi8bJDMbw3HS+YHgUy6w+YUEb+qxW3bHKypghaPYa9u3DMVBiW4xJtqh3PehRBAQgAWSSJ1X5IC1dHs4CXvFZ1IHFgTHxa/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=r6P73IjIGICPMlen7Dz9PuYmagHog0FDx6vR5RaQ6yk=; b=cZ0+HxD/T+29NxzmbkhSI+XpQbwymZK/ZpLpVz8piVWvp8bNGtC03IVtH6anWeTMUUWZewJ4lLkDEbTMHjVz7iEHAA4VhA25xR4bQO1lSl1i6jd8x1/cYUY2SgBT86vA2iquEowHYGFMu8wpc/nSywlkvpUXWxQaUV0h8ZOEQ1EjvyaHURS21Nm/g1tIPe3bGqqnlJuOtcwZyc0mt9lULH03f4t4LQtvovV2z5QZqZfOZv2GMl+559qpxM0WdXaVaPOYXnGJig0leiWXJHlPr998F0B8C6u2s49CFEmUKAwxNhqZeqDfkmFY29IOXmBeCU4XRDc9g+cD+uGIzwvvzA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r6P73IjIGICPMlen7Dz9PuYmagHog0FDx6vR5RaQ6yk=; b=M+GtHtkoYYzCw1n+/+6T9QwiXT/YpNFMfy02RJ8qSo/XyeFh1akwEwlI83GxN3k4ZHu9ttc0f30DLjz96ew9W7BmXYp96f2WFoRb2cbHf4vF35cwdu8L9KB3Zu7UpakNO6Hmc4l08Kaa4Z2fh/NiVNDUpROpASgr6dqsliTFT5cSyRvWxI0CCZG82lVk/sVAj7PNisORDns5JQyHplvFavPN8GwJx41rK+HQOEQj1Dx2GGASwYWm8vDSE0ras9ue4rrA2GsFDfetXtqS7HqWoze0qPwwesDDygGJDJpJWObSEQS+Q6+pxmrUyJVp9Xep277EqkkJaTAs/zDcY54w/g== Received: from DU2PR04CA0233.eurprd04.prod.outlook.com (2603:10a6:10:2b1::28) by PAXPR07MB7821.eurprd07.prod.outlook.com (2603:10a6:102:13c::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.37; Wed, 28 Feb 2024 10:17:23 +0000 Received: from DU6PEPF0000B620.eurprd02.prod.outlook.com (2603:10a6:10:2b1:cafe::32) by DU2PR04CA0233.outlook.office365.com (2603:10a6:10:2b1::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.27 via Frontend Transport; Wed, 28 Feb 2024 10:17:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU6PEPF0000B620.mail.protection.outlook.com (10.167.8.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 10:17:22 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.68) with Microsoft SMTP Server id 15.2.1258.12; Wed, 28 Feb 2024 11:17:22 +0100 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 1A85D380061; Wed, 28 Feb 2024 11:17:22 +0100 (CET) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , =?utf-8?q?Morten_Br=C3=B8rup?= , Stephen Hemminger , =?utf-8?q?Mattias_R=C3=B6nn?= =?utf-8?q?blom?= Subject: [RFC v5 4/6] power: keep per-lcore state in lcore variable Date: Wed, 28 Feb 2024 11:09:26 +0100 Message-ID: <20240228100928.524277-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240228100928.524277-1-mattias.ronnblom@ericsson.com> References: <20240225150330.517225-2-mattias.ronnblom@ericsson.com> <20240228100928.524277-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU6PEPF0000B620:EE_|PAXPR07MB7821:EE_ X-MS-Office365-Filtering-Correlation-Id: b369456c-0003-4d25-66dd-08dc38467430 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: T8jOL4VOpU9YFzpF/KMbE4nE1vDxFezT7FQZa9ZYfvP1NVlEgRo+LQkbiZmqMHkgvZ4Z6jiEre7D3+vo8216Blk64kjHh3GAw0h+qpFpuYk1S9K9TPpSIxX4yW+seH56fE1C6mSEnahm2gkHB5LDVIXUe6+WyDkkA9jfzydWOY0RPX7WVCYtXM991ExnFPIk60hB73R35/xSMLIh9deL0Tz16KcogBDPLHOl4t3jtv4OhBu2q3Bixj59Oj1zVzKYMVSBYY1Rn0JUXE+t3MDxr2n8qo3++ovQXgcG1bvVhZCoRrO/pXdelf76pYoYrhdHzK+AhIU0pCC3GiXL4zzQcM4ZlWQZroJ24XgEiiXskK4v/AmYxRC4riiHvNPbTeZrPtTq+2/rfI7JUfCJmH1lV3ygUCtffCTd7FTEIliqd+DbwbsoVrw22Jc/Azd8TCWfqFrfrwa1VjLKAFDaHDIui5Lxo/4p/yX+3MWirQ49umjiwoOnE2wJ78A4JOxpbzHLYQ0/bnFRGZLO0/MHJUB7eaU1Rd3dpX5ynZiP62KMbqxizrjvWwUjq3OkT1fkUGGG79C7q3Mr0DQLcuQD6kCW/g3XC+SirtREDT8B+uhJGiCJKuHtZews4ZBjXyRtTrnlGRRGNojOKyh+Fh1G9khP20mqTUtRwzOQRQR9pl7091VCBw/jgDXnMSLgkfO8JUYdTDGDcFblNIcUUoqsH0fctmGVOhAUQDMbb4NGcRx5mtidx5hYPIJtnbodjnQQql50 X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(82310400014)(36860700004); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 10:17:22.7025 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b369456c-0003-4d25-66dd-08dc38467430 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF0000B620.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR07MB7821 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Replace static array of cache-aligned structs with an lcore variable, to slightly benefit code simplicity and performance. RFC v3: * Replace for loop with FOREACH macro. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup --- lib/power/rte_power_pmd_mgmt.c | 36 ++++++++++++++++------------------ 1 file changed, 17 insertions(+), 19 deletions(-) diff --git a/lib/power/rte_power_pmd_mgmt.c b/lib/power/rte_power_pmd_mgmt.c index 591fc69f36..595c8091e6 100644 --- a/lib/power/rte_power_pmd_mgmt.c +++ b/lib/power/rte_power_pmd_mgmt.c @@ -5,6 +5,7 @@ #include #include +#include #include #include #include @@ -68,8 +69,8 @@ struct pmd_core_cfg { /**< Number of queues ready to enter power optimized state */ uint64_t sleep_target; /**< Prevent a queue from triggering sleep multiple times */ -} __rte_cache_aligned; -static struct pmd_core_cfg lcore_cfgs[RTE_MAX_LCORE]; +}; +static RTE_LCORE_VAR_HANDLE(struct pmd_core_cfg, lcore_cfgs); static inline bool queue_equal(const union queue *l, const union queue *r) @@ -252,12 +253,11 @@ clb_multiwait(uint16_t port_id __rte_unused, uint16_t qidx __rte_unused, struct rte_mbuf **pkts __rte_unused, uint16_t nb_rx, uint16_t max_pkts __rte_unused, void *arg) { - const unsigned int lcore = rte_lcore_id(); struct queue_list_entry *queue_conf = arg; struct pmd_core_cfg *lcore_conf; const bool empty = nb_rx == 0; - lcore_conf = &lcore_cfgs[lcore]; + lcore_conf = RTE_LCORE_VAR_VALUE(lcore_cfgs); /* early exit */ if (likely(!empty)) @@ -317,13 +317,12 @@ clb_pause(uint16_t port_id __rte_unused, uint16_t qidx __rte_unused, struct rte_mbuf **pkts __rte_unused, uint16_t nb_rx, uint16_t max_pkts __rte_unused, void *arg) { - const unsigned int lcore = rte_lcore_id(); struct queue_list_entry *queue_conf = arg; struct pmd_core_cfg *lcore_conf; const bool empty = nb_rx == 0; uint32_t pause_duration = rte_power_pmd_mgmt_get_pause_duration(); - lcore_conf = &lcore_cfgs[lcore]; + lcore_conf = RTE_LCORE_VAR_VALUE(lcore_cfgs); if (likely(!empty)) /* early exit */ @@ -358,9 +357,8 @@ clb_scale_freq(uint16_t port_id __rte_unused, uint16_t qidx __rte_unused, struct rte_mbuf **pkts __rte_unused, uint16_t nb_rx, uint16_t max_pkts __rte_unused, void *arg) { - const unsigned int lcore = rte_lcore_id(); const bool empty = nb_rx == 0; - struct pmd_core_cfg *lcore_conf = &lcore_cfgs[lcore]; + struct pmd_core_cfg *lcore_conf = RTE_LCORE_VAR_VALUE(lcore_cfgs); struct queue_list_entry *queue_conf = arg; if (likely(!empty)) { @@ -518,7 +516,7 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lcore_id, uint16_t port_id, goto end; } - lcore_cfg = &lcore_cfgs[lcore_id]; + lcore_cfg = RTE_LCORE_VAR_LCORE_VALUE(lcore_id, lcore_cfgs); /* check if other queues are stopped as well */ ret = cfg_queues_stopped(lcore_cfg); @@ -619,7 +617,7 @@ rte_power_ethdev_pmgmt_queue_disable(unsigned int lcore_id, } /* no need to check queue id as wrong queue id would not be enabled */ - lcore_cfg = &lcore_cfgs[lcore_id]; + lcore_cfg = RTE_LCORE_VAR_LCORE_VALUE(lcore_id, lcore_cfgs); /* check if other queues are stopped as well */ ret = cfg_queues_stopped(lcore_cfg); @@ -769,21 +767,21 @@ rte_power_pmd_mgmt_get_scaling_freq_max(unsigned int lcore) } RTE_INIT(rte_power_ethdev_pmgmt_init) { - size_t i; - int j; + struct pmd_core_cfg *lcore_cfg; + int i; + + RTE_LCORE_VAR_ALLOC(lcore_cfgs); /* initialize all tailqs */ - for (i = 0; i < RTE_DIM(lcore_cfgs); i++) { - struct pmd_core_cfg *cfg = &lcore_cfgs[i]; - TAILQ_INIT(&cfg->head); - } + RTE_LCORE_VAR_FOREACH_VALUE(lcore_cfg, lcore_cfgs) + TAILQ_INIT(&lcore_cfg->head); /* initialize config defaults */ emptypoll_max = 512; pause_duration = 1; /* scaling defaults out of range to ensure not used unless set by user or app */ - for (j = 0; j < RTE_MAX_LCORE; j++) { - scale_freq_min[j] = 0; - scale_freq_max[j] = UINT32_MAX; + for (i = 0; i < RTE_MAX_LCORE; i++) { + scale_freq_min[i] = 0; + scale_freq_max[i] = UINT32_MAX; } } From patchwork Wed Feb 28 10:09:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 137420 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 050B843C1F; Wed, 28 Feb 2024 11:18:06 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C3B0242EC1; Wed, 28 Feb 2024 11:17:40 +0100 (CET) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04on2087.outbound.protection.outlook.com [40.107.8.87]) by mails.dpdk.org (Postfix) with ESMTP id 5C2AF40ED3 for ; Wed, 28 Feb 2024 11:17:25 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SGoSO4jGvaxmFPllsjxRswyEVJ7uqA1gVOjSuqGaqMRrGqSahnuLQcRqptnEmWJYLxY+Bw/+NWMuiWHnzuWTb3UKhEKVc9/SDor7WuAddIPXCeg8mG6g9VrDh0wLcBTjlRQSaG9hyrVqlcXAvul16ycLWV5JvoY75lkaiU5yzN1y9DtHZntdEDUPCuxZe/DlZ5VIouujSPzLF6WuZiDVJ73/AHoHWKnJngLqXONRLOTUTEHTDDh4y8WJRwUPEmEYOHDGCoJ7ooIw9L+R8qmHVPC9Woe86zZytjbc2u4PWue3u9/1SubRTi9UFQtnWmyfQkJPTW6ewCZJ74ZGUnc5Sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Zje9q7DJMu0UxiO9XmpivsL8yq5KgzW3xDAjmWeERKg=; b=HrZfh05V12EiQaLsONty88HgfJRApkg8n+Uymjfht9D6+0MQw9Fb2eLpnXIvlzHdnTZyT8gC1tqSd0nuuBbnF8r3Ae9W0twI8UV1xmV+2avbM5qOd/2nE3R7DDTgrsV6vfeMXXpc2Jzvj3BLmIaz1OHk6+KTzsToDB9IdR5EDqDt1hTgRJi7/iahF6rKljfBYp2yeZY4DKbVymjGhPeSmsXHpcbVMCJ6XkWSnIMtZkecZgLxv9wjVayM7O6jaIvPM0mIU8giRILQK0CT8XA0WooQ5Z+uf8vbdLb+A44xpo8o+Lh6lEnn3LrkfZ5Y0TDXurSxbLmElfl7mRWi9+wpaw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Zje9q7DJMu0UxiO9XmpivsL8yq5KgzW3xDAjmWeERKg=; b=gj/OzfJHCvh64uZ4Vh5beX+KFdxN07acjxWw7jb6wIBvsXg8YTRIsT2mN+Ia/6FovzbQhfDIjKuMXs2D1TnJ8qrn5W+kfCu7B4csjNe8+CKsn4YbEdFAiRmLzUZ1CNqkPht3Zv5VHyRT/S/yO+SC68Osdog97AO8bNpWwkW5JFaJmw3EYNBqMf7Qzhq/IKrkdPIqMq+tM/yYKM9avD1WxhkL0Ywoqha9IrLqs0We1WiLvvfvhRc4pdDMiE4yzBzNdgtAoRpNxLOF5nXF1L6rDP2xqxY6PS46VKlB6Fc1Q1dna7zClNsW/O4xwqjiZzSHidLF1sH9ulrpHzWGSpEsMA== Received: from DU7P195CA0016.EURP195.PROD.OUTLOOK.COM (2603:10a6:10:54d::23) by AM8PR07MB8278.eurprd07.prod.outlook.com (2603:10a6:20b:325::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.36; Wed, 28 Feb 2024 10:17:23 +0000 Received: from DU2PEPF00028D00.eurprd03.prod.outlook.com (2603:10a6:10:54d:cafe::7b) by DU7P195CA0016.outlook.office365.com (2603:10a6:10:54d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.50 via Frontend Transport; Wed, 28 Feb 2024 10:17:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU2PEPF00028D00.mail.protection.outlook.com (10.167.242.184) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 10:17:23 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.61) with Microsoft SMTP Server id 15.2.1258.12; Wed, 28 Feb 2024 11:17:22 +0100 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 8F57E380061; Wed, 28 Feb 2024 11:17:22 +0100 (CET) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , =?utf-8?q?Morten_Br=C3=B8rup?= , Stephen Hemminger , =?utf-8?q?Mattias_R=C3=B6nn?= =?utf-8?q?blom?= Subject: [RFC v5 5/6] service: keep per-lcore state in lcore variable Date: Wed, 28 Feb 2024 11:09:27 +0100 Message-ID: <20240228100928.524277-6-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240228100928.524277-1-mattias.ronnblom@ericsson.com> References: <20240225150330.517225-2-mattias.ronnblom@ericsson.com> <20240228100928.524277-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF00028D00:EE_|AM8PR07MB8278:EE_ X-MS-Office365-Filtering-Correlation-Id: a6d4beeb-cf77-4422-af00-08dc38467481 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7RjevXsCnfLplDaGx04wqwbeWayTACEos6ws/S2lQmmxqZTInIeV5F5PN+3WO+raqw6UhjFDmnpEkrs5JI2CYRmgv/QMg33PjdRa1D+ankJY3dVvU5MsToHZcVo1SZ9KdpVuJIDkV/mPCUfHpLZW2zEvxj26EQTg72yeM04WyiDR3GZS/5D3Jvc2+VnOFfMFCf1Pj9AQIZOjXFubXPIQjhZK1wx6YRYqq3M9ISmawtQ8jZUFNt0Z9oWHqLM1taBObR0jwu/K+hrVWoOL3oklZdPvXENU0q8aH9sU9qCIR3uJzNbX+LHzcpiMItckZoCBDR8hojGsJJ80QVif52h+qpfwjXtd6yplxV0JY9QGfTYyW5u9ZE5pSg0pp4ffunYyMOKYygstXlHI3LqZWI6tJxgxlKBQ22hQOfdes7PoBWsL/XxnYYb/vb78PPMe++aCaK17NXirFXI6vULMy6qOusWRBSp48g2C++Hr314DzEvMRdm3gvNiQBB7FbhrPZwLY8lwfjYYybBhJUu3nMEUR6EtjF/7dnH29yfUimqjig25QNnZqNKKkR2y9SF++iK2dEacdgH28KP6uOhKn6sTHXUq4nWjiHThFQIZlRhAw2oaNC+NkXmwtzmw34D5vd4njZxJGjZ6OsAB7cKSbjwu04J3ibVN78C+dEWJAAHqoH8xNdd1NL84VdPH5Ncx2ynlcmQ+0ezRG/mwswQgPM8F0F80QQ6cX+JLjTStsliXeqcdgiTJzs5jKC1xj3YrXgh7 X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(36860700004)(82310400014); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 10:17:23.2508 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a6d4beeb-cf77-4422-af00-08dc38467481 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF00028D00.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR07MB8278 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Replace static array of cache-aligned structs with an lcore variable, to slightly benefit code simplicity and performance. RFC v5: * Fix lcore value pointer bug introduced by RFC v4. RFC v4: * Remove strange-looking lcore value lookup potentially containing invalid lcore id. (Morten Brørup) * Replace misplaced tab with space. (Morten Brørup) Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup --- lib/eal/common/rte_service.c | 118 ++++++++++++++++++++--------------- 1 file changed, 67 insertions(+), 51 deletions(-) diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c index d959c91459..5429ddce41 100644 --- a/lib/eal/common/rte_service.c +++ b/lib/eal/common/rte_service.c @@ -11,6 +11,7 @@ #include #include +#include #include #include #include @@ -75,7 +76,7 @@ struct core_state { static uint32_t rte_service_count; static struct rte_service_spec_impl *rte_services; -static struct core_state *lcore_states; +static RTE_LCORE_VAR_HANDLE(struct core_state, lcore_states); static uint32_t rte_service_library_initialized; int32_t @@ -101,11 +102,12 @@ rte_service_init(void) goto fail_mem; } - lcore_states = rte_calloc("rte_service_core_states", RTE_MAX_LCORE, - sizeof(struct core_state), RTE_CACHE_LINE_SIZE); - if (!lcore_states) { - EAL_LOG(ERR, "error allocating core states array"); - goto fail_mem; + if (lcore_states == NULL) + RTE_LCORE_VAR_ALLOC(lcore_states); + else { + struct core_state *cs; + RTE_LCORE_VAR_FOREACH_VALUE(cs, lcore_states) + memset(cs, 0, sizeof(struct core_state)); } int i; @@ -122,7 +124,6 @@ rte_service_init(void) return 0; fail_mem: rte_free(rte_services); - rte_free(lcore_states); return -ENOMEM; } @@ -136,7 +137,6 @@ rte_service_finalize(void) rte_eal_mp_wait_lcore(); rte_free(rte_services); - rte_free(lcore_states); rte_service_library_initialized = 0; } @@ -286,7 +286,6 @@ rte_service_component_register(const struct rte_service_spec *spec, int32_t rte_service_component_unregister(uint32_t id) { - uint32_t i; struct rte_service_spec_impl *s; SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); @@ -294,9 +293,10 @@ rte_service_component_unregister(uint32_t id) s->internal_flags &= ~(SERVICE_F_REGISTERED); + struct core_state *cs; /* clear the run-bit in all cores */ - for (i = 0; i < RTE_MAX_LCORE; i++) - lcore_states[i].service_mask &= ~(UINT64_C(1) << id); + RTE_LCORE_VAR_FOREACH_VALUE(cs, lcore_states) + cs->service_mask &= ~(UINT64_C(1) << id); memset(&rte_services[id], 0, sizeof(struct rte_service_spec_impl)); @@ -454,7 +454,10 @@ rte_service_may_be_active(uint32_t id) return -EINVAL; for (i = 0; i < lcore_count; i++) { - if (lcore_states[ids[i]].service_active_on_lcore[id]) + struct core_state *cs = + RTE_LCORE_VAR_LCORE_VALUE(ids[i], lcore_states); + + if (cs->service_active_on_lcore[id]) return 1; } @@ -464,7 +467,7 @@ rte_service_may_be_active(uint32_t id) int32_t rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe) { - struct core_state *cs = &lcore_states[rte_lcore_id()]; + struct core_state *cs = RTE_LCORE_VAR_VALUE(lcore_states); struct rte_service_spec_impl *s; SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); @@ -486,8 +489,7 @@ service_runner_func(void *arg) { RTE_SET_USED(arg); uint8_t i; - const int lcore = rte_lcore_id(); - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_VALUE(lcore_states); rte_atomic_store_explicit(&cs->thread_active, 1, rte_memory_order_seq_cst); @@ -533,13 +535,15 @@ service_runner_func(void *arg) int32_t rte_service_lcore_may_be_active(uint32_t lcore) { - if (lcore >= RTE_MAX_LCORE || !lcore_states[lcore].is_service_core) + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); + + if (lcore >= RTE_MAX_LCORE || !cs->is_service_core) return -EINVAL; /* Load thread_active using ACQUIRE to avoid instructions dependent on * the result being re-ordered before this load completes. */ - return rte_atomic_load_explicit(&lcore_states[lcore].thread_active, + return rte_atomic_load_explicit(&cs->thread_active, rte_memory_order_acquire); } @@ -547,9 +551,11 @@ int32_t rte_service_lcore_count(void) { int32_t count = 0; - uint32_t i; - for (i = 0; i < RTE_MAX_LCORE; i++) - count += lcore_states[i].is_service_core; + + struct core_state *cs; + RTE_LCORE_VAR_FOREACH_VALUE(cs, lcore_states) + count += cs->is_service_core; + return count; } @@ -566,7 +572,8 @@ rte_service_lcore_list(uint32_t array[], uint32_t n) uint32_t i; uint32_t idx = 0; for (i = 0; i < RTE_MAX_LCORE; i++) { - struct core_state *cs = &lcore_states[i]; + struct core_state *cs = + RTE_LCORE_VAR_LCORE_VALUE(i, lcore_states); if (cs->is_service_core) { array[idx] = i; idx++; @@ -582,7 +589,7 @@ rte_service_lcore_count_services(uint32_t lcore) if (lcore >= RTE_MAX_LCORE) return -EINVAL; - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); if (!cs->is_service_core) return -ENOTSUP; @@ -634,30 +641,31 @@ rte_service_start_with_defaults(void) static int32_t service_update(uint32_t sid, uint32_t lcore, uint32_t *set, uint32_t *enabled) { + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); + /* validate ID, or return error value */ if (!service_valid(sid) || lcore >= RTE_MAX_LCORE || - !lcore_states[lcore].is_service_core) + !cs->is_service_core) return -EINVAL; uint64_t sid_mask = UINT64_C(1) << sid; if (set) { - uint64_t lcore_mapped = lcore_states[lcore].service_mask & - sid_mask; + uint64_t lcore_mapped = cs->service_mask & sid_mask; if (*set && !lcore_mapped) { - lcore_states[lcore].service_mask |= sid_mask; + cs->service_mask |= sid_mask; rte_atomic_fetch_add_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } if (!*set && lcore_mapped) { - lcore_states[lcore].service_mask &= ~(sid_mask); + cs->service_mask &= ~(sid_mask); rte_atomic_fetch_sub_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } } if (enabled) - *enabled = !!(lcore_states[lcore].service_mask & (sid_mask)); + *enabled = !!(cs->service_mask & (sid_mask)); return 0; } @@ -685,13 +693,14 @@ set_lcore_state(uint32_t lcore, int32_t state) { /* mark core state in hugepage backed config */ struct rte_config *cfg = rte_eal_get_configuration(); + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); cfg->lcore_role[lcore] = state; /* mark state in process local lcore_config */ lcore_config[lcore].core_role = state; /* update per-lcore optimized state tracking */ - lcore_states[lcore].is_service_core = (state == ROLE_SERVICE); + cs->is_service_core = (state == ROLE_SERVICE); rte_eal_trace_service_lcore_state_change(lcore, state); } @@ -702,14 +711,16 @@ rte_service_lcore_reset_all(void) /* loop over cores, reset all to mask 0 */ uint32_t i; for (i = 0; i < RTE_MAX_LCORE; i++) { - if (lcore_states[i].is_service_core) { - lcore_states[i].service_mask = 0; + struct core_state *cs = + RTE_LCORE_VAR_LCORE_VALUE(i, lcore_states); + if (cs->is_service_core) { + cs->service_mask = 0; set_lcore_state(i, ROLE_RTE); /* runstate act as guard variable Use * store-release memory order here to synchronize * with load-acquire in runstate read functions. */ - rte_atomic_store_explicit(&lcore_states[i].runstate, + rte_atomic_store_explicit(&cs->runstate, RUNSTATE_STOPPED, rte_memory_order_release); } } @@ -725,17 +736,19 @@ rte_service_lcore_add(uint32_t lcore) { if (lcore >= RTE_MAX_LCORE) return -EINVAL; - if (lcore_states[lcore].is_service_core) + + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); + if (cs->is_service_core) return -EALREADY; set_lcore_state(lcore, ROLE_SERVICE); /* ensure that after adding a core the mask and state are defaults */ - lcore_states[lcore].service_mask = 0; + cs->service_mask = 0; /* Use store-release memory order here to synchronize with * load-acquire in runstate read functions. */ - rte_atomic_store_explicit(&lcore_states[lcore].runstate, RUNSTATE_STOPPED, + rte_atomic_store_explicit(&cs->runstate, RUNSTATE_STOPPED, rte_memory_order_release); return rte_eal_wait_lcore(lcore); @@ -747,7 +760,7 @@ rte_service_lcore_del(uint32_t lcore) if (lcore >= RTE_MAX_LCORE) return -EINVAL; - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); if (!cs->is_service_core) return -EINVAL; @@ -771,7 +784,7 @@ rte_service_lcore_start(uint32_t lcore) if (lcore >= RTE_MAX_LCORE) return -EINVAL; - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); if (!cs->is_service_core) return -EINVAL; @@ -801,6 +814,8 @@ rte_service_lcore_start(uint32_t lcore) int32_t rte_service_lcore_stop(uint32_t lcore) { + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); + if (lcore >= RTE_MAX_LCORE) return -EINVAL; @@ -808,12 +823,11 @@ rte_service_lcore_stop(uint32_t lcore) * memory order here to synchronize with store-release * in runstate update functions. */ - if (rte_atomic_load_explicit(&lcore_states[lcore].runstate, rte_memory_order_acquire) == + if (rte_atomic_load_explicit(&cs->runstate, rte_memory_order_acquire) == RUNSTATE_STOPPED) return -EALREADY; uint32_t i; - struct core_state *cs = &lcore_states[lcore]; uint64_t service_mask = cs->service_mask; for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) { @@ -834,7 +848,7 @@ rte_service_lcore_stop(uint32_t lcore) /* Use store-release memory order here to synchronize with * load-acquire in runstate read functions. */ - rte_atomic_store_explicit(&lcore_states[lcore].runstate, RUNSTATE_STOPPED, + rte_atomic_store_explicit(&cs->runstate, RUNSTATE_STOPPED, rte_memory_order_release); rte_eal_trace_service_lcore_stop(lcore); @@ -845,7 +859,7 @@ rte_service_lcore_stop(uint32_t lcore) static uint64_t lcore_attr_get_loops(unsigned int lcore) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); return rte_atomic_load_explicit(&cs->loops, rte_memory_order_relaxed); } @@ -853,7 +867,7 @@ lcore_attr_get_loops(unsigned int lcore) static uint64_t lcore_attr_get_cycles(unsigned int lcore) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); return rte_atomic_load_explicit(&cs->cycles, rte_memory_order_relaxed); } @@ -861,7 +875,7 @@ lcore_attr_get_cycles(unsigned int lcore) static uint64_t lcore_attr_get_service_calls(uint32_t service_id, unsigned int lcore) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); return rte_atomic_load_explicit(&cs->service_stats[service_id].calls, rte_memory_order_relaxed); @@ -870,7 +884,7 @@ lcore_attr_get_service_calls(uint32_t service_id, unsigned int lcore) static uint64_t lcore_attr_get_service_cycles(uint32_t service_id, unsigned int lcore) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); return rte_atomic_load_explicit(&cs->service_stats[service_id].cycles, rte_memory_order_relaxed); @@ -886,7 +900,10 @@ attr_get(uint32_t id, lcore_attr_get_fun lcore_attr_get) uint64_t sum = 0; for (lcore = 0; lcore < RTE_MAX_LCORE; lcore++) { - if (lcore_states[lcore].is_service_core) + struct core_state *cs = + RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); + + if (cs->is_service_core) sum += lcore_attr_get(id, lcore); } @@ -930,12 +947,11 @@ int32_t rte_service_lcore_attr_get(uint32_t lcore, uint32_t attr_id, uint64_t *attr_value) { - struct core_state *cs; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); if (lcore >= RTE_MAX_LCORE || !attr_value) return -EINVAL; - cs = &lcore_states[lcore]; if (!cs->is_service_core) return -ENOTSUP; @@ -960,7 +976,8 @@ rte_service_attr_reset_all(uint32_t id) return -EINVAL; for (lcore = 0; lcore < RTE_MAX_LCORE; lcore++) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = + RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); cs->service_stats[id] = (struct service_stats) {}; } @@ -971,12 +988,11 @@ rte_service_attr_reset_all(uint32_t id) int32_t rte_service_lcore_attr_reset_all(uint32_t lcore) { - struct core_state *cs; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); if (lcore >= RTE_MAX_LCORE) return -EINVAL; - cs = &lcore_states[lcore]; if (!cs->is_service_core) return -ENOTSUP; @@ -1011,7 +1027,7 @@ static void service_dump_calls_per_lcore(FILE *f, uint32_t lcore) { uint32_t i; - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE_VALUE(lcore, lcore_states); fprintf(f, "%02d\t", lcore); for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) { From patchwork Wed Feb 28 10:09:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 137419 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6B87043C1F; Wed, 28 Feb 2024 11:17:59 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 84BA942D7B; Wed, 28 Feb 2024 11:17:39 +0100 (CET) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04on2085.outbound.protection.outlook.com [40.107.8.85]) by mails.dpdk.org (Postfix) with ESMTP id 59B6340DDB for ; Wed, 28 Feb 2024 11:17:25 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=i/Zw9oycLmYtpe/VHYtep8+wB03v+U1OEwBqTsNs8/Sqneln8JeaZBvS7Ejo9VxhsGFgdiD3h5MUZaJNEesL3n43APcr4HVPE1x0fagloga4oV+85xWjjBjxrvkWiRQLgH9Ho+Nucajrmd8rBA+Q0ZkNtMW7ao3880Ej6M7pFmCiDz278wyr/+epZOsALpyuLsXDJJFF9nhXzP5qTYj/CGUjyy0oanGufApcMYYh5wQF0YnQ8GdcKTEq7SHmEp9u2B2V+rJZ2T2TZYPZbunoka/jgSDfJldG6+XTqNR0ehi5nvIpz1kRNgXE1RcP1+UXJJ3VhTZ1arjoKmcjPPJGKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S/nn+/osSbRePXDyt2OXgfcTtQ3j8AB1+LstbQNCjHY=; b=aIVLcgEIdfgsznnyrdZeMATPw4y22jlLrvAjK+5V2khjKJI/f+my77Iiu7c/NsBwn2rJP6QJWxA+6Y5aQNfzghAeMyodTfADqbaPlAKz9wmkr5y2raGFvjd96A569LVQARhx9WgpxOIU2I6vZWzfKptD635J8rcnsKwgGBPh0fdci5oj9aYD8fwRlna56/cUoaYuqHXN6RziSpZBBqiqUBBIcG7NDL5x2PYNZNZmauOuay4eIj56LFltFx/riLx0mexAGuLmnYMvTNy6f20ypeXoAHy3LTi/Q+GYmfX/BnqpaIOhLwaV5yR8MTYHhVcF0DN3wcf5mOuMtw/jpS97rQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S/nn+/osSbRePXDyt2OXgfcTtQ3j8AB1+LstbQNCjHY=; b=QSnj+u13FULOpRtKZ11GPGrc4bqVw+Rl9A1g9rz5yavxBS2DAAZZhudRDamMP8U6UXpklBYLd66kxYLCcWI9hXhZhY4Xqpdhi2u5W6ocJJ2p2kXYK/uerYlGu6oQS/yJAqJk0IGKopW7n/zC9TMi8B8d7shIf3c4/HA0yLlqgeBB9XAciyL78sNEXi4znsPPuXzQRHGBMjIKTgWDbUDZrFsbZlv0XDPs6cVxnem+wiXuJjEXbSKxci0lX+1iBfJLW0o7zg1IiSQfzP0tDkzbXi2ZK5u0Z52GG6QWtl8NSi2cbTNuZAnVVml8V4ZP31ziPqg+diyeh/7BmWXAdlUP8Q== Received: from DB6PR0301CA0084.eurprd03.prod.outlook.com (2603:10a6:6:30::31) by AS8PR07MB7301.eurprd07.prod.outlook.com (2603:10a6:20b:23b::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.41; Wed, 28 Feb 2024 10:17:23 +0000 Received: from DU6PEPF0000B61F.eurprd02.prod.outlook.com (2603:10a6:6:30:cafe::10) by DB6PR0301CA0084.outlook.office365.com (2603:10a6:6:30::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.27 via Frontend Transport; Wed, 28 Feb 2024 10:17:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU6PEPF0000B61F.mail.protection.outlook.com (10.167.8.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 10:17:23 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.68) with Microsoft SMTP Server id 15.2.1258.12; Wed, 28 Feb 2024 11:17:23 +0100 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 1A220380061; Wed, 28 Feb 2024 11:17:23 +0100 (CET) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , =?utf-8?q?Morten_Br=C3=B8rup?= , Stephen Hemminger , =?utf-8?q?Mattias_R=C3=B6nn?= =?utf-8?q?blom?= Subject: [RFC v5 6/6] eal: keep per-lcore power intrinsics state in lcore variable Date: Wed, 28 Feb 2024 11:09:28 +0100 Message-ID: <20240228100928.524277-7-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240228100928.524277-1-mattias.ronnblom@ericsson.com> References: <20240225150330.517225-2-mattias.ronnblom@ericsson.com> <20240228100928.524277-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU6PEPF0000B61F:EE_|AS8PR07MB7301:EE_ X-MS-Office365-Filtering-Correlation-Id: daca5272-a3f4-4607-5d16-08dc384674ca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GxklDXeZYGawrSnDY7TKUToc0PefaNiqUi5fSJ/7NeFbpWtsQTUyAF2QR7a2Jz31UH0De22iYgi5UnnNBl4tyNYSUOq28ua/1s231PXQ+KCvZzDfUdS+e2C2s9dT26KTyga/BwkinvPqYvXCVs4KmrB/oy1z5lI28MgzpTQnBsB0ShrNzZPkVDVKjpxacyqpjpHW/8vY0hxT5n6m8tvapIBl1eij9A9F5B9zGcnz6vO5O4NAidgZ2FL1Uyy+KE+GFndEKACXNO486T55dMqu75ASPlQd6+JuXwz0oyslbs+WQPxiB9+wDndpauKsFL2+SSz7VSPnRIkECb7lEPqiw/gnI7BV6wzOJf6lILiOItGUsNjQ1G2Kr+E4FYVinInclHNTxcCYXegPzChZ5cHHmtqZx2Tyajl61HhGrmhvw3KbzbQw1YkvXQYDwpx6Ct9Y3ek3b5JHPq2dZ8f5ycQnqI/Jfzwk8UWxZnARTA+uARKVHkFNTcYx/9n8o1smXPFkT/odN6gLqfIVufjdI9MjCePISpVbRhaz4yevQr8Kto2lPr5Fw0NjCmYTGXU+d79lia3cgcae0HtfhNm5sJvd9t05jMgz67DJkaCKKEYI0qfAh9AV+rBGtOfzfa9oyM7OeptWVUxinetoheRAh6IlUrxiuMKeejuy8Cgv0c4B6ptJnGNKHJ1xDLE+Q6kN/c/7z0A1HTko5Olw54FqglMzOrDh6XJcEBFpkqQOl78hTVrjdhJM1AjQiG0ci92ME47b X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(82310400014)(36860700004); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 10:17:23.7115 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: daca5272-a3f4-4607-5d16-08dc384674ca X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF0000B61F.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR07MB7301 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Keep per-lcore power intrinsics state in a lcore variable to reduce cache working set size and avoid any CPU next-line-prefetching causing false sharing. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup --- lib/eal/x86/rte_power_intrinsics.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/lib/eal/x86/rte_power_intrinsics.c b/lib/eal/x86/rte_power_intrinsics.c index 532a2e646b..23d1761f0a 100644 --- a/lib/eal/x86/rte_power_intrinsics.c +++ b/lib/eal/x86/rte_power_intrinsics.c @@ -4,6 +4,7 @@ #include #include +#include #include #include @@ -12,10 +13,14 @@ /* * Per-lcore structure holding current status of C0.2 sleeps. */ -static struct power_wait_status { +struct power_wait_status { rte_spinlock_t lock; volatile void *monitor_addr; /**< NULL if not currently sleeping */ -} __rte_cache_aligned wait_status[RTE_MAX_LCORE]; +}; + +RTE_LCORE_VAR_HANDLE(struct power_wait_status, wait_status); + +RTE_LCORE_VAR_INIT(wait_status); /* * This function uses UMONITOR/UMWAIT instructions and will enter C0.2 state. @@ -170,7 +175,7 @@ rte_power_monitor(const struct rte_power_monitor_cond *pmc, if (pmc->fn == NULL) return -EINVAL; - s = &wait_status[lcore_id]; + s = RTE_LCORE_VAR_LCORE_VALUE(lcore_id, wait_status); /* update sleep address */ rte_spinlock_lock(&s->lock); @@ -262,7 +267,7 @@ rte_power_monitor_wakeup(const unsigned int lcore_id) if (lcore_id >= RTE_MAX_LCORE) return -EINVAL; - s = &wait_status[lcore_id]; + s = RTE_LCORE_VAR_LCORE_VALUE(lcore_id, wait_status); /* * There is a race condition between sleep, wakeup and locking, but we @@ -301,8 +306,8 @@ int rte_power_monitor_multi(const struct rte_power_monitor_cond pmc[], const uint32_t num, const uint64_t tsc_timestamp) { - const unsigned int lcore_id = rte_lcore_id(); - struct power_wait_status *s = &wait_status[lcore_id]; + struct power_wait_status *s = RTE_LCORE_VAR_VALUE(wait_status); + uint32_t i, rc; /* check if supported */