From patchwork Sun May 5 07:33:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 139866 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2BD9843FAC; Sun, 5 May 2024 09:44:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 371824067E; Sun, 5 May 2024 09:43:45 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2064.outbound.protection.outlook.com [40.107.20.64]) by mails.dpdk.org (Postfix) with ESMTP id CC132402E1 for ; Sun, 5 May 2024 09:43:37 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SRpqcUrcdlASfEUy6XWrTjsaplVX2VGImWIKVsyhcDO1OvFDTdSv+Wki5/POb7GawBhlThvWbnKBQ8t0OB9OwB/9DHmDfPsDi0YJ/hW9D91L0WXrGryqR9Bl+30xQH3TifbiCbMF+/f6TmJklfP7enWEks4YMyVLiv9EWYj/RCx1D65cqMtb8nYzMTEvscd13QICUfdhcxlDbG5zoVXDq9AB9YSUdA/EqH9cJ8yfnH+qrF4/7A25tnmw5iOrhsyXx/waHq0vIOWNE9tolPHValNSvptxzKuy5/h7Z04SaXPI3l6VvbjB2heSG8yHm8IstyG/R27+bDYcrDeZQ2OOJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Mx2k72mHtMfvaE5BD+qrppgugwcEHxL9T3OLZibFkgE=; b=EI0LpDyEybiPrmEhYAUj2Wrms2ABOIC6CgCpTV+v+8bjd7OeffnjbwCM+ZZjCzhB/ovdBXXgyh9bbJX6hLB0LMgmG33Ewy8EnBLIn/gw4dz2XWNBeWU27iKEeWdUECTyvbZeF/RsUMrCq3tZwjG13AJxWu7sP3M9oj95O4S6ZVwux2GE4YRtd7b/DPaT5nOIIj9t02vbdoJzb2mSn86A0kiCSgBOFuN+xlej3EwDI6GLQ3vVSOmBKF+VqcIiJAKd/2yvtRl9w7eRjomF+6m/DI/aYDHpU1wu+eMZVz0WynwyAeTmv1rPdQiyzzwvk2SnL6nkLMBsZP57f5ngvnuyIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mx2k72mHtMfvaE5BD+qrppgugwcEHxL9T3OLZibFkgE=; b=qaU7zNhxSRBSAOdXKQ1KrJst9E3O4mPwlQptaj4fsbvnM3AwUzvXJZu0CrCiAlyW9gqIWCszmVvn+2hi7jssQ1Njr9zvzsZb7hOwVQnUu6TC0dgirkWC/ONv95UuXmu4Kk3L7FmtoOKUwfx+C4Z1q/XXbgzUDASBllIG4zmw2JwOyNyL0WV9q+EP632krfSZ+CfW0kzWwaK0Zw7fJAi8e1EMtwOjJIGczjVKy6yjgetLNN88e9OACAoGRykZI7gjpUiRCi25zudycGx9JNN0bKt75ud0xh+nZ/VVrvAmB+ziy84ht0Lu1nRMF310uJiMjHXDZgvZJxkOnBxByXeNFg== Received: from DUZPR01CA0337.eurprd01.prod.exchangelabs.com (2603:10a6:10:4b8::10) by AM9PR07MB7812.eurprd07.prod.outlook.com (2603:10a6:20b:2fe::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39; Sun, 5 May 2024 07:43:35 +0000 Received: from DB1PEPF000509E7.eurprd03.prod.outlook.com (2603:10a6:10:4b8:cafe::af) by DUZPR01CA0337.outlook.office365.com (2603:10a6:10:4b8::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39 via Frontend Transport; Sun, 5 May 2024 07:43:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF000509E7.mail.protection.outlook.com (10.167.242.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.19 via Frontend Transport; Sun, 5 May 2024 07:43:35 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.69) with Microsoft SMTP Server id 15.2.1544.9; Sun, 5 May 2024 09:43:34 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 884D038007A; Sun, 5 May 2024 09:43:34 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , =?utf-8?q?Morten_Br=C3=B8rup?= , Tyler Retzlaff , Stephen Hemminger , Harry van Haaren , =?utf-8?q?Mattias_R=C3=B6nnb?= =?utf-8?q?lom?= Subject: [RFC v5 5/6] service: use multi-word bitset to represent service flags Date: Sun, 5 May 2024 09:33:12 +0200 Message-ID: <20240505073313.118515-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240505073313.118515-1-mattias.ronnblom@ericsson.com> References: <20240216102348.480407-1-mattias.ronnblom@ericsson.com> <20240505073313.118515-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509E7:EE_|AM9PR07MB7812:EE_ X-MS-Office365-Filtering-Correlation-Id: 3c325b78-45fd-41cb-61d5-08dc6cd711cd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230031|1800799015|376005|36860700004|82310400017; X-Microsoft-Antispam-Message-Info: =?utf-8?q?+6yyeeD/uH+2sYN8Y1/TP4RiNPzkpn1?= =?utf-8?q?5SFpM0XeHkjmKXsGgt52uS7/y/EUWbGEwyPwhcQaHEQKoAVcfr6kb3nEjP8lEWQdQ?= =?utf-8?q?h6EHT1tWFvCs1QRkk8WmKfp0CtD9dWCXAmiXdqdH17GSGjzFniiIQgPgJ5Cqz3ucS?= =?utf-8?q?uaf5w5JcjX4g9Jqj0B5XICwAZGlY2KBlKfa527mHgzCCOjzcz01IMQ9tCNcF+v1ux?= =?utf-8?q?y2IDPjaHVUl/XEi6MlvevgPDknuT2IIxALAbQVkUgw7hbUIO/YcdwcCP9bSOF1rNO?= =?utf-8?q?K+Cv9U6C8qzmU1EqcDKSQjrJoRXe9hD3Rt9+lD0rm9LYYT6E4a5Y4W6tZSqGum3lT?= =?utf-8?q?iGB0fe8Pa2/NxrgAdacKGQ19T3LYD+Dz7lp6MR9B5L7Toh1YPzVTsiCNbont637/L?= =?utf-8?q?DMcr1T19EOd8RkWIi8X7vMHGXU5IGxrx/jt7Oe+d551omEPu93pQEMBjQ2t+u16MS?= =?utf-8?q?ZeKim3y4xEt9nt21zs/RUlDWXJ/D1/5Clk/bpdrXgEvRKWH3rwBVH+vH2JwLPTso0?= =?utf-8?q?ZKH6q4rHDIsBErC/0nLvwnCZjJBd5Kic4bsbuNVVmeiV3aSuPf1DwjGrxxyjIJMKt?= =?utf-8?q?4kT6A4Cf5rj64tTjqICbv05BY4c5wdQ6ulrE7egdRrIUpO2YSx5Xv05o+HCGUZCYG?= =?utf-8?q?sdqM55dPzDjha+knoBtWYsF9ClkUJeG6uB/ashi4HMn9VS4Z3GqbhkxN3l9M6bQ6p?= =?utf-8?q?LGkzgvXPrZa/J7EugmvwQSKvvNNaLAjh13UfLpU2WKAB4thK1XSghM7VFAyAti/+I?= =?utf-8?q?CnyMBvEzBH2AuDysXSno+Qg+7rhU2DggVIpHxzCU0jJQOyK0jOEnbQIbrBeWUVqwT?= =?utf-8?q?cFTIMST+A4Fcefrhkonv5piPwO/7Vf2HSCgn/CSIBXSwR8+WSoWYzW/lts9r8c+Pb?= =?utf-8?q?eD7IumMiPw59SJ8c0jGExr9C7CQHw+Su6nrnQiVEQOD28RCZZTC9sAPEDvq76785b?= =?utf-8?q?bgkoIL7tiZn//JKxscEB0OMCWFMXb+bv1qn7qYjD8OtgubX60dcygzkas0lFlamVO?= =?utf-8?q?lqvg2Iwzw7oqFvi1b+Y1FR+G309fSazMrXtcPhOZv7ki5vBCm/jCNs9AMBBccGHDK?= =?utf-8?q?zoYQHt4oWmdf466n8d4GHdqT0GEebNSC9jmk0iR8zytvBX7c3MEu9Fv6hwb7Xf10Y?= =?utf-8?q?51k60SeGMaK1RajEJrZ6H2YEulsehzEzH+bwV/22KDUwYB2f/LxPLyhygrp4dz9Do?= =?utf-8?q?Jde57SKHk5jM2UG1LoY3vde1koiSxD+7FC68jvezvKHd6v5JbPF6OhBq1PR4kyqx8?= =?utf-8?q?tDu6pD7bWwFkVcdg5Yrz1qqqa2akNc5jfKFr8rFTKbA9t2gKqjZpxeybf3KspRJfh?= =?utf-8?q?+13k0VZkpXAN?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2024 07:43:35.1427 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3c325b78-45fd-41cb-61d5-08dc6cd711cd X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509E7.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR07MB7812 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Use a multi-word bitset to track which services are mapped to which lcores, allowing the RTE_SERVICE_NUM_MAX compile-time constant to be > 64. Replace array-of-bytes service-currently-active flags with a more compact multi-word bitset-based representation, reducing memory footprint somewhat. Signed-off-by: Mattias Rönnblom --- lib/eal/common/rte_service.c | 70 ++++++++++++++---------------------- 1 file changed, 27 insertions(+), 43 deletions(-) diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c index 56379930b6..ec0f47e141 100644 --- a/lib/eal/common/rte_service.c +++ b/lib/eal/common/rte_service.c @@ -11,6 +11,7 @@ #include #include +#include #include #include #include @@ -63,11 +64,11 @@ struct service_stats { /* the internal values of a service core */ struct __rte_cache_aligned core_state { /* map of services IDs are run on this core */ - uint64_t service_mask; + RTE_BITSET_DECLARE(mapped_services, RTE_SERVICE_NUM_MAX); RTE_ATOMIC(uint8_t) runstate; /* running or stopped */ RTE_ATOMIC(uint8_t) thread_active; /* indicates when thread is in service_run() */ uint8_t is_service_core; /* set if core is currently a service core */ - uint8_t service_active_on_lcore[RTE_SERVICE_NUM_MAX]; + RTE_BITSET_DECLARE(service_active_on_lcore, RTE_SERVICE_NUM_MAX); RTE_ATOMIC(uint64_t) loops; RTE_ATOMIC(uint64_t) cycles; struct service_stats service_stats[RTE_SERVICE_NUM_MAX]; @@ -81,11 +82,6 @@ static uint32_t rte_service_library_initialized; int32_t rte_service_init(void) { - /* Hard limit due to the use of an uint64_t-based bitmask (and the - * clzl intrinsic). - */ - RTE_BUILD_BUG_ON(RTE_SERVICE_NUM_MAX > 64); - if (rte_service_library_initialized) { EAL_LOG(NOTICE, "service library init() called, init flag %d", @@ -296,7 +292,7 @@ rte_service_component_unregister(uint32_t id) /* clear the run-bit in all cores */ for (i = 0; i < RTE_MAX_LCORE; i++) - lcore_states[i].service_mask &= ~(UINT64_C(1) << id); + rte_bitset_clear(lcore_states[i].mapped_services, id); memset(&rte_services[id], 0, sizeof(struct rte_service_spec_impl)); @@ -410,7 +406,7 @@ service_runner_do_callback(struct rte_service_spec_impl *s, /* Expects the service 's' is valid. */ static int32_t -service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, +service_run(uint32_t i, struct core_state *cs, const uint64_t *mapped_services, struct rte_service_spec_impl *s, uint32_t serialize_mt_unsafe) { if (!s) @@ -424,12 +420,12 @@ service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, RUNSTATE_RUNNING || rte_atomic_load_explicit(&s->app_runstate, rte_memory_order_acquire) != RUNSTATE_RUNNING || - !(service_mask & (UINT64_C(1) << i))) { - cs->service_active_on_lcore[i] = 0; + !rte_bitset_test(mapped_services, i)) { + rte_bitset_clear(cs->service_active_on_lcore, i); return -ENOEXEC; } - cs->service_active_on_lcore[i] = 1; + rte_bitset_set(cs->service_active_on_lcore, i); if ((service_mt_safe(s) == 0) && (serialize_mt_unsafe == 1)) { if (!rte_spinlock_trylock(&s->execute_lock)) @@ -454,7 +450,7 @@ rte_service_may_be_active(uint32_t id) return -EINVAL; for (i = 0; i < lcore_count; i++) { - if (lcore_states[ids[i]].service_active_on_lcore[id]) + if (rte_bitset_test(lcore_states[ids[i]].service_active_on_lcore, id)) return 1; } @@ -474,7 +470,9 @@ rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe) */ rte_atomic_fetch_add_explicit(&s->num_mapped_cores, 1, rte_memory_order_relaxed); - int ret = service_run(id, cs, UINT64_MAX, s, serialize_mt_unsafe); + RTE_BITSET_DECLARE(all_services, RTE_SERVICE_NUM_MAX); + rte_bitset_set_all(all_services, RTE_SERVICE_NUM_MAX); + int ret = service_run(id, cs, all_services, s, serialize_mt_unsafe); rte_atomic_fetch_sub_explicit(&s->num_mapped_cores, 1, rte_memory_order_relaxed); @@ -485,7 +483,6 @@ static int32_t service_runner_func(void *arg) { RTE_SET_USED(arg); - uint8_t i; const int lcore = rte_lcore_id(); struct core_state *cs = &lcore_states[lcore]; @@ -497,20 +494,11 @@ service_runner_func(void *arg) */ while (rte_atomic_load_explicit(&cs->runstate, rte_memory_order_acquire) == RUNSTATE_RUNNING) { + ssize_t id; - const uint64_t service_mask = cs->service_mask; - uint8_t start_id; - uint8_t end_id; - - if (service_mask == 0) - continue; - - start_id = rte_ctz64(service_mask); - end_id = 64 - rte_clz64(service_mask); - - for (i = start_id; i < end_id; i++) { + RTE_BITSET_FOREACH_SET(id, cs->mapped_services, RTE_SERVICE_NUM_MAX) { /* return value ignored as no change to code flow */ - service_run(i, cs, service_mask, service_get(i), 1); + service_run(id, cs, cs->mapped_services, service_get(id), 1); } rte_atomic_store_explicit(&cs->loops, cs->loops + 1, rte_memory_order_relaxed); @@ -519,8 +507,7 @@ service_runner_func(void *arg) /* Switch off this core for all services, to ensure that future * calls to may_be_active() know this core is switched off. */ - for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) - cs->service_active_on_lcore[i] = 0; + rte_bitset_clear_all(cs->service_active_on_lcore, RTE_SERVICE_NUM_MAX); /* Use SEQ CST memory ordering to avoid any re-ordering around * this store, ensuring that once this store is visible, the service @@ -586,7 +573,7 @@ rte_service_lcore_count_services(uint32_t lcore) if (!cs->is_service_core) return -ENOTSUP; - return rte_popcount64(cs->service_mask); + return rte_bitset_count_set(cs->mapped_services, RTE_SERVICE_NUM_MAX); } int32_t @@ -639,25 +626,23 @@ service_update(uint32_t sid, uint32_t lcore, uint32_t *set, uint32_t *enabled) !lcore_states[lcore].is_service_core) return -EINVAL; - uint64_t sid_mask = UINT64_C(1) << sid; if (set) { - uint64_t lcore_mapped = lcore_states[lcore].service_mask & - sid_mask; + uint64_t lcore_mapped = rte_bitset_test(lcore_states[lcore].mapped_services, sid); if (*set && !lcore_mapped) { - lcore_states[lcore].service_mask |= sid_mask; + rte_bitset_set(lcore_states[lcore].mapped_services, sid); rte_atomic_fetch_add_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } if (!*set && lcore_mapped) { - lcore_states[lcore].service_mask &= ~(sid_mask); + rte_bitset_clear(lcore_states[lcore].mapped_services, sid); rte_atomic_fetch_sub_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } } if (enabled) - *enabled = !!(lcore_states[lcore].service_mask & (sid_mask)); + *enabled = rte_bitset_test(lcore_states[lcore].mapped_services, sid); return 0; } @@ -699,11 +684,11 @@ set_lcore_state(uint32_t lcore, int32_t state) int32_t rte_service_lcore_reset_all(void) { - /* loop over cores, reset all to mask 0 */ + /* loop over cores, reset all mapped services */ uint32_t i; for (i = 0; i < RTE_MAX_LCORE; i++) { if (lcore_states[i].is_service_core) { - lcore_states[i].service_mask = 0; + rte_bitset_clear_all(lcore_states[i].mapped_services, RTE_SERVICE_NUM_MAX); set_lcore_state(i, ROLE_RTE); /* runstate act as guard variable Use * store-release memory order here to synchronize @@ -731,7 +716,7 @@ rte_service_lcore_add(uint32_t lcore) set_lcore_state(lcore, ROLE_SERVICE); /* ensure that after adding a core the mask and state are defaults */ - lcore_states[lcore].service_mask = 0; + rte_bitset_clear_all(lcore_states[lcore].mapped_services, RTE_SERVICE_NUM_MAX); /* Use store-release memory order here to synchronize with * load-acquire in runstate read functions. */ @@ -814,12 +799,11 @@ rte_service_lcore_stop(uint32_t lcore) uint32_t i; struct core_state *cs = &lcore_states[lcore]; - uint64_t service_mask = cs->service_mask; for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) { - int32_t enabled = service_mask & (UINT64_C(1) << i); - int32_t service_running = rte_service_runstate_get(i); - int32_t only_core = (1 == + bool enabled = rte_bitset_test(cs->mapped_services, i); + bool service_running = rte_service_runstate_get(i); + bool only_core = (1 == rte_atomic_load_explicit(&rte_services[i].num_mapped_cores, rte_memory_order_relaxed));