From patchwork Tue Jul 4 21:43:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 129261 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CDFCE42DAB; Tue, 4 Jul 2023 23:49:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 738A840A4B; Tue, 4 Jul 2023 23:49:55 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2076.outbound.protection.outlook.com [40.107.21.76]) by mails.dpdk.org (Postfix) with ESMTP id 5EA5A406B7; Tue, 4 Jul 2023 23:49:53 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EFCdHeznpV/dcSeaN7JGI547Ev74nHZ8EzyiMpbLCj1osO7ebp2IjQh1pSAUuMG/j+dPHR4ETVHIQEf7SfxTA2kJKU2PTsydXWheD0Jhp8ht6r2Uge83b6CSD6xjJkBZcixXqWCci0t9UMXgRLZQ/i+eBwZEATcfA+TrSgy7S7yJjtKQZqsK+5lwMk+Hhl6LkRChkalskz80wRe1Fh4xmVG19c362J07PikFrnUYb1JpWLV1QMWEVRo3Lch8gAhGw1j0mhSye4jkXZwJeJYJfExbs/UXUtXNjdEIbney9XsbQmePuKnjCHkPUsglHuNH/WqTj47v6FaEaiawP/H9FA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=niZXTGS6L0EyGe/DU5WtpHT4uUh1OdsQSzJRUvb+bos=; b=CR+vAAf0nU0zWFNB9E6Eihr+dWDkl/YxFK3HAUxGfIZSnVYpkfkdQ2rIQ1DAE0oHzx1DqMQe6s3wt/Et5HzCUR2KtkjxJB235j9V7ujHf27L7tcgpgFpdocO3zhKETb90OKeCZspXX+RE+BVtPTbfYyty7DqV2t3wtFoZMeLppPL0e8MtMp5b7W4yW7DDAuEgQUtScGAkEE8fTJ+8ueXTHo7gXdF52k2yx/l2908LCx9YNWkiErYlv9oeV8R6nCp8c2W6WqQiQX6+wrvaTnhZCLnkqfc3kPrtlCDIFww965g4q2FTpQDaxp8be7JnqXI+p3eiacuUR7MTrFyM5PVLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=niZXTGS6L0EyGe/DU5WtpHT4uUh1OdsQSzJRUvb+bos=; b=P8njRmg+dL0w6BGu9cuLbmYaw2gyCZ23Dg8sgDBXi4gjsoNf0Kgha3SmY/grXS05r0rwYCnXNIlrRrJ7JZwqRw/4C5ZuBYqepE4cNgdTqeFscEs32MNHpUJ8821C+XfnT2f/FTo/Z58uWbNBluCE+FxT/Kmbyn3xh2RhldBOO0M= Received: from AS9PR06CA0214.eurprd06.prod.outlook.com (2603:10a6:20b:45e::19) by PR3PR07MB8291.eurprd07.prod.outlook.com (2603:10a6:102:17e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.24; Tue, 4 Jul 2023 21:49:52 +0000 Received: from AM0EUR02FT054.eop-EUR02.prod.protection.outlook.com (2603:10a6:20b:45e:cafe::c6) by AS9PR06CA0214.outlook.office365.com (2603:10a6:20b:45e::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.18 via Frontend Transport; Tue, 4 Jul 2023 21:49:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM0EUR02FT054.mail.protection.outlook.com (10.13.55.248) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.6565.18 via Frontend Transport; Tue, 4 Jul 2023 21:49:51 +0000 Received: from ESESBMB505.ericsson.se (153.88.183.172) by ESESSMB501.ericsson.se (153.88.183.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.27; Tue, 4 Jul 2023 23:49:51 +0200 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.188) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Tue, 4 Jul 2023 23:49:51 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 6DF03380061; Tue, 4 Jul 2023 23:49:51 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: Harry van Haaren , Stephen Hemminger CC: , , Suanming Mou , , , =?utf-8?q?Mattias_R=C3=B6nnblom?= , Subject: [PATCH] service: avoid worker lcore exit deadlock Date: Tue, 4 Jul 2023 23:43:39 +0200 Message-ID: <20230704214339.244271-1-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM0EUR02FT054:EE_|PR3PR07MB8291:EE_ X-MS-Office365-Filtering-Correlation-Id: 18a36844-f9a2-45d9-cb42-08db7cd898b0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QxrNU7iFd9h0krHkRRCxMYDQ0FtiFZRJZVz6mjUgjpMCgWUbviseeytdX4wZCWggHAlNypuPJn1SD39x6p9gpuRGth5Q/Rngt66sUt6aifdq5wD+7wVYlPgQHoRc9lEzkI2dGeBQED6vLI4nDEoodqpmBZc0wsy/hoMQ6tPqpne3AgMmHCT/xlKjX0ahoiPTTlDQU8NShdBUWIjsNwGqf41izu0t4QMpW2WSH9BhLQBexnjb1UsU4fFhfPR3Jwp2pR/o2vHGKHLIqGqaf1Wk2wkA83S3J+HMclb0MKVKFuLRBbmqjMBKPoT4cCBRILkrkYUb7eWdhP5U4la0LEp/nuXaHMI0gGSVUF40wtbzrEMtHT1+hQgtbvqFcSF1oSZ92hHj/OfJVDNpq0lHRBHiWKxDcHrLveLO2sz/KdMw+0xGCs0czqlOklyLgJg2OCgkyx3QEd3okQ2yaU+aczOviCzb2usQNtXwrynjxEFT9nyzdbsXceJUa1pDb/69PeSzUrzjsCJik0yp8+mDWVqZWj+8Re9HRjO258yOOtShCzTlz4pygHHblLjuvNFMICkrEGtj7u/xSqEUFvSNIEUKI8iOB35VsnqMCFiQA4xdWGewNhcqyQf1+miS5wudXj0xXY8BFTSlwpNIQ/20TQRUIrSJx3qQqwzMuLVOz1i6cPFFyhChbMzbJZMecK1AlZbavpdzjyXAVCjxljeDSol6AVzOxdstjhFOzLwr0R5ErmEmZuo9JNbK5R34phXHEPf4mQgE6TiyKfC0BLQ7obq5ElIoBgZh84AbxDjcI12neEg= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230028)(4636009)(136003)(39860400002)(396003)(376002)(346002)(451199021)(40470700004)(36840700001)(46966006)(1076003)(40480700001)(82960400001)(82740400003)(356005)(7636003)(40460700003)(6266002)(316002)(4326008)(70586007)(70206006)(36860700001)(2616005)(47076005)(336012)(83380400001)(82310400005)(186003)(26005)(478600001)(110136005)(54906003)(2906002)(8936002)(8676002)(36756003)(5660300002)(86362001)(41300700001)(6666004); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jul 2023 21:49:51.8935 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 18a36844-f9a2-45d9-cb42-08db7cd898b0 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM0EUR02FT054.eop-EUR02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR07MB8291 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: 20230629203720.682f90c3@hermes.local Errors-To: dev-bounces@dpdk.org Calling rte_exit() from a worker lcore thread causes a deadlock in rte_service_finalize(). This patch makes rte_service_finalize() deadlock-free by avoiding the need to synchronize with service lcore threads, which in turn is achieved by moving service and per-lcore state from the heap to being statically allocated. The BSS segment increases with ~156 kB (on x86_64 with default RTE_MAX_LCORE and RTE_SERVICE_NUM_MAX). According to the service perf autotest, this change also results in a slight reduction of service framework overhead. Fixes: 33666b448f15 ("service: fix crash on exit") Cc: harry.van.haaren@intel.com Cc: stable@dpdk.org Signed-off-by: Mattias Rönnblom Acked-by: Harry van Haaren --- lib/eal/common/rte_service.c | 28 ++-------------------------- 1 file changed, 2 insertions(+), 26 deletions(-) diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c index 94e872a08a..2ea0c0c0d9 100644 --- a/lib/eal/common/rte_service.c +++ b/lib/eal/common/rte_service.c @@ -15,7 +15,6 @@ #include #include #include -#include #include #include @@ -74,8 +73,8 @@ struct core_state { } __rte_cache_aligned; static uint32_t rte_service_count; -static struct rte_service_spec_impl *rte_services; -static struct core_state *lcore_states; +static struct rte_service_spec_impl rte_services[RTE_SERVICE_NUM_MAX]; +static struct core_state lcore_states[RTE_MAX_LCORE]; static uint32_t rte_service_library_initialized; int32_t @@ -93,21 +92,6 @@ rte_service_init(void) return -EALREADY; } - rte_services = rte_calloc("rte_services", RTE_SERVICE_NUM_MAX, - sizeof(struct rte_service_spec_impl), - RTE_CACHE_LINE_SIZE); - if (!rte_services) { - RTE_LOG(ERR, EAL, "error allocating rte services array\n"); - goto fail_mem; - } - - lcore_states = rte_calloc("rte_service_core_states", RTE_MAX_LCORE, - sizeof(struct core_state), RTE_CACHE_LINE_SIZE); - if (!lcore_states) { - RTE_LOG(ERR, EAL, "error allocating core states array\n"); - goto fail_mem; - } - int i; struct rte_config *cfg = rte_eal_get_configuration(); for (i = 0; i < RTE_MAX_LCORE; i++) { @@ -120,10 +104,6 @@ rte_service_init(void) rte_service_library_initialized = 1; return 0; -fail_mem: - rte_free(rte_services); - rte_free(lcore_states); - return -ENOMEM; } void @@ -133,10 +113,6 @@ rte_service_finalize(void) return; rte_service_lcore_reset_all(); - rte_eal_mp_wait_lcore(); - - rte_free(rte_services); - rte_free(lcore_states); rte_service_library_initialized = 0; }