From patchwork Fri Jun 26 14:47:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 72262 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 44C56A051C; Fri, 26 Jun 2020 16:48:22 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EFA471C1AB; Fri, 26 Jun 2020 16:48:17 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id 5A37D1C196 for ; Fri, 26 Jun 2020 16:48:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593182893; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tgAkQXWyU6ppia54pOxgztU/wTH6cJJnwTDZt98OIII=; b=I+XukwNOTaN+e0/GfuUcEmY6YmkA7cEbohk/rEOyKWXsSQRhBGwQGJWahCuzRwKdFQv8Ua Dd/sdvYMJIfUQVo5BXYGA+X7nAimE96RHh1Thl4P74e1Im9ox1nfmyaOiptTceM9Yg2Mga a+gO8vSaA/3UOoHZjbRTXdJpa1aWIRU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-147-chDlx7MROTe4lIqrUnXdNw-1; Fri, 26 Jun 2020 10:48:09 -0400 X-MC-Unique: chDlx7MROTe4lIqrUnXdNw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D56AF804002; Fri, 26 Jun 2020 14:48:07 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4852B7FEBE; Fri, 26 Jun 2020 14:48:02 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, thomas@monjalon.net, arybchenko@solarflare.com, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Harini Ramakrishnan , Omar Cardona , Pallavi Kadam , Ranjit Menon Date: Fri, 26 Jun 2020 16:47:28 +0200 Message-Id: <20200626144736.11011-2-david.marchand@redhat.com> In-Reply-To: <20200626144736.11011-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200626144736.11011-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 1/9] eal: relocate per thread symbols to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" We have per lcore thread symbols scattered in OS implementations but common code relies on them. Move all of them in common. RTE_PER_LCORE(_socket_id) and RTE_PER_LCORE(_cpuset) have public accessors and are not exported through the library map, they can be made static. Signed-off-by: David Marchand Reviewed-by: Olivier Matz --- lib/librte_eal/common/eal_common_thread.c | 5 ++++- lib/librte_eal/freebsd/eal_thread.c | 4 ---- lib/librte_eal/include/rte_lcore.h | 1 - lib/librte_eal/linux/eal_thread.c | 4 ---- lib/librte_eal/windows/eal_thread.c | 4 ---- 5 files changed, 4 insertions(+), 14 deletions(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 370bb1b634..a5f67d811c 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -21,7 +21,10 @@ #include "eal_private.h" #include "eal_thread.h" -RTE_DECLARE_PER_LCORE(unsigned , _socket_id); +RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; +static RTE_DEFINE_PER_LCORE(unsigned int, _socket_id) = + (unsigned int)SOCKET_ID_ANY; +static RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); unsigned rte_socket_id(void) { diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index b52019782a..40676d9ef5 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -25,10 +25,6 @@ #include "eal_private.h" #include "eal_thread.h" -RTE_DEFINE_PER_LCORE(unsigned, _lcore_id) = LCORE_ID_ANY; -RTE_DEFINE_PER_LCORE(unsigned, _socket_id) = (unsigned)SOCKET_ID_ANY; -RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); - /* * Send a message to a slave lcore identified by slave_id to call a * function f with argument arg. Once the execution is done, the diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 339046bc86..5c1d1926e9 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -23,7 +23,6 @@ extern "C" { #define LCORE_ID_ANY UINT32_MAX /**< Any lcore. */ RTE_DECLARE_PER_LCORE(unsigned, _lcore_id); /**< Per thread "lcore id". */ -RTE_DECLARE_PER_LCORE(rte_cpuset_t, _cpuset); /**< Per thread "cpuset". */ /** * Get a lcore's role. diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index cd9d6e0ebf..a52ebef3a4 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -25,10 +25,6 @@ #include "eal_private.h" #include "eal_thread.h" -RTE_DEFINE_PER_LCORE(unsigned, _lcore_id) = LCORE_ID_ANY; -RTE_DEFINE_PER_LCORE(unsigned, _socket_id) = (unsigned)SOCKET_ID_ANY; -RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); - /* * Send a message to a slave lcore identified by slave_id to call a * function f with argument arg. Once the execution is done, the diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index 3dd56519c9..f12a2ec6ad 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -16,10 +16,6 @@ #include "eal_private.h" #include "eal_windows.h" -RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; -RTE_DEFINE_PER_LCORE(unsigned int, _socket_id) = (unsigned int)SOCKET_ID_ANY; -RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); - /* * Send a message to a slave lcore identified by slave_id to call a * function f with argument arg. Once the execution is done, the From patchwork Fri Jun 26 14:47:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 72263 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 57758A051C; Fri, 26 Jun 2020 16:48:30 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 33E101C0C0; Fri, 26 Jun 2020 16:48:23 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id EC0951C0BF for ; Fri, 26 Jun 2020 16:48:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593182901; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J8bcJr1ym6XYlcLwbnSKM4jOhL8PmhczSwCFWrkuVeU=; b=HEX/ChXT1Q1961SNYgP03sQwF2vf4dTnOhfy15QfmIfzBcri4A4IQM/w/r2wt1FwVjvHMb YeWWr0s1sJJYmS3HmlsFgNwaJ8xePHnz+6DTohE7yIz4pgw6cusXN3+Rzqt1AMYD3+Edim fyOaUNXrko8DbAiLYU+XMOv2CNrJMYI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-453-2IG9KW5vPfyQiPvsPdpZTg-1; Fri, 26 Jun 2020 10:48:17 -0400 X-MC-Unique: 2IG9KW5vPfyQiPvsPdpZTg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 61038A0C02; Fri, 26 Jun 2020 14:48:15 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 553179CFCC; Fri, 26 Jun 2020 14:48:09 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, thomas@monjalon.net, arybchenko@solarflare.com, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Neil Horman , Cunming Liang , Konstantin Ananyev , Olivier Matz Date: Fri, 26 Jun 2020 16:47:29 +0200 Message-Id: <20200626144736.11011-3-david.marchand@redhat.com> In-Reply-To: <20200626144736.11011-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200626144736.11011-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 2/9] eal: fix multiple definition of per lcore thread id X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Because of the inline accessor + static declaration in rte_gettid(), we end up with multiple symbols for RTE_PER_LCORE(_thread_id). Each compilation unit will pay a cost when accessing this information for the first time. $ nm build/app/dpdk-testpmd | grep per_lcore__thread_id 0000000000000054 d per_lcore__thread_id.5037 0000000000000040 d per_lcore__thread_id.5103 0000000000000048 d per_lcore__thread_id.5259 000000000000004c d per_lcore__thread_id.5259 0000000000000044 d per_lcore__thread_id.5933 0000000000000058 d per_lcore__thread_id.6261 0000000000000050 d per_lcore__thread_id.7378 000000000000005c d per_lcore__thread_id.7496 000000000000000c d per_lcore__thread_id.8016 0000000000000010 d per_lcore__thread_id.8431 Make it global as part of the DPDK_21 stable ABI. Fixes: ef76436c6834 ("eal: get unique thread id") Signed-off-by: David Marchand Acked-by: Ray Kinsella Reviewed-by: Olivier Matz --- lib/librte_eal/common/eal_common_thread.c | 1 + lib/librte_eal/include/rte_eal.h | 3 ++- lib/librte_eal/rte_eal_version.map | 7 +++++++ 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index a5f67d811c..280c64bb76 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -22,6 +22,7 @@ #include "eal_thread.h" RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; +RTE_DEFINE_PER_LCORE(int, _thread_id) = -1; static RTE_DEFINE_PER_LCORE(unsigned int, _socket_id) = (unsigned int)SOCKET_ID_ANY; static RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); diff --git a/lib/librte_eal/include/rte_eal.h b/lib/librte_eal/include/rte_eal.h index 2f9ed298de..2edf8c6556 100644 --- a/lib/librte_eal/include/rte_eal.h +++ b/lib/librte_eal/include/rte_eal.h @@ -447,6 +447,8 @@ enum rte_intr_mode rte_eal_vfio_intr_mode(void); */ int rte_sys_gettid(void); +RTE_DECLARE_PER_LCORE(int, _thread_id); + /** * Get system unique thread id. * @@ -456,7 +458,6 @@ int rte_sys_gettid(void); */ static inline int rte_gettid(void) { - static RTE_DEFINE_PER_LCORE(int, _thread_id) = -1; if (RTE_PER_LCORE(_thread_id) == -1) RTE_PER_LCORE(_thread_id) = rte_sys_gettid(); return RTE_PER_LCORE(_thread_id); diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 196eef5afa..0d42d44ce9 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -221,6 +221,13 @@ DPDK_20.0 { local: *; }; +DPDK_21 { + global: + + per_lcore__thread_id; + +} DPDK_20.0; + EXPERIMENTAL { global: From patchwork Fri Jun 26 14:47:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 72264 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F2FA9A051C; Fri, 26 Jun 2020 16:48:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9432C1C1BB; Fri, 26 Jun 2020 16:48:31 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id 09F9D1C1BB for ; Fri, 26 Jun 2020 16:48:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593182909; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A1k7SYh/jon3zucNP3/cndWYaOpJ9vSVSLEBmwHn9/0=; b=AgBmQsqWJXGrtkdSDZqupmlbYxaqWKiGtLGrqOGlfvCt2jndnyzwjJLuacoh6zkFbZgIF3 TF+n23dKcCqpqM23eRlBpGqd1KWBHVkpr7oUTyBMdPWUeuspSKpNOxEsUJ5QJtCS0z7qmm Ml9YhuzxOQ9geo6L2UmmCbC8AxVU75k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-19-u2e7VfQsPAyjCQylQiGhcg-1; Fri, 26 Jun 2020 10:48:25 -0400 X-MC-Unique: u2e7VfQsPAyjCQylQiGhcg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B6B1C804001; Fri, 26 Jun 2020 14:48:23 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 167F39CFE0; Fri, 26 Jun 2020 14:48:16 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, thomas@monjalon.net, arybchenko@solarflare.com, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Harini Ramakrishnan , Omar Cardona , Pallavi Kadam , Ranjit Menon Date: Fri, 26 Jun 2020 16:47:30 +0200 Message-Id: <20200626144736.11011-4-david.marchand@redhat.com> In-Reply-To: <20200626144736.11011-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200626144736.11011-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 3/9] eal: introduce thread init helper X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Introduce a helper responsible for initialising the per thread context. We can then have a unified context for EAL and non-EAL threads and remove copy/paste'd OS-specific helpers. Per EAL thread CPU affinity setting is separated from the thread init. It is to accommodate with Windows EAL where CPU affinity is not set at the moment. Besides, having affinity set by the master lcore in FreeBSD and Linux will make it possible to detect errors rather than panic in the child thread. But the cleanup when such an event happens is left for later. Signed-off-by: David Marchand --- Changes since v1: - rebased on master, removed Windows workarounds wrt gettid and traces support, --- lib/librte_eal/common/eal_common_thread.c | 51 +++++++++++++---------- lib/librte_eal/common/eal_thread.h | 8 ++-- lib/librte_eal/freebsd/eal.c | 14 ++++++- lib/librte_eal/freebsd/eal_thread.c | 32 +------------- lib/librte_eal/linux/eal.c | 15 ++++++- lib/librte_eal/linux/eal_thread.c | 32 +------------- lib/librte_eal/windows/eal.c | 3 +- lib/librte_eal/windows/eal_thread.c | 10 +---- 8 files changed, 66 insertions(+), 99 deletions(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 280c64bb76..afb30236c5 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -71,20 +71,10 @@ eal_cpuset_socket_id(rte_cpuset_t *cpusetp) return socket_id; } -int -rte_thread_set_affinity(rte_cpuset_t *cpusetp) +static void +thread_update_affinity(rte_cpuset_t *cpusetp) { - int s; - unsigned lcore_id; - pthread_t tid; - - tid = pthread_self(); - - s = pthread_setaffinity_np(tid, sizeof(rte_cpuset_t), cpusetp); - if (s != 0) { - RTE_LOG(ERR, EAL, "pthread_setaffinity_np failed\n"); - return -1; - } + unsigned int lcore_id = rte_lcore_id(); /* store socket_id in TLS for quick access */ RTE_PER_LCORE(_socket_id) = @@ -94,14 +84,24 @@ rte_thread_set_affinity(rte_cpuset_t *cpusetp) memmove(&RTE_PER_LCORE(_cpuset), cpusetp, sizeof(rte_cpuset_t)); - lcore_id = rte_lcore_id(); if (lcore_id != (unsigned)LCORE_ID_ANY) { /* EAL thread will update lcore_config */ lcore_config[lcore_id].socket_id = RTE_PER_LCORE(_socket_id); memmove(&lcore_config[lcore_id].cpuset, cpusetp, sizeof(rte_cpuset_t)); } +} +int +rte_thread_set_affinity(rte_cpuset_t *cpusetp) +{ + if (pthread_setaffinity_np(pthread_self(), sizeof(rte_cpuset_t), + cpusetp) != 0) { + RTE_LOG(ERR, EAL, "pthread_setaffinity_np failed\n"); + return -1; + } + + thread_update_affinity(cpusetp); return 0; } @@ -147,6 +147,19 @@ eal_thread_dump_affinity(char *str, unsigned size) return ret; } +void +rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset) +{ + /* set the lcore ID in per-lcore memory area */ + RTE_PER_LCORE(_lcore_id) = lcore_id; + + /* acquire system unique id */ + rte_gettid(); + + thread_update_affinity(cpuset); + + __rte_trace_mem_per_thread_alloc(); +} struct rte_thread_ctrl_params { void *(*start_routine)(void *); @@ -154,16 +167,14 @@ struct rte_thread_ctrl_params { pthread_barrier_t configured; }; -static void *rte_thread_init(void *arg) +static void *ctrl_thread_init(void *arg) { int ret; - rte_cpuset_t *cpuset = &internal_config.ctrl_cpuset; struct rte_thread_ctrl_params *params = arg; void *(*start_routine)(void *) = params->start_routine; void *routine_arg = params->arg; - /* Store cpuset in TLS for quick access */ - memmove(&RTE_PER_LCORE(_cpuset), cpuset, sizeof(rte_cpuset_t)); + rte_thread_init(rte_lcore_id(), &internal_config.ctrl_cpuset); ret = pthread_barrier_wait(¶ms->configured); if (ret == PTHREAD_BARRIER_SERIAL_THREAD) { @@ -171,8 +182,6 @@ static void *rte_thread_init(void *arg) free(params); } - __rte_trace_mem_per_thread_alloc(); - return start_routine(routine_arg); } @@ -194,7 +203,7 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, pthread_barrier_init(¶ms->configured, NULL, 2); - ret = pthread_create(thread, attr, rte_thread_init, (void *)params); + ret = pthread_create(thread, attr, ctrl_thread_init, (void *)params); if (ret != 0) { free(params); return -ret; diff --git a/lib/librte_eal/common/eal_thread.h b/lib/librte_eal/common/eal_thread.h index b40ed249ed..da5e7c93ba 100644 --- a/lib/librte_eal/common/eal_thread.h +++ b/lib/librte_eal/common/eal_thread.h @@ -16,12 +16,14 @@ __rte_noreturn void *eal_thread_loop(void *arg); /** - * Init per-lcore info for master thread + * Init per-lcore info in current thread. * * @param lcore_id - * identifier of master lcore + * identifier of lcore. + * @param cpuset + * CPU affinity for this thread. */ -void eal_thread_init_master(unsigned lcore_id); +void rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset); /** * Get the NUMA socket id from cpu id. diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 00fc66bf7f..13e5de006f 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -877,7 +877,14 @@ rte_eal_init(int argc, char **argv) eal_check_mem_on_local_socket(); - eal_thread_init_master(rte_config.master_lcore); + if (pthread_setaffinity_np(pthread_self(), sizeof(rte_cpuset_t), + &lcore_config[rte_config.master_lcore].cpuset) != 0) { + rte_eal_init_alert("Cannot set affinity"); + rte_errno = EINVAL; + return -1; + } + rte_thread_init(rte_config.master_lcore, + &lcore_config[rte_config.master_lcore].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); @@ -908,6 +915,11 @@ rte_eal_init(int argc, char **argv) snprintf(thread_name, sizeof(thread_name), "lcore-slave-%d", i); rte_thread_setname(lcore_config[i].thread_id, thread_name); + + ret = pthread_setaffinity_np(lcore_config[i].thread_id, + sizeof(rte_cpuset_t), &lcore_config[i].cpuset); + if (ret != 0) + rte_panic("Cannot set affinity\n"); } /* diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index 40676d9ef5..c1fb8eb2d8 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -66,29 +66,6 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned slave_id) return rc; } -/* set affinity for current thread */ -static int -eal_thread_set_affinity(void) -{ - unsigned lcore_id = rte_lcore_id(); - - /* acquire system unique id */ - rte_gettid(); - - /* update EAL thread core affinity */ - return rte_thread_set_affinity(&lcore_config[lcore_id].cpuset); -} - -void eal_thread_init_master(unsigned lcore_id) -{ - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); -} - /* main loop of threads */ __rte_noreturn void * eal_thread_loop(__rte_unused void *arg) @@ -113,19 +90,12 @@ eal_thread_loop(__rte_unused void *arg) m2s = lcore_config[lcore_id].pipe_master2slave[0]; s2m = lcore_config[lcore_id].pipe_slave2master[1]; - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); + rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%p;cpuset=[%s%s])\n", lcore_id, thread_id, cpuset, ret == 0 ? "" : "..."); - __rte_trace_mem_per_thread_alloc(); rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); /* read on our pipe to get commands */ diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 28a8b78517..8894cea50a 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -1205,10 +1205,16 @@ rte_eal_init(int argc, char **argv) eal_check_mem_on_local_socket(); - eal_thread_init_master(rte_config.master_lcore); + if (pthread_setaffinity_np(pthread_self(), sizeof(rte_cpuset_t), + &lcore_config[rte_config.master_lcore].cpuset) != 0) { + rte_eal_init_alert("Cannot set affinity"); + rte_errno = EINVAL; + return -1; + } + rte_thread_init(rte_config.master_lcore, + &lcore_config[rte_config.master_lcore].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "Master lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", rte_config.master_lcore, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); @@ -1240,6 +1246,11 @@ rte_eal_init(int argc, char **argv) if (ret != 0) RTE_LOG(DEBUG, EAL, "Cannot set name for lcore thread\n"); + + ret = pthread_setaffinity_np(lcore_config[i].thread_id, + sizeof(rte_cpuset_t), &lcore_config[i].cpuset); + if (ret != 0) + rte_panic("Cannot set affinity\n"); } /* diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index a52ebef3a4..07aec0c44d 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -66,29 +66,6 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned slave_id) return rc; } -/* set affinity for current EAL thread */ -static int -eal_thread_set_affinity(void) -{ - unsigned lcore_id = rte_lcore_id(); - - /* acquire system unique id */ - rte_gettid(); - - /* update EAL thread core affinity */ - return rte_thread_set_affinity(&lcore_config[lcore_id].cpuset); -} - -void eal_thread_init_master(unsigned lcore_id) -{ - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); -} - /* main loop of threads */ __rte_noreturn void * eal_thread_loop(__rte_unused void *arg) @@ -113,19 +90,12 @@ eal_thread_loop(__rte_unused void *arg) m2s = lcore_config[lcore_id].pipe_master2slave[0]; s2m = lcore_config[lcore_id].pipe_slave2master[1]; - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); + rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", lcore_id, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); - __rte_trace_mem_per_thread_alloc(); rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); /* read on our pipe to get commands */ diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c index 97c8427c73..adfaa00275 100644 --- a/lib/librte_eal/windows/eal.c +++ b/lib/librte_eal/windows/eal.c @@ -367,7 +367,8 @@ rte_eal_init(int argc, char **argv) return -1; } - eal_thread_init_master(rte_config.master_lcore); + rte_thread_init(rte_config.master_lcore, + &lcore_config[rte_config.master_lcore].cpuset); RTE_LCORE_FOREACH_SLAVE(i) { diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index f12a2ec6ad..4f01881240 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -53,13 +53,6 @@ rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int slave_id) return 0; } -void -eal_thread_init_master(unsigned int lcore_id) -{ - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; -} - /* main loop of threads */ void * eal_thread_loop(void *arg __rte_unused) @@ -84,8 +77,7 @@ eal_thread_loop(void *arg __rte_unused) m2s = lcore_config[lcore_id].pipe_master2slave[0]; s2m = lcore_config[lcore_id].pipe_slave2master[1]; - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; + rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s])\n", lcore_id, (uintptr_t)thread_id, cpuset); From patchwork Fri Jun 26 14:47:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 72265 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A510A051C; Fri, 26 Jun 2020 16:48:52 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EFD9B1C11F; Fri, 26 Jun 2020 16:48:51 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 7EFDB1C1C7 for ; Fri, 26 Jun 2020 16:48:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593182930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OLexyaCBy0Jai0Bt9gMZUzaWqHJhOPIAmiH15OMi4js=; b=cA6ubJNJdzb5Cemy1ZK8Uc4HPOs6y0WnIwxTiC/pmvoN8zfjKkqLIDqaBUoEbzliskUern RHAqWw9jsgZAyNMs10Eg6jn8yxP6ma97iyme+l4tgTDPcGq6hK9vtik4GkZj3Ef4TQx7kz igg5KoEcilQY0CGrgtMIQieagxhu/lQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-291-tPrGNmqAN3miCxuTnY_Gfw-1; Fri, 26 Jun 2020 10:48:48 -0400 X-MC-Unique: tPrGNmqAN3miCxuTnY_Gfw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BAF50A0C00; Fri, 26 Jun 2020 14:48:45 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 64AD79CFDF; Fri, 26 Jun 2020 14:48:26 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, thomas@monjalon.net, arybchenko@solarflare.com, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Jerin Jacob , Sunil Kumar Kori , Neil Horman , Harini Ramakrishnan , Omar Cardona , Pallavi Kadam , Ranjit Menon Date: Fri, 26 Jun 2020 16:47:31 +0200 Message-Id: <20200626144736.11011-5-david.marchand@redhat.com> In-Reply-To: <20200626144736.11011-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200626144736.11011-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 4/9] eal: introduce thread uninit helper X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is a preparation step for dynamically unregistering threads. Since we explicitly allocate a per thread trace buffer in rte_thread_init, add an internal helper to free this buffer. Signed-off-by: David Marchand --- Note: I preferred renaming the current internal function to free all threads trace buffers (new name trace_mem_free()) and reuse the previous name (trace_mem_per_thread_free()) when freeing this buffer for a given thread. Changes since v2: - added missing stub for windows tracing support, - moved free symbol to exported (experimental) ABI as a counterpart of the alloc symbol we already had, Changes since v1: - rebased on master, removed Windows workaround wrt traces support, --- lib/librte_eal/common/eal_common_thread.c | 9 ++++ lib/librte_eal/common/eal_common_trace.c | 51 +++++++++++++++++++---- lib/librte_eal/common/eal_thread.h | 5 +++ lib/librte_eal/common/eal_trace.h | 2 +- lib/librte_eal/include/rte_trace_point.h | 9 ++++ lib/librte_eal/rte_eal_version.map | 3 ++ lib/librte_eal/windows/eal.c | 5 +++ 7 files changed, 75 insertions(+), 9 deletions(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index afb30236c5..3b30cc99d9 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -20,6 +20,7 @@ #include "eal_internal_cfg.h" #include "eal_private.h" #include "eal_thread.h" +#include "eal_trace.h" RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; RTE_DEFINE_PER_LCORE(int, _thread_id) = -1; @@ -161,6 +162,14 @@ rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset) __rte_trace_mem_per_thread_alloc(); } +void +rte_thread_uninit(void) +{ + __rte_trace_mem_per_thread_free(); + + RTE_PER_LCORE(_lcore_id) = LCORE_ID_ANY; +} + struct rte_thread_ctrl_params { void *(*start_routine)(void *); void *arg; diff --git a/lib/librte_eal/common/eal_common_trace.c b/lib/librte_eal/common/eal_common_trace.c index 875553d7e5..3e620d76ed 100644 --- a/lib/librte_eal/common/eal_common_trace.c +++ b/lib/librte_eal/common/eal_common_trace.c @@ -101,7 +101,7 @@ eal_trace_fini(void) { if (!rte_trace_is_enabled()) return; - trace_mem_per_thread_free(); + trace_mem_free(); trace_metadata_destroy(); eal_trace_args_free(); } @@ -370,24 +370,59 @@ __rte_trace_mem_per_thread_alloc(void) rte_spinlock_unlock(&trace->lock); } +static void +trace_mem_per_thread_free_unlocked(struct thread_mem_meta *meta) +{ + if (meta->area == TRACE_AREA_HUGEPAGE) + eal_free_no_trace(meta->mem); + else if (meta->area == TRACE_AREA_HEAP) + free(meta->mem); +} + +void +__rte_trace_mem_per_thread_free(void) +{ + struct trace *trace = trace_obj_get(); + struct __rte_trace_header *header; + uint32_t count; + + if (RTE_PER_LCORE(trace_mem) == NULL) + return; + + header = RTE_PER_LCORE(trace_mem); + rte_spinlock_lock(&trace->lock); + for (count = 0; count < trace->nb_trace_mem_list; count++) { + if (trace->lcore_meta[count].mem == header) + break; + } + if (count != trace->nb_trace_mem_list) { + struct thread_mem_meta *meta = &trace->lcore_meta[count]; + + trace_mem_per_thread_free_unlocked(meta); + if (count != trace->nb_trace_mem_list - 1) { + memmove(meta, meta + 1, + sizeof(*meta) * + (trace->nb_trace_mem_list - count - 1)); + } + trace->nb_trace_mem_list--; + } + rte_spinlock_unlock(&trace->lock); +} + void -trace_mem_per_thread_free(void) +trace_mem_free(void) { struct trace *trace = trace_obj_get(); uint32_t count; - void *mem; if (!rte_trace_is_enabled()) return; rte_spinlock_lock(&trace->lock); for (count = 0; count < trace->nb_trace_mem_list; count++) { - mem = trace->lcore_meta[count].mem; - if (trace->lcore_meta[count].area == TRACE_AREA_HUGEPAGE) - eal_free_no_trace(mem); - else if (trace->lcore_meta[count].area == TRACE_AREA_HEAP) - free(mem); + trace_mem_per_thread_free_unlocked(&trace->lcore_meta[count]); } + trace->nb_trace_mem_list = 0; rte_spinlock_unlock(&trace->lock); } diff --git a/lib/librte_eal/common/eal_thread.h b/lib/librte_eal/common/eal_thread.h index da5e7c93ba..4ecd8fd53a 100644 --- a/lib/librte_eal/common/eal_thread.h +++ b/lib/librte_eal/common/eal_thread.h @@ -25,6 +25,11 @@ __rte_noreturn void *eal_thread_loop(void *arg); */ void rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset); +/** + * Uninitialize per-lcore info for current thread. + */ +void rte_thread_uninit(void); + /** * Get the NUMA socket id from cpu id. * This function is private to EAL. diff --git a/lib/librte_eal/common/eal_trace.h b/lib/librte_eal/common/eal_trace.h index 8f60616156..bbb6e1645c 100644 --- a/lib/librte_eal/common/eal_trace.h +++ b/lib/librte_eal/common/eal_trace.h @@ -106,7 +106,7 @@ int trace_metadata_create(void); void trace_metadata_destroy(void); int trace_mkdir(void); int trace_epoch_time_save(void); -void trace_mem_per_thread_free(void); +void trace_mem_free(void); /* EAL interface */ int eal_trace_init(void); diff --git a/lib/librte_eal/include/rte_trace_point.h b/lib/librte_eal/include/rte_trace_point.h index 377c2414aa..686b86fdb1 100644 --- a/lib/librte_eal/include/rte_trace_point.h +++ b/lib/librte_eal/include/rte_trace_point.h @@ -230,6 +230,15 @@ __rte_trace_point_fp_is_enabled(void) __rte_experimental void __rte_trace_mem_per_thread_alloc(void); +/** + * @internal + * + * Free trace memory buffer per thread. + * + */ +__rte_experimental +void __rte_trace_mem_per_thread_free(void); + /** * @internal * diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 0d42d44ce9..5831eea4b0 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -393,6 +393,9 @@ EXPERIMENTAL { rte_trace_point_lookup; rte_trace_regexp; rte_trace_save; + + # added in 20.08 + __rte_trace_mem_per_thread_free; }; INTERNAL { diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c index adfaa00275..27a44c49ff 100644 --- a/lib/librte_eal/windows/eal.c +++ b/lib/librte_eal/windows/eal.c @@ -255,6 +255,11 @@ __rte_trace_mem_per_thread_alloc(void) { } +void +__rte_trace_mem_per_thread_free(void) +{ +} + void __rte_trace_point_emit_field(size_t sz, const char *field, const char *type) From patchwork Fri Jun 26 14:47:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 72266 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 03E92A051C; Fri, 26 Jun 2020 16:49:04 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E028F1C1AA; Fri, 26 Jun 2020 16:48:59 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 2FAFF1C0BF for ; Fri, 26 Jun 2020 16:48:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593182937; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BXucOJ5qnjOdHBrJ5591+N7T6Lo29P0IPNPkyRfk2p8=; b=TPnnAJ1W0nmHkIYUQWnnEKfHVTEvi+rTEuPUd3qNbRxzMCioCIMSzhkefJlDoMjc4oTiRX 4WGxDa2O2zynNtvm8yvD8mAwXHsguRUKRVnnDsakr0Wfm1JwXFtJMFkI+85e+6M2yUEi96 gjPCjR+jh0DwMSqhY+YKtPFFhryW1ec= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-351-mQK3R4eAPzKUEDGJpJ5moQ-1; Fri, 26 Jun 2020 10:48:53 -0400 X-MC-Unique: mQK3R4eAPzKUEDGJpJ5moQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4804C10059BD; Fri, 26 Jun 2020 14:48:52 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2C4959CFCA; Fri, 26 Jun 2020 14:48:47 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, thomas@monjalon.net, arybchenko@solarflare.com, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org Date: Fri, 26 Jun 2020 16:47:32 +0200 Message-Id: <20200626144736.11011-6-david.marchand@redhat.com> In-Reply-To: <20200626144736.11011-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200626144736.11011-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 5/9] eal: move lcore role code X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" For consistency sake, move all lcore role code in the dedicated compilation unit / header. Signed-off-by: David Marchand Reviewed-by: Olivier Matz --- lib/librte_eal/common/eal_common_lcore.c | 11 +++++++ lib/librte_eal/common/eal_common_thread.c | 11 ------- lib/librte_eal/include/rte_eal.h | 9 ------ lib/librte_eal/include/rte_lcore.h | 37 ++++++++++++++--------- 4 files changed, 34 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 5404922a87..86d32a3dd7 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -63,6 +63,17 @@ rte_eal_lcore_role(unsigned int lcore_id) return cfg->lcore_role[lcore_id]; } +int +rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + + if (lcore_id >= RTE_MAX_LCORE) + return -EINVAL; + + return cfg->lcore_role[lcore_id] == role; +} + int rte_lcore_is_enabled(unsigned int lcore_id) { struct rte_config *cfg = rte_eal_get_configuration(); diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 3b30cc99d9..a7ae0691bf 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -33,17 +33,6 @@ unsigned rte_socket_id(void) return RTE_PER_LCORE(_socket_id); } -int -rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role) -{ - struct rte_config *cfg = rte_eal_get_configuration(); - - if (lcore_id >= RTE_MAX_LCORE) - return -EINVAL; - - return cfg->lcore_role[lcore_id] == role; -} - static int eal_cpuset_socket_id(rte_cpuset_t *cpusetp) { diff --git a/lib/librte_eal/include/rte_eal.h b/lib/librte_eal/include/rte_eal.h index 2edf8c6556..0913d1947c 100644 --- a/lib/librte_eal/include/rte_eal.h +++ b/lib/librte_eal/include/rte_eal.h @@ -31,15 +31,6 @@ extern "C" { /* Maximum thread_name length. */ #define RTE_MAX_THREAD_NAME_LEN 16 -/** - * The lcore role (used in RTE or not). - */ -enum rte_lcore_role_t { - ROLE_RTE, - ROLE_OFF, - ROLE_SERVICE, -}; - /** * The type of process in a linux, multi-process setup */ diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 5c1d1926e9..3968c40693 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -24,6 +24,15 @@ extern "C" { RTE_DECLARE_PER_LCORE(unsigned, _lcore_id); /**< Per thread "lcore id". */ +/** + * The lcore role (used in RTE or not). + */ +enum rte_lcore_role_t { + ROLE_RTE, + ROLE_OFF, + ROLE_SERVICE, +}; + /** * Get a lcore's role. * @@ -34,6 +43,20 @@ RTE_DECLARE_PER_LCORE(unsigned, _lcore_id); /**< Per thread "lcore id". */ */ enum rte_lcore_role_t rte_eal_lcore_role(unsigned int lcore_id); +/** + * Test if the core supplied has a specific role + * + * @param lcore_id + * The identifier of the lcore, which MUST be between 0 and + * RTE_MAX_LCORE-1. + * @param role + * The role to be checked against. + * @return + * Boolean value: positive if test is true; otherwise returns 0. + */ +int +rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role); + /** * Return the Application thread ID of the execution unit. * @@ -283,20 +306,6 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg); -/** - * Test if the core supplied has a specific role - * - * @param lcore_id - * The identifier of the lcore, which MUST be between 0 and - * RTE_MAX_LCORE-1. - * @param role - * The role to be checked against. - * @return - * Boolean value: positive if test is true; otherwise returns 0. - */ -int -rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role); - #ifdef __cplusplus } #endif From patchwork Fri Jun 26 14:47:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 72267 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6BB34A051C; Fri, 26 Jun 2020 16:49:36 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 544DE1C0BF; Fri, 26 Jun 2020 16:49:36 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by dpdk.org (Postfix) with ESMTP id 8A2271C0B8 for ; Fri, 26 Jun 2020 16:49:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593182974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HoVXtARxAtcU4NkgxK5yosEbQS+bOBUdjKPZu1r0XIU=; b=ElOro01I7EtR7dchcwGhVs7ilVElzUUvSQaKXh2uK2zc+C7ASEdegXQM1UZkTjYTJel+x2 Gx0w8UNBU0QTOsgif/jKxQTfEVDhirKyowd0YDxK5BTCbOKk7zyOYYpRGceXXrSmDZd8zz ZVWYd9hOLTyCnZZUDJBWjyI/ps+s+So= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-294-njwkrWSJPpquoJTVQ-EHhA-1; Fri, 26 Jun 2020 10:49:30 -0400 X-MC-Unique: njwkrWSJPpquoJTVQ-EHhA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9E18C190C822; Fri, 26 Jun 2020 14:49:00 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 265419CFCA; Fri, 26 Jun 2020 14:48:56 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, thomas@monjalon.net, arybchenko@solarflare.com, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, John McNamara , Marko Kovacevic , Anatoly Burakov , Olivier Matz , Neil Horman Date: Fri, 26 Jun 2020 16:47:33 +0200 Message-Id: <20200626144736.11011-7-david.marchand@redhat.com> In-Reply-To: <20200626144736.11011-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200626144736.11011-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 6/9] eal: register non-EAL threads as lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" DPDK allows calling some part of its API from a non-EAL thread but this has some limitations. OVS (and other applications) has its own thread management but still want to avoid such limitations by hacking RTE_PER_LCORE(_lcore_id) and faking EAL threads potentially unknown of some DPDK component. Introduce a new API to register non-EAL thread and associate them to a free lcore with a new NON_EAL role. This role denotes lcores that do not run DPDK mainloop and as such prevents use of rte_eal_wait_lcore() and consorts. Multiprocess is not supported as the need for cohabitation with this new feature is unclear at the moment. Signed-off-by: David Marchand Acked-by: Andrew Rybchenko --- Changes since v2: - refused multiprocess init once rte_thread_register got called, and vice versa, - added warning on multiprocess in rte_thread_register doxygen, Changes since v1: - moved cleanup on lcore role code in patch 5, - added unit test, - updated documentation, - changed naming from "external thread" to "registered non-EAL thread" --- MAINTAINERS | 1 + app/test/Makefile | 1 + app/test/autotest_data.py | 6 + app/test/meson.build | 2 + app/test/test_lcores.c | 139 ++++++++++++++++++ doc/guides/howto/debug_troubleshoot.rst | 5 +- .../prog_guide/env_abstraction_layer.rst | 22 +-- doc/guides/prog_guide/mempool_lib.rst | 2 +- lib/librte_eal/common/eal_common_lcore.c | 50 ++++++- lib/librte_eal/common/eal_common_mcfg.c | 36 +++++ lib/librte_eal/common/eal_common_thread.c | 33 +++++ lib/librte_eal/common/eal_memcfg.h | 10 ++ lib/librte_eal/common/eal_private.h | 18 +++ lib/librte_eal/freebsd/eal.c | 4 + lib/librte_eal/include/rte_lcore.h | 25 +++- lib/librte_eal/linux/eal.c | 4 + lib/librte_eal/rte_eal_version.map | 2 + lib/librte_mempool/rte_mempool.h | 11 +- 18 files changed, 349 insertions(+), 22 deletions(-) create mode 100644 app/test/test_lcores.c diff --git a/MAINTAINERS b/MAINTAINERS index 45b6c3a990..88c6d85e07 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -182,6 +182,7 @@ F: app/test/test_cycles.c F: app/test/test_debug.c F: app/test/test_eal* F: app/test/test_errno.c +F: app/test/test_lcores.c F: app/test/test_logs.c F: app/test/test_memcpy* F: app/test/test_per_lcore.c diff --git a/app/test/Makefile b/app/test/Makefile index 7b96a03a64..4a8dea2425 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -97,6 +97,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += test_flow_classify.c endif SRCS-y += test_rwlock.c +SRCS-y += test_lcores.c SRCS-$(CONFIG_RTE_LIBRTE_STACK) += test_stack.c SRCS-$(CONFIG_RTE_LIBRTE_STACK) += test_stack_perf.c diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py index 238ab631b4..4b7da45e09 100644 --- a/app/test/autotest_data.py +++ b/app/test/autotest_data.py @@ -62,6 +62,12 @@ "Func": rwlock_autotest, "Report": None, }, + { + "Name": "Lcores autotest", + "Command": "lcores_autotest", + "Func": default_autotest, + "Report": None, + }, { "Name": "Logs autotest", "Command": "logs_autotest", diff --git a/app/test/meson.build b/app/test/meson.build index 5233ead46e..a57477b7cc 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -67,6 +67,7 @@ test_sources = files('commands.c', 'test_ipsec_perf.c', 'test_kni.c', 'test_kvargs.c', + 'test_lcores.c', 'test_logs.c', 'test_lpm.c', 'test_lpm6.c', @@ -206,6 +207,7 @@ fast_tests = [ ['hash_autotest', true], ['interrupt_autotest', true], ['ipfrag_autotest', false], + ['lcores_autotest', true], ['logs_autotest', true], ['lpm_autotest', true], ['lpm6_autotest', true], diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c new file mode 100644 index 0000000000..864bcbade7 --- /dev/null +++ b/app/test/test_lcores.c @@ -0,0 +1,139 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2020 Red Hat, Inc. + */ + +#include +#include + +#include + +#include "test.h" + +struct thread_context { + enum { INIT, ERROR, DONE } state; + bool lcore_id_any; + pthread_t id; + unsigned int *registered_count; +}; +static void *thread_loop(void *arg) +{ + struct thread_context *t = arg; + unsigned int lcore_id; + + lcore_id = rte_lcore_id(); + if (lcore_id != LCORE_ID_ANY) { + printf("Incorrect lcore id for new thread %u\n", lcore_id); + t->state = ERROR; + } + rte_thread_register(); + lcore_id = rte_lcore_id(); + if ((t->lcore_id_any && lcore_id != LCORE_ID_ANY) || + (!t->lcore_id_any && lcore_id == LCORE_ID_ANY)) { + printf("Could not register new thread, got %u while %sexpecting %u\n", + lcore_id, t->lcore_id_any ? "" : "not ", LCORE_ID_ANY); + t->state = ERROR; + } + /* Report register happened to the control thread. */ + __atomic_add_fetch(t->registered_count, 1, __ATOMIC_RELEASE); + + /* Wait for release from the control thread. */ + while (__atomic_load_n(t->registered_count, __ATOMIC_ACQUIRE) != 0) + ; + rte_thread_unregister(); + lcore_id = rte_lcore_id(); + if (lcore_id != LCORE_ID_ANY) { + printf("Could not unregister new thread, %u still assigned\n", + lcore_id); + t->state = ERROR; + } + + if (t->state != ERROR) + t->state = DONE; + + return NULL; +} + +static int +test_non_eal_lcores(unsigned int eal_threads_count) +{ + struct thread_context thread_contexts[RTE_MAX_LCORE]; + unsigned int non_eal_threads_count; + unsigned int registered_count; + struct thread_context *t; + unsigned int i; + int ret; + + non_eal_threads_count = 0; + registered_count = 0; + + /* Try to create as many threads as possible. */ + for (i = 0; i < RTE_MAX_LCORE - eal_threads_count; i++) { + t = &thread_contexts[i]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = false; + if (pthread_create(&t->id, NULL, thread_loop, t) != 0) + break; + non_eal_threads_count++; + } + printf("non-EAL threads count: %u\n", non_eal_threads_count); + /* Wait all non-EAL threads to register. */ + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + + /* We managed to create the max number of threads, let's try to create + * one more. This will allow one more check. + */ + if (eal_threads_count + non_eal_threads_count < RTE_MAX_LCORE) + goto skip_lcore_any; + t = &thread_contexts[non_eal_threads_count]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = true; + if (pthread_create(&t->id, NULL, thread_loop, t) == 0) { + non_eal_threads_count++; + printf("non-EAL threads count: %u\n", non_eal_threads_count); + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + } + +skip_lcore_any: + /* Release all threads, and check their states. */ + __atomic_store_n(®istered_count, 0, __ATOMIC_RELEASE); + ret = 0; + for (i = 0; i < non_eal_threads_count; i++) { + t = &thread_contexts[i]; + pthread_join(t->id, NULL); + if (t->state != DONE) + ret = -1; + } + + return ret; +} + +static int +test_lcores(void) +{ + unsigned int eal_threads_count = 0; + unsigned int i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + if (!rte_lcore_has_role(i, ROLE_OFF)) + eal_threads_count++; + } + if (eal_threads_count == 0) { + printf("Something is broken, no EAL thread detected.\n"); + return TEST_FAILED; + } + printf("EAL threads count: %u, RTE_MAX_LCORE=%u\n", eal_threads_count, + RTE_MAX_LCORE); + + if (test_non_eal_lcores(eal_threads_count) < 0) + return TEST_FAILED; + + return TEST_SUCCESS; +} + +REGISTER_TEST_COMMAND(lcores_autotest, test_lcores); diff --git a/doc/guides/howto/debug_troubleshoot.rst b/doc/guides/howto/debug_troubleshoot.rst index cef016b2fe..5a46f5fba3 100644 --- a/doc/guides/howto/debug_troubleshoot.rst +++ b/doc/guides/howto/debug_troubleshoot.rst @@ -307,8 +307,9 @@ Custom worker function :numref:`dtg_distributor_worker`. #. Configuration issue isolation - * Identify core role using ``rte_eal_lcore_role`` to identify RTE, OFF and - SERVICE. Check performance functions are mapped to run on the cores. + * Identify core role using ``rte_eal_lcore_role`` to identify RTE, OFF, + SERVICE and NON_EAL. Check performance functions are mapped to run on the + cores. * For high-performance execution logic ensure running it on correct NUMA and non-master core. diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 48a2fec066..f64ae953d1 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -564,9 +564,13 @@ It's also compatible with the pattern of corelist('-l') option. non-EAL pthread support ~~~~~~~~~~~~~~~~~~~~~~~ -It is possible to use the DPDK execution context with any user pthread (aka. Non-EAL pthreads). -In a non-EAL pthread, the *_lcore_id* is always LCORE_ID_ANY which identifies that it is not an EAL thread with a valid, unique, *_lcore_id*. -Some libraries will use an alternative unique ID (e.g. TID), some will not be impacted at all, and some will work but with limitations (e.g. timer and mempool libraries). +It is possible to use the DPDK execution context with any user pthread (aka. non-EAL pthreads). +There are two kinds of non-EAL pthreads: + +- a registered non-EAL pthread with a valid *_lcore_id* that was successfully assigned by calling ``rte_thread_register()``, +- a non registered non-EAL pthread with a LCORE_ID_ANY, + +For non registered non-EAL pthread (with a LCORE_ID_ANY *_lcore_id*), some libraries will use an alternative unique ID (e.g. TID), some will not be impacted at all, and some will work but with limitations (e.g. timer and mempool libraries). All these impacts are mentioned in :ref:`known_issue_label` section. @@ -613,9 +617,9 @@ Known Issues + rte_mempool The rte_mempool uses a per-lcore cache inside the mempool. - For non-EAL pthreads, ``rte_lcore_id()`` will not return a valid number. - So for now, when rte_mempool is used with non-EAL pthreads, the put/get operations will bypass the default mempool cache and there is a performance penalty because of this bypass. - Only user-owned external caches can be used in a non-EAL context in conjunction with ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()`` that accept an explicit cache parameter. + For unregistered non-EAL pthreads, ``rte_lcore_id()`` will not return a valid number. + So for now, when rte_mempool is used with unregistered non-EAL pthreads, the put/get operations will bypass the default mempool cache and there is a performance penalty because of this bypass. + Only user-owned external caches can be used in an unregistered non-EAL context in conjunction with ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()`` that accept an explicit cache parameter. + rte_ring @@ -660,15 +664,15 @@ Known Issues + rte_timer - Running ``rte_timer_manage()`` on a non-EAL pthread is not allowed. However, resetting/stopping the timer from a non-EAL pthread is allowed. + Running ``rte_timer_manage()`` on an unregistered non-EAL pthread is not allowed. However, resetting/stopping the timer from a non-EAL pthread is allowed. + rte_log - In non-EAL pthreads, there is no per thread loglevel and logtype, global loglevels are used. + In unregistered non-EAL pthreads, there is no per thread loglevel and logtype, global loglevels are used. + misc - The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in a non-EAL pthread. + The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread. cgroup control ~~~~~~~~~~~~~~ diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst index f8b430d656..e3e1f940be 100644 --- a/doc/guides/prog_guide/mempool_lib.rst +++ b/doc/guides/prog_guide/mempool_lib.rst @@ -103,7 +103,7 @@ The maximum size of the cache is static and is defined at compilation time (CONF Alternatively to the internal default per-lcore local cache, an application can create and manage external caches through the ``rte_mempool_cache_create()``, ``rte_mempool_cache_free()`` and ``rte_mempool_cache_flush()`` calls. These user-owned caches can be explicitly passed to ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()``. The ``rte_mempool_default_cache()`` call returns the default internal cache if any. -In contrast to the default caches, user-owned caches can be used by non-EAL threads too. +In contrast to the default caches, user-owned caches can be used by unregistered non-EAL threads too. Mempool Handlers ------------------------ diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 86d32a3dd7..a61824a779 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -6,13 +6,15 @@ #include #include -#include -#include -#include -#include #include #include +#include +#include +#include +#include +#include +#include "eal_memcfg.h" #include "eal_private.h" #include "eal_thread.h" @@ -220,3 +222,43 @@ rte_socket_id_by_idx(unsigned int idx) } return config->numa_nodes[idx]; } + +static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; + +unsigned int +eal_lcore_non_eal_allocate(void) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + unsigned int lcore_id; + + if (cfg->process_type == RTE_PROC_SECONDARY || + !eal_mcfg_forbid_multiprocess()) { + RTE_LOG(ERR, EAL, "Multiprocess in use, cannot allocate new lcore.\n"); + return RTE_MAX_LCORE; + } + rte_spinlock_lock(&lcore_lock); + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] != ROLE_OFF) + continue; + cfg->lcore_role[lcore_id] = ROLE_NON_EAL; + cfg->lcore_count++; + break; + } + if (lcore_id == RTE_MAX_LCORE) + RTE_LOG(DEBUG, EAL, "No lcore available.\n"); + rte_spinlock_unlock(&lcore_lock); + return lcore_id; +} + +void +eal_lcore_non_eal_release(unsigned int lcore_id) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + + rte_spinlock_lock(&lcore_lock); + if (cfg->lcore_role[lcore_id] == ROLE_NON_EAL) { + cfg->lcore_role[lcore_id] = ROLE_OFF; + cfg->lcore_count--; + } + rte_spinlock_unlock(&lcore_lock); +} diff --git a/lib/librte_eal/common/eal_common_mcfg.c b/lib/librte_eal/common/eal_common_mcfg.c index 49d3ed0ce5..5b42d454e2 100644 --- a/lib/librte_eal/common/eal_common_mcfg.c +++ b/lib/librte_eal/common/eal_common_mcfg.c @@ -44,6 +44,42 @@ eal_mcfg_check_version(void) return 0; } +enum mp_status { + MP_UNKNOWN, + MP_FORBIDDEN, + MP_ENABLED, +}; + +static bool +eal_mcfg_set_mp_status(enum mp_status status) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + uint8_t expected; + uint8_t desired; + + expected = MP_UNKNOWN; + desired = status; + if (__atomic_compare_exchange_n(&mcfg->mp_status, &expected, desired, + false, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST)) + return true; + + return __atomic_load_n(&mcfg->mp_status, __ATOMIC_RELAXED) == desired; +} + +bool +eal_mcfg_forbid_multiprocess(void) +{ + assert(rte_eal_get_configuration()->process_type == RTE_PROC_PRIMARY); + return eal_mcfg_set_mp_status(MP_FORBIDDEN); +} + +bool +eal_mcfg_enable_multiprocess(void) +{ + assert(rte_eal_get_configuration()->process_type == RTE_PROC_SECONDARY); + return eal_mcfg_set_mp_status(MP_ENABLED); +} + void eal_mcfg_update_internal(void) { diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index a7ae0691bf..1cbddc4b5b 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -236,3 +236,36 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, pthread_join(*thread, NULL); return -ret; } + +void +rte_thread_register(void) +{ + unsigned int lcore_id; + rte_cpuset_t cpuset; + + /* EAL init flushes all lcores, we can't register before. */ + assert(internal_config.init_complete == 1); + if (pthread_getaffinity_np(pthread_self(), sizeof(cpuset), + &cpuset) != 0) + CPU_ZERO(&cpuset); + lcore_id = eal_lcore_non_eal_allocate(); + if (lcore_id >= RTE_MAX_LCORE) + lcore_id = LCORE_ID_ANY; + rte_thread_init(lcore_id, &cpuset); + if (lcore_id != LCORE_ID_ANY) + RTE_LOG(DEBUG, EAL, "Registered non-EAL thread as lcore %u.\n", + lcore_id); +} + +void +rte_thread_unregister(void) +{ + unsigned int lcore_id = rte_lcore_id(); + + if (lcore_id != LCORE_ID_ANY) + eal_lcore_non_eal_release(lcore_id); + rte_thread_uninit(); + if (lcore_id != LCORE_ID_ANY) + RTE_LOG(DEBUG, EAL, "Unregistered non-EAL thread (was lcore %u).\n", + lcore_id); +} diff --git a/lib/librte_eal/common/eal_memcfg.h b/lib/librte_eal/common/eal_memcfg.h index 583fcb5953..340e523c6a 100644 --- a/lib/librte_eal/common/eal_memcfg.h +++ b/lib/librte_eal/common/eal_memcfg.h @@ -41,6 +41,8 @@ struct rte_mem_config { rte_rwlock_t memory_hotplug_lock; /**< Indicates whether memory hotplug request is in progress. */ + uint8_t mp_status; /**< Indicates whether multiprocess can be used. */ + /* memory segments and zones */ struct rte_fbarray memzones; /**< Memzone descriptors. */ @@ -91,6 +93,14 @@ eal_mcfg_wait_complete(void); int eal_mcfg_check_version(void); +/* mark primary process as not supporting multi-process. */ +bool +eal_mcfg_forbid_multiprocess(void); + +/* instruct primary process that a secondary process attached once. */ +bool +eal_mcfg_enable_multiprocess(void); + /* set mem config as complete */ void eal_mcfg_complete(void); diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 0592fcd694..73238ff157 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -396,6 +396,24 @@ uint64_t get_tsc_freq(void); */ uint64_t get_tsc_freq_arch(void); +/** + * Allocate a free lcore to associate to a non-EAL thread. + * + * @return + * - the id of a lcore with role ROLE_NON_EAL on success. + * - RTE_MAX_LCORE if none was available. + */ +unsigned int eal_lcore_non_eal_allocate(void); + +/** + * Release the lcore used by a non-EAL thread. + * Counterpart of eal_lcore_non_eal_allocate(). + * + * @param lcore_id + * The lcore with role ROLE_NON_EAL to release. + */ +void eal_lcore_non_eal_release(unsigned int lcore_id); + /** * Prepare physical memory mapping * i.e. hugepages on Linux and diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 13e5de006f..32a3d999b8 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -424,6 +424,10 @@ rte_config_init(void) } if (rte_eal_config_reattach() < 0) return -1; + if (!eal_mcfg_enable_multiprocess()) { + RTE_LOG(ERR, EAL, "Primary process refused secondary attachment\n"); + return -1; + } eal_mcfg_update_internal(); break; case RTE_PROC_AUTO: diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 3968c40693..43747e88df 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -31,6 +31,7 @@ enum rte_lcore_role_t { ROLE_RTE, ROLE_OFF, ROLE_SERVICE, + ROLE_NON_EAL, }; /** @@ -67,7 +68,8 @@ rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role); * to run threads with lcore IDs 0, 1, 2 and 3 on physical core 10.. * * @return - * Logical core ID (in EAL thread) or LCORE_ID_ANY (in non-EAL thread) + * Logical core ID (in EAL thread or registered non-EAL thread) or + * LCORE_ID_ANY (in unregistered non-EAL thread) */ static inline unsigned rte_lcore_id(void) @@ -279,6 +281,27 @@ int rte_thread_setname(pthread_t id, const char *name); __rte_experimental int rte_thread_getname(pthread_t id, char *name, size_t len); +/** + * Register current non-EAL thread as a lcore. + * + * @note This API is not compatible with the multi-process feature: + * - if a primary process registers a non-EAL thread, then no secondary process + * will initialise. + * - if a secondary process initialises successfully, trying to register a + * non-EAL thread from either primary or secondary processes will always end + * up with the thread getting LCORE_ID_ANY as lcore. + */ +__rte_experimental +void +rte_thread_register(void); + +/** + * Unregister current thread and release lcore if one was associated. + */ +__rte_experimental +void +rte_thread_unregister(void); + /** * Create a control thread. * diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 8894cea50a..1d90d1c0e3 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -514,6 +514,10 @@ rte_config_init(void) } if (rte_eal_config_reattach() < 0) return -1; + if (!eal_mcfg_enable_multiprocess()) { + RTE_LOG(ERR, EAL, "Primary process refused secondary attachment\n"); + return -1; + } eal_mcfg_update_internal(); break; case RTE_PROC_AUTO: diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 5831eea4b0..39c41d445d 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -396,6 +396,8 @@ EXPERIMENTAL { # added in 20.08 __rte_trace_mem_per_thread_free; + rte_thread_register; + rte_thread_unregister; }; INTERNAL { diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h index 652d19f9f1..9e0ee052b3 100644 --- a/lib/librte_mempool/rte_mempool.h +++ b/lib/librte_mempool/rte_mempool.h @@ -28,9 +28,9 @@ * rte_mempool_get() or rte_mempool_put() are designed to be called from an EAL * thread due to the internal per-lcore cache. Due to the lack of caching, * rte_mempool_get() or rte_mempool_put() performance will suffer when called - * by non-EAL threads. Instead, non-EAL threads should call - * rte_mempool_generic_get() or rte_mempool_generic_put() with a user cache - * created with rte_mempool_cache_create(). + * by unregistered non-EAL threads. Instead, unregistered non-EAL threads + * should call rte_mempool_generic_get() or rte_mempool_generic_put() with a + * user cache created with rte_mempool_cache_create(). */ #include @@ -1233,7 +1233,7 @@ void rte_mempool_dump(FILE *f, struct rte_mempool *mp); /** * Create a user-owned mempool cache. * - * This can be used by non-EAL threads to enable caching when they + * This can be used by unregistered non-EAL threads to enable caching when they * interact with a mempool. * * @param size @@ -1264,7 +1264,8 @@ rte_mempool_cache_free(struct rte_mempool_cache *cache); * @param lcore_id * The logical core id. * @return - * A pointer to the mempool cache or NULL if disabled or non-EAL thread. + * A pointer to the mempool cache or NULL if disabled or unregistered non-EAL + * thread. */ static __rte_always_inline struct rte_mempool_cache * rte_mempool_default_cache(struct rte_mempool *mp, unsigned lcore_id) From patchwork Fri Jun 26 14:47:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 72269 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1D3F9A051C; Fri, 26 Jun 2020 16:49:56 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 975EE1C1E6; Fri, 26 Jun 2020 16:49:44 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id DBEC21C0BF for ; Fri, 26 Jun 2020 16:49:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593182975; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RnytCEhFIYQ3y13AI23baxR4eC8kmB6mXJNs5hs86hQ=; b=DHPhkh5N3y11v48KysSgYKkU8PxsRFQGXqYN5N56uJMT+NGuwnBWhnpolndeXy3Ce/foNf VtWpEB1PgL3U30ZLdzwT1B3kY+y5bdRhaIDyhZP5WVsMvHjJIruCh9IafSpkjQnPT39LC5 7P8pGumJyz82VCvlf/0YGicEN4im/Pk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-423-ydIKYRR6OiSEuW82tulw4w-1; Fri, 26 Jun 2020 10:49:32 -0400 X-MC-Unique: ydIKYRR6OiSEuW82tulw4w-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8875719200F8; Fri, 26 Jun 2020 14:49:05 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2F6BA9CFDD; Fri, 26 Jun 2020 14:49:01 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, thomas@monjalon.net, arybchenko@solarflare.com, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Neil Horman Date: Fri, 26 Jun 2020 16:47:34 +0200 Message-Id: <20200626144736.11011-8-david.marchand@redhat.com> In-Reply-To: <20200626144736.11011-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200626144736.11011-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 7/9] eal: add lcore init callbacks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" DPDK components and applications can have their say when a new lcore is initialized. For this, they can register a callback for initializing and releasing their private data. Signed-off-by: David Marchand --- Changes since v2: - added missing test, - fixed rollback on lcore register, Changes since v1: - added unit test (since missing some coverage, for v3), - preferred callback and removed mention of notification, --- app/test/test_lcores.c | 230 +++++++++++++++++++++++ lib/librte_eal/common/eal_common_lcore.c | 138 +++++++++++++- lib/librte_eal/common/eal_private.h | 3 +- lib/librte_eal/include/rte_lcore.h | 68 +++++++ lib/librte_eal/rte_eal_version.map | 2 + 5 files changed, 435 insertions(+), 6 deletions(-) diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c index 864bcbade7..e36dceedf9 100644 --- a/app/test/test_lcores.c +++ b/app/test/test_lcores.c @@ -5,6 +5,7 @@ #include #include +#include #include #include "test.h" @@ -113,6 +114,229 @@ test_non_eal_lcores(unsigned int eal_threads_count) return ret; } +struct limit_lcore_context { + unsigned int init; + unsigned int max; + unsigned int uninit; +}; +static int +limit_lcores_init(unsigned int lcore_id __rte_unused, void *arg) +{ + struct limit_lcore_context *l = arg; + + l->init++; + if (l->init > l->max) + return -1; + return 0; +} +static void +limit_lcores_uninit(unsigned int lcore_id __rte_unused, void *arg) +{ + struct limit_lcore_context *l = arg; + + l->uninit++; +} + +static int +test_lcores_callback(unsigned int eal_threads_count) +{ + struct limit_lcore_context l; + void *handle; + + /* Refuse last lcore => callback register error. */ + memset(&l, 0, sizeof(l)); + l.max = eal_threads_count - 1; + handle = rte_lcore_callback_register("limit", limit_lcores_init, + limit_lcores_uninit, &l); + if (handle != NULL) { + printf("lcore callback register should have failed\n"); + goto error; + } + /* Refusal happens at the n th call to the init callback. + * Besides, n - 1 were accepted, so we expect as many uninit calls when + * the rollback happens. + */ + if (l.init != eal_threads_count) { + printf("lcore callback register failed but incorrect init calls, expected %u, got %u\n", + eal_threads_count, l.init); + goto error; + } + if (l.uninit != eal_threads_count - 1) { + printf("lcore callback register failed but incorrect uninit calls, expected %u, got %u\n", + eal_threads_count - 1, l.uninit); + goto error; + } + + /* Accept all lcore and unregister. */ + memset(&l, 0, sizeof(l)); + l.max = eal_threads_count; + handle = rte_lcore_callback_register("limit", limit_lcores_init, + limit_lcores_uninit, &l); + if (handle == NULL) { + printf("lcore callback register failed\n"); + goto error; + } + if (l.uninit != 0) { + printf("lcore callback register succeeded but incorrect uninit calls, expected 0, got %u\n", + l.uninit); + goto error; + } + rte_lcore_callback_unregister(handle); + handle = NULL; + if (l.init != eal_threads_count) { + printf("lcore callback unregister done but incorrect init calls, expected %u, got %u\n", + eal_threads_count, l.init); + goto error; + } + if (l.uninit != eal_threads_count) { + printf("lcore callback unregister done but incorrect uninit calls, expected %u, got %u\n", + eal_threads_count, l.uninit); + goto error; + } + + return 0; + +error: + if (handle != NULL) + rte_lcore_callback_unregister(handle); + + return -1; +} + +static int +test_non_eal_lcores_callback(unsigned int eal_threads_count) +{ + struct thread_context thread_contexts[2]; + unsigned int non_eal_threads_count; + struct limit_lcore_context l[2]; + unsigned int registered_count; + struct thread_context *t; + void *handle[2]; + unsigned int i; + int ret; + + memset(l, 0, sizeof(l)); + handle[0] = handle[1] = NULL; + non_eal_threads_count = 0; + registered_count = 0; + + /* This test requires two empty slots to be sure lcore init refusal is + * because of callback execution. + */ + if (eal_threads_count + 2 >= RTE_MAX_LCORE) + return 0; + + /* Register two callbacks: + * - first one accepts any lcore, + * - second one accepts all EAL lcore + one more for the first non-EAL + * thread, then refuses the next lcore. + */ + l[0].max = UINT_MAX; + handle[0] = rte_lcore_callback_register("no_limit", limit_lcores_init, + limit_lcores_uninit, &l[0]); + if (handle[0] == NULL) { + printf("lcore callback [0] register failed\n"); + goto error; + } + l[1].max = eal_threads_count + 1; + handle[1] = rte_lcore_callback_register("limit", limit_lcores_init, + limit_lcores_uninit, &l[1]); + if (handle[1] == NULL) { + printf("lcore callback [1] register failed\n"); + goto error; + } + if (l[0].init != eal_threads_count || l[1].init != eal_threads_count) { + printf("lcore callbacks register succeeded but incorrect init calls, expected %u, %u, got %u, %u\n", + eal_threads_count, eal_threads_count, + l[0].init, l[1].init); + goto error; + } + if (l[0].uninit != 0 || l[1].uninit != 0) { + printf("lcore callbacks register succeeded but incorrect uninit calls, expected 0, 1, got %u, %u\n", + l[0].uninit, l[1].uninit); + goto error; + } + /* First thread that expects a valid lcore id. */ + t = &thread_contexts[0]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = false; + if (pthread_create(&t->id, NULL, thread_loop, t) != 0) + goto cleanup_threads; + non_eal_threads_count++; + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + if (l[0].init != eal_threads_count + 1 || + l[1].init != eal_threads_count + 1) { + printf("Incorrect init calls, expected %u, %u, got %u, %u\n", + eal_threads_count + 1, eal_threads_count + 1, + l[0].init, l[1].init); + goto cleanup_threads; + } + if (l[0].uninit != 0 || l[1].uninit != 0) { + printf("Incorrect uninit calls, expected 0, 0, got %u, %u\n", + l[0].uninit, l[1].uninit); + goto cleanup_threads; + } + /* Second thread, that expects LCORE_ID_ANY because of init refusal. */ + t = &thread_contexts[1]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = true; + if (pthread_create(&t->id, NULL, thread_loop, t) != 0) + goto cleanup_threads; + non_eal_threads_count++; + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + if (l[0].init != eal_threads_count + 2 || + l[1].init != eal_threads_count + 2) { + printf("Incorrect init calls, expected %u, %u, got %u, %u\n", + eal_threads_count + 2, eal_threads_count + 2, + l[0].init, l[1].init); + goto cleanup_threads; + } + if (l[0].uninit != 1 || l[1].uninit != 0) { + printf("Incorrect uninit calls, expected 1, 0, got %u, %u\n", + l[0].uninit, l[1].uninit); + goto cleanup_threads; + } + /* Release all threads, and check their states. */ + __atomic_store_n(®istered_count, 0, __ATOMIC_RELEASE); + ret = 0; + for (i = 0; i < non_eal_threads_count; i++) { + t = &thread_contexts[i]; + pthread_join(t->id, NULL); + if (t->state != DONE) + ret = -1; + } + if (ret < 0) + goto error; + if (l[0].uninit != 2 || l[1].uninit != 1) { + printf("Threads reported having successfully registered and unregistered, but incorrect uninit calls, expected 2, 1, got %u, %u\n", + l[0].uninit, l[1].uninit); + goto error; + } + rte_lcore_callback_unregister(handle[0]); + rte_lcore_callback_unregister(handle[1]); + return 0; + +cleanup_threads: + /* Release all threads */ + __atomic_store_n(®istered_count, 0, __ATOMIC_RELEASE); + for (i = 0; i < non_eal_threads_count; i++) { + t = &thread_contexts[i]; + pthread_join(t->id, NULL); + } +error: + if (handle[1] != NULL) + rte_lcore_callback_unregister(handle[1]); + if (handle[0] != NULL) + rte_lcore_callback_unregister(handle[0]); + return -1; +} + static int test_lcores(void) { @@ -133,6 +357,12 @@ test_lcores(void) if (test_non_eal_lcores(eal_threads_count) < 0) return TEST_FAILED; + if (test_lcores_callback(eal_threads_count) < 0) + return TEST_FAILED; + + if (test_non_eal_lcores_callback(eal_threads_count) < 0) + return TEST_FAILED; + return TEST_SUCCESS; } diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index a61824a779..52c46a4cea 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -224,11 +224,114 @@ rte_socket_id_by_idx(unsigned int idx) } static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; +struct lcore_callback { + TAILQ_ENTRY(lcore_callback) next; + char *name; + rte_lcore_init_cb init; + rte_lcore_uninit_cb uninit; + void *arg; +}; +static TAILQ_HEAD(lcore_callbacks_head, lcore_callback) lcore_callbacks = + TAILQ_HEAD_INITIALIZER(lcore_callbacks); + +static int +callback_init(struct lcore_callback *callback, unsigned int lcore_id) +{ + if (callback->init == NULL) + return 0; + RTE_LOG(DEBUG, EAL, "Call init for lcore callback %s, lcore_id %u\n", + callback->name, lcore_id); + return callback->init(lcore_id, callback->arg); +} + +static void +callback_uninit(struct lcore_callback *callback, unsigned int lcore_id) +{ + if (callback->uninit == NULL) + return; + RTE_LOG(DEBUG, EAL, "Call uninit for lcore callback %s, lcore_id %u\n", + callback->name, lcore_id); + callback->uninit(lcore_id, callback->arg); +} + +void * +rte_lcore_callback_register(const char *name, rte_lcore_init_cb init, + rte_lcore_uninit_cb uninit, void *arg) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback; + unsigned int lcore_id; + + callback = calloc(1, sizeof(*callback)); + if (callback == NULL) + return NULL; + if (asprintf(&callback->name, "%s-%p", name, arg) == -1) { + free(callback); + return NULL; + } + callback->init = init; + callback->uninit = uninit; + callback->arg = arg; + rte_spinlock_lock(&lcore_lock); + if (callback->init == NULL) + goto no_init; + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] == ROLE_OFF) + continue; + if (callback_init(callback, lcore_id) == 0) + continue; + /* Callback refused init for this lcore, uninitialize all + * previous lcore. + */ + for (; lcore_id != 0; lcore_id--) { + if (cfg->lcore_role[lcore_id - 1] == ROLE_OFF) + continue; + callback_uninit(callback, lcore_id - 1); + } + free(callback); + callback = NULL; + goto out; + } +no_init: + TAILQ_INSERT_TAIL(&lcore_callbacks, callback, next); + RTE_LOG(DEBUG, EAL, "Registered new lcore callback %s (%sinit, %suninit).\n", + callback->name, callback->init == NULL ? "NO " : "", + callback->uninit == NULL ? "NO " : ""); +out: + rte_spinlock_unlock(&lcore_lock); + return callback; +} + +void +rte_lcore_callback_unregister(void *handle) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback = handle; + unsigned int lcore_id; + + rte_spinlock_lock(&lcore_lock); + if (callback->uninit == NULL) + goto no_uninit; + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] == ROLE_OFF) + continue; + callback_uninit(callback, lcore_id); + } +no_uninit: + TAILQ_REMOVE(&lcore_callbacks, callback, next); + rte_spinlock_unlock(&lcore_lock); + RTE_LOG(DEBUG, EAL, "Unregistered lcore callback %s-%p.\n", + callback->name, callback->arg); + free(callback->name); + free(callback); +} unsigned int eal_lcore_non_eal_allocate(void) { struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback; + struct lcore_callback *prev; unsigned int lcore_id; if (cfg->process_type == RTE_PROC_SECONDARY || @@ -244,8 +347,29 @@ eal_lcore_non_eal_allocate(void) cfg->lcore_count++; break; } - if (lcore_id == RTE_MAX_LCORE) + if (lcore_id == RTE_MAX_LCORE) { RTE_LOG(DEBUG, EAL, "No lcore available.\n"); + goto out; + } + TAILQ_FOREACH(callback, &lcore_callbacks, next) { + if (callback_init(callback, lcore_id) == 0) + continue; + /* Callback refused init for this lcore, call uninit for all + * previous callbacks. + */ + prev = TAILQ_PREV(callback, lcore_callbacks_head, next); + while (prev != NULL) { + callback_uninit(prev, lcore_id); + prev = TAILQ_PREV(prev, lcore_callbacks_head, next); + } + RTE_LOG(DEBUG, EAL, "Initialization refused for lcore %u.\n", + lcore_id); + cfg->lcore_role[lcore_id] = ROLE_OFF; + cfg->lcore_count--; + lcore_id = RTE_MAX_LCORE; + goto out; + } +out: rte_spinlock_unlock(&lcore_lock); return lcore_id; } @@ -254,11 +378,15 @@ void eal_lcore_non_eal_release(unsigned int lcore_id) { struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback; rte_spinlock_lock(&lcore_lock); - if (cfg->lcore_role[lcore_id] == ROLE_NON_EAL) { - cfg->lcore_role[lcore_id] = ROLE_OFF; - cfg->lcore_count--; - } + if (cfg->lcore_role[lcore_id] != ROLE_NON_EAL) + goto out; + TAILQ_FOREACH(callback, &lcore_callbacks, next) + callback_uninit(callback, lcore_id); + cfg->lcore_role[lcore_id] = ROLE_OFF; + cfg->lcore_count--; +out: rte_spinlock_unlock(&lcore_lock); } diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 73238ff157..fc79564111 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -401,7 +401,8 @@ uint64_t get_tsc_freq_arch(void); * * @return * - the id of a lcore with role ROLE_NON_EAL on success. - * - RTE_MAX_LCORE if none was available. + * - RTE_MAX_LCORE if none was available or initializing was refused (see + * rte_lcore_callback_register). */ unsigned int eal_lcore_non_eal_allocate(void); diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 43747e88df..5a2d6ca7af 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -229,6 +229,74 @@ unsigned int rte_get_next_lcore(unsigned int i, int skip_master, int wrap); i X-Patchwork-Id: 72268 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2FB2EA051C; Fri, 26 Jun 2020 16:49:45 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C0CEF1C1AB; Fri, 26 Jun 2020 16:49:37 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by dpdk.org (Postfix) with ESMTP id C880D1C0BF for ; Fri, 26 Jun 2020 16:49:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593182974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kbFmOn4qyuSb4/j5djbtjssbBhbTUgcRMWE7urNI9Os=; b=fJTVSUaJPZi8VSG9jQ3niKWloskYzAg6d2IuxkPJDZ5jzg2FlR8/K8EBFdWu722LSM9/C7 OL6X4/Zpdc8emQCJYVHSj46/c4E0TM7iYHbslKodhVRpV57VrOQUih+6xDVCMWqCMx/+06 bBCEGy74QGWzYPXp29+uPvLFGoB5XYA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-378-QqKOmgb-ORmPoR4_esM2Zg-1; Fri, 26 Jun 2020 10:49:32 -0400 X-MC-Unique: QqKOmgb-ORmPoR4_esM2Zg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5A79F805EFA; Fri, 26 Jun 2020 14:49:11 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 20B7580881; Fri, 26 Jun 2020 14:49:06 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, thomas@monjalon.net, arybchenko@solarflare.com, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Neil Horman Date: Fri, 26 Jun 2020 16:47:35 +0200 Message-Id: <20200626144736.11011-9-david.marchand@redhat.com> In-Reply-To: <20200626144736.11011-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200626144736.11011-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 8/9] eal: add lcore iterators X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a helper to iterate all lcores. The iterator callback is read-only wrt the lcores list. Implement a dump function on top of this for debugging. Signed-off-by: David Marchand Reviewed-by: Olivier Matz --- Changes since v2: - added rte_lcore_dump calls in unit test, for basic check, Changes since v1: - introduced lcore iterators and implemented rte_lcore_dump, this iterator mechanism can then be used outside of EAL, --- app/test/test_lcores.c | 3 + lib/librte_eal/common/eal_common_lcore.c | 77 ++++++++++++++++++++--- lib/librte_eal/common/eal_common_thread.c | 16 +++-- lib/librte_eal/common/eal_thread.h | 13 +++- lib/librte_eal/freebsd/eal.c | 2 +- lib/librte_eal/freebsd/eal_thread.c | 2 +- lib/librte_eal/include/rte_lcore.h | 47 +++++++++++++- lib/librte_eal/linux/eal.c | 2 +- lib/librte_eal/linux/eal_thread.c | 2 +- lib/librte_eal/rte_eal_version.map | 2 + 10 files changed, 143 insertions(+), 23 deletions(-) diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c index e36dceedf9..dd6fa466c8 100644 --- a/app/test/test_lcores.c +++ b/app/test/test_lcores.c @@ -302,6 +302,7 @@ test_non_eal_lcores_callback(unsigned int eal_threads_count) l[0].uninit, l[1].uninit); goto cleanup_threads; } + rte_lcore_dump(stdout); /* Release all threads, and check their states. */ __atomic_store_n(®istered_count, 0, __ATOMIC_RELEASE); ret = 0; @@ -313,6 +314,7 @@ test_non_eal_lcores_callback(unsigned int eal_threads_count) } if (ret < 0) goto error; + rte_lcore_dump(stdout); if (l[0].uninit != 2 || l[1].uninit != 1) { printf("Threads reported having successfully registered and unregistered, but incorrect uninit calls, expected 2, 1, got %u, %u\n", l[0].uninit, l[1].uninit); @@ -353,6 +355,7 @@ test_lcores(void) } printf("EAL threads count: %u, RTE_MAX_LCORE=%u\n", eal_threads_count, RTE_MAX_LCORE); + rte_lcore_dump(stdout); if (test_non_eal_lcores(eal_threads_count) < 0) return TEST_FAILED; diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 52c46a4cea..a801cc537a 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -12,7 +12,7 @@ #include #include #include -#include +#include #include "eal_memcfg.h" #include "eal_private.h" @@ -223,7 +223,7 @@ rte_socket_id_by_idx(unsigned int idx) return config->numa_nodes[idx]; } -static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; +static rte_rwlock_t lcore_lock = RTE_RWLOCK_INITIALIZER; struct lcore_callback { TAILQ_ENTRY(lcore_callback) next; char *name; @@ -272,7 +272,7 @@ rte_lcore_callback_register(const char *name, rte_lcore_init_cb init, callback->init = init; callback->uninit = uninit; callback->arg = arg; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); if (callback->init == NULL) goto no_init; for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { @@ -298,7 +298,7 @@ rte_lcore_callback_register(const char *name, rte_lcore_init_cb init, callback->name, callback->init == NULL ? "NO " : "", callback->uninit == NULL ? "NO " : ""); out: - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); return callback; } @@ -309,7 +309,7 @@ rte_lcore_callback_unregister(void *handle) struct lcore_callback *callback = handle; unsigned int lcore_id; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); if (callback->uninit == NULL) goto no_uninit; for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { @@ -319,7 +319,7 @@ rte_lcore_callback_unregister(void *handle) } no_uninit: TAILQ_REMOVE(&lcore_callbacks, callback, next); - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); RTE_LOG(DEBUG, EAL, "Unregistered lcore callback %s-%p.\n", callback->name, callback->arg); free(callback->name); @@ -339,7 +339,7 @@ eal_lcore_non_eal_allocate(void) RTE_LOG(ERR, EAL, "Multiprocess in use, cannot allocate new lcore.\n"); return RTE_MAX_LCORE; } - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { if (cfg->lcore_role[lcore_id] != ROLE_OFF) continue; @@ -370,7 +370,7 @@ eal_lcore_non_eal_allocate(void) goto out; } out: - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); return lcore_id; } @@ -380,7 +380,7 @@ eal_lcore_non_eal_release(unsigned int lcore_id) struct rte_config *cfg = rte_eal_get_configuration(); struct lcore_callback *callback; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); if (cfg->lcore_role[lcore_id] != ROLE_NON_EAL) goto out; TAILQ_FOREACH(callback, &lcore_callbacks, next) @@ -388,5 +388,62 @@ eal_lcore_non_eal_release(unsigned int lcore_id) cfg->lcore_role[lcore_id] = ROLE_OFF; cfg->lcore_count--; out: - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); +} + +int +rte_lcore_iterate(rte_lcore_iterate_cb cb, void *arg) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + unsigned int lcore_id; + int ret = 0; + + rte_rwlock_read_lock(&lcore_lock); + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] == ROLE_OFF) + continue; + ret = cb(lcore_id, arg); + if (ret != 0) + break; + } + rte_rwlock_read_unlock(&lcore_lock); + return ret; +} + +static int +lcore_dump_cb(unsigned int lcore_id, void *arg) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + char cpuset[RTE_CPU_AFFINITY_STR_LEN]; + const char *role; + FILE *f = arg; + int ret; + + switch (cfg->lcore_role[lcore_id]) { + case ROLE_RTE: + role = "RTE"; + break; + case ROLE_SERVICE: + role = "SERVICE"; + break; + case ROLE_NON_EAL: + role = "NON_EAL"; + break; + default: + role = "UNKNOWN"; + break; + } + + ret = eal_thread_dump_affinity(&lcore_config[lcore_id].cpuset, cpuset, + sizeof(cpuset)); + fprintf(f, "lcore %u, socket %u, role %s, cpuset %s%s\n", lcore_id, + rte_lcore_to_socket_id(lcore_id), role, cpuset, + ret == 0 ? "" : "..."); + return 0; +} + +void +rte_lcore_dump(FILE *f) +{ + rte_lcore_iterate(lcore_dump_cb, f); } diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 1cbddc4b5b..e00374b06f 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -104,17 +104,14 @@ rte_thread_get_affinity(rte_cpuset_t *cpusetp) } int -eal_thread_dump_affinity(char *str, unsigned size) +eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size) { - rte_cpuset_t cpuset; unsigned cpu; int ret; unsigned int out = 0; - rte_thread_get_affinity(&cpuset); - for (cpu = 0; cpu < CPU_SETSIZE; cpu++) { - if (!CPU_ISSET(cpu, &cpuset)) + if (!CPU_ISSET(cpu, cpuset)) continue; ret = snprintf(str + out, @@ -137,6 +134,15 @@ eal_thread_dump_affinity(char *str, unsigned size) return ret; } +int +eal_thread_dump_current_affinity(char *str, unsigned int size) +{ + rte_cpuset_t cpuset; + + rte_thread_get_affinity(&cpuset); + return eal_thread_dump_affinity(&cpuset, str, size); +} + void rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset) { diff --git a/lib/librte_eal/common/eal_thread.h b/lib/librte_eal/common/eal_thread.h index 4ecd8fd53a..13ec252e01 100644 --- a/lib/librte_eal/common/eal_thread.h +++ b/lib/librte_eal/common/eal_thread.h @@ -47,13 +47,15 @@ unsigned eal_cpu_socket_id(unsigned cpu_id); #define RTE_CPU_AFFINITY_STR_LEN 256 /** - * Dump the current pthread cpuset. + * Dump the cpuset as a human readable string. * This function is private to EAL. * * Note: * If the dump size is greater than the size of given buffer, * the string will be truncated and with '\0' at the end. * + * @param cpuset + * The CPU affinity object to dump. * @param str * The string buffer the cpuset will dump to. * @param size @@ -62,6 +64,13 @@ unsigned eal_cpu_socket_id(unsigned cpu_id); * 0 for success, -1 if truncation happens. */ int -eal_thread_dump_affinity(char *str, unsigned size); +eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size); + +/** + * Dump the current thread cpuset. + * This is a wrapper on eal_thread_dump_affinity(). + */ +int +eal_thread_dump_current_affinity(char *str, unsigned int size); #endif /* EAL_THREAD_H */ diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 32a3d999b8..b93c6fa909 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -890,7 +890,7 @@ rte_eal_init(int argc, char **argv) rte_thread_init(rte_config.master_lcore, &lcore_config[rte_config.master_lcore].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "Master lcore %u is ready (tid=%p;cpuset=[%s%s])\n", rte_config.master_lcore, thread_id, cpuset, diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index c1fb8eb2d8..b1a3619f51 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -92,7 +92,7 @@ eal_thread_loop(__rte_unused void *arg) rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%p;cpuset=[%s%s])\n", lcore_id, thread_id, cpuset, ret == 0 ? "" : "..."); diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 5a2d6ca7af..a9bcbbc25d 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -261,8 +261,8 @@ typedef void (*rte_lcore_uninit_cb)(unsigned int lcore_id, void *arg); * If this step succeeds, the callbacks are put in the lcore callbacks list * that will get called for each lcore allocation/release. * - * Note: callbacks execution is serialised under a lock protecting the lcores - * and callbacks list. + * Note: callbacks execution is serialised under a write lock protecting the + * lcores and callbacks list. * * @param name * A name serving as a small description for this callback. @@ -297,6 +297,49 @@ __rte_experimental void rte_lcore_callback_unregister(void *handle); +/** + * Callback prototype for iterating over lcores. + * + * @param lcore_id + * The lcore to consider. + * @param arg + * An opaque pointer coming from the caller. + * @return + * - 0 lets the iteration continue. + * - !0 makes the iteration stop. + */ +typedef int (*rte_lcore_iterate_cb)(unsigned int lcore_id, void *arg); + +/** + * Iterate on all active lcores (ROLE_RTE, ROLE_SERVICE and ROLE_NON_EAL). + * No modification on the lcore states is allowed in the callback. + * + * Note: as opposed to init/uninit callbacks, iteration callbacks can be + * invoked in parallel as they are run under a read lock protecting the lcores + * and callbacks list. + * + * @param cb + * The callback that gets passed each lcore. + * @param arg + * An opaque pointer passed to cb. + * @return + * Same return code as the callback last invocation (see rte_lcore_iterate_cb + * description). + */ +__rte_experimental +int +rte_lcore_iterate(rte_lcore_iterate_cb cb, void *arg); + +/** + * List all lcores. + * + * @param f + * The output stream where the dump should be sent. + */ +__rte_experimental +void +rte_lcore_dump(FILE *f); + /** * Set core affinity of the current thread. * Support both EAL and non-EAL thread and update TLS. diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 1d90d1c0e3..dfa92b11a1 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -1218,7 +1218,7 @@ rte_eal_init(int argc, char **argv) rte_thread_init(rte_config.master_lcore, &lcore_config[rte_config.master_lcore].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "Master lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", rte_config.master_lcore, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index 07aec0c44d..22d9bc8c01 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -92,7 +92,7 @@ eal_thread_loop(__rte_unused void *arg) rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", lcore_id, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index aeee7cf431..23a1565dfb 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -398,6 +398,8 @@ EXPERIMENTAL { __rte_trace_mem_per_thread_free; rte_lcore_callback_register; rte_lcore_callback_unregister; + rte_lcore_dump; + rte_lcore_iterate; rte_thread_register; rte_thread_unregister; }; From patchwork Fri Jun 26 14:47:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 72270 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E0CB4A051C; Fri, 26 Jun 2020 16:50:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6CF6B1C21C; Fri, 26 Jun 2020 16:49:47 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id 7FD091C1E5 for ; Fri, 26 Jun 2020 16:49:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593182983; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/QCM7m5LmHLFgqxCX3fK0yv7GSP7ic7sl0LPlCRa/qc=; b=GLp/1YLgvylrwErKReUuop6RGv/2T1iA0N8YTy6zN2aFD47OPJMXzreiDPB2JCR4JC3Rc2 EqkDIO2v3OCeqvWwXk/qacDMNKJxw6WocZXzI9ud9yQx1kjtrKAyNTpI15654RRo0xmS7h veb/Yvvu4CLqoYTDIHs7QsElI1iTQow= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-431-3ey9GWBnMZC1agP79psQ6A-1; Fri, 26 Jun 2020 10:49:38 -0400 X-MC-Unique: 3ey9GWBnMZC1agP79psQ6A-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2036B8C420B; Fri, 26 Jun 2020 14:49:17 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9FA9CB3A60; Fri, 26 Jun 2020 14:49:13 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, thomas@monjalon.net, arybchenko@solarflare.com, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, "Artem V. Andreev" Date: Fri, 26 Jun 2020 16:47:36 +0200 Message-Id: <20200626144736.11011-10-david.marchand@redhat.com> In-Reply-To: <20200626144736.11011-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200626144736.11011-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v4 9/9] mempool/bucket: handle non-EAL lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Convert to new lcore API to support non-EAL lcores. Signed-off-by: David Marchand Reviewed-by: Andrew Rybchenko --- Changes since v3: - addressed Andrew comments, --- drivers/mempool/bucket/rte_mempool_bucket.c | 130 ++++++++++++-------- 1 file changed, 81 insertions(+), 49 deletions(-) diff --git a/drivers/mempool/bucket/rte_mempool_bucket.c b/drivers/mempool/bucket/rte_mempool_bucket.c index 5ce1ef16fb..8b9daa9782 100644 --- a/drivers/mempool/bucket/rte_mempool_bucket.c +++ b/drivers/mempool/bucket/rte_mempool_bucket.c @@ -55,6 +55,7 @@ struct bucket_data { struct rte_ring *shared_orphan_ring; struct rte_mempool *pool; unsigned int bucket_mem_size; + void *lcore_callback_handle; }; static struct bucket_stack * @@ -345,6 +346,23 @@ bucket_dequeue_contig_blocks(struct rte_mempool *mp, void **first_obj_table, return 0; } +struct bucket_count_per_lcore_ctx { + const struct bucket_data *bd; + unsigned int count; +}; + +static int +bucket_count_per_lcore(unsigned int lcore_id, void *arg) +{ + struct bucket_count_per_lcore_ctx *bplc = arg; + + bplc->count += bplc->bd->obj_per_bucket * + bplc->bd->buckets[lcore_id]->top; + bplc->count += + rte_ring_count(bplc->bd->adoption_buffer_rings[lcore_id]); + return 0; +} + static void count_underfilled_buckets(struct rte_mempool *mp, void *opaque, @@ -373,23 +391,64 @@ count_underfilled_buckets(struct rte_mempool *mp, static unsigned int bucket_get_count(const struct rte_mempool *mp) { - const struct bucket_data *bd = mp->pool_data; - unsigned int count = - bd->obj_per_bucket * rte_ring_count(bd->shared_bucket_ring) + - rte_ring_count(bd->shared_orphan_ring); - unsigned int i; + struct bucket_count_per_lcore_ctx bplc; - for (i = 0; i < RTE_MAX_LCORE; i++) { - if (!rte_lcore_is_enabled(i)) - continue; - count += bd->obj_per_bucket * bd->buckets[i]->top + - rte_ring_count(bd->adoption_buffer_rings[i]); - } + bplc.bd = mp->pool_data; + bplc.count = bplc.bd->obj_per_bucket * + rte_ring_count(bplc.bd->shared_bucket_ring); + bplc.count += rte_ring_count(bplc.bd->shared_orphan_ring); + rte_lcore_iterate(bucket_count_per_lcore, &bplc); rte_mempool_mem_iter((struct rte_mempool *)(uintptr_t)mp, - count_underfilled_buckets, &count); + count_underfilled_buckets, &bplc.count); + + return bplc.count; +} + +static int +bucket_init_per_lcore(unsigned int lcore_id, void *arg) +{ + char rg_name[RTE_RING_NAMESIZE]; + struct bucket_data *bd = arg; + struct rte_mempool *mp; + int rg_flags; + int rc; + + mp = bd->pool; + bd->buckets[lcore_id] = bucket_stack_create(mp, + mp->size / bd->obj_per_bucket); + if (bd->buckets[lcore_id] == NULL) + goto error; + + rc = snprintf(rg_name, sizeof(rg_name), RTE_MEMPOOL_MZ_FORMAT ".a%u", + mp->name, lcore_id); + if (rc < 0 || rc >= (int)sizeof(rg_name)) + goto error; + + rg_flags = RING_F_SC_DEQ; + if (mp->flags & MEMPOOL_F_SP_PUT) + rg_flags |= RING_F_SP_ENQ; + bd->adoption_buffer_rings[lcore_id] = rte_ring_create(rg_name, + rte_align32pow2(mp->size + 1), mp->socket_id, rg_flags); + if (bd->adoption_buffer_rings[lcore_id] == NULL) + goto error; - return count; + return 0; +error: + rte_free(bd->buckets[lcore_id]); + bd->buckets[lcore_id] = NULL; + return -1; +} + +static void +bucket_uninit_per_lcore(unsigned int lcore_id, void *arg) +{ + struct bucket_data *bd = arg; + + rte_ring_free(bd->adoption_buffer_rings[lcore_id]); + bd->adoption_buffer_rings[lcore_id] = NULL; + rte_free(bd->buckets[lcore_id]); + bd->buckets[lcore_id] = NULL; } static int @@ -399,7 +458,6 @@ bucket_alloc(struct rte_mempool *mp) int rc = 0; char rg_name[RTE_RING_NAMESIZE]; struct bucket_data *bd; - unsigned int i; unsigned int bucket_header_size; size_t pg_sz; @@ -429,36 +487,17 @@ bucket_alloc(struct rte_mempool *mp) /* eventually this should be a tunable parameter */ bd->bucket_stack_thresh = (mp->size / bd->obj_per_bucket) * 4 / 3; + bd->lcore_callback_handle = rte_lcore_callback_register("bucket", + bucket_init_per_lcore, bucket_uninit_per_lcore, bd); + if (bd->lcore_callback_handle == NULL) { + rc = -ENOMEM; + goto no_mem_for_stacks; + } + if (mp->flags & MEMPOOL_F_SP_PUT) rg_flags |= RING_F_SP_ENQ; if (mp->flags & MEMPOOL_F_SC_GET) rg_flags |= RING_F_SC_DEQ; - - for (i = 0; i < RTE_MAX_LCORE; i++) { - if (!rte_lcore_is_enabled(i)) - continue; - bd->buckets[i] = - bucket_stack_create(mp, mp->size / bd->obj_per_bucket); - if (bd->buckets[i] == NULL) { - rc = -ENOMEM; - goto no_mem_for_stacks; - } - rc = snprintf(rg_name, sizeof(rg_name), - RTE_MEMPOOL_MZ_FORMAT ".a%u", mp->name, i); - if (rc < 0 || rc >= (int)sizeof(rg_name)) { - rc = -ENAMETOOLONG; - goto no_mem_for_stacks; - } - bd->adoption_buffer_rings[i] = - rte_ring_create(rg_name, rte_align32pow2(mp->size + 1), - mp->socket_id, - rg_flags | RING_F_SC_DEQ); - if (bd->adoption_buffer_rings[i] == NULL) { - rc = -rte_errno; - goto no_mem_for_stacks; - } - } - rc = snprintf(rg_name, sizeof(rg_name), RTE_MEMPOOL_MZ_FORMAT ".0", mp->name); if (rc < 0 || rc >= (int)sizeof(rg_name)) { @@ -498,11 +537,8 @@ bucket_alloc(struct rte_mempool *mp) rte_ring_free(bd->shared_orphan_ring); cannot_create_shared_orphan_ring: invalid_shared_orphan_ring: + rte_lcore_callback_unregister(bd->lcore_callback_handle); no_mem_for_stacks: - for (i = 0; i < RTE_MAX_LCORE; i++) { - rte_free(bd->buckets[i]); - rte_ring_free(bd->adoption_buffer_rings[i]); - } rte_free(bd); no_mem_for_data: rte_errno = -rc; @@ -512,16 +548,12 @@ bucket_alloc(struct rte_mempool *mp) static void bucket_free(struct rte_mempool *mp) { - unsigned int i; struct bucket_data *bd = mp->pool_data; if (bd == NULL) return; - for (i = 0; i < RTE_MAX_LCORE; i++) { - rte_free(bd->buckets[i]); - rte_ring_free(bd->adoption_buffer_rings[i]); - } + rte_lcore_callback_unregister(bd->lcore_callback_handle); rte_ring_free(bd->shared_orphan_ring); rte_ring_free(bd->shared_bucket_ring);