From patchwork Mon Jun 22 13:25:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71969 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 29371A0519; Mon, 22 Jun 2020 15:26:10 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 14B5B1D6DA; Mon, 22 Jun 2020 15:26:01 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id DC8371D6C9 for ; Mon, 22 Jun 2020 15:25:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592832357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tgAkQXWyU6ppia54pOxgztU/wTH6cJJnwTDZt98OIII=; b=X+ZZzKJDtjT4uXedE7unW+39NqGw4JVq0IVEKmD3GPNOnlhY9tm1++Is2/NLuUfI4QIT5m ygYZgI3WkIB5+l5BahPJ1Qisd4BzctnwMLkSKpGDe4tuCCj8+PFX8A/biBFLauzK+s3q/q kO0mpr2fSPiSy9kkf0fzlAu0zuwSxRM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-419-VyauCBQZPlePbFUHrmyzKA-1; Mon, 22 Jun 2020 09:25:55 -0400 X-MC-Unique: VyauCBQZPlePbFUHrmyzKA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D075E1B18BC1; Mon, 22 Jun 2020 13:25:53 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id AD2F671679; Mon, 22 Jun 2020 13:25:49 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Harini Ramakrishnan , Omar Cardona , Pallavi Kadam , Ranjit Menon Date: Mon, 22 Jun 2020 15:25:23 +0200 Message-Id: <20200622132531.21857-2-david.marchand@redhat.com> In-Reply-To: <20200622132531.21857-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v3 1/9] eal: relocate per thread symbols to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" We have per lcore thread symbols scattered in OS implementations but common code relies on them. Move all of them in common. RTE_PER_LCORE(_socket_id) and RTE_PER_LCORE(_cpuset) have public accessors and are not exported through the library map, they can be made static. Signed-off-by: David Marchand --- lib/librte_eal/common/eal_common_thread.c | 5 ++++- lib/librte_eal/freebsd/eal_thread.c | 4 ---- lib/librte_eal/include/rte_lcore.h | 1 - lib/librte_eal/linux/eal_thread.c | 4 ---- lib/librte_eal/windows/eal_thread.c | 4 ---- 5 files changed, 4 insertions(+), 14 deletions(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 370bb1b634..a5f67d811c 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -21,7 +21,10 @@ #include "eal_private.h" #include "eal_thread.h" -RTE_DECLARE_PER_LCORE(unsigned , _socket_id); +RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; +static RTE_DEFINE_PER_LCORE(unsigned int, _socket_id) = + (unsigned int)SOCKET_ID_ANY; +static RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); unsigned rte_socket_id(void) { diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index b52019782a..40676d9ef5 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -25,10 +25,6 @@ #include "eal_private.h" #include "eal_thread.h" -RTE_DEFINE_PER_LCORE(unsigned, _lcore_id) = LCORE_ID_ANY; -RTE_DEFINE_PER_LCORE(unsigned, _socket_id) = (unsigned)SOCKET_ID_ANY; -RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); - /* * Send a message to a slave lcore identified by slave_id to call a * function f with argument arg. Once the execution is done, the diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 339046bc86..5c1d1926e9 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -23,7 +23,6 @@ extern "C" { #define LCORE_ID_ANY UINT32_MAX /**< Any lcore. */ RTE_DECLARE_PER_LCORE(unsigned, _lcore_id); /**< Per thread "lcore id". */ -RTE_DECLARE_PER_LCORE(rte_cpuset_t, _cpuset); /**< Per thread "cpuset". */ /** * Get a lcore's role. diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index cd9d6e0ebf..a52ebef3a4 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -25,10 +25,6 @@ #include "eal_private.h" #include "eal_thread.h" -RTE_DEFINE_PER_LCORE(unsigned, _lcore_id) = LCORE_ID_ANY; -RTE_DEFINE_PER_LCORE(unsigned, _socket_id) = (unsigned)SOCKET_ID_ANY; -RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); - /* * Send a message to a slave lcore identified by slave_id to call a * function f with argument arg. Once the execution is done, the diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index 3dd56519c9..f12a2ec6ad 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -16,10 +16,6 @@ #include "eal_private.h" #include "eal_windows.h" -RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; -RTE_DEFINE_PER_LCORE(unsigned int, _socket_id) = (unsigned int)SOCKET_ID_ANY; -RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); - /* * Send a message to a slave lcore identified by slave_id to call a * function f with argument arg. Once the execution is done, the From patchwork Mon Jun 22 13:25:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71970 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 93695A0519; Mon, 22 Jun 2020 15:26:18 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7808E1D6E6; Mon, 22 Jun 2020 15:26:05 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id 30E911D6D5 for ; Mon, 22 Jun 2020 15:26:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592832363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J8bcJr1ym6XYlcLwbnSKM4jOhL8PmhczSwCFWrkuVeU=; b=WGz998036FuLdT9+d8nun0caiUAjQy8CqCZPO9BnZEuGTEVIXJAbUm1PmjUUNULcFNMa/3 wM6h3eN+icG9JjSXrhOqZbsDjrLm6MZrDBJlEgpL2CmzhND8gbUu6aLOrceqAXyHgvspAP cClB1TL7ULvuf98CO9TkM+wlSbAYHOs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-26-GVk6xorqMCmuBJEH2RpI_A-1; Mon, 22 Jun 2020 09:26:02 -0400 X-MC-Unique: GVk6xorqMCmuBJEH2RpI_A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8902610059A1; Mon, 22 Jun 2020 13:25:59 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9A0701DC; Mon, 22 Jun 2020 13:25:54 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Neil Horman , Cunming Liang , Konstantin Ananyev , Olivier Matz Date: Mon, 22 Jun 2020 15:25:24 +0200 Message-Id: <20200622132531.21857-3-david.marchand@redhat.com> In-Reply-To: <20200622132531.21857-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v3 2/9] eal: fix multiple definition of per lcore thread id X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Because of the inline accessor + static declaration in rte_gettid(), we end up with multiple symbols for RTE_PER_LCORE(_thread_id). Each compilation unit will pay a cost when accessing this information for the first time. $ nm build/app/dpdk-testpmd | grep per_lcore__thread_id 0000000000000054 d per_lcore__thread_id.5037 0000000000000040 d per_lcore__thread_id.5103 0000000000000048 d per_lcore__thread_id.5259 000000000000004c d per_lcore__thread_id.5259 0000000000000044 d per_lcore__thread_id.5933 0000000000000058 d per_lcore__thread_id.6261 0000000000000050 d per_lcore__thread_id.7378 000000000000005c d per_lcore__thread_id.7496 000000000000000c d per_lcore__thread_id.8016 0000000000000010 d per_lcore__thread_id.8431 Make it global as part of the DPDK_21 stable ABI. Fixes: ef76436c6834 ("eal: get unique thread id") Signed-off-by: David Marchand Acked-by: Ray Kinsella --- lib/librte_eal/common/eal_common_thread.c | 1 + lib/librte_eal/include/rte_eal.h | 3 ++- lib/librte_eal/rte_eal_version.map | 7 +++++++ 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index a5f67d811c..280c64bb76 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -22,6 +22,7 @@ #include "eal_thread.h" RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; +RTE_DEFINE_PER_LCORE(int, _thread_id) = -1; static RTE_DEFINE_PER_LCORE(unsigned int, _socket_id) = (unsigned int)SOCKET_ID_ANY; static RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); diff --git a/lib/librte_eal/include/rte_eal.h b/lib/librte_eal/include/rte_eal.h index 2f9ed298de..2edf8c6556 100644 --- a/lib/librte_eal/include/rte_eal.h +++ b/lib/librte_eal/include/rte_eal.h @@ -447,6 +447,8 @@ enum rte_intr_mode rte_eal_vfio_intr_mode(void); */ int rte_sys_gettid(void); +RTE_DECLARE_PER_LCORE(int, _thread_id); + /** * Get system unique thread id. * @@ -456,7 +458,6 @@ int rte_sys_gettid(void); */ static inline int rte_gettid(void) { - static RTE_DEFINE_PER_LCORE(int, _thread_id) = -1; if (RTE_PER_LCORE(_thread_id) == -1) RTE_PER_LCORE(_thread_id) = rte_sys_gettid(); return RTE_PER_LCORE(_thread_id); diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 196eef5afa..0d42d44ce9 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -221,6 +221,13 @@ DPDK_20.0 { local: *; }; +DPDK_21 { + global: + + per_lcore__thread_id; + +} DPDK_20.0; + EXPERIMENTAL { global: From patchwork Mon Jun 22 13:25:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71971 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 664F3A0519; Mon, 22 Jun 2020 15:26:28 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DFC2B1D6EB; Mon, 22 Jun 2020 15:26:13 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id C7E131D6ED for ; Mon, 22 Jun 2020 15:26:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592832369; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9zTPi/8u5JOSyLh9kfSFIm6TQC9+SZQeMQ3QtXTKuEE=; b=EuvKVS3crK9HTCSKwMYp6bq7JjE6Ehq+MkZHpK9DWs+RgOm3N6x1snGjRVlddNZ/xbVC0J FhSV4MKQLbCIxUCErAsM014BApZ5LD5NmvvUd4vjnSZxKPrxcbuRqY7WiqBMQuvP0ZWHE4 QHg9tICMHaa2qRVDytUHBrFAPcS6olE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-229-p5OELwifPY64VdLBAQOT6g-1; Mon, 22 Jun 2020 09:26:05 -0400 X-MC-Unique: p5OELwifPY64VdLBAQOT6g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B8D3F800053; Mon, 22 Jun 2020 13:26:03 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 729F371671; Mon, 22 Jun 2020 13:26:00 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Harini Ramakrishnan , Omar Cardona , Pallavi Kadam , Ranjit Menon Date: Mon, 22 Jun 2020 15:25:25 +0200 Message-Id: <20200622132531.21857-4-david.marchand@redhat.com> In-Reply-To: <20200622132531.21857-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v3 3/9] eal: introduce thread init helper X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Introduce a helper responsible for initialising the per thread context. We can then have a unified context for EAL and non-EAL threads and remove copy/paste'd OS-specific helpers. Per EAL thread CPU affinity setting is separated from the thread init. It is to accommodate with Windows EAL where CPU affinity is not set at the moment. Besides, having affinity set by the master lcore in FreeBSD and Linux will make it possible to detect errors rather than panic in the child thread. But the cleanup when such an event happens is left for later. Signed-off-by: David Marchand --- Changes since v1: - rebased on master, removed Windows workarounds wrt gettid and traces support, --- lib/librte_eal/common/eal_common_thread.c | 51 +++++++++++++---------- lib/librte_eal/common/eal_thread.h | 8 ++-- lib/librte_eal/freebsd/eal.c | 14 ++++++- lib/librte_eal/freebsd/eal_thread.c | 32 +------------- lib/librte_eal/linux/eal.c | 15 ++++++- lib/librte_eal/linux/eal_thread.c | 32 +------------- lib/librte_eal/windows/eal.c | 3 +- lib/librte_eal/windows/eal_thread.c | 10 +---- 8 files changed, 66 insertions(+), 99 deletions(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 280c64bb76..afb30236c5 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -71,20 +71,10 @@ eal_cpuset_socket_id(rte_cpuset_t *cpusetp) return socket_id; } -int -rte_thread_set_affinity(rte_cpuset_t *cpusetp) +static void +thread_update_affinity(rte_cpuset_t *cpusetp) { - int s; - unsigned lcore_id; - pthread_t tid; - - tid = pthread_self(); - - s = pthread_setaffinity_np(tid, sizeof(rte_cpuset_t), cpusetp); - if (s != 0) { - RTE_LOG(ERR, EAL, "pthread_setaffinity_np failed\n"); - return -1; - } + unsigned int lcore_id = rte_lcore_id(); /* store socket_id in TLS for quick access */ RTE_PER_LCORE(_socket_id) = @@ -94,14 +84,24 @@ rte_thread_set_affinity(rte_cpuset_t *cpusetp) memmove(&RTE_PER_LCORE(_cpuset), cpusetp, sizeof(rte_cpuset_t)); - lcore_id = rte_lcore_id(); if (lcore_id != (unsigned)LCORE_ID_ANY) { /* EAL thread will update lcore_config */ lcore_config[lcore_id].socket_id = RTE_PER_LCORE(_socket_id); memmove(&lcore_config[lcore_id].cpuset, cpusetp, sizeof(rte_cpuset_t)); } +} +int +rte_thread_set_affinity(rte_cpuset_t *cpusetp) +{ + if (pthread_setaffinity_np(pthread_self(), sizeof(rte_cpuset_t), + cpusetp) != 0) { + RTE_LOG(ERR, EAL, "pthread_setaffinity_np failed\n"); + return -1; + } + + thread_update_affinity(cpusetp); return 0; } @@ -147,6 +147,19 @@ eal_thread_dump_affinity(char *str, unsigned size) return ret; } +void +rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset) +{ + /* set the lcore ID in per-lcore memory area */ + RTE_PER_LCORE(_lcore_id) = lcore_id; + + /* acquire system unique id */ + rte_gettid(); + + thread_update_affinity(cpuset); + + __rte_trace_mem_per_thread_alloc(); +} struct rte_thread_ctrl_params { void *(*start_routine)(void *); @@ -154,16 +167,14 @@ struct rte_thread_ctrl_params { pthread_barrier_t configured; }; -static void *rte_thread_init(void *arg) +static void *ctrl_thread_init(void *arg) { int ret; - rte_cpuset_t *cpuset = &internal_config.ctrl_cpuset; struct rte_thread_ctrl_params *params = arg; void *(*start_routine)(void *) = params->start_routine; void *routine_arg = params->arg; - /* Store cpuset in TLS for quick access */ - memmove(&RTE_PER_LCORE(_cpuset), cpuset, sizeof(rte_cpuset_t)); + rte_thread_init(rte_lcore_id(), &internal_config.ctrl_cpuset); ret = pthread_barrier_wait(¶ms->configured); if (ret == PTHREAD_BARRIER_SERIAL_THREAD) { @@ -171,8 +182,6 @@ static void *rte_thread_init(void *arg) free(params); } - __rte_trace_mem_per_thread_alloc(); - return start_routine(routine_arg); } @@ -194,7 +203,7 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, pthread_barrier_init(¶ms->configured, NULL, 2); - ret = pthread_create(thread, attr, rte_thread_init, (void *)params); + ret = pthread_create(thread, attr, ctrl_thread_init, (void *)params); if (ret != 0) { free(params); return -ret; diff --git a/lib/librte_eal/common/eal_thread.h b/lib/librte_eal/common/eal_thread.h index b40ed249ed..da5e7c93ba 100644 --- a/lib/librte_eal/common/eal_thread.h +++ b/lib/librte_eal/common/eal_thread.h @@ -16,12 +16,14 @@ __rte_noreturn void *eal_thread_loop(void *arg); /** - * Init per-lcore info for master thread + * Init per-lcore info in current thread. * * @param lcore_id - * identifier of master lcore + * identifier of lcore. + * @param cpuset + * CPU affinity for this thread. */ -void eal_thread_init_master(unsigned lcore_id); +void rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset); /** * Get the NUMA socket id from cpu id. diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index c41f265fac..b5ea11df16 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -877,7 +877,14 @@ rte_eal_init(int argc, char **argv) eal_check_mem_on_local_socket(); - eal_thread_init_master(rte_config.master_lcore); + if (pthread_setaffinity_np(pthread_self(), sizeof(rte_cpuset_t), + &lcore_config[rte_config.master_lcore].cpuset) != 0) { + rte_eal_init_alert("Cannot set affinity"); + rte_errno = EINVAL; + return -1; + } + rte_thread_init(rte_config.master_lcore, + &lcore_config[rte_config.master_lcore].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); @@ -908,6 +915,11 @@ rte_eal_init(int argc, char **argv) snprintf(thread_name, sizeof(thread_name), "lcore-slave-%d", i); rte_thread_setname(lcore_config[i].thread_id, thread_name); + + ret = pthread_setaffinity_np(lcore_config[i].thread_id, + sizeof(rte_cpuset_t), &lcore_config[i].cpuset); + if (ret != 0) + rte_panic("Cannot set affinity\n"); } /* diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index 40676d9ef5..c1fb8eb2d8 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -66,29 +66,6 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned slave_id) return rc; } -/* set affinity for current thread */ -static int -eal_thread_set_affinity(void) -{ - unsigned lcore_id = rte_lcore_id(); - - /* acquire system unique id */ - rte_gettid(); - - /* update EAL thread core affinity */ - return rte_thread_set_affinity(&lcore_config[lcore_id].cpuset); -} - -void eal_thread_init_master(unsigned lcore_id) -{ - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); -} - /* main loop of threads */ __rte_noreturn void * eal_thread_loop(__rte_unused void *arg) @@ -113,19 +90,12 @@ eal_thread_loop(__rte_unused void *arg) m2s = lcore_config[lcore_id].pipe_master2slave[0]; s2m = lcore_config[lcore_id].pipe_slave2master[1]; - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); + rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%p;cpuset=[%s%s])\n", lcore_id, thread_id, cpuset, ret == 0 ? "" : "..."); - __rte_trace_mem_per_thread_alloc(); rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); /* read on our pipe to get commands */ diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index f162124a37..8638376b8a 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -1205,10 +1205,16 @@ rte_eal_init(int argc, char **argv) eal_check_mem_on_local_socket(); - eal_thread_init_master(rte_config.master_lcore); + if (pthread_setaffinity_np(pthread_self(), sizeof(rte_cpuset_t), + &lcore_config[rte_config.master_lcore].cpuset) != 0) { + rte_eal_init_alert("Cannot set affinity"); + rte_errno = EINVAL; + return -1; + } + rte_thread_init(rte_config.master_lcore, + &lcore_config[rte_config.master_lcore].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "Master lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", rte_config.master_lcore, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); @@ -1240,6 +1246,11 @@ rte_eal_init(int argc, char **argv) if (ret != 0) RTE_LOG(DEBUG, EAL, "Cannot set name for lcore thread\n"); + + ret = pthread_setaffinity_np(lcore_config[i].thread_id, + sizeof(rte_cpuset_t), &lcore_config[i].cpuset); + if (ret != 0) + rte_panic("Cannot set affinity\n"); } /* diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index a52ebef3a4..07aec0c44d 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -66,29 +66,6 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned slave_id) return rc; } -/* set affinity for current EAL thread */ -static int -eal_thread_set_affinity(void) -{ - unsigned lcore_id = rte_lcore_id(); - - /* acquire system unique id */ - rte_gettid(); - - /* update EAL thread core affinity */ - return rte_thread_set_affinity(&lcore_config[lcore_id].cpuset); -} - -void eal_thread_init_master(unsigned lcore_id) -{ - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); -} - /* main loop of threads */ __rte_noreturn void * eal_thread_loop(__rte_unused void *arg) @@ -113,19 +90,12 @@ eal_thread_loop(__rte_unused void *arg) m2s = lcore_config[lcore_id].pipe_master2slave[0]; s2m = lcore_config[lcore_id].pipe_slave2master[1]; - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); + rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", lcore_id, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); - __rte_trace_mem_per_thread_alloc(); rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); /* read on our pipe to get commands */ diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c index 427a5557fa..23de12ab43 100644 --- a/lib/librte_eal/windows/eal.c +++ b/lib/librte_eal/windows/eal.c @@ -354,7 +354,8 @@ rte_eal_init(int argc, char **argv) return -1; } - eal_thread_init_master(rte_config.master_lcore); + rte_thread_init(rte_config.master_lcore, + &lcore_config[rte_config.master_lcore].cpuset); RTE_LCORE_FOREACH_SLAVE(i) { diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index f12a2ec6ad..4f01881240 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -53,13 +53,6 @@ rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int slave_id) return 0; } -void -eal_thread_init_master(unsigned int lcore_id) -{ - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; -} - /* main loop of threads */ void * eal_thread_loop(void *arg __rte_unused) @@ -84,8 +77,7 @@ eal_thread_loop(void *arg __rte_unused) m2s = lcore_config[lcore_id].pipe_master2slave[0]; s2m = lcore_config[lcore_id].pipe_slave2master[1]; - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; + rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s])\n", lcore_id, (uintptr_t)thread_id, cpuset); From patchwork Mon Jun 22 13:25:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71972 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A8B30A0519; Mon, 22 Jun 2020 15:26:39 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 32C031D6F5; Mon, 22 Jun 2020 15:26:20 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id 419221D6F3 for ; Mon, 22 Jun 2020 15:26:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592832377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L4FVbKl+mNFqCHWWG9PMea7bpMTOYeVpRTpHtolkjVY=; b=gkSZe3r1/74XL1ugtp6pFqrmsSKOulkVZpZxtRdh/1ojKA5H6OKujuyBiSQ/yD1PJmC3Ct T7xw/iVC4KMMdt+1AgauMRg0GiLe45sYVcC31Yrxmn0GD7ogmoRPsuuNVRzR0eSKjmJ0yu gfruPhLJmG6txn74Q2Z/CHaKetSx/10= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-74-5IcKKVN3PgWxt4jm_S2zgA-1; Mon, 22 Jun 2020 09:26:13 -0400 X-MC-Unique: 5IcKKVN3PgWxt4jm_S2zgA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B612A107ACCA; Mon, 22 Jun 2020 13:26:11 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA09C1DC; Mon, 22 Jun 2020 13:26:04 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Jerin Jacob , Sunil Kumar Kori , Neil Horman , Harini Ramakrishnan , Omar Cardona , Pallavi Kadam , Ranjit Menon Date: Mon, 22 Jun 2020 15:25:26 +0200 Message-Id: <20200622132531.21857-5-david.marchand@redhat.com> In-Reply-To: <20200622132531.21857-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v3 4/9] eal: introduce thread uninit helper X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is a preparation step for dynamically unregistering threads. Since we explicitly allocate a per thread trace buffer in rte_thread_init, add an internal helper to free this buffer. Signed-off-by: David Marchand --- Note: I preferred renaming the current internal function to free all threads trace buffers (new name trace_mem_free()) and reuse the previous name (trace_mem_per_thread_free()) when freeing this buffer for a given thread. Changes since v2: - added missing stub for windows tracing support, - moved free symbol to exported (experimental) ABI as a counterpart of the alloc symbol we already had, Changes since v1: - rebased on master, removed Windows workaround wrt traces support, --- lib/librte_eal/common/eal_common_thread.c | 9 ++++ lib/librte_eal/common/eal_common_trace.c | 51 +++++++++++++++++++---- lib/librte_eal/common/eal_thread.h | 5 +++ lib/librte_eal/common/eal_trace.h | 2 +- lib/librte_eal/include/rte_trace_point.h | 9 ++++ lib/librte_eal/rte_eal_version.map | 3 ++ lib/librte_eal/windows/eal.c | 5 +++ 7 files changed, 75 insertions(+), 9 deletions(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index afb30236c5..3b30cc99d9 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -20,6 +20,7 @@ #include "eal_internal_cfg.h" #include "eal_private.h" #include "eal_thread.h" +#include "eal_trace.h" RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; RTE_DEFINE_PER_LCORE(int, _thread_id) = -1; @@ -161,6 +162,14 @@ rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset) __rte_trace_mem_per_thread_alloc(); } +void +rte_thread_uninit(void) +{ + __rte_trace_mem_per_thread_free(); + + RTE_PER_LCORE(_lcore_id) = LCORE_ID_ANY; +} + struct rte_thread_ctrl_params { void *(*start_routine)(void *); void *arg; diff --git a/lib/librte_eal/common/eal_common_trace.c b/lib/librte_eal/common/eal_common_trace.c index 875553d7e5..3e620d76ed 100644 --- a/lib/librte_eal/common/eal_common_trace.c +++ b/lib/librte_eal/common/eal_common_trace.c @@ -101,7 +101,7 @@ eal_trace_fini(void) { if (!rte_trace_is_enabled()) return; - trace_mem_per_thread_free(); + trace_mem_free(); trace_metadata_destroy(); eal_trace_args_free(); } @@ -370,24 +370,59 @@ __rte_trace_mem_per_thread_alloc(void) rte_spinlock_unlock(&trace->lock); } +static void +trace_mem_per_thread_free_unlocked(struct thread_mem_meta *meta) +{ + if (meta->area == TRACE_AREA_HUGEPAGE) + eal_free_no_trace(meta->mem); + else if (meta->area == TRACE_AREA_HEAP) + free(meta->mem); +} + +void +__rte_trace_mem_per_thread_free(void) +{ + struct trace *trace = trace_obj_get(); + struct __rte_trace_header *header; + uint32_t count; + + if (RTE_PER_LCORE(trace_mem) == NULL) + return; + + header = RTE_PER_LCORE(trace_mem); + rte_spinlock_lock(&trace->lock); + for (count = 0; count < trace->nb_trace_mem_list; count++) { + if (trace->lcore_meta[count].mem == header) + break; + } + if (count != trace->nb_trace_mem_list) { + struct thread_mem_meta *meta = &trace->lcore_meta[count]; + + trace_mem_per_thread_free_unlocked(meta); + if (count != trace->nb_trace_mem_list - 1) { + memmove(meta, meta + 1, + sizeof(*meta) * + (trace->nb_trace_mem_list - count - 1)); + } + trace->nb_trace_mem_list--; + } + rte_spinlock_unlock(&trace->lock); +} + void -trace_mem_per_thread_free(void) +trace_mem_free(void) { struct trace *trace = trace_obj_get(); uint32_t count; - void *mem; if (!rte_trace_is_enabled()) return; rte_spinlock_lock(&trace->lock); for (count = 0; count < trace->nb_trace_mem_list; count++) { - mem = trace->lcore_meta[count].mem; - if (trace->lcore_meta[count].area == TRACE_AREA_HUGEPAGE) - eal_free_no_trace(mem); - else if (trace->lcore_meta[count].area == TRACE_AREA_HEAP) - free(mem); + trace_mem_per_thread_free_unlocked(&trace->lcore_meta[count]); } + trace->nb_trace_mem_list = 0; rte_spinlock_unlock(&trace->lock); } diff --git a/lib/librte_eal/common/eal_thread.h b/lib/librte_eal/common/eal_thread.h index da5e7c93ba..4ecd8fd53a 100644 --- a/lib/librte_eal/common/eal_thread.h +++ b/lib/librte_eal/common/eal_thread.h @@ -25,6 +25,11 @@ __rte_noreturn void *eal_thread_loop(void *arg); */ void rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset); +/** + * Uninitialize per-lcore info for current thread. + */ +void rte_thread_uninit(void); + /** * Get the NUMA socket id from cpu id. * This function is private to EAL. diff --git a/lib/librte_eal/common/eal_trace.h b/lib/librte_eal/common/eal_trace.h index 8f60616156..bbb6e1645c 100644 --- a/lib/librte_eal/common/eal_trace.h +++ b/lib/librte_eal/common/eal_trace.h @@ -106,7 +106,7 @@ int trace_metadata_create(void); void trace_metadata_destroy(void); int trace_mkdir(void); int trace_epoch_time_save(void); -void trace_mem_per_thread_free(void); +void trace_mem_free(void); /* EAL interface */ int eal_trace_init(void); diff --git a/lib/librte_eal/include/rte_trace_point.h b/lib/librte_eal/include/rte_trace_point.h index 377c2414aa..686b86fdb1 100644 --- a/lib/librte_eal/include/rte_trace_point.h +++ b/lib/librte_eal/include/rte_trace_point.h @@ -230,6 +230,15 @@ __rte_trace_point_fp_is_enabled(void) __rte_experimental void __rte_trace_mem_per_thread_alloc(void); +/** + * @internal + * + * Free trace memory buffer per thread. + * + */ +__rte_experimental +void __rte_trace_mem_per_thread_free(void); + /** * @internal * diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 0d42d44ce9..5831eea4b0 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -393,6 +393,9 @@ EXPERIMENTAL { rte_trace_point_lookup; rte_trace_regexp; rte_trace_save; + + # added in 20.08 + __rte_trace_mem_per_thread_free; }; INTERNAL { diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c index 23de12ab43..09cc1aef63 100644 --- a/lib/librte_eal/windows/eal.c +++ b/lib/librte_eal/windows/eal.c @@ -255,6 +255,11 @@ __rte_trace_mem_per_thread_alloc(void) { } +void +__rte_trace_mem_per_thread_free(void) +{ +} + void __rte_trace_point_emit_field(size_t sz, const char *field, const char *type) From patchwork Mon Jun 22 13:25:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71973 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id AC343A0519; Mon, 22 Jun 2020 15:26:51 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1F82E1D702; Mon, 22 Jun 2020 15:26:24 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id 242CC1D6F4 for ; Mon, 22 Jun 2020 15:26:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592832378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BXucOJ5qnjOdHBrJ5591+N7T6Lo29P0IPNPkyRfk2p8=; b=PK3+XNokKKzcC2QaPXs7mqRvHoraXa1adrkKiF+zOUzy79T2iM/fPEMt8GNRfAUvZE/hvx meN+c0cWCf8Un6HWU1gfLoJLW6WpVSUQ5dl7Ht7Oc/HGabZ89IymZh76KU/Q9E25Flqv78 izfhRlwCwK2UBhrBw+SkecaZuADrMas= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-480-8UU8Ul64PXmjkrvoL9V1pw-1; Mon, 22 Jun 2020 09:26:16 -0400 X-MC-Unique: 8UU8Ul64PXmjkrvoL9V1pw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8305C835B40; Mon, 22 Jun 2020 13:26:15 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id B680D1DC; Mon, 22 Jun 2020 13:26:12 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org Date: Mon, 22 Jun 2020 15:25:27 +0200 Message-Id: <20200622132531.21857-6-david.marchand@redhat.com> In-Reply-To: <20200622132531.21857-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v3 5/9] eal: move lcore role code X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" For consistency sake, move all lcore role code in the dedicated compilation unit / header. Signed-off-by: David Marchand --- lib/librte_eal/common/eal_common_lcore.c | 11 +++++++ lib/librte_eal/common/eal_common_thread.c | 11 ------- lib/librte_eal/include/rte_eal.h | 9 ------ lib/librte_eal/include/rte_lcore.h | 37 ++++++++++++++--------- 4 files changed, 34 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 5404922a87..86d32a3dd7 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -63,6 +63,17 @@ rte_eal_lcore_role(unsigned int lcore_id) return cfg->lcore_role[lcore_id]; } +int +rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + + if (lcore_id >= RTE_MAX_LCORE) + return -EINVAL; + + return cfg->lcore_role[lcore_id] == role; +} + int rte_lcore_is_enabled(unsigned int lcore_id) { struct rte_config *cfg = rte_eal_get_configuration(); diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 3b30cc99d9..a7ae0691bf 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -33,17 +33,6 @@ unsigned rte_socket_id(void) return RTE_PER_LCORE(_socket_id); } -int -rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role) -{ - struct rte_config *cfg = rte_eal_get_configuration(); - - if (lcore_id >= RTE_MAX_LCORE) - return -EINVAL; - - return cfg->lcore_role[lcore_id] == role; -} - static int eal_cpuset_socket_id(rte_cpuset_t *cpusetp) { diff --git a/lib/librte_eal/include/rte_eal.h b/lib/librte_eal/include/rte_eal.h index 2edf8c6556..0913d1947c 100644 --- a/lib/librte_eal/include/rte_eal.h +++ b/lib/librte_eal/include/rte_eal.h @@ -31,15 +31,6 @@ extern "C" { /* Maximum thread_name length. */ #define RTE_MAX_THREAD_NAME_LEN 16 -/** - * The lcore role (used in RTE or not). - */ -enum rte_lcore_role_t { - ROLE_RTE, - ROLE_OFF, - ROLE_SERVICE, -}; - /** * The type of process in a linux, multi-process setup */ diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 5c1d1926e9..3968c40693 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -24,6 +24,15 @@ extern "C" { RTE_DECLARE_PER_LCORE(unsigned, _lcore_id); /**< Per thread "lcore id". */ +/** + * The lcore role (used in RTE or not). + */ +enum rte_lcore_role_t { + ROLE_RTE, + ROLE_OFF, + ROLE_SERVICE, +}; + /** * Get a lcore's role. * @@ -34,6 +43,20 @@ RTE_DECLARE_PER_LCORE(unsigned, _lcore_id); /**< Per thread "lcore id". */ */ enum rte_lcore_role_t rte_eal_lcore_role(unsigned int lcore_id); +/** + * Test if the core supplied has a specific role + * + * @param lcore_id + * The identifier of the lcore, which MUST be between 0 and + * RTE_MAX_LCORE-1. + * @param role + * The role to be checked against. + * @return + * Boolean value: positive if test is true; otherwise returns 0. + */ +int +rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role); + /** * Return the Application thread ID of the execution unit. * @@ -283,20 +306,6 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg); -/** - * Test if the core supplied has a specific role - * - * @param lcore_id - * The identifier of the lcore, which MUST be between 0 and - * RTE_MAX_LCORE-1. - * @param role - * The role to be checked against. - * @return - * Boolean value: positive if test is true; otherwise returns 0. - */ -int -rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role); - #ifdef __cplusplus } #endif From patchwork Mon Jun 22 13:25:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71974 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 57093A0519; Mon, 22 Jun 2020 15:27:00 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 792621D6D8; Mon, 22 Jun 2020 15:26:28 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id 3C3AC1D6D8 for ; Mon, 22 Jun 2020 15:26:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592832386; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ER6LVMDIisMPu2vKUZjsUZpoPOS9HWQyQfl5x4D9SjY=; b=esmootTRU5VAb591I1Sl6idd0OrxIbS9GQm3jWP8rX07/f7b2p+eoSmVOiHT92cOlWXwKn ljbiOka2Gd3wRVZLY7IJD5dqJqhtKLq3MOL8EzfsnwThvCpYMwG5GBXMfGQCNmjVTdgPIw /ktUJtjWOtBwxn9suH1NFYV/CCfNQxY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-242-5g8P1Oh4P9qo3KeHZywroQ-1; Mon, 22 Jun 2020 09:26:22 -0400 X-MC-Unique: 5g8P1Oh4P9qo3KeHZywroQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A338710059A0; Mon, 22 Jun 2020 13:26:20 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1F7F41DC; Mon, 22 Jun 2020 13:26:15 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Thomas Monjalon , John McNamara , Marko Kovacevic , Anatoly Burakov , Olivier Matz , Andrew Rybchenko , Neil Horman Date: Mon, 22 Jun 2020 15:25:28 +0200 Message-Id: <20200622132531.21857-7-david.marchand@redhat.com> In-Reply-To: <20200622132531.21857-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v3 6/9] eal: register non-EAL threads as lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" DPDK allows calling some part of its API from a non-EAL thread but this has some limitations. OVS (and other applications) has its own thread management but still want to avoid such limitations by hacking RTE_PER_LCORE(_lcore_id) and faking EAL threads potentially unknown of some DPDK component. Introduce a new API to register non-EAL thread and associate them to a free lcore with a new NON_EAL role. This role denotes lcores that do not run DPDK mainloop and as such prevents use of rte_eal_wait_lcore() and consorts. Signed-off-by: David Marchand Acked-by: Andrew Rybchenko Acked-by: Thomas Monjalon --- Changes since v1: - moved cleanup on lcore role code in patch 5, - added unit test, - updated documentation, - changed naming from "external thread" to "registered non-EAL thread" --- MAINTAINERS | 1 + app/test/Makefile | 1 + app/test/autotest_data.py | 6 + app/test/meson.build | 2 + app/test/test_lcores.c | 139 ++++++++++++++++++ doc/guides/howto/debug_troubleshoot.rst | 5 +- .../prog_guide/env_abstraction_layer.rst | 22 +-- doc/guides/prog_guide/mempool_lib.rst | 2 +- lib/librte_eal/common/eal_common_lcore.c | 44 +++++- lib/librte_eal/common/eal_common_thread.c | 33 +++++ lib/librte_eal/common/eal_private.h | 18 +++ lib/librte_eal/include/rte_lcore.h | 18 ++- lib/librte_eal/rte_eal_version.map | 2 + lib/librte_mempool/rte_mempool.h | 11 +- 14 files changed, 282 insertions(+), 22 deletions(-) create mode 100644 app/test/test_lcores.c diff --git a/MAINTAINERS b/MAINTAINERS index 816696caf2..fe9e74ffbc 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -183,6 +183,7 @@ F: app/test/test_debug.c F: app/test/test_eal* F: app/test/test_errno.c F: app/test/test_interrupts.c +F: app/test/test_lcores.c F: app/test/test_logs.c F: app/test/test_memcpy* F: app/test/test_per_lcore.c diff --git a/app/test/Makefile b/app/test/Makefile index 7b96a03a64..4a8dea2425 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -97,6 +97,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += test_flow_classify.c endif SRCS-y += test_rwlock.c +SRCS-y += test_lcores.c SRCS-$(CONFIG_RTE_LIBRTE_STACK) += test_stack.c SRCS-$(CONFIG_RTE_LIBRTE_STACK) += test_stack_perf.c diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py index fc3fcc159e..600b130966 100644 --- a/app/test/autotest_data.py +++ b/app/test/autotest_data.py @@ -62,6 +62,12 @@ "Func": rwlock_autotest, "Report": None, }, + { + "Name": "Lcores autotest", + "Command": "lcores_autotest", + "Func": default_autotest, + "Report": None, + }, { "Name": "Logs autotest", "Command": "logs_autotest", diff --git a/app/test/meson.build b/app/test/meson.build index 5233ead46e..a57477b7cc 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -67,6 +67,7 @@ test_sources = files('commands.c', 'test_ipsec_perf.c', 'test_kni.c', 'test_kvargs.c', + 'test_lcores.c', 'test_logs.c', 'test_lpm.c', 'test_lpm6.c', @@ -206,6 +207,7 @@ fast_tests = [ ['hash_autotest', true], ['interrupt_autotest', true], ['ipfrag_autotest', false], + ['lcores_autotest', true], ['logs_autotest', true], ['lpm_autotest', true], ['lpm6_autotest', true], diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c new file mode 100644 index 0000000000..864bcbade7 --- /dev/null +++ b/app/test/test_lcores.c @@ -0,0 +1,139 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2020 Red Hat, Inc. + */ + +#include +#include + +#include + +#include "test.h" + +struct thread_context { + enum { INIT, ERROR, DONE } state; + bool lcore_id_any; + pthread_t id; + unsigned int *registered_count; +}; +static void *thread_loop(void *arg) +{ + struct thread_context *t = arg; + unsigned int lcore_id; + + lcore_id = rte_lcore_id(); + if (lcore_id != LCORE_ID_ANY) { + printf("Incorrect lcore id for new thread %u\n", lcore_id); + t->state = ERROR; + } + rte_thread_register(); + lcore_id = rte_lcore_id(); + if ((t->lcore_id_any && lcore_id != LCORE_ID_ANY) || + (!t->lcore_id_any && lcore_id == LCORE_ID_ANY)) { + printf("Could not register new thread, got %u while %sexpecting %u\n", + lcore_id, t->lcore_id_any ? "" : "not ", LCORE_ID_ANY); + t->state = ERROR; + } + /* Report register happened to the control thread. */ + __atomic_add_fetch(t->registered_count, 1, __ATOMIC_RELEASE); + + /* Wait for release from the control thread. */ + while (__atomic_load_n(t->registered_count, __ATOMIC_ACQUIRE) != 0) + ; + rte_thread_unregister(); + lcore_id = rte_lcore_id(); + if (lcore_id != LCORE_ID_ANY) { + printf("Could not unregister new thread, %u still assigned\n", + lcore_id); + t->state = ERROR; + } + + if (t->state != ERROR) + t->state = DONE; + + return NULL; +} + +static int +test_non_eal_lcores(unsigned int eal_threads_count) +{ + struct thread_context thread_contexts[RTE_MAX_LCORE]; + unsigned int non_eal_threads_count; + unsigned int registered_count; + struct thread_context *t; + unsigned int i; + int ret; + + non_eal_threads_count = 0; + registered_count = 0; + + /* Try to create as many threads as possible. */ + for (i = 0; i < RTE_MAX_LCORE - eal_threads_count; i++) { + t = &thread_contexts[i]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = false; + if (pthread_create(&t->id, NULL, thread_loop, t) != 0) + break; + non_eal_threads_count++; + } + printf("non-EAL threads count: %u\n", non_eal_threads_count); + /* Wait all non-EAL threads to register. */ + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + + /* We managed to create the max number of threads, let's try to create + * one more. This will allow one more check. + */ + if (eal_threads_count + non_eal_threads_count < RTE_MAX_LCORE) + goto skip_lcore_any; + t = &thread_contexts[non_eal_threads_count]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = true; + if (pthread_create(&t->id, NULL, thread_loop, t) == 0) { + non_eal_threads_count++; + printf("non-EAL threads count: %u\n", non_eal_threads_count); + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + } + +skip_lcore_any: + /* Release all threads, and check their states. */ + __atomic_store_n(®istered_count, 0, __ATOMIC_RELEASE); + ret = 0; + for (i = 0; i < non_eal_threads_count; i++) { + t = &thread_contexts[i]; + pthread_join(t->id, NULL); + if (t->state != DONE) + ret = -1; + } + + return ret; +} + +static int +test_lcores(void) +{ + unsigned int eal_threads_count = 0; + unsigned int i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + if (!rte_lcore_has_role(i, ROLE_OFF)) + eal_threads_count++; + } + if (eal_threads_count == 0) { + printf("Something is broken, no EAL thread detected.\n"); + return TEST_FAILED; + } + printf("EAL threads count: %u, RTE_MAX_LCORE=%u\n", eal_threads_count, + RTE_MAX_LCORE); + + if (test_non_eal_lcores(eal_threads_count) < 0) + return TEST_FAILED; + + return TEST_SUCCESS; +} + +REGISTER_TEST_COMMAND(lcores_autotest, test_lcores); diff --git a/doc/guides/howto/debug_troubleshoot.rst b/doc/guides/howto/debug_troubleshoot.rst index cef016b2fe..5a46f5fba3 100644 --- a/doc/guides/howto/debug_troubleshoot.rst +++ b/doc/guides/howto/debug_troubleshoot.rst @@ -307,8 +307,9 @@ Custom worker function :numref:`dtg_distributor_worker`. #. Configuration issue isolation - * Identify core role using ``rte_eal_lcore_role`` to identify RTE, OFF and - SERVICE. Check performance functions are mapped to run on the cores. + * Identify core role using ``rte_eal_lcore_role`` to identify RTE, OFF, + SERVICE and NON_EAL. Check performance functions are mapped to run on the + cores. * For high-performance execution logic ensure running it on correct NUMA and non-master core. diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 48a2fec066..f64ae953d1 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -564,9 +564,13 @@ It's also compatible with the pattern of corelist('-l') option. non-EAL pthread support ~~~~~~~~~~~~~~~~~~~~~~~ -It is possible to use the DPDK execution context with any user pthread (aka. Non-EAL pthreads). -In a non-EAL pthread, the *_lcore_id* is always LCORE_ID_ANY which identifies that it is not an EAL thread with a valid, unique, *_lcore_id*. -Some libraries will use an alternative unique ID (e.g. TID), some will not be impacted at all, and some will work but with limitations (e.g. timer and mempool libraries). +It is possible to use the DPDK execution context with any user pthread (aka. non-EAL pthreads). +There are two kinds of non-EAL pthreads: + +- a registered non-EAL pthread with a valid *_lcore_id* that was successfully assigned by calling ``rte_thread_register()``, +- a non registered non-EAL pthread with a LCORE_ID_ANY, + +For non registered non-EAL pthread (with a LCORE_ID_ANY *_lcore_id*), some libraries will use an alternative unique ID (e.g. TID), some will not be impacted at all, and some will work but with limitations (e.g. timer and mempool libraries). All these impacts are mentioned in :ref:`known_issue_label` section. @@ -613,9 +617,9 @@ Known Issues + rte_mempool The rte_mempool uses a per-lcore cache inside the mempool. - For non-EAL pthreads, ``rte_lcore_id()`` will not return a valid number. - So for now, when rte_mempool is used with non-EAL pthreads, the put/get operations will bypass the default mempool cache and there is a performance penalty because of this bypass. - Only user-owned external caches can be used in a non-EAL context in conjunction with ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()`` that accept an explicit cache parameter. + For unregistered non-EAL pthreads, ``rte_lcore_id()`` will not return a valid number. + So for now, when rte_mempool is used with unregistered non-EAL pthreads, the put/get operations will bypass the default mempool cache and there is a performance penalty because of this bypass. + Only user-owned external caches can be used in an unregistered non-EAL context in conjunction with ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()`` that accept an explicit cache parameter. + rte_ring @@ -660,15 +664,15 @@ Known Issues + rte_timer - Running ``rte_timer_manage()`` on a non-EAL pthread is not allowed. However, resetting/stopping the timer from a non-EAL pthread is allowed. + Running ``rte_timer_manage()`` on an unregistered non-EAL pthread is not allowed. However, resetting/stopping the timer from a non-EAL pthread is allowed. + rte_log - In non-EAL pthreads, there is no per thread loglevel and logtype, global loglevels are used. + In unregistered non-EAL pthreads, there is no per thread loglevel and logtype, global loglevels are used. + misc - The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in a non-EAL pthread. + The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread. cgroup control ~~~~~~~~~~~~~~ diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst index f8b430d656..e3e1f940be 100644 --- a/doc/guides/prog_guide/mempool_lib.rst +++ b/doc/guides/prog_guide/mempool_lib.rst @@ -103,7 +103,7 @@ The maximum size of the cache is static and is defined at compilation time (CONF Alternatively to the internal default per-lcore local cache, an application can create and manage external caches through the ``rte_mempool_cache_create()``, ``rte_mempool_cache_free()`` and ``rte_mempool_cache_flush()`` calls. These user-owned caches can be explicitly passed to ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()``. The ``rte_mempool_default_cache()`` call returns the default internal cache if any. -In contrast to the default caches, user-owned caches can be used by non-EAL threads too. +In contrast to the default caches, user-owned caches can be used by unregistered non-EAL threads too. Mempool Handlers ------------------------ diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 86d32a3dd7..7db05428e7 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -6,12 +6,13 @@ #include #include -#include -#include -#include -#include #include #include +#include +#include +#include +#include +#include #include "eal_private.h" #include "eal_thread.h" @@ -220,3 +221,38 @@ rte_socket_id_by_idx(unsigned int idx) } return config->numa_nodes[idx]; } + +static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; + +unsigned int +eal_lcore_non_eal_allocate(void) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + unsigned int lcore_id; + + rte_spinlock_lock(&lcore_lock); + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] != ROLE_OFF) + continue; + cfg->lcore_role[lcore_id] = ROLE_NON_EAL; + cfg->lcore_count++; + break; + } + if (lcore_id == RTE_MAX_LCORE) + RTE_LOG(DEBUG, EAL, "No lcore available.\n"); + rte_spinlock_unlock(&lcore_lock); + return lcore_id; +} + +void +eal_lcore_non_eal_release(unsigned int lcore_id) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + + rte_spinlock_lock(&lcore_lock); + if (cfg->lcore_role[lcore_id] == ROLE_NON_EAL) { + cfg->lcore_role[lcore_id] = ROLE_OFF; + cfg->lcore_count--; + } + rte_spinlock_unlock(&lcore_lock); +} diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index a7ae0691bf..1cbddc4b5b 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -236,3 +236,36 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, pthread_join(*thread, NULL); return -ret; } + +void +rte_thread_register(void) +{ + unsigned int lcore_id; + rte_cpuset_t cpuset; + + /* EAL init flushes all lcores, we can't register before. */ + assert(internal_config.init_complete == 1); + if (pthread_getaffinity_np(pthread_self(), sizeof(cpuset), + &cpuset) != 0) + CPU_ZERO(&cpuset); + lcore_id = eal_lcore_non_eal_allocate(); + if (lcore_id >= RTE_MAX_LCORE) + lcore_id = LCORE_ID_ANY; + rte_thread_init(lcore_id, &cpuset); + if (lcore_id != LCORE_ID_ANY) + RTE_LOG(DEBUG, EAL, "Registered non-EAL thread as lcore %u.\n", + lcore_id); +} + +void +rte_thread_unregister(void) +{ + unsigned int lcore_id = rte_lcore_id(); + + if (lcore_id != LCORE_ID_ANY) + eal_lcore_non_eal_release(lcore_id); + rte_thread_uninit(); + if (lcore_id != LCORE_ID_ANY) + RTE_LOG(DEBUG, EAL, "Unregistered non-EAL thread (was lcore %u).\n", + lcore_id); +} diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 0592fcd694..73238ff157 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -396,6 +396,24 @@ uint64_t get_tsc_freq(void); */ uint64_t get_tsc_freq_arch(void); +/** + * Allocate a free lcore to associate to a non-EAL thread. + * + * @return + * - the id of a lcore with role ROLE_NON_EAL on success. + * - RTE_MAX_LCORE if none was available. + */ +unsigned int eal_lcore_non_eal_allocate(void); + +/** + * Release the lcore used by a non-EAL thread. + * Counterpart of eal_lcore_non_eal_allocate(). + * + * @param lcore_id + * The lcore with role ROLE_NON_EAL to release. + */ +void eal_lcore_non_eal_release(unsigned int lcore_id); + /** * Prepare physical memory mapping * i.e. hugepages on Linux and diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 3968c40693..ea86220394 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -31,6 +31,7 @@ enum rte_lcore_role_t { ROLE_RTE, ROLE_OFF, ROLE_SERVICE, + ROLE_NON_EAL, }; /** @@ -67,7 +68,8 @@ rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role); * to run threads with lcore IDs 0, 1, 2 and 3 on physical core 10.. * * @return - * Logical core ID (in EAL thread) or LCORE_ID_ANY (in non-EAL thread) + * Logical core ID (in EAL thread or registered non-EAL thread) or + * LCORE_ID_ANY (in unregistered non-EAL thread) */ static inline unsigned rte_lcore_id(void) @@ -279,6 +281,20 @@ int rte_thread_setname(pthread_t id, const char *name); __rte_experimental int rte_thread_getname(pthread_t id, char *name, size_t len); +/** + * Register current non-EAL thread as a lcore. + */ +__rte_experimental +void +rte_thread_register(void); + +/** + * Unregister current thread and release lcore if one was associated. + */ +__rte_experimental +void +rte_thread_unregister(void); + /** * Create a control thread. * diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 5831eea4b0..39c41d445d 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -396,6 +396,8 @@ EXPERIMENTAL { # added in 20.08 __rte_trace_mem_per_thread_free; + rte_thread_register; + rte_thread_unregister; }; INTERNAL { diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h index 652d19f9f1..9e0ee052b3 100644 --- a/lib/librte_mempool/rte_mempool.h +++ b/lib/librte_mempool/rte_mempool.h @@ -28,9 +28,9 @@ * rte_mempool_get() or rte_mempool_put() are designed to be called from an EAL * thread due to the internal per-lcore cache. Due to the lack of caching, * rte_mempool_get() or rte_mempool_put() performance will suffer when called - * by non-EAL threads. Instead, non-EAL threads should call - * rte_mempool_generic_get() or rte_mempool_generic_put() with a user cache - * created with rte_mempool_cache_create(). + * by unregistered non-EAL threads. Instead, unregistered non-EAL threads + * should call rte_mempool_generic_get() or rte_mempool_generic_put() with a + * user cache created with rte_mempool_cache_create(). */ #include @@ -1233,7 +1233,7 @@ void rte_mempool_dump(FILE *f, struct rte_mempool *mp); /** * Create a user-owned mempool cache. * - * This can be used by non-EAL threads to enable caching when they + * This can be used by unregistered non-EAL threads to enable caching when they * interact with a mempool. * * @param size @@ -1264,7 +1264,8 @@ rte_mempool_cache_free(struct rte_mempool_cache *cache); * @param lcore_id * The logical core id. * @return - * A pointer to the mempool cache or NULL if disabled or non-EAL thread. + * A pointer to the mempool cache or NULL if disabled or unregistered non-EAL + * thread. */ static __rte_always_inline struct rte_mempool_cache * rte_mempool_default_cache(struct rte_mempool *mp, unsigned lcore_id) From patchwork Mon Jun 22 13:25:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71976 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 132E7A0519; Mon, 22 Jun 2020 15:27:21 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AA95D1D6FF; Mon, 22 Jun 2020 15:26:38 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id C874F1D712 for ; Mon, 22 Jun 2020 15:26:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592832396; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=81uM3e/oSpVn5Pa75qNu1oZLta8CzbIVgnijzAH6PaE=; b=SGPU7y+5ilAWZGdh71U0zNq+YEqxHzxqAu6xFHl8ENrBwL1lV992aRTasyZlE9eEIG+mlp E8PDoUb5tcuv3+6pnCQSHOLNty3A7OcuvEtS8uKY9aXuf/ACHAjMiDU8OHWLt6KOYJhVv1 rvprKlaXaXED6jyIq4XngHSsdfdTE+4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-141-qGHvsgUTO36i9fx_Cg0kqg-1; Mon, 22 Jun 2020 09:26:28 -0400 X-MC-Unique: qGHvsgUTO36i9fx_Cg0kqg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C664E8018AB; Mon, 22 Jun 2020 13:26:26 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6F1F671685; Mon, 22 Jun 2020 13:26:21 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Neil Horman Date: Mon, 22 Jun 2020 15:25:29 +0200 Message-Id: <20200622132531.21857-8-david.marchand@redhat.com> In-Reply-To: <20200622132531.21857-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v3 7/9] eal: add lcore init callbacks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" DPDK components and applications can have their say when a new lcore is initialized. For this, they can register a callback for initializing and releasing their private data. Signed-off-by: David Marchand --- Changes since v2: - added missing test, - fixed rollback on lcore register, Changes since v1: - added unit test (since missing some coverage, for v3), - preferred callback and removed mention of notification, --- app/test/test_lcores.c | 230 +++++++++++++++++++++++ lib/librte_eal/common/eal_common_lcore.c | 138 +++++++++++++- lib/librte_eal/common/eal_private.h | 3 +- lib/librte_eal/include/rte_lcore.h | 68 +++++++ lib/librte_eal/rte_eal_version.map | 2 + 5 files changed, 435 insertions(+), 6 deletions(-) diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c index 864bcbade7..e36dceedf9 100644 --- a/app/test/test_lcores.c +++ b/app/test/test_lcores.c @@ -5,6 +5,7 @@ #include #include +#include #include #include "test.h" @@ -113,6 +114,229 @@ test_non_eal_lcores(unsigned int eal_threads_count) return ret; } +struct limit_lcore_context { + unsigned int init; + unsigned int max; + unsigned int uninit; +}; +static int +limit_lcores_init(unsigned int lcore_id __rte_unused, void *arg) +{ + struct limit_lcore_context *l = arg; + + l->init++; + if (l->init > l->max) + return -1; + return 0; +} +static void +limit_lcores_uninit(unsigned int lcore_id __rte_unused, void *arg) +{ + struct limit_lcore_context *l = arg; + + l->uninit++; +} + +static int +test_lcores_callback(unsigned int eal_threads_count) +{ + struct limit_lcore_context l; + void *handle; + + /* Refuse last lcore => callback register error. */ + memset(&l, 0, sizeof(l)); + l.max = eal_threads_count - 1; + handle = rte_lcore_callback_register("limit", limit_lcores_init, + limit_lcores_uninit, &l); + if (handle != NULL) { + printf("lcore callback register should have failed\n"); + goto error; + } + /* Refusal happens at the n th call to the init callback. + * Besides, n - 1 were accepted, so we expect as many uninit calls when + * the rollback happens. + */ + if (l.init != eal_threads_count) { + printf("lcore callback register failed but incorrect init calls, expected %u, got %u\n", + eal_threads_count, l.init); + goto error; + } + if (l.uninit != eal_threads_count - 1) { + printf("lcore callback register failed but incorrect uninit calls, expected %u, got %u\n", + eal_threads_count - 1, l.uninit); + goto error; + } + + /* Accept all lcore and unregister. */ + memset(&l, 0, sizeof(l)); + l.max = eal_threads_count; + handle = rte_lcore_callback_register("limit", limit_lcores_init, + limit_lcores_uninit, &l); + if (handle == NULL) { + printf("lcore callback register failed\n"); + goto error; + } + if (l.uninit != 0) { + printf("lcore callback register succeeded but incorrect uninit calls, expected 0, got %u\n", + l.uninit); + goto error; + } + rte_lcore_callback_unregister(handle); + handle = NULL; + if (l.init != eal_threads_count) { + printf("lcore callback unregister done but incorrect init calls, expected %u, got %u\n", + eal_threads_count, l.init); + goto error; + } + if (l.uninit != eal_threads_count) { + printf("lcore callback unregister done but incorrect uninit calls, expected %u, got %u\n", + eal_threads_count, l.uninit); + goto error; + } + + return 0; + +error: + if (handle != NULL) + rte_lcore_callback_unregister(handle); + + return -1; +} + +static int +test_non_eal_lcores_callback(unsigned int eal_threads_count) +{ + struct thread_context thread_contexts[2]; + unsigned int non_eal_threads_count; + struct limit_lcore_context l[2]; + unsigned int registered_count; + struct thread_context *t; + void *handle[2]; + unsigned int i; + int ret; + + memset(l, 0, sizeof(l)); + handle[0] = handle[1] = NULL; + non_eal_threads_count = 0; + registered_count = 0; + + /* This test requires two empty slots to be sure lcore init refusal is + * because of callback execution. + */ + if (eal_threads_count + 2 >= RTE_MAX_LCORE) + return 0; + + /* Register two callbacks: + * - first one accepts any lcore, + * - second one accepts all EAL lcore + one more for the first non-EAL + * thread, then refuses the next lcore. + */ + l[0].max = UINT_MAX; + handle[0] = rte_lcore_callback_register("no_limit", limit_lcores_init, + limit_lcores_uninit, &l[0]); + if (handle[0] == NULL) { + printf("lcore callback [0] register failed\n"); + goto error; + } + l[1].max = eal_threads_count + 1; + handle[1] = rte_lcore_callback_register("limit", limit_lcores_init, + limit_lcores_uninit, &l[1]); + if (handle[1] == NULL) { + printf("lcore callback [1] register failed\n"); + goto error; + } + if (l[0].init != eal_threads_count || l[1].init != eal_threads_count) { + printf("lcore callbacks register succeeded but incorrect init calls, expected %u, %u, got %u, %u\n", + eal_threads_count, eal_threads_count, + l[0].init, l[1].init); + goto error; + } + if (l[0].uninit != 0 || l[1].uninit != 0) { + printf("lcore callbacks register succeeded but incorrect uninit calls, expected 0, 1, got %u, %u\n", + l[0].uninit, l[1].uninit); + goto error; + } + /* First thread that expects a valid lcore id. */ + t = &thread_contexts[0]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = false; + if (pthread_create(&t->id, NULL, thread_loop, t) != 0) + goto cleanup_threads; + non_eal_threads_count++; + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + if (l[0].init != eal_threads_count + 1 || + l[1].init != eal_threads_count + 1) { + printf("Incorrect init calls, expected %u, %u, got %u, %u\n", + eal_threads_count + 1, eal_threads_count + 1, + l[0].init, l[1].init); + goto cleanup_threads; + } + if (l[0].uninit != 0 || l[1].uninit != 0) { + printf("Incorrect uninit calls, expected 0, 0, got %u, %u\n", + l[0].uninit, l[1].uninit); + goto cleanup_threads; + } + /* Second thread, that expects LCORE_ID_ANY because of init refusal. */ + t = &thread_contexts[1]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = true; + if (pthread_create(&t->id, NULL, thread_loop, t) != 0) + goto cleanup_threads; + non_eal_threads_count++; + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + if (l[0].init != eal_threads_count + 2 || + l[1].init != eal_threads_count + 2) { + printf("Incorrect init calls, expected %u, %u, got %u, %u\n", + eal_threads_count + 2, eal_threads_count + 2, + l[0].init, l[1].init); + goto cleanup_threads; + } + if (l[0].uninit != 1 || l[1].uninit != 0) { + printf("Incorrect uninit calls, expected 1, 0, got %u, %u\n", + l[0].uninit, l[1].uninit); + goto cleanup_threads; + } + /* Release all threads, and check their states. */ + __atomic_store_n(®istered_count, 0, __ATOMIC_RELEASE); + ret = 0; + for (i = 0; i < non_eal_threads_count; i++) { + t = &thread_contexts[i]; + pthread_join(t->id, NULL); + if (t->state != DONE) + ret = -1; + } + if (ret < 0) + goto error; + if (l[0].uninit != 2 || l[1].uninit != 1) { + printf("Threads reported having successfully registered and unregistered, but incorrect uninit calls, expected 2, 1, got %u, %u\n", + l[0].uninit, l[1].uninit); + goto error; + } + rte_lcore_callback_unregister(handle[0]); + rte_lcore_callback_unregister(handle[1]); + return 0; + +cleanup_threads: + /* Release all threads */ + __atomic_store_n(®istered_count, 0, __ATOMIC_RELEASE); + for (i = 0; i < non_eal_threads_count; i++) { + t = &thread_contexts[i]; + pthread_join(t->id, NULL); + } +error: + if (handle[1] != NULL) + rte_lcore_callback_unregister(handle[1]); + if (handle[0] != NULL) + rte_lcore_callback_unregister(handle[0]); + return -1; +} + static int test_lcores(void) { @@ -133,6 +357,12 @@ test_lcores(void) if (test_non_eal_lcores(eal_threads_count) < 0) return TEST_FAILED; + if (test_lcores_callback(eal_threads_count) < 0) + return TEST_FAILED; + + if (test_non_eal_lcores_callback(eal_threads_count) < 0) + return TEST_FAILED; + return TEST_SUCCESS; } diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 7db05428e7..3737a75ece 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -223,11 +223,114 @@ rte_socket_id_by_idx(unsigned int idx) } static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; +struct lcore_callback { + TAILQ_ENTRY(lcore_callback) next; + char *name; + rte_lcore_init_cb init; + rte_lcore_uninit_cb uninit; + void *arg; +}; +static TAILQ_HEAD(lcore_callbacks_head, lcore_callback) lcore_callbacks = + TAILQ_HEAD_INITIALIZER(lcore_callbacks); + +static int +callback_init(struct lcore_callback *callback, unsigned int lcore_id) +{ + if (callback->init == NULL) + return 0; + RTE_LOG(DEBUG, EAL, "Call init for lcore callback %s, lcore_id %u\n", + callback->name, lcore_id); + return callback->init(lcore_id, callback->arg); +} + +static void +callback_uninit(struct lcore_callback *callback, unsigned int lcore_id) +{ + if (callback->uninit == NULL) + return; + RTE_LOG(DEBUG, EAL, "Call uninit for lcore callback %s, lcore_id %u\n", + callback->name, lcore_id); + callback->uninit(lcore_id, callback->arg); +} + +void * +rte_lcore_callback_register(const char *name, rte_lcore_init_cb init, + rte_lcore_uninit_cb uninit, void *arg) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback; + unsigned int lcore_id; + + callback = calloc(1, sizeof(*callback)); + if (callback == NULL) + return NULL; + if (asprintf(&callback->name, "%s-%p", name, arg) == -1) { + free(callback); + return NULL; + } + callback->init = init; + callback->uninit = uninit; + callback->arg = arg; + rte_spinlock_lock(&lcore_lock); + if (callback->init == NULL) + goto no_init; + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] == ROLE_OFF) + continue; + if (callback_init(callback, lcore_id) == 0) + continue; + /* Callback refused init for this lcore, uninitialize all + * previous lcore. + */ + for (; lcore_id != 0; lcore_id--) { + if (cfg->lcore_role[lcore_id - 1] == ROLE_OFF) + continue; + callback_uninit(callback, lcore_id - 1); + } + free(callback); + callback = NULL; + goto out; + } +no_init: + TAILQ_INSERT_TAIL(&lcore_callbacks, callback, next); + RTE_LOG(DEBUG, EAL, "Registered new lcore callback %s (%sinit, %suninit).\n", + callback->name, callback->init == NULL ? "NO " : "", + callback->uninit == NULL ? "NO " : ""); +out: + rte_spinlock_unlock(&lcore_lock); + return callback; +} + +void +rte_lcore_callback_unregister(void *handle) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback = handle; + unsigned int lcore_id; + + rte_spinlock_lock(&lcore_lock); + if (callback->uninit == NULL) + goto no_uninit; + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] == ROLE_OFF) + continue; + callback_uninit(callback, lcore_id); + } +no_uninit: + TAILQ_REMOVE(&lcore_callbacks, callback, next); + rte_spinlock_unlock(&lcore_lock); + RTE_LOG(DEBUG, EAL, "Unregistered lcore callback %s-%p.\n", + callback->name, callback->arg); + free(callback->name); + free(callback); +} unsigned int eal_lcore_non_eal_allocate(void) { struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback; + struct lcore_callback *prev; unsigned int lcore_id; rte_spinlock_lock(&lcore_lock); @@ -238,8 +341,29 @@ eal_lcore_non_eal_allocate(void) cfg->lcore_count++; break; } - if (lcore_id == RTE_MAX_LCORE) + if (lcore_id == RTE_MAX_LCORE) { RTE_LOG(DEBUG, EAL, "No lcore available.\n"); + goto out; + } + TAILQ_FOREACH(callback, &lcore_callbacks, next) { + if (callback_init(callback, lcore_id) == 0) + continue; + /* Callback refused init for this lcore, call uninit for all + * previous callbacks. + */ + prev = TAILQ_PREV(callback, lcore_callbacks_head, next); + while (prev != NULL) { + callback_uninit(prev, lcore_id); + prev = TAILQ_PREV(prev, lcore_callbacks_head, next); + } + RTE_LOG(DEBUG, EAL, "Initialization refused for lcore %u.\n", + lcore_id); + cfg->lcore_role[lcore_id] = ROLE_OFF; + cfg->lcore_count--; + lcore_id = RTE_MAX_LCORE; + goto out; + } +out: rte_spinlock_unlock(&lcore_lock); return lcore_id; } @@ -248,11 +372,15 @@ void eal_lcore_non_eal_release(unsigned int lcore_id) { struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback; rte_spinlock_lock(&lcore_lock); - if (cfg->lcore_role[lcore_id] == ROLE_NON_EAL) { - cfg->lcore_role[lcore_id] = ROLE_OFF; - cfg->lcore_count--; - } + if (cfg->lcore_role[lcore_id] != ROLE_NON_EAL) + goto out; + TAILQ_FOREACH(callback, &lcore_callbacks, next) + callback_uninit(callback, lcore_id); + cfg->lcore_role[lcore_id] = ROLE_OFF; + cfg->lcore_count--; +out: rte_spinlock_unlock(&lcore_lock); } diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 73238ff157..fc79564111 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -401,7 +401,8 @@ uint64_t get_tsc_freq_arch(void); * * @return * - the id of a lcore with role ROLE_NON_EAL on success. - * - RTE_MAX_LCORE if none was available. + * - RTE_MAX_LCORE if none was available or initializing was refused (see + * rte_lcore_callback_register). */ unsigned int eal_lcore_non_eal_allocate(void); diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index ea86220394..27b29a1f87 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -229,6 +229,74 @@ unsigned int rte_get_next_lcore(unsigned int i, int skip_master, int wrap); i X-Patchwork-Id: 71975 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18B33A0519; Mon, 22 Jun 2020 15:27:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1816D1D701; Mon, 22 Jun 2020 15:26:36 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by dpdk.org (Postfix) with ESMTP id 409721D6F4 for ; Mon, 22 Jun 2020 15:26:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592832393; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K0WrpxKsIha1tr/zyuu0rpgB1yysq/9Je9UDFTWtfhE=; b=aCRhc/Yq6+KuoCwE1OI1WzgSZY20crcWuD4tdAmKTEx4EP1iONwkA5jggc07MvidTaKXnM TQx1FRCSa52o50FexRisJNWfmdXECK9HjL3EB9ejSgPnSJ4Ke5VsRRwjE3tTEy+I7spvBu 3PTNm1qiRl/SEp6RMQQWi3HLeWBE5fI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-498-WpJ0iQq_N1WwbLamzHrVMw-1; Mon, 22 Jun 2020 09:26:31 -0400 X-MC-Unique: WpJ0iQq_N1WwbLamzHrVMw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 903E510059A0; Mon, 22 Jun 2020 13:26:30 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 89B311DC; Mon, 22 Jun 2020 13:26:27 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Neil Horman Date: Mon, 22 Jun 2020 15:25:30 +0200 Message-Id: <20200622132531.21857-9-david.marchand@redhat.com> In-Reply-To: <20200622132531.21857-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v3 8/9] eal: add lcore iterators X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a helper to iterate all lcores. The iterator callback is read-only wrt the lcores list. Implement a dump function on top of this for debugging. Signed-off-by: David Marchand --- Changes since v2: - added rte_lcore_dump calls in unit test, for basic check, Changes since v1: - introduced lcore iterators and implemented rte_lcore_dump, this iterator mechanism can then be used outside of EAL, --- app/test/test_lcores.c | 3 + lib/librte_eal/common/eal_common_lcore.c | 77 ++++++++++++++++++++--- lib/librte_eal/common/eal_common_thread.c | 16 +++-- lib/librte_eal/common/eal_thread.h | 13 +++- lib/librte_eal/freebsd/eal.c | 2 +- lib/librte_eal/freebsd/eal_thread.c | 2 +- lib/librte_eal/include/rte_lcore.h | 47 +++++++++++++- lib/librte_eal/linux/eal.c | 2 +- lib/librte_eal/linux/eal_thread.c | 2 +- lib/librte_eal/rte_eal_version.map | 2 + 10 files changed, 143 insertions(+), 23 deletions(-) diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c index e36dceedf9..dd6fa466c8 100644 --- a/app/test/test_lcores.c +++ b/app/test/test_lcores.c @@ -302,6 +302,7 @@ test_non_eal_lcores_callback(unsigned int eal_threads_count) l[0].uninit, l[1].uninit); goto cleanup_threads; } + rte_lcore_dump(stdout); /* Release all threads, and check their states. */ __atomic_store_n(®istered_count, 0, __ATOMIC_RELEASE); ret = 0; @@ -313,6 +314,7 @@ test_non_eal_lcores_callback(unsigned int eal_threads_count) } if (ret < 0) goto error; + rte_lcore_dump(stdout); if (l[0].uninit != 2 || l[1].uninit != 1) { printf("Threads reported having successfully registered and unregistered, but incorrect uninit calls, expected 2, 1, got %u, %u\n", l[0].uninit, l[1].uninit); @@ -353,6 +355,7 @@ test_lcores(void) } printf("EAL threads count: %u, RTE_MAX_LCORE=%u\n", eal_threads_count, RTE_MAX_LCORE); + rte_lcore_dump(stdout); if (test_non_eal_lcores(eal_threads_count) < 0) return TEST_FAILED; diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 3737a75ece..5a54b18fd6 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -12,7 +12,7 @@ #include #include #include -#include +#include #include "eal_private.h" #include "eal_thread.h" @@ -222,7 +222,7 @@ rte_socket_id_by_idx(unsigned int idx) return config->numa_nodes[idx]; } -static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; +static rte_rwlock_t lcore_lock = RTE_RWLOCK_INITIALIZER; struct lcore_callback { TAILQ_ENTRY(lcore_callback) next; char *name; @@ -271,7 +271,7 @@ rte_lcore_callback_register(const char *name, rte_lcore_init_cb init, callback->init = init; callback->uninit = uninit; callback->arg = arg; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); if (callback->init == NULL) goto no_init; for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { @@ -297,7 +297,7 @@ rte_lcore_callback_register(const char *name, rte_lcore_init_cb init, callback->name, callback->init == NULL ? "NO " : "", callback->uninit == NULL ? "NO " : ""); out: - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); return callback; } @@ -308,7 +308,7 @@ rte_lcore_callback_unregister(void *handle) struct lcore_callback *callback = handle; unsigned int lcore_id; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); if (callback->uninit == NULL) goto no_uninit; for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { @@ -318,7 +318,7 @@ rte_lcore_callback_unregister(void *handle) } no_uninit: TAILQ_REMOVE(&lcore_callbacks, callback, next); - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); RTE_LOG(DEBUG, EAL, "Unregistered lcore callback %s-%p.\n", callback->name, callback->arg); free(callback->name); @@ -333,7 +333,7 @@ eal_lcore_non_eal_allocate(void) struct lcore_callback *prev; unsigned int lcore_id; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { if (cfg->lcore_role[lcore_id] != ROLE_OFF) continue; @@ -364,7 +364,7 @@ eal_lcore_non_eal_allocate(void) goto out; } out: - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); return lcore_id; } @@ -374,7 +374,7 @@ eal_lcore_non_eal_release(unsigned int lcore_id) struct rte_config *cfg = rte_eal_get_configuration(); struct lcore_callback *callback; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); if (cfg->lcore_role[lcore_id] != ROLE_NON_EAL) goto out; TAILQ_FOREACH(callback, &lcore_callbacks, next) @@ -382,5 +382,62 @@ eal_lcore_non_eal_release(unsigned int lcore_id) cfg->lcore_role[lcore_id] = ROLE_OFF; cfg->lcore_count--; out: - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); +} + +int +rte_lcore_iterate(rte_lcore_iterate_cb cb, void *arg) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + unsigned int lcore_id; + int ret = 0; + + rte_rwlock_read_lock(&lcore_lock); + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] == ROLE_OFF) + continue; + ret = cb(lcore_id, arg); + if (ret != 0) + break; + } + rte_rwlock_read_unlock(&lcore_lock); + return ret; +} + +static int +lcore_dump_cb(unsigned int lcore_id, void *arg) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + char cpuset[RTE_CPU_AFFINITY_STR_LEN]; + const char *role; + FILE *f = arg; + int ret; + + switch (cfg->lcore_role[lcore_id]) { + case ROLE_RTE: + role = "RTE"; + break; + case ROLE_SERVICE: + role = "SERVICE"; + break; + case ROLE_NON_EAL: + role = "NON_EAL"; + break; + default: + role = "UNKNOWN"; + break; + } + + ret = eal_thread_dump_affinity(&lcore_config[lcore_id].cpuset, cpuset, + sizeof(cpuset)); + fprintf(f, "lcore %u, socket %u, role %s, cpuset %s%s\n", lcore_id, + rte_lcore_to_socket_id(lcore_id), role, cpuset, + ret == 0 ? "" : "..."); + return 0; +} + +void +rte_lcore_dump(FILE *f) +{ + rte_lcore_iterate(lcore_dump_cb, f); } diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 1cbddc4b5b..e00374b06f 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -104,17 +104,14 @@ rte_thread_get_affinity(rte_cpuset_t *cpusetp) } int -eal_thread_dump_affinity(char *str, unsigned size) +eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size) { - rte_cpuset_t cpuset; unsigned cpu; int ret; unsigned int out = 0; - rte_thread_get_affinity(&cpuset); - for (cpu = 0; cpu < CPU_SETSIZE; cpu++) { - if (!CPU_ISSET(cpu, &cpuset)) + if (!CPU_ISSET(cpu, cpuset)) continue; ret = snprintf(str + out, @@ -137,6 +134,15 @@ eal_thread_dump_affinity(char *str, unsigned size) return ret; } +int +eal_thread_dump_current_affinity(char *str, unsigned int size) +{ + rte_cpuset_t cpuset; + + rte_thread_get_affinity(&cpuset); + return eal_thread_dump_affinity(&cpuset, str, size); +} + void rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset) { diff --git a/lib/librte_eal/common/eal_thread.h b/lib/librte_eal/common/eal_thread.h index 4ecd8fd53a..13ec252e01 100644 --- a/lib/librte_eal/common/eal_thread.h +++ b/lib/librte_eal/common/eal_thread.h @@ -47,13 +47,15 @@ unsigned eal_cpu_socket_id(unsigned cpu_id); #define RTE_CPU_AFFINITY_STR_LEN 256 /** - * Dump the current pthread cpuset. + * Dump the cpuset as a human readable string. * This function is private to EAL. * * Note: * If the dump size is greater than the size of given buffer, * the string will be truncated and with '\0' at the end. * + * @param cpuset + * The CPU affinity object to dump. * @param str * The string buffer the cpuset will dump to. * @param size @@ -62,6 +64,13 @@ unsigned eal_cpu_socket_id(unsigned cpu_id); * 0 for success, -1 if truncation happens. */ int -eal_thread_dump_affinity(char *str, unsigned size); +eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size); + +/** + * Dump the current thread cpuset. + * This is a wrapper on eal_thread_dump_affinity(). + */ +int +eal_thread_dump_current_affinity(char *str, unsigned int size); #endif /* EAL_THREAD_H */ diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index b5ea11df16..69a6f7d8c4 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -886,7 +886,7 @@ rte_eal_init(int argc, char **argv) rte_thread_init(rte_config.master_lcore, &lcore_config[rte_config.master_lcore].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "Master lcore %u is ready (tid=%p;cpuset=[%s%s])\n", rte_config.master_lcore, thread_id, cpuset, diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index c1fb8eb2d8..b1a3619f51 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -92,7 +92,7 @@ eal_thread_loop(__rte_unused void *arg) rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%p;cpuset=[%s%s])\n", lcore_id, thread_id, cpuset, ret == 0 ? "" : "..."); diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 27b29a1f87..4dee7cbcd7 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -261,8 +261,8 @@ typedef void (*rte_lcore_uninit_cb)(unsigned int lcore_id, void *arg); * If this step succeeds, the callbacks are put in the lcore callbacks list * that will get called for each lcore allocation/release. * - * Note: callbacks execution is serialised under a lock protecting the lcores - * and callbacks list. + * Note: callbacks execution is serialised under a write lock protecting the + * lcores and callbacks list. * * @param name * A name serving as a small description for this callback. @@ -297,6 +297,49 @@ __rte_experimental void rte_lcore_callback_unregister(void *handle); +/** + * Callback prototype for iterating over lcores. + * + * @param lcore_id + * The lcore to consider. + * @param arg + * An opaque pointer coming from the caller. + * @return + * - 0 lets the iteration continue. + * - !0 makes the iteration stop. + */ +typedef int (*rte_lcore_iterate_cb)(unsigned int lcore_id, void *arg); + +/** + * Iterate on all active lcores (ROLE_RTE, ROLE_SERVICE and ROLE_NON_EAL). + * No modification on the lcore states is allowed in the callback. + * + * Note: as opposed to init/uninit callbacks, iteration callbacks can be + * invoked in parallel as they are run under a read lock protecting the lcores + * and callbacks list. + * + * @param cb + * The callback that gets passed each lcore. + * @param arg + * An opaque pointer passed to cb. + * @return + * Same return code as the callback last invocation (see rte_lcore_iterate_cb + * description). + */ +__rte_experimental +int +rte_lcore_iterate(rte_lcore_iterate_cb cb, void *arg); + +/** + * List all lcores. + * + * @param f + * The output stream where the dump should be sent. + */ +__rte_experimental +void +rte_lcore_dump(FILE *f); + /** * Set core affinity of the current thread. * Support both EAL and non-EAL thread and update TLS. diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 8638376b8a..2f0efd7cd3 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -1214,7 +1214,7 @@ rte_eal_init(int argc, char **argv) rte_thread_init(rte_config.master_lcore, &lcore_config[rte_config.master_lcore].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "Master lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", rte_config.master_lcore, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index 07aec0c44d..22d9bc8c01 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -92,7 +92,7 @@ eal_thread_loop(__rte_unused void *arg) rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", lcore_id, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index aeee7cf431..23a1565dfb 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -398,6 +398,8 @@ EXPERIMENTAL { __rte_trace_mem_per_thread_free; rte_lcore_callback_register; rte_lcore_callback_unregister; + rte_lcore_dump; + rte_lcore_iterate; rte_thread_register; rte_thread_unregister; }; From patchwork Mon Jun 22 13:25:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71977 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 71EBFA0519; Mon, 22 Jun 2020 15:27:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0EC651D6F2; Mon, 22 Jun 2020 15:26:42 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 9C7791D71A for ; Mon, 22 Jun 2020 15:26:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592832400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bEYsvCW+vyIix+YFDyxULkm2zyme4u/RiWUnIA/e5lI=; b=R57o9t9sA42XgYOX2/kuMnlcrsMVhw6Wl254bZ1mn/sCaYTIrwe8H3kvpA9yAvBk1XBmZv xJIsY20eBgEQM1dD4HCV0DZjmhgBbDJdXsq98V4X+iEyKdRnwEp4SpEdZ5c3yV684lo2QT IJXWQgrIMNKurxZ5mso4Yb0fy6vHGbg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-287-iWw3yGGsOBq-5Ir72UNv4A-1; Mon, 22 Jun 2020 09:26:36 -0400 X-MC-Unique: iWw3yGGsOBq-5Ir72UNv4A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D94101B18BD1; Mon, 22 Jun 2020 13:26:34 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.229]) by smtp.corp.redhat.com (Postfix) with ESMTP id 795A21DC; Mon, 22 Jun 2020 13:26:31 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, "Artem V. Andreev" , Andrew Rybchenko Date: Mon, 22 Jun 2020 15:25:31 +0200 Message-Id: <20200622132531.21857-10-david.marchand@redhat.com> In-Reply-To: <20200622132531.21857-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v3 9/9] mempool/bucket: handle non-EAL lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Convert to new lcore API to support non-EAL lcores. Signed-off-by: David Marchand --- drivers/mempool/bucket/rte_mempool_bucket.c | 131 ++++++++++++-------- 1 file changed, 82 insertions(+), 49 deletions(-) diff --git a/drivers/mempool/bucket/rte_mempool_bucket.c b/drivers/mempool/bucket/rte_mempool_bucket.c index 5ce1ef16fb..0b4f42d330 100644 --- a/drivers/mempool/bucket/rte_mempool_bucket.c +++ b/drivers/mempool/bucket/rte_mempool_bucket.c @@ -55,6 +55,7 @@ struct bucket_data { struct rte_ring *shared_orphan_ring; struct rte_mempool *pool; unsigned int bucket_mem_size; + void *lcore_callback_handle; }; static struct bucket_stack * @@ -345,6 +346,22 @@ bucket_dequeue_contig_blocks(struct rte_mempool *mp, void **first_obj_table, return 0; } +struct bucket_per_lcore_ctx { + const struct bucket_data *bd; + unsigned int count; +}; + +static int +count_per_lcore(unsigned int lcore_id, void *arg) +{ + struct bucket_per_lcore_ctx *ctx = arg; + + ctx->count += ctx->bd->obj_per_bucket * + ctx->bd->buckets[lcore_id]->top; + ctx->count += rte_ring_count(ctx->bd->adoption_buffer_rings[lcore_id]); + return 0; +} + static void count_underfilled_buckets(struct rte_mempool *mp, void *opaque, @@ -373,23 +390,66 @@ count_underfilled_buckets(struct rte_mempool *mp, static unsigned int bucket_get_count(const struct rte_mempool *mp) { - const struct bucket_data *bd = mp->pool_data; - unsigned int count = - bd->obj_per_bucket * rte_ring_count(bd->shared_bucket_ring) + - rte_ring_count(bd->shared_orphan_ring); - unsigned int i; + struct bucket_per_lcore_ctx ctx; - for (i = 0; i < RTE_MAX_LCORE; i++) { - if (!rte_lcore_is_enabled(i)) - continue; - count += bd->obj_per_bucket * bd->buckets[i]->top + - rte_ring_count(bd->adoption_buffer_rings[i]); - } + ctx.bd = mp->pool_data; + ctx.count = ctx.bd->obj_per_bucket * + rte_ring_count(ctx.bd->shared_bucket_ring); + ctx.count += rte_ring_count(ctx.bd->shared_orphan_ring); + rte_lcore_iterate(count_per_lcore, &ctx); rte_mempool_mem_iter((struct rte_mempool *)(uintptr_t)mp, - count_underfilled_buckets, &count); + count_underfilled_buckets, &ctx.count); + + return ctx.count; +} + +static int +bucket_init_per_lcore(unsigned int lcore_id, void *arg) +{ + char rg_name[RTE_RING_NAMESIZE]; + struct bucket_data *bd = arg; + struct rte_mempool *mp; + int rg_flags; + int rc; + + mp = bd->pool; + bd->buckets[lcore_id] = bucket_stack_create(mp, + mp->size / bd->obj_per_bucket); + if (bd->buckets[lcore_id] == NULL) + goto error; + + rc = snprintf(rg_name, sizeof(rg_name), RTE_MEMPOOL_MZ_FORMAT ".a%u", + mp->name, lcore_id); + if (rc < 0 || rc >= (int)sizeof(rg_name)) + goto error; + + rg_flags = RING_F_SC_DEQ; + if (mp->flags & MEMPOOL_F_SP_PUT) + rg_flags |= RING_F_SP_ENQ; + if (mp->flags & MEMPOOL_F_SC_GET) + rg_flags |= RING_F_SC_DEQ; + bd->adoption_buffer_rings[lcore_id] = rte_ring_create(rg_name, + rte_align32pow2(mp->size + 1), mp->socket_id, rg_flags); + if (bd->adoption_buffer_rings[lcore_id] == NULL) + goto error; - return count; + return 0; +error: + rte_free(bd->buckets[lcore_id]); + bd->buckets[lcore_id] = NULL; + return -1; +} + +static void +bucket_uninit_per_lcore(unsigned int lcore_id, void *arg) +{ + struct bucket_data *bd = arg; + + rte_ring_free(bd->adoption_buffer_rings[lcore_id]); + bd->adoption_buffer_rings[lcore_id] = NULL; + rte_free(bd->buckets[lcore_id]); + bd->buckets[lcore_id] = NULL; } static int @@ -399,7 +459,6 @@ bucket_alloc(struct rte_mempool *mp) int rc = 0; char rg_name[RTE_RING_NAMESIZE]; struct bucket_data *bd; - unsigned int i; unsigned int bucket_header_size; size_t pg_sz; @@ -429,36 +488,17 @@ bucket_alloc(struct rte_mempool *mp) /* eventually this should be a tunable parameter */ bd->bucket_stack_thresh = (mp->size / bd->obj_per_bucket) * 4 / 3; + bd->lcore_callback_handle = rte_lcore_callback_register("bucket", + bucket_init_per_lcore, bucket_uninit_per_lcore, bd); + if (bd->lcore_callback_handle == NULL) { + rc = -ENOMEM; + goto no_mem_for_stacks; + } + if (mp->flags & MEMPOOL_F_SP_PUT) rg_flags |= RING_F_SP_ENQ; if (mp->flags & MEMPOOL_F_SC_GET) rg_flags |= RING_F_SC_DEQ; - - for (i = 0; i < RTE_MAX_LCORE; i++) { - if (!rte_lcore_is_enabled(i)) - continue; - bd->buckets[i] = - bucket_stack_create(mp, mp->size / bd->obj_per_bucket); - if (bd->buckets[i] == NULL) { - rc = -ENOMEM; - goto no_mem_for_stacks; - } - rc = snprintf(rg_name, sizeof(rg_name), - RTE_MEMPOOL_MZ_FORMAT ".a%u", mp->name, i); - if (rc < 0 || rc >= (int)sizeof(rg_name)) { - rc = -ENAMETOOLONG; - goto no_mem_for_stacks; - } - bd->adoption_buffer_rings[i] = - rte_ring_create(rg_name, rte_align32pow2(mp->size + 1), - mp->socket_id, - rg_flags | RING_F_SC_DEQ); - if (bd->adoption_buffer_rings[i] == NULL) { - rc = -rte_errno; - goto no_mem_for_stacks; - } - } - rc = snprintf(rg_name, sizeof(rg_name), RTE_MEMPOOL_MZ_FORMAT ".0", mp->name); if (rc < 0 || rc >= (int)sizeof(rg_name)) { @@ -498,11 +538,8 @@ bucket_alloc(struct rte_mempool *mp) rte_ring_free(bd->shared_orphan_ring); cannot_create_shared_orphan_ring: invalid_shared_orphan_ring: + rte_lcore_callback_unregister(bd->lcore_callback_handle); no_mem_for_stacks: - for (i = 0; i < RTE_MAX_LCORE; i++) { - rte_free(bd->buckets[i]); - rte_ring_free(bd->adoption_buffer_rings[i]); - } rte_free(bd); no_mem_for_data: rte_errno = -rc; @@ -512,16 +549,12 @@ bucket_alloc(struct rte_mempool *mp) static void bucket_free(struct rte_mempool *mp) { - unsigned int i; struct bucket_data *bd = mp->pool_data; if (bd == NULL) return; - for (i = 0; i < RTE_MAX_LCORE; i++) { - rte_free(bd->buckets[i]); - rte_ring_free(bd->adoption_buffer_rings[i]); - } + rte_lcore_callback_unregister(bd->lcore_callback_handle); rte_ring_free(bd->shared_orphan_ring); rte_ring_free(bd->shared_bucket_ring);