From patchwork Fri Jun 19 16:22:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71807 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0F9F7A051A; Fri, 19 Jun 2020 18:23:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D60411BF6C; Fri, 19 Jun 2020 18:23:09 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 4E3AD1BF5F for ; Fri, 19 Jun 2020 18:23:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583787; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tgAkQXWyU6ppia54pOxgztU/wTH6cJJnwTDZt98OIII=; b=XqEs3qlUG7ss5jBeQeslBy0dsqb3hrqMBC3ZMJD2OQOmUB4X88Js7BoS0t6yiC0KUSYsrV 0hllcBVsmjApfCbwtM7kSZdVWNFbyDlzCvMUb3MdG0kLzwH1RjtSvG2qP6I4t1xRbQtlMQ WDe7Vs31Kq8Fw+vrMEKMM6QrhswxeIg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-27-oz5LTZQgMlqn8_oNDf6PYw-1; Fri, 19 Jun 2020 12:23:03 -0400 X-MC-Unique: oz5LTZQgMlqn8_oNDf6PYw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 30AE718C6ED1; Fri, 19 Jun 2020 16:23:02 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id 31BD260BF4; Fri, 19 Jun 2020 16:22:57 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Harini Ramakrishnan , Omar Cardona , Pallavi Kadam , Ranjit Menon Date: Fri, 19 Jun 2020 18:22:36 +0200 Message-Id: <20200619162244.8239-2-david.marchand@redhat.com> In-Reply-To: <20200619162244.8239-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200619162244.8239-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 1/9] eal: relocate per thread symbols to common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" We have per lcore thread symbols scattered in OS implementations but common code relies on them. Move all of them in common. RTE_PER_LCORE(_socket_id) and RTE_PER_LCORE(_cpuset) have public accessors and are not exported through the library map, they can be made static. Signed-off-by: David Marchand --- lib/librte_eal/common/eal_common_thread.c | 5 ++++- lib/librte_eal/freebsd/eal_thread.c | 4 ---- lib/librte_eal/include/rte_lcore.h | 1 - lib/librte_eal/linux/eal_thread.c | 4 ---- lib/librte_eal/windows/eal_thread.c | 4 ---- 5 files changed, 4 insertions(+), 14 deletions(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 370bb1b634..a5f67d811c 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -21,7 +21,10 @@ #include "eal_private.h" #include "eal_thread.h" -RTE_DECLARE_PER_LCORE(unsigned , _socket_id); +RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; +static RTE_DEFINE_PER_LCORE(unsigned int, _socket_id) = + (unsigned int)SOCKET_ID_ANY; +static RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); unsigned rte_socket_id(void) { diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index b52019782a..40676d9ef5 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -25,10 +25,6 @@ #include "eal_private.h" #include "eal_thread.h" -RTE_DEFINE_PER_LCORE(unsigned, _lcore_id) = LCORE_ID_ANY; -RTE_DEFINE_PER_LCORE(unsigned, _socket_id) = (unsigned)SOCKET_ID_ANY; -RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); - /* * Send a message to a slave lcore identified by slave_id to call a * function f with argument arg. Once the execution is done, the diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 339046bc86..5c1d1926e9 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -23,7 +23,6 @@ extern "C" { #define LCORE_ID_ANY UINT32_MAX /**< Any lcore. */ RTE_DECLARE_PER_LCORE(unsigned, _lcore_id); /**< Per thread "lcore id". */ -RTE_DECLARE_PER_LCORE(rte_cpuset_t, _cpuset); /**< Per thread "cpuset". */ /** * Get a lcore's role. diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index cd9d6e0ebf..a52ebef3a4 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -25,10 +25,6 @@ #include "eal_private.h" #include "eal_thread.h" -RTE_DEFINE_PER_LCORE(unsigned, _lcore_id) = LCORE_ID_ANY; -RTE_DEFINE_PER_LCORE(unsigned, _socket_id) = (unsigned)SOCKET_ID_ANY; -RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); - /* * Send a message to a slave lcore identified by slave_id to call a * function f with argument arg. Once the execution is done, the diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index 3dd56519c9..f12a2ec6ad 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -16,10 +16,6 @@ #include "eal_private.h" #include "eal_windows.h" -RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; -RTE_DEFINE_PER_LCORE(unsigned int, _socket_id) = (unsigned int)SOCKET_ID_ANY; -RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); - /* * Send a message to a slave lcore identified by slave_id to call a * function f with argument arg. Once the execution is done, the From patchwork Fri Jun 19 16:22:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71808 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8A7AEA051A; Fri, 19 Jun 2020 18:23:22 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4A4411BF75; Fri, 19 Jun 2020 18:23:12 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 88EDF1BF74 for ; Fri, 19 Jun 2020 18:23:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583790; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J8bcJr1ym6XYlcLwbnSKM4jOhL8PmhczSwCFWrkuVeU=; b=ReHcnbJT/n9EZ3VK6lT3EBBqRZpA9/qUoDOumkcL6Sw4xMrzC/P6RIU0A8TqLj6C/va7t8 NaecMITlj/FhRT45rv+8RYqwZso4Ru29X58WeiwZTQHvFr8QYyGe6wyEixTnGt5hsewJw0 RWcA7uzluNHLk/XZeM4NYyg9Xy5KpTs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-505-oxvRt05sNHWYNp1vTNJTWQ-1; Fri, 19 Jun 2020 12:23:08 -0400 X-MC-Unique: oxvRt05sNHWYNp1vTNJTWQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1C86418C6ECF; Fri, 19 Jun 2020 16:23:07 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0EC1260BF4; Fri, 19 Jun 2020 16:23:02 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Neil Horman , Cunming Liang , Konstantin Ananyev , Olivier Matz Date: Fri, 19 Jun 2020 18:22:37 +0200 Message-Id: <20200619162244.8239-3-david.marchand@redhat.com> In-Reply-To: <20200619162244.8239-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200619162244.8239-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 2/9] eal: fix multiple definition of per lcore thread id X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Because of the inline accessor + static declaration in rte_gettid(), we end up with multiple symbols for RTE_PER_LCORE(_thread_id). Each compilation unit will pay a cost when accessing this information for the first time. $ nm build/app/dpdk-testpmd | grep per_lcore__thread_id 0000000000000054 d per_lcore__thread_id.5037 0000000000000040 d per_lcore__thread_id.5103 0000000000000048 d per_lcore__thread_id.5259 000000000000004c d per_lcore__thread_id.5259 0000000000000044 d per_lcore__thread_id.5933 0000000000000058 d per_lcore__thread_id.6261 0000000000000050 d per_lcore__thread_id.7378 000000000000005c d per_lcore__thread_id.7496 000000000000000c d per_lcore__thread_id.8016 0000000000000010 d per_lcore__thread_id.8431 Make it global as part of the DPDK_21 stable ABI. Fixes: ef76436c6834 ("eal: get unique thread id") Signed-off-by: David Marchand Acked-by: Ray Kinsella --- lib/librte_eal/common/eal_common_thread.c | 1 + lib/librte_eal/include/rte_eal.h | 3 ++- lib/librte_eal/rte_eal_version.map | 7 +++++++ 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index a5f67d811c..280c64bb76 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -22,6 +22,7 @@ #include "eal_thread.h" RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; +RTE_DEFINE_PER_LCORE(int, _thread_id) = -1; static RTE_DEFINE_PER_LCORE(unsigned int, _socket_id) = (unsigned int)SOCKET_ID_ANY; static RTE_DEFINE_PER_LCORE(rte_cpuset_t, _cpuset); diff --git a/lib/librte_eal/include/rte_eal.h b/lib/librte_eal/include/rte_eal.h index 2f9ed298de..2edf8c6556 100644 --- a/lib/librte_eal/include/rte_eal.h +++ b/lib/librte_eal/include/rte_eal.h @@ -447,6 +447,8 @@ enum rte_intr_mode rte_eal_vfio_intr_mode(void); */ int rte_sys_gettid(void); +RTE_DECLARE_PER_LCORE(int, _thread_id); + /** * Get system unique thread id. * @@ -456,7 +458,6 @@ int rte_sys_gettid(void); */ static inline int rte_gettid(void) { - static RTE_DEFINE_PER_LCORE(int, _thread_id) = -1; if (RTE_PER_LCORE(_thread_id) == -1) RTE_PER_LCORE(_thread_id) = rte_sys_gettid(); return RTE_PER_LCORE(_thread_id); diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 196eef5afa..0d42d44ce9 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -221,6 +221,13 @@ DPDK_20.0 { local: *; }; +DPDK_21 { + global: + + per_lcore__thread_id; + +} DPDK_20.0; + EXPERIMENTAL { global: From patchwork Fri Jun 19 16:22:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71809 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2399AA051A; Fri, 19 Jun 2020 18:23:32 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 816B01BF7D; Fri, 19 Jun 2020 18:23:19 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id A748E1BF7D for ; Fri, 19 Jun 2020 18:23:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583797; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9zTPi/8u5JOSyLh9kfSFIm6TQC9+SZQeMQ3QtXTKuEE=; b=TeU90yQ7y1/JP8KS9jpjrZyVfRZ37r0yJNmiJjvx/UXyBvnhVdrCKG+D1tVkl/Yg94X7nw ifIimS4BrhRrkPXw+XYh8dG6t4p1xogz+GQLseoMxb0hPNATo3gF8s9AdhRSz3V5gOXNXY smMA8Yxs7UOI8c/vBYSizs3T+oLFEvs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-342-U2K-C_x0O3ewMHZ8XtDcGg-1; Fri, 19 Jun 2020 12:23:13 -0400 X-MC-Unique: U2K-C_x0O3ewMHZ8XtDcGg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7F1D215B63; Fri, 19 Jun 2020 16:23:11 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id AC77260BF4; Fri, 19 Jun 2020 16:23:07 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Harini Ramakrishnan , Omar Cardona , Pallavi Kadam , Ranjit Menon Date: Fri, 19 Jun 2020 18:22:38 +0200 Message-Id: <20200619162244.8239-4-david.marchand@redhat.com> In-Reply-To: <20200619162244.8239-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200619162244.8239-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 3/9] eal: introduce thread init helper X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Introduce a helper responsible for initialising the per thread context. We can then have a unified context for EAL and non-EAL threads and remove copy/paste'd OS-specific helpers. Per EAL thread CPU affinity setting is separated from the thread init. It is to accommodate with Windows EAL where CPU affinity is not set at the moment. Besides, having affinity set by the master lcore in FreeBSD and Linux will make it possible to detect errors rather than panic in the child thread. But the cleanup when such an event happens is left for later. Signed-off-by: David Marchand --- Changes since v1: - rebased on master, removed Windows workarounds wrt gettid and traces support, --- lib/librte_eal/common/eal_common_thread.c | 51 +++++++++++++---------- lib/librte_eal/common/eal_thread.h | 8 ++-- lib/librte_eal/freebsd/eal.c | 14 ++++++- lib/librte_eal/freebsd/eal_thread.c | 32 +------------- lib/librte_eal/linux/eal.c | 15 ++++++- lib/librte_eal/linux/eal_thread.c | 32 +------------- lib/librte_eal/windows/eal.c | 3 +- lib/librte_eal/windows/eal_thread.c | 10 +---- 8 files changed, 66 insertions(+), 99 deletions(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 280c64bb76..afb30236c5 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -71,20 +71,10 @@ eal_cpuset_socket_id(rte_cpuset_t *cpusetp) return socket_id; } -int -rte_thread_set_affinity(rte_cpuset_t *cpusetp) +static void +thread_update_affinity(rte_cpuset_t *cpusetp) { - int s; - unsigned lcore_id; - pthread_t tid; - - tid = pthread_self(); - - s = pthread_setaffinity_np(tid, sizeof(rte_cpuset_t), cpusetp); - if (s != 0) { - RTE_LOG(ERR, EAL, "pthread_setaffinity_np failed\n"); - return -1; - } + unsigned int lcore_id = rte_lcore_id(); /* store socket_id in TLS for quick access */ RTE_PER_LCORE(_socket_id) = @@ -94,14 +84,24 @@ rte_thread_set_affinity(rte_cpuset_t *cpusetp) memmove(&RTE_PER_LCORE(_cpuset), cpusetp, sizeof(rte_cpuset_t)); - lcore_id = rte_lcore_id(); if (lcore_id != (unsigned)LCORE_ID_ANY) { /* EAL thread will update lcore_config */ lcore_config[lcore_id].socket_id = RTE_PER_LCORE(_socket_id); memmove(&lcore_config[lcore_id].cpuset, cpusetp, sizeof(rte_cpuset_t)); } +} +int +rte_thread_set_affinity(rte_cpuset_t *cpusetp) +{ + if (pthread_setaffinity_np(pthread_self(), sizeof(rte_cpuset_t), + cpusetp) != 0) { + RTE_LOG(ERR, EAL, "pthread_setaffinity_np failed\n"); + return -1; + } + + thread_update_affinity(cpusetp); return 0; } @@ -147,6 +147,19 @@ eal_thread_dump_affinity(char *str, unsigned size) return ret; } +void +rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset) +{ + /* set the lcore ID in per-lcore memory area */ + RTE_PER_LCORE(_lcore_id) = lcore_id; + + /* acquire system unique id */ + rte_gettid(); + + thread_update_affinity(cpuset); + + __rte_trace_mem_per_thread_alloc(); +} struct rte_thread_ctrl_params { void *(*start_routine)(void *); @@ -154,16 +167,14 @@ struct rte_thread_ctrl_params { pthread_barrier_t configured; }; -static void *rte_thread_init(void *arg) +static void *ctrl_thread_init(void *arg) { int ret; - rte_cpuset_t *cpuset = &internal_config.ctrl_cpuset; struct rte_thread_ctrl_params *params = arg; void *(*start_routine)(void *) = params->start_routine; void *routine_arg = params->arg; - /* Store cpuset in TLS for quick access */ - memmove(&RTE_PER_LCORE(_cpuset), cpuset, sizeof(rte_cpuset_t)); + rte_thread_init(rte_lcore_id(), &internal_config.ctrl_cpuset); ret = pthread_barrier_wait(¶ms->configured); if (ret == PTHREAD_BARRIER_SERIAL_THREAD) { @@ -171,8 +182,6 @@ static void *rte_thread_init(void *arg) free(params); } - __rte_trace_mem_per_thread_alloc(); - return start_routine(routine_arg); } @@ -194,7 +203,7 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, pthread_barrier_init(¶ms->configured, NULL, 2); - ret = pthread_create(thread, attr, rte_thread_init, (void *)params); + ret = pthread_create(thread, attr, ctrl_thread_init, (void *)params); if (ret != 0) { free(params); return -ret; diff --git a/lib/librte_eal/common/eal_thread.h b/lib/librte_eal/common/eal_thread.h index b40ed249ed..da5e7c93ba 100644 --- a/lib/librte_eal/common/eal_thread.h +++ b/lib/librte_eal/common/eal_thread.h @@ -16,12 +16,14 @@ __rte_noreturn void *eal_thread_loop(void *arg); /** - * Init per-lcore info for master thread + * Init per-lcore info in current thread. * * @param lcore_id - * identifier of master lcore + * identifier of lcore. + * @param cpuset + * CPU affinity for this thread. */ -void eal_thread_init_master(unsigned lcore_id); +void rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset); /** * Get the NUMA socket id from cpu id. diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index c41f265fac..b5ea11df16 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -877,7 +877,14 @@ rte_eal_init(int argc, char **argv) eal_check_mem_on_local_socket(); - eal_thread_init_master(rte_config.master_lcore); + if (pthread_setaffinity_np(pthread_self(), sizeof(rte_cpuset_t), + &lcore_config[rte_config.master_lcore].cpuset) != 0) { + rte_eal_init_alert("Cannot set affinity"); + rte_errno = EINVAL; + return -1; + } + rte_thread_init(rte_config.master_lcore, + &lcore_config[rte_config.master_lcore].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); @@ -908,6 +915,11 @@ rte_eal_init(int argc, char **argv) snprintf(thread_name, sizeof(thread_name), "lcore-slave-%d", i); rte_thread_setname(lcore_config[i].thread_id, thread_name); + + ret = pthread_setaffinity_np(lcore_config[i].thread_id, + sizeof(rte_cpuset_t), &lcore_config[i].cpuset); + if (ret != 0) + rte_panic("Cannot set affinity\n"); } /* diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index 40676d9ef5..c1fb8eb2d8 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -66,29 +66,6 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned slave_id) return rc; } -/* set affinity for current thread */ -static int -eal_thread_set_affinity(void) -{ - unsigned lcore_id = rte_lcore_id(); - - /* acquire system unique id */ - rte_gettid(); - - /* update EAL thread core affinity */ - return rte_thread_set_affinity(&lcore_config[lcore_id].cpuset); -} - -void eal_thread_init_master(unsigned lcore_id) -{ - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); -} - /* main loop of threads */ __rte_noreturn void * eal_thread_loop(__rte_unused void *arg) @@ -113,19 +90,12 @@ eal_thread_loop(__rte_unused void *arg) m2s = lcore_config[lcore_id].pipe_master2slave[0]; s2m = lcore_config[lcore_id].pipe_slave2master[1]; - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); + rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%p;cpuset=[%s%s])\n", lcore_id, thread_id, cpuset, ret == 0 ? "" : "..."); - __rte_trace_mem_per_thread_alloc(); rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); /* read on our pipe to get commands */ diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index f162124a37..8638376b8a 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -1205,10 +1205,16 @@ rte_eal_init(int argc, char **argv) eal_check_mem_on_local_socket(); - eal_thread_init_master(rte_config.master_lcore); + if (pthread_setaffinity_np(pthread_self(), sizeof(rte_cpuset_t), + &lcore_config[rte_config.master_lcore].cpuset) != 0) { + rte_eal_init_alert("Cannot set affinity"); + rte_errno = EINVAL; + return -1; + } + rte_thread_init(rte_config.master_lcore, + &lcore_config[rte_config.master_lcore].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "Master lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", rte_config.master_lcore, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); @@ -1240,6 +1246,11 @@ rte_eal_init(int argc, char **argv) if (ret != 0) RTE_LOG(DEBUG, EAL, "Cannot set name for lcore thread\n"); + + ret = pthread_setaffinity_np(lcore_config[i].thread_id, + sizeof(rte_cpuset_t), &lcore_config[i].cpuset); + if (ret != 0) + rte_panic("Cannot set affinity\n"); } /* diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index a52ebef3a4..07aec0c44d 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -66,29 +66,6 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned slave_id) return rc; } -/* set affinity for current EAL thread */ -static int -eal_thread_set_affinity(void) -{ - unsigned lcore_id = rte_lcore_id(); - - /* acquire system unique id */ - rte_gettid(); - - /* update EAL thread core affinity */ - return rte_thread_set_affinity(&lcore_config[lcore_id].cpuset); -} - -void eal_thread_init_master(unsigned lcore_id) -{ - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); -} - /* main loop of threads */ __rte_noreturn void * eal_thread_loop(__rte_unused void *arg) @@ -113,19 +90,12 @@ eal_thread_loop(__rte_unused void *arg) m2s = lcore_config[lcore_id].pipe_master2slave[0]; s2m = lcore_config[lcore_id].pipe_slave2master[1]; - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; - - /* set CPU affinity */ - if (eal_thread_set_affinity() < 0) - rte_panic("cannot set affinity\n"); + rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", lcore_id, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); - __rte_trace_mem_per_thread_alloc(); rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); /* read on our pipe to get commands */ diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c index 427a5557fa..23de12ab43 100644 --- a/lib/librte_eal/windows/eal.c +++ b/lib/librte_eal/windows/eal.c @@ -354,7 +354,8 @@ rte_eal_init(int argc, char **argv) return -1; } - eal_thread_init_master(rte_config.master_lcore); + rte_thread_init(rte_config.master_lcore, + &lcore_config[rte_config.master_lcore].cpuset); RTE_LCORE_FOREACH_SLAVE(i) { diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index f12a2ec6ad..4f01881240 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -53,13 +53,6 @@ rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int slave_id) return 0; } -void -eal_thread_init_master(unsigned int lcore_id) -{ - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; -} - /* main loop of threads */ void * eal_thread_loop(void *arg __rte_unused) @@ -84,8 +77,7 @@ eal_thread_loop(void *arg __rte_unused) m2s = lcore_config[lcore_id].pipe_master2slave[0]; s2m = lcore_config[lcore_id].pipe_slave2master[1]; - /* set the lcore ID in per-lcore memory area */ - RTE_PER_LCORE(_lcore_id) = lcore_id; + rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s])\n", lcore_id, (uintptr_t)thread_id, cpuset); From patchwork Fri Jun 19 16:22:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71810 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 52ACBA051A; Fri, 19 Jun 2020 18:23:45 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 750B31BF8A; Fri, 19 Jun 2020 18:23:22 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id BF08D1BF84 for ; Fri, 19 Jun 2020 18:23:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nTRGiDB7hN+RD2fSZB3QWQYxfLcHBbpBtbL3h9OosqE=; b=W9zjsCyhAohGZhWGfC8jCfFHO1Mk9XUtPpbdDg+l9ozcwTOfT9CaKqqaIMbpFqZKOVuG2i qCUiNuIicmIVuDyDj6YleG601jXcZmq1QugKi5y7bWkegb0fDqUe8kt2RWpW36c3HyDfVP +NrqWY+JSg7jZUHwPEZOmAMtESFlnDU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-121-O7BRS4y3OXmumK6VxMoSFA-1; Fri, 19 Jun 2020 12:23:18 -0400 X-MC-Unique: O7BRS4y3OXmumK6VxMoSFA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E68FE18C6ED1; Fri, 19 Jun 2020 16:23:16 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id 892F360BF4; Fri, 19 Jun 2020 16:23:12 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Jerin Jacob , Sunil Kumar Kori Date: Fri, 19 Jun 2020 18:22:39 +0200 Message-Id: <20200619162244.8239-5-david.marchand@redhat.com> In-Reply-To: <20200619162244.8239-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200619162244.8239-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 4/9] eal: introduce thread uninit helper X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is a preparation step for dynamically unregistering threads. Since we explicitly allocate a per thread trace buffer in rte_thread_init, add an internal helper to free this buffer. Signed-off-by: David Marchand --- Note: I preferred renaming the current internal function to free all threads trace buffers (new name trace_mem_free()) and reuse the previous name (trace_mem_per_thread_free()) when freeing this buffer for a given thread. Changes since v1: - rebased on master, removed Windows workaround wrt traces support, --- lib/librte_eal/common/eal_common_thread.c | 9 +++++ lib/librte_eal/common/eal_common_trace.c | 49 +++++++++++++++++++---- lib/librte_eal/common/eal_thread.h | 5 +++ lib/librte_eal/common/eal_trace.h | 1 + 4 files changed, 57 insertions(+), 7 deletions(-) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index afb30236c5..9b9af75dc1 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -20,6 +20,7 @@ #include "eal_internal_cfg.h" #include "eal_private.h" #include "eal_thread.h" +#include "eal_trace.h" RTE_DEFINE_PER_LCORE(unsigned int, _lcore_id) = LCORE_ID_ANY; RTE_DEFINE_PER_LCORE(int, _thread_id) = -1; @@ -161,6 +162,14 @@ rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset) __rte_trace_mem_per_thread_alloc(); } +void +rte_thread_uninit(void) +{ + trace_mem_per_thread_free(); + + RTE_PER_LCORE(_lcore_id) = LCORE_ID_ANY; +} + struct rte_thread_ctrl_params { void *(*start_routine)(void *); void *arg; diff --git a/lib/librte_eal/common/eal_common_trace.c b/lib/librte_eal/common/eal_common_trace.c index 875553d7e5..cd2d217b02 100644 --- a/lib/librte_eal/common/eal_common_trace.c +++ b/lib/librte_eal/common/eal_common_trace.c @@ -101,7 +101,7 @@ eal_trace_fini(void) { if (!rte_trace_is_enabled()) return; - trace_mem_per_thread_free(); + trace_mem_free(); trace_metadata_destroy(); eal_trace_args_free(); } @@ -370,24 +370,59 @@ __rte_trace_mem_per_thread_alloc(void) rte_spinlock_unlock(&trace->lock); } +static void +trace_mem_per_thread_free_unlocked(struct thread_mem_meta *meta) +{ + if (meta->area == TRACE_AREA_HUGEPAGE) + eal_free_no_trace(meta->mem); + else if (meta->area == TRACE_AREA_HEAP) + free(meta->mem); +} + void trace_mem_per_thread_free(void) +{ + struct trace *trace = trace_obj_get(); + struct __rte_trace_header *header; + uint32_t count; + + if (RTE_PER_LCORE(trace_mem) == NULL) + return; + + header = RTE_PER_LCORE(trace_mem); + rte_spinlock_lock(&trace->lock); + for (count = 0; count < trace->nb_trace_mem_list; count++) { + if (trace->lcore_meta[count].mem == header) + break; + } + if (count != trace->nb_trace_mem_list) { + struct thread_mem_meta *meta = &trace->lcore_meta[count]; + + trace_mem_per_thread_free_unlocked(meta); + if (count != trace->nb_trace_mem_list - 1) { + memmove(meta, meta + 1, + sizeof(*meta) * + (trace->nb_trace_mem_list - count - 1)); + } + trace->nb_trace_mem_list--; + } + rte_spinlock_unlock(&trace->lock); +} + +void +trace_mem_free(void) { struct trace *trace = trace_obj_get(); uint32_t count; - void *mem; if (!rte_trace_is_enabled()) return; rte_spinlock_lock(&trace->lock); for (count = 0; count < trace->nb_trace_mem_list; count++) { - mem = trace->lcore_meta[count].mem; - if (trace->lcore_meta[count].area == TRACE_AREA_HUGEPAGE) - eal_free_no_trace(mem); - else if (trace->lcore_meta[count].area == TRACE_AREA_HEAP) - free(mem); + trace_mem_per_thread_free_unlocked(&trace->lcore_meta[count]); } + trace->nb_trace_mem_list = 0; rte_spinlock_unlock(&trace->lock); } diff --git a/lib/librte_eal/common/eal_thread.h b/lib/librte_eal/common/eal_thread.h index da5e7c93ba..4ecd8fd53a 100644 --- a/lib/librte_eal/common/eal_thread.h +++ b/lib/librte_eal/common/eal_thread.h @@ -25,6 +25,11 @@ __rte_noreturn void *eal_thread_loop(void *arg); */ void rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset); +/** + * Uninitialize per-lcore info for current thread. + */ +void rte_thread_uninit(void); + /** * Get the NUMA socket id from cpu id. * This function is private to EAL. diff --git a/lib/librte_eal/common/eal_trace.h b/lib/librte_eal/common/eal_trace.h index 8f60616156..92c5951c3a 100644 --- a/lib/librte_eal/common/eal_trace.h +++ b/lib/librte_eal/common/eal_trace.h @@ -106,6 +106,7 @@ int trace_metadata_create(void); void trace_metadata_destroy(void); int trace_mkdir(void); int trace_epoch_time_save(void); +void trace_mem_free(void); void trace_mem_per_thread_free(void); /* EAL interface */ From patchwork Fri Jun 19 16:22:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71811 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BD8ACA051A; Fri, 19 Jun 2020 18:23:56 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A33691BF83; Fri, 19 Jun 2020 18:23:27 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id F21271BF8E for ; Fri, 19 Jun 2020 18:23:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583805; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FU8p2jBcOc3aPAatFVCPey8QqKAzxg29a9uD1DsltWs=; b=IBBzB7yaHoWovuaFsfyPbxqQQlIdZyGCwZ52dq7fbbHMKo+kGqrS9ardUwIqSdn2xwiOD1 CACen4sIUZIJiWJvccPbM2w/pUMFKDSnt1GI9yePYGX9nKiNPgDks5Hh6zjtJKaB1597/o XgRF5zA70NC6cIWH/hXHz/E/s27paPo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-202-GyrcBBJ6OX-UsMQnTBWBpw-1; Fri, 19 Jun 2020 12:23:23 -0400 X-MC-Unique: GyrcBBJ6OX-UsMQnTBWBpw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 28DF8107ACCA; Fri, 19 Jun 2020 16:23:22 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8355860BF4; Fri, 19 Jun 2020 16:23:17 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org Date: Fri, 19 Jun 2020 18:22:40 +0200 Message-Id: <20200619162244.8239-6-david.marchand@redhat.com> In-Reply-To: <20200619162244.8239-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200619162244.8239-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 5/9] eal: move lcore role code X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" For consistency sake, move all lcore role code in the dedicated compilation unit / header. Signed-off-by: David Marchand --- lib/librte_eal/common/eal_common_lcore.c | 11 +++++++ lib/librte_eal/common/eal_common_thread.c | 11 ------- lib/librte_eal/include/rte_eal.h | 9 ------ lib/librte_eal/include/rte_lcore.h | 37 ++++++++++++++--------- 4 files changed, 34 insertions(+), 34 deletions(-) diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 5404922a87..86d32a3dd7 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -63,6 +63,17 @@ rte_eal_lcore_role(unsigned int lcore_id) return cfg->lcore_role[lcore_id]; } +int +rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + + if (lcore_id >= RTE_MAX_LCORE) + return -EINVAL; + + return cfg->lcore_role[lcore_id] == role; +} + int rte_lcore_is_enabled(unsigned int lcore_id) { struct rte_config *cfg = rte_eal_get_configuration(); diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 9b9af75dc1..4f96c50424 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -33,17 +33,6 @@ unsigned rte_socket_id(void) return RTE_PER_LCORE(_socket_id); } -int -rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role) -{ - struct rte_config *cfg = rte_eal_get_configuration(); - - if (lcore_id >= RTE_MAX_LCORE) - return -EINVAL; - - return cfg->lcore_role[lcore_id] == role; -} - static int eal_cpuset_socket_id(rte_cpuset_t *cpusetp) { diff --git a/lib/librte_eal/include/rte_eal.h b/lib/librte_eal/include/rte_eal.h index 2edf8c6556..0913d1947c 100644 --- a/lib/librte_eal/include/rte_eal.h +++ b/lib/librte_eal/include/rte_eal.h @@ -31,15 +31,6 @@ extern "C" { /* Maximum thread_name length. */ #define RTE_MAX_THREAD_NAME_LEN 16 -/** - * The lcore role (used in RTE or not). - */ -enum rte_lcore_role_t { - ROLE_RTE, - ROLE_OFF, - ROLE_SERVICE, -}; - /** * The type of process in a linux, multi-process setup */ diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 5c1d1926e9..3968c40693 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -24,6 +24,15 @@ extern "C" { RTE_DECLARE_PER_LCORE(unsigned, _lcore_id); /**< Per thread "lcore id". */ +/** + * The lcore role (used in RTE or not). + */ +enum rte_lcore_role_t { + ROLE_RTE, + ROLE_OFF, + ROLE_SERVICE, +}; + /** * Get a lcore's role. * @@ -34,6 +43,20 @@ RTE_DECLARE_PER_LCORE(unsigned, _lcore_id); /**< Per thread "lcore id". */ */ enum rte_lcore_role_t rte_eal_lcore_role(unsigned int lcore_id); +/** + * Test if the core supplied has a specific role + * + * @param lcore_id + * The identifier of the lcore, which MUST be between 0 and + * RTE_MAX_LCORE-1. + * @param role + * The role to be checked against. + * @return + * Boolean value: positive if test is true; otherwise returns 0. + */ +int +rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role); + /** * Return the Application thread ID of the execution unit. * @@ -283,20 +306,6 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg); -/** - * Test if the core supplied has a specific role - * - * @param lcore_id - * The identifier of the lcore, which MUST be between 0 and - * RTE_MAX_LCORE-1. - * @param role - * The role to be checked against. - * @return - * Boolean value: positive if test is true; otherwise returns 0. - */ -int -rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role); - #ifdef __cplusplus } #endif From patchwork Fri Jun 19 16:22:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71812 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A935A051A; Fri, 19 Jun 2020 18:24:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 16B9E1BF82; Fri, 19 Jun 2020 18:23:37 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 24E931BF73 for ; Fri, 19 Jun 2020 18:23:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583814; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eQc4RB2dd6uOyDlUd/T1eVWEOzpzNsfDEUktmd5RTtA=; b=J9TAIbEyws2fWpJOQ/67l9FbpSeJXLfcupqObxP0oPQAYwTzZRd5Yr7XkINIEcVgl35i7D tMsHaVtwT0XIlWC7fy7TyfqiW/GogUglCBAK/STmh0WTAQ/oSsrLop/Rcf4Bn1alj+5m23 aGaO3Ld8yLazOakk/QoawnUdw/2iVLg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-245-4789JZB9PY-saOVaFZZoOA-1; Fri, 19 Jun 2020 12:23:30 -0400 X-MC-Unique: 4789JZB9PY-saOVaFZZoOA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7E429835B4A; Fri, 19 Jun 2020 16:23:28 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1003E61985; Fri, 19 Jun 2020 16:23:22 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Thomas Monjalon , John McNamara , Marko Kovacevic , Anatoly Burakov , Olivier Matz , Andrew Rybchenko , Neil Horman Date: Fri, 19 Jun 2020 18:22:41 +0200 Message-Id: <20200619162244.8239-7-david.marchand@redhat.com> In-Reply-To: <20200619162244.8239-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200619162244.8239-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 6/9] eal: register non-EAL threads as lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" DPDK allows calling some part of its API from a non-EAL thread but this has some limitations. OVS (and other applications) has its own thread management but still want to avoid such limitations by hacking RTE_PER_LCORE(_lcore_id) and faking EAL threads potentially unknown of some DPDK component. Introduce a new API to register non-EAL thread and associate them to a free lcore with a new NON_EAL role. This role denotes lcores that do not run DPDK mainloop and as such prevents use of rte_eal_wait_lcore() and consorts. Signed-off-by: David Marchand --- Changes since v1: - moved cleanup on lcore role code in patch 5, - added unit test, - updated documentation, - changed naming from "external thread" to "registered non-EAL thread" --- MAINTAINERS | 1 + app/test/Makefile | 1 + app/test/autotest_data.py | 6 + app/test/meson.build | 2 + app/test/test_lcores.c | 139 ++++++++++++++++++ doc/guides/howto/debug_troubleshoot.rst | 5 +- .../prog_guide/env_abstraction_layer.rst | 22 +-- doc/guides/prog_guide/mempool_lib.rst | 2 +- lib/librte_eal/common/eal_common_lcore.c | 44 +++++- lib/librte_eal/common/eal_common_thread.c | 33 +++++ lib/librte_eal/common/eal_private.h | 18 +++ lib/librte_eal/include/rte_lcore.h | 18 ++- lib/librte_eal/rte_eal_version.map | 4 + lib/librte_mempool/rte_mempool.h | 11 +- 14 files changed, 284 insertions(+), 22 deletions(-) create mode 100644 app/test/test_lcores.c diff --git a/MAINTAINERS b/MAINTAINERS index 816696caf2..fe9e74ffbc 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -183,6 +183,7 @@ F: app/test/test_debug.c F: app/test/test_eal* F: app/test/test_errno.c F: app/test/test_interrupts.c +F: app/test/test_lcores.c F: app/test/test_logs.c F: app/test/test_memcpy* F: app/test/test_per_lcore.c diff --git a/app/test/Makefile b/app/test/Makefile index 7b96a03a64..4a8dea2425 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -97,6 +97,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += test_flow_classify.c endif SRCS-y += test_rwlock.c +SRCS-y += test_lcores.c SRCS-$(CONFIG_RTE_LIBRTE_STACK) += test_stack.c SRCS-$(CONFIG_RTE_LIBRTE_STACK) += test_stack_perf.c diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py index fc3fcc159e..600b130966 100644 --- a/app/test/autotest_data.py +++ b/app/test/autotest_data.py @@ -62,6 +62,12 @@ "Func": rwlock_autotest, "Report": None, }, + { + "Name": "Lcores autotest", + "Command": "lcores_autotest", + "Func": default_autotest, + "Report": None, + }, { "Name": "Logs autotest", "Command": "logs_autotest", diff --git a/app/test/meson.build b/app/test/meson.build index 5233ead46e..a57477b7cc 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -67,6 +67,7 @@ test_sources = files('commands.c', 'test_ipsec_perf.c', 'test_kni.c', 'test_kvargs.c', + 'test_lcores.c', 'test_logs.c', 'test_lpm.c', 'test_lpm6.c', @@ -206,6 +207,7 @@ fast_tests = [ ['hash_autotest', true], ['interrupt_autotest', true], ['ipfrag_autotest', false], + ['lcores_autotest', true], ['logs_autotest', true], ['lpm_autotest', true], ['lpm6_autotest', true], diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c new file mode 100644 index 0000000000..155b255b94 --- /dev/null +++ b/app/test/test_lcores.c @@ -0,0 +1,139 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2020 Red Hat, Inc. + */ + +#include +#include + +#include + +#include "test.h" + +struct thread_context { + enum { INIT, ERROR, DONE } state; + bool lcore_id_any; + pthread_t id; + unsigned int *registered_count; +}; +static void *thread_loop(void *arg) +{ + struct thread_context *t = arg; + unsigned int lcore_id; + + lcore_id = rte_lcore_id(); + if (lcore_id != LCORE_ID_ANY) { + printf("Incorrect lcore id for new thread %u\n", lcore_id); + t->state = ERROR; + } + rte_thread_register(); + lcore_id = rte_lcore_id(); + if ((t->lcore_id_any && lcore_id != LCORE_ID_ANY) || + (!t->lcore_id_any && lcore_id == LCORE_ID_ANY)) { + printf("Could not register new thread, got %u while %sexpecting %u\n", + lcore_id, t->lcore_id_any ? "" : "not ", LCORE_ID_ANY); + t->state = ERROR; + } + /* Report register happened to the control thread. */ + __atomic_add_fetch(t->registered_count, 1, __ATOMIC_RELEASE); + + /* Wait for release from the control thread. */ + while (__atomic_load_n(t->registered_count, __ATOMIC_ACQUIRE) != 0) + ; + rte_thread_unregister(); + lcore_id = rte_lcore_id(); + if (lcore_id != LCORE_ID_ANY) { + printf("Could not unregister new thread, %u still assigned\n", + lcore_id); + t->state = ERROR; + } + + if (t->state != ERROR) + t->state = DONE; + + return NULL; +} + +static int +test_non_eal_lcores(unsigned int eal_threads_count) +{ + struct thread_context thread_contexts[RTE_MAX_LCORE]; + unsigned int non_eal_threads_count; + struct thread_context *t; + unsigned int registered_count; + unsigned int i; + int ret; + + non_eal_threads_count = 0; + registered_count = 0; + + /* Try to create as many threads as possible. */ + for (i = 0; i < RTE_MAX_LCORE - eal_threads_count; i++) { + t = &thread_contexts[i]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = false; + if (pthread_create(&t->id, NULL, thread_loop, t) != 0) + break; + non_eal_threads_count++; + } + printf("non-EAL threads count: %u\n", non_eal_threads_count); + /* Wait all non-EAL threads to register. */ + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + + /* We managed to create the max number of threads, let's try to create + * one more. This will allow one more check. + */ + if (eal_threads_count + non_eal_threads_count < RTE_MAX_LCORE) + goto skip_lcore_any; + t = &thread_contexts[non_eal_threads_count]; + t->state = INIT; + t->registered_count = ®istered_count; + t->lcore_id_any = true; + if (pthread_create(&t->id, NULL, thread_loop, t) == 0) { + non_eal_threads_count++; + printf("non-EAL threads count: %u\n", non_eal_threads_count); + while (__atomic_load_n(®istered_count, __ATOMIC_ACQUIRE) != + non_eal_threads_count) + ; + } + +skip_lcore_any: + /* Release all threads, and check their states. */ + __atomic_store_n(®istered_count, 0, __ATOMIC_RELEASE); + ret = 0; + for (i = 0; i < non_eal_threads_count; i++) { + t = &thread_contexts[i]; + pthread_join(t->id, NULL); + if (t->state != DONE) + ret = -1; + } + + return ret; +} + +static int +test_lcores(void) +{ + unsigned int eal_threads_count = 0; + unsigned int i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + if (!rte_lcore_has_role(i, ROLE_OFF)) + eal_threads_count++; + } + if (eal_threads_count == 0) { + printf("Something is broken, no EAL thread detected.\n"); + return TEST_FAILED; + } + printf("EAL threads count: %u, RTE_MAX_LCORE=%u\n", eal_threads_count, + RTE_MAX_LCORE); + + if (test_non_eal_lcores(eal_threads_count) < 0) + return TEST_FAILED; + + return TEST_SUCCESS; +} + +REGISTER_TEST_COMMAND(lcores_autotest, test_lcores); diff --git a/doc/guides/howto/debug_troubleshoot.rst b/doc/guides/howto/debug_troubleshoot.rst index cef016b2fe..5a46f5fba3 100644 --- a/doc/guides/howto/debug_troubleshoot.rst +++ b/doc/guides/howto/debug_troubleshoot.rst @@ -307,8 +307,9 @@ Custom worker function :numref:`dtg_distributor_worker`. #. Configuration issue isolation - * Identify core role using ``rte_eal_lcore_role`` to identify RTE, OFF and - SERVICE. Check performance functions are mapped to run on the cores. + * Identify core role using ``rte_eal_lcore_role`` to identify RTE, OFF, + SERVICE and NON_EAL. Check performance functions are mapped to run on the + cores. * For high-performance execution logic ensure running it on correct NUMA and non-master core. diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 48a2fec066..f64ae953d1 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -564,9 +564,13 @@ It's also compatible with the pattern of corelist('-l') option. non-EAL pthread support ~~~~~~~~~~~~~~~~~~~~~~~ -It is possible to use the DPDK execution context with any user pthread (aka. Non-EAL pthreads). -In a non-EAL pthread, the *_lcore_id* is always LCORE_ID_ANY which identifies that it is not an EAL thread with a valid, unique, *_lcore_id*. -Some libraries will use an alternative unique ID (e.g. TID), some will not be impacted at all, and some will work but with limitations (e.g. timer and mempool libraries). +It is possible to use the DPDK execution context with any user pthread (aka. non-EAL pthreads). +There are two kinds of non-EAL pthreads: + +- a registered non-EAL pthread with a valid *_lcore_id* that was successfully assigned by calling ``rte_thread_register()``, +- a non registered non-EAL pthread with a LCORE_ID_ANY, + +For non registered non-EAL pthread (with a LCORE_ID_ANY *_lcore_id*), some libraries will use an alternative unique ID (e.g. TID), some will not be impacted at all, and some will work but with limitations (e.g. timer and mempool libraries). All these impacts are mentioned in :ref:`known_issue_label` section. @@ -613,9 +617,9 @@ Known Issues + rte_mempool The rte_mempool uses a per-lcore cache inside the mempool. - For non-EAL pthreads, ``rte_lcore_id()`` will not return a valid number. - So for now, when rte_mempool is used with non-EAL pthreads, the put/get operations will bypass the default mempool cache and there is a performance penalty because of this bypass. - Only user-owned external caches can be used in a non-EAL context in conjunction with ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()`` that accept an explicit cache parameter. + For unregistered non-EAL pthreads, ``rte_lcore_id()`` will not return a valid number. + So for now, when rte_mempool is used with unregistered non-EAL pthreads, the put/get operations will bypass the default mempool cache and there is a performance penalty because of this bypass. + Only user-owned external caches can be used in an unregistered non-EAL context in conjunction with ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()`` that accept an explicit cache parameter. + rte_ring @@ -660,15 +664,15 @@ Known Issues + rte_timer - Running ``rte_timer_manage()`` on a non-EAL pthread is not allowed. However, resetting/stopping the timer from a non-EAL pthread is allowed. + Running ``rte_timer_manage()`` on an unregistered non-EAL pthread is not allowed. However, resetting/stopping the timer from a non-EAL pthread is allowed. + rte_log - In non-EAL pthreads, there is no per thread loglevel and logtype, global loglevels are used. + In unregistered non-EAL pthreads, there is no per thread loglevel and logtype, global loglevels are used. + misc - The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in a non-EAL pthread. + The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread. cgroup control ~~~~~~~~~~~~~~ diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst index f8b430d656..e3e1f940be 100644 --- a/doc/guides/prog_guide/mempool_lib.rst +++ b/doc/guides/prog_guide/mempool_lib.rst @@ -103,7 +103,7 @@ The maximum size of the cache is static and is defined at compilation time (CONF Alternatively to the internal default per-lcore local cache, an application can create and manage external caches through the ``rte_mempool_cache_create()``, ``rte_mempool_cache_free()`` and ``rte_mempool_cache_flush()`` calls. These user-owned caches can be explicitly passed to ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()``. The ``rte_mempool_default_cache()`` call returns the default internal cache if any. -In contrast to the default caches, user-owned caches can be used by non-EAL threads too. +In contrast to the default caches, user-owned caches can be used by unregistered non-EAL threads too. Mempool Handlers ------------------------ diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 86d32a3dd7..7db05428e7 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -6,12 +6,13 @@ #include #include -#include -#include -#include -#include #include #include +#include +#include +#include +#include +#include #include "eal_private.h" #include "eal_thread.h" @@ -220,3 +221,38 @@ rte_socket_id_by_idx(unsigned int idx) } return config->numa_nodes[idx]; } + +static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; + +unsigned int +eal_lcore_non_eal_allocate(void) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + unsigned int lcore_id; + + rte_spinlock_lock(&lcore_lock); + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] != ROLE_OFF) + continue; + cfg->lcore_role[lcore_id] = ROLE_NON_EAL; + cfg->lcore_count++; + break; + } + if (lcore_id == RTE_MAX_LCORE) + RTE_LOG(DEBUG, EAL, "No lcore available.\n"); + rte_spinlock_unlock(&lcore_lock); + return lcore_id; +} + +void +eal_lcore_non_eal_release(unsigned int lcore_id) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + + rte_spinlock_lock(&lcore_lock); + if (cfg->lcore_role[lcore_id] == ROLE_NON_EAL) { + cfg->lcore_role[lcore_id] = ROLE_OFF; + cfg->lcore_count--; + } + rte_spinlock_unlock(&lcore_lock); +} diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 4f96c50424..338194832b 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -236,3 +236,36 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, pthread_join(*thread, NULL); return -ret; } + +void +rte_thread_register(void) +{ + unsigned int lcore_id; + rte_cpuset_t cpuset; + + /* EAL init flushes all lcores, we can't register before. */ + assert(internal_config.init_complete == 1); + if (pthread_getaffinity_np(pthread_self(), sizeof(cpuset), + &cpuset) != 0) + CPU_ZERO(&cpuset); + lcore_id = eal_lcore_non_eal_allocate(); + if (lcore_id >= RTE_MAX_LCORE) + lcore_id = LCORE_ID_ANY; + rte_thread_init(lcore_id, &cpuset); + if (lcore_id != LCORE_ID_ANY) + RTE_LOG(DEBUG, EAL, "Registered non-EAL thread as lcore %u.\n", + lcore_id); +} + +void +rte_thread_unregister(void) +{ + unsigned int lcore_id = rte_lcore_id(); + + if (lcore_id != LCORE_ID_ANY) + eal_lcore_non_eal_release(lcore_id); + rte_thread_uninit(); + if (lcore_id != LCORE_ID_ANY) + RTE_LOG(DEBUG, EAL, "Unregistered non-EAL thread (was lcore %u).\n", + lcore_id); +} diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 0592fcd694..73238ff157 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -396,6 +396,24 @@ uint64_t get_tsc_freq(void); */ uint64_t get_tsc_freq_arch(void); +/** + * Allocate a free lcore to associate to a non-EAL thread. + * + * @return + * - the id of a lcore with role ROLE_NON_EAL on success. + * - RTE_MAX_LCORE if none was available. + */ +unsigned int eal_lcore_non_eal_allocate(void); + +/** + * Release the lcore used by a non-EAL thread. + * Counterpart of eal_lcore_non_eal_allocate(). + * + * @param lcore_id + * The lcore with role ROLE_NON_EAL to release. + */ +void eal_lcore_non_eal_release(unsigned int lcore_id); + /** * Prepare physical memory mapping * i.e. hugepages on Linux and diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 3968c40693..ea86220394 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -31,6 +31,7 @@ enum rte_lcore_role_t { ROLE_RTE, ROLE_OFF, ROLE_SERVICE, + ROLE_NON_EAL, }; /** @@ -67,7 +68,8 @@ rte_lcore_has_role(unsigned int lcore_id, enum rte_lcore_role_t role); * to run threads with lcore IDs 0, 1, 2 and 3 on physical core 10.. * * @return - * Logical core ID (in EAL thread) or LCORE_ID_ANY (in non-EAL thread) + * Logical core ID (in EAL thread or registered non-EAL thread) or + * LCORE_ID_ANY (in unregistered non-EAL thread) */ static inline unsigned rte_lcore_id(void) @@ -279,6 +281,20 @@ int rte_thread_setname(pthread_t id, const char *name); __rte_experimental int rte_thread_getname(pthread_t id, char *name, size_t len); +/** + * Register current non-EAL thread as a lcore. + */ +__rte_experimental +void +rte_thread_register(void); + +/** + * Unregister current thread and release lcore if one was associated. + */ +__rte_experimental +void +rte_thread_unregister(void); + /** * Create a control thread. * diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 0d42d44ce9..5503dd7620 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -393,6 +393,10 @@ EXPERIMENTAL { rte_trace_point_lookup; rte_trace_regexp; rte_trace_save; + + # added in 20.08 + rte_thread_register; + rte_thread_unregister; }; INTERNAL { diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h index 652d19f9f1..9e0ee052b3 100644 --- a/lib/librte_mempool/rte_mempool.h +++ b/lib/librte_mempool/rte_mempool.h @@ -28,9 +28,9 @@ * rte_mempool_get() or rte_mempool_put() are designed to be called from an EAL * thread due to the internal per-lcore cache. Due to the lack of caching, * rte_mempool_get() or rte_mempool_put() performance will suffer when called - * by non-EAL threads. Instead, non-EAL threads should call - * rte_mempool_generic_get() or rte_mempool_generic_put() with a user cache - * created with rte_mempool_cache_create(). + * by unregistered non-EAL threads. Instead, unregistered non-EAL threads + * should call rte_mempool_generic_get() or rte_mempool_generic_put() with a + * user cache created with rte_mempool_cache_create(). */ #include @@ -1233,7 +1233,7 @@ void rte_mempool_dump(FILE *f, struct rte_mempool *mp); /** * Create a user-owned mempool cache. * - * This can be used by non-EAL threads to enable caching when they + * This can be used by unregistered non-EAL threads to enable caching when they * interact with a mempool. * * @param size @@ -1264,7 +1264,8 @@ rte_mempool_cache_free(struct rte_mempool_cache *cache); * @param lcore_id * The logical core id. * @return - * A pointer to the mempool cache or NULL if disabled or non-EAL thread. + * A pointer to the mempool cache or NULL if disabled or unregistered non-EAL + * thread. */ static __rte_always_inline struct rte_mempool_cache * rte_mempool_default_cache(struct rte_mempool *mp, unsigned lcore_id) From patchwork Fri Jun 19 16:22:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71813 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 32016A051A; Fri, 19 Jun 2020 18:24:23 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C55C71BFA4; Fri, 19 Jun 2020 18:23:40 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 9A9E71BFA4 for ; Fri, 19 Jun 2020 18:23:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583819; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s2K90QaYJuM84cIxWxdTQRDdmUZD6NQvdYliXIc5wOA=; b=c0yYdDFj3KNsY/goY/61PIihnA6PF7SQbFlxesmtncUX0WAM959pYunxgtivDnqEeWw/tY uWxVfg+oIq+u67PQyC2xA3rVJm8Jdw0zlva5//e5DfQxkEa+vN7ALP7w9Vd8rOWeuxC6au wOVIb+oFf/5zb1YnwGPnFSzBKgJ0yK0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-7-MHUUXaMMPrOuj4-6XCeZWw-1; Fri, 19 Jun 2020 12:23:35 -0400 X-MC-Unique: MHUUXaMMPrOuj4-6XCeZWw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CD9B41005512; Fri, 19 Jun 2020 16:23:33 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3348560BF4; Fri, 19 Jun 2020 16:23:28 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Neil Horman Date: Fri, 19 Jun 2020 18:22:42 +0200 Message-Id: <20200619162244.8239-8-david.marchand@redhat.com> In-Reply-To: <20200619162244.8239-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200619162244.8239-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 7/9] eal: add lcore init callbacks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" DPDK components and applications can have their say when a new lcore is initialized. For this, they can register a callback for initializing and releasing their private data. Signed-off-by: David Marchand --- Changes since v1: - added unit test (since missing some coverage, for v3), - preferred callback and removed mention of notification, --- app/test/test_lcores.c | 95 ++++++++++++++++ lib/librte_eal/common/eal_common_lcore.c | 139 ++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 3 +- lib/librte_eal/include/rte_lcore.h | 68 +++++++++++ lib/librte_eal/rte_eal_version.map | 2 + 5 files changed, 301 insertions(+), 6 deletions(-) diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c index 155b255b94..cf2439f71d 100644 --- a/app/test/test_lcores.c +++ b/app/test/test_lcores.c @@ -5,6 +5,7 @@ #include #include +#include #include #include "test.h" @@ -113,6 +114,95 @@ test_non_eal_lcores(unsigned int eal_threads_count) return ret; } +struct limit_lcore_context { + unsigned int init; + unsigned int max; + unsigned int uninit; +}; +static int +limit_lcores_init(unsigned int lcore_id __rte_unused, void *arg) +{ + struct limit_lcore_context *l = arg; + + l->init++; + if (l->init > l->max) + return -1; + return 0; +} +static void +limit_lcores_uninit(unsigned int lcore_id __rte_unused, void *arg) +{ + struct limit_lcore_context *l = arg; + + l->uninit++; +} + +static int +test_lcores_callback(unsigned int eal_threads_count) +{ + struct limit_lcore_context l; + void *handle; + + /* Refuse last lcore => callback register error. */ + memset(&l, 0, sizeof(l)); + l.max = eal_threads_count - 1; + handle = rte_lcore_callback_register("limit", limit_lcores_init, + limit_lcores_uninit, &l); + if (handle != NULL) { + printf("lcore callback register should have failed\n"); + goto error; + } + /* Refusal happens at the n th call to the init callback. + * Besides, n - 1 were accepted, so we expect as many uninit calls when + * the rollback happens. + */ + if (l.init != eal_threads_count) { + printf("lcore init calls failed: expected %u, got %u\n", + eal_threads_count, l.init); + goto error; + } + if (l.uninit != eal_threads_count - 1) { + printf("lcore uninit calls failed: expected %u, got %u\n", + eal_threads_count - 1, l.uninit); + goto error; + } + + /* Accept all lcore and unregister. */ + memset(&l, 0, sizeof(l)); + l.max = eal_threads_count; + handle = rte_lcore_callback_register("limit", limit_lcores_init, + limit_lcores_uninit, &l); + if (handle == NULL) { + printf("lcore callback register failed\n"); + goto error; + } + if (l.uninit != 0) { + printf("lcore uninit got called %u times during register\n", + l.uninit); + goto error; + } + rte_lcore_callback_unregister(handle); + handle = NULL; + if (l.init != eal_threads_count) { + printf("lcore init got called %u times during unregister (expected %u)\n", + l.init, eal_threads_count); + goto error; + } + if (l.uninit != eal_threads_count) { + printf("lcore uninit calls failed: expected %u, got %u\n", + eal_threads_count, l.uninit); + goto error; + } + + return 0; + +error: + if (handle != NULL) + rte_lcore_callback_unregister(handle); + + return -1; +} + static int test_lcores(void) { @@ -133,6 +223,11 @@ test_lcores(void) if (test_non_eal_lcores(eal_threads_count) < 0) return TEST_FAILED; + if (test_lcores_callback(eal_threads_count) < 0) + return TEST_FAILED; + + /* FIXME: missing a test on callback + registering non-EAL threads */ + return TEST_SUCCESS; } diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 7db05428e7..5140026b6c 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -223,11 +223,114 @@ rte_socket_id_by_idx(unsigned int idx) } static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; +struct lcore_callback { + TAILQ_ENTRY(lcore_callback) next; + char *name; + rte_lcore_init_cb init; + rte_lcore_uninit_cb uninit; + void *arg; +}; +static TAILQ_HEAD(lcore_callbacks_head, lcore_callback) lcore_callbacks = + TAILQ_HEAD_INITIALIZER(lcore_callbacks); + +static int +callback_init(struct lcore_callback *callback, unsigned int lcore_id) +{ + if (callback->init == NULL) + return 0; + RTE_LOG(DEBUG, EAL, "Call init for lcore callback %s, lcore_id %u\n", + callback->name, lcore_id); + return callback->init(lcore_id, callback->arg); +} + +static void +callback_uninit(struct lcore_callback *callback, unsigned int lcore_id) +{ + if (callback->uninit == NULL) + return; + RTE_LOG(DEBUG, EAL, "Call uninit for lcore callback %s, lcore_id %u\n", + callback->name, lcore_id); + callback->uninit(lcore_id, callback->arg); +} + +void * +rte_lcore_callback_register(const char *name, rte_lcore_init_cb init, + rte_lcore_uninit_cb uninit, void *arg) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback; + unsigned int lcore_id; + + callback = calloc(1, sizeof(*callback)); + if (callback == NULL) + return NULL; + if (asprintf(&callback->name, "%s-%p", name, arg) == -1) { + free(callback); + return NULL; + } + callback->init = init; + callback->uninit = uninit; + callback->arg = arg; + rte_spinlock_lock(&lcore_lock); + if (callback->init == NULL) + goto no_init; + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] == ROLE_OFF) + continue; + if (callback_init(callback, lcore_id) == 0) + continue; + /* Callback refused init for this lcore, uninitialize all + * previous lcore. + */ + for (; lcore_id != 0; lcore_id--) { + if (cfg->lcore_role[lcore_id - 1] == ROLE_OFF) + continue; + callback_uninit(callback, lcore_id - 1); + } + free(callback); + callback = NULL; + goto out; + } +no_init: + TAILQ_INSERT_TAIL(&lcore_callbacks, callback, next); + RTE_LOG(DEBUG, EAL, "Registered new lcore callback %s (%sinit, %suninit).\n", + callback->name, callback->init == NULL ? "NO " : "", + callback->uninit == NULL ? "NO " : ""); +out: + rte_spinlock_unlock(&lcore_lock); + return callback; +} + +void +rte_lcore_callback_unregister(void *handle) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback = handle; + unsigned int lcore_id; + + rte_spinlock_lock(&lcore_lock); + if (callback->uninit == NULL) + goto no_uninit; + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] == ROLE_OFF) + continue; + callback_uninit(callback, lcore_id); + } +no_uninit: + TAILQ_REMOVE(&lcore_callbacks, callback, next); + rte_spinlock_unlock(&lcore_lock); + RTE_LOG(DEBUG, EAL, "Unregistered lcore callback %s-%p.\n", + callback->name, callback->arg); + free(callback->name); + free(callback); +} unsigned int eal_lcore_non_eal_allocate(void) { struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback; + struct lcore_callback *prev; unsigned int lcore_id; rte_spinlock_lock(&lcore_lock); @@ -238,8 +341,30 @@ eal_lcore_non_eal_allocate(void) cfg->lcore_count++; break; } - if (lcore_id == RTE_MAX_LCORE) + if (lcore_id == RTE_MAX_LCORE) { RTE_LOG(DEBUG, EAL, "No lcore available.\n"); + goto out; + } + TAILQ_FOREACH(callback, &lcore_callbacks, next) { + if (callback_init(callback, lcore_id) == 0) + continue; + /* Callback refused init for this lcore, call uninit for all + * previous callbacks. + */ + prev = TAILQ_PREV(callback, lcore_callbacks_head, next); + while (prev != NULL) { + callback_uninit(prev, lcore_id); + prev = TAILQ_PREV(callback, lcore_callbacks_head, + next); + } + RTE_LOG(DEBUG, EAL, "Initialization refused for lcore %u.\n", + lcore_id); + cfg->lcore_role[lcore_id] = ROLE_OFF; + cfg->lcore_count--; + lcore_id = RTE_MAX_LCORE; + goto out; + } +out: rte_spinlock_unlock(&lcore_lock); return lcore_id; } @@ -248,11 +373,15 @@ void eal_lcore_non_eal_release(unsigned int lcore_id) { struct rte_config *cfg = rte_eal_get_configuration(); + struct lcore_callback *callback; rte_spinlock_lock(&lcore_lock); - if (cfg->lcore_role[lcore_id] == ROLE_NON_EAL) { - cfg->lcore_role[lcore_id] = ROLE_OFF; - cfg->lcore_count--; - } + if (cfg->lcore_role[lcore_id] != ROLE_NON_EAL) + goto out; + TAILQ_FOREACH(callback, &lcore_callbacks, next) + callback_uninit(callback, lcore_id); + cfg->lcore_role[lcore_id] = ROLE_OFF; + cfg->lcore_count--; +out: rte_spinlock_unlock(&lcore_lock); } diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 73238ff157..fc79564111 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -401,7 +401,8 @@ uint64_t get_tsc_freq_arch(void); * * @return * - the id of a lcore with role ROLE_NON_EAL on success. - * - RTE_MAX_LCORE if none was available. + * - RTE_MAX_LCORE if none was available or initializing was refused (see + * rte_lcore_callback_register). */ unsigned int eal_lcore_non_eal_allocate(void); diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index ea86220394..27b29a1f87 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -229,6 +229,74 @@ unsigned int rte_get_next_lcore(unsigned int i, int skip_master, int wrap); i X-Patchwork-Id: 71814 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 91455A051A; Fri, 19 Jun 2020 18:24:33 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 012241BF9C; Fri, 19 Jun 2020 18:23:45 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by dpdk.org (Postfix) with ESMTP id 64C511BFA9 for ; Fri, 19 Jun 2020 18:23:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583822; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VWVmp5bktRjZ30dT0ELFo0W+YDdJykbSyroZlCTQEDw=; b=fOiOvU3PpAQ4srNLYYv8mUjDAjNNKdHN1f5IkE3XpAuWTZOtNs54SWh/n6unaAQYOKcyOT P3uJU0y2m6CdIwEh3/IUHfYppMdh9YTsiEp4AaVd3o8H7HZdt+TyZBXDAJSrUux2omEEhQ iRDoqTjPza8+xU23ohdwvZjQ+NJx4QI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-152-ncAqGIHBMfWbx2uvVQF90Q-1; Fri, 19 Jun 2020 12:23:38 -0400 X-MC-Unique: ncAqGIHBMfWbx2uvVQF90Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7E0B61800D42; Fri, 19 Jun 2020 16:23:37 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id B484060E1C; Fri, 19 Jun 2020 16:23:34 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, Neil Horman Date: Fri, 19 Jun 2020 18:22:43 +0200 Message-Id: <20200619162244.8239-9-david.marchand@redhat.com> In-Reply-To: <20200619162244.8239-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200619162244.8239-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 8/9] eal: add lcore iterators X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a helper to iterate all lcores. The iterator callback is read-only wrt the lcores list. Implement a dump function on top of this for debugging. Signed-off-by: David Marchand --- Changes since v1: - introduced lcore iterators and implemented rte_lcore_dump, this iterator mechanism can then be used outside of EAL, --- lib/librte_eal/common/eal_common_lcore.c | 77 ++++++++++++++++++++--- lib/librte_eal/common/eal_common_thread.c | 16 +++-- lib/librte_eal/common/eal_thread.h | 13 +++- lib/librte_eal/freebsd/eal.c | 2 +- lib/librte_eal/freebsd/eal_thread.c | 2 +- lib/librte_eal/include/rte_lcore.h | 47 +++++++++++++- lib/librte_eal/linux/eal.c | 2 +- lib/librte_eal/linux/eal_thread.c | 2 +- lib/librte_eal/rte_eal_version.map | 2 + 9 files changed, 140 insertions(+), 23 deletions(-) diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c index 5140026b6c..2524356a88 100644 --- a/lib/librte_eal/common/eal_common_lcore.c +++ b/lib/librte_eal/common/eal_common_lcore.c @@ -12,7 +12,7 @@ #include #include #include -#include +#include #include "eal_private.h" #include "eal_thread.h" @@ -222,7 +222,7 @@ rte_socket_id_by_idx(unsigned int idx) return config->numa_nodes[idx]; } -static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; +static rte_rwlock_t lcore_lock = RTE_RWLOCK_INITIALIZER; struct lcore_callback { TAILQ_ENTRY(lcore_callback) next; char *name; @@ -271,7 +271,7 @@ rte_lcore_callback_register(const char *name, rte_lcore_init_cb init, callback->init = init; callback->uninit = uninit; callback->arg = arg; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); if (callback->init == NULL) goto no_init; for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { @@ -297,7 +297,7 @@ rte_lcore_callback_register(const char *name, rte_lcore_init_cb init, callback->name, callback->init == NULL ? "NO " : "", callback->uninit == NULL ? "NO " : ""); out: - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); return callback; } @@ -308,7 +308,7 @@ rte_lcore_callback_unregister(void *handle) struct lcore_callback *callback = handle; unsigned int lcore_id; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); if (callback->uninit == NULL) goto no_uninit; for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { @@ -318,7 +318,7 @@ rte_lcore_callback_unregister(void *handle) } no_uninit: TAILQ_REMOVE(&lcore_callbacks, callback, next); - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); RTE_LOG(DEBUG, EAL, "Unregistered lcore callback %s-%p.\n", callback->name, callback->arg); free(callback->name); @@ -333,7 +333,7 @@ eal_lcore_non_eal_allocate(void) struct lcore_callback *prev; unsigned int lcore_id; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { if (cfg->lcore_role[lcore_id] != ROLE_OFF) continue; @@ -365,7 +365,7 @@ eal_lcore_non_eal_allocate(void) goto out; } out: - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); return lcore_id; } @@ -375,7 +375,7 @@ eal_lcore_non_eal_release(unsigned int lcore_id) struct rte_config *cfg = rte_eal_get_configuration(); struct lcore_callback *callback; - rte_spinlock_lock(&lcore_lock); + rte_rwlock_write_lock(&lcore_lock); if (cfg->lcore_role[lcore_id] != ROLE_NON_EAL) goto out; TAILQ_FOREACH(callback, &lcore_callbacks, next) @@ -383,5 +383,62 @@ eal_lcore_non_eal_release(unsigned int lcore_id) cfg->lcore_role[lcore_id] = ROLE_OFF; cfg->lcore_count--; out: - rte_spinlock_unlock(&lcore_lock); + rte_rwlock_write_unlock(&lcore_lock); +} + +int +rte_lcore_iterate(rte_lcore_iterate_cb cb, void *arg) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + unsigned int lcore_id; + int ret = 0; + + rte_rwlock_read_lock(&lcore_lock); + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (cfg->lcore_role[lcore_id] == ROLE_OFF) + continue; + ret = cb(lcore_id, arg); + if (ret != 0) + break; + } + rte_rwlock_read_unlock(&lcore_lock); + return ret; +} + +static int +lcore_dump_cb(unsigned int lcore_id, void *arg) +{ + struct rte_config *cfg = rte_eal_get_configuration(); + char cpuset[RTE_CPU_AFFINITY_STR_LEN]; + const char *role; + FILE *f = arg; + int ret; + + switch (cfg->lcore_role[lcore_id]) { + case ROLE_RTE: + role = "RTE"; + break; + case ROLE_SERVICE: + role = "SERVICE"; + break; + case ROLE_NON_EAL: + role = "NON_EAL"; + break; + default: + role = "UNKNOWN"; + break; + } + + ret = eal_thread_dump_affinity(&lcore_config[lcore_id].cpuset, cpuset, + sizeof(cpuset)); + fprintf(f, "lcore %u, socket %u, role %s, cpuset %s%s\n", lcore_id, + rte_lcore_to_socket_id(lcore_id), role, cpuset, + ret == 0 ? "" : "..."); + return 0; +} + +void +rte_lcore_dump(FILE *f) +{ + rte_lcore_iterate(lcore_dump_cb, f); } diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 338194832b..5dc0b12f42 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -104,17 +104,14 @@ rte_thread_get_affinity(rte_cpuset_t *cpusetp) } int -eal_thread_dump_affinity(char *str, unsigned size) +eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size) { - rte_cpuset_t cpuset; unsigned cpu; int ret; unsigned int out = 0; - rte_thread_get_affinity(&cpuset); - for (cpu = 0; cpu < CPU_SETSIZE; cpu++) { - if (!CPU_ISSET(cpu, &cpuset)) + if (!CPU_ISSET(cpu, cpuset)) continue; ret = snprintf(str + out, @@ -137,6 +134,15 @@ eal_thread_dump_affinity(char *str, unsigned size) return ret; } +int +eal_thread_dump_current_affinity(char *str, unsigned int size) +{ + rte_cpuset_t cpuset; + + rte_thread_get_affinity(&cpuset); + return eal_thread_dump_affinity(&cpuset, str, size); +} + void rte_thread_init(unsigned int lcore_id, rte_cpuset_t *cpuset) { diff --git a/lib/librte_eal/common/eal_thread.h b/lib/librte_eal/common/eal_thread.h index 4ecd8fd53a..13ec252e01 100644 --- a/lib/librte_eal/common/eal_thread.h +++ b/lib/librte_eal/common/eal_thread.h @@ -47,13 +47,15 @@ unsigned eal_cpu_socket_id(unsigned cpu_id); #define RTE_CPU_AFFINITY_STR_LEN 256 /** - * Dump the current pthread cpuset. + * Dump the cpuset as a human readable string. * This function is private to EAL. * * Note: * If the dump size is greater than the size of given buffer, * the string will be truncated and with '\0' at the end. * + * @param cpuset + * The CPU affinity object to dump. * @param str * The string buffer the cpuset will dump to. * @param size @@ -62,6 +64,13 @@ unsigned eal_cpu_socket_id(unsigned cpu_id); * 0 for success, -1 if truncation happens. */ int -eal_thread_dump_affinity(char *str, unsigned size); +eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size); + +/** + * Dump the current thread cpuset. + * This is a wrapper on eal_thread_dump_affinity(). + */ +int +eal_thread_dump_current_affinity(char *str, unsigned int size); #endif /* EAL_THREAD_H */ diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index b5ea11df16..69a6f7d8c4 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -886,7 +886,7 @@ rte_eal_init(int argc, char **argv) rte_thread_init(rte_config.master_lcore, &lcore_config[rte_config.master_lcore].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "Master lcore %u is ready (tid=%p;cpuset=[%s%s])\n", rte_config.master_lcore, thread_id, cpuset, diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index c1fb8eb2d8..b1a3619f51 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -92,7 +92,7 @@ eal_thread_loop(__rte_unused void *arg) rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%p;cpuset=[%s%s])\n", lcore_id, thread_id, cpuset, ret == 0 ? "" : "..."); diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index 27b29a1f87..4dee7cbcd7 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -261,8 +261,8 @@ typedef void (*rte_lcore_uninit_cb)(unsigned int lcore_id, void *arg); * If this step succeeds, the callbacks are put in the lcore callbacks list * that will get called for each lcore allocation/release. * - * Note: callbacks execution is serialised under a lock protecting the lcores - * and callbacks list. + * Note: callbacks execution is serialised under a write lock protecting the + * lcores and callbacks list. * * @param name * A name serving as a small description for this callback. @@ -297,6 +297,49 @@ __rte_experimental void rte_lcore_callback_unregister(void *handle); +/** + * Callback prototype for iterating over lcores. + * + * @param lcore_id + * The lcore to consider. + * @param arg + * An opaque pointer coming from the caller. + * @return + * - 0 lets the iteration continue. + * - !0 makes the iteration stop. + */ +typedef int (*rte_lcore_iterate_cb)(unsigned int lcore_id, void *arg); + +/** + * Iterate on all active lcores (ROLE_RTE, ROLE_SERVICE and ROLE_NON_EAL). + * No modification on the lcore states is allowed in the callback. + * + * Note: as opposed to init/uninit callbacks, iteration callbacks can be + * invoked in parallel as they are run under a read lock protecting the lcores + * and callbacks list. + * + * @param cb + * The callback that gets passed each lcore. + * @param arg + * An opaque pointer passed to cb. + * @return + * Same return code as the callback last invocation (see rte_lcore_iterate_cb + * description). + */ +__rte_experimental +int +rte_lcore_iterate(rte_lcore_iterate_cb cb, void *arg); + +/** + * List all lcores. + * + * @param f + * The output stream where the dump should be sent. + */ +__rte_experimental +void +rte_lcore_dump(FILE *f); + /** * Set core affinity of the current thread. * Support both EAL and non-EAL thread and update TLS. diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 8638376b8a..2f0efd7cd3 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -1214,7 +1214,7 @@ rte_eal_init(int argc, char **argv) rte_thread_init(rte_config.master_lcore, &lcore_config[rte_config.master_lcore].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "Master lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", rte_config.master_lcore, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index 07aec0c44d..22d9bc8c01 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -92,7 +92,7 @@ eal_thread_loop(__rte_unused void *arg) rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); - ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", lcore_id, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index c3e762c1d9..3aeb5b11ab 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -397,6 +397,8 @@ EXPERIMENTAL { # added in 20.08 rte_lcore_callback_register; rte_lcore_callback_unregister; + rte_lcore_dump; + rte_lcore_iterate; rte_thread_register; rte_thread_unregister; }; From patchwork Fri Jun 19 16:22:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 71815 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 56669A051A; Fri, 19 Jun 2020 18:24:44 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 52AE81BFB2; Fri, 19 Jun 2020 18:23:50 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by dpdk.org (Postfix) with ESMTP id 4F7981BFA1 for ; Fri, 19 Jun 2020 18:23:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583827; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bEYsvCW+vyIix+YFDyxULkm2zyme4u/RiWUnIA/e5lI=; b=Np2TA7YY6OcdxHIPpmmagzWfhw59WHWmVupacDW0MwbFYRwALnuw9k1SNu3thZKk/foJNZ RadkI38OVaZWpd5KDIZDEMl7V4cqxX6Lu+YiWG0nTXawrwcykyX8xav8qG0UKKbxaMTV7o h4jYjsu8HR3Sea3rZSJOUEmYTbzsLrA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-52-sNGbzPoLOaiOivNsPIH7Ow-1; Fri, 19 Jun 2020 12:23:43 -0400 X-MC-Unique: sNGbzPoLOaiOivNsPIH7Ow-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8039D1800D42; Fri, 19 Jun 2020 16:23:41 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1E3B260CCC; Fri, 19 Jun 2020 16:23:37 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org, "Artem V. Andreev" , Andrew Rybchenko Date: Fri, 19 Jun 2020 18:22:44 +0200 Message-Id: <20200619162244.8239-10-david.marchand@redhat.com> In-Reply-To: <20200619162244.8239-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> <20200619162244.8239-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v2 9/9] mempool/bucket: handle non-EAL lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Convert to new lcore API to support non-EAL lcores. Signed-off-by: David Marchand --- drivers/mempool/bucket/rte_mempool_bucket.c | 131 ++++++++++++-------- 1 file changed, 82 insertions(+), 49 deletions(-) diff --git a/drivers/mempool/bucket/rte_mempool_bucket.c b/drivers/mempool/bucket/rte_mempool_bucket.c index 5ce1ef16fb..0b4f42d330 100644 --- a/drivers/mempool/bucket/rte_mempool_bucket.c +++ b/drivers/mempool/bucket/rte_mempool_bucket.c @@ -55,6 +55,7 @@ struct bucket_data { struct rte_ring *shared_orphan_ring; struct rte_mempool *pool; unsigned int bucket_mem_size; + void *lcore_callback_handle; }; static struct bucket_stack * @@ -345,6 +346,22 @@ bucket_dequeue_contig_blocks(struct rte_mempool *mp, void **first_obj_table, return 0; } +struct bucket_per_lcore_ctx { + const struct bucket_data *bd; + unsigned int count; +}; + +static int +count_per_lcore(unsigned int lcore_id, void *arg) +{ + struct bucket_per_lcore_ctx *ctx = arg; + + ctx->count += ctx->bd->obj_per_bucket * + ctx->bd->buckets[lcore_id]->top; + ctx->count += rte_ring_count(ctx->bd->adoption_buffer_rings[lcore_id]); + return 0; +} + static void count_underfilled_buckets(struct rte_mempool *mp, void *opaque, @@ -373,23 +390,66 @@ count_underfilled_buckets(struct rte_mempool *mp, static unsigned int bucket_get_count(const struct rte_mempool *mp) { - const struct bucket_data *bd = mp->pool_data; - unsigned int count = - bd->obj_per_bucket * rte_ring_count(bd->shared_bucket_ring) + - rte_ring_count(bd->shared_orphan_ring); - unsigned int i; + struct bucket_per_lcore_ctx ctx; - for (i = 0; i < RTE_MAX_LCORE; i++) { - if (!rte_lcore_is_enabled(i)) - continue; - count += bd->obj_per_bucket * bd->buckets[i]->top + - rte_ring_count(bd->adoption_buffer_rings[i]); - } + ctx.bd = mp->pool_data; + ctx.count = ctx.bd->obj_per_bucket * + rte_ring_count(ctx.bd->shared_bucket_ring); + ctx.count += rte_ring_count(ctx.bd->shared_orphan_ring); + rte_lcore_iterate(count_per_lcore, &ctx); rte_mempool_mem_iter((struct rte_mempool *)(uintptr_t)mp, - count_underfilled_buckets, &count); + count_underfilled_buckets, &ctx.count); + + return ctx.count; +} + +static int +bucket_init_per_lcore(unsigned int lcore_id, void *arg) +{ + char rg_name[RTE_RING_NAMESIZE]; + struct bucket_data *bd = arg; + struct rte_mempool *mp; + int rg_flags; + int rc; + + mp = bd->pool; + bd->buckets[lcore_id] = bucket_stack_create(mp, + mp->size / bd->obj_per_bucket); + if (bd->buckets[lcore_id] == NULL) + goto error; + + rc = snprintf(rg_name, sizeof(rg_name), RTE_MEMPOOL_MZ_FORMAT ".a%u", + mp->name, lcore_id); + if (rc < 0 || rc >= (int)sizeof(rg_name)) + goto error; + + rg_flags = RING_F_SC_DEQ; + if (mp->flags & MEMPOOL_F_SP_PUT) + rg_flags |= RING_F_SP_ENQ; + if (mp->flags & MEMPOOL_F_SC_GET) + rg_flags |= RING_F_SC_DEQ; + bd->adoption_buffer_rings[lcore_id] = rte_ring_create(rg_name, + rte_align32pow2(mp->size + 1), mp->socket_id, rg_flags); + if (bd->adoption_buffer_rings[lcore_id] == NULL) + goto error; - return count; + return 0; +error: + rte_free(bd->buckets[lcore_id]); + bd->buckets[lcore_id] = NULL; + return -1; +} + +static void +bucket_uninit_per_lcore(unsigned int lcore_id, void *arg) +{ + struct bucket_data *bd = arg; + + rte_ring_free(bd->adoption_buffer_rings[lcore_id]); + bd->adoption_buffer_rings[lcore_id] = NULL; + rte_free(bd->buckets[lcore_id]); + bd->buckets[lcore_id] = NULL; } static int @@ -399,7 +459,6 @@ bucket_alloc(struct rte_mempool *mp) int rc = 0; char rg_name[RTE_RING_NAMESIZE]; struct bucket_data *bd; - unsigned int i; unsigned int bucket_header_size; size_t pg_sz; @@ -429,36 +488,17 @@ bucket_alloc(struct rte_mempool *mp) /* eventually this should be a tunable parameter */ bd->bucket_stack_thresh = (mp->size / bd->obj_per_bucket) * 4 / 3; + bd->lcore_callback_handle = rte_lcore_callback_register("bucket", + bucket_init_per_lcore, bucket_uninit_per_lcore, bd); + if (bd->lcore_callback_handle == NULL) { + rc = -ENOMEM; + goto no_mem_for_stacks; + } + if (mp->flags & MEMPOOL_F_SP_PUT) rg_flags |= RING_F_SP_ENQ; if (mp->flags & MEMPOOL_F_SC_GET) rg_flags |= RING_F_SC_DEQ; - - for (i = 0; i < RTE_MAX_LCORE; i++) { - if (!rte_lcore_is_enabled(i)) - continue; - bd->buckets[i] = - bucket_stack_create(mp, mp->size / bd->obj_per_bucket); - if (bd->buckets[i] == NULL) { - rc = -ENOMEM; - goto no_mem_for_stacks; - } - rc = snprintf(rg_name, sizeof(rg_name), - RTE_MEMPOOL_MZ_FORMAT ".a%u", mp->name, i); - if (rc < 0 || rc >= (int)sizeof(rg_name)) { - rc = -ENAMETOOLONG; - goto no_mem_for_stacks; - } - bd->adoption_buffer_rings[i] = - rte_ring_create(rg_name, rte_align32pow2(mp->size + 1), - mp->socket_id, - rg_flags | RING_F_SC_DEQ); - if (bd->adoption_buffer_rings[i] == NULL) { - rc = -rte_errno; - goto no_mem_for_stacks; - } - } - rc = snprintf(rg_name, sizeof(rg_name), RTE_MEMPOOL_MZ_FORMAT ".0", mp->name); if (rc < 0 || rc >= (int)sizeof(rg_name)) { @@ -498,11 +538,8 @@ bucket_alloc(struct rte_mempool *mp) rte_ring_free(bd->shared_orphan_ring); cannot_create_shared_orphan_ring: invalid_shared_orphan_ring: + rte_lcore_callback_unregister(bd->lcore_callback_handle); no_mem_for_stacks: - for (i = 0; i < RTE_MAX_LCORE; i++) { - rte_free(bd->buckets[i]); - rte_ring_free(bd->adoption_buffer_rings[i]); - } rte_free(bd); no_mem_for_data: rte_errno = -rc; @@ -512,16 +549,12 @@ bucket_alloc(struct rte_mempool *mp) static void bucket_free(struct rte_mempool *mp) { - unsigned int i; struct bucket_data *bd = mp->pool_data; if (bd == NULL) return; - for (i = 0; i < RTE_MAX_LCORE; i++) { - rte_free(bd->buckets[i]); - rte_ring_free(bd->adoption_buffer_rings[i]); - } + rte_lcore_callback_unregister(bd->lcore_callback_handle); rte_ring_free(bd->shared_orphan_ring); rte_ring_free(bd->shared_bucket_ring);