From patchwork Fri Aug 16 12:14:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Harris, James R" X-Patchwork-Id: 57730 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id ACB3F1BF21; Fri, 16 Aug 2019 21:19:27 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 091861BF19 for ; Fri, 16 Aug 2019 21:19:25 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Aug 2019 12:19:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,394,1559545200"; d="scan'208";a="377512597" Received: from jrharri1-skx.ch.intel.com (HELO [127.0.1.1]) ([143.182.137.73]) by fmsmga006.fm.intel.com with ESMTP; 16 Aug 2019 12:19:25 -0700 From: Jim Harris To: dev@dpdk.org, anatoly.burakov@intel.com Date: Fri, 16 Aug 2019 05:14:18 -0700 Message-ID: <156595765801.19075.15759465891004687257.stgit@jrharri1-skx> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v3 1/2] timer: use rte_mp_msg to pass TSC hz to secondary procs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" rte_eal_init() is much faster in secondary processes since hugepages don't need to be zeroed. But there's still non-trivial delays in the timer subsystem initialization due to the 100ms sleep used to calculate TSC hz. So use the rte_mp_msg framework to allow secondary processes to get the TSC hz from the primary process. This cuts rte_eal_init() execution time in a secondary process from 165ms to 66ms in my test program. Signed-off-by: Jim Harris --- lib/librte_eal/common/eal_common_timer.c | 70 +++++++++++++++++++++++++++++- 1 file changed, 68 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/eal_common_timer.c b/lib/librte_eal/common/eal_common_timer.c index 145543de7..3eb0309f5 100644 --- a/lib/librte_eal/common/eal_common_timer.c +++ b/lib/librte_eal/common/eal_common_timer.c @@ -15,9 +15,16 @@ #include #include #include +#include #include "eal_private.h" +#define EAL_TIMER_MP "eal_timer_mp_sync" + +struct timer_mp_param { + uint64_t tsc; +}; + /* The frequency of the RDTSC timer resolution */ static uint64_t eal_tsc_resolution_hz; @@ -74,8 +81,8 @@ estimate_tsc_freq(void) return RTE_ALIGN_MUL_NEAR(rte_rdtsc() - start, CYC_PER_10MHZ); } -void -set_tsc_freq(void) +static void +set_tsc_freq_primary(void) { uint64_t freq; @@ -89,6 +96,65 @@ set_tsc_freq(void) eal_tsc_resolution_hz = freq; } +static void +set_tsc_freq_secondary(void) +{ + struct rte_mp_msg mp_req = {0}; + struct rte_mp_reply mp_reply = {0}; + struct timer_mp_param *r; + struct timespec ts = {.tv_sec = 1, .tv_nsec = 0}; + + strcpy(mp_req.name, EAL_TIMER_MP); + if (rte_mp_request_sync(&mp_req, &mp_reply, &ts) || mp_reply.nb_received != 1) { + /* We weren't able to get the tsc hz from the primary process. So we will + * just calculate it here in the secondary process instead. + */ + set_tsc_freq_primary(); + free(mp_reply.msgs); + return; + } + + r = (struct timer_mp_param *)mp_reply.msgs[0].param; + eal_tsc_resolution_hz = r->tsc; + free(mp_reply.msgs); +} + +static int +timer_mp_primary(__attribute__((unused)) const struct rte_mp_msg *msg, const void *peer) +{ + struct rte_mp_msg reply; + struct timer_mp_param *r = (struct timer_mp_param *)reply.param; + + memset(&reply, 0, sizeof(reply)); + r->tsc = eal_tsc_resolution_hz; + strcpy(reply.name, EAL_TIMER_MP); + reply.len_param = sizeof(*r); + + return rte_mp_reply(&reply, peer); +} + +void +set_tsc_freq(void) +{ + int rc; + + /* We use a 100ms timer to calculate the TSC hz. We can save this 100ms in + * secondary processes, by getting the TSC hz from the primary process. + * So register an mp_action callback in the primary process, which secondary + * processes will use to get the TSC hz. + */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + set_tsc_freq_primary(); + rc = rte_mp_action_register(EAL_TIMER_MP, timer_mp_primary); + if (rc) { + RTE_LOG(WARNING, EAL, "Could not register mp_action - secondary " + " processes will calculate TSC independently.\n"); + } + } else { + set_tsc_freq_secondary(); + } +} + void rte_delay_us_callback_register(void (*userfunc)(unsigned int)) { rte_delay_us = userfunc; From patchwork Fri Aug 16 12:14:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Harris, James R" X-Patchwork-Id: 57731 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A5F6B1BF15; Fri, 16 Aug 2019 21:19:33 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 4C9071BF0D for ; Fri, 16 Aug 2019 21:19:31 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Aug 2019 12:19:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,394,1559545200"; d="scan'208";a="377512622" Received: from jrharri1-skx.ch.intel.com (HELO [127.0.1.1]) ([143.182.137.73]) by fmsmga006.fm.intel.com with ESMTP; 16 Aug 2019 12:19:30 -0700 From: Jim Harris To: dev@dpdk.org, anatoly.burakov@intel.com Date: Fri, 16 Aug 2019 05:14:23 -0700 Message-ID: <156595766333.19075.11387685364063679396.stgit@jrharri1-skx> In-Reply-To: <156595765801.19075.15759465891004687257.stgit@jrharri1-skx> References: <156595765801.19075.15759465891004687257.stgit@jrharri1-skx> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v3 2/2] timer: don't check tsc flags in secondary processes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" check_tsc_flags() parses /proc/cpuinfo and prints warning messages if any cores don't have constant_tsc and nonstop_tsc. It has no functional meaning. This consumes a noticeable amount of time in secondary processes - on my test system, it consumes 21ms out of the 66ms total execution time for rte_eal_init(). So let's just skip checking these flags in secondary processes. Since the primary process is already parsing the entirety of /proc/cpuinfo, the warning printed in the primary process should be sufficient. Signed-off-by: Jim Harris --- lib/librte_eal/linux/eal/eal_timer.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/lib/librte_eal/linux/eal/eal_timer.c b/lib/librte_eal/linux/eal/eal_timer.c index 76ec17034..ce447d43b 100644 --- a/lib/librte_eal/linux/eal/eal_timer.c +++ b/lib/librte_eal/linux/eal/eal_timer.c @@ -198,6 +198,15 @@ check_tsc_flags(void) char line[512]; FILE *stream; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* This function just prints warnings if TSC is not constant + * and has no functional meaning. It also checks *all* cores + * on the system, not just the ones configured for this process. + * So don't bother rechecking again in secondary processes. + */ + return; + } + stream = fopen("/proc/cpuinfo", "r"); if (!stream) { RTE_LOG(WARNING, EAL, "WARNING: Unable to open /proc/cpuinfo\n");