From patchwork Tue Aug 22 16:11:47 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hunt, David" X-Patchwork-Id: 27722 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id E34A37D36; Tue, 22 Aug 2017 18:12:40 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id C7AD07D15 for ; Tue, 22 Aug 2017 18:12:37 +0200 (CEST) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Aug 2017 09:12:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.41,412,1498546800"; d="scan'208";a="142675193" Received: from silpixa00397898.ir.intel.com (HELO silpixa00397898.ger.corp.intel.com) ([10.237.223.116]) by fmsmga005.fm.intel.com with ESMTP; 22 Aug 2017 09:12:08 -0700 From: David Hunt To: dev@dpdk.org Cc: david.hunt@intel.com Date: Tue, 22 Aug 2017 17:11:47 +0100 Message-Id: <1503418310-162535-2-git-send-email-david.hunt@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1503418310-162535-1-git-send-email-david.hunt@intel.com> References: <1503418310-162535-1-git-send-email-david.hunt@intel.com> Subject: [dpdk-dev] [PATCH v1 1/4] lib/librte_power: add per-core turbo capability X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adds a new set of APIs to allow per-core turbo enable-disable. Signed-off-by: David Hunt --- lib/librte_power/channel_commands.h | 2 + lib/librte_power/rte_power.c | 9 ++ lib/librte_power/rte_power.h | 41 +++++++++ lib/librte_power/rte_power_acpi_cpufreq.c | 143 ++++++++++++++++++++++++++++++ lib/librte_power/rte_power_acpi_cpufreq.h | 40 +++++++++ lib/librte_power/rte_power_kvm_vm.c | 19 ++++ lib/librte_power/rte_power_kvm_vm.h | 35 +++++++- 7 files changed, 288 insertions(+), 1 deletion(-) diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h index 383897b..484085b 100644 --- a/lib/librte_power/channel_commands.h +++ b/lib/librte_power/channel_commands.h @@ -52,6 +52,8 @@ extern "C" { #define CPU_POWER_SCALE_DOWN 2 #define CPU_POWER_SCALE_MAX 3 #define CPU_POWER_SCALE_MIN 4 +#define CPU_POWER_ENABLE_TURBO 5 +#define CPU_POWER_DISABLE_TURBO 6 struct channel_packet { uint64_t resource_id; /**< core_num, device */ diff --git a/lib/librte_power/rte_power.c b/lib/librte_power/rte_power.c index 998ed1c..b327a86 100644 --- a/lib/librte_power/rte_power.c +++ b/lib/librte_power/rte_power.c @@ -50,6 +50,9 @@ rte_power_freq_change_t rte_power_freq_up = NULL; rte_power_freq_change_t rte_power_freq_down = NULL; rte_power_freq_change_t rte_power_freq_max = NULL; rte_power_freq_change_t rte_power_freq_min = NULL; +rte_power_freq_change_t rte_power_turbo_status; +rte_power_freq_change_t rte_power_freq_enable_turbo; +rte_power_freq_change_t rte_power_freq_disable_turbo; int rte_power_set_env(enum power_management_env env) @@ -65,6 +68,9 @@ rte_power_set_env(enum power_management_env env) rte_power_freq_down = rte_power_acpi_cpufreq_freq_down; rte_power_freq_min = rte_power_acpi_cpufreq_freq_min; rte_power_freq_max = rte_power_acpi_cpufreq_freq_max; + rte_power_turbo_status = rte_power_acpi_turbo_status; + rte_power_freq_enable_turbo = rte_power_acpi_enable_turbo; + rte_power_freq_disable_turbo = rte_power_acpi_disable_turbo; } else if (env == PM_ENV_KVM_VM) { rte_power_freqs = rte_power_kvm_vm_freqs; rte_power_get_freq = rte_power_kvm_vm_get_freq; @@ -73,6 +79,9 @@ rte_power_set_env(enum power_management_env env) rte_power_freq_down = rte_power_kvm_vm_freq_down; rte_power_freq_min = rte_power_kvm_vm_freq_min; rte_power_freq_max = rte_power_kvm_vm_freq_max; + rte_power_turbo_status = rte_power_kvm_vm_turbo_status; + rte_power_freq_enable_turbo = rte_power_kvm_vm_enable_turbo; + rte_power_freq_disable_turbo = rte_power_kvm_vm_disable_turbo; } else { RTE_LOG(ERR, POWER, "Invalid Power Management Environment(%d) set\n", env); diff --git a/lib/librte_power/rte_power.h b/lib/librte_power/rte_power.h index 67e0ec0..b17b7a5 100644 --- a/lib/librte_power/rte_power.h +++ b/lib/librte_power/rte_power.h @@ -236,6 +236,47 @@ extern rte_power_freq_change_t rte_power_freq_max; */ extern rte_power_freq_change_t rte_power_freq_min; +/** + * Query the Turbo Boost status of a specific lcore. + * Review each environments specific documentation for usage.. + * + * @param lcore_id + * lcore id. + * + * @return + * - 1 Turbo Boost is enabled for this lcore. + * - 0 Turbo Boost is disabled for this lcore. + * - Negative on error. + */ +extern rte_power_freq_change_t rte_power_turbo_status; + +/** + * Enable Turbo Boost for this lcore. + * Review each environments specific documentation for usage.. + * + * @param lcore_id + * lcore id. + * + * @return + * - 0 on success. + * - Negative on error. + */ +extern rte_power_freq_change_t rte_power_freq_enable_turbo; + +/** + * Disable Turbo Boost for this lcore. + * Review each environments specific documentation for usage.. + * + * @param lcore_id + * lcore id. + * + * @return + * - 0 on success. + * - Negative on error. + */ +extern rte_power_freq_change_t rte_power_freq_disable_turbo; + + #ifdef __cplusplus } #endif diff --git a/lib/librte_power/rte_power_acpi_cpufreq.c b/lib/librte_power/rte_power_acpi_cpufreq.c index a56c9b5..6695f59 100644 --- a/lib/librte_power/rte_power_acpi_cpufreq.c +++ b/lib/librte_power/rte_power_acpi_cpufreq.c @@ -87,6 +87,14 @@ #define POWER_SYSFILE_SETSPEED \ "/sys/devices/system/cpu/cpu%u/cpufreq/scaling_setspeed" +/* + * MSR related + */ +#define PLATFORM_INFO 0x0CE +#define TURBO_RATIO_LIMIT 0x1AD +#define IA32_PERF_CTL 0x199 +#define CORE_TURBO_DISABLE_BIT ((uint64_t)1<<32) + enum power_state { POWER_IDLE = 0, POWER_ONGOING, @@ -543,3 +551,138 @@ rte_power_acpi_cpufreq_freq_min(unsigned lcore_id) /* Frequencies in the array are from high to low. */ return set_freq_internal(pi, pi->nb_freqs - 1); } + + +static int +rdmsr(int lcore, int msr, uint64_t *val) +{ + char filename[32]; + int fd; + int retval; + + sprintf(filename, "/dev/cpu/%d/msr", lcore); + fd = open(filename, O_RDONLY); + if (fd < 0) + return fd; + + retval = pread(fd, val, sizeof(uint64_t), msr); + if (retval < 0) { + close(fd); + return retval; + } + close(fd); + return 0; +} + +static int +wrmsr(int lcore, int msr, uint64_t val) +{ + char filename[32]; + int fd; + int retval; + + sprintf(filename, "/dev/cpu/%d/msr", lcore); + fd = open(filename, O_WRONLY); + if (fd < 0) + return fd; + + retval = pwrite(fd, (void *)&val, sizeof(uint64_t), msr); + if (retval < 0) { + close(fd); + return retval; + } + close(fd); + return 0; +} + +int +rte_power_acpi_turbo_status(unsigned int lcore_id) +{ + uint64_t val; + int retval; + + if (lcore_id >= RTE_MAX_LCORE) { + RTE_LOG(ERR, POWER, "Invalid lcore ID\n"); + return -1; + } + +#if defined(RTE_ARCH_I686) || defined(RTE_ARCH_X86_64) + retval = rdmsr(lcore_id, IA32_PERF_CTL, &val); + if (retval) + return retval; + else + return(!(val & CORE_TURBO_DISABLE_BIT)); +#else + return 0 +#endif +} + + +int +rte_power_acpi_enable_turbo(unsigned int lcore_id) +{ + uint64_t val; + int retval; + + if (lcore_id >= RTE_MAX_LCORE) { + RTE_LOG(ERR, POWER, "Invalid lcore ID\n"); + return -1; + } + +#if defined(RTE_ARCH_I686) || defined(RTE_ARCH_X86_64) + /* + * The low byte of 1ADh MSR contains max recomended ratio when a small + * number of cores are active. Use this ratio when turbo is enabled. + */ + retval = rdmsr(lcore_id, TURBO_RATIO_LIMIT, &val); + if (retval) + return retval; + + val = (val & 0x00ff) << 8; /* Move to second lowest byte */ + val &= ~CORE_TURBO_DISABLE_BIT; /* Switch bit off to enable turbo */ + + retval = wrmsr(lcore_id, IA32_PERF_CTL, val); + if (retval) + return retval; + else + return 0; +#else + return 0; +#endif +} + +int +rte_power_acpi_disable_turbo(unsigned int lcore_id) +{ + uint64_t val; + int retval; + + if (lcore_id >= RTE_MAX_LCORE) { + RTE_LOG(ERR, POWER, "Invalid lcore ID\n"); + return -1; + } + +#if defined(RTE_ARCH_I686) || defined(RTE_ARCH_X86_64) + /* + * 0CEh MSR contains max non-turbo ratio in bits 8-15. Use this + * for the freq when turbo is disabled for that core. + */ + retval = rdmsr(lcore_id, PLATFORM_INFO, &val); + if (retval) + return retval; + + val = val & 0xff00; /* Only need second lowest byte */ + val |= CORE_TURBO_DISABLE_BIT; /* Switch bit on to disable turbo */ + + retval = wrmsr(lcore_id, IA32_PERF_CTL, val); + if (retval) + return retval; + + /* Try to set freq to max by default coming out of turbo */ + if (rte_power_acpi_cpufreq_freq_max(lcore_id) < 0) { + RTE_LOG(ERR, POWER, "Failed to set frequency of lcore %u to max\n", + lcore_id); + } +#endif + return 0; +} diff --git a/lib/librte_power/rte_power_acpi_cpufreq.h b/lib/librte_power/rte_power_acpi_cpufreq.h index 68578e9..eee0ca0 100644 --- a/lib/librte_power/rte_power_acpi_cpufreq.h +++ b/lib/librte_power/rte_power_acpi_cpufreq.h @@ -185,6 +185,46 @@ int rte_power_acpi_cpufreq_freq_max(unsigned lcore_id); */ int rte_power_acpi_cpufreq_freq_min(unsigned lcore_id); +/** + * Get the turbo status of a specific lcore. + * It should be protected outside of this function for threadsafe. + * + * @param lcore_id + * lcore id. + * + * @return + * - 1 Turbo Boost is enabled on this lcore. + * - 0 Turbo Boost is disabled on this lcore. + * - Negative on error. + */ +int rte_power_acpi_turbo_status(unsigned int lcore_id); + +/** + * Enable Turbo Boost on a specific lcore. + * It should be protected outside of this function for threadsafe. + * + * @param lcore_id + * lcore id. + * + * @return + * - 0 Turbo Boost is enabled successfully on this lcore. + * - Negative on error. + */ +int rte_power_acpi_enable_turbo(unsigned int lcore_id); + +/** + * Disable Turbo Boost on a specific lcore. + * It should be protected outside of this function for threadsafe. + * + * @param lcore_id + * lcore id. + * + * @return + * - 0 Turbo Boost disabled successfully on this lcore. + * - Negative on error. + */ +int rte_power_acpi_disable_turbo(unsigned int lcore_id); + #ifdef __cplusplus } #endif diff --git a/lib/librte_power/rte_power_kvm_vm.c b/lib/librte_power/rte_power_kvm_vm.c index a1badf3..9906062 100644 --- a/lib/librte_power/rte_power_kvm_vm.c +++ b/lib/librte_power/rte_power_kvm_vm.c @@ -134,3 +134,22 @@ rte_power_kvm_vm_freq_min(unsigned lcore_id) { return send_msg(lcore_id, CPU_POWER_SCALE_MIN); } + +int +rte_power_kvm_vm_turbo_status(__attribute__((unused)) unsigned int lcore_id) +{ + RTE_LOG(ERR, POWER, "rte_power_turbo_status is not implemented for Virtual Machine Power Management\n"); + return -ENOTSUP; +} + +int +rte_power_kvm_vm_enable_turbo(unsigned int lcore_id) +{ + return send_msg(lcore_id, CPU_POWER_ENABLE_TURBO); +} + +int +rte_power_kvm_vm_disable_turbo(unsigned int lcore_id) +{ + return send_msg(lcore_id, CPU_POWER_DISABLE_TURBO); +} diff --git a/lib/librte_power/rte_power_kvm_vm.h b/lib/librte_power/rte_power_kvm_vm.h index dcbc878..9af41d6 100644 --- a/lib/librte_power/rte_power_kvm_vm.h +++ b/lib/librte_power/rte_power_kvm_vm.h @@ -172,8 +172,41 @@ int rte_power_kvm_vm_freq_max(unsigned lcore_id); */ int rte_power_kvm_vm_freq_min(unsigned lcore_id); +/** + * It should be protected outside of this function for threadsafe. + * + * @param lcore_id + * lcore id. + * + * @return + * -ENOTSUP + */ +int rte_power_kvm_vm_turbo_status(unsigned int lcore_id); + +/** + * It should be protected outside of this function for threadsafe. + * + * @param lcore_id + * lcore id. + * + * @return + * - 1 on success. + * - Negative on error. + */ +int rte_power_kvm_vm_enable_turbo(unsigned int lcore_id); + +/** + * It should be protected outside of this function for threadsafe. + * + * @param lcore_id + * lcore id. + * + * @return + * - 1 on success. + * - Negative on error. + */ +int rte_power_kvm_vm_disable_turbo(unsigned int lcore_id); #ifdef __cplusplus } #endif - #endif