From patchwork Fri Nov 11 10:20:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 119800 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C78D6A0542; Fri, 11 Nov 2022 11:20:53 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7D2B242D0E; Fri, 11 Nov 2022 11:20:52 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id B68FA42D0D for ; Fri, 11 Nov 2022 11:20:50 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 72FDF1FB; Fri, 11 Nov 2022 02:20:56 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.116]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A85463F73D; Fri, 11 Nov 2022 02:20:47 -0800 (PST) From: Feifei Wang To: Ruifeng Wang Cc: dev@dpdk.org, david.hunt@intel.com, david.marchand@redhat.com, thomas@monjalon.net, stephen@networkplumber.org, nd@arm.com, Feifei Wang Subject: [PATCH v4 1/4] eal: add 8 bits case for wait scheme Date: Fri, 11 Nov 2022 18:20:36 +0800 Message-Id: <20221111102039.3567666-2-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221111102039.3567666-1-feifei.wang2@arm.com> References: <20220825064251.2637274-1-feifei.wang2@arm.com> <20221111102039.3567666-1-feifei.wang2@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For wait scheme generic helper, add 8 bits case. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang --- lib/eal/arm/include/rte_pause_64.h | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/lib/eal/arm/include/rte_pause_64.h b/lib/eal/arm/include/rte_pause_64.h index fe4d42b1ea..c21600ca96 100644 --- a/lib/eal/arm/include/rte_pause_64.h +++ b/lib/eal/arm/include/rte_pause_64.h @@ -31,6 +31,25 @@ static inline void rte_pause(void) /* Put processor into low power WFE(Wait For Event) state. */ #define __RTE_ARM_WFE() { asm volatile("wfe" : : : "memory"); } +/* + * Atomic exclusive load from addr, it returns the 8-bit content of + * *addr while making it 'monitored', when it is written by someone + * else, the 'monitored' state is cleared and an event is generated + * implicitly to exit WFE. + */ +#define __RTE_ARM_LOAD_EXC_8(src, dst, memorder) { \ + if (memorder == __ATOMIC_RELAXED) { \ + asm volatile("ldxrb %w[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r" (src) \ + : "memory"); \ + } else { \ + asm volatile("ldaxrb %w[tmp], [%x[addr]]" \ + : [tmp] "=&r" (dst) \ + : [addr] "r" (src) \ + : "memory"); \ + } } + /* * Atomic exclusive load from addr, it returns the 16-bit content of * *addr while making it 'monitored', when it is written by someone @@ -111,9 +130,11 @@ static inline void rte_pause(void) } } \ #define __RTE_ARM_LOAD_EXC(src, dst, memorder, size) { \ - RTE_BUILD_BUG_ON(size != 16 && size != 32 && \ - size != 64 && size != 128); \ - if (size == 16) \ + RTE_BUILD_BUG_ON(size != 8 && size != 16 && \ + size != 32 && size != 64 && size != 128); \ + if (size == 8) \ + __RTE_ARM_LOAD_EXC_8(src, dst, memorder) \ + else if (size == 16) \ __RTE_ARM_LOAD_EXC_16(src, dst, memorder) \ else if (size == 32) \ __RTE_ARM_LOAD_EXC_32(src, dst, memorder) \ From patchwork Fri Nov 11 10:20:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 119801 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5047CA0542; Fri, 11 Nov 2022 11:20:58 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 67C1742D17; Fri, 11 Nov 2022 11:20:55 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id EB11F42D17 for ; Fri, 11 Nov 2022 11:20:53 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9F3C323A; Fri, 11 Nov 2022 02:20:59 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.116]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D34EE3F73D; Fri, 11 Nov 2022 02:20:50 -0800 (PST) From: Feifei Wang To: Ruifeng Wang Cc: dev@dpdk.org, david.hunt@intel.com, david.marchand@redhat.com, thomas@monjalon.net, stephen@networkplumber.org, nd@arm.com, Feifei Wang Subject: [PATCH v4 2/4] eal: add power mgmt support on Arm Date: Fri, 11 Nov 2022 18:20:37 +0800 Message-Id: <20221111102039.3567666-3-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221111102039.3567666-1-feifei.wang2@arm.com> References: <20220825064251.2637274-1-feifei.wang2@arm.com> <20221111102039.3567666-1-feifei.wang2@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For Arm aarch, use WFE instruction to enable power monitor API, and use SEV instruction to enable wake up API. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang --- lib/eal/arm/include/rte_pause_64.h | 5 ++- lib/eal/arm/rte_cpuflags.c | 5 +++ lib/eal/arm/rte_power_intrinsics.c | 72 ++++++++++++++++++++++++++++-- 3 files changed, 78 insertions(+), 4 deletions(-) diff --git a/lib/eal/arm/include/rte_pause_64.h b/lib/eal/arm/include/rte_pause_64.h index c21600ca96..5f70e97481 100644 --- a/lib/eal/arm/include/rte_pause_64.h +++ b/lib/eal/arm/include/rte_pause_64.h @@ -25,9 +25,12 @@ static inline void rte_pause(void) #ifdef RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED -/* Send an event to quit WFE. */ +/* Send a local event to quit WFE. */ #define __RTE_ARM_SEVL() { asm volatile("sevl" : : : "memory"); } +/* Send a global event to quit WFE for all cores. */ +#define __RTE_ARM_SEV() { asm volatile("sev" : : : "memory"); } + /* Put processor into low power WFE(Wait For Event) state. */ #define __RTE_ARM_WFE() { asm volatile("wfe" : : : "memory"); } diff --git a/lib/eal/arm/rte_cpuflags.c b/lib/eal/arm/rte_cpuflags.c index 93461191c7..90b80709fd 100644 --- a/lib/eal/arm/rte_cpuflags.c +++ b/lib/eal/arm/rte_cpuflags.c @@ -163,4 +163,9 @@ void rte_cpu_get_intrinsics_support(struct rte_cpu_intrinsics *intrinsics) { memset(intrinsics, 0, sizeof(*intrinsics)); + +#ifdef RTE_ARM_USE_WFE + intrinsics->power_monitor = 1; +#endif + } diff --git a/lib/eal/arm/rte_power_intrinsics.c b/lib/eal/arm/rte_power_intrinsics.c index 13f6a3264d..d7d8d7af2f 100644 --- a/lib/eal/arm/rte_power_intrinsics.c +++ b/lib/eal/arm/rte_power_intrinsics.c @@ -6,17 +6,75 @@ #include "rte_power_intrinsics.h" +#ifdef RTE_ARM_USE_WFE +static inline int +__check_val_size(const uint8_t sz) +{ + switch (sz) { + case sizeof(uint8_t): /* fall-through */ + case sizeof(uint16_t): /* fall-through */ + case sizeof(uint32_t): /* fall-through */ + case sizeof(uint64_t): /* fall-through */ + return 0; + default: + /* unexpected size */ + return -1; + } +} +#endif + /** - * This function is not supported on ARM. + * This function uses WFE instruction to make lcore suspend + * execution on ARM. + * Note that timestamp based timeout is not supported yet. */ int rte_power_monitor(const struct rte_power_monitor_cond *pmc, const uint64_t tsc_timestamp) { - RTE_SET_USED(pmc); RTE_SET_USED(tsc_timestamp); +#ifdef RTE_ARM_USE_WFE + const unsigned int lcore_id = rte_lcore_id(); + uint64_t cur_value; + + /* prevent non-EAL thread from using this API */ + if (lcore_id >= RTE_MAX_LCORE) + return -EINVAL; + + if (pmc == NULL) + return -EINVAL; + + if (__check_val_size(pmc->size) < 0) + return -EINVAL; + + if (pmc->fn == NULL) + return -EINVAL; + + switch (pmc->size) { + case sizeof(uint8_t): + __RTE_ARM_LOAD_EXC_8(pmc->addr, cur_value, __ATOMIC_RELAXED); + __RTE_ARM_WFE() + break; + case sizeof(uint16_t): + __RTE_ARM_LOAD_EXC_16(pmc->addr, cur_value, __ATOMIC_RELAXED); + __RTE_ARM_WFE() + break; + case sizeof(uint32_t): + __RTE_ARM_LOAD_EXC_32(pmc->addr, cur_value, __ATOMIC_RELAXED); + __RTE_ARM_WFE() + break; + case sizeof(uint64_t): + __RTE_ARM_LOAD_EXC_64(pmc->addr, cur_value, __ATOMIC_RELAXED); + __RTE_ARM_WFE() + } + + return 0; +#else + RTE_SET_USED(pmc); + return -ENOTSUP; +#endif } /** @@ -31,14 +89,22 @@ rte_power_pause(const uint64_t tsc_timestamp) } /** - * This function is not supported on ARM. + * This function uses SEV instruction to wake up all cores + * on ARM. + * Note that lcore_id is not used here. */ int rte_power_monitor_wakeup(const unsigned int lcore_id) { RTE_SET_USED(lcore_id); +#ifdef RTE_ARM_USE_WFE + __RTE_ARM_SEV() + + return 0; +#else return -ENOTSUP; +#endif } int From patchwork Fri Nov 11 10:20:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 119802 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 07B74A0542; Fri, 11 Nov 2022 11:21:03 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4B08942D1E; Fri, 11 Nov 2022 11:20:58 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 176FD42D1E for ; Fri, 11 Nov 2022 11:20:57 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CA8C61FB; Fri, 11 Nov 2022 02:21:02 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.116]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0A77F3F73D; Fri, 11 Nov 2022 02:20:53 -0800 (PST) From: Feifei Wang To: David Hunt Cc: dev@dpdk.org, david.marchand@redhat.com, thomas@monjalon.net, stephen@networkplumber.org, nd@arm.com, Feifei Wang , Ruifeng Wang Subject: [PATCH v4 3/4] power: add power monitor support check Date: Fri, 11 Nov 2022 18:20:38 +0800 Message-Id: <20221111102039.3567666-4-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221111102039.3567666-1-feifei.wang2@arm.com> References: <20220825064251.2637274-1-feifei.wang2@arm.com> <20221111102039.3567666-1-feifei.wang2@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On ARM, WFE instruction, which is used to power monitor, has no timeout wake-up mechanism, and if users want to exit power monitor actively, the main lcore is needed to send SEV instruction to wake up other lcores. So it is necessary to ensure the main lcore does not enter the power-monitor state. To solve this, main core support check on ARM is added. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang --- lib/power/rte_power_pmd_mgmt.c | 12 ++++++++++++ lib/power/rte_power_pmd_mgmt.h | 1 + 2 files changed, 13 insertions(+) diff --git a/lib/power/rte_power_pmd_mgmt.c b/lib/power/rte_power_pmd_mgmt.c index ca1840387c..3f3478b2c1 100644 --- a/lib/power/rte_power_pmd_mgmt.c +++ b/lib/power/rte_power_pmd_mgmt.c @@ -490,6 +490,18 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lcore_id, uint16_t port_id, rte_rx_callback_fn clb; int ret; +#if defined(RTE_ARCH_ARM64) + /* Ensure the main lcore does not enter the power-monitor state, + * so that it can be used to wake up other lcores on ARM. + * This is due to WFE instruction has no timeout wake-up mechanism, + * and if users want to exit actively, the main lcore is needed + * to send SEV instruction to wake up other lcores. + */ + if (lcore_id == rte_get_main_lcore() && + mode == RTE_POWER_MGMT_TYPE_MONITOR) + return EPERM; +#endif + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); if (queue_id >= RTE_MAX_QUEUES_PER_PORT || lcore_id >= RTE_MAX_LCORE) { diff --git a/lib/power/rte_power_pmd_mgmt.h b/lib/power/rte_power_pmd_mgmt.h index 7ae6ef2d32..b1a0989b88 100644 --- a/lib/power/rte_power_pmd_mgmt.h +++ b/lib/power/rte_power_pmd_mgmt.h @@ -52,6 +52,7 @@ enum rte_power_pmd_mgmt_type { * The power management scheme to use for specified Rx queue. * @return * 0 on success + * EPERM main core cannot be supported on ARM * <0 on error */ __rte_experimental From patchwork Fri Nov 11 10:20:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 119803 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A6719A0542; Fri, 11 Nov 2022 11:21:07 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3E7CC42D25; Fri, 11 Nov 2022 11:21:01 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 42F5642D28 for ; Fri, 11 Nov 2022 11:21:00 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F39451FB; Fri, 11 Nov 2022 02:21:05 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.116]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 344E23F73D; Fri, 11 Nov 2022 02:20:56 -0800 (PST) From: Feifei Wang To: David Hunt Cc: dev@dpdk.org, david.marchand@redhat.com, thomas@monjalon.net, stephen@networkplumber.org, nd@arm.com, Feifei Wang , Ruifeng Wang Subject: [PATCH v4 4/4] examples/l3fwd-power: add power monitor wake up API Date: Fri, 11 Nov 2022 18:20:39 +0800 Message-Id: <20221111102039.3567666-5-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221111102039.3567666-1-feifei.wang2@arm.com> References: <20220825064251.2637274-1-feifei.wang2@arm.com> <20221111102039.3567666-1-feifei.wang2@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For ARM aarch, power monitor uses WFE instruction to enable, which can not exit automatically within the time limit. This means 'rte_power_monitor_wakeup' API needs to be called to wake up sleep cores if there is no store operation to monitored address. For other aarch, 'rte_power_monitor_wakeup' API can also make sure all lcores wake up from sleeping state. Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Acked-by: David Hunt --- examples/l3fwd-power/main.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index fd3ade330f..a47a91ce6f 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -3113,6 +3113,19 @@ main(int argc, char **argv) if (app_mode == APP_MODE_EMPTY_POLL || app_mode == APP_MODE_TELEMETRY) launch_timer(rte_lcore_id()); + /* wake up all worker cores from sleeping state */ + if (pmgmt_type == RTE_POWER_MGMT_TYPE_MONITOR) { + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + if (lcore_id == rte_get_main_lcore()) + continue; + + rte_power_monitor_wakeup(lcore_id); + } + } + RTE_LCORE_FOREACH_WORKER(lcore_id) { if (rte_eal_wait_lcore(lcore_id) < 0) return -1;