[v2,3/3] examples/l3fwd-power: enable PMD power monitor on Arm
Checks
Commit Message
For Arm aarch, power monitor uses WFE instruction to enable, which can
not exit automatically within the time limit. This means
'rte_power_monitor_wakeup' API needs to be called to wake up sleep cores
if there is no store operation to monitored address.
Furthermore, we disable power monitor feature on the main core so that
it can be used to wake up other sleeping cores after it exit from loop.
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: David Hunt <david.hunt@intel.com>
---
examples/l3fwd-power/main.c | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
Comments
On Mon, 7 Nov 2022 15:04:49 +0800
Feifei Wang <feifei.wang2@arm.com> wrote:
> + /* Ensure the main lcore does not enter the power-monitor state,
> + * so that it can be used to wake up other lcores on ARM.
> + * This is due to WFE instruction has no timeout wake-up mechanism,
> + * and if users want to exit actively, the main lcore is needed
> + * to send SEV instruction to wake up other lcores.
> + */
> + unsigned int main_lcore = rte_get_main_lcore();
This can be done in a simpler an cleaner manner with a continue statement
earlier in the loop.
diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c
index fd3ade330f82..115535fd4cd7 100644
--- a/examples/l3fwd-power/main.c
+++ b/examples/l3fwd-power/main.c
@@ -2984,6 +2984,16 @@ main(int argc, char **argv)
}
if (app_mode == APP_MODE_PMD_MGMT && !baseline_enabled) {
+#ifdef RTE_ARCH_ARM64
+ /* Ensure the main lcore does not enter the power-monitor state,
+ * so that it can be used to wake up other lcores on ARM.
+ * This is due to WFE instruction has no timeout wake-up mechanism,
+ * and if users want to exit actively, the main lcore is needed
+ * to send SEV instruction to wake up other lcors.
+ */
+ if (lcore_id == rte_get_main_lcore())
+ continue;
+#endif
/* Set power_pmd_mgmt configs passed by user */
rte_power_pmd_mgmt_set_emptypoll_max(max_empty_polls);
ret = rte_power_pmd_mgmt_set_pause_duration(pause_duration);
Hi, Stephen
> -----邮件原件-----
> 发件人: Stephen Hemminger <stephen@networkplumber.org>
> 发送时间: Tuesday, November 8, 2022 12:02 AM
> 收件人: Feifei Wang <Feifei.Wang2@arm.com>
> 抄送: David Hunt <david.hunt@intel.com>; dev@dpdk.org;
> david.marchand@redhat.com; thomas@monjalon.net; nd <nd@arm.com>;
> Ruifeng Wang <Ruifeng.Wang@arm.com>
> 主题: Re: [PATCH v2 3/3] examples/l3fwd-power: enable PMD power
> monitor on Arm
>
> On Mon, 7 Nov 2022 15:04:49 +0800
> Feifei Wang <feifei.wang2@arm.com> wrote:
>
> > + /* Ensure the main lcore does not enter the
> power-monitor state,
> > + * so that it can be used to wake up other
> lcores on ARM.
> > + * This is due to WFE instruction has no
> timeout wake-up mechanism,
> > + * and if users want to exit actively, the main
> lcore is needed
> > + * to send SEV instruction to wake up other
> lcores.
> > + */
> > + unsigned int main_lcore =
> rte_get_main_lcore();
>
> This can be done in a simpler an cleaner manner with a continue statement
> earlier in the loop.
>
> diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c
> index fd3ade330f82..115535fd4cd7 100644
> --- a/examples/l3fwd-power/main.c
> +++ b/examples/l3fwd-power/main.c
> @@ -2984,6 +2984,16 @@ main(int argc, char **argv)
> }
>
> if (app_mode == APP_MODE_PMD_MGMT
> && !baseline_enabled) {
> +#ifdef RTE_ARCH_ARM64
> + /* Ensure the main lcore does not enter the
> power-monitor state,
> + * so that it can be used to wake up other
> lcores on ARM.
> + * This is due to WFE instruction has no
> timeout wake-up mechanism,
> + * and if users want to exit actively, the main
> lcore is needed
> + * to send SEV instruction to wake up other
> lcors.
> + */
> + if (lcore_id == rte_get_main_lcore())
> + continue;
> +#endif
> /* Set power_pmd_mgmt configs passed by
> user */
>
> rte_power_pmd_mgmt_set_emptypoll_max(max_empty_polls);
> ret =
> rte_power_pmd_mgmt_set_pause_duration(pause_duration);
Thanks for the comment.
There maybe some problems for this change. This is due to that we just want to disable power monitor
feature on the main core when "app_mode == APP_MODE_PMD_MGMT && pmgmt_type == RTE_POWER_MGMT_TYPE_MONITOR".
When “pmgmt_type == RTE_POWER_MGMT_TYPE_PAUSE || pmgmt_type == RTE_POWER_MGMT_TYPE_SCALE", main core
power management mode can be enabled.
Best Regards
Feifei
@@ -3006,6 +3006,25 @@ main(int argc, char **argv)
"Error setting scaling freq max: err=%d, lcore %d\n",
ret, lcore_id);
+#if defined(RTE_ARCH_ARM64)
+ /* Ensure the main lcore does not enter the power-monitor state,
+ * so that it can be used to wake up other lcores on ARM.
+ * This is due to WFE instruction has no timeout wake-up mechanism,
+ * and if users want to exit actively, the main lcore is needed
+ * to send SEV instruction to wake up other lcores.
+ */
+ unsigned int main_lcore = rte_get_main_lcore();
+ if (lcore_id != main_lcore ||
+ pmgmt_type != RTE_POWER_MGMT_TYPE_MONITOR) {
+ ret = rte_power_ethdev_pmgmt_queue_enable(
+ lcore_id, portid, queueid,
+ pmgmt_type);
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE,
+ "rte_power_ethdev_pmgmt_queue_enable: err=%d, port=%d\n",
+ ret, portid);
+ }
+#else
ret = rte_power_ethdev_pmgmt_queue_enable(
lcore_id, portid, queueid,
pmgmt_type);
@@ -3013,6 +3032,7 @@ main(int argc, char **argv)
rte_exit(EXIT_FAILURE,
"rte_power_ethdev_pmgmt_queue_enable: err=%d, port=%d\n",
ret, portid);
+#endif
}
}
}
@@ -3113,6 +3133,19 @@ main(int argc, char **argv)
if (app_mode == APP_MODE_EMPTY_POLL || app_mode == APP_MODE_TELEMETRY)
launch_timer(rte_lcore_id());
+ /* wake up all worker cores from sleeping state */
+ if (app_mode == APP_MODE_PMD_MGMT) {
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ if (rte_lcore_is_enabled(lcore_id) == 0)
+ continue;
+
+ if (lcore_id == rte_get_main_lcore())
+ continue;
+
+ rte_power_monitor_wakeup(lcore_id);
+ }
+ }
+
RTE_LCORE_FOREACH_WORKER(lcore_id) {
if (rte_eal_wait_lcore(lcore_id) < 0)
return -1;