[v3,3/3] examples/l3fwd-power: enable PMD power monitor on Arm
Checks
Commit Message
For Arm aarch, power monitor uses WFE instruction to enable, which can
not exit automatically within the time limit. This means
'rte_power_monitor_wakeup' API needs to be called to wake up sleep cores
if there is no store operation to monitored address.
Furthermore, we disable power monitor feature on the main core so that
it can be used to wake up other sleeping cores after it exit from loop.
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: David Hunt <david.hunt@intel.com>
---
examples/l3fwd-power/main.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
Comments
11/11/2022 08:26, Feifei Wang:
> For Arm aarch, power monitor uses WFE instruction to enable, which can
> not exit automatically within the time limit. This means
> 'rte_power_monitor_wakeup' API needs to be called to wake up sleep cores
> if there is no store operation to monitored address.
>
> Furthermore, we disable power monitor feature on the main core so that
> it can be used to wake up other sleeping cores after it exit from loop.
>
> Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Acked-by: David Hunt <david.hunt@intel.com>
> ---
> +#if defined(RTE_ARCH_ARM64)
> + /* Ensure the main lcore does not enter the power-monitor state,
> + * so that it can be used to wake up other lcores on ARM.
> + * This is due to WFE instruction has no timeout wake-up mechanism,
> + * and if users want to exit actively, the main lcore is needed
> + * to send SEV instruction to wake up other lcores.
> + */
> + unsigned int main_lcore = rte_get_main_lcore();
> + if (lcore_id == main_lcore &&
> + pmgmt_type == RTE_POWER_MGMT_TYPE_MONITOR)
> + continue;
> +#endif
We need to have this logic abstracted in the API for all architectures.
We cannot afford having such #ifdef per CPU in the application.
Hi, Thomas
> -----邮件原件-----
> 发件人: Thomas Monjalon <thomas@monjalon.net>
> 发送时间: Friday, November 11, 2022 4:22 PM
> 收件人: Feifei Wang <Feifei.Wang2@arm.com>
> 抄送: David Hunt <david.hunt@intel.com>; dev@dpdk.org;
> david.marchand@redhat.com; stephen@networkplumber.org; nd
> <nd@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>
> 主题: Re: [PATCH v3 3/3] examples/l3fwd-power: enable PMD power
> monitor on Arm
>
> 11/11/2022 08:26, Feifei Wang:
> > For Arm aarch, power monitor uses WFE instruction to enable, which can
> > not exit automatically within the time limit. This means
> > 'rte_power_monitor_wakeup' API needs to be called to wake up sleep
> > cores if there is no store operation to monitored address.
> >
> > Furthermore, we disable power monitor feature on the main core so that
> > it can be used to wake up other sleeping cores after it exit from loop.
> >
> > Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> > Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Acked-by: David Hunt <david.hunt@intel.com>
> > ---
> > +#if defined(RTE_ARCH_ARM64)
> > + /* Ensure the main lcore does not enter the
> power-monitor state,
> > + * so that it can be used to wake up other
> lcores on ARM.
> > + * This is due to WFE instruction has no
> timeout wake-up mechanism,
> > + * and if users want to exit actively, the main
> lcore is needed
> > + * to send SEV instruction to wake up other
> lcores.
> > + */
> > + unsigned int main_lcore =
> rte_get_main_lcore();
> > + if (lcore_id == main_lcore &&
> > + pmgmt_type ==
> RTE_POWER_MGMT_TYPE_MONITOR)
> > + continue;
> > +#endif
>
> We need to have this logic abstracted in the API for all architectures.
> We cannot afford having such #ifdef per CPU in the application.
Thanks for the comment. I will move this into API.
Best Regards
Feifei
>
@@ -3006,6 +3006,18 @@ main(int argc, char **argv)
"Error setting scaling freq max: err=%d, lcore %d\n",
ret, lcore_id);
+#if defined(RTE_ARCH_ARM64)
+ /* Ensure the main lcore does not enter the power-monitor state,
+ * so that it can be used to wake up other lcores on ARM.
+ * This is due to WFE instruction has no timeout wake-up mechanism,
+ * and if users want to exit actively, the main lcore is needed
+ * to send SEV instruction to wake up other lcores.
+ */
+ unsigned int main_lcore = rte_get_main_lcore();
+ if (lcore_id == main_lcore &&
+ pmgmt_type == RTE_POWER_MGMT_TYPE_MONITOR)
+ continue;
+#endif
ret = rte_power_ethdev_pmgmt_queue_enable(
lcore_id, portid, queueid,
pmgmt_type);
@@ -3113,6 +3125,19 @@ main(int argc, char **argv)
if (app_mode == APP_MODE_EMPTY_POLL || app_mode == APP_MODE_TELEMETRY)
launch_timer(rte_lcore_id());
+ /* wake up all worker cores from sleeping state */
+ if (pmgmt_type == RTE_POWER_MGMT_TYPE_MONITOR) {
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ if (rte_lcore_is_enabled(lcore_id) == 0)
+ continue;
+
+ if (lcore_id == rte_get_main_lcore())
+ continue;
+
+ rte_power_monitor_wakeup(lcore_id);
+ }
+ }
+
RTE_LCORE_FOREACH_WORKER(lcore_id) {
if (rte_eal_wait_lcore(lcore_id) < 0)
return -1;