[v2,3/3] examples/l3fwd-power: enable PMD power monitor on Arm

Message ID 20221107070449.3319078-4-feifei.wang2@arm.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series Enable PMD power management on Arm |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-aarch64-unit-testing success Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-x86_64-unit-testing success Testing PASS
ci/iol-x86_64-compile-testing success Testing PASS
ci/intel-Testing success Testing PASS
ci/github-robot: build success github build: passed
ci/iol-aarch64-compile-testing success Testing PASS

Commit Message

Feifei Wang Nov. 7, 2022, 7:04 a.m. UTC
  For Arm aarch, power monitor uses WFE instruction to enable, which can
not exit automatically within the time limit. This means
'rte_power_monitor_wakeup' API needs to be called to wake up sleep cores
if there is no store operation to monitored address.

Furthermore, we disable power monitor feature on the main core so that
it can be used to wake up other sleeping cores after it exit from loop.

Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: David Hunt <david.hunt@intel.com>
---
 examples/l3fwd-power/main.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
  

Comments

Stephen Hemminger Nov. 7, 2022, 4:01 p.m. UTC | #1
On Mon,  7 Nov 2022 15:04:49 +0800
Feifei Wang <feifei.wang2@arm.com> wrote:

> +				/* Ensure the main lcore does not enter the power-monitor state,
> +				 * so that it can be used to wake up other lcores on ARM.
> +				 * This is due to WFE instruction has no timeout wake-up mechanism,
> +				 * and if users want to exit actively, the main lcore is needed
> +				 * to send SEV instruction to wake up other lcores.
> +				 */
> +				unsigned int main_lcore = rte_get_main_lcore();

This can be done in a simpler an cleaner manner with a continue statement
earlier in the loop.

diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c
index fd3ade330f82..115535fd4cd7 100644
--- a/examples/l3fwd-power/main.c
+++ b/examples/l3fwd-power/main.c
@@ -2984,6 +2984,16 @@ main(int argc, char **argv)
 			}
 
 			if (app_mode == APP_MODE_PMD_MGMT && !baseline_enabled) {
+#ifdef RTE_ARCH_ARM64
+				/* Ensure the main lcore does not enter the power-monitor state,
+				 * so that it can be used to wake up other lcores on ARM.
+				 * This is due to WFE instruction has no timeout wake-up mechanism,
+				 * and if users want to exit actively, the main lcore is needed
+				 * to send SEV instruction to wake up other lcors.
+				 */
+				if (lcore_id == rte_get_main_lcore())
+					continue;
+#endif
 				/* Set power_pmd_mgmt configs passed by user */
 				rte_power_pmd_mgmt_set_emptypoll_max(max_empty_polls);
 				ret = rte_power_pmd_mgmt_set_pause_duration(pause_duration);
  
Feifei Wang Nov. 8, 2022, 3:25 a.m. UTC | #2
Hi, Stephen

> -----邮件原件-----
> 发件人: Stephen Hemminger <stephen@networkplumber.org>
> 发送时间: Tuesday, November 8, 2022 12:02 AM
> 收件人: Feifei Wang <Feifei.Wang2@arm.com>
> 抄送: David Hunt <david.hunt@intel.com>; dev@dpdk.org;
> david.marchand@redhat.com; thomas@monjalon.net; nd <nd@arm.com>;
> Ruifeng Wang <Ruifeng.Wang@arm.com>
> 主题: Re: [PATCH v2 3/3] examples/l3fwd-power: enable PMD power
> monitor on Arm
> 
> On Mon,  7 Nov 2022 15:04:49 +0800
> Feifei Wang <feifei.wang2@arm.com> wrote:
> 
> > +				/* Ensure the main lcore does not enter the
> power-monitor state,
> > +				 * so that it can be used to wake up other
> lcores on ARM.
> > +				 * This is due to WFE instruction has no
> timeout wake-up mechanism,
> > +				 * and if users want to exit actively, the main
> lcore is needed
> > +				 * to send SEV instruction to wake up other
> lcores.
> > +				 */
> > +				unsigned int main_lcore =
> rte_get_main_lcore();
> 
> This can be done in a simpler an cleaner manner with a continue statement
> earlier in the loop.
> 
> diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c
> index fd3ade330f82..115535fd4cd7 100644
> --- a/examples/l3fwd-power/main.c
> +++ b/examples/l3fwd-power/main.c
> @@ -2984,6 +2984,16 @@ main(int argc, char **argv)
>  			}
> 
>  			if (app_mode == APP_MODE_PMD_MGMT
> && !baseline_enabled) {
> +#ifdef RTE_ARCH_ARM64
> +				/* Ensure the main lcore does not enter the
> power-monitor state,
> +				 * so that it can be used to wake up other
> lcores on ARM.
> +				 * This is due to WFE instruction has no
> timeout wake-up mechanism,
> +				 * and if users want to exit actively, the main
> lcore is needed
> +				 * to send SEV instruction to wake up other
> lcors.
> +				 */
> +				if (lcore_id == rte_get_main_lcore())
> +					continue;
> +#endif
>  				/* Set power_pmd_mgmt configs passed by
> user */
> 
> 	rte_power_pmd_mgmt_set_emptypoll_max(max_empty_polls);
>  				ret =
> rte_power_pmd_mgmt_set_pause_duration(pause_duration);

Thanks for the comment.
There maybe some problems for this change. This is due to that we just want to disable power monitor
feature on the main core when "app_mode == APP_MODE_PMD_MGMT && pmgmt_type == RTE_POWER_MGMT_TYPE_MONITOR".
When “pmgmt_type == RTE_POWER_MGMT_TYPE_PAUSE || pmgmt_type == RTE_POWER_MGMT_TYPE_SCALE", main core
power management mode can be enabled.

Best Regards
Feifei
  

Patch

diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c
index fd3ade330f..ffd2ad686a 100644
--- a/examples/l3fwd-power/main.c
+++ b/examples/l3fwd-power/main.c
@@ -3006,6 +3006,25 @@  main(int argc, char **argv)
 						"Error setting scaling freq max: err=%d, lcore %d\n",
 							ret, lcore_id);
 
+#if defined(RTE_ARCH_ARM64)
+				/* Ensure the main lcore does not enter the power-monitor state,
+				 * so that it can be used to wake up other lcores on ARM.
+				 * This is due to WFE instruction has no timeout wake-up mechanism,
+				 * and if users want to exit actively, the main lcore is needed
+				 * to send SEV instruction to wake up other lcores.
+				 */
+				unsigned int main_lcore = rte_get_main_lcore();
+				if (lcore_id != main_lcore ||
+						pmgmt_type != RTE_POWER_MGMT_TYPE_MONITOR) {
+					ret = rte_power_ethdev_pmgmt_queue_enable(
+							lcore_id, portid, queueid,
+							pmgmt_type);
+					if (ret < 0)
+						rte_exit(EXIT_FAILURE,
+							"rte_power_ethdev_pmgmt_queue_enable: err=%d, port=%d\n",
+							ret, portid);
+				}
+#else
 				ret = rte_power_ethdev_pmgmt_queue_enable(
 						lcore_id, portid, queueid,
 						pmgmt_type);
@@ -3013,6 +3032,7 @@  main(int argc, char **argv)
 					rte_exit(EXIT_FAILURE,
 						"rte_power_ethdev_pmgmt_queue_enable: err=%d, port=%d\n",
 							ret, portid);
+#endif
 			}
 		}
 	}
@@ -3113,6 +3133,19 @@  main(int argc, char **argv)
 	if (app_mode == APP_MODE_EMPTY_POLL || app_mode == APP_MODE_TELEMETRY)
 		launch_timer(rte_lcore_id());
 
+	/* wake up all worker cores from sleeping state */
+	if (app_mode == APP_MODE_PMD_MGMT) {
+		for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+			if (rte_lcore_is_enabled(lcore_id) == 0)
+				continue;
+
+			if (lcore_id == rte_get_main_lcore())
+				continue;
+
+			rte_power_monitor_wakeup(lcore_id);
+		}
+	}
+
 	RTE_LCORE_FOREACH_WORKER(lcore_id) {
 		if (rte_eal_wait_lcore(lcore_id) < 0)
 			return -1;