[0/9] examples/vm_power: 100% Busy Polling

  This patch set adds the capability to do out-of-band power
monitoring on a system. It uses a thread to monitor the branch
counters in the targeted cores, and calculates the branch ratio
if the running code.

If the branch ratop is low (0.01), then
the code is most likely running in a tight poll loop and doing
nothing, i.e. receiving no packets. In this case we scale down
the frequency of that core.

If the branch ratio is higher (>0.01), then it is likely that
the code is receiving and processing packets. In this case, we
scale up the frequency of that core.

The cpu counters are read via /dev/cpu/x/msr, so requires the
msr kernel module to be loaded. Because this method is used,
the patch set is implemented with one file for x86 systems, and
another for non-x86 systems, with conditional compilation in
the Makefile. The non-x86 functions are stubs, and do not
currently implement any functionality.

The vm_power_manager app has been modified to take a new parameter
   --core-list or -l
which takes a list of cores in a comma-separated list format,
e.g. 1,3,5-7,9, which resolvest to a core list of 1,3,5,6,7,9
These cores will then be enabled for oob monitoring. When the
OOB monitoring thread starts, it reads the branch hits/miss
counters of each monitored core, and scales up/down accordingly.

The guest_cli app has also been modified to allow sending of a
policy of type BRANCH_RATIO where all of the cores included in
the policy will be monitored by the vm_power_manager oob thread.

v2 changes:
   * Add the guest_cli patch into this patch set, including the
     ability to set the policy to BRANCH_RATIO.
     http://patches.dpdk.org/patch/40742/
   * When vm_power_manger receives a policy with type BRANCH_RATIO,
     add the relevant cores to the monitoring thread.

v3 changes:
   * Added a command line parameter to alloe changing of the
     default branch ratio threshold. can now use -b 0.3 or
     --branch-ratio=0.3 to set the ratio for scaling up/down.

[1/9] examples/vm_power: add check for port count
[2/9] examples/vm_power: add core list parameter
[3/9] examples/vm_power: add oob monitoring functions
[4/9] examples/vm_power: allow greater than 64 cores
[5/9] examples/vm_power: add thread for oob core monitor
[6/9] examples/vm_power: add port-list to command line
[7/9] examples/vm_power: add branch ratio policy type
[8/9] examples/vm_power: add cli args to guest app
[9/9] examples/vm_power: make branch ratio configurable
  

26/06/2018 11:23, David Hunt:
> This patch set adds the capability to do out-of-band power
> monitoring on a system. It uses a thread to monitor the branch
> counters in the targeted cores, and calculates the branch ratio
> if the running code.
> 
> If the branch ratop is low (0.01), then
> the code is most likely running in a tight poll loop and doing
> nothing, i.e. receiving no packets. In this case we scale down
> the frequency of that core.
> 
> If the branch ratio is higher (>0.01), then it is likely that
> the code is receiving and processing packets. In this case, we
> scale up the frequency of that core.
> 
> The cpu counters are read via /dev/cpu/x/msr, so requires the
> msr kernel module to be loaded. Because this method is used,
> the patch set is implemented with one file for x86 systems, and
> another for non-x86 systems, with conditional compilation in
> the Makefile. The non-x86 functions are stubs, and do not
> currently implement any functionality.
> 
> The vm_power_manager app has been modified to take a new parameter
>    --core-list or -l
> which takes a list of cores in a comma-separated list format,
> e.g. 1,3,5-7,9, which resolvest to a core list of 1,3,5,6,7,9
> These cores will then be enabled for oob monitoring. When the
> OOB monitoring thread starts, it reads the branch hits/miss
> counters of each monitored core, and scales up/down accordingly.

It looks to be a feature which could be integrated in DPDK libs.
Why choosing to implement it fully in an example?
  

Hi Thomas,

On 12/7/2018 8:09 PM, Thomas Monjalon wrote:
> 26/06/2018 11:23, David Hunt:
>> This patch set adds the capability to do out-of-band power
>> monitoring on a system. It uses a thread to monitor the branch
>> counters in the targeted cores, and calculates the branch ratio
>> if the running code.
>>
>> If the branch ratop is low (0.01), then
>> the code is most likely running in a tight poll loop and doing
>> nothing, i.e. receiving no packets. In this case we scale down
>> the frequency of that core.
>>
>> If the branch ratio is higher (>0.01), then it is likely that
>> the code is receiving and processing packets. In this case, we
>> scale up the frequency of that core.
>>
>> The cpu counters are read via /dev/cpu/x/msr, so requires the
>> msr kernel module to be loaded. Because this method is used,
>> the patch set is implemented with one file for x86 systems, and
>> another for non-x86 systems, with conditional compilation in
>> the Makefile. The non-x86 functions are stubs, and do not
>> currently implement any functionality.
>>
>> The vm_power_manager app has been modified to take a new parameter
>>     --core-list or -l
>> which takes a list of cores in a comma-separated list format,
>> e.g. 1,3,5-7,9, which resolvest to a core list of 1,3,5,6,7,9
>> These cores will then be enabled for oob monitoring. When the
>> OOB monitoring thread starts, it reads the branch hits/miss
>> counters of each monitored core, and scales up/down accordingly.
> It looks to be a feature which could be integrated in DPDK libs.
> Why choosing to implement it fully in an example?

I needed to set up a thread that looped tightly (~100uS interval) and 
run it on it's
own core. From what I have seen in other cases, it is usually the 
application that
allocates cores and decides what to run on them. I did think about putting
some of it in a library, but for this case I thought it made more sense 
to keep
it purely as a sample app.

Regards,
Dave.
  

13/07/2018 10:31, Hunt, David:
> Hi Thomas,
> 
> On 12/7/2018 8:09 PM, Thomas Monjalon wrote:
> > 26/06/2018 11:23, David Hunt:
> >> This patch set adds the capability to do out-of-band power
> >> monitoring on a system. It uses a thread to monitor the branch
> >> counters in the targeted cores, and calculates the branch ratio
> >> if the running code.
> >>
> >> If the branch ratop is low (0.01), then
> >> the code is most likely running in a tight poll loop and doing
> >> nothing, i.e. receiving no packets. In this case we scale down
> >> the frequency of that core.
> >>
> >> If the branch ratio is higher (>0.01), then it is likely that
> >> the code is receiving and processing packets. In this case, we
> >> scale up the frequency of that core.
> >>
> >> The cpu counters are read via /dev/cpu/x/msr, so requires the
> >> msr kernel module to be loaded. Because this method is used,
> >> the patch set is implemented with one file for x86 systems, and
> >> another for non-x86 systems, with conditional compilation in
> >> the Makefile. The non-x86 functions are stubs, and do not
> >> currently implement any functionality.
> >>
> >> The vm_power_manager app has been modified to take a new parameter
> >>     --core-list or -l
> >> which takes a list of cores in a comma-separated list format,
> >> e.g. 1,3,5-7,9, which resolvest to a core list of 1,3,5,6,7,9
> >> These cores will then be enabled for oob monitoring. When the
> >> OOB monitoring thread starts, it reads the branch hits/miss
> >> counters of each monitored core, and scales up/down accordingly.
> > It looks to be a feature which could be integrated in DPDK libs.
> > Why choosing to implement it fully in an example?
> 
> I needed to set up a thread that looped tightly (~100uS interval) and 
> run it on it's
> own core. From what I have seen in other cases, it is usually the 
> application that
> allocates cores and decides what to run on them. I did think about putting
> some of it in a library, but for this case I thought it made more sense 
> to keep
> it purely as a sample app.

I feel some code deserves to be in a library.
For instance, having different implementations per CPU is a good reason
to make a library.
  

On 13/7/2018 9:33 AM, Thomas Monjalon wrote:
> 13/07/2018 10:31, Hunt, David:
>> Hi Thomas,
>>
>> On 12/7/2018 8:09 PM, Thomas Monjalon wrote:
>>> 26/06/2018 11:23, David Hunt:
>>>> This patch set adds the capability to do out-of-band power
>>>> monitoring on a system. It uses a thread to monitor the branch
>>>> counters in the targeted cores, and calculates the branch ratio
>>>> if the running code.
>>>>
>>>> If the branch ratop is low (0.01), then
>>>> the code is most likely running in a tight poll loop and doing
>>>> nothing, i.e. receiving no packets. In this case we scale down
>>>> the frequency of that core.
>>>>
>>>> If the branch ratio is higher (>0.01), then it is likely that
>>>> the code is receiving and processing packets. In this case, we
>>>> scale up the frequency of that core.
>>>>
>>>> The cpu counters are read via /dev/cpu/x/msr, so requires the
>>>> msr kernel module to be loaded. Because this method is used,
>>>> the patch set is implemented with one file for x86 systems, and
>>>> another for non-x86 systems, with conditional compilation in
>>>> the Makefile. The non-x86 functions are stubs, and do not
>>>> currently implement any functionality.
>>>>
>>>> The vm_power_manager app has been modified to take a new parameter
>>>>      --core-list or -l
>>>> which takes a list of cores in a comma-separated list format,
>>>> e.g. 1,3,5-7,9, which resolvest to a core list of 1,3,5,6,7,9
>>>> These cores will then be enabled for oob monitoring. When the
>>>> OOB monitoring thread starts, it reads the branch hits/miss
>>>> counters of each monitored core, and scales up/down accordingly.
>>> It looks to be a feature which could be integrated in DPDK libs.
>>> Why choosing to implement it fully in an example?
>> I needed to set up a thread that looped tightly (~100uS interval) and
>> run it on it's
>> own core. From what I have seen in other cases, it is usually the
>> application that
>> allocates cores and decides what to run on them. I did think about putting
>> some of it in a library, but for this case I thought it made more sense
>> to keep
>> it purely as a sample app.
> I feel some code deserves to be in a library.
> For instance, having different implementations per CPU is a good reason
> to make a library.
>

Sure, I can look at moving some of the code into the library in a future 
release. However, I
believe it's OK as it is for the current merge window.

Regards,
Dave.
  

13/07/2018 10:43, Hunt, David:
> 
> On 13/7/2018 9:33 AM, Thomas Monjalon wrote:
> > 13/07/2018 10:31, Hunt, David:
> >> Hi Thomas,
> >>
> >> On 12/7/2018 8:09 PM, Thomas Monjalon wrote:
> >>> 26/06/2018 11:23, David Hunt:
> >>>> This patch set adds the capability to do out-of-band power
> >>>> monitoring on a system. It uses a thread to monitor the branch
> >>>> counters in the targeted cores, and calculates the branch ratio
> >>>> if the running code.
> >>>>
> >>>> If the branch ratop is low (0.01), then
> >>>> the code is most likely running in a tight poll loop and doing
> >>>> nothing, i.e. receiving no packets. In this case we scale down
> >>>> the frequency of that core.
> >>>>
> >>>> If the branch ratio is higher (>0.01), then it is likely that
> >>>> the code is receiving and processing packets. In this case, we
> >>>> scale up the frequency of that core.
> >>>>
> >>>> The cpu counters are read via /dev/cpu/x/msr, so requires the
> >>>> msr kernel module to be loaded. Because this method is used,
> >>>> the patch set is implemented with one file for x86 systems, and
> >>>> another for non-x86 systems, with conditional compilation in
> >>>> the Makefile. The non-x86 functions are stubs, and do not
> >>>> currently implement any functionality.
> >>>>
> >>>> The vm_power_manager app has been modified to take a new parameter
> >>>>      --core-list or -l
> >>>> which takes a list of cores in a comma-separated list format,
> >>>> e.g. 1,3,5-7,9, which resolvest to a core list of 1,3,5,6,7,9
> >>>> These cores will then be enabled for oob monitoring. When the
> >>>> OOB monitoring thread starts, it reads the branch hits/miss
> >>>> counters of each monitored core, and scales up/down accordingly.
> >>> It looks to be a feature which could be integrated in DPDK libs.
> >>> Why choosing to implement it fully in an example?
> >> I needed to set up a thread that looped tightly (~100uS interval) and
> >> run it on it's
> >> own core. From what I have seen in other cases, it is usually the
> >> application that
> >> allocates cores and decides what to run on them. I did think about putting
> >> some of it in a library, but for this case I thought it made more sense
> >> to keep
> >> it purely as a sample app.
> > I feel some code deserves to be in a library.
> > For instance, having different implementations per CPU is a good reason
> > to make a library.
> >
> 
> Sure, I can look at moving some of the code into the library in a future 
> release. However, I
> believe it's OK as it is for the current merge window.

I will to pull it in 18.08-rc2 if compilation is fine.