mbox series

[v2,0/2] ethdev: add random item support

Message ID 20230911074149.697944-1-michaelba@nvidia.com (mailing list archive)
Headers
Series ethdev: add random item support |

Message

Michael Baum Sept. 11, 2023, 7:41 a.m. UTC 
  Add support for matching random value using new "rte_flow_item_random"
structure.
This random value is not based on the packet data/headers.
Application shouldn't assume that this value is kept during the life
time of the packet.

v2:
 - Rabase.
 - Fix copy-paste mistake in release notes.

Michael Baum (2):
  ethdev: add random item support
  app/testpmd: add random item support

 app/test-pmd/cmdline_flow.c                 | 30 ++++++++++++++++++-
 doc/guides/nics/features/default.ini        |  1 +
 doc/guides/prog_guide/rte_flow.rst          | 11 +++++++
 doc/guides/rel_notes/release_23_11.rst      |  4 +++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |  4 +++
 lib/ethdev/rte_flow.c                       |  1 +
 lib/ethdev/rte_flow.h                       | 33 +++++++++++++++++++++
 7 files changed, 83 insertions(+), 1 deletion(-)

Comments

Morten Brørup Sept. 11, 2023, 4:55 p.m. UTC | #1 
> From: Michael Baum [mailto:michaelba@nvidia.com]
> Sent: Monday, 11 September 2023 08.42
> 
> Add support for matching random value using new "rte_flow_item_random"
> structure.

I get it. It can be used for things like stochastic sampling.

However, it doesn't provide support for e.g. 1/100 or 1/500.

So here's a suggestion:

Instead of "value" (which is irrelevant) & "mask" (which is what really controls the probability), wouldn't it be better if "value" was an inverse probability (and "mask" was irrelevant)? E.g. value=500 means that there is a 1 of 500 probability of a match.

Would this kind of random item better serve the purpose?

Or is the random item, in its current form, also required for other purposes?
Stephen Hemminger Sept. 11, 2023, 5:53 p.m. UTC | #2 
On Mon, 11 Sep 2023 18:55:45 +0200
Morten Brørup <mb@smartsharesystems.com> wrote:

> > From: Michael Baum [mailto:michaelba@nvidia.com]
> > Sent: Monday, 11 September 2023 08.42
> > 
> > Add support for matching random value using new
> > "rte_flow_item_random" structure.  
> 
> I get it. It can be used for things like stochastic sampling.
> 
> However, it doesn't provide support for e.g. 1/100 or 1/500.
> 
> So here's a suggestion:
> 
> Instead of "value" (which is irrelevant) & "mask" (which is what
> really controls the probability), wouldn't it be better if "value"
> was an inverse probability (and "mask" was irrelevant)? E.g.
> value=500 means that there is a 1 of 500 probability of a match.
> 
> Would this kind of random item better serve the purpose?
> 
> Or is the random item, in its current form, also required for other
> purposes?
> 

For netem (Linux kernel) needed to support expressing a percent in fixed
point number. The solution was to express it as:
    Percent = X / UINT32_MAX
Michael Baum Sept. 12, 2023, 8:40 a.m. UTC | #3 
On Mon, 11 Sep 2023 18:55:45 +0200
Morten Brørup <mb@smartsharesystems.com> wrote: 
> 
> > From: Michael Baum [mailto:michaelba@nvidia.com]
> > Sent: Monday, 11 September 2023 08.42
> >
> > Add support for matching random value using new "rte_flow_item_random"
> > structure.
> 
> I get it. It can be used for things like stochastic sampling.
> 
> However, it doesn't provide support for e.g. 1/100 or 1/500.

It supports those values using "last" field in addition to "spec" and "mask".
It matches a range of random values for any requested percentage.

> 
> So here's a suggestion:
> 
> Instead of "value" (which is irrelevant) & "mask" (which is what really controls the
> probability), wouldn't it be better if "value" was an inverse probability (and
> "mask" was irrelevant)? E.g. value=500 means that there is a 1 of 500 probability
> of a match.
> 
> Would this kind of random item better serve the purpose?
> 
> Or is the random item, in its current form, also required for other purposes?

The random item is more generic than stochastic sampling, it can implement distribution. 
When application wants to distribute the traffic between ports/queues, it can match all random value range with send to port/queue actions.
Michael Baum Oct. 12, 2023, 9:48 a.m. UTC | #4 
On  Monday, 11 September 2023 20:53 Stephen Hemminger wrote: 
> 
> On Mon, 11 Sep 2023 18:55:45 +0200
> Morten Brørup <mb@smartsharesystems.com> wrote:
> 
> > > From: Michael Baum [mailto:michaelba@nvidia.com]
> > > Sent: Monday, 11 September 2023 08.42
> > >
> > > Add support for matching random value using new
> > > "rte_flow_item_random" structure.
> >
> > I get it. It can be used for things like stochastic sampling.
> >
> > However, it doesn't provide support for e.g. 1/100 or 1/500.
> >
> > So here's a suggestion:
> >
> > Instead of "value" (which is irrelevant) & "mask" (which is what
> > really controls the probability), wouldn't it be better if "value"
> > was an inverse probability (and "mask" was irrelevant)? E.g.
> > value=500 means that there is a 1 of 500 probability of a match.
> >
> > Would this kind of random item better serve the purpose?
> >
> > Or is the random item, in its current form, also required for other
> > purposes?
> >
> 
> For netem (Linux kernel) needed to support expressing a percent in fixed point
> number. The solution was to express it as:
>     Percent = X / UINT32_MAX

I concur with the notion that numbers can be utilized for probability calculations. However, it's important to acknowledge that when employing numbers, we are essentially working with a superset of possibilities due to their inherent versatility.

To illustrate this, consider the scenario where we transmit a numerical value to an application and use that same value across different stages of the pipeline. Take, for instance, the following use case:

In the initial stage, there is a 20% probability of selecting a packet for sampling before proceeding with the pipeline.

Towards the end of the pipeline, if the packet was initially sampled, we sample it once more.

The challenge arises when attempting to replicate this process solely using a numerical value, as there is no guarantee that the same packet will be chosen both in the initial and final stages. Conversely, by duplicating the random number and using it as a reference for matching, we can effectively implement the aforementioned usage scenario.

I hope this clarifies the distinction and benefits of utilizing random numbers in our probability calculations.