[v3,0/7] support the flow-based traffic sampling

This patch set implement the flow sampling for mlx5 driver.

The solution is introduced a new rte_flow action that will sample the incoming traffic and send a duplicated traffic with the specified ratio to the application, while the original packet will continue to the target destination.

If the sample ratio value be set to 1, means that the packets would be completely mirrored. The sample packet can be assigned with different set of actions from the original packet.

MLX5 PMD driver will be responsible for validate and translate the sample action while creating a flow.

v3:
* Remove 'const' of ratio field.
* Update description and commit messages.

v2:
* Rebase patches based on the latest code.
* Update rte_flow and release documents.
* Fix the compile error.
* Removed unnecessary change in [PATCH 7/8] net/mlx5: update the metadata register c0 support since FDB will use 5-tuple to do match.
* Update changes based on the comments.

Jiawei Wang (7):
  ethdev: introduce sample action for rte flow
  common/mlx5: glue for sample action
  common/mlx5: query sampler object capability via DevX
  net/mlx5: add the validate sample action
  net/mlx5: split sample flow into two sub flows
  net/mlx5: update translate function for sample action
  app/testpmd: add testpmd command for sample action

 app/test-pmd/cmdline_flow.c            | 285 ++++++++++++++-
 doc/guides/prog_guide/rte_flow.rst     |  25 ++
 doc/guides/rel_notes/release_20_08.rst |   6 +
 drivers/common/mlx5/Makefile           |   5 +
 drivers/common/mlx5/linux/meson.build  |   2 +
 drivers/common/mlx5/linux/mlx5_glue.c  |  15 +
 drivers/common/mlx5/linux/mlx5_glue.h  |  12 +
 drivers/common/mlx5/mlx5_devx_cmds.c   |  27 ++
 drivers/common/mlx5/mlx5_devx_cmds.h   |   1 +
 drivers/common/mlx5/mlx5_prm.h         |  51 +++
 drivers/net/mlx5/linux/mlx5_os.c       |  14 +
 drivers/net/mlx5/mlx5.c                |  11 +
 drivers/net/mlx5/mlx5.h                |   4 +
 drivers/net/mlx5/mlx5_flow.c           | 274 +++++++++++++-
 drivers/net/mlx5/mlx5_flow.h           |  51 ++-
 drivers/net/mlx5/mlx5_flow_dv.c        | 627 ++++++++++++++++++++++++++++++++-
 lib/librte_ethdev/rte_flow.c           |   1 +
 lib/librte_ethdev/rte_flow.h           |  30 ++
 18 files changed, 1406 insertions(+), 35 deletions(-)

On Mon,  6 Jul 2020 20:51:01 +0300
Jiawei Wang <jiaweiw@mellanox.com> wrote:

> This patch set implement the flow sampling for mlx5 driver.
> 
> The solution is introduced a new rte_flow action that will sample the incoming traffic and send a duplicated traffic with the specified ratio to the application, while the original packet will continue to the target destination.
> 
> If the sample ratio value be set to 1, means that the packets would be completely mirrored. The sample packet can be assigned with different set of actions from the original packet.
> 
> MLX5 PMD driver will be responsible for validate and translate the sample action while creating a flow.
> 

You seem to have ignored my feedback that this could be more useful if it
didn't just support duplication. It should allow sampling and then make
the other rule chain (the one that gets hit after sampling) run.

By allowing a more general form of sampling it could be used for doing
network emulation (or packet manipulation) as well as simple netflow/ipfix style sampling.

Hi Stephen,

> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Subject: Re: [dpdk-dev] [PATCH v3 0/7] support the flow-based traffic sampling
> 
> On Mon,  6 Jul 2020 20:51:01 +0300
> Jiawei Wang <jiaweiw@mellanox.com> wrote:
> 
> > This patch set implement the flow sampling for mlx5 driver.
> >
> > The solution is introduced a new rte_flow action that will sample the
> incoming traffic and send a duplicated traffic with the specified ratio to the
> application, while the original packet will continue to the target destination.
> >
> > If the sample ratio value be set to 1, means that the packets would be
> completely mirrored. The sample packet can be assigned with different set of
> actions from the original packet.
> >
> > MLX5 PMD driver will be responsible for validate and translate the sample
> action while creating a flow.
> >
> 
> You seem to have ignored my feedback that this could be more useful if it
> didn't just support duplication. It should allow sampling and then make
> the other rule chain (the one that gets hit after sampling) run.
> 
> By allowing a more general form of sampling it could be used for doing
> network emulation (or packet manipulation) as well as simple netflow/ipfix
> style sampling.
> 
What is network emulation?
I think that what you are suggesting is a different feature. There is high penalty for 
rules duplication, like I stated in my previous reply.
As I can see there is no plan at least in Mellanox to support this kind of feature you
are suggesting, Since this API in any case is experimental. I suggest that when the need
comes or some other vendor will wish to implement such a feature we
can ether update the API or create a new one.
What do you think?

Best,
Ori