[v4,0/3] Predictable RSS feature

  This patch series introduces predictable RSS feature.
It is based on the idea of searching for partial hash collisions
within Toeplitz hash.

The Toeplitz hash function is a homomorphism between (G, ^) and (H, ^),
where (G, ^) - is a group of tuples and (H, ^) is a group of hashes
with respect to XOR operation. So tuples and hashes could be treated as
n-dimension and 32-dimension vector spaces over GF(2).
So, f(x ^ y) == f(x) ^ f(y)
where f - is the toeplitz hash function and x, y are tuples.

The ability to predict partial collisions allows user to compute
input hash value with desired LSB values.
Usually number of LSB's are defined by the size of RSS Redirection Table.

There could be number of use cases, for example:
1) NAT. Using this library it is possible to select a new port number
on a translation in the way that rss hash for original tuple will have
the same LSB's as rss hash for reverse tuple.
2) IPSec/MPLS/Vxlan. It is possible to choose tunnel id to be pinned to
a desired queue.
3) TCP stack. It is possible to choose a source port number for outgoing
connections in the way that received replies will be assigned to
desired queue.
4) RSS hash key generation. Hash key initialization with random values
does not guarantee an uniform distribution amongst queues. This library
uses mathematically proved algorithm to complete the rss hash key to
provide the best distribution.

v4:
- RETA_SZ macros were moved to .h
- fixed typos 

v3:
- reworked rte_thash_adjust_tuple()
- added extra comments
- fixed typos
- rte_thash_adjust_tuple() API was putted into the first commit

v2:
- added extra API rte_thash_adjust_tuple()
- added extra tests for rte_thash_adjust_tuple()
- added extra fields to rte_thash_subtuple_helper struct
- fixed typos

Vladimir Medvedkin (3):
  hash: add predictable RSS API
  hash: add predictable RSS implementation
  test/hash: add additional thash tests

 app/test/test_thash.c       | 469 ++++++++++++++++++++++++++++++-
 lib/librte_hash/meson.build |   3 +-
 lib/librte_hash/rte_thash.c | 671 ++++++++++++++++++++++++++++++++++++++++++++
 lib/librte_hash/rte_thash.h | 198 +++++++++++++
 lib/librte_hash/version.map |   8 +
 5 files changed, 1342 insertions(+), 7 deletions(-)
 create mode 100644 lib/librte_hash/rte_thash.c
  

> -----Original Message-----
> From: Medvedkin, Vladimir <vladimir.medvedkin@intel.com>
> Sent: Tuesday, April 13, 2021 6:20 AM
> To: dev@dpdk.org
> Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Chilikin, Andrey
> <andrey.chilikin@intel.com>; Kinsella, Ray <ray.kinsella@intel.com>; Wang,
> Yipeng1 <yipeng1.wang@intel.com>; Gobriel, Sameh
> <sameh.gobriel@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: [PATCH v4 0/3] Predictable RSS feature
> 
> This patch series introduces predictable RSS feature.
> It is based on the idea of searching for partial hash collisions within Toeplitz
> hash.
> 
> The Toeplitz hash function is a homomorphism between (G, ^) and (H, ^),
> where (G, ^) - is a group of tuples and (H, ^) is a group of hashes with respect
> to XOR operation. So tuples and hashes could be treated as n-dimension and
> 32-dimension vector spaces over GF(2).
> So, f(x ^ y) == f(x) ^ f(y)
> where f - is the toeplitz hash function and x, y are tuples.
> 
> The ability to predict partial collisions allows user to compute input hash value
> with desired LSB values.
> Usually number of LSB's are defined by the size of RSS Redirection Table.
> 
> There could be number of use cases, for example:
> 1) NAT. Using this library it is possible to select a new port number on a
> translation in the way that rss hash for original tuple will have the same LSB's
> as rss hash for reverse tuple.
> 2) IPSec/MPLS/Vxlan. It is possible to choose tunnel id to be pinned to a
> desired queue.
> 3) TCP stack. It is possible to choose a source port number for outgoing
> connections in the way that received replies will be assigned to desired
> queue.
> 4) RSS hash key generation. Hash key initialization with random values does
> not guarantee an uniform distribution amongst queues. This library uses
> mathematically proved algorithm to complete the rss hash key to provide the
> best distribution.
> 
> v4:
> - RETA_SZ macros were moved to .h
> - fixed typos
> 
> v3:
> - reworked rte_thash_adjust_tuple()
> - added extra comments
> - fixed typos
> - rte_thash_adjust_tuple() API was putted into the first commit
> 
> v2:
> - added extra API rte_thash_adjust_tuple()
> - added extra tests for rte_thash_adjust_tuple()
> - added extra fields to rte_thash_subtuple_helper struct
> - fixed typos
> 
> Vladimir Medvedkin (3):
>   hash: add predictable RSS API
>   hash: add predictable RSS implementation
>   test/hash: add additional thash tests
> 
>  app/test/test_thash.c       | 469 ++++++++++++++++++++++++++++++-
>  lib/librte_hash/meson.build |   3 +-
>  lib/librte_hash/rte_thash.c | 671
> ++++++++++++++++++++++++++++++++++++++++++++
>  lib/librte_hash/rte_thash.h | 198 +++++++++++++
>  lib/librte_hash/version.map |   8 +
>  5 files changed, 1342 insertions(+), 7 deletions(-)  create mode 100644
> lib/librte_hash/rte_thash.c
> 
> --
> 2.7.4
[Wang, Yipeng] 
Hi, Vladimir, thanks for the new version.
Most parts look good to me, so I acked to not block the integration.
But documentation and a good use example is still needed.
I think it can be added later after RC1.

Thanks!
Yipeng
  

On Wed, Apr 14, 2021 at 8:05 PM Wang, Yipeng1 <yipeng1.wang@intel.com> wrote:
>
> > -----Original Message-----
> > From: Medvedkin, Vladimir <vladimir.medvedkin@intel.com>
> > Sent: Tuesday, April 13, 2021 6:20 AM
> > To: dev@dpdk.org
> > Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Chilikin, Andrey
> > <andrey.chilikin@intel.com>; Kinsella, Ray <ray.kinsella@intel.com>; Wang,
> > Yipeng1 <yipeng1.wang@intel.com>; Gobriel, Sameh
> > <sameh.gobriel@intel.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>
> > Subject: [PATCH v4 0/3] Predictable RSS feature
> >
> > This patch series introduces predictable RSS feature.
> > It is based on the idea of searching for partial hash collisions within Toeplitz
> > hash.
> >
> > The Toeplitz hash function is a homomorphism between (G, ^) and (H, ^),
> > where (G, ^) - is a group of tuples and (H, ^) is a group of hashes with respect
> > to XOR operation. So tuples and hashes could be treated as n-dimension and
> > 32-dimension vector spaces over GF(2).
> > So, f(x ^ y) == f(x) ^ f(y)
> > where f - is the toeplitz hash function and x, y are tuples.
> >
> > The ability to predict partial collisions allows user to compute input hash value
> > with desired LSB values.
> > Usually number of LSB's are defined by the size of RSS Redirection Table.
> >
> > There could be number of use cases, for example:
> > 1) NAT. Using this library it is possible to select a new port number on a
> > translation in the way that rss hash for original tuple will have the same LSB's
> > as rss hash for reverse tuple.
> > 2) IPSec/MPLS/Vxlan. It is possible to choose tunnel id to be pinned to a
> > desired queue.
> > 3) TCP stack. It is possible to choose a source port number for outgoing
> > connections in the way that received replies will be assigned to desired
> > queue.
> > 4) RSS hash key generation. Hash key initialization with random values does
> > not guarantee an uniform distribution amongst queues. This library uses
> > mathematically proved algorithm to complete the rss hash key to provide the
> > best distribution.
> >
> > v4:
> > - RETA_SZ macros were moved to .h
> > - fixed typos
> >
> > v3:
> > - reworked rte_thash_adjust_tuple()
> > - added extra comments
> > - fixed typos
> > - rte_thash_adjust_tuple() API was putted into the first commit
> >
> > v2:
> > - added extra API rte_thash_adjust_tuple()
> > - added extra tests for rte_thash_adjust_tuple()
> > - added extra fields to rte_thash_subtuple_helper struct
> > - fixed typos
> >
> > Vladimir Medvedkin (3):
> >   hash: add predictable RSS API
> >   hash: add predictable RSS implementation
> >   test/hash: add additional thash tests
> >
> >  app/test/test_thash.c       | 469 ++++++++++++++++++++++++++++++-
> >  lib/librte_hash/meson.build |   3 +-
> >  lib/librte_hash/rte_thash.c | 671
> > ++++++++++++++++++++++++++++++++++++++++++++
> >  lib/librte_hash/rte_thash.h | 198 +++++++++++++
> >  lib/librte_hash/version.map |   8 +
> >  5 files changed, 1342 insertions(+), 7 deletions(-)  create mode 100644
> > lib/librte_hash/rte_thash.c
> >
> > --
> > 2.7.4
> [Wang, Yipeng]
> Hi, Vladimir, thanks for the new version.
> Most parts look good to me, so I acked to not block the integration.
> But documentation and a good use example is still needed.
> I think it can be added later after RC1.

This is not something new.
Documentation, release note update comes with the API.

The example had been asked by Stephen too and this is justified.

I can see some comments in patches.

I will wait for a v5 from Vladimir.
If it can be done for rc1, cool, else it will wait for the next release.