[1/3] ethdev: add RSS hash level

Message ID 20191207005919.10962-2-ajit.khaparde@broadcom.com (mailing list archive)
State Changes Requested, archived
Delegated to: Ferruh Yigit
Headers
Series add support for RSS level |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/iol-intel-Performance success Performance Testing PASS
ci/Intel-compilation success Compilation OK
ci/iol-testing success Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS

Commit Message

Ajit Khaparde Dec. 7, 2019, 12:59 a.m. UTC
  This patch adds ability to configure RSS hash level in hardware.
This feature will allow an application to select RSS hash calculation
on outer or inner headers for tunneled packets.

Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 lib/librte_ethdev/rte_ethdev.h | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
  

Comments

Andrew Rybchenko Dec. 7, 2019, 9:13 a.m. UTC | #1
On 12/7/19 3:59 AM, Ajit Khaparde wrote:
> This patch adds ability to configure RSS hash level in hardware.
> This feature will allow an application to select RSS hash calculation
> on outer or inner headers for tunneled packets.
> 
> Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> ---
>   lib/librte_ethdev/rte_ethdev.h | 27 +++++++++++++++++++++++++++
>   1 file changed, 27 insertions(+)
> 
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 18a9defc2..5189bdbab 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -444,11 +444,35 @@ struct rte_vlan_filter_conf {
>    * The *rss_hf* field of the *rss_conf* structure indicates the different
>    * types of IPv4/IPv6 packets to which the RSS hashing must be applied.
>    * Supplying an *rss_hf* equal to zero disables the RSS feature.
> + *
> + * The *rss_level* field of the *rss_conf* structure indicates the
> + * Packet encapsulation level RSS hash @p types apply to.
> + *
> + * - @p 0 requests the default behavior. Depending on the packet
> + *   type, it can mean outermost, innermost, anything in between or
> + *   even no RSS.
> + *
> + *   It basically stands for the innermost encapsulation level RSS
> + *   can be performed on according to PMD and device capabilities.
> + *
> + * - @p 1 requests RSS to be performed on the outermost packet
> + *   encapsulation level.
> + *
> + * - @p 2 and subsequent values request RSS to be performed on the
> + *   specified inner packet encapsulation level, from outermost to
> + *   innermost (lower to higher values).
> + *
> + * Support for values other than @p 0 is dependent on the underlying
> + * hardware in use.
> + *
> + * Requesting a specific RSS level on unrecognized traffic results
> + * in undefined behavior.
>    */
>   struct rte_eth_rss_conf {
>   	uint8_t *rss_key;    /**< If not NULL, 40-byte hash key. */
>   	uint8_t rss_key_len; /**< hash key length in bytes. */
>   	uint64_t rss_hf;     /**< Hash functions to apply - see below. */
> +	uint32_t rss_level;  /**< RSS hash level */
>   };

I'm not sure that offload flag is required in this case.
I think maximum supported rss_level in dev_info will provide
more information and per-queue level does not make sense
in this case. Even if per-queue group control is required,
it should be doable via rte_flow API RSS action.

Anyway, it looks like it is ABI breakage with all consequences.
In 64-bit case it is possible to put it before rss_hf to avoid
ABI breakage, but it will break ABI on 32-bit anyway.

>   /*
> @@ -599,6 +623,8 @@ rte_eth_rss_hf_refine(uint64_t rss_hf)
>   	ETH_RSS_GENEVE | \
>   	ETH_RSS_NVGRE)
>   
> +#define ETH_RSS_LEVEL_DEFAULT	0
> +
>   /*
>    * Definitions used for redirection table entry size.
>    * Some RSS RETA sizes may not be supported by some drivers, check the
> @@ -1103,6 +1129,7 @@ struct rte_eth_conf {
>   #define DEV_RX_OFFLOAD_SCTP_CKSUM	0x00020000
>   #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM  0x00040000
>   #define DEV_RX_OFFLOAD_RSS_HASH		0x00080000
> +#define DEV_RX_OFFLOAD_RSS_LEVEL	0x00100000
>   
>   #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \
>   				 DEV_RX_OFFLOAD_UDP_CKSUM | \
>
  
Ajit Khaparde Dec. 7, 2019, 7:56 p.m. UTC | #2
On Sat, Dec 7, 2019 at 1:14 AM Andrew Rybchenko <arybchenko@solarflare.com>
wrote:

> On 12/7/19 3:59 AM, Ajit Khaparde wrote:
> > This patch adds ability to configure RSS hash level in hardware.
> > This feature will allow an application to select RSS hash calculation
> > on outer or inner headers for tunneled packets.
> >
> > Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> > ---
> >   lib/librte_ethdev/rte_ethdev.h | 27 +++++++++++++++++++++++++++
> >   1 file changed, 27 insertions(+)
> >
> > diff --git a/lib/librte_ethdev/rte_ethdev.h
> b/lib/librte_ethdev/rte_ethdev.h
> > index 18a9defc2..5189bdbab 100644
> > --- a/lib/librte_ethdev/rte_ethdev.h
> > +++ b/lib/librte_ethdev/rte_ethdev.h
> > @@ -444,11 +444,35 @@ struct rte_vlan_filter_conf {
> >    * The *rss_hf* field of the *rss_conf* structure indicates the
> different
> >    * types of IPv4/IPv6 packets to which the RSS hashing must be applied.
> >    * Supplying an *rss_hf* equal to zero disables the RSS feature.
> > + *
> > + * The *rss_level* field of the *rss_conf* structure indicates the
> > + * Packet encapsulation level RSS hash @p types apply to.
> > + *
> > + * - @p 0 requests the default behavior. Depending on the packet
> > + *   type, it can mean outermost, innermost, anything in between or
> > + *   even no RSS.
> > + *
> > + *   It basically stands for the innermost encapsulation level RSS
> > + *   can be performed on according to PMD and device capabilities.
> > + *
> > + * - @p 1 requests RSS to be performed on the outermost packet
> > + *   encapsulation level.
> > + *
> > + * - @p 2 and subsequent values request RSS to be performed on the
> > + *   specified inner packet encapsulation level, from outermost to
> > + *   innermost (lower to higher values).
> > + *
> > + * Support for values other than @p 0 is dependent on the underlying
> > + * hardware in use.
> > + *
> > + * Requesting a specific RSS level on unrecognized traffic results
> > + * in undefined behavior.
> >    */
> >   struct rte_eth_rss_conf {
> >       uint8_t *rss_key;    /**< If not NULL, 40-byte hash key. */
> >       uint8_t rss_key_len; /**< hash key length in bytes. */
> >       uint64_t rss_hf;     /**< Hash functions to apply - see below. */
> > +     uint32_t rss_level;  /**< RSS hash level */
> >   };
>
> I'm not sure that offload flag is required in this case.
> I think maximum supported rss_level in dev_info will provide

more information and per-queue level does not make sense
> in this case. Even if per-queue group control is required,
> it should be doable via rte_flow API RSS action.
>
This is dev config and not flow specific configuration. Ofcourse while
passing
the rss_config, not all the queues may be specified, but that is not a new
behavior and it is upto the application anyway.

Are we transitioning the device level configuration to rte_flow/flow
based scheme?


>
> Anyway, it looks like it is ABI breakage with all consequences.
> In 64-bit case it is possible to put it before rss_hf to avoid
> ABI breakage, but it will break ABI on 32-bit anyway.
>
Right.
I sent the proposal for review early to get it cleaned up and ready
when the window opens.


>
> >   /*
> > @@ -599,6 +623,8 @@ rte_eth_rss_hf_refine(uint64_t rss_hf)
> >       ETH_RSS_GENEVE | \
> >       ETH_RSS_NVGRE)
> >
> > +#define ETH_RSS_LEVEL_DEFAULT        0
> > +
> >   /*
> >    * Definitions used for redirection table entry size.
> >    * Some RSS RETA sizes may not be supported by some drivers, check the
> > @@ -1103,6 +1129,7 @@ struct rte_eth_conf {
> >   #define DEV_RX_OFFLOAD_SCTP_CKSUM   0x00020000
> >   #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM  0x00040000
> >   #define DEV_RX_OFFLOAD_RSS_HASH             0x00080000
> > +#define DEV_RX_OFFLOAD_RSS_LEVEL     0x00100000
> >
> >   #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \
> >                                DEV_RX_OFFLOAD_UDP_CKSUM | \
> >
>
>
  
Stephen Hemminger Dec. 7, 2019, 10:27 p.m. UTC | #3
On Fri,  6 Dec 2019 16:59:17 -0800
Ajit Khaparde <ajit.khaparde@broadcom.com> wrote:

>   */
>  struct rte_eth_rss_conf {
>  	uint8_t *rss_key;    /**< If not NULL, 40-byte hash key. */
>  	uint8_t rss_key_len; /**< hash key length in bytes. */
>  	uint64_t rss_hf;     /**< Hash functions to apply - see below. */
> +	uint32_t rss_level;  /**< RSS hash level */
>  };
>  


This is an API/ABI change which is not allowed per current policy.
API/ABI is frozen since 19.11 until the DPDK 20.11 release

You need to figure out another way to do this.
  
Ananyev, Konstantin Dec. 8, 2019, 6:02 p.m. UTC | #4
Hi Ajit,

> This patch adds ability to configure RSS hash level in hardware.
> This feature will allow an application to select RSS hash calculation
> on outer or inner headers for tunneled packets.
> 
> Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> ---
>  lib/librte_ethdev/rte_ethdev.h | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 18a9defc2..5189bdbab 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -444,11 +444,35 @@ struct rte_vlan_filter_conf {
>   * The *rss_hf* field of the *rss_conf* structure indicates the different
>   * types of IPv4/IPv6 packets to which the RSS hashing must be applied.
>   * Supplying an *rss_hf* equal to zero disables the RSS feature.
> + *
> + * The *rss_level* field of the *rss_conf* structure indicates the
> + * Packet encapsulation level RSS hash @p types apply to.
> + *
> + * - @p 0 requests the default behavior. Depending on the packet
> + *   type, it can mean outermost, innermost, anything in between or
> + *   even no RSS.
> + *
> + *   It basically stands for the innermost encapsulation level RSS
> + *   can be performed on according to PMD and device capabilities.
> + *
> + * - @p 1 requests RSS to be performed on the outermost packet
> + *   encapsulation level.
> + *
> + * - @p 2 and subsequent values request RSS to be performed on the
> + *   specified inner packet encapsulation level, from outermost to
> + *   innermost (lower to higher values).

If it is just to add ability to do rss over inner header, then do we really need
all this concept of 'levels'?
Might be just:

#define DEV_RX_OFFLOAD_RSS_HASH		0x00080000
+#define DEV_RX_OFFLOAD_RSS_HASH_INNER	0x00100000 

and uint8_t in rte_eth_rss_conf (between key_len and rss_hf) to indicate
inner/outer?

> + *
> + * Support for values other than @p 0 is dependent on the underlying
> + * hardware in use.
> + *
> + * Requesting a specific RSS level on unrecognized traffic results
> + * in undefined behavior.
>   */
>  struct rte_eth_rss_conf {
>  	uint8_t *rss_key;    /**< If not NULL, 40-byte hash key. */
>  	uint8_t rss_key_len; /**< hash key length in bytes. */
>  	uint64_t rss_hf;     /**< Hash functions to apply - see below. */
> +	uint32_t rss_level;  /**< RSS hash level */
>  };
> 
>  /*
> @@ -599,6 +623,8 @@ rte_eth_rss_hf_refine(uint64_t rss_hf)
>  	ETH_RSS_GENEVE | \
>  	ETH_RSS_NVGRE)
> 
> +#define ETH_RSS_LEVEL_DEFAULT	0
> +
>  /*
>   * Definitions used for redirection table entry size.
>   * Some RSS RETA sizes may not be supported by some drivers, check the
> @@ -1103,6 +1129,7 @@ struct rte_eth_conf {
>  #define DEV_RX_OFFLOAD_SCTP_CKSUM	0x00020000
>  #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM  0x00040000
>  #define DEV_RX_OFFLOAD_RSS_HASH		0x00080000
> +#define DEV_RX_OFFLOAD_RSS_LEVEL	0x00100000
> 
>  #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \
>  				 DEV_RX_OFFLOAD_UDP_CKSUM | \
> --
> 2.21.0 (Apple Git-122.2)
  
Andrew Rybchenko Dec. 9, 2019, 7:35 a.m. UTC | #5
On 12/7/19 10:56 PM, Ajit Khaparde wrote:
> On Sat, Dec 7, 2019 at 1:14 AM Andrew Rybchenko <arybchenko@solarflare.com>
> wrote:
> 
>> On 12/7/19 3:59 AM, Ajit Khaparde wrote:
>>> This patch adds ability to configure RSS hash level in hardware.
>>> This feature will allow an application to select RSS hash calculation
>>> on outer or inner headers for tunneled packets.
>>>
>>> Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
>>> ---
>>>   lib/librte_ethdev/rte_ethdev.h | 27 +++++++++++++++++++++++++++
>>>   1 file changed, 27 insertions(+)
>>>
>>> diff --git a/lib/librte_ethdev/rte_ethdev.h
>> b/lib/librte_ethdev/rte_ethdev.h
>>> index 18a9defc2..5189bdbab 100644
>>> --- a/lib/librte_ethdev/rte_ethdev.h
>>> +++ b/lib/librte_ethdev/rte_ethdev.h
>>> @@ -444,11 +444,35 @@ struct rte_vlan_filter_conf {
>>>    * The *rss_hf* field of the *rss_conf* structure indicates the
>> different
>>>    * types of IPv4/IPv6 packets to which the RSS hashing must be applied.
>>>    * Supplying an *rss_hf* equal to zero disables the RSS feature.
>>> + *
>>> + * The *rss_level* field of the *rss_conf* structure indicates the
>>> + * Packet encapsulation level RSS hash @p types apply to.
>>> + *
>>> + * - @p 0 requests the default behavior. Depending on the packet
>>> + *   type, it can mean outermost, innermost, anything in between or
>>> + *   even no RSS.
>>> + *
>>> + *   It basically stands for the innermost encapsulation level RSS
>>> + *   can be performed on according to PMD and device capabilities.
>>> + *
>>> + * - @p 1 requests RSS to be performed on the outermost packet
>>> + *   encapsulation level.
>>> + *
>>> + * - @p 2 and subsequent values request RSS to be performed on the
>>> + *   specified inner packet encapsulation level, from outermost to
>>> + *   innermost (lower to higher values).
>>> + *
>>> + * Support for values other than @p 0 is dependent on the underlying
>>> + * hardware in use.
>>> + *
>>> + * Requesting a specific RSS level on unrecognized traffic results
>>> + * in undefined behavior.
>>>    */
>>>   struct rte_eth_rss_conf {
>>>       uint8_t *rss_key;    /**< If not NULL, 40-byte hash key. */
>>>       uint8_t rss_key_len; /**< hash key length in bytes. */
>>>       uint64_t rss_hf;     /**< Hash functions to apply - see below. */
>>> +     uint32_t rss_level;  /**< RSS hash level */
>>>   };
>>
>> I'm not sure that offload flag is required in this case.
>> I think maximum supported rss_level in dev_info will provide
> 
> more information and per-queue level does not make sense

I think information about maximum RSS hash level could be
useful. At least it provides clear information about
limitations instead of attempt to configure with RSS level
equal to 3 and getting error without clear understanding why
if the level is not supported.

>> in this case. Even if per-queue group control is required,
>> it should be doable via rte_flow API RSS action.
>>
> This is dev config and not flow specific configuration. Ofcourse while
> passing
> the rss_config, not all the queues may be specified, but that is not a new
> behavior and it is upto the application anyway.

Yes, of course. I was just trying to explain why Rx offload is
not required and it should be just dev_info field with maximum
RSS level supported.

> Are we transitioning the device level configuration to rte_flow/flow
> based scheme?

No-no, see above.

>> Anyway, it looks like it is ABI breakage with all consequences.
>> In 64-bit case it is possible to put it before rss_hf to avoid
>> ABI breakage, but it will break ABI on 32-bit anyway.
>>
> Right.
> I sent the proposal for review early to get it cleaned up and ready
> when the window opens.

OK, good.

>>
>>>   /*
>>> @@ -599,6 +623,8 @@ rte_eth_rss_hf_refine(uint64_t rss_hf)
>>>       ETH_RSS_GENEVE | \
>>>       ETH_RSS_NVGRE)
>>>
>>> +#define ETH_RSS_LEVEL_DEFAULT        0
>>> +
>>>   /*
>>>    * Definitions used for redirection table entry size.
>>>    * Some RSS RETA sizes may not be supported by some drivers, check the
>>> @@ -1103,6 +1129,7 @@ struct rte_eth_conf {
>>>   #define DEV_RX_OFFLOAD_SCTP_CKSUM   0x00020000
>>>   #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM  0x00040000
>>>   #define DEV_RX_OFFLOAD_RSS_HASH             0x00080000
>>> +#define DEV_RX_OFFLOAD_RSS_LEVEL     0x00100000
>>>
>>>   #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \
>>>                                DEV_RX_OFFLOAD_UDP_CKSUM | \
>>>
>>
>>
  
Ferruh Yigit Dec. 9, 2019, 2:41 p.m. UTC | #6
On 12/9/2019 7:35 AM, Andrew Rybchenko wrote:
> On 12/7/19 10:56 PM, Ajit Khaparde wrote:
>> On Sat, Dec 7, 2019 at 1:14 AM Andrew Rybchenko <arybchenko@solarflare.com>
>> wrote:
>>
>>> On 12/7/19 3:59 AM, Ajit Khaparde wrote:
>>>> This patch adds ability to configure RSS hash level in hardware.
>>>> This feature will allow an application to select RSS hash calculation
>>>> on outer or inner headers for tunneled packets.
>>>>
>>>> Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
>>>> ---
>>>>   lib/librte_ethdev/rte_ethdev.h | 27 +++++++++++++++++++++++++++
>>>>   1 file changed, 27 insertions(+)
>>>>
>>>> diff --git a/lib/librte_ethdev/rte_ethdev.h
>>> b/lib/librte_ethdev/rte_ethdev.h
>>>> index 18a9defc2..5189bdbab 100644
>>>> --- a/lib/librte_ethdev/rte_ethdev.h
>>>> +++ b/lib/librte_ethdev/rte_ethdev.h
>>>> @@ -444,11 +444,35 @@ struct rte_vlan_filter_conf {
>>>>    * The *rss_hf* field of the *rss_conf* structure indicates the
>>> different
>>>>    * types of IPv4/IPv6 packets to which the RSS hashing must be applied.
>>>>    * Supplying an *rss_hf* equal to zero disables the RSS feature.
>>>> + *
>>>> + * The *rss_level* field of the *rss_conf* structure indicates the
>>>> + * Packet encapsulation level RSS hash @p types apply to.
>>>> + *
>>>> + * - @p 0 requests the default behavior. Depending on the packet
>>>> + *   type, it can mean outermost, innermost, anything in between or
>>>> + *   even no RSS.
>>>> + *
>>>> + *   It basically stands for the innermost encapsulation level RSS
>>>> + *   can be performed on according to PMD and device capabilities.
>>>> + *
>>>> + * - @p 1 requests RSS to be performed on the outermost packet
>>>> + *   encapsulation level.
>>>> + *
>>>> + * - @p 2 and subsequent values request RSS to be performed on the
>>>> + *   specified inner packet encapsulation level, from outermost to
>>>> + *   innermost (lower to higher values).
>>>> + *
>>>> + * Support for values other than @p 0 is dependent on the underlying
>>>> + * hardware in use.
>>>> + *
>>>> + * Requesting a specific RSS level on unrecognized traffic results
>>>> + * in undefined behavior.
>>>>    */
>>>>   struct rte_eth_rss_conf {
>>>>       uint8_t *rss_key;    /**< If not NULL, 40-byte hash key. */
>>>>       uint8_t rss_key_len; /**< hash key length in bytes. */
>>>>       uint64_t rss_hf;     /**< Hash functions to apply - see below. */
>>>> +     uint32_t rss_level;  /**< RSS hash level */
>>>>   };
>>>
>>> I'm not sure that offload flag is required in this case.
>>> I think maximum supported rss_level in dev_info will provide
>>
>> more information and per-queue level does not make sense
> 
> I think information about maximum RSS hash level could be
> useful. At least it provides clear information about
> limitations instead of attempt to configure with RSS level
> equal to 3 and getting error without clear understanding why
> if the level is not supported.

+1 to not have 'DEV_RX_OFFLOAD_RSS_LEVEL' offload flag.

The existing 'DEV_RX_OFFLOAD_RSS_HASH' offload flag is just to enabling copying
calculated hash value to mbuf field, RSS configuration still done via the
configure API and related config structs.
Since there is single field in mbuf for rss hash, no need additional offload flags.

But I think we can have new field in rss config, and Andrew's suggestion to have
it in dev_info looks reasonable.

As Konstantin's point, do we really have multiple level, or just inner/outer?
If it is just inner/outer perhaps we can simplify the config/logic to enums
instead of 32bit rss levels?


> 
>>> in this case. Even if per-queue group control is required,
>>> it should be doable via rte_flow API RSS action.
>>>
>> This is dev config and not flow specific configuration. Ofcourse while
>> passing
>> the rss_config, not all the queues may be specified, but that is not a new
>> behavior and it is upto the application anyway.
> 
> Yes, of course. I was just trying to explain why Rx offload is
> not required and it should be just dev_info field with maximum
> RSS level supported.
> 
>> Are we transitioning the device level configuration to rte_flow/flow
>> based scheme?
> 
> No-no, see above.
> 
>>> Anyway, it looks like it is ABI breakage with all consequences.
>>> In 64-bit case it is possible to put it before rss_hf to avoid
>>> ABI breakage, but it will break ABI on 32-bit anyway.
>>>
>> Right.
>> I sent the proposal for review early to get it cleaned up and ready
>> when the window opens.
> 
> OK, good.

Just to reminder that unless something changes that window is 20.11, so there is
quite a lot of time for it.

We can clarify with Thomas, but as far as I understand we still need to update
(and ack) the deprecation notices for possible ABI breakages for next breakage
window.

> 
>>>
>>>>   /*
>>>> @@ -599,6 +623,8 @@ rte_eth_rss_hf_refine(uint64_t rss_hf)
>>>>       ETH_RSS_GENEVE | \
>>>>       ETH_RSS_NVGRE)
>>>>
>>>> +#define ETH_RSS_LEVEL_DEFAULT        0
>>>> +
>>>>   /*
>>>>    * Definitions used for redirection table entry size.
>>>>    * Some RSS RETA sizes may not be supported by some drivers, check the
>>>> @@ -1103,6 +1129,7 @@ struct rte_eth_conf {
>>>>   #define DEV_RX_OFFLOAD_SCTP_CKSUM   0x00020000
>>>>   #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM  0x00040000
>>>>   #define DEV_RX_OFFLOAD_RSS_HASH             0x00080000
>>>> +#define DEV_RX_OFFLOAD_RSS_LEVEL     0x00100000
>>>>
>>>>   #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \
>>>>                                DEV_RX_OFFLOAD_UDP_CKSUM | \
>>>>
>>>
>>>
>
  

Patch

diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 18a9defc2..5189bdbab 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -444,11 +444,35 @@  struct rte_vlan_filter_conf {
  * The *rss_hf* field of the *rss_conf* structure indicates the different
  * types of IPv4/IPv6 packets to which the RSS hashing must be applied.
  * Supplying an *rss_hf* equal to zero disables the RSS feature.
+ *
+ * The *rss_level* field of the *rss_conf* structure indicates the
+ * Packet encapsulation level RSS hash @p types apply to.
+ *
+ * - @p 0 requests the default behavior. Depending on the packet
+ *   type, it can mean outermost, innermost, anything in between or
+ *   even no RSS.
+ *
+ *   It basically stands for the innermost encapsulation level RSS
+ *   can be performed on according to PMD and device capabilities.
+ *
+ * - @p 1 requests RSS to be performed on the outermost packet
+ *   encapsulation level.
+ *
+ * - @p 2 and subsequent values request RSS to be performed on the
+ *   specified inner packet encapsulation level, from outermost to
+ *   innermost (lower to higher values).
+ *
+ * Support for values other than @p 0 is dependent on the underlying
+ * hardware in use.
+ *
+ * Requesting a specific RSS level on unrecognized traffic results
+ * in undefined behavior.
  */
 struct rte_eth_rss_conf {
 	uint8_t *rss_key;    /**< If not NULL, 40-byte hash key. */
 	uint8_t rss_key_len; /**< hash key length in bytes. */
 	uint64_t rss_hf;     /**< Hash functions to apply - see below. */
+	uint32_t rss_level;  /**< RSS hash level */
 };
 
 /*
@@ -599,6 +623,8 @@  rte_eth_rss_hf_refine(uint64_t rss_hf)
 	ETH_RSS_GENEVE | \
 	ETH_RSS_NVGRE)
 
+#define ETH_RSS_LEVEL_DEFAULT	0
+
 /*
  * Definitions used for redirection table entry size.
  * Some RSS RETA sizes may not be supported by some drivers, check the
@@ -1103,6 +1129,7 @@  struct rte_eth_conf {
 #define DEV_RX_OFFLOAD_SCTP_CKSUM	0x00020000
 #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM  0x00040000
 #define DEV_RX_OFFLOAD_RSS_HASH		0x00080000
+#define DEV_RX_OFFLOAD_RSS_LEVEL	0x00100000
 
 #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \
 				 DEV_RX_OFFLOAD_UDP_CKSUM | \