diff mbox series

ethdev: check for Rx RSS distribution and RSS hash

Message ID 1587990889-56408-1-git-send-email-oulijun@huawei.com (mailing list archive)
State Changes Requested, archived
Delegated to: Ferruh Yigit
Headers
Series ethdev: check for Rx RSS distribution and RSS hash |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/iol-intel-Performance fail Performance Testing issues
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-nxp-Performance success Performance Testing PASS
ci/travis-robot warning Travis build: failed
ci/iol-testing success Testing PASS
ci/Intel-compilation success Compilation OK

Commit Message

Lijun Ou April 27, 2020, 12:34 p.m. UTC 
  When rte api checks the Rx RSS distribution is enable but the RSS
hash is disabled, it will return an error.

Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
 lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

Comments

Andrew Rybchenko April 27, 2020, 12:49 p.m. UTC | #1 
On 4/27/20 3:34 PM, Lijun Ou wrote:
> When rte api checks the Rx RSS distribution is enable but the RSS
> hash is disabled, it will return an error.
>
> Signed-off-by: Lijun Ou <oulijun@huawei.com>
> ---
>  lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 0854ef8..07734c4 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>  		goto rollback;
>  	}
>  
> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
> +		RTE_ETHDEV_LOG(ERR,
> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
> +			port_id,
> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
> +		ret = -EINVAL;
> +		goto rollback;
> +	}
> +
>  	/*
>  	 * Setup new number of RX/TX queues and reconfigure device.
>  	 */

NACK. It is perfectly fine to do distribution, but do not need RSS hash
information.
 - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
 - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
   itself from HW to SW
Ferruh Yigit April 27, 2020, 1:27 p.m. UTC | #2 
On 4/27/2020 1:49 PM, Andrew Rybchenko wrote:
> On 4/27/20 3:34 PM, Lijun Ou wrote:
>> When rte api checks the Rx RSS distribution is enable but the RSS
>> hash is disabled, it will return an error.
>>
>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>> ---
>>  lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>  1 file changed, 11 insertions(+)
>>
>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>> index 0854ef8..07734c4 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>  		goto rollback;
>>  	}
>>  
>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>> +		RTE_ETHDEV_LOG(ERR,
>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>> +			port_id,
>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>> +		ret = -EINVAL;
>> +		goto rollback;
>> +	}
>> +
>>  	/*
>>  	 * Setup new number of RX/TX queues and reconfigure device.
>>  	 */
> 
> NACK. It is perfectly fine to do distribution, but do not need RSS hash
> information.
>  - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>  - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>    itself from HW to SW
> 

Hi Lijun,

As Andrew described, now 'DEV_RX_OFFLOAD_RSS_HASH' controls to copy calculated
hash value to 'mbuf::hash::rss' or not. When rss hash copied to mubf,
'PKT_RX_RSS_HASH' flag of 'mbuf::ol_flags' set to notify the application that
hash value is valid. This was a performance optimization.

We know above because we were involved in development of it, if this is not
clean for third party, can you please amend your patch to clarify above behavior?

Thanks,
ferruh
Lijun Ou April 29, 2020, 6:31 a.m. UTC | #3 
在 2020/4/27 20:49, Andrew Rybchenko 写道:
> On 4/27/20 3:34 PM, Lijun Ou wrote:
>> When rte api checks the Rx RSS distribution is enable but the RSS
>> hash is disabled, it will return an error.
>>
>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>> ---
>>   lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>   1 file changed, 11 insertions(+)
>>
>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>> index 0854ef8..07734c4 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>   		goto rollback;
>>   	}
>>   
>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>> +		RTE_ETHDEV_LOG(ERR,
>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>> +			port_id,
>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>> +		ret = -EINVAL;
>> +		goto rollback;
>> +	}
>> +
>>   	/*
>>   	 * Setup new number of RX/TX queues and reconfigure device.
>>   	 */
> 
> NACK. It is perfectly fine to do distribution, but do not need RSS hash
> information.
>   - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>   - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>     itself from HW to SW
> 
Thanks for your detail explanation. I still have a question, what is the 
use of setting DEV_RX_OFFLOAD_HASH, what is the difference between user 
configuration DEV_RX_OFFLOAD_HASH and no configuration?

> .
>
Lijun Ou April 29, 2020, 6:41 a.m. UTC | #4 
在 2020/4/27 21:27, Ferruh Yigit 写道:
> On 4/27/2020 1:49 PM, Andrew Rybchenko wrote:
>> On 4/27/20 3:34 PM, Lijun Ou wrote:
>>> When rte api checks the Rx RSS distribution is enable but the RSS
>>> hash is disabled, it will return an error.
>>>
>>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>>> ---
>>>   lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>>   1 file changed, 11 insertions(+)
>>>
>>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>>> index 0854ef8..07734c4 100644
>>> --- a/lib/librte_ethdev/rte_ethdev.c
>>> +++ b/lib/librte_ethdev/rte_ethdev.c
>>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>   		goto rollback;
>>>   	}
>>>   
>>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>>> +		RTE_ETHDEV_LOG(ERR,
>>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>>> +			port_id,
>>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>>> +		ret = -EINVAL;
>>> +		goto rollback;
>>> +	}
>>> +
>>>   	/*
>>>   	 * Setup new number of RX/TX queues and reconfigure device.
>>>   	 */
>>
>> NACK. It is perfectly fine to do distribution, but do not need RSS hash
>> information.
>>   - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>>   - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>>     itself from HW to SW
>>
> 
> Hi Lijun,
> 
> As Andrew described, now 'DEV_RX_OFFLOAD_RSS_HASH' controls to copy calculated
> hash value to 'mbuf::hash::rss' or not. When rss hash copied to mubf,
> 'PKT_RX_RSS_HASH' flag of 'mbuf::ol_flags' set to notify the application that
> hash value is valid. This was a performance optimization.
>
if the user is not configure the DEV_RX_OFFLOAD_RSS_HASH and the users 
add the ETH_MQ_RX_RSS_FLAG， as a result, the hardware has calculated the 
hash result and distributed, the mbuf::o1_flags have set the 
PKT_RX_RSS_HASH and the mbuf::hash::rss have set the hash result. What 
are the advantage of this configuration with DEV_RX_OFFLOAD_RSS_HASH?

Can I understand it this way, If the user does not apply this rss hash, 
does it mean that rss cannot be used to improve performance, even if the 
hardware has calculated the rss hash result?
> We know above because we were involved in development of it, if this is not
> clean for third party, can you please amend your patch to clarify above behavior?
> 
> Thanks,
> ferruh
> 
> .
>
Ferruh Yigit April 29, 2020, 12:42 p.m. UTC | #5 
On 4/29/2020 7:41 AM, oulijun wrote:
> 
> 
> 在 2020/4/27 21:27, Ferruh Yigit 写道:
>> On 4/27/2020 1:49 PM, Andrew Rybchenko wrote:
>>> On 4/27/20 3:34 PM, Lijun Ou wrote:
>>>> When rte api checks the Rx RSS distribution is enable but the RSS
>>>> hash is disabled, it will return an error.
>>>>
>>>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>>>> ---
>>>>   lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>>>   1 file changed, 11 insertions(+)
>>>>
>>>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>>>> index 0854ef8..07734c4 100644
>>>> --- a/lib/librte_ethdev/rte_ethdev.c
>>>> +++ b/lib/librte_ethdev/rte_ethdev.c
>>>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>>   		goto rollback;
>>>>   	}
>>>>   
>>>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>>>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>>>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>>>> +		RTE_ETHDEV_LOG(ERR,
>>>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>>>> +			port_id,
>>>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>>>> +		ret = -EINVAL;
>>>> +		goto rollback;
>>>> +	}
>>>> +
>>>>   	/*
>>>>   	 * Setup new number of RX/TX queues and reconfigure device.
>>>>   	 */
>>>
>>> NACK. It is perfectly fine to do distribution, but do not need RSS hash
>>> information.
>>>   - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>>>   - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>>>     itself from HW to SW
>>>
>>
>> Hi Lijun,
>>
>> As Andrew described, now 'DEV_RX_OFFLOAD_RSS_HASH' controls to copy calculated
>> hash value to 'mbuf::hash::rss' or not. When rss hash copied to mubf,
>> 'PKT_RX_RSS_HASH' flag of 'mbuf::ol_flags' set to notify the application that
>> hash value is valid. This was a performance optimization.
>>
> if the user is not configure the DEV_RX_OFFLOAD_RSS_HASH and the users 
> add the ETH_MQ_RX_RSS_FLAG， as a result, the hardware has calculated the 
> hash result and distributed, the mbuf::o1_flags have set the 
> PKT_RX_RSS_HASH and the mbuf::hash::rss have set the hash result. What 
> are the advantage of this configuration with DEV_RX_OFFLOAD_RSS_HASH?

If 'DEV_RX_OFFLOAD_RSS_HASH' NOT set, but 'ETH_MQ_RX_RSS_FLAG' set, RSS should
work as expected but 'mbuf::hash::rss' not filled.

This is optimization for the case user doesn't need the calculated hash value,
and seems this optimization has bigger affect in some NICs that pays some cost
to receive the HW calculated hash values.

User requesting 'DEV_RX_OFFLOAD_RSS_HASH' offload should enable driver to update
'mbuf::hash::rss' and 'PKT_RX_RSS_HASH' in 'mbuf::ol_flags' addition to RSS feature.

> 
> Can I understand it this way, If the user does not apply this rss hash, 
> does it mean that rss cannot be used to improve performance, even if the 
> hardware has calculated the rss hash result?

If 'DEV_RX_OFFLOAD_RSS_HASH' is NOT set, application won't able to receive the
HW calculated hash values.

>> We know above because we were involved in development of it, if this is not
>> clean for third party, can you please amend your patch to clarify above behavior?
>>
>> Thanks,
>> ferruh
>>
>> .
>>
>
Ferruh Yigit May 1, 2020, 4:04 p.m. UTC | #6 
On 4/29/2020 1:42 PM, Ferruh Yigit wrote:
> On 4/29/2020 7:41 AM, oulijun wrote:
>>
>>
>> 在 2020/4/27 21:27, Ferruh Yigit 写道:
>>> On 4/27/2020 1:49 PM, Andrew Rybchenko wrote:
>>>> On 4/27/20 3:34 PM, Lijun Ou wrote:
>>>>> When rte api checks the Rx RSS distribution is enable but the RSS
>>>>> hash is disabled, it will return an error.
>>>>>
>>>>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>>>>> ---
>>>>>   lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>>>>   1 file changed, 11 insertions(+)
>>>>>
>>>>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>>>>> index 0854ef8..07734c4 100644
>>>>> --- a/lib/librte_ethdev/rte_ethdev.c
>>>>> +++ b/lib/librte_ethdev/rte_ethdev.c
>>>>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>>>   		goto rollback;
>>>>>   	}
>>>>>   
>>>>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>>>>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>>>>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>>>>> +		RTE_ETHDEV_LOG(ERR,
>>>>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>>>>> +			port_id,
>>>>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>>>>> +		ret = -EINVAL;
>>>>> +		goto rollback;
>>>>> +	}
>>>>> +
>>>>>   	/*
>>>>>   	 * Setup new number of RX/TX queues and reconfigure device.
>>>>>   	 */
>>>>
>>>> NACK. It is perfectly fine to do distribution, but do not need RSS hash
>>>> information.
>>>>   - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>>>>   - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>>>>     itself from HW to SW
>>>>
>>>
>>> Hi Lijun,
>>>
>>> As Andrew described, now 'DEV_RX_OFFLOAD_RSS_HASH' controls to copy calculated
>>> hash value to 'mbuf::hash::rss' or not. When rss hash copied to mubf,
>>> 'PKT_RX_RSS_HASH' flag of 'mbuf::ol_flags' set to notify the application that
>>> hash value is valid. This was a performance optimization.
>>>
>> if the user is not configure the DEV_RX_OFFLOAD_RSS_HASH and the users 
>> add the ETH_MQ_RX_RSS_FLAG， as a result, the hardware has calculated the 
>> hash result and distributed, the mbuf::o1_flags have set the 
>> PKT_RX_RSS_HASH and the mbuf::hash::rss have set the hash result. What 
>> are the advantage of this configuration with DEV_RX_OFFLOAD_RSS_HASH?
> 
> If 'DEV_RX_OFFLOAD_RSS_HASH' NOT set, but 'ETH_MQ_RX_RSS_FLAG' set, RSS should
> work as expected but 'mbuf::hash::rss' not filled.
> 
> This is optimization for the case user doesn't need the calculated hash value,
> and seems this optimization has bigger affect in some NICs that pays some cost
> to receive the HW calculated hash values.
> 
> User requesting 'DEV_RX_OFFLOAD_RSS_HASH' offload should enable driver to update
> 'mbuf::hash::rss' and 'PKT_RX_RSS_HASH' in 'mbuf::ol_flags' addition to RSS feature.

I hope this is clear now, and back to my first point if you think above behavior
is not clear in the doc or API comments, can you please send a patch to clarify
it better?

Thanks,
ferruh

> 
>>
>> Can I understand it this way, If the user does not apply this rss hash, 
>> does it mean that rss cannot be used to improve performance, even if the 
>> hardware has calculated the rss hash result?
> 
> If 'DEV_RX_OFFLOAD_RSS_HASH' is NOT set, application won't able to receive the
> HW calculated hash values.
> 
>>> We know above because we were involved in development of it, if this is not
>>> clean for third party, can you please amend your patch to clarify above behavior?
>>>
>>> Thanks,
>>> ferruh
>>>
>>> .
>>>
>>
>
diff mbox series

Patch

diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 0854ef8..07734c4 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1411,6 +1411,17 @@  rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
 		goto rollback;
 	}
 
+	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
+	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
+	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
+		RTE_ETHDEV_LOG(ERR,
+			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
+			port_id,
+			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
+		ret = -EINVAL;
+		goto rollback;
+	}
+
 	/*
 	 * Setup new number of RX/TX queues and reconfigure device.
 	 */