diff mbox

[dpdk-dev] net/mlx5: setup RSS regardless of queue count

Message ID 20180319163007.11516-1-allain.legacy@windriver.com (mailing list archive)
State Superseded, archived
Delegated to: Shahaf Shuler
Headers

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Allain Legacy March 19, 2018, 4:30 p.m. UTC 
  From: Dahir Osman <dahir.osman@windriver.com>

In some environments it is desirable to have the NIC perform RSS
normally on the packet regardless of the number of queues configured.
The RSS hash result that is stored in the mbuf can then be used by
the application to make decisions about how to distribute workloads
to threads, secondary processes, or even virtual machines if the
application is a virtual switch.  This change to the mlx5 driver
aligns with how other drivers in the Intel family work.

Signed-off-by: Allain Legacy <allain.legacy@windriver.com>
---
 drivers/net/mlx5/mlx5_rxq.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comments

Nélio Laranjeiro March 20, 2018, 12:26 p.m. UTC | #1 
On Mon, Mar 19, 2018 at 11:30:07AM -0500, Allain Legacy wrote:
> From: Dahir Osman <dahir.osman@windriver.com>
> 
> In some environments it is desirable to have the NIC perform RSS
> normally on the packet regardless of the number of queues configured.
> The RSS hash result that is stored in the mbuf can then be used by
> the application to make decisions about how to distribute workloads
> to threads, secondary processes, or even virtual machines if the
> application is a virtual switch.  This change to the mlx5 driver
> aligns with how other drivers in the Intel family work.
> 
> Signed-off-by: Allain Legacy <allain.legacy@windriver.com>
> ---
>  drivers/net/mlx5/mlx5_rxq.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
> index ff58c4921..e6b05b0ad 100644
> --- a/drivers/net/mlx5/mlx5_rxq.c
> +++ b/drivers/net/mlx5/mlx5_rxq.c
> @@ -1028,7 +1028,8 @@ mlx5_priv_rxq_new(struct priv *priv, uint16_t idx, uint16_t desc,
>  	      tmpl->rxq.crc_present ? "disabled" : "enabled",
>  	      tmpl->rxq.crc_present << 2);
>  	/* Save port ID. */
> -	tmpl->rxq.rss_hash = priv->rxqs_n > 1;
> +	tmpl->rxq.rss_hash = (!!(dev->data->dev_conf.rxmode.mq_mode &
> +				 ETH_MQ_RX_RSS));
>  	tmpl->rxq.port_id = dev->data->port_id;
>  	tmpl->priv = priv;
>  	tmpl->rxq.mp = mp;
> -- 
> 2.12.1

Unfortunately, is not enough to have a valid RSS hash result when the
PMD has a single Rx queue, a little more work needs to be handled in the
mlx5_flow.c engine to configure the hash field in Verbs Hash Rx queues
when a single queues is being used.

Thanks,
Allain Legacy March 20, 2018, 1:10 p.m. UTC | #2 
> -----Original Message-----
> From: Nélio Laranjeiro [mailto:nelio.laranjeiro@6wind.com]
> Sent: Tuesday, March 20, 2018 8:26 AM
...
> 
> Unfortunately, is not enough to have a valid RSS hash result when the PMD
> has a single Rx queue, a little more work needs to be handled in the
> mlx5_flow.c engine to configure the hash field in Verbs Hash Rx queues
> when a single queues is being used.

Ok, thanks.  We will take another look at this to see why it seems to be working in our environment.

Allain
Yongseok Koh March 20, 2018, 11:07 p.m. UTC | #3 
On Tue, Mar 20, 2018 at 01:26:08PM +0100, Nélio Laranjeiro wrote:
> On Mon, Mar 19, 2018 at 11:30:07AM -0500, Allain Legacy wrote:
> > From: Dahir Osman <dahir.osman@windriver.com>
> > 
> > In some environments it is desirable to have the NIC perform RSS
> > normally on the packet regardless of the number of queues configured.
> > The RSS hash result that is stored in the mbuf can then be used by
> > the application to make decisions about how to distribute workloads
> > to threads, secondary processes, or even virtual machines if the
> > application is a virtual switch.  This change to the mlx5 driver
> > aligns with how other drivers in the Intel family work.
> > 
> > Signed-off-by: Allain Legacy <allain.legacy@windriver.com>
> > ---
> >  drivers/net/mlx5/mlx5_rxq.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
> > index ff58c4921..e6b05b0ad 100644
> > --- a/drivers/net/mlx5/mlx5_rxq.c
> > +++ b/drivers/net/mlx5/mlx5_rxq.c
> > @@ -1028,7 +1028,8 @@ mlx5_priv_rxq_new(struct priv *priv, uint16_t idx, uint16_t desc,
> >  	      tmpl->rxq.crc_present ? "disabled" : "enabled",
> >  	      tmpl->rxq.crc_present << 2);
> >  	/* Save port ID. */
> > -	tmpl->rxq.rss_hash = priv->rxqs_n > 1;
> > +	tmpl->rxq.rss_hash = (!!(dev->data->dev_conf.rxmode.mq_mode &
> > +				 ETH_MQ_RX_RSS));
> >  	tmpl->rxq.port_id = dev->data->port_id;
> >  	tmpl->priv = priv;
> >  	tmpl->rxq.mp = mp;
> > -- 
> > 2.12.1
> 
> Unfortunately, is not enough to have a valid RSS hash result when the
> PMD has a single Rx queue, a little more work needs to be handled in the
> mlx5_flow.c engine to configure the hash field in Verbs Hash Rx queues
> when a single queues is being used.

It is good to have such feature like described in the commit message. And from
Allain's other email, it seems datapath of mlx5 ignores valid hash result from
HW for no reason if the number of queue is one. Allain, you wanted to fix that,
didn't you?

Nelio, can you please share a bit more idea of what should be done further in
mlx5_flow.c for this, so that Allain can come up with a right patch?


Thanks,
Yongseok
Nélio Laranjeiro March 21, 2018, 8:33 a.m. UTC | #4 
On Tue, Mar 20, 2018 at 04:07:11PM -0700, Yongseok Koh wrote:
> On Tue, Mar 20, 2018 at 01:26:08PM +0100, Nélio Laranjeiro wrote:
> > On Mon, Mar 19, 2018 at 11:30:07AM -0500, Allain Legacy wrote:
> > > From: Dahir Osman <dahir.osman@windriver.com>
> > > 
> > > In some environments it is desirable to have the NIC perform RSS
> > > normally on the packet regardless of the number of queues configured.
> > > The RSS hash result that is stored in the mbuf can then be used by
> > > the application to make decisions about how to distribute workloads
> > > to threads, secondary processes, or even virtual machines if the
> > > application is a virtual switch.  This change to the mlx5 driver
> > > aligns with how other drivers in the Intel family work.
> > > 
> > > Signed-off-by: Allain Legacy <allain.legacy@windriver.com>
> > > ---
> > >  drivers/net/mlx5/mlx5_rxq.c | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
> > > index ff58c4921..e6b05b0ad 100644
> > > --- a/drivers/net/mlx5/mlx5_rxq.c
> > > +++ b/drivers/net/mlx5/mlx5_rxq.c
> > > @@ -1028,7 +1028,8 @@ mlx5_priv_rxq_new(struct priv *priv, uint16_t idx, uint16_t desc,
> > >  	      tmpl->rxq.crc_present ? "disabled" : "enabled",
> > >  	      tmpl->rxq.crc_present << 2);
> > >  	/* Save port ID. */
> > > -	tmpl->rxq.rss_hash = priv->rxqs_n > 1;
> > > +	tmpl->rxq.rss_hash = (!!(dev->data->dev_conf.rxmode.mq_mode &
> > > +				 ETH_MQ_RX_RSS));
> > >  	tmpl->rxq.port_id = dev->data->port_id;
> > >  	tmpl->priv = priv;
> > >  	tmpl->rxq.mp = mp;
> > > -- 
> > > 2.12.1
> > 
> > Unfortunately, is not enough to have a valid RSS hash result when the
> > PMD has a single Rx queue, a little more work needs to be handled in the
> > mlx5_flow.c engine to configure the hash field in Verbs Hash Rx queues
> > when a single queues is being used.
> 
> It is good to have such feature like described in the commit message. And from
> Allain's other email, it seems datapath of mlx5 ignores valid hash result from
> HW for no reason if the number of queue is one.

Single queue means no RSS, it is not the datapath which ignore it, it is
the control plane which does not configure it.

> Allain, you wanted to fix that, didn't you?

From the patch above, it seems :)

> Nelio, can you please share a bit more idea of what should be done further in
> mlx5_flow.c for this, so that Allain can come up with a right patch?

The engine is not trivial as it needs to convert flow API to Verbs flows
which the restrictions from the MLX5 kernel driver.
Trying to give instructions ends by writing it down directly as I need
to test them to see if it does not break your requirements also.

Allain,  I will finalise this patch (today) and make some test, I'll
contact you once it is finish to see if the behavior you expect is
reached.

Thanks,
Allain Legacy March 21, 2018, 12:43 p.m. UTC | #5 
> -----Original Message-----
> From: Nélio Laranjeiro [mailto:nelio.laranjeiro@6wind.com]
> Sent: Wednesday, March 21, 2018 4:34 AM
<...>
> The engine is not trivial as it needs to convert flow API to Verbs flows which
> the restrictions from the MLX5 kernel driver.
> Trying to give instructions ends by writing it down directly as I need to test
> them to see if it does not break your requirements also.
> 
> Allain,  I will finalise this patch (today) and make some test, I'll contact you
> once it is finish to see if the behavior you expect is reached.
> 
Great, thanks!
diff mbox

Patch

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index ff58c4921..e6b05b0ad 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -1028,7 +1028,8 @@  mlx5_priv_rxq_new(struct priv *priv, uint16_t idx, uint16_t desc,
 	      tmpl->rxq.crc_present ? "disabled" : "enabled",
 	      tmpl->rxq.crc_present << 2);
 	/* Save port ID. */
-	tmpl->rxq.rss_hash = priv->rxqs_n > 1;
+	tmpl->rxq.rss_hash = (!!(dev->data->dev_conf.rxmode.mq_mode &
+				 ETH_MQ_RX_RSS));
 	tmpl->rxq.port_id = dev->data->port_id;
 	tmpl->priv = priv;
 	tmpl->rxq.mp = mp;