[dpdk-dev,v3,04/10] app/testpmd: convert to new Ethdev Tx offloads API

Message ID ae1ada03c1d83b074e920650da2614bc6d007f7d.1514281260.git.shahafs@mellanox.com (mailing list archive)
State Superseded, archived
Headers

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation fail apply patch file failure

Commit Message

Shahaf Shuler Dec. 26, 2017, 9:44 a.m. UTC
  Ethdev Tx offloads API has changed since:

commit cba7f53b717d ("ethdev: introduce Tx queue offloads API")

Convert the application to use the new API.

This patch mandates the port to be stopped when configure the Tx
offloads. This is because the PMD must be aware to the offloads changes
on the device and queue configuration.

Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
---
 app/test-pmd/cmdline.c | 90 ++++++++++++++++++++++++++++++++++++++++++---
 app/test-pmd/config.c  | 55 ++++++++++++++++++---------
 app/test-pmd/testpmd.c |  3 ++
 3 files changed, 124 insertions(+), 24 deletions(-)
  

Comments

Wenzhuo Lu Jan. 9, 2018, 5:27 a.m. UTC | #1
Hi Shahaf,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shahaf Shuler
> Sent: Tuesday, December 26, 2017 5:44 PM
> To: Wu, Jingjing <jingjing.wu@intel.com>; Yigit, Ferruh
> <ferruh.yigit@intel.com>
> Cc: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new Ethdev
> Tx offloads API
> 
> Ethdev Tx offloads API has changed since:
> 
> commit cba7f53b717d ("ethdev: introduce Tx queue offloads API")
> 
> Convert the application to use the new API.
> 
> This patch mandates the port to be stopped when configure the Tx offloads.
> This is because the PMD must be aware to the offloads changes on the
> device and queue configuration.
> 
> Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
> ---
>  app/test-pmd/cmdline.c | 90
> ++++++++++++++++++++++++++++++++++++++++++---
>  app/test-pmd/config.c  | 55 ++++++++++++++++++---------  app/test-
> pmd/testpmd.c |  3 ++
>  3 files changed, 124 insertions(+), 24 deletions(-)
> 
> diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index
> d8c73a9b1..58125839a 100644
> --- a/app/test-pmd/cmdline.c
> +++ b/app/test-pmd/cmdline.c
> @@ -3439,7 +3439,14 @@ cmd_tx_vlan_set_parsed(void *parsed_result,  {
>  	struct cmd_tx_vlan_set_result *res = parsed_result;
> 
> +	if (!port_is_stopped(res->port_id)) {
> +		printf("Please stop port %d first\n", res->port_id);
> +		return;
> +	}
> +
>  	tx_vlan_set(res->port_id, res->vlan_id);
> +
> +	cmd_reconfig_device_queue(res->port_id, 1, 1);
>  }
> 
I do have some concern about this behavior change, 'port_is_stopped' and ' cmd_reconfig_device_queue '.
1, seems this behavior change is not necessary for using the new offload API. Maybe splitting this patch to 2 is better.
2, some NICs doesn't need to be stopped or re-configured to make vlan functions enabled. That's why the original code doesn't have this restriction. Maybe figuring out a way to do the restriction in the driver layer is better.
  
Shahaf Shuler Jan. 9, 2018, 6:47 a.m. UTC | #2
Hi,

Tuesday, January 9, 2018 7:28 AM, Lu, Wenzhuo:
> Hi Shahaf,
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shahaf Shuler
> > Sent: Tuesday, December 26, 2017 5:44 PM
> > To: Wu, Jingjing <jingjing.wu@intel.com>; Yigit, Ferruh
> > <ferruh.yigit@intel.com>
> > Cc: dev@dpdk.org
> > Subject: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new
> > Ethdev Tx offloads API
> >
> > Ethdev Tx offloads API has changed since:
> >
> > commit cba7f53b717d ("ethdev: introduce Tx queue offloads API")
> >
> > Convert the application to use the new API.
> >
> > This patch mandates the port to be stopped when configure the Tx
> offloads.
> > This is because the PMD must be aware to the offloads changes on the
> > device and queue configuration.
> >
> > Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
> > ---
> >  app/test-pmd/cmdline.c | 90
> > ++++++++++++++++++++++++++++++++++++++++++---
> >  app/test-pmd/config.c  | 55 ++++++++++++++++++---------  app/test-
> > pmd/testpmd.c |  3 ++
> >  3 files changed, 124 insertions(+), 24 deletions(-)
> >
> > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index
> > d8c73a9b1..58125839a 100644
> > --- a/app/test-pmd/cmdline.c
> > +++ b/app/test-pmd/cmdline.c
> > @@ -3439,7 +3439,14 @@ cmd_tx_vlan_set_parsed(void *parsed_result,
> {
> >  	struct cmd_tx_vlan_set_result *res = parsed_result;
> >
> > +	if (!port_is_stopped(res->port_id)) {
> > +		printf("Please stop port %d first\n", res->port_id);
> > +		return;
> > +	}
> > +
> >  	tx_vlan_set(res->port_id, res->vlan_id);
> > +
> > +	cmd_reconfig_device_queue(res->port_id, 1, 1);
> >  }
> >
> I do have some concern about this behavior change, 'port_is_stopped' and '
> cmd_reconfig_device_queue '.
> 1, seems this behavior change is not necessary for using the new offload API.
> Maybe splitting this patch to 2 is better.

It is related. Because as part of the offloads API the Tx offloads configuration is set on the rte_eth_dev_configure. To configure a device it must be stopped in advance. 

Also, per my understanding, the only API which allows offloads configuration on the flight are the Rx VLAN offloads. 
As you can see, this API is respected and no need to check the port status in it

> 2, some NICs doesn't need to be stopped or re-configured to make vlan
> functions enabled. 

Only for the Rx vlan. The Tx vlan insert is not configured on the flight. 

That's why the original code doesn't have this restriction.
> Maybe figuring out a way to do the restriction in the driver layer is better.

The original code doesn't have such restriction because testpmd wasn't configuring Tx offloads at all. It used it's own Tx offloads enum (which I removed on later patches) to indicate which offloads is set. From the device perspective all the Tx offloads should be set. 
It did had a way to configure offloads with txqflags parameter, but this required queues reconfigurations.
  
Wenzhuo Lu Jan. 9, 2018, 7:13 a.m. UTC | #3
Hi Shahaf,

> -----Original Message-----
> From: Shahaf Shuler [mailto:shahafs@mellanox.com]
> Sent: Tuesday, January 9, 2018 2:48 PM
> To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new
> Ethdev Tx offloads API
> 
> Hi,
> 
> Tuesday, January 9, 2018 7:28 AM, Lu, Wenzhuo:
> > Hi Shahaf,
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shahaf Shuler
> > > Sent: Tuesday, December 26, 2017 5:44 PM
> > > To: Wu, Jingjing <jingjing.wu@intel.com>; Yigit, Ferruh
> > > <ferruh.yigit@intel.com>
> > > Cc: dev@dpdk.org
> > > Subject: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new
> > > Ethdev Tx offloads API
> > >
> > > Ethdev Tx offloads API has changed since:
> > >
> > > commit cba7f53b717d ("ethdev: introduce Tx queue offloads API")
> > >
> > > Convert the application to use the new API.
> > >
> > > This patch mandates the port to be stopped when configure the Tx
> > offloads.
> > > This is because the PMD must be aware to the offloads changes on the
> > > device and queue configuration.
> > >
> > > Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
> > > ---
> > >  app/test-pmd/cmdline.c | 90
> > > ++++++++++++++++++++++++++++++++++++++++++---
> > >  app/test-pmd/config.c  | 55 ++++++++++++++++++---------  app/test-
> > > pmd/testpmd.c |  3 ++
> > >  3 files changed, 124 insertions(+), 24 deletions(-)
> > >
> > > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index
> > > d8c73a9b1..58125839a 100644
> > > --- a/app/test-pmd/cmdline.c
> > > +++ b/app/test-pmd/cmdline.c
> > > @@ -3439,7 +3439,14 @@ cmd_tx_vlan_set_parsed(void *parsed_result,
> > {
> > >  	struct cmd_tx_vlan_set_result *res = parsed_result;
> > >
> > > +	if (!port_is_stopped(res->port_id)) {
> > > +		printf("Please stop port %d first\n", res->port_id);
> > > +		return;
> > > +	}
> > > +
> > >  	tx_vlan_set(res->port_id, res->vlan_id);
> > > +
> > > +	cmd_reconfig_device_queue(res->port_id, 1, 1);
> > >  }
> > >
> > I do have some concern about this behavior change, 'port_is_stopped' and
> '
> > cmd_reconfig_device_queue '.
> > 1, seems this behavior change is not necessary for using the new offload
> API.
> > Maybe splitting this patch to 2 is better.
> 
> It is related. Because as part of the offloads API the Tx offloads configuration
> is set on the rte_eth_dev_configure. To configure a device it must be
> stopped in advance.
> 
> Also, per my understanding, the only API which allows offloads configuration
> on the flight are the Rx VLAN offloads.
> As you can see, this API is respected and no need to check the port status in
> it
> 
> > 2, some NICs doesn't need to be stopped or re-configured to make vlan
> > functions enabled.
> 
> Only for the Rx vlan. The Tx vlan insert is not configured on the flight.
> 
> That's why the original code doesn't have this restriction.
> > Maybe figuring out a way to do the restriction in the driver layer is better.
> 
> The original code doesn't have such restriction because testpmd wasn't
> configuring Tx offloads at all. It used it's own Tx offloads enum (which I
> removed on later patches) to indicate which offloads is set. From the device
> perspective all the Tx offloads should be set.
> It did had a way to configure offloads with txqflags parameter, but this
> required queues reconfigurations.
' cmd_tx_vlan_set_parsed' is the function which configures TX vlan insertion. This patch adds the restriction in it.
For example, on ixgbe it does support on the fly configuration. APP doesn't need to stop the device or reconfigure the queues.
  
Shahaf Shuler Jan. 9, 2018, 10:02 a.m. UTC | #4
Tuesday, January 9, 2018 9:14 AM, Lu, Wenzhuo:
> > The original code doesn't have such restriction because testpmd wasn't
> > configuring Tx offloads at all. It used it's own Tx offloads enum
> > (which I removed on later patches) to indicate which offloads is set.
> > From the device perspective all the Tx offloads should be set.
> > It did had a way to configure offloads with txqflags parameter, but
> > this required queues reconfigurations.
> ' cmd_tx_vlan_set_parsed' is the function which configures TX vlan insertion.
> This patch adds the restriction in it.
> For example, on ixgbe it does support on the fly configuration. APP doesn't
> need to stop the device or reconfigure the queues.

What is mean is that in the old offloads API, all the Tx offloads were enabled by default, besides the ones which were masked with txqflags.
This is why testpmd didn't have any limitation on changing the Tx offloads. All of them were enabled, it just needed to choose with which one to use.

Now, with the new offloads API, the Tx offloads are disabled by default, and enabled per application need on device and queue configuration.
In order to call device/queue configuration the port must be stopped. This is requirement from ethdev API.

Now, there is an API in ethdev to configure Rx VLAN offloads on the flight for supported devices.
If there are devices which supports it also in the Tx side, and you see a good use case for that, then need to think of new API in ethdev to configure Tx VLAN offloads on the flight.

However, the plain conversion of testpmd from the old to the new API requires the port to be stopped for all of the Tx offloads.
  
Wenzhuo Lu Jan. 9, 2018, 11:37 a.m. UTC | #5
Hi Shahaf,

> -----Original Message-----
> From: Shahaf Shuler [mailto:shahafs@mellanox.com]
> Sent: Tuesday, January 9, 2018 6:02 PM
> To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new
> Ethdev Tx offloads API
> 
> Tuesday, January 9, 2018 9:14 AM, Lu, Wenzhuo:
> > > The original code doesn't have such restriction because testpmd
> > > wasn't configuring Tx offloads at all. It used it's own Tx offloads
> > > enum (which I removed on later patches) to indicate which offloads is set.
> > > From the device perspective all the Tx offloads should be set.
> > > It did had a way to configure offloads with txqflags parameter, but
> > > this required queues reconfigurations.
> > ' cmd_tx_vlan_set_parsed' is the function which configures TX vlan
> insertion.
> > This patch adds the restriction in it.
> > For example, on ixgbe it does support on the fly configuration. APP
> > doesn't need to stop the device or reconfigure the queues.
> 
> What is mean is that in the old offloads API, all the Tx offloads were enabled
> by default, besides the ones which were masked with txqflags.
> This is why testpmd didn't have any limitation on changing the Tx offloads.
> All of them were enabled, it just needed to choose with which one to use.
> 
> Now, with the new offloads API, the Tx offloads are disabled by default, and
> enabled per application need on device and queue configuration.
> In order to call device/queue configuration the port must be stopped. This is
> requirement from ethdev API.
> 
> Now, there is an API in ethdev to configure Rx VLAN offloads on the flight for
> supported devices.
> If there are devices which supports it also in the Tx side, and you see a good
> use case for that, then need to think of new API in ethdev to configure Tx
> VLAN offloads on the flight.
> 
> However, the plain conversion of testpmd from the old to the new API
> requires the port to be stopped for all of the Tx offloads.
Let's just talk about TX vlan insertion. Actually there's no ethdev API for it. And no configuration for it. Because vlan insertion is simple, it only need to add a vlan tag when transmitting packets.
You can see it's per packet thing. APP can set the ol_flags and vlan_tci in the mbuf of every packet to enable this function for the packet or not. I think it does be a on the fly function on any NIC.
  
Ananyev, Konstantin Jan. 9, 2018, 12:14 p.m. UTC | #6
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Lu, Wenzhuo
> Sent: Tuesday, January 9, 2018 11:37 AM
> To: Shahaf Shuler <shahafs@mellanox.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new Ethdev Tx offloads API
> 
> Hi Shahaf,
> 
> > -----Original Message-----
> > From: Shahaf Shuler [mailto:shahafs@mellanox.com]
> > Sent: Tuesday, January 9, 2018 6:02 PM
> > To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Wu, Jingjing
> > <jingjing.wu@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
> > Cc: dev@dpdk.org
> > Subject: RE: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new
> > Ethdev Tx offloads API
> >
> > Tuesday, January 9, 2018 9:14 AM, Lu, Wenzhuo:
> > > > The original code doesn't have such restriction because testpmd
> > > > wasn't configuring Tx offloads at all. It used it's own Tx offloads
> > > > enum (which I removed on later patches) to indicate which offloads is set.
> > > > From the device perspective all the Tx offloads should be set.
> > > > It did had a way to configure offloads with txqflags parameter, but
> > > > this required queues reconfigurations.
> > > ' cmd_tx_vlan_set_parsed' is the function which configures TX vlan
> > insertion.
> > > This patch adds the restriction in it.
> > > For example, on ixgbe it does support on the fly configuration. APP
> > > doesn't need to stop the device or reconfigure the queues.
> >
> > What is mean is that in the old offloads API, all the Tx offloads were enabled
> > by default, besides the ones which were masked with txqflags.
> > This is why testpmd didn't have any limitation on changing the Tx offloads.
> > All of them were enabled, it just needed to choose with which one to use.
> >
> > Now, with the new offloads API, the Tx offloads are disabled by default, and
> > enabled per application need on device and queue configuration.
> > In order to call device/queue configuration the port must be stopped. This is
> > requirement from ethdev API.
> >
> > Now, there is an API in ethdev to configure Rx VLAN offloads on the flight for
> > supported devices.
> > If there are devices which supports it also in the Tx side, and you see a good
> > use case for that, then need to think of new API in ethdev to configure Tx
> > VLAN offloads on the flight.
> >
> > However, the plain conversion of testpmd from the old to the new API
> > requires the port to be stopped for all of the Tx offloads.
> Let's just talk about TX vlan insertion. Actually there's no ethdev API for it. And no configuration for it. Because vlan insertion is simple, it
> only need to add a vlan tag when transmitting packets.

You still need an appropriate TX function be selected, otherwise mbuf->vlan_tci will be just ignored.
So you still has to specify at queue_setup flag that you would like to have VLAN insertion offload enabled. 
Konstantin

> You can see it's per packet thing. APP can set the ol_flags and vlan_tci in the mbuf of every packet to enable this function for the packet or
> not. I think it does be a on the fly function on any NIC.
  
Wenzhuo Lu Jan. 10, 2018, 12:37 a.m. UTC | #7
Hi Konstantin,

> -----Original Message-----
> From: Ananyev, Konstantin
> Sent: Tuesday, January 9, 2018 8:15 PM
> To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Shahaf Shuler
> <shahafs@mellanox.com>; Wu, Jingjing <jingjing.wu@intel.com>; Yigit,
> Ferruh <ferruh.yigit@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new
> Ethdev Tx offloads API
> 
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Lu, Wenzhuo
> > Sent: Tuesday, January 9, 2018 11:37 AM
> > To: Shahaf Shuler <shahafs@mellanox.com>; Wu, Jingjing
> > <jingjing.wu@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new
> > Ethdev Tx offloads API
> >
> > Hi Shahaf,
> >
> > > -----Original Message-----
> > > From: Shahaf Shuler [mailto:shahafs@mellanox.com]
> > > Sent: Tuesday, January 9, 2018 6:02 PM
> > > To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Wu, Jingjing
> > > <jingjing.wu@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
> > > Cc: dev@dpdk.org
> > > Subject: RE: [dpdk-dev] [PATCH v3 04/10] app/testpmd: convert to new
> > > Ethdev Tx offloads API
> > >
> > > Tuesday, January 9, 2018 9:14 AM, Lu, Wenzhuo:
> > > > > The original code doesn't have such restriction because testpmd
> > > > > wasn't configuring Tx offloads at all. It used it's own Tx
> > > > > offloads enum (which I removed on later patches) to indicate which
> offloads is set.
> > > > > From the device perspective all the Tx offloads should be set.
> > > > > It did had a way to configure offloads with txqflags parameter,
> > > > > but this required queues reconfigurations.
> > > > ' cmd_tx_vlan_set_parsed' is the function which configures TX vlan
> > > insertion.
> > > > This patch adds the restriction in it.
> > > > For example, on ixgbe it does support on the fly configuration.
> > > > APP doesn't need to stop the device or reconfigure the queues.
> > >
> > > What is mean is that in the old offloads API, all the Tx offloads
> > > were enabled by default, besides the ones which were masked with
> txqflags.
> > > This is why testpmd didn't have any limitation on changing the Tx
> offloads.
> > > All of them were enabled, it just needed to choose with which one to use.
> > >
> > > Now, with the new offloads API, the Tx offloads are disabled by
> > > default, and enabled per application need on device and queue
> configuration.
> > > In order to call device/queue configuration the port must be
> > > stopped. This is requirement from ethdev API.
> > >
> > > Now, there is an API in ethdev to configure Rx VLAN offloads on the
> > > flight for supported devices.
> > > If there are devices which supports it also in the Tx side, and you
> > > see a good use case for that, then need to think of new API in
> > > ethdev to configure Tx VLAN offloads on the flight.
> > >
> > > However, the plain conversion of testpmd from the old to the new API
> > > requires the port to be stopped for all of the Tx offloads.
> > Let's just talk about TX vlan insertion. Actually there's no ethdev
> > API for it. And no configuration for it. Because vlan insertion is simple, it
> only need to add a vlan tag when transmitting packets.
> 
> You still need an appropriate TX function be selected, otherwise mbuf-
> >vlan_tci will be just ignored.
> So you still has to specify at queue_setup flag that you would like to have
> VLAN insertion offload enabled.
> Konstantin
O, it's a good point. I didn't consider the TX path selection. 

Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>

> 
> > You can see it's per packet thing. APP can set the ol_flags and
> > vlan_tci in the mbuf of every packet to enable this function for the packet
> or not. I think it does be a on the fly function on any NIC.
  

Patch

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index d8c73a9b1..58125839a 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -3439,7 +3439,14 @@  cmd_tx_vlan_set_parsed(void *parsed_result,
 {
 	struct cmd_tx_vlan_set_result *res = parsed_result;
 
+	if (!port_is_stopped(res->port_id)) {
+		printf("Please stop port %d first\n", res->port_id);
+		return;
+	}
+
 	tx_vlan_set(res->port_id, res->vlan_id);
+
+	cmd_reconfig_device_queue(res->port_id, 1, 1);
 }
 
 cmdline_parse_token_string_t cmd_tx_vlan_set_tx_vlan =
@@ -3486,7 +3493,14 @@  cmd_tx_vlan_set_qinq_parsed(void *parsed_result,
 {
 	struct cmd_tx_vlan_set_qinq_result *res = parsed_result;
 
+	if (!port_is_stopped(res->port_id)) {
+		printf("Please stop port %d first\n", res->port_id);
+		return;
+	}
+
 	tx_qinq_set(res->port_id, res->vlan_id, res->vlan_id_outer);
+
+	cmd_reconfig_device_queue(res->port_id, 1, 1);
 }
 
 cmdline_parse_token_string_t cmd_tx_vlan_set_qinq_tx_vlan =
@@ -3592,7 +3606,14 @@  cmd_tx_vlan_reset_parsed(void *parsed_result,
 {
 	struct cmd_tx_vlan_reset_result *res = parsed_result;
 
+	if (!port_is_stopped(res->port_id)) {
+		printf("Please stop port %d first\n", res->port_id);
+		return;
+	}
+
 	tx_vlan_reset(res->port_id);
+
+	cmd_reconfig_device_queue(res->port_id, 1, 1);
 }
 
 cmdline_parse_token_string_t cmd_tx_vlan_reset_tx_vlan =
@@ -3685,11 +3706,16 @@  cmd_csum_parsed(void *parsed_result,
 	struct cmd_csum_result *res = parsed_result;
 	int hw = 0;
 	uint16_t mask = 0;
+	uint64_t csum_offloads = 0;
 
 	if (port_id_is_invalid(res->port_id, ENABLED_WARN)) {
 		printf("invalid port %d\n", res->port_id);
 		return;
 	}
+	if (!port_is_stopped(res->port_id)) {
+		printf("Please stop port %d first\n", res->port_id);
+		return;
+	}
 
 	if (!strcmp(res->mode, "set")) {
 
@@ -3698,22 +3724,34 @@  cmd_csum_parsed(void *parsed_result,
 
 		if (!strcmp(res->proto, "ip")) {
 			mask = TESTPMD_TX_OFFLOAD_IP_CKSUM;
+			csum_offloads |= DEV_TX_OFFLOAD_IPV4_CKSUM;
 		} else if (!strcmp(res->proto, "udp")) {
 			mask = TESTPMD_TX_OFFLOAD_UDP_CKSUM;
+			csum_offloads |= DEV_TX_OFFLOAD_UDP_CKSUM;
 		} else if (!strcmp(res->proto, "tcp")) {
 			mask = TESTPMD_TX_OFFLOAD_TCP_CKSUM;
+			csum_offloads |= DEV_TX_OFFLOAD_TCP_CKSUM;
 		} else if (!strcmp(res->proto, "sctp")) {
 			mask = TESTPMD_TX_OFFLOAD_SCTP_CKSUM;
+			csum_offloads |= DEV_TX_OFFLOAD_SCTP_CKSUM;
 		} else if (!strcmp(res->proto, "outer-ip")) {
 			mask = TESTPMD_TX_OFFLOAD_OUTER_IP_CKSUM;
+			csum_offloads |= DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM;
 		}
 
-		if (hw)
+		if (hw) {
 			ports[res->port_id].tx_ol_flags |= mask;
-		else
+			ports[res->port_id].dev_conf.txmode.offloads |=
+							csum_offloads;
+		} else {
 			ports[res->port_id].tx_ol_flags &= (~mask);
+			ports[res->port_id].dev_conf.txmode.offloads &=
+							(~csum_offloads);
+		}
 	}
 	csum_show(res->port_id);
+
+	cmd_reconfig_device_queue(res->port_id, 1, 1);
 }
 
 cmdline_parse_token_string_t cmd_csum_csum =
@@ -3837,15 +3875,24 @@  cmd_tso_set_parsed(void *parsed_result,
 
 	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
 		return;
+	if (!port_is_stopped(res->port_id)) {
+		printf("Please stop port %d first\n", res->port_id);
+		return;
+	}
 
 	if (!strcmp(res->mode, "set"))
 		ports[res->port_id].tso_segsz = res->tso_segsz;
 
-	if (ports[res->port_id].tso_segsz == 0)
+	if (ports[res->port_id].tso_segsz == 0) {
+		ports[res->port_id].dev_conf.txmode.offloads &=
+						~DEV_TX_OFFLOAD_TCP_TSO;
 		printf("TSO for non-tunneled packets is disabled\n");
-	else
+	} else {
+		ports[res->port_id].dev_conf.txmode.offloads |=
+						DEV_TX_OFFLOAD_TCP_TSO;
 		printf("TSO segment size for non-tunneled packets is %d\n",
 			ports[res->port_id].tso_segsz);
+	}
 
 	/* display warnings if configuration is not supported by the NIC */
 	rte_eth_dev_info_get(res->port_id, &dev_info);
@@ -3854,6 +3901,8 @@  cmd_tso_set_parsed(void *parsed_result,
 		printf("Warning: TSO enabled but not "
 			"supported by port %d\n", res->port_id);
 	}
+
+	cmd_reconfig_device_queue(res->port_id, 1, 1);
 }
 
 cmdline_parse_token_string_t cmd_tso_set_tso =
@@ -3939,13 +3988,27 @@  cmd_tunnel_tso_set_parsed(void *parsed_result,
 
 	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
 		return;
+	if (!port_is_stopped(res->port_id)) {
+		printf("Please stop port %d first\n", res->port_id);
+		return;
+	}
 
 	if (!strcmp(res->mode, "set"))
 		ports[res->port_id].tunnel_tso_segsz = res->tso_segsz;
 
-	if (ports[res->port_id].tunnel_tso_segsz == 0)
+	if (ports[res->port_id].tunnel_tso_segsz == 0) {
+		ports[res->port_id].dev_conf.txmode.offloads &=
+			~(DEV_TX_OFFLOAD_VXLAN_TNL_TSO |
+			  DEV_TX_OFFLOAD_GRE_TNL_TSO |
+			  DEV_TX_OFFLOAD_IPIP_TNL_TSO |
+			  DEV_TX_OFFLOAD_GENEVE_TNL_TSO);
 		printf("TSO for tunneled packets is disabled\n");
-	else {
+	} else {
+		ports[res->port_id].dev_conf.txmode.offloads |=
+			(DEV_TX_OFFLOAD_VXLAN_TNL_TSO |
+			 DEV_TX_OFFLOAD_GRE_TNL_TSO |
+			 DEV_TX_OFFLOAD_IPIP_TNL_TSO |
+			 DEV_TX_OFFLOAD_GENEVE_TNL_TSO);
 		printf("TSO segment size for tunneled packets is %d\n",
 			ports[res->port_id].tunnel_tso_segsz);
 
@@ -3971,6 +4034,8 @@  cmd_tunnel_tso_set_parsed(void *parsed_result,
 			printf("Warning: csum set outer-ip must be set to hw "
 				"if outer L3 is IPv4; not necessary for IPv6\n");
 	}
+
+	cmd_reconfig_device_queue(res->port_id, 1, 1);
 }
 
 cmdline_parse_token_string_t cmd_tunnel_tso_set_tso =
@@ -13012,8 +13077,13 @@  cmd_set_macsec_offload_on_parsed(
 
 	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
+	if (!port_is_stopped(port_id)) {
+		printf("Please stop port %d first\n", port_id);
+		return;
+	}
 
 	ports[port_id].tx_ol_flags |= TESTPMD_TX_OFFLOAD_MACSEC;
+	ports[port_id].dev_conf.txmode.offloads |= DEV_TX_OFFLOAD_MACSEC_INSERT;
 #ifdef RTE_LIBRTE_IXGBE_PMD
 	ret = rte_pmd_ixgbe_macsec_enable(port_id, en, rp);
 #endif
@@ -13022,6 +13092,7 @@  cmd_set_macsec_offload_on_parsed(
 
 	switch (ret) {
 	case 0:
+		cmd_reconfig_device_queue(port_id, 1, 1);
 		break;
 	case -ENODEV:
 		printf("invalid port_id %d\n", port_id);
@@ -13096,14 +13167,21 @@  cmd_set_macsec_offload_off_parsed(
 
 	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
+	if (!port_is_stopped(port_id)) {
+		printf("Please stop port %d first\n", port_id);
+		return;
+	}
 
 	ports[port_id].tx_ol_flags &= ~TESTPMD_TX_OFFLOAD_MACSEC;
+	ports[port_id].dev_conf.txmode.offloads &=
+					~DEV_TX_OFFLOAD_MACSEC_INSERT;
 #ifdef RTE_LIBRTE_IXGBE_PMD
 	ret = rte_pmd_ixgbe_macsec_disable(port_id);
 #endif
 
 	switch (ret) {
 	case 0:
+		cmd_reconfig_device_queue(port_id, 1, 1);
 		break;
 	case -ENODEV:
 		printf("invalid port_id %d\n", port_id);
diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index 3bc99be76..d0056fa69 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -616,8 +616,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_VLAN_INSERT) {
 		printf("VLAN insert:                   ");
-		if (ports[port_id].tx_ol_flags &
-		    TESTPMD_TX_OFFLOAD_INSERT_VLAN)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_VLAN_INSERT)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -634,8 +634,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_QINQ_INSERT) {
 		printf("Double VLANs insert:           ");
-		if (ports[port_id].tx_ol_flags &
-		    TESTPMD_TX_OFFLOAD_INSERT_QINQ)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_QINQ_INSERT)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -643,7 +643,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_IPV4_CKSUM) {
 		printf("TX IPv4 checksum:              ");
-		if (ports[port_id].tx_ol_flags & TESTPMD_TX_OFFLOAD_IP_CKSUM)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_IPV4_CKSUM)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -651,7 +652,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_UDP_CKSUM) {
 		printf("TX UDP checksum:               ");
-		if (ports[port_id].tx_ol_flags & TESTPMD_TX_OFFLOAD_UDP_CKSUM)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_UDP_CKSUM)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -659,7 +661,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_TCP_CKSUM) {
 		printf("TX TCP checksum:               ");
-		if (ports[port_id].tx_ol_flags & TESTPMD_TX_OFFLOAD_TCP_CKSUM)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_TCP_CKSUM)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -667,7 +670,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_SCTP_CKSUM) {
 		printf("TX SCTP checksum:              ");
-		if (ports[port_id].tx_ol_flags & TESTPMD_TX_OFFLOAD_SCTP_CKSUM)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_SCTP_CKSUM)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -675,8 +679,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM) {
 		printf("TX Outer IPv4 checksum:        ");
-		if (ports[port_id].tx_ol_flags &
-		    TESTPMD_TX_OFFLOAD_OUTER_IP_CKSUM)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -684,7 +688,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_TCP_TSO) {
 		printf("TX TCP segmentation:           ");
-		if (ports[port_id].tso_segsz != 0)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_TCP_TSO)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -692,7 +697,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_UDP_TSO) {
 		printf("TX UDP segmentation:           ");
-		if (ports[port_id].tso_segsz != 0)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_UDP_TSO)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -700,7 +706,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_VXLAN_TNL_TSO) {
 		printf("TSO for VXLAN tunnel packet:   ");
-		if (ports[port_id].tunnel_tso_segsz)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_VXLAN_TNL_TSO)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -708,7 +715,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_GRE_TNL_TSO) {
 		printf("TSO for GRE tunnel packet:     ");
-		if (ports[port_id].tunnel_tso_segsz)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_GRE_TNL_TSO)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -716,7 +724,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_IPIP_TNL_TSO) {
 		printf("TSO for IPIP tunnel packet:    ");
-		if (ports[port_id].tunnel_tso_segsz)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_IPIP_TNL_TSO)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -724,7 +733,8 @@  port_offload_cap_display(portid_t port_id)
 
 	if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_GENEVE_TNL_TSO) {
 		printf("TSO for GENEVE tunnel packet:  ");
-		if (ports[port_id].tunnel_tso_segsz)
+		if (ports[port_id].dev_conf.txmode.offloads &
+		    DEV_TX_OFFLOAD_GENEVE_TNL_TSO)
 			printf("on\n");
 		else
 			printf("off\n");
@@ -1704,8 +1714,10 @@  rxtx_config_display(void)
 				tx_conf->tx_thresh.pthresh,
 				tx_conf->tx_thresh.hthresh,
 				tx_conf->tx_thresh.wthresh);
-		printf("  TX RS bit threshold=%d - TXQ flags=0x%"PRIx32"\n",
-				tx_conf->tx_rs_thresh, tx_conf->txq_flags);
+		printf("  TX RS bit threshold=%d - TXQ flags=0x%"PRIx32""
+		       " - TXQ offloads=0x%"PRIx64"\n",
+				tx_conf->tx_rs_thresh, tx_conf->txq_flags,
+				tx_conf->offloads);
 	}
 }
 
@@ -2783,6 +2795,7 @@  void
 tx_vlan_set(portid_t port_id, uint16_t vlan_id)
 {
 	int vlan_offload;
+
 	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
@@ -2796,6 +2809,7 @@  tx_vlan_set(portid_t port_id, uint16_t vlan_id)
 
 	tx_vlan_reset(port_id);
 	ports[port_id].tx_ol_flags |= TESTPMD_TX_OFFLOAD_INSERT_VLAN;
+	ports[port_id].dev_conf.txmode.offloads |= DEV_TX_OFFLOAD_VLAN_INSERT;
 	ports[port_id].tx_vlan_id = vlan_id;
 }
 
@@ -2803,6 +2817,7 @@  void
 tx_qinq_set(portid_t port_id, uint16_t vlan_id, uint16_t vlan_id_outer)
 {
 	int vlan_offload;
+
 	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
 	if (vlan_id_is_invalid(vlan_id))
@@ -2818,6 +2833,7 @@  tx_qinq_set(portid_t port_id, uint16_t vlan_id, uint16_t vlan_id_outer)
 
 	tx_vlan_reset(port_id);
 	ports[port_id].tx_ol_flags |= TESTPMD_TX_OFFLOAD_INSERT_QINQ;
+	ports[port_id].dev_conf.txmode.offloads |= DEV_TX_OFFLOAD_QINQ_INSERT;
 	ports[port_id].tx_vlan_id = vlan_id;
 	ports[port_id].tx_vlan_id_outer = vlan_id_outer;
 }
@@ -2829,6 +2845,9 @@  tx_vlan_reset(portid_t port_id)
 		return;
 	ports[port_id].tx_ol_flags &= ~(TESTPMD_TX_OFFLOAD_INSERT_VLAN |
 				TESTPMD_TX_OFFLOAD_INSERT_QINQ);
+	ports[port_id].dev_conf.txmode.offloads &=
+				~(DEV_TX_OFFLOAD_VLAN_INSERT |
+				  DEV_TX_OFFLOAD_QINQ_INSERT);
 	ports[port_id].tx_vlan_id = 0;
 	ports[port_id].tx_vlan_id_outer = 0;
 }
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 77154ef3b..0087438bc 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1498,6 +1498,9 @@  start_port(portid_t pid)
 		}
 		if (port->need_reconfig_queues > 0) {
 			port->need_reconfig_queues = 0;
+			port->tx_conf.txq_flags = ETH_TXQ_FLAGS_IGNORE;
+			/* Apply Tx offloads configuration */
+			port->tx_conf.offloads = port->dev_conf.txmode.offloads;
 			/* setup tx queues */
 			for (qi = 0; qi < nb_txq; qi++) {
 				if ((numa_support) &&