[v3,1/1] app/testpmd: control passing Rx metadata to PMD
Checks
Commit Message
Presently, Rx metadata is sent to PMD by default, leading
to a performance drop as processing for the same in rx path
takes extra cycles.
Hence, introducing command line argument, 'nic-to-pmd-rx-metadata'
to control passing rx metadata to PMD. By default it’s disabled.
Signed-off-by: Hanumanth Pothula <hpothula@marvell.com>
v3:
- Updated run_app.rst with the new command line argument,
nic-to-pmd-rx-metadata.
- Updated commit text.
v2:
- taken cared alignment issues
- renamed command line argument from rx-metadata to nic-to-pmd-rx-metadata
- renamed variable name from rx-metadata to nic_to_pmd_rx_metadata
---
app/test-pmd/parameters.c | 4 ++++
app/test-pmd/testpmd.c | 6 +++++-
app/test-pmd/testpmd.h | 2 ++
doc/guides/testpmd_app_ug/run_app.rst | 3 +++
4 files changed, 14 insertions(+), 1 deletion(-)
Comments
On 10/6/22 21:35, Hanumanth Pothula wrote:
> Presently, Rx metadata is sent to PMD by default, leading
> to a performance drop as processing for the same in rx path
> takes extra cycles.
>
> Hence, introducing command line argument, 'nic-to-pmd-rx-metadata'
> to control passing rx metadata to PMD. By default it’s disabled.
>
> Signed-off-by: Hanumanth Pothula <hpothula@marvell.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>
> v3:
> - Updated run_app.rst with the new command line argument,
> nic-to-pmd-rx-metadata.
> - Updated commit text.
> v2:
> - taken cared alignment issues
> - renamed command line argument from rx-metadata to nic-to-pmd-rx-metadata
> - renamed variable name from rx-metadata to nic_to_pmd_rx_metadata
Applied to dpdk-next-net/main, thanks.
17/10/2022 10:32, Andrew Rybchenko:
> On 10/6/22 21:35, Hanumanth Pothula wrote:
> > Presently, Rx metadata is sent to PMD by default, leading
> > to a performance drop as processing for the same in rx path
> > takes extra cycles.
> >
> > Hence, introducing command line argument, 'nic-to-pmd-rx-metadata'
> > to control passing rx metadata to PMD. By default it’s disabled.
> >
> > Signed-off-by: Hanumanth Pothula <hpothula@marvell.com>
>
> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>
> Applied to dpdk-next-net/main, thanks.
I'm not sure this patch is really acceptable.
It is disabling Rx metadata by default just for benchmarking reason
because your driver is doing some processing even if metadata is not required.
From a user perspective, if a command requesting metadata is entered,
it won't work until we enable this new option on startup.
It looks terrible.
Please tell me I misunderstood something.
27/10/2022 09:34, Thomas Monjalon:
> 17/10/2022 10:32, Andrew Rybchenko:
> > On 10/6/22 21:35, Hanumanth Pothula wrote:
> > > Presently, Rx metadata is sent to PMD by default, leading
> > > to a performance drop as processing for the same in rx path
> > > takes extra cycles.
> > >
> > > Hence, introducing command line argument, 'nic-to-pmd-rx-metadata'
> > > to control passing rx metadata to PMD. By default it’s disabled.
> > >
> > > Signed-off-by: Hanumanth Pothula <hpothula@marvell.com>
> >
> > Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> >
> > Applied to dpdk-next-net/main, thanks.
>
> I'm not sure this patch is really acceptable.
> It is disabling Rx metadata by default just for benchmarking reason
> because your driver is doing some processing even if metadata is not required.
>
> From a user perspective, if a command requesting metadata is entered,
> it won't work until we enable this new option on startup.
> It looks terrible.
>
> Please tell me I misunderstood something.
While pulling, I see that the name is not compliant with others.
I think it should start with "enable-", use hyphens and be sorted.
I'll drop it from the pull for now, we can have it in -rc3.
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Thursday, October 27, 2022 6:25 PM
> To: Hanumanth Reddy Pothula <hpothula@marvell.com>; Andrew
> Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Cc: dev@dpdk.org; Aman Singh <aman.deep.singh@intel.com>; Yuying
> Zhang <yuying.zhang@intel.com>; dev@dpdk.org; Jerin Jacob
> Kollanukkaran <jerinj@marvell.com>; Nithin Kumar Dabilpuram
> <ndabilpuram@marvell.com>; rasland@nvidia.com; orika@nvidia.com;
> viacheslavo@nvidia.com
> Subject: [EXT] Re: [PATCH v3 1/1] app/testpmd: control passing Rx
> metadata to PMD
>
> External Email
>
> ----------------------------------------------------------------------
> 27/10/2022 09:34, Thomas Monjalon:
> > 17/10/2022 10:32, Andrew Rybchenko:
> > > On 10/6/22 21:35, Hanumanth Pothula wrote:
> > > > Presently, Rx metadata is sent to PMD by default, leading to a
> > > > performance drop as processing for the same in rx path takes extra
> > > > cycles.
> > > >
> > > > Hence, introducing command line argument, 'nic-to-pmd-rx-metadata'
> > > > to control passing rx metadata to PMD. By default it’s disabled.
> > > >
> > > > Signed-off-by: Hanumanth Pothula <hpothula@marvell.com>
> > >
> > > Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> > >
> > > Applied to dpdk-next-net/main, thanks.
> >
> > I'm not sure this patch is really acceptable.
> > It is disabling Rx metadata by default just for benchmarking reason
> > because your driver is doing some processing even if metadata is not
> required.
> >
> > From a user perspective, if a command requesting metadata is entered,
> > it won't work until we enable this new option on startup.
> > It looks terrible.
> >
> > Please tell me I misunderstood something.
>
> While pulling, I see that the name is not compliant with others.
> I think it should start with "enable-", use hyphens and be sorted.
>
> I'll drop it from the pull for now, we can have it in -rc3.
>
@Thomas Monjalon I missed your comment, sorry for the delayed response.
Sending Rx metadata to PMD is added recently, which breaking our driver performance.
Normally any feature added to testpmd will be disabled by default, to make sure it won't affect other code(PMD).
Hence adding new testpmd command line argument to disable this feature by default.
Presently, below three Rx meta features are sent to PMD by default.
/** The NIC is able to deliver flag (if set) with packets to the PMD. */
#define RTE_ETH_RX_METADATA_USER_FLAG RTE_BIT64(0)
/** The NIC is able to deliver mark ID with packets to the PMD. */
#define RTE_ETH_RX_METADATA_USER_MARK RTE_BIT64(1)
/** The NIC is able to deliver tunnel ID with packets to the PMD. */
#define RTE_ETH_RX_METADATA_TUNNEL_ID RTE_BIT64(2)
02/12/2022 17:14, Hanumanth Reddy Pothula:
> -----Original Message-----
> > 27/10/2022 09:34, Thomas Monjalon:
> > > 17/10/2022 10:32, Andrew Rybchenko:
> > > > On 10/6/22 21:35, Hanumanth Pothula wrote:
> > > > > Presently, Rx metadata is sent to PMD by default, leading to a
> > > > > performance drop as processing for the same in rx path takes extra
> > > > > cycles.
> > > > >
> > > > > Hence, introducing command line argument, 'nic-to-pmd-rx-metadata'
> > > > > to control passing rx metadata to PMD. By default it’s disabled.
> > > > >
> > > > > Signed-off-by: Hanumanth Pothula <hpothula@marvell.com>
> > > >
> > > > Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> > > >
> > > > Applied to dpdk-next-net/main, thanks.
> > >
> > > I'm not sure this patch is really acceptable.
> > > It is disabling Rx metadata by default just for benchmarking reason
> > > because your driver is doing some processing even if metadata is not
> > required.
> > >
> > > From a user perspective, if a command requesting metadata is entered,
> > > it won't work until we enable this new option on startup.
> > > It looks terrible.
> > >
> > > Please tell me I misunderstood something.
> >
> > While pulling, I see that the name is not compliant with others.
> > I think it should start with "enable-", use hyphens and be sorted.
> >
> > I'll drop it from the pull for now, we can have it in -rc3.
> >
>
> @Thomas Monjalon I missed your comment, sorry for the delayed response.
>
> Sending Rx metadata to PMD is added recently, which breaking our driver performance.
> Normally any feature added to testpmd will be disabled by default, to make sure it won't affect other code(PMD).
> Hence adding new testpmd command line argument to disable this feature by default.
No, disabling by default doesn't mean you should enable with option.
It should be enabled if required by a command.
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Saturday, December 3, 2022 1:11 AM
> To: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>; Hanumanth
> Reddy Pothula <hpothula@marvell.com>
> Cc: dev@dpdk.org; Aman Singh <aman.deep.singh@intel.com>; Yuying
> Zhang <yuying.zhang@intel.com>; dev@dpdk.org; Jerin Jacob
> Kollanukkaran <jerinj@marvell.com>; Nithin Kumar Dabilpuram
> <ndabilpuram@marvell.com>; rasland@nvidia.com; orika@nvidia.com;
> viacheslavo@nvidia.com
> Subject: Re: [EXT] Re: [PATCH v3 1/1] app/testpmd: control passing Rx
> metadata to PMD
>
> 02/12/2022 17:14, Hanumanth Reddy Pothula:
> > -----Original Message-----
> > > 27/10/2022 09:34, Thomas Monjalon:
> > > > 17/10/2022 10:32, Andrew Rybchenko:
> > > > > On 10/6/22 21:35, Hanumanth Pothula wrote:
> > > > > > Presently, Rx metadata is sent to PMD by default, leading to a
> > > > > > performance drop as processing for the same in rx path takes
> > > > > > extra cycles.
> > > > > >
> > > > > > Hence, introducing command line argument, 'nic-to-pmd-rx-
> metadata'
> > > > > > to control passing rx metadata to PMD. By default it’s disabled.
> > > > > >
> > > > > > Signed-off-by: Hanumanth Pothula <hpothula@marvell.com>
> > > > >
> > > > > Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> > > > >
> > > > > Applied to dpdk-next-net/main, thanks.
> > > >
> > > > I'm not sure this patch is really acceptable.
> > > > It is disabling Rx metadata by default just for benchmarking
> > > > reason because your driver is doing some processing even if
> > > > metadata is not
> > > required.
> > > >
> > > > From a user perspective, if a command requesting metadata is
> > > > entered, it won't work until we enable this new option on startup.
> > > > It looks terrible.
> > > >
> > > > Please tell me I misunderstood something.
> > >
> > > While pulling, I see that the name is not compliant with others.
> > > I think it should start with "enable-", use hyphens and be sorted.
> > >
> > > I'll drop it from the pull for now, we can have it in -rc3.
> > >
> >
> > @Thomas Monjalon I missed your comment, sorry for the delayed
> response.
> >
> > Sending Rx metadata to PMD is added recently, which breaking our driver
> performance.
> > Normally any feature added to testpmd will be disabled by default, to
> make sure it won't affect other code(PMD).
> > Hence adding new testpmd command line argument to disable this
> feature by default.
>
> No, disabling by default doesn't mean you should enable with option.
> It should be enabled if required by a command.
>
Yeah, got it, user can enable/disable the feature on the fly without restarting the application(testpmd).
Will introduce new command, like below,
'port config rx-nic-to-pmd-metadata on/off'
Please let me know your thoughts on this command implementation.
If it's fine, will start implementing the command. Suggest if there is any other better way to implement the same.
05/12/2022 08:59, Hanumanth Reddy Pothula:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 02/12/2022 17:14, Hanumanth Reddy Pothula:
> > > -----Original Message-----
> > > > 27/10/2022 09:34, Thomas Monjalon:
> > > > > 17/10/2022 10:32, Andrew Rybchenko:
> > > > > > On 10/6/22 21:35, Hanumanth Pothula wrote:
> > > > > > > Presently, Rx metadata is sent to PMD by default, leading to a
> > > > > > > performance drop as processing for the same in rx path takes
> > > > > > > extra cycles.
> > > > > > >
> > > > > > > Hence, introducing command line argument, 'nic-to-pmd-rx-
> > metadata'
> > > > > > > to control passing rx metadata to PMD. By default it’s disabled.
> > > > > > >
> > > > > > > Signed-off-by: Hanumanth Pothula <hpothula@marvell.com>
> > > > > >
> > > > > > Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> > > > > >
> > > > > > Applied to dpdk-next-net/main, thanks.
> > > > >
> > > > > I'm not sure this patch is really acceptable.
> > > > > It is disabling Rx metadata by default just for benchmarking
> > > > > reason because your driver is doing some processing even if
> > > > > metadata is not
> > > > required.
> > > > >
> > > > > From a user perspective, if a command requesting metadata is
> > > > > entered, it won't work until we enable this new option on startup.
> > > > > It looks terrible.
> > > > >
> > > > > Please tell me I misunderstood something.
> > > >
> > > > While pulling, I see that the name is not compliant with others.
> > > > I think it should start with "enable-", use hyphens and be sorted.
> > > >
> > > > I'll drop it from the pull for now, we can have it in -rc3.
> > > >
> > >
> > > @Thomas Monjalon I missed your comment, sorry for the delayed
> > response.
> > >
> > > Sending Rx metadata to PMD is added recently, which breaking our driver
> > performance.
> > > Normally any feature added to testpmd will be disabled by default, to
> > make sure it won't affect other code(PMD).
> > > Hence adding new testpmd command line argument to disable this
> > feature by default.
> >
> > No, disabling by default doesn't mean you should enable with option.
> > It should be enabled if required by a command.
> >
> Yeah, got it, user can enable/disable the feature on the fly without restarting the application(testpmd).
> Will introduce new command, like below,
> 'port config rx-nic-to-pmd-metadata on/off'
>
> Please let me know your thoughts on this command implementation.
> If it's fine, will start implementing the command. Suggest if there is any other better way to implement the same.
What about enabling automatically when a flow command requests metadata?
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Monday, December 5, 2022 10:29
> To: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>; Hanumanth Reddy
> Pothula <hpothula@marvell.com>
> Cc: dev@dpdk.org; Aman Singh <aman.deep.singh@intel.com>; Yuying Zhang
> <yuying.zhang@intel.com>; dev@dpdk.org; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Nithin Kumar Dabilpuram <ndabilpuram@marvell.com>;
> Raslan Darawsheh <rasland@nvidia.com>; Ori Kam <orika@nvidia.com>; Slava
> Ovsiienko <viacheslavo@nvidia.com>
> Subject: Re: [EXT] Re: [PATCH v3 1/1] app/testpmd: control passing Rx
> metadata to PMD
>
> 05/12/2022 08:59, Hanumanth Reddy Pothula:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 02/12/2022 17:14, Hanumanth Reddy Pothula:
> > > > -----Original Message-----
> > > > > 27/10/2022 09:34, Thomas Monjalon:
> > > > > > 17/10/2022 10:32, Andrew Rybchenko:
> > > > > > > On 10/6/22 21:35, Hanumanth Pothula wrote:
> > > > > > > > Presently, Rx metadata is sent to PMD by default, leading
> > > > > > > > to a performance drop as processing for the same in rx
> > > > > > > > path takes extra cycles.
> > > > > > > >
> > > > > > > > Hence, introducing command line argument, 'nic-to-pmd-rx-
> > > metadata'
> > > > > > > > to control passing rx metadata to PMD. By default it’s
> disabled.
> > > > > > > >
> > > > > > > > Signed-off-by: Hanumanth Pothula <hpothula@marvell.com>
> > > > > > >
> > > > > > > Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> > > > > > >
> > > > > > > Applied to dpdk-next-net/main, thanks.
> > > > > >
> > > > > > I'm not sure this patch is really acceptable.
> > > > > > It is disabling Rx metadata by default just for benchmarking
> > > > > > reason because your driver is doing some processing even if
> > > > > > metadata is not
> > > > > required.
> > > > > >
> > > > > > From a user perspective, if a command requesting metadata is
> > > > > > entered, it won't work until we enable this new option on startup.
> > > > > > It looks terrible.
> > > > > >
> > > > > > Please tell me I misunderstood something.
> > > > >
> > > > > While pulling, I see that the name is not compliant with others.
> > > > > I think it should start with "enable-", use hyphens and be sorted.
> > > > >
> > > > > I'll drop it from the pull for now, we can have it in -rc3.
> > > > >
> > > >
> > > > @Thomas Monjalon I missed your comment, sorry for the delayed
> > > response.
> > > >
> > > > Sending Rx metadata to PMD is added recently, which breaking our
> > > > driver
> > > performance.
> > > > Normally any feature added to testpmd will be disabled by default,
> > > > to
> > > make sure it won't affect other code(PMD).
> > > > Hence adding new testpmd command line argument to disable this
> > > feature by default.
> > >
> > > No, disabling by default doesn't mean you should enable with option.
> > > It should be enabled if required by a command.
> > >
> > Yeah, got it, user can enable/disable the feature on the fly without
> restarting the application(testpmd).
> > Will introduce new command, like below,
> > 'port config rx-nic-to-pmd-metadata on/off'
> >
> > Please let me know your thoughts on this command implementation.
> > If it's fine, will start implementing the command. Suggest if there is any
> other better way to implement the same.
>
> What about enabling automatically when a flow command requests metadata?
>
Another option - enable on defining metadata dynamic field/flag in mbuf.
mlx5 PMD, for example follows this approach.
Do we need disabling this in real application? If application is designed to use
metadata it is unlikely it stops using these ones.
With best regards,
Slava
@@ -202,6 +202,7 @@ usage(char* progname)
printf(" --hairpin-mode=0xXX: bitmask set the hairpin port mode.\n"
" 0x10 - explicit Tx rule, 0x02 - hairpin ports paired\n"
" 0x01 - hairpin ports loop, 0x00 - hairpin port self\n");
+ printf(" --nic-to-pmd-rx-metadata: let the NIC deliver per-packet Rx metadata to PMD\n");
}
#ifdef RTE_LIB_CMDLINE
@@ -695,6 +696,7 @@ launch_args_parse(int argc, char** argv)
{ "record-burst-stats", 0, 0, 0 },
{ PARAM_NUM_PROCS, 1, 0, 0 },
{ PARAM_PROC_ID, 1, 0, 0 },
+ { "nic-to-pmd-rx-metadata", 0, 0, 0 },
{ 0, 0, 0, 0 },
};
@@ -1434,6 +1436,8 @@ launch_args_parse(int argc, char** argv)
num_procs = atoi(optarg);
if (!strcmp(lgopts[opt_idx].name, PARAM_PROC_ID))
proc_id = atoi(optarg);
+ if (!strcmp(lgopts[opt_idx].name, "nic-to-pmd-rx-metadata"))
+ nic_to_pmd_rx_metadata = 1;
break;
case 'h':
usage(argv[0]);
@@ -411,6 +411,9 @@ uint8_t clear_ptypes = true;
/* Hairpin ports configuration mode. */
uint16_t hairpin_mode;
+/* Send Rx metadata */
+uint8_t nic_to_pmd_rx_metadata;
+
/* Pretty printing of ethdev events */
static const char * const eth_event_desc[] = {
[RTE_ETH_EVENT_UNKNOWN] = "unknown",
@@ -1595,7 +1598,8 @@ init_config_port_offloads(portid_t pid, uint32_t socket_id)
int ret;
int i;
- eth_rx_metadata_negotiate_mp(pid);
+ if (nic_to_pmd_rx_metadata)
+ eth_rx_metadata_negotiate_mp(pid);
port->dev_conf.txmode = tx_mode;
port->dev_conf.rxmode = rx_mode;
@@ -615,6 +615,8 @@ extern struct rte_ether_addr peer_eth_addrs[RTE_MAX_ETHPORTS];
extern uint32_t burst_tx_delay_time; /**< Burst tx delay time(us) for mac-retry. */
extern uint32_t burst_tx_retry_num; /**< Burst tx retry number for mac-retry. */
+extern uint8_t nic_to_pmd_rx_metadata;
+
#ifdef RTE_LIB_GRO
#define GRO_DEFAULT_ITEM_NUM_PER_FLOW 32
#define GRO_DEFAULT_FLOW_NUM (RTE_GRO_MAX_BURST_ITEM_NUM / \
@@ -539,6 +539,9 @@ The command line options are:
The default value is 0. Hairpin will use single port mode and implicit Tx flow mode.
+* ``--nic_to_pmd_rx_metadata``
+
+ Enable passing Rx metadata to PMD.
Testpmd Multi-Process Command-line Options
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~