[v6,2/2] app/testpmd: add send scheduling test capability
Checks
Commit Message
This commit adds testpmd capability to provide timestamps on the packets
being sent in the txonly mode. This includes:
- SEND_ON_TIMESTAMP support
new device Tx offload capability support added, example:
testpmd> port config 0 tx_offload send_on_timestamp on
- set txtimes, registers field and flag, example:
testpmd> set txtimes 1000000,0
This command enables the packet send scheduling on timestamps if
the first parameter is not zero, generic format:
testpmd> set txtimes (inter),(intra)
where:
inter - is the delay between the bursts in the device clock units.
intra - is the delay between the packets within the burst specified
in the device clock units
As the result the bursts of packet will be transmitted with
specific delay between the packets within the burst and specific
delay between the bursts. The rte_eth_get_clock() is supposed to be
engaged to get the current device clock value and provide
the reference for the timestamps.
- show txtimes, displays the timing settings
- txonly burst time pattern
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
---
app/test-pmd/cmdline.c | 59 ++++++++++++++++++++--
app/test-pmd/config.c | 43 ++++++++++++++++
app/test-pmd/testpmd.c | 3 ++
app/test-pmd/testpmd.h | 3 ++
app/test-pmd/txonly.c | 77 +++++++++++++++++++++++++++--
doc/guides/testpmd_app_ug/testpmd_funcs.rst | 36 +++++++++++++-
6 files changed, 212 insertions(+), 9 deletions(-)
Comments
On 7/9/2020 1:36 PM, Viacheslav Ovsiienko wrote:
> This commit adds testpmd capability to provide timestamps on the packets
> being sent in the txonly mode. This includes:
>
> - SEND_ON_TIMESTAMP support
> new device Tx offload capability support added, example:
>
> testpmd> port config 0 tx_offload send_on_timestamp on
>
> - set txtimes, registers field and flag, example:
>
> testpmd> set txtimes 1000000,0
>
> This command enables the packet send scheduling on timestamps if
> the first parameter is not zero, generic format:
>
> testpmd> set txtimes (inter),(intra)
>
> where:
>
> inter - is the delay between the bursts in the device clock units.
> intra - is the delay between the packets within the burst specified
> in the device clock units
>
> As the result the bursts of packet will be transmitted with
> specific delay between the packets within the burst and specific
> delay between the bursts. The rte_eth_get_clock() is supposed to be
> engaged to get the current device clock value and provide
> the reference for the timestamps.
>
> - show txtimes, displays the timing settings
> - txonly burst time pattern
>
> Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
<...>
> +cmdline_parse_inst_t cmd_set_txtimes = {
> + .f = cmd_set_txtimes_parsed,
> + .data = NULL,
> + .help_str = "set txtimes <inter_burst>,<intra_burst>",
> + .tokens = {
> + (void *)&cmd_set_txtimes_keyword,
> + (void *)&cmd_set_txtimes_name,
> + (void *)&cmd_set_txtimes_value,
> + NULL,
> + },
> +};
Can you please update 'cmd_help_long_parsed()' with command updates?
<...>
> void
> +show_tx_pkt_times(void)
> +{
> + printf("Interburst gap: %u\n", tx_pkt_times[0]);
> + printf("Intraburst gap: %u\n", tx_pkt_times[1]);
> +}
> +
> +void
> +set_tx_pkt_times(unsigned int *tx_times)
> +{
> + int offset;
> + int flag;
> +
> + static const struct rte_mbuf_dynfield desc_offs = {
> + .name = RTE_MBUF_DYNFIELD_TIMESTAMP_NAME,
> + .size = sizeof(uint64_t),
> + .align = __alignof__(uint64_t),
> + };
> + static const struct rte_mbuf_dynflag desc_flag = {
> + .name = RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME,
> + };
> +
> + offset = rte_mbuf_dynfield_register(&desc_offs);
> + if (offset < 0 && rte_errno != EEXIST)
> + printf("Dynamic timestamp field registration error: %d",
> + rte_errno);
> + flag = rte_mbuf_dynflag_register(&desc_flag);
> + if (flag < 0 && rte_errno != EEXIST)
> + printf("Dynamic timestamp flag registration error: %d",
> + rte_errno);
You are not checking at all if the device supports
'DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP' offload or if it is configured or not, but
blindly registering the dynamic fields.
'DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP' seems not really used, as mentioned in prev
patch I would be OK to drop the flag.
> + tx_pkt_times[0] = tx_times[0];
> + tx_pkt_times[1] = tx_times[1];
I think it is better to rename 'tx_pkt_times[0]' -> 'tx_pkt_times_inter',
'tx_pkt_times[1]' --> 'tx_pkt_times_intra' to increase code readability.
<...>
> --- a/app/test-pmd/txonly.c
> +++ b/app/test-pmd/txonly.c
> @@ -53,6 +53,11 @@
> static struct rte_ipv4_hdr pkt_ip_hdr; /**< IP header of transmitted packets. */
> RTE_DEFINE_PER_LCORE(uint8_t, _ip_var); /**< IP address variation */
> static struct rte_udp_hdr pkt_udp_hdr; /**< UDP header of tx packets. */
> +RTE_DEFINE_PER_LCORE(uint64_t, ts_qskew); /**< Timestamp offset per queue */
> +static uint64_t ts_mask; /**< Timestamp dynamic flag mask */
> +static int32_t ts_off; /**< Timestamp dynamic field offset */
> +static bool ts_enable; /**< Timestamp enable */
> +static uint64_t ts_init[RTE_MAX_ETHPORTS];
What do you think renaming the 'ts_' prefix to long 'timestamp_' prefix, will
variable names be too long? When you are out of this patch context and reading
code 'ts_init' is not that expressive.
<...>
> @@ -213,6 +219,50 @@
> copy_buf_to_pkt(&pkt_udp_hdr, sizeof(pkt_udp_hdr), pkt,
> sizeof(struct rte_ether_hdr) +
> sizeof(struct rte_ipv4_hdr));
> + if (unlikely(ts_enable)) {
> + uint64_t skew = RTE_PER_LCORE(ts_qskew);
> + struct {
> + rte_be32_t signature;
> + rte_be16_t pkt_idx;
> + rte_be16_t queue_idx;
> + rte_be64_t ts;
> + } ts_mark;
> +
> + if (unlikely(!skew)) {
> + struct rte_eth_dev *dev = &rte_eth_devices[fs->tx_port];
> + unsigned int txqs_n = dev->data->nb_tx_queues;
> + uint64_t phase = tx_pkt_times[0] * fs->tx_queue /
> + (txqs_n ? txqs_n : 1);
> + /*
> + * Initialize the scheduling time phase shift
> + * depending on queue index.
> + */
> + skew = ts_init[fs->tx_port] + tx_pkt_times[0] + phase;
> + RTE_PER_LCORE(ts_qskew) = skew;
> + }
> + ts_mark.pkt_idx = rte_cpu_to_be_16(idx);
> + ts_mark.queue_idx = rte_cpu_to_be_16(fs->tx_queue);
> + ts_mark.signature = rte_cpu_to_be_32(0xBEEFC0DE);
> + if (unlikely(!idx)) {
> + skew += tx_pkt_times[0];
> + pkt->ol_flags |= ts_mask;
> + *RTE_MBUF_DYNFIELD(pkt, ts_off, uint64_t *) = skew;
> + RTE_PER_LCORE(ts_qskew) = skew;
> + ts_mark.ts = rte_cpu_to_be_64(skew);
> + } else if (tx_pkt_times[1]) {
> + skew += tx_pkt_times[1];
> + pkt->ol_flags |= ts_mask;
> + *RTE_MBUF_DYNFIELD(pkt, ts_off, uint64_t *) = skew;
> + RTE_PER_LCORE(ts_qskew) = skew;
> + ts_mark.ts = rte_cpu_to_be_64(skew);
> + } else {
> + ts_mark.ts = RTE_BE64(0);
> + }
> + copy_buf_to_pkt(&ts_mark, sizeof(ts_mark), pkt,
> + sizeof(struct rte_ether_hdr) +
> + sizeof(struct rte_ipv4_hdr) +
> + sizeof(pkt_udp_hdr));
timestamp data seems put into packet payload, I assume this is for debug, is
there any intendent target app for the packets?
<...>
> static void
> -tx_only_begin(__rte_unused portid_t pi)
> +tx_only_begin(portid_t pi)
> {
> uint16_t pkt_data_len;
> + int dynf;
>
> pkt_data_len = (uint16_t) (tx_pkt_length - (
> sizeof(struct rte_ether_hdr) +
> sizeof(struct rte_ipv4_hdr) +
> sizeof(struct rte_udp_hdr)));
> setup_pkt_udp_ip_headers(&pkt_ip_hdr, &pkt_udp_hdr, pkt_data_len);
> +
> + ts_enable = false;
> + ts_mask = 0;
> + ts_off = -1;
> + RTE_PER_LCORE(ts_qskew) = 0;
> + dynf = rte_mbuf_dynflag_lookup
> + (RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME, NULL);
> + if (dynf >= 0)
> + ts_mask = 1ULL << dynf;
> + dynf = rte_mbuf_dynfield_lookup
> + (RTE_MBUF_DYNFIELD_TIMESTAMP_NAME, NULL);
> + if (dynf >= 0)
> + ts_off = dynf;
> + ts_enable = tx_pkt_times[0] && ts_mask && ts_off >= 0 &&
> + !rte_eth_read_clock(pi, &ts_init[pi]);
'rte_eth_read_clock()' support is a 'must' to have this feature. Can you please
clarify this in the document/commit_log?
<...>
> }
>
> struct fwd_engine tx_only_engine = {
> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> index a808b6a..00413cc 100644
> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> @@ -266,7 +266,7 @@ show config
> Displays the configuration of the application.
> The configuration comes from the command-line, the runtime or the application defaults::
>
> - testpmd> show config (rxtx|cores|fwd|txpkts)
> + testpmd> show config (rxtx|cores|fwd|txpkts|txtimes)
>
> The available information categories are:
>
> @@ -278,6 +278,8 @@ The available information categories are:
>
> * ``txpkts``: Packets to TX configuration.
>
> +* ``txtimes``: Burst time pattern for Tx only mode.
The description is not clear, can you pleaes give a little more detail?
> +
> For example:
>
> .. code-block:: console
> @@ -725,6 +727,38 @@ Set the length of each segment of the TX-ONLY packets or length of packet for FL
>
> Where x[,y]* represents a CSV list of values, without white space.
>
> +set txtimes
> +~~~~~~~~~~
WARNING: Title underline too short.
> +
> +Configure the timing burst pattern for Tx only mode. This command enables
> +the packet send scheduling on dynamic timestamp mbuf field and configures
> +timing pattern in Tx only mode. In this mode, if scheduling is enabled
> +application provides timestamps in the packets being sent. It is possible
> +to configure delay (in unspecified device clock units) between bursts
> +and between the packets within the burst::
> +
> + testpmd> set txtimes (inter),(intra)
> +
> +where:
> +
> +* ``inter`` is the delay between the bursts in the device clock units.
> + If ``intra`` is zero, this is the time between the beginnings of the
> + first packets in the neighbour bursts, if ``intra`` is not zero,
> + ``inter`` specifies the time between the beginning of the first packet
> + of the current burst and the beginning of the last packet of the
> + previous burst. If ``inter`` parameter is zero the send scheduling
> + on timestamps is disabled (default).
> +
> +* ``intra`` is the delay between the packets within the burst specified
> + in the device clock units. The number of packets in the burst is defined
> + by regular burst setting. If ``intra`` parameter is zero no timestamps
> + provided in the packets excepting the first one in the burst.
> +
> +As the result the bursts of packet will be transmitted with specific
> +delays between the packets within the burst and specific delay between
> +the bursts. The rte_eth_get_clock() is supposed to be engaged to get the
'rte_eth_read_clock()'?
Hi Ferruh,
Thanks a lot for the comments, addressed all of them.
With best regards, Slava
> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Friday, July 10, 2020 2:58
> To: Slava Ovsiienko <viacheslavo@mellanox.com>; dev@dpdk.org
> Cc: Matan Azrad <matan@mellanox.com>; Raslan Darawsheh
> <rasland@mellanox.com>; olivier.matz@6wind.com;
> bernard.iremonger@intel.com; thomas@monjalon.com
> Subject: Re: [dpdk-dev] [PATCH v6 2/2] app/testpmd: add send scheduling
> test capability
>
> On 7/9/2020 1:36 PM, Viacheslav Ovsiienko wrote:
> > This commit adds testpmd capability to provide timestamps on the
> > packets being sent in the txonly mode. This includes:
> >
> > - SEND_ON_TIMESTAMP support
> > new device Tx offload capability support added, example:
> >
> > testpmd> port config 0 tx_offload send_on_timestamp on
> >
> > - set txtimes, registers field and flag, example:
> >
> > testpmd> set txtimes 1000000,0
> >
> > This command enables the packet send scheduling on timestamps if
> > the first parameter is not zero, generic format:
> >
> > testpmd> set txtimes (inter),(intra)
> >
> > where:
> >
> > inter - is the delay between the bursts in the device clock units.
> > intra - is the delay between the packets within the burst specified
> > in the device clock units
> >
> > As the result the bursts of packet will be transmitted with
> > specific delay between the packets within the burst and specific
> > delay between the bursts. The rte_eth_get_clock() is supposed to be
> > engaged to get the current device clock value and provide
> > the reference for the timestamps.
> >
> > - show txtimes, displays the timing settings
> > - txonly burst time pattern
> >
> > Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
>
> <...>
>
> > +cmdline_parse_inst_t cmd_set_txtimes = {
> > + .f = cmd_set_txtimes_parsed,
> > + .data = NULL,
> > + .help_str = "set txtimes <inter_burst>,<intra_burst>",
> > + .tokens = {
> > + (void *)&cmd_set_txtimes_keyword,
> > + (void *)&cmd_set_txtimes_name,
> > + (void *)&cmd_set_txtimes_value,
> > + NULL,
> > + },
> > +};
>
> Can you please update 'cmd_help_long_parsed()' with command updates?
>
> <...>
>
> > void
> > +show_tx_pkt_times(void)
> > +{
> > + printf("Interburst gap: %u\n", tx_pkt_times[0]);
> > + printf("Intraburst gap: %u\n", tx_pkt_times[1]); }
> > +
> > +void
> > +set_tx_pkt_times(unsigned int *tx_times) {
> > + int offset;
> > + int flag;
> > +
> > + static const struct rte_mbuf_dynfield desc_offs = {
> > + .name = RTE_MBUF_DYNFIELD_TIMESTAMP_NAME,
> > + .size = sizeof(uint64_t),
> > + .align = __alignof__(uint64_t),
> > + };
> > + static const struct rte_mbuf_dynflag desc_flag = {
> > + .name = RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME,
> > + };
> > +
> > + offset = rte_mbuf_dynfield_register(&desc_offs);
> > + if (offset < 0 && rte_errno != EEXIST)
> > + printf("Dynamic timestamp field registration error: %d",
> > + rte_errno);
> > + flag = rte_mbuf_dynflag_register(&desc_flag);
> > + if (flag < 0 && rte_errno != EEXIST)
> > + printf("Dynamic timestamp flag registration error: %d",
> > + rte_errno);
>
> You are not checking at all if the device supports
> 'DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP' offload or if it is configured or
> not, but blindly registering the dynamic fields.
> 'DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP' seems not really used, as
> mentioned in prev patch I would be OK to drop the flag.
>
> > + tx_pkt_times[0] = tx_times[0];
> > + tx_pkt_times[1] = tx_times[1];
>
> I think it is better to rename 'tx_pkt_times[0]' -> 'tx_pkt_times_inter',
> 'tx_pkt_times[1]' --> 'tx_pkt_times_intra' to increase code readability.
>
> <...>
>
> > --- a/app/test-pmd/txonly.c
> > +++ b/app/test-pmd/txonly.c
> > @@ -53,6 +53,11 @@
> > static struct rte_ipv4_hdr pkt_ip_hdr; /**< IP header of transmitted
> > packets. */ RTE_DEFINE_PER_LCORE(uint8_t, _ip_var); /**< IP address
> > variation */ static struct rte_udp_hdr pkt_udp_hdr; /**< UDP header
> > of tx packets. */
> > +RTE_DEFINE_PER_LCORE(uint64_t, ts_qskew); /**< Timestamp offset per
> > +queue */ static uint64_t ts_mask; /**< Timestamp dynamic flag mask */
> > +static int32_t ts_off; /**< Timestamp dynamic field offset */ static
> > +bool ts_enable; /**< Timestamp enable */ static uint64_t
> > +ts_init[RTE_MAX_ETHPORTS];
>
> What do you think renaming the 'ts_' prefix to long 'timestamp_' prefix, will
> variable names be too long? When you are out of this patch context and
> reading code 'ts_init' is not that expressive.
>
> <...>
>
> > @@ -213,6 +219,50 @@
> > copy_buf_to_pkt(&pkt_udp_hdr, sizeof(pkt_udp_hdr), pkt,
> > sizeof(struct rte_ether_hdr) +
> > sizeof(struct rte_ipv4_hdr));
> > + if (unlikely(ts_enable)) {
> > + uint64_t skew = RTE_PER_LCORE(ts_qskew);
> > + struct {
> > + rte_be32_t signature;
> > + rte_be16_t pkt_idx;
> > + rte_be16_t queue_idx;
> > + rte_be64_t ts;
> > + } ts_mark;
> > +
> > + if (unlikely(!skew)) {
> > + struct rte_eth_dev *dev = &rte_eth_devices[fs-
> >tx_port];
> > + unsigned int txqs_n = dev->data->nb_tx_queues;
> > + uint64_t phase = tx_pkt_times[0] * fs->tx_queue /
> > + (txqs_n ? txqs_n : 1);
> > + /*
> > + * Initialize the scheduling time phase shift
> > + * depending on queue index.
> > + */
> > + skew = ts_init[fs->tx_port] + tx_pkt_times[0] +
> phase;
> > + RTE_PER_LCORE(ts_qskew) = skew;
> > + }
> > + ts_mark.pkt_idx = rte_cpu_to_be_16(idx);
> > + ts_mark.queue_idx = rte_cpu_to_be_16(fs->tx_queue);
> > + ts_mark.signature = rte_cpu_to_be_32(0xBEEFC0DE);
> > + if (unlikely(!idx)) {
> > + skew += tx_pkt_times[0];
> > + pkt->ol_flags |= ts_mask;
> > + *RTE_MBUF_DYNFIELD(pkt, ts_off, uint64_t *) =
> skew;
> > + RTE_PER_LCORE(ts_qskew) = skew;
> > + ts_mark.ts = rte_cpu_to_be_64(skew);
> > + } else if (tx_pkt_times[1]) {
> > + skew += tx_pkt_times[1];
> > + pkt->ol_flags |= ts_mask;
> > + *RTE_MBUF_DYNFIELD(pkt, ts_off, uint64_t *) =
> skew;
> > + RTE_PER_LCORE(ts_qskew) = skew;
> > + ts_mark.ts = rte_cpu_to_be_64(skew);
> > + } else {
> > + ts_mark.ts = RTE_BE64(0);
> > + }
> > + copy_buf_to_pkt(&ts_mark, sizeof(ts_mark), pkt,
> > + sizeof(struct rte_ether_hdr) +
> > + sizeof(struct rte_ipv4_hdr) +
> > + sizeof(pkt_udp_hdr));
>
> timestamp data seems put into packet payload, I assume this is for debug, is
> there any intendent target app for the packets?
>
> <...>
>
> > static void
> > -tx_only_begin(__rte_unused portid_t pi)
> > +tx_only_begin(portid_t pi)
> > {
> > uint16_t pkt_data_len;
> > + int dynf;
> >
> > pkt_data_len = (uint16_t) (tx_pkt_length - (
> > sizeof(struct rte_ether_hdr) +
> > sizeof(struct rte_ipv4_hdr) +
> > sizeof(struct rte_udp_hdr)));
> > setup_pkt_udp_ip_headers(&pkt_ip_hdr, &pkt_udp_hdr,
> pkt_data_len);
> > +
> > + ts_enable = false;
> > + ts_mask = 0;
> > + ts_off = -1;
> > + RTE_PER_LCORE(ts_qskew) = 0;
> > + dynf = rte_mbuf_dynflag_lookup
> > +
> (RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME, NULL);
> > + if (dynf >= 0)
> > + ts_mask = 1ULL << dynf;
> > + dynf = rte_mbuf_dynfield_lookup
> > + (RTE_MBUF_DYNFIELD_TIMESTAMP_NAME,
> NULL);
> > + if (dynf >= 0)
> > + ts_off = dynf;
> > + ts_enable = tx_pkt_times[0] && ts_mask && ts_off >= 0 &&
> > + !rte_eth_read_clock(pi, &ts_init[pi]);
>
> 'rte_eth_read_clock()' support is a 'must' to have this feature. Can you please
> clarify this in the document/commit_log?
>
> <...>
>
> > }
> >
> > struct fwd_engine tx_only_engine = {
> > diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> > b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> > index a808b6a..00413cc 100644
> > --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> > +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> > @@ -266,7 +266,7 @@ show config
> > Displays the configuration of the application.
> > The configuration comes from the command-line, the runtime or the
> application defaults::
> >
> > - testpmd> show config (rxtx|cores|fwd|txpkts)
> > + testpmd> show config (rxtx|cores|fwd|txpkts|txtimes)
> >
> > The available information categories are:
> >
> > @@ -278,6 +278,8 @@ The available information categories are:
> >
> > * ``txpkts``: Packets to TX configuration.
> >
> > +* ``txtimes``: Burst time pattern for Tx only mode.
>
> The description is not clear, can you pleaes give a little more detail?
>
> > +
> > For example:
> >
> > .. code-block:: console
> > @@ -725,6 +727,38 @@ Set the length of each segment of the TX-ONLY
> > packets or length of packet for FL
> >
> > Where x[,y]* represents a CSV list of values, without white space.
> >
> > +set txtimes
> > +~~~~~~~~~~
>
> WARNING: Title underline too short.
>
> > +
> > +Configure the timing burst pattern for Tx only mode. This command
> > +enables the packet send scheduling on dynamic timestamp mbuf field
> > +and configures timing pattern in Tx only mode. In this mode, if
> > +scheduling is enabled application provides timestamps in the packets
> > +being sent. It is possible to configure delay (in unspecified device
> > +clock units) between bursts and between the packets within the burst::
> > +
> > + testpmd> set txtimes (inter),(intra)
> > +
> > +where:
> > +
> > +* ``inter`` is the delay between the bursts in the device clock units.
> > + If ``intra`` is zero, this is the time between the beginnings of
> > +the
> > + first packets in the neighbour bursts, if ``intra`` is not zero,
> > + ``inter`` specifies the time between the beginning of the first
> > +packet
> > + of the current burst and the beginning of the last packet of the
> > + previous burst. If ``inter`` parameter is zero the send scheduling
> > + on timestamps is disabled (default).
> > +
> > +* ``intra`` is the delay between the packets within the burst
> > +specified
> > + in the device clock units. The number of packets in the burst is
> > +defined
> > + by regular burst setting. If ``intra`` parameter is zero no
> > +timestamps
> > + provided in the packets excepting the first one in the burst.
> > +
> > +As the result the bursts of packet will be transmitted with specific
> > +delays between the packets within the burst and specific delay
> > +between the bursts. The rte_eth_get_clock() is supposed to be engaged
> > +to get the
>
> 'rte_eth_read_clock()'?
@@ -3930,6 +3930,52 @@ struct cmd_set_txsplit_result {
},
};
+/* *** SET TIMES FOR TXONLY PACKETS SCHEDULING ON TIMESTAMPS *** */
+
+struct cmd_set_txtimes_result {
+ cmdline_fixed_string_t cmd_keyword;
+ cmdline_fixed_string_t txtimes;
+ cmdline_fixed_string_t tx_times;
+};
+
+static void
+cmd_set_txtimes_parsed(void *parsed_result,
+ __rte_unused struct cmdline *cl,
+ __rte_unused void *data)
+{
+ struct cmd_set_txtimes_result *res;
+ unsigned int tx_times[2] = {0, 0};
+ unsigned int n_times;
+
+ res = parsed_result;
+ n_times = parse_item_list(res->tx_times, "tx times",
+ 2, tx_times, 0);
+ if (n_times == 2)
+ set_tx_pkt_times(tx_times);
+}
+
+cmdline_parse_token_string_t cmd_set_txtimes_keyword =
+ TOKEN_STRING_INITIALIZER(struct cmd_set_txtimes_result,
+ cmd_keyword, "set");
+cmdline_parse_token_string_t cmd_set_txtimes_name =
+ TOKEN_STRING_INITIALIZER(struct cmd_set_txtimes_result,
+ txtimes, "txtimes");
+cmdline_parse_token_string_t cmd_set_txtimes_value =
+ TOKEN_STRING_INITIALIZER(struct cmd_set_txtimes_result,
+ tx_times, NULL);
+
+cmdline_parse_inst_t cmd_set_txtimes = {
+ .f = cmd_set_txtimes_parsed,
+ .data = NULL,
+ .help_str = "set txtimes <inter_burst>,<intra_burst>",
+ .tokens = {
+ (void *)&cmd_set_txtimes_keyword,
+ (void *)&cmd_set_txtimes_name,
+ (void *)&cmd_set_txtimes_value,
+ NULL,
+ },
+};
+
/* *** ADD/REMOVE ALL VLAN IDENTIFIERS TO/FROM A PORT VLAN RX FILTER *** */
struct cmd_rx_vlan_filter_all_result {
cmdline_fixed_string_t rx_vlan;
@@ -7418,6 +7464,8 @@ static void cmd_showcfg_parsed(void *parsed_result,
pkt_fwd_config_display(&cur_fwd_config);
else if (!strcmp(res->what, "txpkts"))
show_tx_pkt_segments();
+ else if (!strcmp(res->what, "txtimes"))
+ show_tx_pkt_times();
}
cmdline_parse_token_string_t cmd_showcfg_show =
@@ -7426,12 +7474,12 @@ static void cmd_showcfg_parsed(void *parsed_result,
TOKEN_STRING_INITIALIZER(struct cmd_showcfg_result, cfg, "config");
cmdline_parse_token_string_t cmd_showcfg_what =
TOKEN_STRING_INITIALIZER(struct cmd_showcfg_result, what,
- "rxtx#cores#fwd#txpkts");
+ "rxtx#cores#fwd#txpkts#txtimes");
cmdline_parse_inst_t cmd_showcfg = {
.f = cmd_showcfg_parsed,
.data = NULL,
- .help_str = "show config rxtx|cores|fwd|txpkts",
+ .help_str = "show config rxtx|cores|fwd|txpkts|txtimes",
.tokens = {
(void *)&cmd_showcfg_show,
(void *)&cmd_showcfg_port,
@@ -18670,7 +18718,8 @@ struct cmd_config_per_port_tx_offload_result {
"sctp_cksum#tcp_tso#udp_tso#outer_ipv4_cksum#"
"qinq_insert#vxlan_tnl_tso#gre_tnl_tso#"
"ipip_tnl_tso#geneve_tnl_tso#macsec_insert#"
- "mt_lockfree#multi_segs#mbuf_fast_free#security");
+ "mt_lockfree#multi_segs#mbuf_fast_free#security#"
+ "send_on_timestamp");
cmdline_parse_token_string_t cmd_config_per_port_tx_offload_result_on_off =
TOKEN_STRING_INITIALIZER
(struct cmd_config_per_port_tx_offload_result,
@@ -18755,7 +18804,8 @@ struct cmd_config_per_port_tx_offload_result {
"sctp_cksum|tcp_tso|udp_tso|outer_ipv4_cksum|"
"qinq_insert|vxlan_tnl_tso|gre_tnl_tso|"
"ipip_tnl_tso|geneve_tnl_tso|macsec_insert|"
- "mt_lockfree|multi_segs|mbuf_fast_free|security on|off",
+ "mt_lockfree|multi_segs|mbuf_fast_free|security|"
+ "send_on_timestamp on|off",
.tokens = {
(void *)&cmd_config_per_port_tx_offload_result_port,
(void *)&cmd_config_per_port_tx_offload_result_config,
@@ -19427,6 +19477,7 @@ struct cmd_showport_macs_result {
(cmdline_parse_inst_t *)&cmd_set_log,
(cmdline_parse_inst_t *)&cmd_set_txpkts,
(cmdline_parse_inst_t *)&cmd_set_txsplit,
+ (cmdline_parse_inst_t *)&cmd_set_txtimes,
(cmdline_parse_inst_t *)&cmd_set_fwd_list,
(cmdline_parse_inst_t *)&cmd_set_fwd_mask,
(cmdline_parse_inst_t *)&cmd_set_fwd_mode,
@@ -1046,6 +1046,15 @@ static int bus_match_all(const struct rte_bus *bus, const void *data)
printf("off\n");
}
+ if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP) {
+ printf("Tx scheduling on timestamp: ");
+ if (ports[port_id].dev_conf.txmode.offloads &
+ DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP)
+ printf("on\n");
+ else
+ printf("off\n");
+ }
+
}
int
@@ -3067,6 +3076,40 @@ struct igb_ring_desc_16_bytes {
}
void
+show_tx_pkt_times(void)
+{
+ printf("Interburst gap: %u\n", tx_pkt_times[0]);
+ printf("Intraburst gap: %u\n", tx_pkt_times[1]);
+}
+
+void
+set_tx_pkt_times(unsigned int *tx_times)
+{
+ int offset;
+ int flag;
+
+ static const struct rte_mbuf_dynfield desc_offs = {
+ .name = RTE_MBUF_DYNFIELD_TIMESTAMP_NAME,
+ .size = sizeof(uint64_t),
+ .align = __alignof__(uint64_t),
+ };
+ static const struct rte_mbuf_dynflag desc_flag = {
+ .name = RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME,
+ };
+
+ offset = rte_mbuf_dynfield_register(&desc_offs);
+ if (offset < 0 && rte_errno != EEXIST)
+ printf("Dynamic timestamp field registration error: %d",
+ rte_errno);
+ flag = rte_mbuf_dynflag_register(&desc_flag);
+ if (flag < 0 && rte_errno != EEXIST)
+ printf("Dynamic timestamp flag registration error: %d",
+ rte_errno);
+ tx_pkt_times[0] = tx_times[0];
+ tx_pkt_times[1] = tx_times[1];
+}
+
+void
setup_gro(const char *onoff, portid_t port_id)
{
if (!rte_eth_dev_is_valid_port(port_id)) {
@@ -223,6 +223,9 @@ struct fwd_engine * fwd_engines[] = {
uint8_t txonly_multi_flow;
/**< Whether multiple flows are generated in TXONLY mode. */
+uint32_t tx_pkt_times[2];
+/**< Timings for send scheduling in TXONLY mode. */
+
uint16_t nb_pkt_per_burst = DEF_PKT_BURST; /**< Number of packets per burst. */
uint16_t mb_mempool_cache = DEF_MBUF_CACHE; /**< Size of mbuf mempool cache. */
@@ -442,6 +442,7 @@ struct queue_stats_mappings {
extern uint16_t tx_pkt_length; /**< Length of TXONLY packet */
extern uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT]; /**< Seg. lengths */
extern uint8_t tx_pkt_nb_segs; /**< Number of segments in TX packets */
+extern uint32_t tx_pkt_times[2];
enum tx_pkt_split {
TX_PKT_SPLIT_OFF,
@@ -794,6 +795,8 @@ void vlan_tpid_set(portid_t port_id, enum rte_vlan_type vlan_type,
void set_verbose_level(uint16_t vb_level);
void set_tx_pkt_segments(unsigned *seg_lengths, unsigned nb_segs);
void show_tx_pkt_segments(void);
+void set_tx_pkt_times(unsigned int *tx_times);
+void show_tx_pkt_times(void);
void set_tx_pkt_split(const char *name);
void set_nb_pkt_per_burst(uint16_t pkt_burst);
char *list_pkt_forwarding_modes(void);
@@ -53,6 +53,11 @@
static struct rte_ipv4_hdr pkt_ip_hdr; /**< IP header of transmitted packets. */
RTE_DEFINE_PER_LCORE(uint8_t, _ip_var); /**< IP address variation */
static struct rte_udp_hdr pkt_udp_hdr; /**< UDP header of tx packets. */
+RTE_DEFINE_PER_LCORE(uint64_t, ts_qskew); /**< Timestamp offset per queue */
+static uint64_t ts_mask; /**< Timestamp dynamic flag mask */
+static int32_t ts_off; /**< Timestamp dynamic field offset */
+static bool ts_enable; /**< Timestamp enable */
+static uint64_t ts_init[RTE_MAX_ETHPORTS];
static void
copy_buf_to_pkt_segs(void* buf, unsigned len, struct rte_mbuf *pkt,
@@ -150,7 +155,8 @@
static inline bool
pkt_burst_prepare(struct rte_mbuf *pkt, struct rte_mempool *mbp,
struct rte_ether_hdr *eth_hdr, const uint16_t vlan_tci,
- const uint16_t vlan_tci_outer, const uint64_t ol_flags)
+ const uint16_t vlan_tci_outer, const uint64_t ol_flags,
+ const uint16_t idx, const struct fwd_stream *fs)
{
struct rte_mbuf *pkt_segs[RTE_MAX_SEGS_PER_PKT];
struct rte_mbuf *pkt_seg;
@@ -213,6 +219,50 @@
copy_buf_to_pkt(&pkt_udp_hdr, sizeof(pkt_udp_hdr), pkt,
sizeof(struct rte_ether_hdr) +
sizeof(struct rte_ipv4_hdr));
+ if (unlikely(ts_enable)) {
+ uint64_t skew = RTE_PER_LCORE(ts_qskew);
+ struct {
+ rte_be32_t signature;
+ rte_be16_t pkt_idx;
+ rte_be16_t queue_idx;
+ rte_be64_t ts;
+ } ts_mark;
+
+ if (unlikely(!skew)) {
+ struct rte_eth_dev *dev = &rte_eth_devices[fs->tx_port];
+ unsigned int txqs_n = dev->data->nb_tx_queues;
+ uint64_t phase = tx_pkt_times[0] * fs->tx_queue /
+ (txqs_n ? txqs_n : 1);
+ /*
+ * Initialize the scheduling time phase shift
+ * depending on queue index.
+ */
+ skew = ts_init[fs->tx_port] + tx_pkt_times[0] + phase;
+ RTE_PER_LCORE(ts_qskew) = skew;
+ }
+ ts_mark.pkt_idx = rte_cpu_to_be_16(idx);
+ ts_mark.queue_idx = rte_cpu_to_be_16(fs->tx_queue);
+ ts_mark.signature = rte_cpu_to_be_32(0xBEEFC0DE);
+ if (unlikely(!idx)) {
+ skew += tx_pkt_times[0];
+ pkt->ol_flags |= ts_mask;
+ *RTE_MBUF_DYNFIELD(pkt, ts_off, uint64_t *) = skew;
+ RTE_PER_LCORE(ts_qskew) = skew;
+ ts_mark.ts = rte_cpu_to_be_64(skew);
+ } else if (tx_pkt_times[1]) {
+ skew += tx_pkt_times[1];
+ pkt->ol_flags |= ts_mask;
+ *RTE_MBUF_DYNFIELD(pkt, ts_off, uint64_t *) = skew;
+ RTE_PER_LCORE(ts_qskew) = skew;
+ ts_mark.ts = rte_cpu_to_be_64(skew);
+ } else {
+ ts_mark.ts = RTE_BE64(0);
+ }
+ copy_buf_to_pkt(&ts_mark, sizeof(ts_mark), pkt,
+ sizeof(struct rte_ether_hdr) +
+ sizeof(struct rte_ipv4_hdr) +
+ sizeof(pkt_udp_hdr));
+ }
/*
* Complete first mbuf of packet and append it to the
* burst of packets to be transmitted.
@@ -275,7 +325,8 @@
if (unlikely(!pkt_burst_prepare(pkts_burst[nb_pkt], mbp,
ð_hdr, vlan_tci,
vlan_tci_outer,
- ol_flags))) {
+ ol_flags,
+ nb_pkt, fs))) {
rte_mempool_put_bulk(mbp,
(void **)&pkts_burst[nb_pkt],
nb_pkt_per_burst - nb_pkt);
@@ -290,7 +341,8 @@
if (unlikely(!pkt_burst_prepare(pkt, mbp, ð_hdr,
vlan_tci,
vlan_tci_outer,
- ol_flags))) {
+ ol_flags,
+ nb_pkt, fs))) {
rte_pktmbuf_free(pkt);
break;
}
@@ -302,6 +354,7 @@
return;
nb_tx = rte_eth_tx_burst(fs->tx_port, fs->tx_queue, pkts_burst, nb_pkt);
+
/*
* Retry if necessary
*/
@@ -342,15 +395,31 @@
}
static void
-tx_only_begin(__rte_unused portid_t pi)
+tx_only_begin(portid_t pi)
{
uint16_t pkt_data_len;
+ int dynf;
pkt_data_len = (uint16_t) (tx_pkt_length - (
sizeof(struct rte_ether_hdr) +
sizeof(struct rte_ipv4_hdr) +
sizeof(struct rte_udp_hdr)));
setup_pkt_udp_ip_headers(&pkt_ip_hdr, &pkt_udp_hdr, pkt_data_len);
+
+ ts_enable = false;
+ ts_mask = 0;
+ ts_off = -1;
+ RTE_PER_LCORE(ts_qskew) = 0;
+ dynf = rte_mbuf_dynflag_lookup
+ (RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME, NULL);
+ if (dynf >= 0)
+ ts_mask = 1ULL << dynf;
+ dynf = rte_mbuf_dynfield_lookup
+ (RTE_MBUF_DYNFIELD_TIMESTAMP_NAME, NULL);
+ if (dynf >= 0)
+ ts_off = dynf;
+ ts_enable = tx_pkt_times[0] && ts_mask && ts_off >= 0 &&
+ !rte_eth_read_clock(pi, &ts_init[pi]);
}
struct fwd_engine tx_only_engine = {
@@ -266,7 +266,7 @@ show config
Displays the configuration of the application.
The configuration comes from the command-line, the runtime or the application defaults::
- testpmd> show config (rxtx|cores|fwd|txpkts)
+ testpmd> show config (rxtx|cores|fwd|txpkts|txtimes)
The available information categories are:
@@ -278,6 +278,8 @@ The available information categories are:
* ``txpkts``: Packets to TX configuration.
+* ``txtimes``: Burst time pattern for Tx only mode.
+
For example:
.. code-block:: console
@@ -725,6 +727,38 @@ Set the length of each segment of the TX-ONLY packets or length of packet for FL
Where x[,y]* represents a CSV list of values, without white space.
+set txtimes
+~~~~~~~~~~
+
+Configure the timing burst pattern for Tx only mode. This command enables
+the packet send scheduling on dynamic timestamp mbuf field and configures
+timing pattern in Tx only mode. In this mode, if scheduling is enabled
+application provides timestamps in the packets being sent. It is possible
+to configure delay (in unspecified device clock units) between bursts
+and between the packets within the burst::
+
+ testpmd> set txtimes (inter),(intra)
+
+where:
+
+* ``inter`` is the delay between the bursts in the device clock units.
+ If ``intra`` is zero, this is the time between the beginnings of the
+ first packets in the neighbour bursts, if ``intra`` is not zero,
+ ``inter`` specifies the time between the beginning of the first packet
+ of the current burst and the beginning of the last packet of the
+ previous burst. If ``inter`` parameter is zero the send scheduling
+ on timestamps is disabled (default).
+
+* ``intra`` is the delay between the packets within the burst specified
+ in the device clock units. The number of packets in the burst is defined
+ by regular burst setting. If ``intra`` parameter is zero no timestamps
+ provided in the packets excepting the first one in the burst.
+
+As the result the bursts of packet will be transmitted with specific
+delays between the packets within the burst and specific delay between
+the bursts. The rte_eth_get_clock() is supposed to be engaged to get the
+current device clock value and provide the reference for the timestamps.
+
set txsplit
~~~~~~~~~~~