From patchwork Fri Jul 10 12:39:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 73742 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 37DB4A052A; Fri, 10 Jul 2020 14:39:49 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DB5ED1DC26; Fri, 10 Jul 2020 14:39:48 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id B63781D9AB for ; Fri, 10 Jul 2020 14:39:47 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with SMTP; 10 Jul 2020 15:39:46 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06ACdkFA028228; Fri, 10 Jul 2020 15:39:46 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id 06ACdkO6017844; Fri, 10 Jul 2020 12:39:46 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id 06ACditj017841; Fri, 10 Jul 2020 12:39:44 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, olivier.matz@6wind.com, arybchenko@solarflare.com, thomas@monjalon.net, ferruh.yigit@intel.com Date: Fri, 10 Jul 2020 12:39:41 +0000 Message-Id: <1594384782-17799-1-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591771085-24959-1-git-send-email-viacheslavo@mellanox.com> References: <1591771085-24959-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v7 1/2] mbuf: introduce accurate packet Tx scheduling X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" There is the requirement on some networks for precise traffic timing management. The ability to send (and, generally speaking, receive) the packets at the very precisely specified moment of time provides the opportunity to support the connections with Time Division Multiplexing using the contemporary general purpose NIC without involving an auxiliary hardware. For example, the supporting of O-RAN Fronthaul interface is one of the promising features for potentially usage of the precise time management for the egress packets. The main objective of this patchset is to specify the way how applications can provide the moment of time at what the packet transmission must be started and to describe in preliminary the supporting this feature from mlx5 PMD side [1]. The new dynamic timestamp field is proposed, it provides some timing information, the units and time references (initial phase) are not explicitly defined but are maintained always the same for a given port. Some devices allow to query rte_eth_read_clock() that will return the current device timestamp. The dynamic timestamp flag tells whether the field contains actual timestamp value. For the packets being sent this value can be used by PMD to schedule packet sending. The device clock is opaque entity, the units and frequency are vendor specific and might depend on hardware capabilities and configurations. If might (or not) be synchronized with real time via PTP, might (or not) be synchronous with CPU clock (for example if NIC and CPU share the same clock source there might be no any drift between the NIC and CPU clocks), etc. After PKT_RX_TIMESTAMP flag and fixed timestamp field supposed deprecation and obsoleting, these dynamic flag and field might be used to manage the timestamps on receiving datapath as well. Having the dedicated flags for Rx/Tx timestamps allows applications not to perform explicit flags reset on forwarding and not to promote received timestamps to the transmitting datapath by default. The static PKT_RX_TIMESTAMP is considered as candidate to become the dynamic flag and this move should be discussed. When PMD sees the "rte_dynfield_timestamp" set on the packet being sent it tries to synchronize the time of packet appearing on the wire with the specified packet timestamp. If the specified one is in the past it should be ignored, if one is in the distant future it should be capped with some reasonable value (in range of seconds). These specific cases ("too late" and "distant future") can be optionally reported via device xstats to assist applications to detect the time-related problems. There is no any packet reordering according timestamps is supposed, neither within packet burst, nor between packets, it is an entirely application responsibility to generate packets and its timestamps in desired order. The timestamps can be put only in the first packet in the burst providing the entire burst scheduling. PMD reports the ability to synchronize packet sending on timestamp with new offload flag: This is palliative and might be replaced with new eth_dev API about reporting/managing the supported dynamic flags and its related features. This API would break ABI compatibility and can't be introduced at the moment, so is postponed to 20.11. For testing purposes it is proposed to update testpmd "txonly" forwarding mode routine. With this update testpmd application generates the packets and sets the dynamic timestamps according to specified time pattern if it sees the "rte_dynfield_timestamp" is registered. The new testpmd command is proposed to configure sending pattern: set tx_times , - the delay between the packets within the burst specified in the device clock units. The number of packets in the burst is defined by txburst parameter - the delay between the bursts in the device clock units As the result the bursts of packet will be transmitted with specific delays between the packets within the burst and specific delay between the bursts. The rte_eth_read_clock is supposed to be engaged to get the current device clock value and provide the reference for the timestamps. [1] http://patches.dpdk.org/patch/73714/ Signed-off-by: Viacheslav Ovsiienko Acked-by: Olivier Matz Reviewed-by: Ferruh Yigit --- v1->v4: - dedicated dynamic Tx timestamp flag instead of shared with Rx v4->v5: - elaborated commit message - more words about device clocks added, - note about dedicated Rx/Tx timestamp flags added v5->v6: - release notes are updated v6->v7: - commit message is updated - testpmd checks the supported offloads before registering dynamic timestamp flag/field --- doc/guides/rel_notes/release_20_08.rst | 7 +++++++ lib/librte_ethdev/rte_ethdev.c | 1 + lib/librte_ethdev/rte_ethdev.h | 4 ++++ lib/librte_mbuf/rte_mbuf_dyn.h | 31 +++++++++++++++++++++++++++++++ 4 files changed, 43 insertions(+) diff --git a/doc/guides/rel_notes/release_20_08.rst b/doc/guides/rel_notes/release_20_08.rst index 988474c..bdea389 100644 --- a/doc/guides/rel_notes/release_20_08.rst +++ b/doc/guides/rel_notes/release_20_08.rst @@ -81,6 +81,13 @@ New Features Added the RegEx library which provides an API for offload of regular expressions search operations to hardware or software accelerator devices. + +* **Introduced send packet scheduling on the timestamps.** + + Added the new mbuf dynamic field and flag to provide timestamp on what packet + transmitting can be synchronized. The device Tx offload flag is added to + indicate the PMD supports send scheduling. + * **Updated PCAP driver.** Updated PCAP driver with new features and improvements, including: diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 7022bd7..c48ca2a 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -160,6 +160,7 @@ struct rte_eth_xstats_name_off { RTE_TX_OFFLOAD_BIT2STR(UDP_TNL_TSO), RTE_TX_OFFLOAD_BIT2STR(IP_TNL_TSO), RTE_TX_OFFLOAD_BIT2STR(OUTER_UDP_CKSUM), + RTE_TX_OFFLOAD_BIT2STR(SEND_ON_TIMESTAMP), }; #undef RTE_TX_OFFLOAD_BIT2STR diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h index 631b146..97313a0 100644 --- a/lib/librte_ethdev/rte_ethdev.h +++ b/lib/librte_ethdev/rte_ethdev.h @@ -1178,6 +1178,10 @@ struct rte_eth_conf { /** Device supports outer UDP checksum */ #define DEV_TX_OFFLOAD_OUTER_UDP_CKSUM 0x00100000 +/** Device supports send on timestamp */ +#define DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP 0x00200000 + + #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001 /**< Device supports Rx queue setup after device started*/ #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002 diff --git a/lib/librte_mbuf/rte_mbuf_dyn.h b/lib/librte_mbuf/rte_mbuf_dyn.h index 96c3631..8407230 100644 --- a/lib/librte_mbuf/rte_mbuf_dyn.h +++ b/lib/librte_mbuf/rte_mbuf_dyn.h @@ -250,4 +250,35 @@ int rte_mbuf_dynflag_lookup(const char *name, #define RTE_MBUF_DYNFIELD_METADATA_NAME "rte_flow_dynfield_metadata" #define RTE_MBUF_DYNFLAG_METADATA_NAME "rte_flow_dynflag_metadata" +/** + * The timestamp dynamic field provides some timing information, the + * units and time references (initial phase) are not explicitly defined + * but are maintained always the same for a given port. Some devices allow + * to query rte_eth_read_clock() that will return the current device + * timestamp. The dynamic Tx timestamp flag tells whether the field contains + * actual timestamp value for the packets being sent, this value can be + * used by PMD to schedule packet sending. + * + * After PKT_RX_TIMESTAMP flag and fixed timestamp field deprecation + * and obsoleting, the dedicated Rx timestamp flag is supposed to be + * introduced and the shared dynamic timestamp field will be used + * to handle the timestamps on receiving datapath as well. + */ +#define RTE_MBUF_DYNFIELD_TIMESTAMP_NAME "rte_dynfield_timestamp" + +/** + * When PMD sees the RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME flag set on the + * packet being sent it tries to synchronize the time of packet appearing + * on the wire with the specified packet timestamp. If the specified one + * is in the past it should be ignored, if one is in the distant future + * it should be capped with some reasonable value (in range of seconds). + * + * There is no any packet reordering according to timestamps is supposed, + * neither for packet within the burst, nor for the whole bursts, it is + * an entirely application responsibility to generate packets and its + * timestamps in desired order. The timestamps might be put only in + * the first packet in the burst providing the entire burst scheduling. + */ +#define RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME "rte_dynflag_tx_timestamp" + #endif From patchwork Fri Jul 10 12:39:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 73743 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8100FA052A; Fri, 10 Jul 2020 14:39:55 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 941E11DD6C; Fri, 10 Jul 2020 14:39:54 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 696E81DD6A for ; Fri, 10 Jul 2020 14:39:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with SMTP; 10 Jul 2020 15:39:48 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06ACdmfl028233; Fri, 10 Jul 2020 15:39:48 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id 06ACdm8i017848; Fri, 10 Jul 2020 12:39:48 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id 06ACdm9I017847; Fri, 10 Jul 2020 12:39:48 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, olivier.matz@6wind.com, arybchenko@solarflare.com, thomas@monjalon.net, ferruh.yigit@intel.com Date: Fri, 10 Jul 2020 12:39:42 +0000 Message-Id: <1594384782-17799-2-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594384782-17799-1-git-send-email-viacheslavo@mellanox.com> References: <1591771085-24959-1-git-send-email-viacheslavo@mellanox.com> <1594384782-17799-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v7 2/2] app/testpmd: add send scheduling test capability X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit adds testpmd capability to provide timestamps on the packets being sent in the txonly mode. This includes: - SEND_ON_TIMESTAMP support new device Tx offload capability support added, example: testpmd> port config 0 tx_offload send_on_timestamp on - set txtimes, registers field and flag, example: testpmd> set txtimes 1000000,0 This command enables the packet send scheduling on timestamps if the first parameter is not zero, generic format: testpmd> set txtimes (inter),(intra) where: inter - is the delay between the bursts in the device clock units. If "intra" (next parameter) is zero, this is the time between the beginnings of the first packets in the neighbour bursts, if "intra" is not zero, "inter" specifies the time between the beginning of the first packet of the current burst and the beginning of the last packet of the previous burst. If "inter"parameter is zero the send scheduling on timestamps is disabled (default). intra - is the delay between the packets within the burst specified in the device clock units. The number of packets in the burst is defined by regular burst setting. If "intra" parameter is zero no timestamps provided in the packets excepting the first one in the burst. As the result the bursts of packet will be transmitted with specific delay between the packets within the burst and specific delay between the bursts. The rte_eth_read_clock() is supposed to be engaged to get the current device clock value and provide the reference for the timestamps. If there is no supported rte_eth_read_clock() there will be no provided send scheduling on the device. - show txtimes, displays the timing settings - txonly burst time pattern Signed-off-by: Viacheslav Ovsiienko --- app/test-pmd/cmdline.c | 63 ++++++++++++++++++++-- app/test-pmd/config.c | 61 +++++++++++++++++++++ app/test-pmd/testpmd.c | 6 +++ app/test-pmd/testpmd.h | 4 ++ app/test-pmd/txonly.c | 83 +++++++++++++++++++++++++++-- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 38 ++++++++++++- 6 files changed, 246 insertions(+), 9 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 39ad938..def0709 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -294,6 +294,10 @@ static void cmd_help_long_parsed(void *parsed_result, " Right now only applicable for CSUM and TXONLY" " modes\n\n" + "set txtimes (x, y)\n" + " Set the scheduling on timestamps" + " timings for the TXOMLY mode\n\n" + "set corelist (x[,y]*)\n" " Set the list of forwarding cores.\n\n" @@ -3930,6 +3934,52 @@ struct cmd_set_txsplit_result { }, }; +/* *** SET TIMES FOR TXONLY PACKETS SCHEDULING ON TIMESTAMPS *** */ + +struct cmd_set_txtimes_result { + cmdline_fixed_string_t cmd_keyword; + cmdline_fixed_string_t txtimes; + cmdline_fixed_string_t tx_times; +}; + +static void +cmd_set_txtimes_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, + __rte_unused void *data) +{ + struct cmd_set_txtimes_result *res; + unsigned int tx_times[2] = {0, 0}; + unsigned int n_times; + + res = parsed_result; + n_times = parse_item_list(res->tx_times, "tx times", + 2, tx_times, 0); + if (n_times == 2) + set_tx_pkt_times(tx_times); +} + +cmdline_parse_token_string_t cmd_set_txtimes_keyword = + TOKEN_STRING_INITIALIZER(struct cmd_set_txtimes_result, + cmd_keyword, "set"); +cmdline_parse_token_string_t cmd_set_txtimes_name = + TOKEN_STRING_INITIALIZER(struct cmd_set_txtimes_result, + txtimes, "txtimes"); +cmdline_parse_token_string_t cmd_set_txtimes_value = + TOKEN_STRING_INITIALIZER(struct cmd_set_txtimes_result, + tx_times, NULL); + +cmdline_parse_inst_t cmd_set_txtimes = { + .f = cmd_set_txtimes_parsed, + .data = NULL, + .help_str = "set txtimes ,", + .tokens = { + (void *)&cmd_set_txtimes_keyword, + (void *)&cmd_set_txtimes_name, + (void *)&cmd_set_txtimes_value, + NULL, + }, +}; + /* *** ADD/REMOVE ALL VLAN IDENTIFIERS TO/FROM A PORT VLAN RX FILTER *** */ struct cmd_rx_vlan_filter_all_result { cmdline_fixed_string_t rx_vlan; @@ -7418,6 +7468,8 @@ static void cmd_showcfg_parsed(void *parsed_result, pkt_fwd_config_display(&cur_fwd_config); else if (!strcmp(res->what, "txpkts")) show_tx_pkt_segments(); + else if (!strcmp(res->what, "txtimes")) + show_tx_pkt_times(); } cmdline_parse_token_string_t cmd_showcfg_show = @@ -7426,12 +7478,12 @@ static void cmd_showcfg_parsed(void *parsed_result, TOKEN_STRING_INITIALIZER(struct cmd_showcfg_result, cfg, "config"); cmdline_parse_token_string_t cmd_showcfg_what = TOKEN_STRING_INITIALIZER(struct cmd_showcfg_result, what, - "rxtx#cores#fwd#txpkts"); + "rxtx#cores#fwd#txpkts#txtimes"); cmdline_parse_inst_t cmd_showcfg = { .f = cmd_showcfg_parsed, .data = NULL, - .help_str = "show config rxtx|cores|fwd|txpkts", + .help_str = "show config rxtx|cores|fwd|txpkts|txtimes", .tokens = { (void *)&cmd_showcfg_show, (void *)&cmd_showcfg_port, @@ -18670,7 +18722,8 @@ struct cmd_config_per_port_tx_offload_result { "sctp_cksum#tcp_tso#udp_tso#outer_ipv4_cksum#" "qinq_insert#vxlan_tnl_tso#gre_tnl_tso#" "ipip_tnl_tso#geneve_tnl_tso#macsec_insert#" - "mt_lockfree#multi_segs#mbuf_fast_free#security"); + "mt_lockfree#multi_segs#mbuf_fast_free#security#" + "send_on_timestamp"); cmdline_parse_token_string_t cmd_config_per_port_tx_offload_result_on_off = TOKEN_STRING_INITIALIZER (struct cmd_config_per_port_tx_offload_result, @@ -18755,7 +18808,8 @@ struct cmd_config_per_port_tx_offload_result { "sctp_cksum|tcp_tso|udp_tso|outer_ipv4_cksum|" "qinq_insert|vxlan_tnl_tso|gre_tnl_tso|" "ipip_tnl_tso|geneve_tnl_tso|macsec_insert|" - "mt_lockfree|multi_segs|mbuf_fast_free|security on|off", + "mt_lockfree|multi_segs|mbuf_fast_free|security|" + "send_on_timestamp on|off", .tokens = { (void *)&cmd_config_per_port_tx_offload_result_port, (void *)&cmd_config_per_port_tx_offload_result_config, @@ -19427,6 +19481,7 @@ struct cmd_showport_macs_result { (cmdline_parse_inst_t *)&cmd_set_log, (cmdline_parse_inst_t *)&cmd_set_txpkts, (cmdline_parse_inst_t *)&cmd_set_txsplit, + (cmdline_parse_inst_t *)&cmd_set_txtimes, (cmdline_parse_inst_t *)&cmd_set_fwd_list, (cmdline_parse_inst_t *)&cmd_set_fwd_mask, (cmdline_parse_inst_t *)&cmd_set_fwd_mode, diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index 53a9b97..48a8790 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -1046,6 +1046,15 @@ static int bus_match_all(const struct rte_bus *bus, const void *data) printf("off\n"); } + if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP) { + printf("Tx scheduling on timestamp: "); + if (ports[port_id].dev_conf.txmode.offloads & + DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP) + printf("on\n"); + else + printf("off\n"); + } + } int @@ -3067,6 +3076,58 @@ struct igb_ring_desc_16_bytes { } void +show_tx_pkt_times(void) +{ + printf("Interburst gap: %u\n", tx_pkt_times_inter); + printf("Intraburst gap: %u\n", tx_pkt_times_intra); +} + +void +set_tx_pkt_times(unsigned int *tx_times) +{ + uint16_t port_id; + int offload_found = 0; + int offset; + int flag; + + static const struct rte_mbuf_dynfield desc_offs = { + .name = RTE_MBUF_DYNFIELD_TIMESTAMP_NAME, + .size = sizeof(uint64_t), + .align = __alignof__(uint64_t), + }; + static const struct rte_mbuf_dynflag desc_flag = { + .name = RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME, + }; + + RTE_ETH_FOREACH_DEV(port_id) { + struct rte_eth_dev_info dev_info = { 0 }; + int ret; + + ret = rte_eth_dev_info_get(port_id, &dev_info); + if (ret == 0 && dev_info.tx_offload_capa & + DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP) { + offload_found = 1; + break; + } + } + if (!offload_found) { + printf("No device supporting Tx timestamp scheduling found, " + "dynamic flag and field not registered\n"); + return; + } + offset = rte_mbuf_dynfield_register(&desc_offs); + if (offset < 0 && rte_errno != EEXIST) + printf("Dynamic timestamp field registration error: %d", + rte_errno); + flag = rte_mbuf_dynflag_register(&desc_flag); + if (flag < 0 && rte_errno != EEXIST) + printf("Dynamic timestamp flag registration error: %d", + rte_errno); + tx_pkt_times_inter = tx_times[0]; + tx_pkt_times_intra = tx_times[1]; +} + +void setup_gro(const char *onoff, portid_t port_id) { if (!rte_eth_dev_is_valid_port(port_id)) { diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index f5feb62..d1216e0 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -223,6 +223,12 @@ struct fwd_engine * fwd_engines[] = { uint8_t txonly_multi_flow; /**< Whether multiple flows are generated in TXONLY mode. */ +uint32_t tx_pkt_times_inter; +/**< Timings for send scheduling in TXONLY mode, time between bursts. */ + +uint32_t tx_pkt_times_intra; +/**< Timings for send scheduling in TXONLY mode, time between packets. */ + uint16_t nb_pkt_per_burst = DEF_PKT_BURST; /**< Number of packets per burst. */ uint16_t mb_mempool_cache = DEF_MBUF_CACHE; /**< Size of mbuf mempool cache. */ diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index a803750..9825649 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -442,6 +442,8 @@ struct queue_stats_mappings { extern uint16_t tx_pkt_length; /**< Length of TXONLY packet */ extern uint16_t tx_pkt_seg_lengths[RTE_MAX_SEGS_PER_PKT]; /**< Seg. lengths */ extern uint8_t tx_pkt_nb_segs; /**< Number of segments in TX packets */ +extern uint32_t tx_pkt_times_intra; +extern uint32_t tx_pkt_times_inter; enum tx_pkt_split { TX_PKT_SPLIT_OFF, @@ -794,6 +796,8 @@ void vlan_tpid_set(portid_t port_id, enum rte_vlan_type vlan_type, void set_verbose_level(uint16_t vb_level); void set_tx_pkt_segments(unsigned *seg_lengths, unsigned nb_segs); void show_tx_pkt_segments(void); +void set_tx_pkt_times(unsigned int *tx_times); +void show_tx_pkt_times(void); void set_tx_pkt_split(const char *name); void set_nb_pkt_per_burst(uint16_t pkt_burst); char *list_pkt_forwarding_modes(void); diff --git a/app/test-pmd/txonly.c b/app/test-pmd/txonly.c index 076ccaf..97f4a45 100644 --- a/app/test-pmd/txonly.c +++ b/app/test-pmd/txonly.c @@ -53,6 +53,12 @@ static struct rte_ipv4_hdr pkt_ip_hdr; /**< IP header of transmitted packets. */ RTE_DEFINE_PER_LCORE(uint8_t, _ip_var); /**< IP address variation */ static struct rte_udp_hdr pkt_udp_hdr; /**< UDP header of tx packets. */ +RTE_DEFINE_PER_LCORE(uint64_t, timestamp_qskew); + /**< Timestamp offset per queue */ +static uint64_t timestamp_mask; /**< Timestamp dynamic flag mask */ +static int32_t timestamp_off; /**< Timestamp dynamic field offset */ +static bool timestamp_enable; /**< Timestamp enable */ +static uint64_t timestamp_initial[RTE_MAX_ETHPORTS]; static void copy_buf_to_pkt_segs(void* buf, unsigned len, struct rte_mbuf *pkt, @@ -150,7 +156,8 @@ static inline bool pkt_burst_prepare(struct rte_mbuf *pkt, struct rte_mempool *mbp, struct rte_ether_hdr *eth_hdr, const uint16_t vlan_tci, - const uint16_t vlan_tci_outer, const uint64_t ol_flags) + const uint16_t vlan_tci_outer, const uint64_t ol_flags, + const uint16_t idx, const struct fwd_stream *fs) { struct rte_mbuf *pkt_segs[RTE_MAX_SEGS_PER_PKT]; struct rte_mbuf *pkt_seg; @@ -213,6 +220,53 @@ copy_buf_to_pkt(&pkt_udp_hdr, sizeof(pkt_udp_hdr), pkt, sizeof(struct rte_ether_hdr) + sizeof(struct rte_ipv4_hdr)); + if (unlikely(timestamp_enable)) { + uint64_t skew = RTE_PER_LCORE(timestamp_qskew); + struct { + rte_be32_t signature; + rte_be16_t pkt_idx; + rte_be16_t queue_idx; + rte_be64_t ts; + } timestamp_mark; + + if (unlikely(!skew)) { + struct rte_eth_dev *dev = &rte_eth_devices[fs->tx_port]; + unsigned int txqs_n = dev->data->nb_tx_queues; + uint64_t phase = tx_pkt_times_inter * fs->tx_queue / + (txqs_n ? txqs_n : 1); + /* + * Initialize the scheduling time phase shift + * depending on queue index. + */ + skew = timestamp_initial[fs->tx_port] + + tx_pkt_times_inter + phase; + RTE_PER_LCORE(timestamp_qskew) = skew; + } + timestamp_mark.pkt_idx = rte_cpu_to_be_16(idx); + timestamp_mark.queue_idx = rte_cpu_to_be_16(fs->tx_queue); + timestamp_mark.signature = rte_cpu_to_be_32(0xBEEFC0DE); + if (unlikely(!idx)) { + skew += tx_pkt_times_inter; + pkt->ol_flags |= timestamp_mask; + *RTE_MBUF_DYNFIELD + (pkt, timestamp_off, uint64_t *) = skew; + RTE_PER_LCORE(timestamp_qskew) = skew; + timestamp_mark.ts = rte_cpu_to_be_64(skew); + } else if (tx_pkt_times_intra) { + skew += tx_pkt_times_intra; + pkt->ol_flags |= timestamp_mask; + *RTE_MBUF_DYNFIELD + (pkt, timestamp_off, uint64_t *) = skew; + RTE_PER_LCORE(timestamp_qskew) = skew; + timestamp_mark.ts = rte_cpu_to_be_64(skew); + } else { + timestamp_mark.ts = RTE_BE64(0); + } + copy_buf_to_pkt(×tamp_mark, sizeof(timestamp_mark), pkt, + sizeof(struct rte_ether_hdr) + + sizeof(struct rte_ipv4_hdr) + + sizeof(pkt_udp_hdr)); + } /* * Complete first mbuf of packet and append it to the * burst of packets to be transmitted. @@ -275,7 +329,8 @@ if (unlikely(!pkt_burst_prepare(pkts_burst[nb_pkt], mbp, ð_hdr, vlan_tci, vlan_tci_outer, - ol_flags))) { + ol_flags, + nb_pkt, fs))) { rte_mempool_put_bulk(mbp, (void **)&pkts_burst[nb_pkt], nb_pkt_per_burst - nb_pkt); @@ -290,7 +345,8 @@ if (unlikely(!pkt_burst_prepare(pkt, mbp, ð_hdr, vlan_tci, vlan_tci_outer, - ol_flags))) { + ol_flags, + nb_pkt, fs))) { rte_pktmbuf_free(pkt); break; } @@ -302,6 +358,7 @@ return; nb_tx = rte_eth_tx_burst(fs->tx_port, fs->tx_queue, pkts_burst, nb_pkt); + /* * Retry if necessary */ @@ -342,15 +399,33 @@ } static void -tx_only_begin(__rte_unused portid_t pi) +tx_only_begin(portid_t pi) { uint16_t pkt_data_len; + int dynf; pkt_data_len = (uint16_t) (tx_pkt_length - ( sizeof(struct rte_ether_hdr) + sizeof(struct rte_ipv4_hdr) + sizeof(struct rte_udp_hdr))); setup_pkt_udp_ip_headers(&pkt_ip_hdr, &pkt_udp_hdr, pkt_data_len); + + timestamp_enable = false; + timestamp_mask = 0; + timestamp_off = -1; + RTE_PER_LCORE(timestamp_qskew) = 0; + dynf = rte_mbuf_dynflag_lookup + (RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME, NULL); + if (dynf >= 0) + timestamp_mask = 1ULL << dynf; + dynf = rte_mbuf_dynfield_lookup + (RTE_MBUF_DYNFIELD_TIMESTAMP_NAME, NULL); + if (dynf >= 0) + timestamp_off = dynf; + timestamp_enable = tx_pkt_times_inter && + timestamp_mask && + timestamp_off >= 0 && + !rte_eth_read_clock(pi, ×tamp_initial[pi]); } struct fwd_engine tx_only_engine = { diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index a808b6a..3f54787 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -266,7 +266,7 @@ show config Displays the configuration of the application. The configuration comes from the command-line, the runtime or the application defaults:: - testpmd> show config (rxtx|cores|fwd|txpkts) + testpmd> show config (rxtx|cores|fwd|txpkts|txtimes) The available information categories are: @@ -278,6 +278,8 @@ The available information categories are: * ``txpkts``: Packets to TX configuration. +* ``txtimes``: Burst time pattern for Tx only mode. + For example: .. code-block:: console @@ -725,6 +727,40 @@ Set the length of each segment of the TX-ONLY packets or length of packet for FL Where x[,y]* represents a CSV list of values, without white space. +set txtimes +~~~~~~~~~~~ + +Configure the timing burst pattern for Tx only mode. This command enables +the packet send scheduling on dynamic timestamp mbuf field and configures +timing pattern in Tx only mode. In this mode, if scheduling is enabled +application provides timestamps in the packets being sent. It is possible +to configure delay (in unspecified device clock units) between bursts +and between the packets within the burst:: + + testpmd> set txtimes (inter),(intra) + +where: + +* ``inter`` is the delay between the bursts in the device clock units. + If ``intra`` is zero, this is the time between the beginnings of the + first packets in the neighbour bursts, if ``intra`` is not zero, + ``inter`` specifies the time between the beginning of the first packet + of the current burst and the beginning of the last packet of the + previous burst. If ``inter`` parameter is zero the send scheduling + on timestamps is disabled (default). + +* ``intra`` is the delay between the packets within the burst specified + in the device clock units. The number of packets in the burst is defined + by regular burst setting. If ``intra`` parameter is zero no timestamps + provided in the packets excepting the first one in the burst. + +As the result the bursts of packet will be transmitted with specific +delays between the packets within the burst and specific delay between +the bursts. The rte_eth_read_clock() must be supported by the device(s) +and is supposed to be engaged to get the current device clock value +and provide the reference for the timestamps. If there is no supported +rte_eth_read_clock() there will be no send scheduling provided on the port. + set txsplit ~~~~~~~~~~~