[v3,1/3] rte_ethdev: Add API function to read dev clock

  Add rte_eth_read_clock to read the raw clock of a devide.

The main use is to get the device clock conversion co-efficients to be
able to translate the raw clock of the timestamp field of the pkt mbuf
to a local synced time value.

This function was missing to allow users to convert the RX timestamp field
to real time without the complexity of the rte_timesync* facility. One can
derivate the clock frequency by calling twice read_clock and then keep a
common time base.

Signed-off-by: Tom Barbette <barbette@kth.se>
---
 doc/guides/nics/features.rst             |  1 +
 lib/librte_ethdev/rte_ethdev.c           | 13 +++++++
 lib/librte_ethdev/rte_ethdev.h           | 45 ++++++++++++++++++++++++
 lib/librte_ethdev/rte_ethdev_core.h      |  6 ++++
 lib/librte_ethdev/rte_ethdev_version.map |  2 ++
 lib/librte_mbuf/rte_mbuf.h               |  2 ++
 6 files changed, 69 insertions(+)
  

On 4/24/19 8:34 PM, Tom Barbette wrote:
> Add rte_eth_read_clock to read the raw clock of a devide.

Typo above "devide"

> The main use is to get the device clock conversion co-efficients to be
> able to translate the raw clock of the timestamp field of the pkt mbuf
> to a local synced time value.
>
> This function was missing to allow users to convert the RX timestamp field

RX -> Rx

> to real time without the complexity of the rte_timesync* facility. One can
> derivate the clock frequency by calling twice read_clock and then keep a
> common time base.
>
> Signed-off-by: Tom Barbette <barbette@kth.se>

[snip]

> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index d7cfa3d53..22e35d839 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -4170,6 +4170,19 @@ rte_eth_timesync_write_time(uint16_t port_id, const struct timespec *timestamp)
>   								timestamp));
>   }
>   
> +int
> +rte_eth_read_clock(uint16_t port_id, uint64_t *timestamp)
> +{
> +	struct rte_eth_dev *dev;
> +
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> +	dev = &rte_eth_devices[port_id];
> +
> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->read_clock, -ENOTSUP);
> +	return eth_err(port_id, (*dev->dev_ops->read_clock)(dev,
> +								timestamp));

Why timestamp on the next line?

> +}
> +
>   int
>   rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
>   {
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index b8d19c69f..dfaf16a56 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -3793,6 +3793,51 @@ int rte_eth_timesync_read_time(uint16_t port_id, struct timespec *time);
>    */
>   int rte_eth_timesync_write_time(uint16_t port_id, const struct timespec *time);
>   
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Read the current clock counter of an Ethernet device
> + *
> + * This returns the current raw clock value of an Ethernet device.
> + * The value returned here is from the same clock than the one
> + * filling timestamp field of RX packets. Therefore it can be used

RX -> Rx

> + * to compute a precise conversion of the device clock to the real time.
> + *
> + * E.g, a simple heuristic to derivate the frequency would be:
> + * uint64_t start, end;
> + * rte_eth_read_clock(port, start);
> + * rte_delay_ms(100);
> + * rte_eth_read_clock(port, end);
> + * double freq = (end - start) * 10;
> + *
> + * Compute a common reference with:
> + * uint64_t base_time_sec = current_time();
> + * uint64_t base_clock;
> + * rte_eth_read_clock(port, base_clock);
> + *
> + * Then, convert the raw mbuf timestamp with:
> + * base_time_sec + (double)(mbuf->timestamp - base_clock) / freq;
> + *
> + * This simple example will not provide a very good accuracy. One must
> + * at least measure multiple times the frequency and do a regression.
> + * To avoid deviation from the system time, the common reference can
> + * be repeated from time to time. The integer division can also be
> + * converted by a multiplication and a shift for better performance.
> + *
> + * @param port_id
> + *   The port identifier of the Ethernet device.
> + * @param time
> + *   Pointer to the uint64_t that holds the raw clock value.
> + *
> + * @return
> + *   - 0: Success.
> + *   - -ENODEV: The port ID is invalid.
> + *   - -ENOTSUP: The function is not supported by the Ethernet driver.
> + */
> +int __rte_experimental
> +rte_eth_read_clock(uint16_t port_id, uint64_t *timestamp);
> +
>   /**
>    * Config l2 tunnel ether type of an Ethernet device for filtering specific
>    * tunnel packets by ether type.

[snip]

> eth_xstats_get_names_by_id_t xstats_get_names_by_id;
> diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map
> index afcd25599..670e4fad5 100644
> --- a/lib/librte_ethdev/rte_ethdev_version.map
> +++ b/lib/librte_ethdev/rte_ethdev_version.map
> @@ -253,10 +253,12 @@ EXPERIMENTAL {
>   	rte_eth_dev_rx_intr_ctl_q_get_fd;
>   	rte_eth_find_next_of;
>   	rte_eth_find_next_sibling;
> +	rte_eth_read_clock;
>   	rte_eth_switch_domain_alloc;
>   	rte_eth_switch_domain_free;
>   	rte_flow_conv;
>   	rte_flow_expand_rss;
> +	rte_eth_read_clock;

Dup

[snip]
  

> On Apr 24, 2019, at 12:34 PM, Tom Barbette <barbette@kth.se> wrote:
> 
> Add rte_eth_read_clock to read the raw clock of a devide.
> 
> The main use is to get the device clock conversion co-efficients to be
> able to translate the raw clock of the timestamp field of the pkt mbuf
> to a local synced time value.
> 
> This function was missing to allow users to convert the RX timestamp field
> to real time without the complexity of the rte_timesync* facility. One can
> derivate the clock frequency by calling twice read_clock and then keep a
> common time base.
> 
> Signed-off-by: Tom Barbette <barbette@kth.se>
> ---
> doc/guides/nics/features.rst             |  1 +
> lib/librte_ethdev/rte_ethdev.c           | 13 +++++++
> lib/librte_ethdev/rte_ethdev.h           | 45 ++++++++++++++++++++++++
> lib/librte_ethdev/rte_ethdev_core.h      |  6 ++++
> lib/librte_ethdev/rte_ethdev_version.map |  2 ++
> lib/librte_mbuf/rte_mbuf.h               |  2 ++
> 6 files changed, 69 insertions(+)
> 
> diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst
> index c5bf32222..025b7f812 100644
> --- a/doc/guides/nics/features.rst
> +++ b/doc/guides/nics/features.rst
> @@ -602,6 +602,7 @@ Supports Timestamp.
> * **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_TIMESTAMP``.
> * **[provides] mbuf**: ``mbuf.timestamp``.
> * **[provides] rte_eth_dev_info**: ``rx_offload_capa,rx_queue_offload_capa: DEV_RX_OFFLOAD_TIMESTAMP``.
> +* **[related] eth_dev_ops**: ``read_clock``.
> 
> .. _nic_features_macsec_offload:
> 
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index d7cfa3d53..22e35d839 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -4170,6 +4170,19 @@ rte_eth_timesync_write_time(uint16_t port_id, const struct timespec *timestamp)
> 								timestamp));
> }
> 
> +int
> +rte_eth_read_clock(uint16_t port_id, uint64_t *timestamp)
> +{
> +	struct rte_eth_dev *dev;
> +
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> +	dev = &rte_eth_devices[port_id];
> +
> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->read_clock, -ENOTSUP);
> +	return eth_err(port_id, (*dev->dev_ops->read_clock)(dev,
> +								timestamp));
> +}
> +
> int
> rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
> {
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index b8d19c69f..dfaf16a56 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -3793,6 +3793,51 @@ int rte_eth_timesync_read_time(uint16_t port_id, struct timespec *time);
>  */
> int rte_eth_timesync_write_time(uint16_t port_id, const struct timespec *time);
> 
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Read the current clock counter of an Ethernet device
> + *
> + * This returns the current raw clock value of an Ethernet device.
> + * The value returned here is from the same clock than the one
> + * filling timestamp field of RX packets. Therefore it can be used
> + * to compute a precise conversion of the device clock to the real time.
> + *
> + * E.g, a simple heuristic to derivate the frequency would be:
> + * uint64_t start, end;
> + * rte_eth_read_clock(port, start);
> + * rte_delay_ms(100);
> + * rte_eth_read_clock(port, end);
> + * double freq = (end - start) * 10;
> + *
> + * Compute a common reference with:
> + * uint64_t base_time_sec = current_time();
> + * uint64_t base_clock;
> + * rte_eth_read_clock(port, base_clock);
> + *
> + * Then, convert the raw mbuf timestamp with:
> + * base_time_sec + (double)(mbuf->timestamp - base_clock) / freq;
> + *
> + * This simple example will not provide a very good accuracy. One must
> + * at least measure multiple times the frequency and do a regression.
> + * To avoid deviation from the system time, the common reference can
> + * be repeated from time to time. The integer division can also be
> + * converted by a multiplication and a shift for better performance.
> + *
> + * @param port_id
> + *   The port identifier of the Ethernet device.
> + * @param time
> + *   Pointer to the uint64_t that holds the raw clock value.

What is a raw clock value? It took me a bit to find that it is in nano-seconds need to document that point.
> + *
> + * @return
> + *   - 0: Success.
> + *   - -ENODEV: The port ID is invalid.
> + *   - -ENOTSUP: The function is not supported by the Ethernet driver.
> + */
> +int __rte_experimental
> +rte_eth_read_clock(uint16_t port_id, uint64_t *timestamp);

Using timestamp variable is not very descriptive and some other name needs to be used. Also in the driver it is called *clock.

Another question is why does this routine not perform the looping/delaying and calling read_clock and then return frequency instead. If you want a timestamp reading function then this one is not being described that way and I would think it should be done someplace else or should be.
> +
> /**
>  * Config l2 tunnel ether type of an Ethernet device for filtering specific
>  * tunnel packets by ether type.
> diff --git a/lib/librte_ethdev/rte_ethdev_core.h b/lib/librte_ethdev/rte_ethdev_core.h
> index 8f03f83f6..86806b3eb 100644
> --- a/lib/librte_ethdev/rte_ethdev_core.h
> +++ b/lib/librte_ethdev/rte_ethdev_core.h
> @@ -322,6 +322,10 @@ typedef int (*eth_timesync_write_time)(struct rte_eth_dev *dev,
> 				       const struct timespec *timestamp);
> /**< @internal Function used to get time from the device clock */
> 
> +typedef int (*eth_read_clock)(struct rte_eth_dev *dev,
> +				      uint64_t *timestamp);
> +/**< @internal Function used to get the current value of the device clock. */
> +
> typedef int (*eth_get_reg_t)(struct rte_eth_dev *dev,
> 				struct rte_dev_reg_info *info);
> /**< @internal Retrieve registers  */
> @@ -496,6 +500,8 @@ struct eth_dev_ops {
> 	eth_timesync_read_time     timesync_read_time; /** Get the device clock time. */
> 	eth_timesync_write_time    timesync_write_time; /** Set the device clock time. */
> 
> +	eth_read_clock             read_clock;
> +
> 	eth_xstats_get_by_id_t     xstats_get_by_id;
> 	/**< Get extended device statistic values by ID. */
> 	eth_xstats_get_names_by_id_t xstats_get_names_by_id;
> diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map
> index afcd25599..670e4fad5 100644
> --- a/lib/librte_ethdev/rte_ethdev_version.map
> +++ b/lib/librte_ethdev/rte_ethdev_version.map
> @@ -253,10 +253,12 @@ EXPERIMENTAL {
> 	rte_eth_dev_rx_intr_ctl_q_get_fd;
> 	rte_eth_find_next_of;
> 	rte_eth_find_next_sibling;
> +	rte_eth_read_clock;
> 	rte_eth_switch_domain_alloc;
> 	rte_eth_switch_domain_free;
> 	rte_flow_conv;
> 	rte_flow_expand_rss;
> +	rte_eth_read_clock;
> 	rte_mtr_capabilities_get;
> 	rte_mtr_create;
> 	rte_mtr_destroy;
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> index 68415af02..e530a96c5 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -668,6 +668,8 @@ struct rte_mbuf {
> 
> 	/** Valid if PKT_RX_TIMESTAMP is set. The unit and time reference
> 	 * are not normalized but are always the same for a given port.
> +	 * Some devices allow to query rte_eth_read_clock that will return the
> +	 * current device timestamp.

The message here is not a good place for this detail, I would put it in the function call doc above.
> 	 */
> 	uint64_t timestamp;
> 
> -- 
> 2.17.1
> 

Regards,
Keith
  

@Andrew I applied your comments. Thanks.

On 2019-04-25 20:28, Wiles, Keith wrote:
> What is a raw clock value? It took me a bit to find that it is in nano-seconds need to document that point.

It is not in nanosecond, it has no units. Finding the relation between 
the device clock and the real time is the whole point of this API.

> Using timestamp variable is not very descriptive and some other name needs to be used. Also in the driver it is called *clock.

The problem is that the "timestamp offload" feature filling the 
timestamp field of the pktmbuf is already badly named as it is given 
without time unit. Both of them are not timestamp but raw "clock" 
values, a number of ticks at an unknown frequency, with an unknow time 
base (ie is it the number of ticks since boot? Device is up? 1/1/1970?).
One solution would be to have an union in the pktmbuf, changing the 
timestamp field into a "clock" or "timestamp" field. But it's a bit 
overkill.

I propose to change the read_clock API to reference "clock" instead of 
timestamp everywhere.

> Another question is why does this routine not perform the looping/delaying and calling read_clock and then return frequency instead. If you want a timestamp reading function then this one is not being described that way and I would think it should be done someplace else or should be.

The frequency is one thing, and would allow to give a relative time in 
second between packets. In practice though, we change the frequency (as 
Linux does regarding the TSC ticks) to catch up corrections applied by 
NTP on the source clock.
The second thing is the time reference, to convert the clock time (of 
the packets) to the current wall time. One may want to use the monotonic 
clock, the real clock (we do follow both). Or no system clock at all and 
directly follow an NTP source. The point is, having a function to give 
the frequency is not enough. One can derive the frequency and the time 
reference with this API. I could write a helper in a second step to get 
the frequency out of read_clock. But the time reference implies timers, 
choosing a source, and stuff we don't want here, I think...

>> 	/** Valid if PKT_RX_TIMESTAMP is set. The unit and time reference
>> 	 * are not normalized but are always the same for a given port.
>> +	 * Some devices allow to query rte_eth_read_clock that will return the
>> +	 * current device timestamp.
> 
> The message here is not a good place for this detail, I would put it in the function call doc above.

I can remove these lines, but with read_clock referencing only clock I 
feel like it's even more needed. Someone reading the doc will see "oh, I 
can use timestamp offloading to get precise timing of when my packets 
were received, great ! But it has no unit and no reference... What do I 
do with that?". It miss the last step "how I see... I can look at 
rte_eth_read_clock documentation to read more about conversion."

Thanks !

Tom