[v9,03/12] telemetry: initial telemetry infrastructure
Checks
Commit Message
From: Ciara Power <ciara.power@intel.com>
This patch adds the infrastructure and initial code for the telemetry
library.
The telemetry init is registered with eal_init(). We can then check to see
if --telemetry was passed as an eal option. If --telemetry was parsed, then
we call telemetry init at the end of eal init.
Control threads are used to get CPU cycles for telemetry, which are
configured in this patch also.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
---
v9:
- Add telemetry.h to doxygen index/conf (Thomas)
- Add @file section to telemetry.h for Doxygen (Thomas)
- Move -ljansson to patch that introduces json_* calls (Thomas)
---
config/common_base | 5 +
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf.in | 1 +
lib/Makefile | 2 +
lib/librte_telemetry/Makefile | 27 ++++
lib/librte_telemetry/meson.build | 7 +
lib/librte_telemetry/rte_telemetry.c | 123 ++++++++++++++++++
lib/librte_telemetry/rte_telemetry.h | 51 ++++++++
lib/librte_telemetry/rte_telemetry_internal.h | 32 +++++
.../rte_telemetry_version.map | 8 ++
lib/meson.build | 2 +-
mk/rte.app.mk | 3 +-
mk/rte.vars.mk | 6 +
13 files changed, 267 insertions(+), 3 deletions(-)
create mode 100644 lib/librte_telemetry/Makefile
create mode 100644 lib/librte_telemetry/meson.build
create mode 100644 lib/librte_telemetry/rte_telemetry.c
create mode 100644 lib/librte_telemetry/rte_telemetry.h
create mode 100644 lib/librte_telemetry/rte_telemetry_internal.h
create mode 100644 lib/librte_telemetry/rte_telemetry_version.map
Comments
27/10/2018 01:59, Harry van Haaren:
> --- a/mk/rte.vars.mk
> +++ b/mk/rte.vars.mk
> +JANSSON := $(shell pkg-config --exists jansson; echo $$?)
> +ifneq ($(JANSSON),0)
> +$(warning Jansson not found, disabling RTE_LIBRTE_TELEMETRY)
> +CONFIG_RTE_LIBRTE_TELEMETRY = n
> +endif
It fails for cross-compilation.
Example:
When compiling i686 on x86_64 host, no error with
pkg-config --exists jansson
but fails when linking:
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1/../../../libjansson.so when searching for -ljansson
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1/../../../libjansson.a when searching for -ljansson
/usr/bin/ld: skipping incompatible /usr/lib/libjansson.so when searching for -ljansson
/usr/bin/ld: skipping incompatible /usr/lib/libjansson.a when searching for -ljansson
/usr/bin/ld: cannot find -ljansson
Note: there is the same issue with examples/vm_power_manager/Makefile
I start thinking it is not reasonnable to enable an optional dependency
in our Makefiles. Perhaps the only solution is to use meson...
> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Friday, October 26, 2018 6:56 PM
> To: Van Haaren, Harry <harry.van.haaren@intel.com>; Laatz, Kevin
> <kevin.laatz@intel.com>; Nicolau, Radu <radu.nicolau@intel.com>; Hunt, David
> <david.hunt@intel.com>
> Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
> stephen@networkplumber.org; gaetan.rivet@6wind.com; shreyansh.jain@nxp.com;
> mattias.ronnblom@ericsson.com; Ciara Power <ciara.power@intel.com>; Brian
> Archbold <brian.archbold@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v9 03/12] telemetry: initial telemetry
> infrastructure
>
> 27/10/2018 01:59, Harry van Haaren:
> > --- a/mk/rte.vars.mk
> > +++ b/mk/rte.vars.mk
> > +JANSSON := $(shell pkg-config --exists jansson; echo $$?)
> > +ifneq ($(JANSSON),0)
> > +$(warning Jansson not found, disabling RTE_LIBRTE_TELEMETRY)
> > +CONFIG_RTE_LIBRTE_TELEMETRY = n
> > +endif
>
> It fails for cross-compilation.
> Example:
> When compiling i686 on x86_64 host, no error with
> pkg-config --exists jansson
> but fails when linking:
> /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-
> gnu/8.2.1/../../../libjansson.so when searching for -ljansson
> /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-
> gnu/8.2.1/../../../libjansson.a when searching for -ljansson
> /usr/bin/ld: skipping incompatible /usr/lib/libjansson.so when searching
> for -ljansson
> /usr/bin/ld: skipping incompatible /usr/lib/libjansson.a when searching
> for -ljansson
> /usr/bin/ld: cannot find -ljansson
>
> Note: there is the same issue with examples/vm_power_manager/Makefile
>
> I start thinking it is not reasonnable to enable an optional dependency
> in our Makefiles. Perhaps the only solution is to use meson...
Ouch, yes if x86_64 pkg-config finds the available library, and it is not available in its i686 form, or other combinations thereof, things become another-level of complex.
Indeed Meson is (good | getting better) at detecting these things, Bruce knows more IIRC :)
@Thomas, for this release I propose to rework to use CONFIG=N as default for Make, and allow packagers to enable if they wish. Is that a good next-step?
27/10/2018 04:19, Van Haaren, Harry:
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > 27/10/2018 01:59, Harry van Haaren:
> > > --- a/mk/rte.vars.mk
> > > +++ b/mk/rte.vars.mk
> > > +JANSSON := $(shell pkg-config --exists jansson; echo $$?)
> > > +ifneq ($(JANSSON),0)
> > > +$(warning Jansson not found, disabling RTE_LIBRTE_TELEMETRY)
> > > +CONFIG_RTE_LIBRTE_TELEMETRY = n
> > > +endif
> >
> > It fails for cross-compilation.
> > Example:
> > When compiling i686 on x86_64 host, no error with
> > pkg-config --exists jansson
> > but fails when linking:
> > /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-
> > gnu/8.2.1/../../../libjansson.so when searching for -ljansson
> > /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-
> > gnu/8.2.1/../../../libjansson.a when searching for -ljansson
> > /usr/bin/ld: skipping incompatible /usr/lib/libjansson.so when searching
> > for -ljansson
> > /usr/bin/ld: skipping incompatible /usr/lib/libjansson.a when searching
> > for -ljansson
> > /usr/bin/ld: cannot find -ljansson
> >
> > Note: there is the same issue with examples/vm_power_manager/Makefile
> >
> > I start thinking it is not reasonnable to enable an optional dependency
> > in our Makefiles. Perhaps the only solution is to use meson...
>
> Ouch, yes if x86_64 pkg-config finds the available library, and it is not available in its i686 form, or other combinations thereof, things become another-level of complex.
On ArchLinux, I have pkg-config-32 for i686 libs.
And for true cross-compilation with sysroot, some environment variables
are required. See this doc:
https://autotools.io/pkgconfig/cross-compiling.html
> Indeed Meson is (good | getting better) at detecting these things, Bruce knows more IIRC :)
>
> @Thomas, for this release I propose to rework to use CONFIG=N as default for Make, and allow packagers to enable if they wish. Is that a good next-step?
Yes it looks reasonnable disabling telemetry by default in make case.
@@ -756,6 +756,11 @@ CONFIG_RTE_LIBRTE_CMDLINE_DEBUG=n
CONFIG_RTE_LIBRTE_HASH=y
CONFIG_RTE_LIBRTE_HASH_DEBUG=n
+#
+# Compile librte_telemetry
+#
+CONFIG_RTE_LIBRTE_TELEMETRY=y
+
#
# Compile librte_efd
#
@@ -159,7 +159,8 @@ The public API headers are grouped by topics:
[hexdump] (@ref rte_hexdump.h),
[debug] (@ref rte_debug.h),
[log] (@ref rte_log.h),
- [errno] (@ref rte_errno.h)
+ [errno] (@ref rte_errno.h),
+ [telemetry] (@ref rte_telemetry.h)
- **misc**:
[EAL config] (@ref rte_eal.h),
@@ -56,6 +56,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \
@TOPDIR@/lib/librte_sched \
@TOPDIR@/lib/librte_security \
@TOPDIR@/lib/librte_table \
+ @TOPDIR@/lib/librte_telemetry \
@TOPDIR@/lib/librte_timer \
@TOPDIR@/lib/librte_vhost
INPUT += @API_EXAMPLES@
@@ -106,6 +106,8 @@ DEPDIRS-librte_gso := librte_eal librte_mbuf librte_ethdev librte_net
DEPDIRS-librte_gso += librte_mempool
DIRS-$(CONFIG_RTE_LIBRTE_BPF) += librte_bpf
DEPDIRS-librte_bpf := librte_eal librte_mempool librte_mbuf librte_ethdev
+DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry
+DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev
ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
new file mode 100644
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_telemetry.a
+
+CFLAGS += -O3
+CFLAGS += -I$(SRCDIR)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+LDLIBS += -lrte_eal -lrte_ethdev
+LDLIBS += -lrte_metrics
+LDLIBS += -lpthread
+
+EXPORT_MAP := rte_telemetry_version.map
+
+LIBABIVER := 1
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c
+
+# export include files
+SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h
+
+include $(RTE_SDK)/mk/rte.lib.mk
new file mode 100644
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+sources = files('rte_telemetry.c')
+headers = files('rte_telemetry.h', 'rte_telemetry_internal.h')
+deps += ['metrics', 'ethdev']
+cflags += '-DALLOW_EXPERIMENTAL_API'
new file mode 100644
@@ -0,0 +1,123 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <unistd.h>
+#include <pthread.h>
+
+#include <rte_eal.h>
+#include <rte_ethdev.h>
+#include <rte_metrics.h>
+#include <rte_option.h>
+
+#include "rte_telemetry.h"
+#include "rte_telemetry_internal.h"
+
+#define SLEEP_TIME 10
+
+static telemetry_impl *static_telemetry;
+
+static int32_t
+rte_telemetry_run(void *userdata)
+{
+ struct telemetry_impl *telemetry = userdata;
+
+ if (telemetry == NULL) {
+ TELEMETRY_LOG_WARN("TELEMETRY could not be initialised");
+ return -1;
+ }
+
+ return 0;
+}
+
+static void
+*rte_telemetry_run_thread_func(void *userdata)
+{
+ int ret;
+ struct telemetry_impl *telemetry = userdata;
+
+ if (telemetry == NULL) {
+ TELEMETRY_LOG_ERR("%s passed a NULL instance", __func__);
+ pthread_exit(0);
+ }
+
+ while (telemetry->thread_status) {
+ rte_telemetry_run(telemetry);
+ ret = usleep(SLEEP_TIME);
+ if (ret < 0)
+ TELEMETRY_LOG_ERR("Calling thread could not be put to sleep");
+ }
+ pthread_exit(0);
+}
+
+int32_t __rte_experimental
+rte_telemetry_init()
+{
+ int ret;
+ pthread_attr_t attr;
+ const char *telemetry_ctrl_thread = "telemetry";
+
+ if (static_telemetry) {
+ TELEMETRY_LOG_WARN("TELEMETRY structure already initialised");
+ return -EALREADY;
+ }
+
+ static_telemetry = calloc(1, sizeof(struct telemetry_impl));
+ if (static_telemetry == NULL) {
+ TELEMETRY_LOG_ERR("Memory could not be allocated");
+ return -ENOMEM;
+ }
+
+ static_telemetry->socket_id = rte_socket_id();
+ rte_metrics_init(static_telemetry->socket_id);
+
+ ret = pthread_attr_init(&attr);
+ if (ret != 0) {
+ TELEMETRY_LOG_ERR("Pthread attribute init failed");
+ return -EPERM;
+ }
+
+ ret = rte_ctrl_thread_create(&static_telemetry->thread_id,
+ telemetry_ctrl_thread, &attr, rte_telemetry_run_thread_func,
+ (void *)static_telemetry);
+ static_telemetry->thread_status = 1;
+
+ if (ret < 0) {
+ ret = rte_telemetry_cleanup();
+ if (ret < 0)
+ TELEMETRY_LOG_ERR("TELEMETRY cleanup failed");
+ return -EPERM;
+ }
+
+ return 0;
+}
+
+int32_t __rte_experimental
+rte_telemetry_cleanup(void)
+{
+ struct telemetry_impl *telemetry = static_telemetry;
+ telemetry->thread_status = 0;
+ pthread_join(telemetry->thread_id, NULL);
+ free(telemetry);
+ static_telemetry = NULL;
+ return 0;
+}
+
+int telemetry_log_level;
+RTE_INIT(rte_telemetry_register);
+
+static struct rte_option option = {
+ .opt_str = "--telemetry",
+ .cb = &rte_telemetry_init,
+ .enabled = 0
+};
+
+static void
+rte_telemetry_register(void)
+{
+ telemetry_log_level = rte_log_register("lib.telemetry");
+ if (telemetry_log_level >= 0)
+ rte_log_set_level(telemetry_log_level, RTE_LOG_ERR);
+
+ rte_option_register(&option);
+}
new file mode 100644
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <stdint.h>
+
+#ifndef _RTE_TELEMETRY_H_
+#define _RTE_TELEMETRY_H_
+
+/**
+ * @file
+ * RTE Telemetry
+ *
+ * The telemetry library provides a method to retrieve statistics from
+ * DPDK by sending a JSON encoded message over a socket. DPDK will send
+ * a JSON encoded response containing telemetry data.
+ ***/
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Initialize Telemetry
+ *
+ * @return
+ * 0 on successful initialisation.
+ * @return
+ * -ENOMEM on memory allocation error
+ * @return
+ * -EPERM on unknown error failure
+ * @return
+ * -EALREADY if Telemetry is already initialised.
+ */
+int32_t __rte_experimental
+rte_telemetry_init(void);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Clean up and free memory.
+ *
+ * @return
+ * 0 on success
+ * @return
+ * -EPERM on failure
+ */
+int32_t __rte_experimental
+rte_telemetry_cleanup(void);
+
+#endif
new file mode 100644
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <rte_log.h>
+
+#ifndef _RTE_TELEMETRY_INTERNAL_H_
+#define _RTE_TELEMETRY_INTERNAL_H_
+
+/* Logging Macros */
+extern int telemetry_log_level;
+
+#define TELEMETRY_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ##level, telemetry_log_level, "%s(): "fmt "\n", \
+ __func__, ##args)
+
+#define TELEMETRY_LOG_ERR(fmt, args...) \
+ TELEMETRY_LOG(ERR, fmt, ## args)
+
+#define TELEMETRY_LOG_WARN(fmt, args...) \
+ TELEMETRY_LOG(WARNING, fmt, ## args)
+
+#define TELEMETRY_LOG_INFO(fmt, args...) \
+ TELEMETRY_LOG(INFO, fmt, ## args)
+
+typedef struct telemetry_impl {
+ pthread_t thread_id;
+ int thread_status;
+ uint32_t socket_id;
+} telemetry_impl;
+
+#endif
new file mode 100644
@@ -0,0 +1,8 @@
+EXPERIMENTAL {
+ global:
+
+ rte_telemetry_cleanup;
+ rte_telemetry_init;
+
+ local: *;
+};
@@ -25,7 +25,7 @@ libraries = [ 'compat', # just a header, used for versioning
# add pkt framework libs which use other libs from above
'port', 'table', 'pipeline',
# flow_classify lib depends on pkt framework table lib
- 'flow_classify', 'bpf']
+ 'flow_classify', 'bpf', 'telemetry']
default_cflags = machine_args
if cc.has_argument('-Wno-format-truncation')
@@ -51,7 +51,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += --whole-archive
_LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl
_LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += --no-whole-archive
_LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats
-_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics
_LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats
_LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats
_LDLIBS-$(CONFIG_RTE_LIBRTE_POWER) += -lrte_power
@@ -80,6 +79,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SECURITY) += -lrte_security
_LDLIBS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += -lrte_compressdev
_LDLIBS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += -lrte_eventdev
_LDLIBS-$(CONFIG_RTE_LIBRTE_RAWDEV) += -lrte_rawdev
+_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics
+_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson
_LDLIBS-$(CONFIG_RTE_LIBRTE_TIMER) += -lrte_timer
_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL) += -lrte_mempool
_LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring
@@ -52,6 +52,12 @@ ifneq ($(CONFIG_RTE_LIBRTE_E1000_PMD),y)
CONFIG_RTE_LIBRTE_E1000_PMD = $(CONFIG_RTE_LIBRTE_EM_PMD)
endif
+JANSSON := $(shell pkg-config --exists jansson; echo $$?)
+ifneq ($(JANSSON),0)
+$(warning Jansson not found, disabling RTE_LIBRTE_TELEMETRY)
+CONFIG_RTE_LIBRTE_TELEMETRY = n
+endif
+
ifeq ($(RTE_ARCH),)
$(error RTE_ARCH is not defined)
endif