From patchwork Thu Aug 23 12:08:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43811 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 95BAF4C95; Thu, 23 Aug 2018 14:08:29 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 4BD2E2BF7 for ; Thu, 23 Aug 2018 14:08:26 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613827" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:24 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:03 +0100 Message-Id: <1535026093-101872-2-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 01/11] telemetry: initial telemetry infrastructure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds the infrastructure and initial code for the telemetry library. A virtual device is used for telemetry, which is included in this patch. To use telemetry, a command-line argument must be used - "--vdev=telemetry". Control threads are used to get CPU cycles for telemetry, which are configured in this patch also. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold --- config/common_base | 5 + drivers/Makefile | 3 + drivers/meson.build | 3 +- drivers/telemetry/Makefile | 8 ++ drivers/telemetry/meson.build | 6 ++ drivers/telemetry/telemetry/Makefile | 26 +++++ drivers/telemetry/telemetry/meson.build | 5 + .../telemetry/rte_pmd_telemetry_version.map | 9 ++ drivers/telemetry/telemetry/telemetry_driver.c | 40 ++++++++ lib/Makefile | 2 + lib/librte_telemetry/Makefile | 26 +++++ lib/librte_telemetry/meson.build | 7 ++ lib/librte_telemetry/rte_telemetry.c | 108 +++++++++++++++++++++ lib/librte_telemetry/rte_telemetry.h | 40 ++++++++ lib/librte_telemetry/rte_telemetry_internal.h | 32 ++++++ lib/librte_telemetry/rte_telemetry_version.map | 6 ++ lib/meson.build | 2 +- mk/rte.app.mk | 2 + 18 files changed, 328 insertions(+), 2 deletions(-) create mode 100644 drivers/telemetry/Makefile create mode 100644 drivers/telemetry/meson.build create mode 100644 drivers/telemetry/telemetry/Makefile create mode 100644 drivers/telemetry/telemetry/meson.build create mode 100644 drivers/telemetry/telemetry/rte_pmd_telemetry_version.map create mode 100644 drivers/telemetry/telemetry/telemetry_driver.c create mode 100644 lib/librte_telemetry/Makefile create mode 100644 lib/librte_telemetry/meson.build create mode 100644 lib/librte_telemetry/rte_telemetry.c create mode 100644 lib/librte_telemetry/rte_telemetry.h create mode 100644 lib/librte_telemetry/rte_telemetry_internal.h create mode 100644 lib/librte_telemetry/rte_telemetry_version.map diff --git a/config/common_base b/config/common_base index 4bcbaf9..682f8bf 100644 --- a/config/common_base +++ b/config/common_base @@ -716,6 +716,11 @@ CONFIG_RTE_LIBRTE_HASH=y CONFIG_RTE_LIBRTE_HASH_DEBUG=n # +# Compile librte_telemetry +# +CONFIG_RTE_LIBRTE_TELEMETRY=y + +# # Compile librte_efd # CONFIG_RTE_LIBRTE_EFD=y diff --git a/drivers/Makefile b/drivers/Makefile index 7566076..2238290 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -21,5 +21,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += event DEPDIRS-event := common bus mempool net DIRS-$(CONFIG_RTE_LIBRTE_RAWDEV) += raw DEPDIRS-raw := common bus mempool net event +DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry +DEPDIRS-telemetry := common bus mempool + include $(RTE_SDK)/mk/rte.subdir.mk diff --git a/drivers/meson.build b/drivers/meson.build index f94e2fe..d38f22f 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -9,7 +9,8 @@ driver_classes = ['common', 'crypto', # depends on common, bus and mempool (net in future). 'compress', # depends on common, bus, mempool. 'event', # depends on common, bus, mempool and net. - 'raw'] # depends on common, bus, mempool, net and event. + 'raw', # depends on common, bus, mempool, net and event. + 'telemetry'] # depends on common, bus, and mempool. default_cflags = machine_args if cc.has_argument('-Wno-format-truncation') diff --git a/drivers/telemetry/Makefile b/drivers/telemetry/Makefile new file mode 100644 index 0000000..86b0342 --- /dev/null +++ b/drivers/telemetry/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry + +include $(RTE_SDK)/mk/rte.subdir.mk diff --git a/drivers/telemetry/meson.build b/drivers/telemetry/meson.build new file mode 100644 index 0000000..b7eb791 --- /dev/null +++ b/drivers/telemetry/meson.build @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +drivers = ['telemetry'] +config_flag_fmt = 'RTE_LIBRTE_@0@_PMD' +driver_name_fmt = 'rte_pmd_@0@' diff --git a/drivers/telemetry/telemetry/Makefile b/drivers/telemetry/telemetry/Makefile new file mode 100644 index 0000000..e9d98f1 --- /dev/null +++ b/drivers/telemetry/telemetry/Makefile @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# library name +LIB = librte_pmd_telemetry.a + +# build flags +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) + +LDLIBS += -lrte_eal -lrte_mbuf +LDLIBS += -lrte_telemetry +LDLIBS += -lrte_bus_vdev + +# versioning export map +EXPORT_MAP := rte_pmd_telemetry_version.map + +# library version +LIBABIVER := 1 + +# library source files +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := telemetry_driver.c + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/telemetry/telemetry/meson.build b/drivers/telemetry/telemetry/meson.build new file mode 100644 index 0000000..baebcab --- /dev/null +++ b/drivers/telemetry/telemetry/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +sources = files('telemetry_driver.c') +deps += ['telemetry', 'bus_pci', 'bus_vdev'] diff --git a/drivers/telemetry/telemetry/rte_pmd_telemetry_version.map b/drivers/telemetry/telemetry/rte_pmd_telemetry_version.map new file mode 100644 index 0000000..a73e0f2 --- /dev/null +++ b/drivers/telemetry/telemetry/rte_pmd_telemetry_version.map @@ -0,0 +1,9 @@ +DPDK_18.05 { + global: + + telemetry_probe; + telemetry_remove; + + local: *; + +}; diff --git a/drivers/telemetry/telemetry/telemetry_driver.c b/drivers/telemetry/telemetry/telemetry_driver.c new file mode 100644 index 0000000..c56f60c --- /dev/null +++ b/drivers/telemetry/telemetry/telemetry_driver.c @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include + +#include +#include +#include +#include + +static int +telemetry_probe(struct rte_vdev_device *vdev) +{ + int ret; + + RTE_SET_USED(vdev); + ret = rte_telemetry_init(rte_socket_id()); + if (ret < 0) { + printf("Error - Telemetry initialisation failed\n"); + return -1; + } + return 0; +} + +static int +telemetry_remove(struct rte_vdev_device *vdev) +{ + const char *name; + name = rte_vdev_device_name(vdev); + printf("Uninitialising the device: %s\n", name); + return 0; +} + +static struct rte_vdev_driver pmd_telemetry_drv = { + .probe = telemetry_probe, + .remove = telemetry_remove +}; + +RTE_PMD_REGISTER_VDEV(telemetry, pmd_telemetry_drv); diff --git a/lib/Makefile b/lib/Makefile index afa604e..8cbd035 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -105,6 +105,8 @@ DEPDIRS-librte_gso := librte_eal librte_mbuf librte_ethdev librte_net DEPDIRS-librte_gso += librte_mempool DIRS-$(CONFIG_RTE_LIBRTE_BPF) += librte_bpf DEPDIRS-librte_bpf := librte_eal librte_mempool librte_mbuf librte_ethdev +DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry +DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y) DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile new file mode 100644 index 0000000..bda3788 --- /dev/null +++ b/lib/librte_telemetry/Makefile @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# library name +LIB = librte_telemetry.a + +CFLAGS += -O3 +CFLAGS += -I$(SRCDIR) +CFLAGS += -DALLOW_EXPERIMENTAL_API + +LDLIBS += -lrte_eal -lrte_ethdev +LDLIBS += -lrte_metrics + +EXPORT_MAP := rte_telemetry_version.map + +LIBABIVER := 1 + +# library source files +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c + +# export include files +SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build new file mode 100644 index 0000000..7716076 --- /dev/null +++ b/lib/librte_telemetry/meson.build @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +sources = files('rte_telemetry.c') +headers = files('rte_telemetry.h', 'rte_telemetry_internal.h') +deps += ['metrics', 'ethdev'] +cflags += '-DALLOW_EXPERIMENTAL_API' diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c new file mode 100644 index 0000000..8d7b0e3 --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry.c @@ -0,0 +1,108 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include + +#include +#include +#include + +#include "rte_telemetry.h" +#include "rte_telemetry_internal.h" + +#define SLEEP_TIME 10 + +static telemetry_impl *static_telemetry; + +static int32_t +rte_telemetry_run(void *userdata) +{ + struct telemetry_impl *telemetry = (struct telemetry_impl *)userdata; + if (!telemetry) { + TELEMETRY_LOG_WARN("Warning - TELEMETRY could not be " + "initialised\n"); + return -1; + } + + return 0; +} + +static void +*rte_telemetry_run_thread_func(void *userdata) +{ + int ret; + struct telemetry_impl *telemetry = (struct telemetry_impl *)userdata; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - %s passed a NULL instance\n", + __func__); + pthread_exit(0); + } + + while (telemetry->thread_status) { + rte_telemetry_run(telemetry); + ret = usleep(SLEEP_TIME); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Calling thread could not be" + " put to sleep\n"); + } + pthread_exit(0); +} + +int32_t +rte_telemetry_init(uint32_t socket_id) +{ + int ret; + pthread_attr_t attr; + const char *telemetry_ctrl_thread = "telemetry"; + + if (static_telemetry) { + TELEMETRY_LOG_WARN("Warning - TELEMETRY structure already " + "initialised\n"); + return -EALREADY; + } + + static_telemetry = calloc(1, sizeof(struct telemetry_impl)); + if (!static_telemetry) { + TELEMETRY_LOG_ERR("Error - Memory could not be allocated\n"); + return -ENOMEM; + } + + static_telemetry->socket_id = socket_id; + rte_metrics_init(static_telemetry->socket_id); + pthread_attr_init(&attr); + ret = rte_ctrl_thread_create(&static_telemetry->thread_id, + telemetry_ctrl_thread, &attr, rte_telemetry_run_thread_func, + (void *)static_telemetry); + static_telemetry->thread_status = 1; + if (ret < 0) { + ret = rte_telemetry_cleanup(); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - TELEMETRY cleanup failed\n"); + return -EPERM; + } + return 0; +} + +int32_t +rte_telemetry_cleanup(void) +{ + struct telemetry_impl *telemetry = static_telemetry; + telemetry->thread_status = 0; + pthread_join(telemetry->thread_id, NULL); + free(telemetry); + static_telemetry = NULL; + return 0; +} + +int telemetry_log_level; +RTE_INIT(rte_telemetry_log_init); + +static void +rte_telemetry_log_init(void) +{ + telemetry_log_level = rte_log_register("lib.telemetry"); + if (telemetry_log_level >= 0) + rte_log_set_level(telemetry_log_level, RTE_LOG_ERR); +} diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h new file mode 100644 index 0000000..b691845 --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include + +#ifndef _RTE_TELEMETRY_H_ +#define _RTE_TELEMETRY_H_ + +/** + * Get the telemetry_impl structure device pointer initialised. + * + * @param socket_id + * Unsigned integer representing the socket id to be used + * for the telemetry structure. + * + * @return + * 0 on successful initialisation. + * @return + * -ENOMEM on memory allocation error + * @return + * -EPERM on unknown error failure + * @return + * -EALREADY if Telemetry is already initialised. + */ +int32_t +rte_telemetry_init(uint32_t socket_id); + +/** + * Clean up and free memory. + * + * @return + * 0 on success + * @return + * -EPERM on failure + */ +int32_t +rte_telemetry_cleanup(void); + +#endif diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h new file mode 100644 index 0000000..4e810a8 --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include + +#ifndef _RTE_TELEMETRY_INTERNAL_H_ +#define _RTE_TELEMETRY_INTERNAL_H_ + +/* Logging Macros */ +extern int telemetry_log_level; + +#define TELEMETRY_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ##level, telemetry_log_level, "%s(): "fmt "\n", \ + __func__, ##args) + +#define TELEMETRY_LOG_ERR(fmt, args...) \ + TELEMETRY_LOG(ERR, fmt, ## args) + +#define TELEMETRY_LOG_WARN(fmt, args...) \ + TELEMETRY_LOG(WARNING, fmt, ## args) + +#define TELEMETRY_LOG_INFO(fmt, args...) \ + TELEMETRY_LOG(INFO, fmt, ## args) + +typedef struct telemetry_impl { + pthread_t thread_id; + int thread_status; + uint32_t socket_id; +} telemetry_impl; + +#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map new file mode 100644 index 0000000..efd437d --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -0,0 +1,6 @@ +DPDK_18.05 { + global: + + rte_telemetry_init; + local: *; +}; diff --git a/lib/meson.build b/lib/meson.build index eb91f10..fc84b2f 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -24,7 +24,7 @@ libraries = [ 'compat', # just a header, used for versioning # add pkt framework libs which use other libs from above 'port', 'table', 'pipeline', # flow_classify lib depends on pkt framework table lib - 'flow_classify', 'bpf'] + 'flow_classify', 'bpf', 'telemetry'] default_cflags = machine_args if cc.has_argument('-Wno-format-truncation') diff --git a/mk/rte.app.mk b/mk/rte.app.mk index de33883..8551191 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -80,6 +80,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SECURITY) += -lrte_security _LDLIBS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += -lrte_compressdev _LDLIBS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += -lrte_eventdev _LDLIBS-$(CONFIG_RTE_LIBRTE_RAWDEV) += -lrte_rawdev +_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_metrics -lrte_telemetry _LDLIBS-$(CONFIG_RTE_LIBRTE_TIMER) += -lrte_timer _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL) += -lrte_mempool _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring @@ -164,6 +165,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SOFTNIC) += -lrte_pmd_softnic endif _LDLIBS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += -lrte_pmd_sfc_efx _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 +_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_pmd_telemetry _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += -lrte_pmd_tap _LDLIBS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += -lrte_pmd_thunderx_nicvf _LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += -lrte_pmd_vdev_netvsc From patchwork Thu Aug 23 12:08:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43812 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 481E04CA7; Thu, 23 Aug 2018 14:08:32 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 069134C8D for ; Thu, 23 Aug 2018 14:08:27 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613835" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:26 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:04 +0100 Message-Id: <1535026093-101872-3-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 02/11] telemetry: add initial connection socket X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds the telemetry UNIX socket. It is used to allow connections from external clients. On the initial connection from a client, ethdev stats are registered in the metrics library, to allow for their retrieval at a later stage. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold --- lib/librte_telemetry/rte_telemetry.c | 205 ++++++++++++++++++++++++++ lib/librte_telemetry/rte_telemetry_internal.h | 4 + 2 files changed, 209 insertions(+) diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 8d7b0e3..f984929 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -3,21 +3,159 @@ */ #include +#include +#include +#include #include #include #include +#include #include "rte_telemetry.h" #include "rte_telemetry_internal.h" #define SLEEP_TIME 10 +#define DEFAULT_DPDK_PATH "/var/run/.rte_telemetry" + +const char *socket_path = DEFAULT_DPDK_PATH; static telemetry_impl *static_telemetry; +int32_t +rte_telemetry_check_port_activity(int port_id) +{ + int pid; + + RTE_ETH_FOREACH_DEV(pid) { + if (pid == port_id) + return 1; + } + TELEMETRY_LOG_ERR("Error - port_id: %d is invalid, not active\n", + port_id); + return 0; +} + +static int32_t +rte_telemetry_reg_ethdev_to_metrics(uint16_t port_id) +{ + int ret, num_xstats, start_index, i; + struct rte_eth_xstat *eth_xstats; + + if (!rte_eth_dev_is_valid_port(port_id)) { + TELEMETRY_LOG_ERR("Error - port_id: %d is invalid\n", port_id); + return -EINVAL; + } + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + TELEMETRY_LOG_ERR("Error - rte_eth_xstats_get(%u) failed:" + " %d\n", port_id, num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + TELEMETRY_LOG_ERR("Error - Failed to malloc memory for" + " xstats\n"); + return -ENOMEM; + } + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + TELEMETRY_LOG_ERR("Error - rte_eth_xstats_get(%u) len%i" + " failed: %d\n", port_id, num_xstats, ret); + return -EPERM; + } + struct rte_eth_xstat_name *eth_xstats_names; + eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * + num_xstats); + if (eth_xstats_names == NULL) { + free(eth_xstats); + TELEMETRY_LOG_ERR("Error - Failed to malloc memory for" + " xstats_names\n"); + return -ENOMEM; + } + ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, + num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + free(eth_xstats_names); + TELEMETRY_LOG_ERR("Error - rte_eth_xstats_get_names(%u)" + " len%i failed: %d\n", port_id, num_xstats, + ret); + return -EPERM; + } + const char *xstats_names[num_xstats]; + + for (i = 0; i < num_xstats; i++) + xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; + + start_index = rte_metrics_reg_names(xstats_names, num_xstats); + + if (start_index < 0) { + TELEMETRY_LOG_ERR("Error - rte_metrics_reg_names failed -" + " metrics may already be registered\n"); + free(eth_xstats); + free(eth_xstats_names); + return -1; + } + free(eth_xstats_names); + free(eth_xstats); + return start_index; +} + +static int32_t +rte_telemetry_initial_accept(struct telemetry_impl *telemetry) +{ + int pid; + + RTE_ETH_FOREACH_DEV(pid) { + telemetry->reg_index = + rte_telemetry_reg_ethdev_to_metrics(pid); + break; + } + + if (telemetry->reg_index < 0) { + TELEMETRY_LOG_ERR("Error - failed to register ethdev " + "metrics\n"); + return -1; + } + telemetry->metrics_register_done = 1; + + return 0; +} + +static int32_t +rte_telemetry_accept_new_client(struct telemetry_impl *telemetry) +{ + int ret; + if (telemetry->accept_fd == 0 || telemetry->accept_fd == -1) { + ret = listen(telemetry->server_fd, 1); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Listening error with " + "server fd\n"); + return -1; + } + telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); + + if (telemetry->accept_fd > 0 && + telemetry->metrics_register_done == 0) { + ret = rte_telemetry_initial_accept(telemetry); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Failed to run " + "initial configurations/tests\n"); + return -1; + } + } + } + return 0; +} + static int32_t rte_telemetry_run(void *userdata) { + int ret; struct telemetry_impl *telemetry = (struct telemetry_impl *)userdata; if (!telemetry) { TELEMETRY_LOG_WARN("Warning - TELEMETRY could not be " @@ -25,6 +163,12 @@ rte_telemetry_run(void *userdata) return -1; } + ret = rte_telemetry_accept_new_client(telemetry); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Accept and read new client " + "failed\n"); + return -1; + } return 0; } @@ -50,6 +194,51 @@ static void pthread_exit(0); } +static int32_t +rte_telemetry_set_socket_nonblock(int fd) +{ + if (fd < 0) { + TELEMETRY_LOG_ERR("Error - Invalid fd provided\n"); + return -1; + } + + int flags = fcntl(fd, F_GETFL, 0); + if (flags < 0) + flags = 0; + return fcntl(fd, F_SETFL, flags | O_NONBLOCK); +} + +static int32_t +rte_telemetry_create_socket(struct telemetry_impl *telemetry) +{ + int ret; + + if (!telemetry) + return -1; + + telemetry->server_fd = socket(AF_UNIX, SOCK_STREAM, 0); + if (telemetry->server_fd == -1) { + TELEMETRY_LOG_ERR("Error - Failed to open socket\n"); + return -1; + } + ret = rte_telemetry_set_socket_nonblock(telemetry->server_fd); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not set socket to NONBLOCK\n"); + return -1; + } + struct sockaddr_un addr = {0}; + addr.sun_family = AF_UNIX; + strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); + unlink(socket_path); + + if (bind(telemetry->server_fd, (struct sockaddr *)&addr, + sizeof(addr)) < 0) { + TELEMETRY_LOG_ERR("Error - Socket binding error\n"); + return -1; + } + return 0; +} + int32_t rte_telemetry_init(uint32_t socket_id) { @@ -71,6 +260,14 @@ rte_telemetry_init(uint32_t socket_id) static_telemetry->socket_id = socket_id; rte_metrics_init(static_telemetry->socket_id); + ret = rte_telemetry_create_socket(static_telemetry); + if (ret < 0) { + ret = rte_telemetry_cleanup(); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - TELEMETRY cleanup failed\n"); + return -EPERM; + } + pthread_attr_init(&attr); ret = rte_ctrl_thread_create(&static_telemetry->thread_id, telemetry_ctrl_thread, &attr, rte_telemetry_run_thread_func, @@ -88,7 +285,15 @@ rte_telemetry_init(uint32_t socket_id) int32_t rte_telemetry_cleanup(void) { + int ret; struct telemetry_impl *telemetry = static_telemetry; + + ret = close(telemetry->server_fd); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Close TELEMETRY socket failed\n"); + free(telemetry); + return -EPERM; + } telemetry->thread_status = 0; pthread_join(telemetry->thread_id, NULL); free(telemetry); diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index 4e810a8..569d56a 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -24,9 +24,13 @@ extern int telemetry_log_level; TELEMETRY_LOG(INFO, fmt, ## args) typedef struct telemetry_impl { + int accept_fd; + int server_fd; pthread_t thread_id; int thread_status; uint32_t socket_id; + int reg_index; + int metrics_register_done; } telemetry_impl; #endif From patchwork Thu Aug 23 12:08:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43813 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5A3814CA1; Thu, 23 Aug 2018 14:08:34 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id E0AE54C8D for ; Thu, 23 Aug 2018 14:08:28 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613840" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:27 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:05 +0100 Message-Id: <1535026093-101872-4-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 03/11] telemetry: add client feature and sockets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch introduces clients to the telemetry API. When a client makes a connection through the initial telemetry socket, they can send a message through the socket to be parsed. Register messages are expected through this socket, to enable clients to register and have a client socket setup for future communications. A TAILQ is used to store all clients information. Using this, the client sockets are polled for messages, which will later be parsed and dealt with accordingly. Functionality that make use of the client sockets were introduced in this patch also, such as writing to client sockets, and sending error responses. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold --- lib/librte_telemetry/meson.build | 2 + lib/librte_telemetry/rte_telemetry.c | 358 ++++++++++++++++++++++++++ lib/librte_telemetry/rte_telemetry_internal.h | 25 ++ mk/rte.app.mk | 2 +- 4 files changed, 386 insertions(+), 1 deletion(-) diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 7716076..0ccfa36 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -5,3 +5,5 @@ sources = files('rte_telemetry.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h') deps += ['metrics', 'ethdev'] cflags += '-DALLOW_EXPERIMENTAL_API' +jansson = cc.find_library('jansson', required: true) +ext_deps += jansson diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index f984929..e9dd022 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -15,6 +16,8 @@ #include "rte_telemetry.h" #include "rte_telemetry_internal.h" +#define BUF_SIZE 1024 +#define ACTION_POST 1 #define SLEEP_TIME 10 #define DEFAULT_DPDK_PATH "/var/run/.rte_telemetry" @@ -36,6 +39,88 @@ rte_telemetry_check_port_activity(int port_id) return 0; } +int32_t +rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, + const char *json_string) +{ + int ret; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error, could not initialise " + "TELEMETRY_API\n"); + return -1; + } + if (!telemetry->request_client) { + TELEMETRY_LOG_ERR("Error - No client has been chosen to" + " write to\n"); + return -1; + } + if (!json_string) { + TELEMETRY_LOG_ERR("Error - Invalid JSON string!\n"); + return -1; + } + ret = send(telemetry->request_client->fd, + json_string, strlen(json_string), 0); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Failed to write to socket for " + "client: %s\n", telemetry->request_client->file_path); + return -1; + } + return 0; +} + +int32_t +rte_telemetry_send_error_response(struct telemetry_impl *telemetry, + int error_type) +{ + int ret; + const char *status_code, *json_buffer; + json_t *root; + + if (error_type == -EPERM) + status_code = "Status Error: Unknown"; + else if (error_type == -EINVAL) + status_code = "Status Error: Invalid Argument 404"; + else if (error_type == -ENOMEM) + status_code = "Status Error: Memory Allocation Error"; + else { + TELEMETRY_LOG_ERR("Error - invalid error type\n"); + return -EINVAL; + } + + root = json_object(); + + if (!root) { + TELEMETRY_LOG_ERR("Error - Could not create root JSON " + "object\n"); + return -EPERM; + } + + ret = json_object_set_new(root, "status_code", + json_string(status_code)); + + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Status code field cannot be set\n"); + return -EPERM; + } + + ret = json_object_set_new(root, "data", json_null()); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Data field cannot be set\n"); + return -EPERM; + } + + json_buffer = json_dumps(root, JSON_INDENT(2)); + json_decref(root); + + ret = rte_telemetry_write_to_socket(telemetry, json_buffer); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not write to socket\n"); + return -EPERM; + } + return 0; +} + static int32_t rte_telemetry_reg_ethdev_to_metrics(uint16_t port_id) { @@ -127,6 +212,31 @@ rte_telemetry_initial_accept(struct telemetry_impl *telemetry) } static int32_t +rte_telemetry_read_client(struct telemetry_impl *telemetry) +{ + char buf[BUF_SIZE]; + int buffer_read = 0; + errno = 0; + + buffer_read = read(telemetry->accept_fd, buf, BUF_SIZE-1); + buf[buffer_read] = '\0'; + if (buffer_read == -1) { + TELEMETRY_LOG_ERR("Error - Read error\n"); + return -1; + } else if (buffer_read == 0) { + close(telemetry->accept_fd); + telemetry->accept_fd = 0; + } else { + int ret = rte_telemetry_parse_client_message(telemetry, buf); + if (ret < 0) + TELEMETRY_LOG_WARN("Warning - parse message failed\n"); + close(telemetry->accept_fd); + telemetry->accept_fd = 0; + } + return 0; +} + +static int32_t rte_telemetry_accept_new_client(struct telemetry_impl *telemetry) { int ret; @@ -148,6 +258,28 @@ rte_telemetry_accept_new_client(struct telemetry_impl *telemetry) return -1; } } + } else { + ret = rte_telemetry_read_client(telemetry); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - failed to read socket " + "buffer\n"); + return -1; + } + } + return 0; +} + +static int32_t +rte_telemetry_read_client_sockets(struct telemetry_impl *telemetry) +{ + telemetry_client *client; + TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { + char client_buf[BUF_SIZE]; + int bytes = read(client->fd, client_buf, BUF_SIZE-1); + client_buf[bytes] = '\0'; + if (bytes > 0) { + telemetry->request_client = client; + } } return 0; } @@ -169,6 +301,12 @@ rte_telemetry_run(void *userdata) "failed\n"); return -1; } + + ret = rte_telemetry_read_client_sockets(telemetry); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - client socket read failed\n"); + return -1; + } return 0; } @@ -267,6 +405,7 @@ rte_telemetry_init(uint32_t socket_id) TELEMETRY_LOG_ERR("Error - TELEMETRY cleanup failed\n"); return -EPERM; } + TAILQ_INIT(&static_telemetry->client_list_head); pthread_attr_init(&attr); ret = rte_ctrl_thread_create(&static_telemetry->thread_id, @@ -282,12 +421,38 @@ rte_telemetry_init(uint32_t socket_id) return 0; } +static int32_t +rte_telemetry_client_cleanup(struct telemetry_client *client) +{ + int ret; + + ret = close(client->fd); + free(client->file_path); + free(client); + + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Close client socket failed\n"); + return -EPERM; + } + return 0; +} + int32_t rte_telemetry_cleanup(void) { int ret; struct telemetry_impl *telemetry = static_telemetry; + telemetry_client *client, *temp_client; + TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, + temp_client) { + TAILQ_REMOVE(&telemetry->client_list_head, client, client_list); + ret = rte_telemetry_client_cleanup(client); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Client cleanup failed\n"); + return -EPERM; + } + } ret = close(telemetry->server_fd); if (ret < 0) { TELEMETRY_LOG_ERR("Error - Close TELEMETRY socket failed\n"); @@ -301,6 +466,199 @@ rte_telemetry_cleanup(void) return 0; } +int32_t +rte_telemetry_unregister_client(struct telemetry_impl *telemetry, + const char *client_path) +{ + int ret; + + if (!telemetry) { + TELEMETRY_LOG_WARN("Warning - TELEMETRY is not initialised\n"); + return -ENODEV; + } + + if (!client_path) { + TELEMETRY_LOG_ERR("Error - Invalid client path\n"); + goto einval_fail; + } + + if (TAILQ_EMPTY(&telemetry->client_list_head)) { + TELEMETRY_LOG_ERR("Error - there are no clients currently " + "registered\n"); + return -EPERM; + } + telemetry_client *client, *temp_client; + TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, + temp_client) { + if (strcmp(client_path, client->file_path) == 0) { + TAILQ_REMOVE(&telemetry->client_list_head, client, + client_list); + ret = rte_telemetry_client_cleanup(client); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Client cleanup " + "failed\n"); + return -EPERM; + } + return 0; + } + } + TELEMETRY_LOG_WARN("Warning - couldn't find client, possibly not " + "registered yet.\n"); + return -1; + +einval_fail: + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -EINVAL; +} + +int32_t +rte_telemetry_register_client(struct telemetry_impl *telemetry, + const char *client_path) +{ + int ret, fd; + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - could not initialize " + "TELEMETRY API\n"); + return -ENODEV; + } + + if (!client_path) { + TELEMETRY_LOG_ERR("Error - Invalid client path\n"); + return -EINVAL; + } + + telemetry_client *client; + TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { + if (strcmp(client_path, client->file_path) == 0) { + TELEMETRY_LOG_WARN("Warning - '%s' already " + "registered\n", client_path); + return -EINVAL; + } + } + fd = socket(AF_UNIX, SOCK_STREAM, 0); + if (fd == -1) { + TELEMETRY_LOG_ERR("Error - Client socket error\n"); + return -EACCES; + } + ret = rte_telemetry_set_socket_nonblock(fd); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not set socket to NONBLOCK\n"); + return -EPERM; + } + + struct sockaddr_un addrs = {0}; + addrs.sun_family = AF_UNIX; + strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); + telemetry_client *new_client = + (telemetry_client *)malloc(sizeof(telemetry_client)); + new_client->file_path = strdup(client_path); + new_client->fd = fd; + + if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { + TELEMETRY_LOG_ERR("Error - TELEMETRY client connect to %s " + "didn't work\n", client_path); + ret = rte_telemetry_client_cleanup(new_client); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Client cleanup failed\n"); + return -EPERM; + } + return -EINVAL; + } + TAILQ_INSERT_HEAD(&telemetry->client_list_head, new_client, + client_list); + + return 0; +} + +int32_t +rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf) +{ + int ret; + json_error_t error; + json_t *root = json_loads(buf, 0, &error); + + if (!root) { + TELEMETRY_LOG_WARN("Warning - Could not load JSON object from " + "data passed in : %s\n", error.text); + goto fail; + } else if (!json_is_object(root)) { + TELEMETRY_LOG_WARN("Warning - JSON Request is not a JSON " + "object\n"); + json_decref(root); + goto fail; + } + + json_t *action = json_object_get(root, "action"); + if (!action) { + TELEMETRY_LOG_WARN("Warning - Request does not have action " + "field\n"); + goto fail; + } else if (!json_is_integer(action)) { + TELEMETRY_LOG_WARN("Warning - Action value is not an " + "integer\n"); + goto fail; + } + + json_t *command = json_object_get(root, "command"); + if (!command) { + TELEMETRY_LOG_WARN("Warning - Request does not have command " + "field\n"); + goto fail; + } else if (!json_is_string(command)) { + TELEMETRY_LOG_WARN("Warning - Command value is not a string\n"); + goto fail; + } + + int action_int = json_integer_value(action); + if (action_int != ACTION_POST) { + TELEMETRY_LOG_WARN("Warning - Invalid action code\n"); + goto fail; + } + + const char *command_string = json_string_value(command); + if (strcmp(command_string, "clients") != 0) { + TELEMETRY_LOG_WARN("Warning - Invalid command\n"); + goto fail; + } + + json_t *data = json_object_get(root, "data"); + if (!data) { + TELEMETRY_LOG_WARN("Warning - Request does not have data " + "field\n"); + goto fail; + } + + json_t *client_path = json_object_get(data, "client_path"); + if (!client_path) { + TELEMETRY_LOG_WARN("Warning - Request does not have " + "client_path field\n"); + goto fail; + } + if (!json_is_string(client_path)) { + TELEMETRY_LOG_WARN("Warning - client_path value is not a " + "string\n"); + goto fail; + } + + const char *client_path_string = json_string_value(client_path); + + ret = rte_telemetry_register_client(telemetry, client_path_string); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - could not register client\n"); + telemetry->register_fail_count++; + goto fail; + } + + return 0; + +fail: + TELEMETRY_LOG_WARN("Warning - Client attempted to register with " + "invalid message\n"); + return -1; +} + int telemetry_log_level; RTE_INIT(rte_telemetry_log_init); diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index 569d56a..e3292cf 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -3,6 +3,7 @@ */ #include +#include #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ @@ -23,6 +24,12 @@ extern int telemetry_log_level; #define TELEMETRY_LOG_INFO(fmt, args...) \ TELEMETRY_LOG(INFO, fmt, ## args) +typedef struct telemetry_client { + char *file_path; + int fd; + TAILQ_ENTRY(telemetry_client) client_list; +} telemetry_client; + typedef struct telemetry_impl { int accept_fd; int server_fd; @@ -31,6 +38,24 @@ typedef struct telemetry_impl { uint32_t socket_id; int reg_index; int metrics_register_done; + TAILQ_HEAD(, telemetry_client) client_list_head; + struct telemetry_client *request_client; + int register_fail_count; } telemetry_impl; +int32_t +rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); + +int32_t +rte_telemetry_send_error_response(struct telemetry_impl *telemetry, + int error_type); + +int32_t +rte_telemetry_register_client(struct telemetry_impl *telemetry, + const char *client_path); + +int32_t +rte_telemetry_unregister_client(struct telemetry_impl *telemetry, + const char *client_path); + #endif diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 8551191..1963812 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -80,7 +80,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SECURITY) += -lrte_security _LDLIBS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += -lrte_compressdev _LDLIBS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += -lrte_eventdev _LDLIBS-$(CONFIG_RTE_LIBRTE_RAWDEV) += -lrte_rawdev -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_metrics -lrte_telemetry +_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_metrics -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_TIMER) += -lrte_timer _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL) += -lrte_mempool _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring From patchwork Thu Aug 23 12:08:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43814 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1C0534CBB; Thu, 23 Aug 2018 14:08:36 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 8F7F04C9C for ; Thu, 23 Aug 2018 14:08:30 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613847" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:28 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:06 +0100 Message-Id: <1535026093-101872-5-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 04/11] telemetry: add parser for client socket messages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds the parser file. This is used to parse any messages that are received on any of the client sockets. Currently, the unregister functionality works using the parser. Functionality relating to getting statistic values for certain ports will be added in a subsequent patch, however the parsing involved for that command is added in this patch. Some of the parser code included is in preparation for future functionality, that is not implemented yet in this patchset. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold --- lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 4 +- lib/librte_telemetry/rte_telemetry.c | 9 + lib/librte_telemetry/rte_telemetry_internal.h | 3 + lib/librte_telemetry/rte_telemetry_parser.c | 585 ++++++++++++++++++++++++++ lib/librte_telemetry/rte_telemetry_parser.h | 13 + 6 files changed, 613 insertions(+), 2 deletions(-) create mode 100644 lib/librte_telemetry/rte_telemetry_parser.c create mode 100644 lib/librte_telemetry/rte_telemetry_parser.h diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index bda3788..df8fdd9 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -19,6 +19,7 @@ LIBABIVER := 1 # library source files SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 0ccfa36..7450f96 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -1,8 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation -sources = files('rte_telemetry.c') -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h') +sources = files('rte_telemetry.c', 'rte_telemetry_parser.c') +headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') deps += ['metrics', 'ethdev'] cflags += '-DALLOW_EXPERIMENTAL_API' jansson = cc.find_library('jansson', required: true) diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index e9dd022..c6c6612 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -15,6 +15,7 @@ #include "rte_telemetry.h" #include "rte_telemetry_internal.h" +#include "rte_telemetry_parser.h" #define BUF_SIZE 1024 #define ACTION_POST 1 @@ -272,6 +273,8 @@ rte_telemetry_accept_new_client(struct telemetry_impl *telemetry) static int32_t rte_telemetry_read_client_sockets(struct telemetry_impl *telemetry) { + int ret; + telemetry_client *client; TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { char client_buf[BUF_SIZE]; @@ -279,6 +282,12 @@ rte_telemetry_read_client_sockets(struct telemetry_impl *telemetry) client_buf[bytes] = '\0'; if (bytes > 0) { telemetry->request_client = client; + ret = rte_telemetry_parse(telemetry, client_buf); + if (ret < 0) { + TELEMETRY_LOG_WARN("Warning - Parse socket " + "input failed: %i\n", ret); + return -1; + } } } return 0; diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index e3292cf..b057794 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -58,4 +58,7 @@ int32_t rte_telemetry_unregister_client(struct telemetry_impl *telemetry, const char *client_path); +int32_t +rte_telemetry_check_port_activity(int port_id); + #endif diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c new file mode 100644 index 0000000..571c991 --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -0,0 +1,585 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "rte_telemetry_internal.h" + +#define ACTION_GET 0 +#define ACTION_DELETE 2 + +struct command { + char *command_text; + int (*comm_func_ptr)(struct telemetry_impl *, int, json_t *); +} command; + +static int32_t +rte_telemetry_command_clients(struct telemetry_impl *telemetry, int action, + json_t *data) +{ + int ret; + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Invalid telemetry argument\n"); + return -1; + } + + if (action != ACTION_DELETE) { + TELEMETRY_LOG_WARN("Warning - Invalid action for this " + "command\n"); + goto einval_fail; + } + + if (!json_is_object(data)) { + TELEMETRY_LOG_WARN("Warning - Invalid data provided for this " + "command\n"); + goto einval_fail; + } + + json_t *client_path = json_object_get(data, "client_path"); + if (!json_is_string(client_path)) { + TELEMETRY_LOG_WARN("Warning - Command value is not a string\n"); + goto einval_fail; + } + + const char *client_path_string = json_string_value(client_path); + + ret = rte_telemetry_unregister_client(telemetry, client_path_string); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - could not unregister client\n"); + goto einval_fail; + } + + return 0; + +einval_fail: + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; +} + +static int32_t +rte_telemetry_command_ports(struct telemetry_impl *telemetry, int action, + json_t *data) +{ + int ret; + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Invalid telemetry argument\n"); + return -1; + } + + if (!json_is_null(data)) { + TELEMETRY_LOG_WARN("Warning - Data should be NULL JSON object " + "for 'ports' command\n"); + goto einval_fail; + } + + if (action != ACTION_GET) { + TELEMETRY_LOG_WARN("Warning - Invalid action for this " + "command\n"); + goto einval_fail; + } + + return 0; + +einval_fail: + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; +} + +static int32_t +rte_telemetry_command_ports_details(struct telemetry_impl *telemetry, + int action, json_t *data) +{ + int ret; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Invalid telemetry argument\n"); + return -1; + } + + if (action != ACTION_GET) { + TELEMETRY_LOG_WARN("Warning - Invalid action for this " + "command\n"); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + if (!json_is_object(data)) { + TELEMETRY_LOG_WARN("Warning - Invalid data provided for this " + "command\n"); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + json_t *port_ids_json = json_object_get(data, "ports"); + if (!json_is_array(port_ids_json)) { + TELEMETRY_LOG_WARN("Warning - Invalid Port ID array\n"); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + uint64_t num_port_ids = json_array_size(port_ids_json); + int port_ids[num_port_ids]; + RTE_SET_USED(port_ids); + size_t index; + json_t *value; + + json_array_foreach(port_ids_json, index, value) { + if (!json_is_integer(value)) { + TELEMETRY_LOG_WARN("Warning - Port ID given is " + "invalid\n"); + ret = rte_telemetry_send_error_response(telemetry, + -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send " + "error\n"); + return -1; + } + port_ids[index] = json_integer_value(value); + } + + return 0; +} + +static int32_t +rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, + json_t *data) +{ + int ret; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Invalid telemetry argument\n"); + return -1; + } + + if (!json_is_null(data)) { + TELEMETRY_LOG_WARN("Warning - Data should be NULL JSON object " + "for 'port_stats' command\n"); + goto einval_fail; + } + + if (action != ACTION_GET) { + TELEMETRY_LOG_WARN("Warning - Invalid action for this " + "command\n"); + goto einval_fail; + } + + return 0; + +einval_fail: + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; +} + +static int32_t +rte_telemetry_stat_names_to_ids(struct telemetry_impl *telemetry, + const char * const *stat_names, uint32_t *stat_ids, + uint64_t num_stat_names) +{ + struct rte_metric_name *names; + int ret; + + if (stat_names == NULL) { + TELEMETRY_LOG_WARN("Warning - Invalid stat_names argument\n"); + goto einval_fail; + } + + if (num_stat_names <= 0) { + TELEMETRY_LOG_WARN("Warning - Invalid num_stat_names " + "argument\n"); + goto einval_fail; + } + + int num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + TELEMETRY_LOG_ERR("Error - Cannot get metrics count\n"); + goto eperm_fail; + } else if (num_metrics == 0) { + TELEMETRY_LOG_WARN("Warning - No metrics have been " + "registered\n"); + goto eperm_fail; + } + + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (names == NULL) { + TELEMETRY_LOG_ERR("Error - Cannot allocate memory for names\n"); + ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + TELEMETRY_LOG_ERR("Error - Cannot get metrics names\n"); + free(names); + goto eperm_fail; + } + + uint32_t i, k; + k = 0; + for (i = 0; i < (uint32_t)num_stat_names; i++) { + uint32_t j; + for (j = 0; j < (uint32_t)num_metrics; j++) { + if (strcmp(stat_names[i], names[j].name) == 0) { + stat_ids[k] = j; + k++; + break; + } + } + } + + if (k != num_stat_names) { + TELEMETRY_LOG_WARN("Warning - Invalid stat names provided\n"); + free(names); + goto einval_fail; + } + + return 0; + +einval_fail: + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + +eperm_fail: + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; +} + +int32_t +rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, + int action, json_t *data) +{ + int ret, num_metrics; + struct rte_metric_name *names; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Invalid telemetry argument\n"); + return -1; + } + + if (action != ACTION_GET) { + TELEMETRY_LOG_WARN("Warning - Invalid action for this command\n"); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + if (json_is_object(data)) { + TELEMETRY_LOG_WARN("Warning - Invalid data provided for this command\n"); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + TELEMETRY_LOG_ERR("Error - Cannot get metrics count\n"); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } else if (num_metrics == 0) { + TELEMETRY_LOG_ERR("Error - No metrics to display (none have been registered)\n"); + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (!names) { + TELEMETRY_LOG_ERR("Error - Cannot allocate memory\n"); + ret = rte_telemetry_send_error_response(telemetry, + -ENOMEM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + uint64_t num_port_ids = 0; + const char *stat_names[num_metrics]; + uint32_t stat_ids[num_metrics]; + int p; + + RTE_ETH_FOREACH_DEV(p) { + num_port_ids++; + } + if (!num_port_ids) { + TELEMETRY_LOG_WARN("Warning - No active ports\n"); + ret = rte_telemetry_send_error_response(telemetry, + -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + goto fail; + } + + ret = rte_metrics_get_names(names, num_metrics); + int i; + for (i = 0; i < num_metrics; i++) + stat_names[i] = names[i].name; + + ret = rte_telemetry_stat_names_to_ids(telemetry, stat_names, stat_ids, + num_metrics); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not convert stat names to IDs\n"); + goto fail; + } + return 0; + +fail: + free(names); + return -1; +} + +int32_t +rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl + *telemetry, int action, json_t *data) +{ + int ret; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Invalid telemetry argument\n"); + return -1; + } + + if (action != ACTION_GET) { + TELEMETRY_LOG_WARN("Warning - Invalid action for this " + "command\n"); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + if (!json_is_object(data)) { + TELEMETRY_LOG_WARN("Warning - Invalid data provided for this " + "command\n"); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + json_t *port_ids_json = json_object_get(data, "ports"); + json_t *stat_names_json = json_object_get(data, "stats"); + if (!json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { + TELEMETRY_LOG_WARN("Warning - Invalid input data array(s)\n"); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + uint64_t num_port_ids = json_array_size(port_ids_json); + uint32_t port_ids[num_port_ids]; + size_t index; + json_t *value; + + json_array_foreach(port_ids_json, index, value) { + if (!json_is_integer(value)) { + TELEMETRY_LOG_WARN("Warning - Port ID given is not " + "valid\n"); + ret = rte_telemetry_send_error_response(telemetry, + -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send " + "error\n"); + return -1; + } + port_ids[index] = json_integer_value(value); + ret = rte_telemetry_check_port_activity(port_ids[index]); + if (ret < 1) { + ret = rte_telemetry_send_error_response(telemetry, + -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send " + "error\n"); + return -1; + } + } + + uint64_t num_stat_names = json_array_size(stat_names_json); + const char *stat_names[num_stat_names]; + + json_array_foreach(stat_names_json, index, value) { + if (!json_is_string(value)) { + TELEMETRY_LOG_WARN("Warning - Stat Name given is not a " + "string\n"); + ret = rte_telemetry_send_error_response(telemetry, + -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send " + "error\n"); + return -1; + } + stat_names[index] = json_string_value(value); + } + + uint32_t stat_ids[num_stat_names]; + ret = rte_telemetry_stat_names_to_ids(telemetry, stat_names, stat_ids, + num_stat_names); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not convert stat names to " + "IDs\n"); + return -1; + } + return 0; +} + +static int32_t +rte_telemetry_parse_command(struct telemetry_impl *telemetry, int action, + const char *command, json_t *data) +{ + int ret; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Invalid telemetry argument\n"); + return -1; + } + + struct command commands[] = { + {.command_text = "clients", + .comm_func_ptr = &rte_telemetry_command_clients}, + {.command_text = "ports", + .comm_func_ptr = &rte_telemetry_command_ports}, + {.command_text = "ports_details", + .comm_func_ptr = &rte_telemetry_command_ports_details}, + {.command_text = "port_stats", + .comm_func_ptr = &rte_telemetry_command_port_stats}, + {.command_text = "ports_stats_values_by_name", + .comm_func_ptr = + &rte_telemetry_command_ports_stats_values_by_name}, + {.command_text = "ports_all_stat_values", + .comm_func_ptr = + &rte_telemetry_command_ports_all_stat_values} + }; + + const uint32_t num_commands = sizeof(commands)/sizeof(struct command); + uint32_t i; + + for (i = 0; i < num_commands; i++) { + if (strcmp(command, commands[i].command_text) == 0) { + int ret = commands[i].comm_func_ptr(telemetry, action, + data); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Command Function " + "for %s failed\n", + commands[i].command_text); + return -1; + } + return 0; + } + } + TELEMETRY_LOG_WARN("Warning - \"%s\" command not found\n", command); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; +} + +int32_t +rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data) +{ + int ret; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Invalid telemetry argument\n"); + return -1; + } + + json_error_t error; + json_t *root = json_loads(socket_rx_data, 0, &error); + if (!root) { + TELEMETRY_LOG_WARN("Warning - Could not load JSON object from " + "data passed in : %s\n", error.text); + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -EPERM; + } else if (!json_is_object(root)) { + TELEMETRY_LOG_WARN("Warning - JSON Request is not a JSON " + "object\n"); + json_decref(root); + goto einval_fail; + } + + json_t *action = json_object_get(root, "action"); + if (!action) { + TELEMETRY_LOG_WARN("Warning - Request does not have action " + "field\n"); + goto einval_fail; + } else if (!json_is_integer(action)) { + TELEMETRY_LOG_WARN("Warning - Action value is not an " + "integer\n"); + goto einval_fail; + } + + json_t *command = json_object_get(root, "command"); + if (!command) { + TELEMETRY_LOG_WARN("Warning - Request does not have command " + "field\n"); + goto einval_fail; + } else if (!json_is_string(command)) { + TELEMETRY_LOG_WARN("Warning - Command value is not a string\n"); + goto einval_fail; + } + + int action_int = json_integer_value(action); + if (action_int != ACTION_GET && action_int != ACTION_DELETE) { + TELEMETRY_LOG_WARN("Warning - Invalid action code\n"); + goto einval_fail; + } + + const char *command_string = json_string_value(command); + json_t *data = json_object_get(root, "data"); + if (!data) { + TELEMETRY_LOG_WARN("Warning - Request does not have data " + "field\n"); + goto einval_fail; + } + + ret = rte_telemetry_parse_command(telemetry, action_int, command_string, + data); + if (ret < 0) { + TELEMETRY_LOG_WARN("Warning - Could not parse command\n"); + return -EINVAL; + } + + return 0; + +einval_fail: + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -EPERM; + } + return -EINVAL; +} diff --git a/lib/librte_telemetry/rte_telemetry_parser.h b/lib/librte_telemetry/rte_telemetry_parser.h new file mode 100644 index 0000000..63e633d --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_parser.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include "rte_telemetry_internal.h" + +#ifndef _RTE_TELEMETRY_PARSER_H_ +#define _RTE_TELEMETRY_PARSER_H_ + +int32_t +rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data); + +#endif From patchwork Thu Aug 23 12:08:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43815 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B097A4F98; Thu, 23 Aug 2018 14:08:38 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 3C6BA2C4F for ; Thu, 23 Aug 2018 14:08:32 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613854" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:30 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:07 +0100 Message-Id: <1535026093-101872-6-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 05/11] telemetry: update metrics before sending stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds functionality to update the statistics in the metrics library with values from the ethdev stats. Values need to be updated before they are encoded into a JSON message and sent to the client that requested them. The JSON encoding will be added in a subsequent patch. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold --- lib/librte_telemetry/rte_telemetry.c | 128 ++++++++++++++++++++++++++ lib/librte_telemetry/rte_telemetry_internal.h | 4 + lib/librte_telemetry/rte_telemetry_parser.c | 19 ++++ 3 files changed, 151 insertions(+) diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index c6c6612..bd09374 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -40,6 +40,74 @@ rte_telemetry_check_port_activity(int port_id) return 0; } +static int32_t +rte_telemetry_update_metrics_ethdev(struct telemetry_impl *telemetry, + uint16_t port_id, int reg_start_index) +{ + int ret, num_xstats, i; + struct rte_eth_xstat *eth_xstats; + + if (!rte_eth_dev_is_valid_port(port_id)) { + TELEMETRY_LOG_ERR("Error - port_id: %d is invalid\n", port_id); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + ret = rte_telemetry_check_port_activity(port_id); + if (ret < 1) { + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + TELEMETRY_LOG_ERR("Error - rte_eth_xstats_get(%u) failed:" + " %d\n", port_id, num_xstats); + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + TELEMETRY_LOG_ERR("Error - Failed to malloc memory for " + "xstats\n"); + ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + TELEMETRY_LOG_ERR("Error - rte_eth_xstats_get(%u) len%i" + " failed: %d\n", port_id, num_xstats, ret); + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + uint64_t xstats_values[num_xstats]; + + for (i = 0; i < num_xstats; i++) + xstats_values[i] = eth_xstats[i].value; + ret = rte_metrics_update_values(port_id, reg_start_index, + xstats_values, num_xstats); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not update metrics values\n"); + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + free(eth_xstats); + return -1; + } + + free(eth_xstats); + return 0; +} + int32_t rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, const char *json_string) @@ -122,6 +190,66 @@ rte_telemetry_send_error_response(struct telemetry_impl *telemetry, return 0; } +int32_t +rte_telemetry_send_ports_stats_values(uint32_t *metric_ids, int num_metric_ids, + uint32_t *port_ids, int num_port_ids, struct telemetry_impl *telemetry) +{ + int ret; + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Invalid telemetry argument\n"); + return -1; + } + if (metric_ids == NULL) { + TELEMETRY_LOG_ERR("Error - Invalid metric_ids array\n"); + goto einval_fail; + } + if (num_metric_ids < 0) { + TELEMETRY_LOG_ERR("Error - Invalid num_metric_ids, must be " + "positive\n"); + goto einval_fail; + } + if (port_ids == NULL) { + TELEMETRY_LOG_ERR("Error - Invalid port_ids array\n"); + goto einval_fail; + } + if (num_port_ids < 0) { + TELEMETRY_LOG_ERR("Error - Invalid num_port_ids, must be " + "positive\n"); + goto einval_fail; + } + + int i; + for (i = 0; i < num_port_ids; i++) { + if (!rte_eth_dev_is_valid_port(port_ids[i])) { + TELEMETRY_LOG_ERR("Error - Port: %d invalid\n", + port_ids[i]); + goto einval_fail; + } + ret = rte_telemetry_update_metrics_ethdev(telemetry, + port_ids[i], telemetry->reg_index); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - failed to update ethdev " + "metrics\n"); + return -1; + } + } + + char *json_buffer = NULL; + ret = rte_telemetry_write_to_socket(telemetry, json_buffer); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not write to socket\n"); + return -1; + } + return 0; + +einval_fail: + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; +} + + static int32_t rte_telemetry_reg_ethdev_to_metrics(uint16_t port_id) { diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index b057794..ef417f2 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -61,4 +61,8 @@ rte_telemetry_unregister_client(struct telemetry_impl *telemetry, int32_t rte_telemetry_check_port_activity(int port_id); +int32_t +rte_telemetry_send_ports_stats_values(uint32_t *metric_ids, int num_metric_ids, + uint32_t *port_ids, int num_port_ids, struct telemetry_impl *telemetry); + #endif diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 571c991..4c50e41 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -327,8 +327,10 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, const char *stat_names[num_metrics]; uint32_t stat_ids[num_metrics]; int p; + uint32_t port_ids[RTE_MAX_ETHPORTS]; RTE_ETH_FOREACH_DEV(p) { + port_ids[num_port_ids] = p; num_port_ids++; } if (!num_port_ids) { @@ -351,6 +353,14 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, TELEMETRY_LOG_ERR("Error - Could not convert stat names to IDs\n"); goto fail; } + + ret = rte_telemetry_send_ports_stats_values(stat_ids, num_metrics, + port_ids, num_port_ids, telemetry); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Sending ports stats values failed\n"); + goto fail; + } + return 0; fail: @@ -451,6 +461,15 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl "IDs\n"); return -1; } + + ret = rte_telemetry_send_ports_stats_values(stat_ids, num_stat_names, + port_ids, num_port_ids, telemetry); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Sending ports stats values " + "failed\n"); + return -1; + } + return 0; } From patchwork Thu Aug 23 12:08:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43816 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2FE99532C; Thu, 23 Aug 2018 14:08:40 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 72D852C4F for ; Thu, 23 Aug 2018 14:08:33 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613860" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:31 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:08 +0100 Message-Id: <1535026093-101872-7-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 06/11] telemetry: format json response when sending stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds functionality to create a JSON message in order to send it to a client socket. When stats are requested by a client, they are retrieved from the metrics library and encoded in JSON format. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold --- lib/librte_telemetry/rte_telemetry.c | 314 ++++++++++++++++++++++++++++++++++- 1 file changed, 313 insertions(+), 1 deletion(-) diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index bd09374..ba04d3d 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -179,7 +179,7 @@ rte_telemetry_send_error_response(struct telemetry_impl *telemetry, return -EPERM; } - json_buffer = json_dumps(root, JSON_INDENT(2)); + json_buffer = json_dumps(root, 0); json_decref(root); ret = rte_telemetry_write_to_socket(telemetry, json_buffer); @@ -190,6 +190,311 @@ rte_telemetry_send_error_response(struct telemetry_impl *telemetry, return 0; } +static int +rte_telemetry_get_metrics(struct telemetry_impl *telemetry, uint32_t port_id, + struct rte_metric_value *metrics, struct rte_metric_name *names, + int num_metrics) +{ + int ret, num_values; + + if (num_metrics < 0) { + TELEMETRY_LOG_ERR("Error - Invalid metrics count\n"); + goto einval_fail; + } else if (num_metrics == 0) { + TELEMETRY_LOG_ERR("No metrics to display (none have been " + "registered)\n"); + goto eperm_fail; + } + + if (metrics == NULL) { + TELEMETRY_LOG_ERR("Error - metrics must be initialised.\n"); + goto einval_fail; + } + + if (names == NULL) { + TELEMETRY_LOG_ERR("Error - names must be initialised.\n"); + goto einval_fail; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + TELEMETRY_LOG_ERR("Error - Cannot get metrics names\n"); + goto eperm_fail; + } + + num_values = rte_metrics_get_values(port_id, NULL, 0); + ret = rte_metrics_get_values(port_id, metrics, num_values); + if (ret < 0 || ret > num_values) { + TELEMETRY_LOG_ERR("Error - Cannot get metrics values\n"); + goto eperm_fail; + } + return 0; + +eperm_fail: + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + +einval_fail: + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + +} + +static int32_t +rte_telemetry_json_format_stat(struct telemetry_impl *telemetry, json_t *stats, + const char *metric_name, uint64_t metric_value) +{ + int ret; + json_t *stat = json_object(); + + if (!stat) { + TELEMETRY_LOG_ERR("Error - Could not create stat JSON " + "object\n"); + goto eperm_fail; + } + + ret = json_object_set_new(stat, "name", json_string(metric_name)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Stat Name field cannot be set\n"); + goto eperm_fail; + } + + ret = json_object_set_new(stat, "value", json_integer(metric_value)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Stat Value field cannot be set\n"); + goto eperm_fail; + } + + ret = json_array_append_new(stats, stat); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Stat cannot be added to stats json " + "array\n"); + goto eperm_fail; + } + return 0; + +eperm_fail: + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + +} + +static int32_t +rte_telemetry_json_format_port(struct telemetry_impl *telemetry, + uint32_t port_id, json_t *ports, uint32_t *metric_ids, + uint32_t num_metric_ids) +{ + struct rte_metric_value *metrics = 0; + struct rte_metric_name *names = 0; + int num_metrics, ret; + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + TELEMETRY_LOG_ERR("Error - Cannot get metrics count\n"); + goto einval_fail; + } else if (num_metrics == 0) { + TELEMETRY_LOG_ERR("Error - No metrics to display (none have " + "been registered)\n"); + goto eperm_fail; + } + + metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (metrics == NULL || names == NULL) { + TELEMETRY_LOG_ERR("Error - Cannot allocate memory\n"); + free(metrics); + free(names); + int err_ret = rte_telemetry_send_error_response(telemetry, + -ENOMEM); + if (err_ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + } + + ret = rte_telemetry_get_metrics(telemetry, port_id, metrics, names, + num_metrics); + if (ret < 0) { + free(metrics); + free(names); + TELEMETRY_LOG_ERR("Error - rte_telemetry_get_metrics failed\n"); + return -1; + } + + json_t *port = json_object(); + json_t *stats = json_array(); + if (!port || !stats) { + TELEMETRY_LOG_ERR("Error - Could not create port/stats JSON " + "objects\n"); + goto eperm_fail; + } + + ret = json_object_set_new(port, "port", json_integer(port_id)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Port field cannot be set\n"); + goto eperm_fail; + } + + uint32_t i; + for (i = 0; i < num_metric_ids; i++) { + if (metric_ids[i] >= (uint32_t)num_metrics) { + TELEMETRY_LOG_ERR("Error - Metric_id: %d is " + "not valid\n", metric_ids[i]); + goto einval_fail; + } + + int metric_id = metric_ids[i]; + int metric_index = -1; + int metric_name_key = -1; + int32_t j; + + for (j = 0; j < num_metrics; j++) { + if (metrics[j].key == metric_id) { + metric_name_key = metrics[j].key; + metric_index = j; + break; + } + } + + if (metric_name_key < 0 || metric_index < 0) { + TELEMETRY_LOG_ERR("Error - Could not get metric " + "name/index\n"); + goto eperm_fail; + } + + const char *metric_name = names[metric_name_key].name; + uint64_t metric_value = metrics[metric_index].value; + ret = rte_telemetry_json_format_stat(telemetry, stats, + metric_name, metric_value); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Format stat with id: %u " + "failed\n", metric_id); + free(metrics); + free(names); + return -1; + } + } + + if (json_array_size(stats) == 0) + ret = json_object_set_new(port, "stats", json_null()); + else + ret = json_object_set_new(port, "stats", stats); + + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Stats object cannot be set\n"); + goto eperm_fail; + } + + ret = json_array_append_new(ports, port); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Port object cannot be added to " + "ports array\n"); + goto eperm_fail; + } + + free(metrics); + free(names); + return 0; + +eperm_fail: + free(metrics); + free(names); + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + +einval_fail: + free(metrics); + free(names); + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; +} + +static int32_t +rte_telemetry_encode_json_format(struct telemetry_impl *telemetry, + uint32_t *port_ids, uint32_t num_port_ids, uint32_t *metric_ids, + uint32_t num_metric_ids, char **json_buffer) +{ + int ret; + json_t *root, *ports; + + if (num_port_ids <= 0 || num_metric_ids <= 0) { + TELEMETRY_LOG_ERR("Error - Please provide port and metric ids " + "to query\n"); + goto einval_fail; + } + + ports = json_array(); + if (!ports) { + TELEMETRY_LOG_ERR("Error - Could not create ports JSON " + "array\n"); + goto eperm_fail; + } + + uint32_t i; + for (i = 0; i < num_port_ids; i++) { + if (!rte_eth_dev_is_valid_port(port_ids[i])) { + TELEMETRY_LOG_ERR("Error - Port: %d invalid\n", + port_ids[i]); + goto einval_fail; + } + } + + for (i = 0; i < num_port_ids; i++) { + ret = rte_telemetry_json_format_port(telemetry, port_ids[i], + ports, metric_ids, num_metric_ids); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - format port in JSON " + "failed\n"); + return -1; + } + } + + root = json_object(); + if (!root) { + TELEMETRY_LOG_ERR("Error - Could not create root JSON " + "object\n"); + goto eperm_fail; + } + + ret = json_object_set_new(root, "status_code", + json_string("Status OK: 200")); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Status code field cannot be set\n"); + goto eperm_fail; + } + + ret = json_object_set_new(root, "data", ports); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Data field cannot be set\n"); + goto eperm_fail; + } + + *json_buffer = json_dumps(root, JSON_INDENT(2)); + json_decref(root); + return 0; + +eperm_fail: + ret = rte_telemetry_send_error_response(telemetry, -EPERM); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; + +einval_fail: + ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not send error\n"); + return -1; +} + int32_t rte_telemetry_send_ports_stats_values(uint32_t *metric_ids, int num_metric_ids, uint32_t *port_ids, int num_port_ids, struct telemetry_impl *telemetry) @@ -235,6 +540,13 @@ rte_telemetry_send_ports_stats_values(uint32_t *metric_ids, int num_metric_ids, } char *json_buffer = NULL; + ret = rte_telemetry_encode_json_format(telemetry, port_ids, + num_port_ids, metric_ids, num_metric_ids, &json_buffer); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - JSON encode function failed\n"); + return -1; + } + ret = rte_telemetry_write_to_socket(telemetry, json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("Error - Could not write to socket\n"); From patchwork Thu Aug 23 12:08:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43817 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C50955681; Thu, 23 Aug 2018 14:08:41 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 4220A4CB5 for ; Thu, 23 Aug 2018 14:08:35 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613870" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:33 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:09 +0100 Message-Id: <1535026093-101872-8-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 07/11] telemetry: add tests for telemetry api X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds all tests for the Telemetry API. The tests added include a parser test, selftest, and socket messaging tests. The parser tests pass valid and invalid messages to the parser to ensure the correct return values are received. The selftest tests basic functions in the Telemetry API such as registering, unregistering, and initialisation. The socket messaging tests pass messages through the socket and validates the return message, to ensure the Telemetry API is responding correctly. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold --- drivers/telemetry/telemetry/telemetry_driver.c | 7 + lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 4 +- lib/librte_telemetry/rte_telemetry.c | 616 +++++++++++++++++++++- lib/librte_telemetry/rte_telemetry.h | 12 + lib/librte_telemetry/rte_telemetry_internal.h | 3 + lib/librte_telemetry/rte_telemetry_parser_test.c | 574 ++++++++++++++++++++ lib/librte_telemetry/rte_telemetry_parser_test.h | 39 ++ lib/librte_telemetry/rte_telemetry_socket_tests.h | 36 ++ lib/librte_telemetry/rte_telemetry_version.map | 1 + 10 files changed, 1290 insertions(+), 3 deletions(-) create mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c create mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.h create mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h diff --git a/drivers/telemetry/telemetry/telemetry_driver.c b/drivers/telemetry/telemetry/telemetry_driver.c index c56f60c..125a89c 100644 --- a/drivers/telemetry/telemetry/telemetry_driver.c +++ b/drivers/telemetry/telemetry/telemetry_driver.c @@ -15,6 +15,13 @@ telemetry_probe(struct rte_vdev_device *vdev) int ret; RTE_SET_USED(vdev); + ret = rte_telemetry_selftest(); + if (ret < 0) { + printf("Error - Selftest failed\n"); + return -1; + } + printf("Success - Selftest passed\n"); + ret = rte_telemetry_init(rte_socket_id()); if (ret < 0) { printf("Error - Telemetry initialisation failed\n"); diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index df8fdd9..d766c82 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -20,6 +20,7 @@ LIBABIVER := 1 # library source files SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 7450f96..57dd83d 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -1,8 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c') -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') +sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') +headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h', 'rte_telemetry_parser_test.h') deps += ['metrics', 'ethdev'] cflags += '-DALLOW_EXPERIMENTAL_API' jansson = cc.find_library('jansson', required: true) diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index ba04d3d..ecf644b 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -16,16 +16,34 @@ #include "rte_telemetry.h" #include "rte_telemetry_internal.h" #include "rte_telemetry_parser.h" +#include "rte_telemetry_parser_test.h" +#include "rte_telemetry_socket_tests.h" #define BUF_SIZE 1024 #define ACTION_POST 1 #define SLEEP_TIME 10 #define DEFAULT_DPDK_PATH "/var/run/.rte_telemetry" +#define SELFTEST_VALID_CLIENT "/var/run/valid_client" +#define SELFTEST_INVALID_CLIENT "/var/run/invalid_client" +#define SOCKET_TEST_CLIENT_PATH "/var/run/client" const char *socket_path = DEFAULT_DPDK_PATH; static telemetry_impl *static_telemetry; +struct telemetry_message_test { + char *test_name; + int (*test_func_ptr)(struct telemetry_impl *telemetry, int fd); +}; + +struct json_data { + char *status_code; + char *data; + int port; + char *stat_name; + int stat_value; +}; + int32_t rte_telemetry_check_port_activity(int port_id) { @@ -634,7 +652,7 @@ rte_telemetry_reg_ethdev_to_metrics(uint16_t port_id) static int32_t rte_telemetry_initial_accept(struct telemetry_impl *telemetry) { - int pid; + int pid, ret; RTE_ETH_FOREACH_DEV(pid) { telemetry->reg_index = @@ -648,6 +666,17 @@ rte_telemetry_initial_accept(struct telemetry_impl *telemetry) return -1; } telemetry->metrics_register_done = 1; + ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index, + telemetry->server_fd); + if (ret < 0) + return -1; + + ret = rte_telemetry_parser_test(telemetry); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Parser Tests Failed\n"); + return -1; + } + TELEMETRY_LOG_INFO("Success - All Parser Tests Passed\n"); return 0; } @@ -1108,6 +1137,591 @@ rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf) return -1; } +int32_t +rte_telemetry_dummy_client_socket(const char *valid_client_path) +{ + int sockfd = socket(AF_UNIX, SOCK_STREAM, 0); + if (sockfd < 0) { + TELEMETRY_LOG_ERR("Error - Test socket creation failure\n"); + return -1; + } + struct sockaddr_un addr = {0}; + addr.sun_family = AF_UNIX; + strlcpy(addr.sun_path, valid_client_path, sizeof(addr.sun_path)); + unlink(valid_client_path); + + if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { + TELEMETRY_LOG_ERR("Error - Test socket binding failure\n"); + return -1; + } + if (listen(sockfd, 1) < 0) { + TELEMETRY_LOG_ERR("Error - Listen failure\n"); + return -1; + } + return sockfd; +} + +int32_t +rte_telemetry_selftest(void) +{ + const char *invalid_client_path = SELFTEST_INVALID_CLIENT; + const char *valid_client_path = SELFTEST_VALID_CLIENT; + int ret, sockfd; + + TELEMETRY_LOG_INFO("Selftest\n"); + + ret = rte_telemetry_init(rte_socket_id()); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Valid initialisation test" + " failed\n"); + return -1; + } + TELEMETRY_LOG_INFO("Success - Valid initialisation test passed\n"); + + ret = rte_telemetry_init(rte_socket_id()); + if (ret != -EALREADY) { + TELEMETRY_LOG_ERR("Error - Invalid initialisation test " + "failed\n"); + return -1; + } + TELEMETRY_LOG_INFO("Success - Invalid initialisation test passed\n"); + + ret = rte_telemetry_unregister_client(static_telemetry, + invalid_client_path); + if (ret != -EPERM) { + TELEMETRY_LOG_ERR("Error - Invalid unregister test failed\n"); + return -1; + } + TELEMETRY_LOG_INFO("Success - Invalid unregister test passed\n"); + + sockfd = rte_telemetry_dummy_client_socket(valid_client_path); + if (sockfd < 0) { + TELEMETRY_LOG_ERR("Error - Test socket creation failed\n"); + return -1; + } + + ret = rte_telemetry_register_client(static_telemetry, + valid_client_path); + if (ret != 0) { + TELEMETRY_LOG_ERR("Error - Valid register test failed: %i\n", + ret); + return -1; + } + accept(sockfd, NULL, NULL); + TELEMETRY_LOG_INFO("Success - Valid register test passed\n"); + + ret = rte_telemetry_register_client(static_telemetry, + valid_client_path); + if (ret != -EINVAL) { + TELEMETRY_LOG_ERR("Error - Invalid register test failed: %i\n", + ret); + return -1; + } + TELEMETRY_LOG_INFO("Success - Invalid register test passed\n"); + + ret = rte_telemetry_unregister_client(static_telemetry, + invalid_client_path); + if (ret != -1) { + TELEMETRY_LOG_ERR("Error - Invalid unregister test failed: " + "%i\n", ret); + return -1; + } + TELEMETRY_LOG_INFO("Success - Invalid unregister test passed\n"); + + ret = rte_telemetry_unregister_client(static_telemetry, + valid_client_path); + if (ret != 0) { + TELEMETRY_LOG_ERR("Error - Valid unregister test failed: %i" + "\n", ret); + return -1; + } + TELEMETRY_LOG_INFO("Success - Valid unregister test passed\n"); + + ret = rte_telemetry_cleanup(); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - cleanup() test failed\n"); + return -1; + } + TELEMETRY_LOG_INFO("Success - Valid cleanup test passed\n"); + + return 0; +} + +int32_t +rte_telemetry_socket_messaging_testing(int index, int socket) +{ + struct telemetry_impl *telemetry = calloc(1, sizeof(telemetry_impl)); + int fd, bad_send_fd, send_fd, bad_fd, bad_recv_fd, recv_fd, ret; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - could not initialize " + "Telemetry API\n"); + return -1; + } + telemetry->server_fd = socket; + telemetry->reg_index = index; + TELEMETRY_LOG_INFO("Beginning Telemetry socket message Selftest\n"); + rte_telemetry_socket_test_setup(telemetry, &send_fd, &recv_fd); + TELEMETRY_LOG_INFO("Register valid client test\n"); + + ret = rte_telemetry_socket_register_test(telemetry, &fd, send_fd, + recv_fd); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Register valid client test " + "failed!\n"); + free(telemetry); + return -1; + } + TELEMETRY_LOG_INFO("Success - Register valid client test passed!\n"); + + TELEMETRY_LOG_INFO("Register invalid/same client test\n"); + ret = rte_telemetry_socket_test_setup(telemetry, &bad_send_fd, + &bad_recv_fd); + ret = rte_telemetry_socket_register_test(telemetry, &bad_fd, + bad_send_fd, bad_recv_fd); + if (!ret) { + TELEMETRY_LOG_ERR("Error - Register invalid/same client test " + "failed!\n"); + free(telemetry); + return -1; + } + TELEMETRY_LOG_INFO("Success - Register invalid/same client test " + "passed!\n"); + + ret = rte_telemetry_json_socket_message_test(telemetry, fd); + if (ret < 0) { + free(telemetry); + return -1; + } + + return 0; +} + +int32_t +rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, + int send_fd, int recv_fd) +{ + int ret; + char good_req_string[BUF_SIZE]; + + snprintf(good_req_string, sizeof(good_req_string), + "{\"action\":1,\"command\":\"clients\",\"data\":{\"client_path\"" + ":\"%s\"}}", SOCKET_TEST_CLIENT_PATH); + + listen(recv_fd, 1); + ret = send(send_fd, good_req_string, strlen(good_req_string), 0); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not send message over " + "socket\n"); + return -1; + } + rte_telemetry_run(telemetry); + if (telemetry->register_fail_count != 0) + return -1; + *fd = accept(recv_fd, NULL, NULL); + return 0; +} + +int32_t +rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, + int *recv_fd) +{ + int ret; + const char *client_path = SOCKET_TEST_CLIENT_PATH; + *send_fd = socket(AF_UNIX, SOCK_STREAM, 0); + *recv_fd = socket(AF_UNIX, SOCK_STREAM, 0); + + listen(telemetry->server_fd, 5); + struct sockaddr_un addr = {0}; + addr.sun_family = AF_UNIX; + strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); + ret = connect(*send_fd, (struct sockaddr *) &addr, sizeof(addr)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - could not connect socket\n"); + return -1; + } + telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); + + struct sockaddr_un addrs = {0}; + addrs.sun_family = AF_UNIX; + strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); + unlink(client_path); + + ret = bind(*recv_fd, (struct sockaddr *)&addrs, sizeof(addrs)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - could not bind socket\n"); + return -1; + } + + return 0; +} + +static int32_t +rte_telemetry_stat_parse(char *buf, struct json_data *json_data_struct) +{ + json_error_t error; + json_t *root = json_loads(buf, 0, &error); + int arraylen, i; + json_t *status, *dataArray, *port, *stats, *name, *value, + *dataArrayObj, *statsArrayObj; + + stats = NULL; + port = NULL; + name = NULL; + + if (buf == NULL) { + TELEMETRY_LOG_ERR("Error - JSON message is NULL\n"); + return -EINVAL; + } + + if (!root) { + TELEMETRY_LOG_ERR("Error - Could not load JSON object from " + "data passed in : %s\n", error.text); + return -EPERM; + } else if (!json_is_object(root)) { + TELEMETRY_LOG_ERR("Error - JSON Request is not a JSON " + "object\n"); + json_decref(root); + return -EINVAL; + } + + status = json_object_get(root, "status_code"); + if (!status) { + TELEMETRY_LOG_ERR("Error - Request does not have status " + "field\n"); + return -EINVAL; + } else if (!json_is_string(status)) { + TELEMETRY_LOG_ERR("Error - Status value is not a String\n"); + return -EINVAL; + } + + json_data_struct->status_code = strdup(json_string_value(status)); + + dataArray = json_object_get(root, "data"); + if (!dataArray) { + TELEMETRY_LOG_ERR("Error - Request does not have data field\n"); + return -EINVAL; + } + arraylen = json_array_size(dataArray); + if (arraylen == 0) { + json_data_struct->data = "null"; + return -EINVAL; + } + + for (i = 0; i < arraylen; i++) { + dataArrayObj = json_array_get(dataArray, i); + port = json_object_get(dataArrayObj, "port"); + stats = json_object_get(dataArrayObj, "stats"); + } + + if (!port) { + TELEMETRY_LOG_ERR("Error - Request does not have port field\n"); + return -EINVAL; + } + if (!json_is_integer(port)) { + TELEMETRY_LOG_ERR("Error - Port value is not an integer\n"); + return -EINVAL; + } + + json_data_struct->port = json_integer_value(port); + + if (!stats) { + TELEMETRY_LOG_ERR("Error - Request does not have stats " + "field\n"); + return -EINVAL; + } + + arraylen = json_array_size(stats); + for (i = 0; i < arraylen; i++) { + statsArrayObj = json_array_get(stats, i); + name = json_object_get(statsArrayObj, "name"); + value = json_object_get(statsArrayObj, "value"); + } + if (!name) { + TELEMETRY_LOG_ERR("Error - Request does not have name field\n"); + return -EINVAL; + } + if (!json_is_string(name)) { + TELEMETRY_LOG_ERR("Error - Stat name value is not a string\n"); + return -EINVAL; + } + json_data_struct->stat_name = strdup(json_string_value(name)); + + if (!value) { + TELEMETRY_LOG_ERR("Error - Request does not have value " + "field\n"); + return -EINVAL; + } + if (!json_is_integer(value)) { + TELEMETRY_LOG_ERR("Error - Stat value is not an integer\n"); + return -EINVAL; + } + json_data_struct->stat_value = json_integer_value(value); + return 0; +} + +static void +rte_telemetry_free_test_data(struct json_data *data) +{ + free(data->status_code); + free(data->stat_name); + free(data); +} + +int32_t +rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd) +{ + int ret; + char buf[BUF_SIZE]; + struct json_data *data_struct; + const char *status = "Status OK: 200"; + int port = 0; + const char *name = "rx_good_packets"; + int value = 0; + int fail_count = 0; + const char *valid_json_message = "{\"action\":0,\"command\":" + "\"ports_stats_values_by_name\",\"data\":{\"ports\"" + ":[0],\"stats\":[\"rx_good_packets\"]}}"; + + ret = send(fd, valid_json_message, strlen(valid_json_message), 0); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not send message over " + "socket\n"); + return -1; + } + rte_telemetry_run(telemetry); + int buffer_read = 0; + errno = 0; + buffer_read = recv(fd, buf, BUF_SIZE-1, 0); + + if (buffer_read == -1) { + TELEMETRY_LOG_ERR("Error - Read error\n"); + return -1; + } + buf[buffer_read] = '\0'; + data_struct = calloc(1, sizeof(struct json_data)); + ret = rte_telemetry_stat_parse(buf, data_struct); + + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not parse stats\n"); + fail_count++; + } + + if (strcmp(data_struct->status_code, status) != 0) { + TELEMETRY_LOG_ERR(" Error - Status code is invalid\n"); + fail_count++; + } + if (data_struct->port != port) { + TELEMETRY_LOG_ERR("Error - Port is invalid\n"); + fail_count++; + } + if (strcmp(data_struct->stat_name, name) != 0) { + TELEMETRY_LOG_ERR("Error - Stat name is invalid\n"); + fail_count++; + } + if (data_struct->stat_value != value) { + TELEMETRY_LOG_ERR("Error - Stat value is invalid\n"); + fail_count++; + } + rte_telemetry_free_test_data(data_struct); + if (fail_count > 0) + return -1; + + TELEMETRY_LOG_INFO("Success - Passed valid JSON message test " + "passed\n"); + return 0; +} + +int32_t +rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd) +{ + int ret; + char buf[BUF_SIZE]; + int fail_count = 0; + const char *invalid_json = "{]"; + const char *status = "Status Error: Unknown"; + const char *data = "null"; + struct json_data *data_struct; + int buffer_read = 0; + errno = 0; + + ret = send(fd, invalid_json, strlen(invalid_json), 0); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not send message over " + "socket\n"); + return -1; + } + + rte_telemetry_run(telemetry); + buffer_read = recv(fd, buf, BUF_SIZE-1, 0); + + if (buffer_read == -1) { + TELEMETRY_LOG_ERR("Error - Read error\n"); + return -1; + } + buf[buffer_read] = '\0'; + + data_struct = calloc(1, sizeof(struct json_data)); + ret = rte_telemetry_stat_parse(buf, data_struct); + + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not parse stats\n"); + + if (strcmp(data_struct->status_code, status) != 0) { + TELEMETRY_LOG_ERR("Error - Status code is invalid\n"); + fail_count++; + } + if (strcmp(data_struct->data, data) != 0) { + TELEMETRY_LOG_ERR("Error - Data status is invalid\n"); + fail_count++; + } + rte_telemetry_free_test_data(data_struct); + if (fail_count > 0) + return -1; + + TELEMETRY_LOG_INFO("Success - Passed invalid JSON message test\n"); + return 0; +} + +int32_t +rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd) +{ + int ret; + char buf[BUF_SIZE]; + int fail_count = 0; + char *status = "Status Error: Invalid Argument 404"; + char *data = "null"; + struct json_data *data_struct; + const char *invalid_contents = "{\"action\":0,\"command\":" + "\"ports_stats_values_by_name\",\"data\":{\"ports\"" + ":[0],\"stats\":[\"some_invalid_param\"," + "\"another_invalid_param\"]}}"; + int buffer_read = 0; + errno = 0; + + ret = send(fd, invalid_contents, strlen(invalid_contents), 0); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not send message over " + "socket\n"); + return -1; + } + rte_telemetry_run(telemetry); + buffer_read = recv(fd, buf, BUF_SIZE-1, 0); + if (buffer_read == -1) { + TELEMETRY_LOG_ERR("Error - Read error\n"); + return -1; + } + buf[buffer_read] = '\0'; + data_struct = calloc(1, sizeof(struct json_data)); + ret = rte_telemetry_stat_parse(buf, data_struct); + + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not parse stats\n"); + + if (strcmp(data_struct->status_code, status) != 0) { + TELEMETRY_LOG_ERR("Error - Status code is invalid\n"); + fail_count++; + } + if (strcmp(data_struct->data, data) != 0) { + TELEMETRY_LOG_ERR("Error - Data status is invalid\n"); + fail_count++; + } + rte_telemetry_free_test_data(data_struct); + if (fail_count > 0) + return -1; + + TELEMETRY_LOG_INFO("Success - Passed invalid JSON content test\n"); + return 0; +} + +int32_t +rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd) +{ + int ret; + char buf[BUF_SIZE]; + int fail_count = 0; + const char *status = "Status Error: Invalid Argument 404"; + char *data = "null"; + struct json_data *data_struct; + const char *empty_json = "{}"; + int buffer_read = 0; + errno = 0; + + ret = (send(fd, empty_json, strlen(empty_json), 0)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not send message over " + "socket\n"); + return -1; + } + rte_telemetry_run(telemetry); + buffer_read = recv(fd, buf, BUF_SIZE-1, 0); + if (buffer_read == -1) { + TELEMETRY_LOG_ERR("Error - Read error\n"); + return -1; + } + buf[buffer_read] = '\0'; + data_struct = calloc(1, sizeof(struct json_data)); + ret = rte_telemetry_stat_parse(buf, data_struct); + + if (ret < 0) + TELEMETRY_LOG_ERR("Error - Could not parse stats\n"); + + if (strcmp(data_struct->status_code, status) != 0) { + TELEMETRY_LOG_ERR("Error - Status code is invalid\n"); + fail_count++; + } + if (strcmp(data_struct->data, data) != 0) { + TELEMETRY_LOG_ERR("Error - Data status is invalid\n"); + fail_count++; + } + rte_telemetry_free_test_data(data_struct); + if (fail_count > 0) + return -1; + + TELEMETRY_LOG_INFO("Success - Passed JSON empty message test\n"); + return 0; +} + +int32_t +rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, int fd) +{ + uint16_t i; + int ret, fail_count; + + fail_count = 0; + struct telemetry_message_test socket_json_tests[] = { + {.test_name = "Invalid JSON test", + .test_func_ptr = rte_telemetry_invalid_json_test}, + {.test_name = "Valid JSON test", + .test_func_ptr = rte_telemetry_valid_json_test}, + {.test_name = "JSON contents test", + .test_func_ptr = rte_telemetry_json_contents_test}, + {.test_name = "JSON empty tests", + .test_func_ptr = rte_telemetry_json_empty_test} + }; + +#define NUM_TESTS (sizeof(socket_json_tests)/sizeof(socket_json_tests[0])) + + for (i = 0; i < NUM_TESTS; i++) { + TELEMETRY_LOG_INFO("%s\n", + socket_json_tests[i].test_name); + ret = (socket_json_tests[i].test_func_ptr) + (telemetry, fd); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - %s failed\n", + socket_json_tests[i].test_name); + fail_count++; + } + } + if (fail_count > 0) { + TELEMETRY_LOG_ERR("Error - Failed %i JSON socket message " + "test(s)", fail_count); + return -1; + } + TELEMETRY_LOG_INFO("Success - All JSON tests passed\n"); + return 0; +} + int telemetry_log_level; RTE_INIT(rte_telemetry_log_init); diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index b691845..4f187b7 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -37,4 +37,16 @@ rte_telemetry_init(uint32_t socket_id); int32_t rte_telemetry_cleanup(void); +/** + * Runs various tests to ensure telemetry initialisation and register/unregister + * functions are working correctly. + * + * @return + * 0 on success when all tests have passed + * @return + * -1 on failure when the test has failed + */ +int32_t +rte_telemetry_selftest(void); + #endif diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index ef417f2..3e21b79 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -65,4 +65,7 @@ int32_t rte_telemetry_send_ports_stats_values(uint32_t *metric_ids, int num_metric_ids, uint32_t *port_ids, int num_port_ids, struct telemetry_impl *telemetry); +int32_t +rte_telemetry_socket_messaging_testing(int index, int socket); + #endif diff --git a/lib/librte_telemetry/rte_telemetry_parser_test.c b/lib/librte_telemetry/rte_telemetry_parser_test.c new file mode 100644 index 0000000..4fa442d --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_parser_test.c @@ -0,0 +1,574 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "rte_telemetry_parser.h" + +#define ACTION_GET 0 +#define ACTION_DELETE 2 + +#define INV_ACTION_VAL 0 +#define INV_COMMAND_VAL 1 +#define INV_DATA_VAL 2 +#define INV_ACTION_FIELD 3 +#define INV_COMMAND_FIELD 4 +#define INV_DATA_FIELD 5 +#define INV_JSON_FORMAT 6 +#define VALID_REQ 7 + + +#define TEST_CLIENT "/var/run/test_client" + +int32_t +rte_telemetry_create_test_socket(struct telemetry_impl *telemetry, + const char *test_client_path) +{ + + int ret, sockfd; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Telemetry argument has not been " + "initialised\n"); + return -EINVAL; + } + + sockfd = socket(AF_UNIX, SOCK_STREAM, 0); + if (sockfd < 0) { + TELEMETRY_LOG_ERR("Error - Test socket creation failure\n"); + return -1; + } + + struct sockaddr_un addr = {0}; + addr.sun_family = AF_UNIX; + strlcpy(addr.sun_path, test_client_path, sizeof(addr.sun_path)); + unlink(test_client_path); + + if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { + TELEMETRY_LOG_ERR("Error - Test socket binding failure\n"); + return -1; + } + + if (listen(sockfd, 1) < 0) { + TELEMETRY_LOG_ERR("Error - Listen failure\n"); + return -1; + } + + ret = rte_telemetry_register_client(telemetry, test_client_path); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Register dummy client failed: %i", + ret); + return -1; + } + + ret = accept(sockfd, NULL, NULL); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Socket accept failed\n"); + return -1; + } + + struct telemetry_client *client; + TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) + telemetry->request_client = client; + + return 0; +} + +int32_t +rte_telemetry_format_port_stat_ids(int *port_ids, int num_port_ids, + const char * const *stat_names, int num_stat_names, json_t **data) +{ + + int ret; + json_t *stat_names_json_array = NULL; + json_t *port_ids_json_array = NULL; + if (num_port_ids < 0) { + TELEMETRY_LOG_ERR("Error - Port Ids Count invalid\n"); + goto fail; + } + + *data = json_object(); + if (!*data) { + TELEMETRY_LOG_ERR("Error - Data json object creation failed\n"); + goto fail; + } + + port_ids_json_array = json_array(); + if (!port_ids_json_array) { + TELEMETRY_LOG_ERR("Error - port_ids_json_array creation " + "failed\n"); + goto fail; + } + + uint32_t i; + for (i = 0; i < (uint32_t) num_port_ids; i++) { + ret = json_array_append(port_ids_json_array, + json_integer(port_ids[i])); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - JSON array creation " + "failed\n"); + goto fail; + } + } + + ret = json_object_set_new(*data, "ports", port_ids_json_array); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Setting 'ports' value in data object" + " failed\n"); + goto fail; + } + + if (stat_names) { + + if (num_stat_names < 0) { + TELEMETRY_LOG_ERR("Error - Stat Names Count invalid\n"); + goto fail; + } + + stat_names_json_array = json_array(); + if (!stat_names_json_array) { + TELEMETRY_LOG_ERR("Error - stat_names_json_array " + "creation failed\n"); + goto fail; + } + + uint32_t i; + for (i = 0; i < (uint32_t) num_stat_names; i++) { + ret = json_array_append(stat_names_json_array, + json_string(stat_names[i])); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - JSON array creation " + " failed\n"); + goto fail; + } + } + + ret = json_object_set_new(*data, "stats", + stat_names_json_array); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Setting 'stats' value in " + "data object failed\n"); + goto fail; + } + + } + return 0; + +fail: + if (*data) + json_decref(*data); + if (stat_names_json_array) + json_decref(stat_names_json_array); + if (port_ids_json_array) + json_decref(port_ids_json_array); + return -1; +} + +int32_t +rte_telemetry_create_json_request(int action, char *command, + const char *client_path, int *port_ids, int num_port_ids, + const char * const *stat_names, int num_stat_names, char **request, + int inv_choice) +{ + int ret; + json_t *root = json_object(); + + if (!root) { + TELEMETRY_LOG_ERR("Error - Could not create root json " + "object\n"); + goto fail; + } + + if (inv_choice == INV_ACTION_FIELD) { + ret = json_object_set_new(root, "ac--on", json_integer(action)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Setting invalid action " + "field in root object failed\n"); + goto fail; + } + } else { + ret = json_object_set_new(root, "action", json_integer(action)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Setting valid action field " + "in root object failed\n"); + goto fail; + } + } + + if (inv_choice == INV_COMMAND_FIELD) { + ret = json_object_set_new(root, "co---nd", + json_string(command)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Setting invalid command " + "field in root object failed\n"); + goto fail; + } + } else { + ret = json_object_set_new(root, "command", + json_string(command)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Setting valid command " + "field in root object failed\n"); + goto fail; + } + } + + json_t *data = json_null(); + if (client_path) { + data = json_object(); + if (!data) { + TELEMETRY_LOG_ERR("Error - Data json object creation " + "failed\n"); + goto fail; + } + + ret = json_object_set_new(data, "client_path", + json_string(client_path)); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Setting valid client_path " + "field in data object failed\n"); + goto fail; + } + + } else if (port_ids) { + ret = rte_telemetry_format_port_stat_ids(port_ids, num_port_ids, + stat_names, num_stat_names, &data); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Formatting Port/Stat " + "arrays failed\n"); + goto fail; + } + + } + + if (inv_choice == INV_DATA_FIELD) { + ret = json_object_set_new(root, "d--a", data); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Setting invalid data " + "field in data object failed\n"); + goto fail; + } + } else { + ret = json_object_set_new(root, "data", data); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Setting valid data field " + "in data object failed\n"); + goto fail; + } + } + + *request = json_dumps(root, 0); + if (!*request) { + TELEMETRY_LOG_ERR("Error - Converting JSON root object to " + "char* failed\n"); + goto fail; + } + + json_decref(root); + return 0; + +fail: + if (root) + json_decref(root); + return -1; +} + +int32_t +rte_telemetry_send_get_ports_and_stats_request(struct telemetry_impl *telemetry, + int action_choice, char *command_choice, int inv_choice) +{ + int ret; + char *request; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Telemetry argument has not been " + "initialised\n"); + return -EINVAL; + } + + char *client_path_data = NULL; + + if (inv_choice == INV_ACTION_VAL) + action_choice = -1; + else if (inv_choice == INV_COMMAND_VAL) + command_choice = "INVALID_COMMAND"; + else if (inv_choice == INV_DATA_VAL) + client_path_data = "INVALID_DATA"; + + ret = rte_telemetry_create_json_request(action_choice, command_choice, + client_path_data, NULL, -1, NULL, -1, &request, inv_choice); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not create JSON Request\n"); + return -1; + } + + if (inv_choice == INV_JSON_FORMAT) + request++; + + ret = rte_telemetry_parse(telemetry, request); + if (ret < 0) { + TELEMETRY_LOG_WARN("Warning - Could not parse JSON Request\n"); + return -1; + } + return 0; +} + +int32_t +rte_telemetry_send_get_ports_details_request(struct telemetry_impl *telemetry, + int action_choice, int *port_ids, int num_port_ids, int inv_choice) +{ + int ret; + char *request; + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Telemetry argument has not been " + "initialised\n"); + return -EINVAL; + } + + char *command = "ports_details"; + + if (inv_choice == INV_ACTION_VAL) + action_choice = -1; + else if (inv_choice == INV_COMMAND_VAL) + command = "INVALID_COMMAND"; + else if (inv_choice == INV_DATA_VAL) + port_ids = NULL; + + + ret = rte_telemetry_create_json_request(action_choice, command, NULL, + port_ids, num_port_ids, NULL, -1, &request, inv_choice); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not create JSON Request\n"); + return -1; + } + + if (inv_choice == INV_JSON_FORMAT) + request++; + + ret = rte_telemetry_parse(telemetry, request); + if (ret < 0) { + TELEMETRY_LOG_WARN("Warning - Could not parse JSON Request\n"); + return -1; + } + return 0; +} + +int32_t +rte_telemetry_send_stats_values_by_name_request(struct telemetry_impl + *telemetry, int action_choice, int *port_ids, int num_port_ids, + const char * const *stat_names, int num_stat_names, + int inv_choice) +{ + int ret; + char *request; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Telemetry argument has not been " + "initialised\n"); + return -EINVAL; + } + + char *command = "ports_stats_values_by_name"; + + if (inv_choice == INV_ACTION_VAL) + action_choice = -1; + else if (inv_choice == INV_COMMAND_VAL) + command = "INVALID_COMMAND"; + else if (inv_choice == INV_DATA_VAL) { + port_ids = NULL; + stat_names = NULL; + } + + + ret = rte_telemetry_create_json_request(action_choice, command, NULL, + port_ids, num_port_ids, stat_names, num_stat_names, + &request, inv_choice); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not create JSON Request\n"); + return -1; + } + + if (inv_choice == INV_JSON_FORMAT) + request++; + + ret = rte_telemetry_parse(telemetry, request); + if (ret < 0) { + TELEMETRY_LOG_WARN("Warning - Could not parse JSON Request\n"); + return -1; + } + + return 0; +} + +int32_t +rte_telemetry_send_unreg_request(struct telemetry_impl *telemetry, + int action_choice, const char *client_path, int inv_choice) +{ + int ret; + char *request; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Telemetry argument has not been " + "initialised\n"); + return -EINVAL; + } + + char *command = "clients"; + + if (inv_choice == INV_ACTION_VAL) + action_choice = -1; + else if (inv_choice == INV_COMMAND_VAL) + command = "INVALID_COMMAND"; + else if (inv_choice == INV_DATA_VAL) + client_path = NULL; + + ret = rte_telemetry_create_json_request(action_choice, command, + client_path, NULL, -1, NULL, -1, &request, inv_choice); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not create JSON Request\n"); + return -1; + } + + if (inv_choice == INV_JSON_FORMAT) + request++; + + ret = rte_telemetry_parse(telemetry, request); + if (ret < 0) { + TELEMETRY_LOG_WARN("Warning - Could not parse JSON Request\n"); + return -1; + } + return 0; +} + +int32_t +rte_telemetry_parser_test(struct telemetry_impl *telemetry) +{ + int ret; + + if (!telemetry) { + TELEMETRY_LOG_ERR("Error - Telemetry argument has not been " + "initialised\n"); + return -EINVAL; + } + + const char *client_path = TEST_CLIENT; + ret = rte_telemetry_create_test_socket(telemetry, client_path); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Could not create test request " + "client socket\n"); + return -1; + } + + int port_ids[] = {0, 1}; + int num_port_ids = RTE_DIM(port_ids); + + static const char * const stat_names[] = {"tx_good_packets", + "rx_good_packets"}; + int num_stat_names = RTE_DIM(stat_names); + + static const char * const test_types[] = { + "INVALID ACTION VALUE TESTS", + "INVALID COMMAND VALUE TESTS", + "INVALID DATA VALUE TESTS", + "INVALID ACTION FIELD TESTS", + "INVALID COMMAND FIELD TESTS", + "INVALID DATA FIELD TESTS", + "INVALID JSON FORMAT TESTS", + "VALID TESTS" + }; + + +#define NUM_TEST_TYPES (sizeof(test_types)/sizeof(const char * const)) + + uint32_t i; + for (i = 0; i < NUM_TEST_TYPES; i++) { + TELEMETRY_LOG_INFO("%s\n", + test_types[i]); + + ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, + ACTION_GET, "ports", i); + if (ret != 0 && i == VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Get ports valid test " + "failed\n"); + return -EPERM; + } else if (ret != -1 && i != VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Get ports invalid test" + " failed\n"); + return -EPERM; + } + + TELEMETRY_LOG_INFO("Success - Get ports test passed\n"); + + ret = rte_telemetry_send_get_ports_details_request(telemetry, + ACTION_GET, port_ids, num_port_ids, i); + if (ret != 0 && i == VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Get ports details valid\n"); + return -EPERM; + } else if (ret != -1 && i != VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Get ports details " + "invalid\n"); + return -EPERM; + } + + TELEMETRY_LOG_INFO("Success - Get ports details test passed\n"); + + ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, + ACTION_GET, "port_stats", i); + if (ret != 0 && i == VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Get port stats valid " + "test\n"); + return -EPERM; + } else if (ret != -1 && i != VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Get ports stats invalid" + " test failed\n"); + return -EPERM; + } + + TELEMETRY_LOG_INFO("Success - Get ports stats test passed\n"); + + ret = rte_telemetry_send_stats_values_by_name_request(telemetry, + ACTION_GET, port_ids, num_port_ids, + stat_names, num_stat_names, i); + if (ret != 0 && i == VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Get ports stats values by" + " name valid test failed\n"); + return -EPERM; + } else if (ret != -1 && i != VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Get ports stats values by" + " name invalid test failed\n"); + return -EPERM; + } + + TELEMETRY_LOG_INFO("Success - Get ports stats values by name" + " test passed\n"); + + ret = rte_telemetry_send_unreg_request(telemetry, ACTION_DELETE, + client_path, i); + if (ret != 0 && i == VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Deregister valid test " + "failed\n"); + return -EPERM; + } else if (ret != -1 && i != VALID_REQ) { + TELEMETRY_LOG_ERR("Error - Deregister invalid test" + " failed\n"); + return -EPERM; + } + + TELEMETRY_LOG_INFO("Success - Deregister test passed\n"); + } + + return 0; +} diff --git a/lib/librte_telemetry/rte_telemetry_parser_test.h b/lib/librte_telemetry/rte_telemetry_parser_test.h new file mode 100644 index 0000000..6ada852 --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_parser_test.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_PARSER_TEST_H_ +#define _RTE_TELEMETRY_PARSER_TEST_H_ + +int32_t +rte_telemetry_parser_test(struct telemetry_impl *telemetry); + +int32_t +rte_telemetry_format_port_stat_ids(int *port_ids, int num_port_ids, + const char * const stat_names, int num_stat_names, json_t **data); + +int32_t +rte_telemetry_create_json_request(int action, char *command, + const char *client_path, int *port_ids, int num_port_ids, + const char * const stat_names, int num_stat_names, char **request, + int inv_choice); + +int32_t +rte_telemetry_send_get_ports_and_stats_request(struct telemetry_impl *telemetry, + int action_choice, char *command_choice, int inv_choice); + +int32_t +rte_telemetry_send_get_ports_details_request(struct telemetry_impl *telemetry, + int action_choice, int *port_ids, int num_port_ids, int inv_choice); + +int32_t +rte_telemetry_send_stats_values_by_name_request(struct telemetry_impl + *telemetry, int action_choice, int *port_ids, int num_port_ids, + const char * const stat_names, int num_stat_names, + int inv_choice); + +int32_t +rte_telemetry_send_unreg_request(int action_choice, const char *client_path, + int inv_choice); + +#endif diff --git a/lib/librte_telemetry/rte_telemetry_socket_tests.h b/lib/librte_telemetry/rte_telemetry_socket_tests.h new file mode 100644 index 0000000..db9167c --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_socket_tests.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include + +#include "rte_telemetry_internal.h" + +#ifndef _RTE_TELEMETRY_SOCKET_TESTING_H_ +#define _RTE_TELEMETRY_SOCKET_TESTING_H_ + +int32_t +rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, + int fd); + +int32_t +rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd); + +int32_t +rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd); + +int32_t +rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd); + +int32_t +rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd); + +int32_t +rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, + int send_fd, int recv_fd); + +int32_t +rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, + int *recv_fd); + +#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index efd437d..5ce5680 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -2,5 +2,6 @@ DPDK_18.05 { global: rte_telemetry_init; + rte_telemetry_selftest; local: *; }; From patchwork Thu Aug 23 12:08:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43818 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0528558C3; Thu, 23 Aug 2018 14:08:43 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 850AE4CC3 for ; Thu, 23 Aug 2018 14:08:36 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613876" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:35 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:10 +0100 Message-Id: <1535026093-101872-9-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 08/11] telemetry: add vdev kvargs for selftest X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds functionality to run the selftest by passing in an argument to vdev, "selftest". To run the tests that were added in the previous patch, the argument "selftest=1" must be added to the vdev arguments. This will enable the user to run telemetry with or without tests, as required. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold --- drivers/telemetry/telemetry/telemetry_driver.c | 50 ++++++++++++++++++++++---- lib/librte_telemetry/rte_telemetry.c | 35 +++++++++--------- lib/librte_telemetry/rte_telemetry.h | 8 ++++- lib/librte_telemetry/rte_telemetry_internal.h | 1 + 4 files changed, 70 insertions(+), 24 deletions(-) diff --git a/drivers/telemetry/telemetry/telemetry_driver.c b/drivers/telemetry/telemetry/telemetry_driver.c index 125a89c..a97c199 100644 --- a/drivers/telemetry/telemetry/telemetry_driver.c +++ b/drivers/telemetry/telemetry/telemetry_driver.c @@ -8,21 +8,57 @@ #include #include #include +#include + +#define SELFTEST_ARG "selftest" + +static int +assign_selftest(const char *key __rte_unused, const char *value, void *opaque) +{ + int *selftest = opaque; + *selftest = atoi(value); + return 0; +} static int telemetry_probe(struct rte_vdev_device *vdev) { + static const char *const args[] = { + SELFTEST_ARG + }; + + const char *params; int ret; + struct telemetry_args telemetry_args; + telemetry_args.selftest = 0; - RTE_SET_USED(vdev); - ret = rte_telemetry_selftest(); - if (ret < 0) { - printf("Error - Selftest failed\n"); - return -1; + params = rte_vdev_device_args(vdev); + if (params != NULL && params[0] != '\0') { + struct rte_kvargs *kvlist = rte_kvargs_parse(params, args); + + if (!kvlist) { + printf("Error - Could not create kvlist\n"); + } else { + ret = rte_kvargs_process(kvlist, SELFTEST_ARG, + assign_selftest, &telemetry_args.selftest); + if (ret != 0) { + printf("Error - Processing the arguments\n"); + rte_kvargs_free(kvlist); + return ret; + } + } + rte_kvargs_free(kvlist); + } + if (telemetry_args.selftest) { + ret = rte_telemetry_selftest(); + if (ret < 0) { + printf("Error - Selftest failed\n"); + return -1; + } + printf("Success - Selftest passed\n"); } - printf("Success - Selftest passed\n"); - ret = rte_telemetry_init(rte_socket_id()); + ret = rte_telemetry_init(rte_socket_id(), &telemetry_args); if (ret < 0) { printf("Error - Telemetry initialisation failed\n"); return -1; diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index ecf644b..1f7528d 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -666,18 +666,21 @@ rte_telemetry_initial_accept(struct telemetry_impl *telemetry) return -1; } telemetry->metrics_register_done = 1; - ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index, - telemetry->server_fd); - if (ret < 0) - return -1; + if (telemetry->selftest) { + ret = + rte_telemetry_socket_messaging_testing(telemetry->reg_index, + telemetry->server_fd); + if (ret < 0) + return -1; - ret = rte_telemetry_parser_test(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Error - Parser Tests Failed\n"); - return -1; + ret = rte_telemetry_parser_test(telemetry); + if (ret < 0) { + TELEMETRY_LOG_ERR("Error - Parser Tests Failed\n"); + return -1; + } + TELEMETRY_LOG_INFO("Success - All Parser Tests Passed\n"); + telemetry->selftest = 0; } - TELEMETRY_LOG_INFO("Success - All Parser Tests Passed\n"); - return 0; } @@ -856,7 +859,7 @@ rte_telemetry_create_socket(struct telemetry_impl *telemetry) } int32_t -rte_telemetry_init(uint32_t socket_id) +rte_telemetry_init(uint32_t socket_id, struct telemetry_args *args) { int ret; pthread_attr_t attr; @@ -873,7 +876,7 @@ rte_telemetry_init(uint32_t socket_id) TELEMETRY_LOG_ERR("Error - Memory could not be allocated\n"); return -ENOMEM; } - + static_telemetry->selftest = args->selftest; static_telemetry->socket_id = socket_id; rte_metrics_init(static_telemetry->socket_id); ret = rte_telemetry_create_socket(static_telemetry); @@ -1167,10 +1170,10 @@ rte_telemetry_selftest(void) const char *invalid_client_path = SELFTEST_INVALID_CLIENT; const char *valid_client_path = SELFTEST_VALID_CLIENT; int ret, sockfd; + struct telemetry_args args; + args.selftest = 0; - TELEMETRY_LOG_INFO("Selftest\n"); - - ret = rte_telemetry_init(rte_socket_id()); + ret = rte_telemetry_init(rte_socket_id(), &args); if (ret < 0) { TELEMETRY_LOG_ERR("Error - Valid initialisation test" " failed\n"); @@ -1178,7 +1181,7 @@ rte_telemetry_selftest(void) } TELEMETRY_LOG_INFO("Success - Valid initialisation test passed\n"); - ret = rte_telemetry_init(rte_socket_id()); + ret = rte_telemetry_init(rte_socket_id(), &args); if (ret != -EALREADY) { TELEMETRY_LOG_ERR("Error - Invalid initialisation test " "failed\n"); diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index 4f187b7..ac66595 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -7,12 +7,18 @@ #ifndef _RTE_TELEMETRY_H_ #define _RTE_TELEMETRY_H_ +typedef struct telemetry_args { + int selftest; +} telemetry_args; + /** * Get the telemetry_impl structure device pointer initialised. * * @param socket_id * Unsigned integer representing the socket id to be used * for the telemetry structure. + * @param args + * Struct containing arguments from telemetry_args * * @return * 0 on successful initialisation. @@ -24,7 +30,7 @@ * -EALREADY if Telemetry is already initialised. */ int32_t -rte_telemetry_init(uint32_t socket_id); +rte_telemetry_init(uint32_t socket_id, struct telemetry_args *args); /** * Clean up and free memory. diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index 3e21b79..3270711 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -41,6 +41,7 @@ typedef struct telemetry_impl { TAILQ_HEAD(, telemetry_client) client_list_head; struct telemetry_client *request_client; int register_fail_count; + int selftest; } telemetry_impl; int32_t From patchwork Thu Aug 23 12:08:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43819 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4BF2858FE; Thu, 23 Aug 2018 14:08:45 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id E04784CE4 for ; Thu, 23 Aug 2018 14:08:37 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613884" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:36 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:11 +0100 Message-Id: <1535026093-101872-10-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 09/11] doc: add telemetry documentation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds all documentation for telemetry. A description on how to use the Telemetry API with a DPDK application is given in this document. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold Acked-by: Marko Kovacevic --- doc/guides/howto/index.rst | 1 + doc/guides/howto/telemetry.rst | 86 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 87 insertions(+) create mode 100644 doc/guides/howto/telemetry.rst diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst index e13a090..a642a2b 100644 --- a/doc/guides/howto/index.rst +++ b/doc/guides/howto/index.rst @@ -17,3 +17,4 @@ HowTo Guides virtio_user_for_container_networking virtio_user_as_exceptional_path packet_capture_framework + telemetry diff --git a/doc/guides/howto/telemetry.rst b/doc/guides/howto/telemetry.rst new file mode 100644 index 0000000..0a2ae97 --- /dev/null +++ b/doc/guides/howto/telemetry.rst @@ -0,0 +1,86 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +DPDK Telemetry API User Guide +============================== + +This document describes how the Data Plane Development Kit(DPDK) Telemetry API +is used for querying port statistics from incoming traffic. + +Introduction +------------ + +The ``librte_telemetry`` provides the functionality so that users may query +metrics from incoming port traffic. The application which initializes packet +forwarding will act as the server, sending metrics to the requesting application +which acts as the client. + +In DPDK, applications can be used to initialize the ``telemetry`` as a virtual +device. To view incoming traffic on featured ports, the application should be +run first (ie. after ports are configured). Once the application is running, the +service assurance agent(for example the collectd plugin) should be run to begin +querying the API. + +A client connects their Service Assurance application to the DPDK application +via a UNIX socket. Once a connection is established, a client can send JSON +messages to the DPDK application requesting metrics via another UNIX client. +This request is then handled and parsed if valid. The response is then +formatted in JSON and sent back to the requesting client. + +Pre-requisites +~~~~~~~~~~~~~~ + +* Python ≥ 2.5 + +* Jansson library for JSON serialization + +Test Environment +---------------- + +``telemetry`` offers a range of selftests that a client can run within +the DPDK application. This is done by passing in an argument to the +virtual device initialization like so through the EAL command line.:: + + ./dpdk_app --vdev=telemetry,selftest=1 + + +Configuration +------------- + +Enable the telemetry API by modifying the following config option before +building DPDK:: + + CONFIG_RTE_LIBRTE_TELEMETRY=y + +Note: Meson will pick this up automatically if ``libjansson`` is available. + +Running the Application +----------------------- + +The following steps demonstrate how to run the ``telemetry`` API to query all +statistics on all active ports, using the ``telemetry_client`` python script +to query. +Note: This guide assumes packet generation is applicable and the user is +testing with testpmd as a DPDK primary application to forward packets, although +any DPDK application is applicable. + +#. Launch testpmd as the primary application with the ``telemetry`` + virtual device.:: + + ./app/testpmd --vdev=telemetry + +#. Launch the ``telemetry`` python script with a client filepath.:: + + python usertools/telemetry_client.py /var/run/some_client + + The client filepath is going to be used to setup our UNIX connect with the + DPDK primary application, in this case ``testpmd`` + This will initialize a menu where a client can proceed to recursively query + statistics, request statistics once or unregister the file_path, thus exiting + the menu. + +#. Send traffic to any or all available ports from a traffic generator. + Select a query option(recursive or singular polling). + The metrics will then be displayed on the client terminal in JSON format. + +#. Once finished, unregister the client using the menu command. From patchwork Thu Aug 23 12:08:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43820 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AC9E05B1E; Thu, 23 Aug 2018 14:08:46 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 397E14C96 for ; Thu, 23 Aug 2018 14:08:39 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613890" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:37 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:12 +0100 Message-Id: <1535026093-101872-11-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 10/11] usertools: add client python script for telemetry X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds a python script which can be used as a demo client. The script is interactive and will allow the user to register, request statistics, and unregister. To run the script, an argument for the client file path must be passed in: "python telemetry_client.py "file_path". This script is useful to see how the Telemetry API for DPDK is used, and how to make the initial connection. Signed-off-by: Ciara Power Signed-off-by: Brian Archbold --- usertools/telemetry_client.py | 116 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 usertools/telemetry_client.py diff --git a/usertools/telemetry_client.py b/usertools/telemetry_client.py new file mode 100644 index 0000000..ede778c --- /dev/null +++ b/usertools/telemetry_client.py @@ -0,0 +1,116 @@ +# SPDK-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +import socket +import os +import sys +import time + +BUFFER_SIZE = 200000 + +METRICS_REQ = "{\"action\":0,\"command\":\"ports_all_stat_values\",\"data\":null}" +API_REG = "{\"action\":1,\"command\":\"clients\",\"data\":{\"client_path\":\"" +API_UNREG = "{\"action\":2,\"command\":\"clients\",\"data\":{\"client_path\":\"" +DEFAULT_FP = "/var/run/.default_client" + +class Socket: + + def __init__(self): + self.send_fd = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + self.recv_fd = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + self.client_fd = None + + def __del__(self): + try: + self.send_fd.close() + self.recv_fd.close() + self.client_fd.close() + except: + print("Error - Sockets could not be closed") + +class Client: + + def __init__(self): # Creates a client instance + self.socket = Socket() + self.file_path = None + self.choice = None + self.unregistered = 0 + + def __del__(self): + try: + if self.unregistered == 0: + self.unregister(); + except: + print("Error - Client could not be destroyed") + + def getFilepath(self, file_path): # Gets arguments from Command-Line and assigns to instance of client + self.file_path = file_path + + def register(self): # Connects a client to DPDK-instance + if os.path.exists(self.file_path): + os.unlink(self.file_path) + try: + self.socket.recv_fd.bind(self.file_path) + except socket.error as msg: + print ("Error - Socket binding error: " + str(msg) + "\n") + self.socket.recv_fd.settimeout(2) + self.socket.send_fd.connect("/var/run/.rte_telemetry") + JSON = (API_REG + self.file_path + "\"}}") + self.socket.send_fd.sendall(JSON) + self.socket.recv_fd.listen(1) + self.socket.client_fd = self.socket.recv_fd.accept()[0] + + def unregister(self): # Unregister a given client + self.socket.client_fd.send(API_UNREG + self.file_path + "\"}}") + self.socket.client_fd.close() + + def requestMetrics(self): # Requests metrics for given client + self.socket.client_fd.send(METRICS_REQ) + data = self.socket.client_fd.recv(BUFFER_SIZE) + print "\nResponse: \n", str(data) + + def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client + print("\nPlease enter the number of times you'd like to continuously request Metrics:") + n_requests = int(input("\n:")) + print("\033[F") #Removes the user input from screen, cleans it up + print("\033[K") + for i in range(n_requests): + self.requestMetrics() + time.sleep(sleep_time) + + def interactiveMenu(self, sleep_time): # Creates Interactive menu within the script + while self.choice != 3: + print("\nOptions Menu") + print("[1] Send for Metrics for all ports") + print("[2] Send for Metrics for all ports recursively") + print("[3] Unregister client") + + try: + self.choice = int(input("\n:")) + print("\033[F") #Removes the user input for screen, cleans it up + print("\033[K") + if self.choice == 1: + self.requestMetrics() + elif self.choice == 2: + self.repeatedlyRequestMetrics(sleep_time) + elif self.choice == 3: + self.unregister() + self.unregistered = 1 + else: + print("Error - Invalid request choice") + except: + pass + +if __name__ == "__main__": + + sleep_time = 1 + file_path = "" + if (len(sys.argv) == 2): + file_path = sys.argv[1] + else: + print("Warning - No filepath passed, using default (" + DEFAULT_FP + ").") + file_path = DEFAULT_FP + client = Client() + client.getFilepath(file_path) + client.register() + client.interactiveMenu(sleep_time) From patchwork Thu Aug 23 12:08:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43821 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 081BD5B2E; Thu, 23 Aug 2018 14:08:48 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id C56CA548B for ; Thu, 23 Aug 2018 14:08:40 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613894" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:39 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:13 +0100 Message-Id: <1535026093-101872-12-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535026093-101872-1-git-send-email-ciara.power@intel.com> References: <1535026093-101872-1-git-send-email-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH 11/11] telemetry: add collectd plugin patch X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds the patch for the collectd plugin developed for use with the DPDK Telemetry library. The patch is included here to allow users to apply the patch to collectd when using DPDK Telemetry. Further details on applying the collectd patch can be found in the collectd patch commit message. The collectd plugin will be upstreamed to collectd at a later stage. Signed-off-by: Brian Archbold Signed-off-by: Emma Kenny --- ...emetry-add-plugin-for-DPDK-metrics-via-DP.patch | 578 +++++++++++++++++++++ 1 file changed, 578 insertions(+) create mode 100644 lib/librte_telemetry/collectd_plugin/v1-0001-dpdk_telemetry-add-plugin-for-DPDK-metrics-via-DP.patch diff --git a/lib/librte_telemetry/collectd_plugin/v1-0001-dpdk_telemetry-add-plugin-for-DPDK-metrics-via-DP.patch b/lib/librte_telemetry/collectd_plugin/v1-0001-dpdk_telemetry-add-plugin-for-DPDK-metrics-via-DP.patch new file mode 100644 index 0000000..e81d463 --- /dev/null +++ b/lib/librte_telemetry/collectd_plugin/v1-0001-dpdk_telemetry-add-plugin-for-DPDK-metrics-via-DP.patch @@ -0,0 +1,578 @@ +From 4c7660827b471ecd862122aa0e2a90c0a0f8ec97 Mon Sep 17 00:00:00 2001 +From: Emma Kenny +Date: Tue, 21 Aug 2018 15:49:15 +0100 +Subject: [PATCH v1] dpdk_telemetry: add plugin for DPDK metrics via DPDK + Telemetry library + +This patch introduces a new plugin for collectd, which consumes DPDK metrics +via the dpdk_telemetry library. The collectd plugin here provides an +easy way to use the DPDK telemetry API to query ethernet device metrics. + +The collectd plugin retrieves metrics from a DPDK packet forwarding +application by sending a JSON formatted message via a UNIX domain +socket. The DPDK telemetry component will respond with a JSON formatted +reply, delivering the requested metrics. The dpdk_telemetry collectd +plugin parses the JSON data, and publishes the metric values to collectd +for further use. + +This plugin has a dependency on the DPDK Telemetry library, as it must be +"in sync" with the DPDK Telemetry implementation. + +This patch should be applied on the following collectd commit: +fff795c9846bd8fe4bc7f76bcd83a2b8cefb4525 + +Signed-off-by: Emma Kenny +Signed-off-by: Brian Archbold +--- + Makefile.am | 7 + + configure.ac | 4 + + src/collectd.conf.in | 26 ++-- + src/collectd.conf.pod | 35 +++++ + src/dpdk_telemetry.c | 364 ++++++++++++++++++++++++++++++++++++++++++++++++++ + src/types.db | 1 + + 6 files changed, 427 insertions(+), 10 deletions(-) + create mode 100755 src/dpdk_telemetry.c + +diff --git a/Makefile.am b/Makefile.am +index cb40148..66885e2 100644 +--- a/Makefile.am ++++ b/Makefile.am +@@ -884,6 +884,13 @@ dpdkstat_la_CFLAGS = $(AM_CFLAGS) $(LIBDPDK_CFLAGS) + dpdkstat_la_LDFLAGS = $(PLUGIN_LDFLAGS) $(LIBDPDK_LDFLAGS) + dpdkstat_la_LIBADD = $(LIBDPDK_LIBS) + endif ++if BUILD_PLUGIN_DPDK_TELEMETRY ++pkglib_LTLIBRARIES += dpdk_telemetry.la ++dpdk_telemetry_la_SOURCES = src/dpdk_telemetry.c ++dpdk_telemetry_la_CFLAGS = $(AM_CFLAGS) ++dpdk_telemetry_la_LDFLAGS = $(PLUGIN_LDFLAGS) ++dpdk_telemetry_la_LIBADD = -ljansson ++endif + + if BUILD_PLUGIN_DRBD + pkglib_LTLIBRARIES += drbd.la +diff --git a/configure.ac b/configure.ac +index 7bf3718..93ee1a0 100644 +--- a/configure.ac ++++ b/configure.ac +@@ -6289,6 +6289,7 @@ plugin_disk="no" + plugin_drbd="no" + plugin_dpdkevents="no" + plugin_dpdkstat="no" ++plugin_dpdk_telemetry="no" + plugin_entropy="no" + plugin_ethstat="no" + plugin_fhcount="no" +@@ -6349,6 +6350,7 @@ if test "x$ac_system" = "xLinux"; then + plugin_cpufreq="yes" + plugin_disk="yes" + plugin_drbd="yes" ++ plugin_dpdk_telemetry="yes" + plugin_entropy="yes" + plugin_fhcount="yes" + plugin_fscache="yes" +@@ -6710,6 +6712,7 @@ AC_PLUGIN([dns], [$with_libpcap], [DNS traffic analysi + AC_PLUGIN([dpdkevents], [$plugin_dpdkevents], [Events from DPDK]) + AC_PLUGIN([dpdkstat], [$plugin_dpdkstat], [Stats from DPDK]) + AC_PLUGIN([drbd], [$plugin_drbd], [DRBD statistics]) ++AC_PLUGIN([dpdk_telemetry], [$plugin_dpdk_telemetry], [Metrics from DPDK Telemetry]) + AC_PLUGIN([email], [yes], [EMail statistics]) + AC_PLUGIN([entropy], [$plugin_entropy], [Entropy statistics]) + AC_PLUGIN([ethstat], [$plugin_ethstat], [Stats from NIC driver]) +@@ -7132,6 +7135,7 @@ AC_MSG_RESULT([ dns . . . . . . . . . $enable_dns]) + AC_MSG_RESULT([ dpdkevents. . . . . . $enable_dpdkevents]) + AC_MSG_RESULT([ dpdkstat . . . . . . $enable_dpdkstat]) + AC_MSG_RESULT([ drbd . . . . . . . . $enable_drbd]) ++AC_MSG_RESULT([ dpdk_telemetry. . . . $enable_dpdk_telemetry]) + AC_MSG_RESULT([ email . . . . . . . . $enable_email]) + AC_MSG_RESULT([ entropy . . . . . . . $enable_entropy]) + AC_MSG_RESULT([ ethstat . . . . . . . $enable_ethstat]) +diff --git a/src/collectd.conf.in b/src/collectd.conf.in +index af65214..8e9d500 100644 +--- a/src/collectd.conf.in ++++ b/src/collectd.conf.in +@@ -62,12 +62,12 @@ + @LOAD_PLUGIN_LOGFILE@LoadPlugin logfile + @LOAD_PLUGIN_LOG_LOGSTASH@LoadPlugin log_logstash + +-# +-# LogLevel @DEFAULT_LOG_LEVEL@ +-# File STDOUT +-# Timestamp true +-# PrintSeverity false +-# ++ ++ LogLevel @DEFAULT_LOG_LEVEL@ ++ File STDOUT ++ Timestamp true ++ PrintSeverity false ++ + + # + # LogLevel @DEFAULT_LOG_LEVEL@ +@@ -117,6 +117,7 @@ + #@BUILD_PLUGIN_DNS_TRUE@LoadPlugin dns + #@BUILD_PLUGIN_DPDKEVENTS_TRUE@LoadPlugin dpdkevents + #@BUILD_PLUGIN_DPDKSTAT_TRUE@LoadPlugin dpdkstat ++@BUILD_PLUGIN_DPDK_TELEMETRY_TRUE@LoadPlugin dpdk_telemetry + #@BUILD_PLUGIN_DRBD_TRUE@LoadPlugin drbd + #@BUILD_PLUGIN_EMAIL_TRUE@LoadPlugin email + #@BUILD_PLUGIN_ENTROPY_TRUE@LoadPlugin entropy +@@ -394,10 +395,10 @@ + # SubtractGuestState true + # + # +-# +-# DataDir "@localstatedir@/lib/@PACKAGE_NAME@/csv" +-# StoreRates false +-# ++ ++ DataDir "@localstatedir@/lib/@PACKAGE_NAME@/csv" ++ StoreRates false ++ + + # + # +@@ -595,6 +596,11 @@ + # PortName "interface2" + # + ++ ++ ClientSocketPath "/var/run/.client" ++ DpdkSocketPath "/var/run/.rte_telemetry" ++ ++ + # + # SocketFile "@localstatedir@/run/@PACKAGE_NAME@-email" + # SocketGroup "collectd" +diff --git a/src/collectd.conf.pod b/src/collectd.conf.pod +index 6e6d6ea..3cc062f 100644 +--- a/src/collectd.conf.pod ++++ b/src/collectd.conf.pod +@@ -2822,6 +2822,41 @@ convention will be used for the additional ports. + + =back + ++=head2 Plugin C ++ ++ ++The I< dpdk_telemetry > plugin collects DPDK metrics via the dpdk_telemetry library. ++ ++ ++B ++ ++ ++ ++ ClientSocketPath "/var/run/.client" ++ DpdkSocketPath "/var/run/.rte_telemetry" ++ ++ ++ ++B ++ ++ ++=over 2 ++ ++ ++=item B I ++ ++ ++The path to the client socket. ++ ++ ++=item B I ++ ++ ++The path to DPDK Telemetry. ++ ++ ++=back ++ + =head2 Plugin C + + =over 4 +diff --git a/src/dpdk_telemetry.c b/src/dpdk_telemetry.c +new file mode 100755 +index 0000000..773256e +--- /dev/null ++++ b/src/dpdk_telemetry.c +@@ -0,0 +1,364 @@ ++/*- ++ * collectd - src/dpdk_telemetry.c ++ * MIT License ++ * ++ * Copyright(c) 2018 Intel Corporation. All rights reserved. ++ * ++ * Permission is hereby granted, free of charge, to any person obtaining a copy ++ * of ++ * this software and associated documentation files (the "Software"), to deal in ++ * the Software without restriction, including without limitation the rights to ++ * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies ++ * of the Software, and to permit persons to whom the Software is furnished to ++ * do ++ * so, subject to the following conditions: ++ * ++ * The above copyright notice and this permission notice shall be included in ++ * all ++ * copies or substantial portions of the Software. ++ * ++ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ++ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ++ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE ++ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ++ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ++ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ++ * SOFTWARE. ++ * ++ */ ++ ++#include "collectd.h" ++#include "common.h" ++#include "plugin.h" ++#include "utils_time.h" ++ ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++#include ++ ++#define BUF_SIZE 1000000 ++#define PLUGIN_NAME "dpdk_telemetry" ++#define DEFAULT_DPDK_PATH "/var/run/.rte_telemetry" ++#define DEFAULT_CLIENT_PATH "/var/run/.client" ++ ++struct client_info { ++ int s_send; ++ int s_recv; ++ int fd; ++ const char *dpdk_path; ++ const char *client_path; ++ struct sockaddr_un addr; ++ struct sockaddr_un addrs; ++}; ++ ++static struct client_info *client = NULL; ++static char g_client_path[BUF_SIZE]; ++static char g_dpdk_path[BUF_SIZE]; ++ ++static int dpdk_telemetry_config(oconfig_item_t *ci) { ++ int ret, i; ++ ++ INFO(PLUGIN_NAME ": %s:%d", __FUNCTION__, __LINE__); ++ ++ for (i = 0; i < ci->children_num; i++) { ++ oconfig_item_t *child = ci->children + i; ++ ++ if (strcasecmp("ClientSocketPath", child->key) == 0) { ++ ret = cf_util_get_string_buffer(child, g_client_path, ++ sizeof(g_client_path)); ++ } else if (strcasecmp("DpdkSocketPath", child->key) == 0) { ++ ret = cf_util_get_string_buffer(child, g_dpdk_path, sizeof(g_dpdk_path)); ++ } else { ++ ERROR(PLUGIN_NAME ": Unknown configuration parameter" ++ "\"%s\"", ++ child->key); ++ ret = -1; ++ } ++ ++ if (ret < 0) { ++ INFO(PLUGIN_NAME ": %s:%d ret =%d", __FUNCTION__, __LINE__, ret); ++ return ret; ++ } ++ } ++ return 0; ++} ++ ++static int dpdk_telemetry_parse(json_t *stats, json_t *port, int portid) { ++ json_t *statsArrayObj; ++ if (!stats) { ++ ERROR("dpdk_telemetry: Stats pointer is invalid\n"); ++ return -1; ++ } ++ ++ if (!port) { ++ ERROR("dpdk_telemetry: Port pointer is invalid\n"); ++ return -1; ++ } ++ ++ if (portid < 0) { ++ ERROR("dpdk_telemetry: portid is invalid\n"); ++ return -1; ++ } ++ ++ json_t *name, *value; ++ const char *name_string; ++ int value_int, statslen, i; ++ statslen = json_array_size(stats); ++ for (i = 0; i < statslen; i++) { ++ statsArrayObj = json_array_get(stats, i); ++ name = json_object_get(statsArrayObj, "name"); ++ value = json_object_get(statsArrayObj, "value"); ++ if (!name) { ++ ERROR("dpdk_telemetry: Request does not have name field\n"); ++ return -1; ++ } ++ if (!json_is_string(name)) { ++ ERROR("dpdk_telemetry: Stat name value is not a string\n"); ++ return -1; ++ } ++ name_string = json_string_value(name); ++ if (!value) { ++ ERROR("dpdk_telemetry: Request does not have value name\n"); ++ return -1; ++ } ++ if (!json_is_integer(value)) { ++ ERROR("dpdk_telemetry: Stat value is not an integer\n"); ++ return -1; ++ } ++ ++ char dev_name[BUF_SIZE]; ++ snprintf(dev_name, sizeof(dev_name), "%s.%d", name_string, portid); ++ value_int = json_integer_value(value); ++ value_t dpdk_telemetry_values[1]; ++ value_list_t dpdk_telemetry_vl = VALUE_LIST_INIT; ++ dpdk_telemetry_values[0].counter = value_int; ++ dpdk_telemetry_vl.values = dpdk_telemetry_values; ++ dpdk_telemetry_vl.values_len = 1; ++ dpdk_telemetry_vl.time = cdtime(); ++ snprintf(dpdk_telemetry_vl.host, sizeof(dpdk_telemetry_vl.host), "%s", ++ hostname_g); ++ snprintf(dpdk_telemetry_vl.plugin, sizeof(dpdk_telemetry_vl.plugin), ++ "dpdk_telemetry"); ++ snprintf(dpdk_telemetry_vl.plugin_instance, ++ sizeof(dpdk_telemetry_vl.plugin_instance), "%s", dev_name); ++ snprintf(dpdk_telemetry_vl.type, sizeof(dpdk_telemetry_vl.type), ++ "dpdk_telemetry"); ++ snprintf(dpdk_telemetry_vl.type_instance, ++ sizeof(dpdk_telemetry_vl.type_instance), "%s", name_string); ++ ++ int ret = plugin_dispatch_values(&dpdk_telemetry_vl); ++ if (ret < 0) { ++ ERROR("dpdk_telemetry: Failed to dispatch values"); ++ return -1; ++ } ++ } ++ return 0; ++} ++ ++static int parse_json(char *buf) { ++ ++ if (!buf) { ++ ERROR("dpdk_telemetry: buf pointer is invalid\n"); ++ return -1; ++ } ++ json_error_t error; ++ json_t *root = json_loads(buf, 0, &error); ++ int arraylen, i; ++ json_t *status, *dataArray, *stats, *dataArrayObj; ++ stats = NULL; ++ ++ if (!root) { ++ ERROR("dpdk_telemetry: Could not load JSON object from data passed in" ++ " : %s\n", ++ error.text); ++ return -1; ++ } else if (!json_is_object(root)) { ++ ERROR("dpdk_telemetry: JSON Request is not a JSON object\n"); ++ json_decref(root); ++ return -1; ++ } ++ ++ status = json_object_get(root, "status_code"); ++ if (!status) { ++ ERROR("dpdk_telemetry: Request does not have status field\n"); ++ return -1; ++ } else if (!json_is_string(status)) { ++ ERROR("dpdk_telemetry: Status value is not a string\n"); ++ return -1; ++ } ++ dataArray = json_object_get(root, "data"); ++ if (!dataArray) { ++ ERROR("dpdk_telemetry: Request does not have data field\n"); ++ return -1; ++ } ++ arraylen = json_array_size(dataArray); ++ if (!arraylen) { ++ ERROR("dpdk_telemetry: No data to get\n"); ++ return -1; ++ } ++ ++ for (i = 0; i < arraylen; i++) { ++ json_t *port; ++ dataArrayObj = json_array_get(dataArray, i); ++ port = json_object_get(dataArrayObj, "port"); ++ stats = json_object_get(dataArrayObj, "stats"); ++ if (!port) { ++ ERROR("dpdk_telemetry: Request does not have port field\n"); ++ return -1; ++ } ++ if (!json_is_integer(port)) { ++ ERROR("dpdk_telemetry: Port value is not an integer\n"); ++ return -1; ++ } ++ ++ if (!stats) { ++ ERROR("dpdk_telemetry: Request does not have stats field\n"); ++ return -1; ++ } ++ dpdk_telemetry_parse(stats, port, i); ++ } ++ return 0; ++} ++ ++static int dpdk_telemetry_cleanup(void) { ++ if (!client) { ++ WARNING("dpdk_telemetry: instance pointer is NULL, cleanup() has already " ++ "been called\n"); ++ return -1; ++ } ++ close(client->s_send); ++ close(client->s_recv); ++ close(client->fd); ++ free(client); ++ client = NULL; ++ return 0; ++} ++ ++static int dpdk_telemetry_read(user_data_t *ud) { ++ INFO(PLUGIN_NAME ": %s:%d", __FUNCTION__, __LINE__); ++ struct client_info *client = (struct client_info *)ud->data; ++ char buffer[BUF_SIZE]; ++ int bytes, ret; ++ char *json_string = "{\"action\":0,\"command\":" ++ "\"ports_all_stat_values\",\"data\":null}"; ++ if (send(client->fd, json_string, strlen(json_string), 0) < 0) { ++ ERROR("dpdk_telemetry: Could not send stats\n"); ++ return -1; ++ } ++ bytes = recv(client->fd, buffer, sizeof(buffer), 0); ++ buffer[bytes] = '\0'; ++ if (bytes < 0) { ++ ERROR("dpdk_telemetry: Could not receive stats\n"); ++ return -1; ++ } ++ ret = parse_json(buffer); ++ if (ret < 0) { ++ ERROR("dpdk_telemetry: Parsing failed\n"); ++ return -1; ++ } ++ return 0; ++} ++ ++static int dpdk_telemetry_init(void) { ++ INFO(PLUGIN_NAME ": %s:%d", __FUNCTION__, __LINE__); ++ char message[BUF_SIZE]; ++ ++ client = calloc(1, sizeof(struct client_info)); ++ if (!client) { ++ ERROR("dpdk_telemetry: Memory could not be allocated\n"); ++ return -1; ++ } ++ /*Here we look up the length of the g_dpdk_path string ++ * If it has a length we use it, otherwise we fall back to default ++ * See dpdk_telemetry_config() for details ++ */ ++ client->dpdk_path = (strlen(g_dpdk_path)) ? g_dpdk_path : DEFAULT_DPDK_PATH; ++ client->client_path = ++ (strlen(g_client_path)) ? g_client_path : DEFAULT_CLIENT_PATH; ++ client->s_send = socket(AF_UNIX, SOCK_STREAM, 0); ++ if (client->s_send < 0) { ++ ERROR("dpdk_telemetry: Failed to open socket\n"); ++ dpdk_telemetry_cleanup(); ++ return -1; ++ } ++ client->s_recv = socket(AF_UNIX, SOCK_STREAM, 0); ++ if (client->s_recv < 0) { ++ ERROR("dpdk_telemetry: Failed to open message socket\n"); ++ dpdk_telemetry_cleanup(); ++ return -1; ++ } ++ client->addr.sun_family = AF_UNIX; ++ snprintf(client->addr.sun_path, sizeof(client->addr.sun_path), "%s", ++ client->dpdk_path); ++ if (connect(client->s_send, (struct sockaddr *)&client->addr, ++ sizeof(client->addr)) < 0) { ++ ERROR("dpdk_telemetry: Failed to connect\n"); ++ dpdk_telemetry_cleanup(); ++ return -1; ++ } ++ client->addrs.sun_family = AF_UNIX; ++ snprintf(client->addrs.sun_path, sizeof(client->addrs.sun_path), "%s", ++ client->client_path); ++ unlink(client->client_path); ++ if (bind(client->s_recv, (struct sockaddr *)&client->addrs, ++ sizeof(client->addrs)) < 0) { ++ ERROR("dpdk_telemetry: Failed to bind\n"); ++ dpdk_telemetry_cleanup(); ++ return -1; ++ } ++ if (listen(client->s_recv, 1) < 0) { ++ ERROR("dpdk_telemetry: Listen failed\n"); ++ dpdk_telemetry_cleanup(); ++ return -1; ++ } ++ snprintf(message, sizeof(message), "{\"action\":1,\"command\":\"clients\"" ++ ",\"data\":{\"client_path\":\"%s\"}}", ++ client->client_path); ++ if (send(client->s_send, message, strlen(message), 0) < 0) { ++ ERROR("dpdk_telemetry: Could not send register message\n"); ++ dpdk_telemetry_cleanup(); ++ return -1; ++ } ++ client->fd = accept(client->s_recv, NULL, NULL); ++ if (client->fd < 0) { ++ ERROR("dpdk_telemetry: Failed to accept\n"); ++ dpdk_telemetry_cleanup(); ++ return -1; ++ } ++ user_data_t ud; ++ memset(&ud, 0, sizeof(ud)); ++ ud.data = (void *)client; ++ plugin_register_complex_read(NULL, "dpdk_telemetry", dpdk_telemetry_read, 0, ++ &ud); ++ return 0; ++} ++ ++static int dpdk_telemetry_shutdown(void) { ++ INFO(PLUGIN_NAME ": %s:%d", __FUNCTION__, __LINE__); ++ char msg[BUF_SIZE]; ++ int ret; ++ ++ snprintf(msg, sizeof(msg), "{\"action\":2,\"command\":\"clients\"" ++ ",\"data\":{\"client_path\":\"%s\"}}", ++ client->client_path); ++ ret = send(client->fd, msg, strlen(msg), 0); ++ if (ret < 0) { ++ ERROR("dpdk_telemetry: Could not send unregister message\n"); ++ dpdk_telemetry_cleanup(); ++ return -1; ++ } ++ dpdk_telemetry_cleanup(); ++ return 0; ++} ++ ++void module_register(void) { ++ plugin_register_init("dpdk_telemetry", dpdk_telemetry_init); ++ plugin_register_complex_config("dpdk_telemetry", dpdk_telemetry_config); ++ plugin_register_shutdown("dpdk_telemetry", dpdk_telemetry_shutdown); ++} +diff --git a/src/types.db b/src/types.db +index f4933ee..4517b3c 100644 +--- a/src/types.db ++++ b/src/types.db +@@ -76,6 +76,7 @@ dns_transfer value:DERIVE:0:U + dns_update value:DERIVE:0:U + dns_zops value:DERIVE:0:U + domain_state state:GAUGE:0:U, reason:GAUGE:0:U ++dpdk_telemetry value:DERIVE:0:U + drbd_resource value:DERIVE:0:U + duration seconds:GAUGE:0:U + email_check value:GAUGE:0:U +-- +2.9.5 +