From patchwork Thu Aug 23 12:08:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Power, Ciara" X-Patchwork-Id: 43810 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DC49E2BF7; Thu, 23 Aug 2018 14:08:26 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 8F9BA2BF7 for ; Thu, 23 Aug 2018 14:08:25 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Aug 2018 05:08:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,278,1531810800"; d="scan'208";a="256613820" Received: from silpixa00399503.ir.intel.com ([10.237.222.102]) by fmsmga005.fm.intel.com with ESMTP; 23 Aug 2018 05:08:22 -0700 From: Ciara Power To: harry.van.haaren@intel.com, brian.archbold@intel.com, emma.kenny@intel.com, ciara.power@intel.com Cc: dev@dpdk.org Date: Thu, 23 Aug 2018 13:08:02 +0100 Message-Id: <1535026093-101872-1-git-send-email-ciara.power@intel.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 00/11] introduce telemetry library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patchset introduces a Telemetry library for DPDK Service Assurance. This library provides an easy way to query DPDK Ethdev metrics. The telemetry library provides a method for a service assurance component to retrieve metrics from a DPDK packet forwarding application. Communicating from the service assurance component to DPDK is done using a UNIX domain socket, passing a JSON formatted string. A reply is sent (again a JSON formatted string) of the current DPDK metrics. The telemetry component makes use of the existing rte_metrics library to query values. The values to be transmitted via the telemetry infrastructure must be present in the Metrics library. Currently the ethdev values are pushed to the metrics library, and the queried from there – there is an open question on how applications would like this to occur. Currently only ethdev to metrics functionality is implemented, however other subsystems like crypto, eventdev, keepalive etc can use similar mechanisms. Exposing DPDK Telemetry via a socket interface enables service assurance agents like collectd to consume data from DPDK. This is vital for monitoring, fault-detection, and error reporting. A collectd plugin has been created to interact with the DPDK Telemetry component, showing how it can be used in practice. The collectd plugin patch can be found at lib/librte_telemetry/collectd_plugin/v1-0001-dpdk_telemetry-add-plugin- for-DPDK-metrics-via-DP.patch, this will be upstreamed to collectd at a later stage. Further instructions on applying the collectd patch can be found in the collectd patch commit message. A small python script is provided in ./usertools/telemetry_client.py to quick-start using DPDK Telemetry. Ciara Power and Brian Archbold (10): telemetry: initial telemetry infrastructure telemetry: add initial connection socket telemetry: add client feature and sockets telemetry: add parser for client socket messages telemetry: update metrics before sending stats telemetry: format json response when sending stats telemetry: add tests for telemetry api telemetry: add vdev kvargs for selftest doc: add telemetry documentation usertools: add client python script for telemetry Emma Kenny (1) : telemetry: add collectd plugin patch config/common_base | 5 + doc/guides/howto/index.rst | 1 + doc/guides/howto/telemetry.rst | 86 + drivers/Makefile | 3 + drivers/meson.build | 3 +- drivers/telemetry/Makefile | 8 + drivers/telemetry/meson.build | 6 + drivers/telemetry/telemetry/Makefile | 26 + drivers/telemetry/telemetry/meson.build | 5 + .../telemetry/rte_pmd_telemetry_version.map | 9 + drivers/telemetry/telemetry/telemetry_driver.c | 83 + lib/Makefile | 2 + lib/librte_telemetry/Makefile | 28 + ...emetry-add-plugin-for-DPDK-metrics-via-DP.patch | 578 +++++++ lib/librte_telemetry/meson.build | 9 + lib/librte_telemetry/rte_telemetry.c | 1737 ++++++++++++++++++++ lib/librte_telemetry/rte_telemetry.h | 58 + lib/librte_telemetry/rte_telemetry_internal.h | 72 + lib/librte_telemetry/rte_telemetry_parser.c | 604 +++++++ lib/librte_telemetry/rte_telemetry_parser.h | 13 + lib/librte_telemetry/rte_telemetry_parser_test.c | 574 +++++++ lib/librte_telemetry/rte_telemetry_parser_test.h | 39 + lib/librte_telemetry/rte_telemetry_socket_tests.h | 36 + lib/librte_telemetry/rte_telemetry_version.map | 7 + lib/meson.build | 2 +- mk/rte.app.mk | 2 + usertools/telemetry_client.py | 116 ++ 27 files changed, 4110 insertions(+), 2 deletions(-) create mode 100644 doc/guides/howto/telemetry.rst create mode 100644 drivers/telemetry/Makefile create mode 100644 drivers/telemetry/meson.build create mode 100644 drivers/telemetry/telemetry/Makefile create mode 100644 drivers/telemetry/telemetry/meson.build create mode 100644 drivers/telemetry/telemetry/rte_pmd_telemetry_version.map create mode 100644 drivers/telemetry/telemetry/telemetry_driver.c create mode 100644 lib/librte_telemetry/Makefile create mode 100644 lib/librte_telemetry/collectd_plugin/v1-0001-dpdk_telemetry-add-plugin-for-DPDK-metrics-via-DP.patch create mode 100644 lib/librte_telemetry/meson.build create mode 100644 lib/librte_telemetry/rte_telemetry.c create mode 100644 lib/librte_telemetry/rte_telemetry.h create mode 100644 lib/librte_telemetry/rte_telemetry_internal.h create mode 100644 lib/librte_telemetry/rte_telemetry_parser.c create mode 100644 lib/librte_telemetry/rte_telemetry_parser.h create mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c create mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.h create mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h create mode 100644 lib/librte_telemetry/rte_telemetry_version.map create mode 100644 usertools/telemetry_client.py