From patchwork Fri Sep 9 09:35:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116131 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EB6C0A0540; Fri, 9 Sep 2022 11:36:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6DC6442B71; Fri, 9 Sep 2022 11:35:51 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 1CD2541141 for ; Fri, 9 Sep 2022 11:35:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716147; x=1694252147; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8gY7ypd+bbmXlOdpfNoiEm8gtMeL86IWU2icgCarunQ=; b=hbng/ETeODLrq2IHDtBVZut/Su2woWeNIXpKxa8epo3usSAo5jNIvbbS 5A1yCLnTBlDEevdP8c3XJiAphj8Hs/fQt1R2TfFBFCQC5FTA1MiCZsJTl dtRSzdvVvGIDOaAqt0z92tE/FVf9pfPlzYycK4JayO81EbgDnHcYLRTqk SnBfeBytazYWYWFNsg3bhOMc0DomuIXZ++VwkDgxj0zWCYBaqF1ogKxjD lFgf0SVCnbHMdXivI6A/aNtMNUt0/+VPT9pE4Blfmob/QzB9/jxpI84/+ /QGo8YTe/q9OK5RDSzGj9LKki0DIIjScaNfBrXENBEbFZAsWZ8cggHDB2 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="298241087" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="298241087" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996349" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:39 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 01/13] telemetry: limit characters allowed in dictionary names Date: Fri, 9 Sep 2022 10:35:11 +0100 Message-Id: <20220909093523.471727-2-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org To save issues with encoding the names of values in dicts, we limit the allowed names to a subset of character values. This list of allowed characters can be expanded as necessary in future. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- lib/telemetry/rte_telemetry.h | 8 ++++++++ lib/telemetry/telemetry_data.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/lib/telemetry/rte_telemetry.h b/lib/telemetry/rte_telemetry.h index d586dd0fc1..a0d21d6b7f 100644 --- a/lib/telemetry/rte_telemetry.h +++ b/lib/telemetry/rte_telemetry.h @@ -64,6 +64,10 @@ rte_tel_data_start_array(struct rte_tel_data *d, enum rte_tel_value_type type); /** * Start a dictionary of values for returning from a callback * + * Dictionaries consist of key-values pairs to be returned, where the keys, + * or names, are strings and the values can be any of the types supported by telemetry. + * Name strings may only contain alphanumeric characters as well as '_' or '/' + * * @param d * The data structure passed to the callback * @return @@ -159,6 +163,7 @@ rte_tel_data_add_array_container(struct rte_tel_data *d, * The data structure passed to the callback * @param name * The name the value is to be stored under in the dict + * Must contain only alphanumeric characters or the symbols: '_' or '/' * @param val * The string to be stored in the dict * @return @@ -177,6 +182,7 @@ rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, * The data structure passed to the callback * @param name * The name the value is to be stored under in the dict + * Must contain only alphanumeric characters or the symbols: '_' or '/' * @param val * The number to be stored in the dict * @return @@ -193,6 +199,7 @@ rte_tel_data_add_dict_int(struct rte_tel_data *d, const char *name, int val); * The data structure passed to the callback * @param name * The name the value is to be stored under in the dict + * Must contain only alphanumeric characters or the symbols: '_' or '/' * @param val * The number to be stored in the dict * @return @@ -212,6 +219,7 @@ rte_tel_data_add_dict_u64(struct rte_tel_data *d, * The data structure passed to the callback * @param name * The name the value is to be stored under in the dict. + * Must contain only alphanumeric characters or the symbols: '_' or '/' * @param val * The pointer to the container to be stored in the dict. * @param keep diff --git a/lib/telemetry/telemetry_data.c b/lib/telemetry/telemetry_data.c index e14ae3c4d4..b5cd74b25b 100644 --- a/lib/telemetry/telemetry_data.c +++ b/lib/telemetry/telemetry_data.c @@ -3,6 +3,8 @@ */ #undef RTE_USE_LIBBSD +#include + #include #include "telemetry_data.h" @@ -92,6 +94,24 @@ rte_tel_data_add_array_container(struct rte_tel_data *d, return 0; } +static bool +valid_name(const char *name) +{ + char allowed[128] = { + ['0' ... '9'] = 1, + ['A' ... 'Z'] = 1, + ['a' ... 'z'] = 1, + ['_'] = 1, + ['/'] = 1, + }; + while (*name != '\0') { + if ((size_t)*name >= RTE_DIM(allowed) || allowed[(int)*name] == 0) + return false; + name++; + } + return true; +} + int rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, const char *val) @@ -104,6 +124,9 @@ rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) return -ENOSPC; + if (!valid_name(name)) + return -EINVAL; + d->data_len++; e->type = RTE_TEL_STRING_VAL; vbytes = strlcpy(e->value.sval, val, RTE_TEL_MAX_STRING_LEN); @@ -123,6 +146,9 @@ rte_tel_data_add_dict_int(struct rte_tel_data *d, const char *name, int val) if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) return -ENOSPC; + if (!valid_name(name)) + return -EINVAL; + d->data_len++; e->type = RTE_TEL_INT_VAL; e->value.ival = val; @@ -140,6 +166,9 @@ rte_tel_data_add_dict_u64(struct rte_tel_data *d, if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) return -ENOSPC; + if (!valid_name(name)) + return -EINVAL; + d->data_len++; e->type = RTE_TEL_U64_VAL; e->value.u64val = val; @@ -161,6 +190,9 @@ rte_tel_data_add_dict_container(struct rte_tel_data *d, const char *name, if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) return -ENOSPC; + if (!valid_name(name)) + return -EINVAL; + d->data_len++; e->type = RTE_TEL_CONTAINER; e->value.container.data = val; From patchwork Fri Sep 9 09:35:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116132 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A06CA0540; Fri, 9 Sep 2022 11:36:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 46ECD42B75; Fri, 9 Sep 2022 11:35:52 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id E499E4003F for ; Fri, 9 Sep 2022 11:35:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716149; x=1694252149; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=meo/0pHiYWPh3MuMPnK0dfOYgBnhXJTnsuJS8dOq1pk=; b=JEuvBVzIdncbULqlZvEZxeYqHkZs5nAM8uxwFTYmXQdJf+jaPCdjEgZh mAUwn0tTaAzfmLv89xDKa+WZXT+QsCteCFpdOp4Bdn4USj/fUTk598/jC ECFmRntWl9nOcxYc/6Qi8qTluQgaIJKjhYZr9ZUqS7nA/GVbvdMJ5tIzl yRacg0PqIZuvWyIelFHhPYUi0DW41Ns3aO5vCajPDTed3ZQBMPcDxfgsa 49Ss1JpptKtSG8WSjSGgiFq38AsxRQX1dWcBQ0zTUTdfEKRKFgzngwUTV 1Qu0A8ISsl8UcDhRIhd4IUFd+uXJdzSUTevuSOBeAMas+ax1wZX6Fz/6y g==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="298241089" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="298241089" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996353" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:40 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 02/13] test/telemetry_json: print success or failure per subtest Date: Fri, 9 Sep 2022 10:35:12 +0100 Message-Id: <20220909093523.471727-3-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org rather than just printing out success or failure at the end of the test only, print out "OK" or "ERROR" for each individual test case within the overall test. As part of this, ensure each case returns 0 on success and any other value on failure. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- app/test/test_telemetry_json.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c index 790181d316..748b7cfe5a 100644 --- a/app/test/test_telemetry_json.c +++ b/app/test/test_telemetry_json.c @@ -102,8 +102,10 @@ test_large_array_element(void) used = rte_tel_json_add_array_string(buf, sizeof(buf), used, str); printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != 0) + return -1; - return strlen(buf) != 0; + return strncmp(expected, buf, sizeof(buf)); } static int @@ -117,20 +119,33 @@ test_large_obj_element(void) used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, str, 0); printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != 0) + return -1; - return strlen(buf) != 0; + return strncmp(expected, buf, sizeof(buf)); } +typedef int (*test_fn)(void); + static int test_telemetry_json(void) { - if (test_basic_array() < 0 || - test_basic_obj() < 0 || - test_overflow_array() < 0 || - test_overflow_obj() < 0 || - test_large_array_element() < 0 || - test_large_obj_element() < 0) - return -1; + unsigned int i; + test_fn fns[] = { + test_basic_array, + test_basic_obj, + test_overflow_array, + test_overflow_obj, + test_large_array_element, + test_large_obj_element, + }; + for (i = 0; i < RTE_DIM(fns); i++) + if (fns[i]() == 0) + printf("OK\n"); + else { + printf("ERROR\n"); + return -1; + } return 0; } From patchwork Fri Sep 9 09:35:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116127 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 451CDA00C5; Fri, 9 Sep 2022 11:35:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DEB0340A7E; Fri, 9 Sep 2022 11:35:45 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 296164003F for ; Fri, 9 Sep 2022 11:35:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716144; x=1694252144; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eNnDeAG62U3mZwyXgXT6/8QAGFCvQwMYYrVwb1JwZNE=; b=hXo1c/DxW0dL1Yw7hzprR67AMOJQSTOjkpXuDHUNCo6ZP2IR7snBj3i/ j105snlV3i+0dABspfZuUf6iRV2hSdSFr5Bu6jkasqD90mMrRj9gWLToc N0noMruUhsplN+bNb8Qc6aLDPeT2GLhHrdeOZQPe0HFa3NdvbVRh/h89W qf9YKcvSQohvXvBfPpTeIdjZN6NHIm0KOJHP0CogDArFEm6ooELjlMaqi lRBucpfs1UW5WozG7IkQ7LoE9baXSVZMpFMDg8iNN1HcvMLyxiyVU+s4b pQNay/DaSWVRzlPAi1VL77oueLR7W4qEp5ryzq4UhHYNQMdBE/MQqD42o w==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437120" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437120" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996357" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:41 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 03/13] telemetry: fix escaping of invalid json characters Date: Fri, 9 Sep 2022 10:35:13 +0100 Message-Id: <20220909093523.471727-4-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For string values returned from telemetry, escape any values that cannot normally appear in a json string. According to the json spec[1], the characters than need to be handled are control chars (char value < 0x20) and '"' and '\' characters. To handle this, we replace the snprintf call with a separate string copying and encapsulation routine which checks each character as it copies it to the final array. [1] https://www.rfc-editor.org/rfc/rfc8259.txt Fixes: 6dd571fd07c3 ("telemetry: introduce new functionality") Bugzilla ID: 1037 Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- lib/telemetry/telemetry.c | 11 +++++--- lib/telemetry/telemetry_json.h | 48 +++++++++++++++++++++++++++++++++- 2 files changed, 55 insertions(+), 4 deletions(-) diff --git a/lib/telemetry/telemetry.c b/lib/telemetry/telemetry.c index c6fd03a5ab..7188b1905c 100644 --- a/lib/telemetry/telemetry.c +++ b/lib/telemetry/telemetry.c @@ -232,9 +232,14 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s) MAX_CMD_LEN, cmd ? cmd : "none"); break; case RTE_TEL_STRING: - used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":\"%.*s\"}", - MAX_CMD_LEN, cmd, - RTE_TEL_MAX_SINGLE_STRING_LEN, d->data.str); + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", + MAX_CMD_LEN, cmd); + cb_data_buf = &out_buf[prefix_used]; + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + + used = rte_tel_json_str(cb_data_buf, buf_len, 0, d->data.str); + used += prefix_used; + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); break; case RTE_TEL_DICT: prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", diff --git a/lib/telemetry/telemetry_json.h b/lib/telemetry/telemetry_json.h index db70690274..13df5d07e3 100644 --- a/lib/telemetry/telemetry_json.h +++ b/lib/telemetry/telemetry_json.h @@ -44,6 +44,52 @@ __json_snprintf(char *buf, const int len, const char *format, ...) return 0; /* nothing written or modified */ } +static const char control_chars[0x20] = { + ['\n'] = 'n', + ['\r'] = 'r', + ['\t'] = 't', +}; + +/** + * @internal + * Does the same as __json_snprintf(buf, len, "\"%s\"", str) + * except that it does proper escaping as necessary. + * Drops any invalid characters we don't support + */ +static inline int +__json_format_str(char *buf, const int len, const char *str) +{ + char tmp[len]; + int tmpidx = 0; + + tmp[tmpidx++] = '"'; + while (*str != '\0') { + if (*str < (int)RTE_DIM(control_chars)) { + int idx = *str; /* compilers don't like char type as index */ + if (control_chars[idx] != 0) { + tmp[tmpidx++] = '\\'; + tmp[tmpidx++] = control_chars[idx]; + } + } else if (*str == '"' || *str == '\\') { + tmp[tmpidx++] = '\\'; + tmp[tmpidx++] = *str; + } else + tmp[tmpidx++] = *str; + /* we always need space for closing quote and null character. + * Ensuring at least two free characters also means we can always take an + * escaped character like "\n" without overflowing + */ + if (tmpidx > len - 2) + return 0; + str++; + } + tmp[tmpidx++] = '"'; + tmp[tmpidx] = '\0'; + + strcpy(buf, tmp); + return tmpidx; +} + /* Copies an empty array into the provided buffer. */ static inline int rte_tel_json_empty_array(char *buf, const int len, const int used) @@ -62,7 +108,7 @@ rte_tel_json_empty_obj(char *buf, const int len, const int used) static inline int rte_tel_json_str(char *buf, const int len, const int used, const char *str) { - return used + __json_snprintf(buf + used, len - used, "\"%s\"", str); + return used + __json_format_str(buf + used, len - used, str); } /* Appends a string into the JSON array in the provided buffer. */ From patchwork Fri Sep 9 09:35:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116129 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 07B3CA00C5; Fri, 9 Sep 2022 11:35:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8835E42802; Fri, 9 Sep 2022 11:35:49 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id BF6B44003F for ; Fri, 9 Sep 2022 11:35:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716144; x=1694252144; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gZWLrdtU0OzHT+9MXNKB2CVPrd9knLEq6KTVr/zgowQ=; b=PfoYLQmwPrRXOIGB7tCqJrki974YhLBFgJzqcFqucdCfMR2o/IMRJbH6 H1Y7PJIJ1zrAsB3ClNptEXYuPduJJ0I5vcbtDcStKN3PokEwKlVp7Dvs+ 8fM6aUaTOPL/HBM/P2QHPcxl+8nXQV8rSNOfJLZ9npvG1NyeShZf1APVm 0hrEjV5/f2cFv5p6DnGo6Oe13eQDjeyk8wz5nCoQJ14gG9RNqU2CvT+xi zFXt4YLg55orhRupKSYIyjhdtmhSRFKAc3dgD1viwLEsgfG7FeDDL+s8T s/1TAq0psZiZCTwh1wRDidYM3VTx/H9rURO11qd+3xe+zh+fVBGcXH6ck g==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437123" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437123" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996364" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:43 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 04/13] test/telemetry_json: add test for string character escaping Date: Fri, 9 Sep 2022 10:35:14 +0100 Message-Id: <20220909093523.471727-5-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add unit test to validate that when creating a string response in json, that characters such as \n or quotes are properly escaped. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- app/test/test_telemetry_json.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c index 748b7cfe5a..955c2e5b1b 100644 --- a/app/test/test_telemetry_json.c +++ b/app/test/test_telemetry_json.c @@ -125,6 +125,22 @@ test_large_obj_element(void) return strncmp(expected, buf, sizeof(buf)); } +static int +test_string_char_escaping(void) +{ + static const char str[] = "A string across\ntwo lines and \"with quotes\"!"; + const char *expected = "\"A string across\\ntwo lines and \\\"with quotes\\\"!\""; + char buf[sizeof(str) + 10]; + int used = 0; + + used = rte_tel_json_str(buf, sizeof(buf), used, str); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != (int)strlen(expected)) + return -1; + + return strncmp(expected, buf, sizeof(buf)); +} + typedef int (*test_fn)(void); static int @@ -138,6 +154,7 @@ test_telemetry_json(void) test_overflow_obj, test_large_array_element, test_large_obj_element, + test_string_char_escaping, }; for (i = 0; i < RTE_DIM(fns); i++) if (fns[i]() == 0) From patchwork Fri Sep 9 09:35:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116130 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1065FA0540; Fri, 9 Sep 2022 11:36:05 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AEB1E42905; Fri, 9 Sep 2022 11:35:50 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 4F0B141155 for ; Fri, 9 Sep 2022 11:35:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716147; x=1694252147; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Wc4msroVjF7qakmFzGILlAml5a7kYcygtN+xKqYE3pE=; b=ibKfyboAkmbzR/JMv8UKQgQxAlG40NzAJdRKKUagrKhiYiBK3ejYt6N7 QhvYEASX7+tBBreTKtCP0qt0UzEEiUax3QYYLuPmL/yuc7Pc5a7nMYxfr 9dBYUeKUdyQonEI/fPLU/55XxL0k9bqmVafA6izd6LVCmpZXMUvgikRaX fmFXXol2SWeNN/Z7hiRiYwo41DF8haJQCm9vtpLfwyWk4qccEZuLCyx7B XiuE30uadrQwDtOPXbG1WEGwKCY12AodAd1/RHb1ZnHkP1i9l1Bv4nD/o famCKa6tMHVLR+c4OuwW1t+sBaa06CYgA0lhiJ0SpRVpVUwUYE5lTeQ73 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437126" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437126" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996375" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:44 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 05/13] telemetry: add escaping of strings in arrays Date: Fri, 9 Sep 2022 10:35:15 +0100 Message-Id: <20220909093523.471727-6-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When strings are added to an array variable, we need to properly escape the invalid json characters in the strings. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- lib/telemetry/telemetry_json.h | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/lib/telemetry/telemetry_json.h b/lib/telemetry/telemetry_json.h index 13df5d07e3..c4442a0bf0 100644 --- a/lib/telemetry/telemetry_json.h +++ b/lib/telemetry/telemetry_json.h @@ -52,17 +52,22 @@ static const char control_chars[0x20] = { /** * @internal - * Does the same as __json_snprintf(buf, len, "\"%s\"", str) - * except that it does proper escaping as necessary. + * This function acts the same as __json_snprintf(buf, len, "%s%s%s", prefix, str, suffix) + * except that it does proper escaping of "str" as necessary. Prefix and suffix should be compile- + * time constants not needing escaping. * Drops any invalid characters we don't support */ static inline int -__json_format_str(char *buf, const int len, const char *str) +__json_format_str(char *buf, const int len, const char *prefix, const char *str, const char *suffix) { char tmp[len]; int tmpidx = 0; - tmp[tmpidx++] = '"'; + while (*prefix != '\0' && tmpidx < len) + tmp[tmpidx++] = *prefix++; + if (tmpidx >= len) + return 0; + while (*str != '\0') { if (*str < (int)RTE_DIM(control_chars)) { int idx = *str; /* compilers don't like char type as index */ @@ -75,7 +80,7 @@ __json_format_str(char *buf, const int len, const char *str) tmp[tmpidx++] = *str; } else tmp[tmpidx++] = *str; - /* we always need space for closing quote and null character. + /* we always need space for (at minimum) closing quote and null character. * Ensuring at least two free characters also means we can always take an * escaped character like "\n" without overflowing */ @@ -83,7 +88,12 @@ __json_format_str(char *buf, const int len, const char *str) return 0; str++; } - tmp[tmpidx++] = '"'; + + while (*suffix != '\0' && tmpidx < len) + tmp[tmpidx++] = *suffix++; + if (tmpidx >= len) + return 0; + tmp[tmpidx] = '\0'; strcpy(buf, tmp); @@ -108,7 +118,7 @@ rte_tel_json_empty_obj(char *buf, const int len, const int used) static inline int rte_tel_json_str(char *buf, const int len, const int used, const char *str) { - return used + __json_format_str(buf + used, len - used, str); + return used + __json_format_str(buf + used, len - used, "\"", str, "\""); } /* Appends a string into the JSON array in the provided buffer. */ @@ -118,9 +128,9 @@ rte_tel_json_add_array_string(char *buf, const int len, const int used, { int ret, end = used - 1; /* strip off final delimiter */ if (used <= 2) /* assume empty, since minimum is '[]' */ - return __json_snprintf(buf, len, "[\"%s\"]", str); + return __json_format_str(buf, len, "[\"", str, "\"]"); - ret = __json_snprintf(buf + end, len - end, ",\"%s\"]", str); + ret = __json_format_str(buf + end, len - end, ",\"", str, "\"]"); return ret == 0 ? used : end + ret; } From patchwork Fri Sep 9 09:35:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116135 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 27D26A0540; Fri, 9 Sep 2022 11:36:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B9FBF4282B; Fri, 9 Sep 2022 11:35:54 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 3D38042B6E for ; Fri, 9 Sep 2022 11:35:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716151; x=1694252151; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VAyWKjsvQiWKxGEr4fSSeNKCNtAL1l3fvVm+jXAWGsc=; b=Fc51G8XcZLwdrkeOmpd1UztbC/b+MbkkO+CjnlErhL0Tkk/4cBW2WNpd MvaET1E3Vda+pOgh7kgrhQwlCHfj/2vUoDCWu/JyTJgtXMuYdMJat9JqQ sKWPKe7wGOuajT/kmzukWIAr5k/AT92otWRs98PeBB1j6IRAbXCJjq2Rq ir1shrFJGeWJA1+tLhl44T5Is1L7Efo41IEOtsLKW30YQoTO040Bng3aE 8EDig1FH9Z7opUzq5bKwvtxzy6eogQm7XNozmOY46aXR3RF7MfQZQChci Pq0U/xmccmlrMoCDowKU866VL8gco6KGqzhuW0ikFJ1Zo8Bl2B8/KUweT w==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437128" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437128" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996380" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:45 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 06/13] test/telemetry-json: add test for escaping strings in arrays Date: Fri, 9 Sep 2022 10:35:16 +0100 Message-Id: <20220909093523.471727-7-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add test-case to validate that when adding strings to arrays, the strings are properly escaped to remove any invalid characters. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- app/test/test_telemetry_json.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c index 955c2e5b1b..31a13ea1d7 100644 --- a/app/test/test_telemetry_json.c +++ b/app/test/test_telemetry_json.c @@ -141,6 +141,29 @@ test_string_char_escaping(void) return strncmp(expected, buf, sizeof(buf)); } +static int +test_array_char_escaping(void) +{ + /* "meaning of life", with tab between first two words, '\n' at end, + * and "life" in quotes, followed by "all the fish" in quotes + */ + const char *expected = "[\"meaning\\tof \\\"life\\\"\\n\",\"\\\"all the fish\\\"\"]"; + char buf[1024]; + int used = 0; + + used = rte_tel_json_empty_array(buf, sizeof(buf), used); + if (used != 2 || strcmp(buf, "[]")) + return -1; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, "meaning\tof \"life\"\n"); + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, "\"all the fish\""); + + printf("buf = '%s', expected = '%s'\n", buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + typedef int (*test_fn)(void); static int @@ -155,6 +178,7 @@ test_telemetry_json(void) test_large_array_element, test_large_obj_element, test_string_char_escaping, + test_array_char_escaping, }; for (i = 0; i < RTE_DIM(fns); i++) if (fns[i]() == 0) From patchwork Fri Sep 9 09:35:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116133 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B839FA0540; Fri, 9 Sep 2022 11:36:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 16F0F42B79; Fri, 9 Sep 2022 11:35:53 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 5924742829 for ; Fri, 9 Sep 2022 11:35:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716150; x=1694252150; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=j54IeXZePIv63X4WGZqjquKQa6jTxa9vk57lowo70H0=; b=eaG2QyJhqCk8aAk/Pjif+8dv6PNPzO6PfvVjKCOvadCW5qRqbdJl74Yh /24YQiEHyTK2UzOIqoz91ZMhcVmJew7w98mNMfFmTg6vJQ3s8ErC6CmZI G4IDjnfRB6WC+JNniPPIcTNX8sZ4mNrKpVKwHDHQkaHA0TjF88wO3f+L5 spg/+JBpcIZfz6Ocfi1XPSAQJU9rYZc58SwxxRBSrYD6s5A6lk3Kcw+vM NHjZWj5OSH7IAfr8ABl68tOcW6qpU/MWcFxOgEFA2FCk1FdReRMr49gc5 8x3Qry2WEmTWFn0QjvHpi9TF/7n+AQZy8UeH+3xBSlvwskmzhr9BkDGGJ g==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437131" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437131" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996388" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:46 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 07/13] telemetry: add escaping of strings in dicts Date: Fri, 9 Sep 2022 10:35:17 +0100 Message-Id: <20220909093523.471727-8-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When strings are added to an dict variable, we need to properly escape the invalid json characters in the strings. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- lib/telemetry/telemetry_json.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/telemetry/telemetry_json.h b/lib/telemetry/telemetry_json.h index c4442a0bf0..e3fae7c30d 100644 --- a/lib/telemetry/telemetry_json.h +++ b/lib/telemetry/telemetry_json.h @@ -54,7 +54,7 @@ static const char control_chars[0x20] = { * @internal * This function acts the same as __json_snprintf(buf, len, "%s%s%s", prefix, str, suffix) * except that it does proper escaping of "str" as necessary. Prefix and suffix should be compile- - * time constants not needing escaping. + * time constants, or values not needing escaping. * Drops any invalid characters we don't support */ static inline int @@ -219,12 +219,16 @@ static inline int rte_tel_json_add_obj_str(char *buf, const int len, const int used, const char *name, const char *val) { + char tmp_name[RTE_TEL_MAX_STRING_LEN + 5]; int ret, end = used - 1; + + /* names are limited to certain characters so need no escaping */ + snprintf(tmp_name, sizeof(tmp_name), "{\"%s\":\"", name); if (used <= 2) /* assume empty, since minimum is '{}' */ - return __json_snprintf(buf, len, "{\"%s\":\"%s\"}", name, val); + return __json_format_str(buf, len, tmp_name, val, "\"}"); - ret = __json_snprintf(buf + end, len - end, ",\"%s\":\"%s\"}", - name, val); + tmp_name[0] = ','; /* replace '{' with ',' at start */ + ret = __json_format_str(buf + end, len - end, tmp_name, val, "\"}"); return ret == 0 ? used : end + ret; } From patchwork Fri Sep 9 09:35:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116134 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D94C5A0540; Fri, 9 Sep 2022 11:36:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E87CB42B7E; Fri, 9 Sep 2022 11:35:53 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 0927F42829 for ; Fri, 9 Sep 2022 11:35:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716151; x=1694252151; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ybuJsbCS+BMl1wZU5d0yuc0/cz5tUQOOhQf4ftnns8M=; b=dS8+ws7k+GntD+tL/RxCYgmJkGD/ZmtYK++/cilUqXKFePjZ8xnRPvEG oyWf5tNOyTa6N2lGclh6gu6xfH33zGcEIeLQf6+dpkdhxdQ98nZydBtad VVet1lr2YHS57FUn+0kPIV9vi96du7KRnopuZKf05JmYQ27vf07zaE8AV PiKksIaAFBk+UENA7noNvF2JvZrKJdiU8MUeqv9a0ZThb2uvUcdpHkSoP AlZIIByr+CfSvQfi1Q+HHrBGygodIUt2ChH/7tRYr7Mq1E6k7RyehEqek tc7AoFxa/NCDF85mo8rR6GlfoTsMYQ0v82eE7EQvIiV1/jJePFW5FlJC7 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437135" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437135" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996409" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:48 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 08/13] test/telemetry_json: add test for string escaping in objects Date: Fri, 9 Sep 2022 10:35:18 +0100 Message-Id: <20220909093523.471727-9-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add a test-case to validate that when adding strings either as the name or the value of an entry in an object, that all values are escaped properly. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- app/test/test_telemetry_json.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c index 31a13ea1d7..184c3ba9f1 100644 --- a/app/test/test_telemetry_json.c +++ b/app/test/test_telemetry_json.c @@ -164,6 +164,29 @@ test_array_char_escaping(void) return strncmp(expected, buf, sizeof(buf)); } +static int +test_obj_char_escaping(void) +{ + const char *expected = "{\"good\":\"Clint Eastwood\\n\"," + "\"bad\":\"Lee\\tVan\\tCleef\"," + "\"ugly\":\"\\rEli Wallach\"}"; + char buf[1024]; + int used = 0; + + used = rte_tel_json_empty_obj(buf, sizeof(buf), used); + if (used != 2 || strcmp(buf, "{}")) + return -1; + + used = rte_tel_json_add_obj_str(buf, sizeof(buf), used, "good", "Clint Eastwood\n"); + used = rte_tel_json_add_obj_str(buf, sizeof(buf), used, "bad", "Lee\tVan\tCleef"); + used = rte_tel_json_add_obj_str(buf, sizeof(buf), used, "ugly", "\rEli Wallach"); + + printf("buf = '%s', expected = '%s'\n", buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + typedef int (*test_fn)(void); static int @@ -179,6 +202,7 @@ test_telemetry_json(void) test_large_obj_element, test_string_char_escaping, test_array_char_escaping, + test_obj_char_escaping }; for (i = 0; i < RTE_DIM(fns); i++) if (fns[i]() == 0) From patchwork Fri Sep 9 09:35:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116136 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CFB94A0540; Fri, 9 Sep 2022 11:36:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E456442B8E; Fri, 9 Sep 2022 11:35:55 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id B061242829 for ; Fri, 9 Sep 2022 11:35:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716151; x=1694252151; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GnQTKladdNCFmAE4dqRh46rqOJm9KH1WTCgMQJrDXLw=; b=fpIrL18ptIxfAzc1Ju80LLcAOT7HsuV9NyX/lce+OCTkvVzP8Xl+iUX2 RfnatflTthHtMO6ehn39f2C1ptDfK583p58aH+fhnSgUzl+6zGANH33Fm +7v/vndnYTAyK69GYL2/5oiZehFPGa4Mutcqje5tyhp9lG+hfMq+ySa5A MVQy0pGPawmwdocntjDItUo87WAciPLepb4NbJL4e0Ve0/Bg5qXyolxFC kdNOQTXZvJ8K+1dKwaHmbh1bUMJKbcbf1+O10mx1pPv6RaZSSFmsomDI5 vAOCBr2YxcbmHUe/yC2XMQWo7bRgcg93gtH1Wa3XuWDc7DLQhxi4dbZl5 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437141" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437141" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996439" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:49 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 09/13] telemetry: limit command characters Date: Fri, 9 Sep 2022 10:35:19 +0100 Message-Id: <20220909093523.471727-10-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Limit the telemetry command characters to the minimum set needed for current implementations. This prevents issues with invalid json characters needing to be escaped on replies. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- doc/guides/rel_notes/deprecation.rst | 8 -------- lib/telemetry/telemetry.c | 7 +++++++ 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst index e7583cae4c..d1c93ca7e3 100644 --- a/doc/guides/rel_notes/deprecation.rst +++ b/doc/guides/rel_notes/deprecation.rst @@ -212,14 +212,6 @@ Deprecation Notices * metrics: The function ``rte_metrics_init`` will have a non-void return in order to notify errors instead of calling ``rte_exit``. -* telemetry: The allowed characters in names for dictionary values - will be limited to alphanumeric characters - and a small subset of additional printable characters. - This will ensure that all dictionary parameter names can be output - without escaping in JSON - or in any future output format used. - Names for the telemetry commands will be similarly limited. - The parameters for telemetry commands are unaffected by this change. - * net/octeontx_ep: The driver ``octeontx_ep`` was to support OCTEON TX line of products. It will be renamed to ``octeon_ep`` in DPDK 22.11 to apply for diff --git a/lib/telemetry/telemetry.c b/lib/telemetry/telemetry.c index 7188b1905c..03651e947d 100644 --- a/lib/telemetry/telemetry.c +++ b/lib/telemetry/telemetry.c @@ -70,12 +70,19 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help) { struct cmd_callback *new_callbacks; + const char *cmdp = cmd; int i = 0; if (strlen(cmd) >= MAX_CMD_LEN || fn == NULL || cmd[0] != '/' || strlen(help) >= RTE_TEL_MAX_STRING_LEN) return -EINVAL; + while (*cmdp != '\0') { + if (!isalnum(*cmdp) && *cmdp != '_' && *cmdp != '/') + return -EINVAL; + cmdp++; + } + rte_spinlock_lock(&callback_sl); new_callbacks = realloc(callbacks, sizeof(callbacks[0]) * (num_callbacks + 1)); if (new_callbacks == NULL) { From patchwork Fri Sep 9 09:35:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116137 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5CA4BA0540; Fri, 9 Sep 2022 11:36:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B679842B93; Fri, 9 Sep 2022 11:35:56 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 32D5142B7B for ; Fri, 9 Sep 2022 11:35:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716153; x=1694252153; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bezmzFO8bJWQbCOTDbMgk6PPTOjoBIx0uEqoISmniCM=; b=Hkd8CQD90cY3bNbfjiDJG2iO2byxbp8a8N1L2qlz/9XyMHwRwtTNKqz9 rpMY9pWIhNhRKM0VwVGQZ0XWHb+buTwpWAX7e9FeOuEe4drBeKek/oMIe EePgkmIbVJl/nOSE8pDjALK7sWarXK7tWTdRTmD+twF04bXtg+WQJxns2 FtYwPsvmr45pSKON8yrIH61377m0ct1547WvEa6Uy8xbBE+HHWQS1ytQJ z/OzgOCQscRkfVq425OCESarlwVs2rauZpSxPVNAZW2Zvz5IdS7fVYaIn tEG0G8BiidcQwohEE7ZqpcKkevd/yAEVx0sxq3Q3Z8yHf3jcF49IHj8NA Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437145" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437145" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996442" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:50 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 10/13] test/telemetry_data: refactor for maintainability Date: Fri, 9 Sep 2022 10:35:20 +0100 Message-Id: <20220909093523.471727-11-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org To help with the writing and maintaining of test cases in this file we can make the following changes to it: - rename non-test-case functions i.e. the infrastructure functions, to not start with "test_", so that each sub-test case can be identified by starting with that prefix. - add a comment at the start of the file explaining how tests are to be written and managed, so as to keep consistency. - add a trivial test-case for returning a simple string value to use as a reference example for those wanting to add test cases. - improve the key macro used for validating the output from each function, so that the standard json preamble can be skipped for each function. This hides more of the infrastructure implementation from the user i.e. they don't need to worry what the actual command used is called, and also shortens the output strings so we can avoid line splitting in most cases. - add clearing the "response_data" structure to the loop calling each test to avoid each test function having to do so individually. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- app/test/test_telemetry_data.c | 101 ++++++++++++++++++++------------- 1 file changed, 60 insertions(+), 41 deletions(-) diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c index 73eee293a1..9d174dc99f 100644 --- a/app/test/test_telemetry_data.c +++ b/app/test/test_telemetry_data.c @@ -21,18 +21,45 @@ #define TELEMETRY_VERSION "v2" #define REQUEST_CMD "/test" #define BUF_SIZE 1024 -#define TEST_OUTPUT(exp) test_output(__func__, exp) +#define CHECK_OUTPUT(exp) check_output(__func__, "{\"" REQUEST_CMD "\":" exp "}") + +/* + * Runs a series of test cases, checking the output of telemetry for various different types of + * responses. On init, a single connection to DPDK telemetry is made, and a single telemetry + * callback "/test" is registered. That callback always returns the value of the static global + * variable "response_data", so each test case builds up that structure, and then calls the + * "check_output" function to ensure the response received over the socket for "/test" matches + * that expected for the response_data value populated. + * + * NOTE: + * - each test case function in this file should be added to the "test_cases" array in + * test_telemetry_data function at the bottom of the file. + * - each test case function should populate the "response_data" global variable (below) + * with the appropriate values which would be returned from a simulated telemetry function. + * Then the test case function should have "return CHECK_OUTPUT();" as it's + * last line. The test infrastructure will then validate that the output when returning + * "response_data" structure matches that in "". + * - the response_data structure will be zeroed on entry to each test function, so each function + * can begin with a call to "rte_tel_data_string/start_array/start_dict" as so desired. + * - the expected_output for each function can be just the actual json data from the + * "response_data" value. The CHECK_OUTPUT macro will include the appropriate "{\"/test\": ... }" + * structure around the json output. + * + * See test_simple_string(), or test_case_array_int() for a basic examples of test cases. + */ + static struct rte_tel_data response_data; static int sock; + /* * This function is the callback registered with Telemetry to be used when * the /test command is requested. This callback returns the global data built * up by the individual test cases. */ static int -test_cb(const char *cmd __rte_unused, const char *params __rte_unused, +telemetry_test_cb(const char *cmd __rte_unused, const char *params __rte_unused, struct rte_tel_data *d) { *d = response_data; @@ -46,7 +73,7 @@ test_cb(const char *cmd __rte_unused, const char *params __rte_unused, * and is compared to the actual response received from Telemetry. */ static int -test_output(const char *func_name, const char *expected) +check_output(const char *func_name, const char *expected) { int bytes; char buf[BUF_SIZE * 16]; @@ -66,6 +93,14 @@ test_output(const char *func_name, const char *expected) return strncmp(expected, buf, sizeof(buf)); } +static int +test_simple_string(void) +{ + rte_tel_data_string(&response_data, "Simple string"); + + return CHECK_OUTPUT("\"Simple string\""); +} + static int test_dict_with_array_int_values(void) { @@ -77,7 +112,6 @@ test_dict_with_array_int_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_INT_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); for (i = 0; i < 5; i++) { @@ -90,8 +124,7 @@ test_dict_with_array_int_values(void) rte_tel_data_add_dict_container(&response_data, "dict_1", child_data2, 0); - return TEST_OUTPUT("{\"/test\":{\"dict_0\":[0,1,2,3,4]," - "\"dict_1\":[0,1,2,3,4]}}"); + return CHECK_OUTPUT("{\"dict_0\":[0,1,2,3,4],\"dict_1\":[0,1,2,3,4]}"); } static int @@ -105,7 +138,6 @@ test_array_with_array_int_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_INT_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_array(&response_data, RTE_TEL_CONTAINER); for (i = 0; i < 5; i++) { @@ -115,18 +147,18 @@ test_array_with_array_int_values(void) rte_tel_data_add_array_container(&response_data, child_data, 0); rte_tel_data_add_array_container(&response_data, child_data2, 0); - return TEST_OUTPUT("{\"/test\":[[0,1,2,3,4],[0,1,2,3,4]]}"); + return CHECK_OUTPUT("[[0,1,2,3,4],[0,1,2,3,4]]"); } static int test_case_array_int(void) { int i; - memset(&response_data, 0, sizeof(response_data)); + rte_tel_data_start_array(&response_data, RTE_TEL_INT_VAL); for (i = 0; i < 5; i++) rte_tel_data_add_array_int(&response_data, i); - return TEST_OUTPUT("{\"/test\":[0,1,2,3,4]}"); + return CHECK_OUTPUT("[0,1,2,3,4]"); } static int @@ -135,7 +167,6 @@ test_case_add_dict_int(void) int i = 0; char name_of_value[8]; - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); for (i = 0; i < 5; i++) { @@ -143,14 +174,12 @@ test_case_add_dict_int(void) rte_tel_data_add_dict_int(&response_data, name_of_value, i); } - return TEST_OUTPUT("{\"/test\":{\"dict_0\":0,\"dict_1\":1,\"dict_2\":2," - "\"dict_3\":3,\"dict_4\":4}}"); + return CHECK_OUTPUT("{\"dict_0\":0,\"dict_1\":1,\"dict_2\":2,\"dict_3\":3,\"dict_4\":4}"); } static int test_case_array_string(void) { - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_array(&response_data, RTE_TEL_STRING_VAL); rte_tel_data_add_array_string(&response_data, "aaaa"); rte_tel_data_add_array_string(&response_data, "bbbb"); @@ -158,14 +187,12 @@ test_case_array_string(void) rte_tel_data_add_array_string(&response_data, "dddd"); rte_tel_data_add_array_string(&response_data, "eeee"); - return TEST_OUTPUT("{\"/test\":[\"aaaa\",\"bbbb\",\"cccc\",\"dddd\"," - "\"eeee\"]}"); + return CHECK_OUTPUT("[\"aaaa\",\"bbbb\",\"cccc\",\"dddd\",\"eeee\"]"); } static int test_case_add_dict_string(void) { - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); rte_tel_data_add_dict_string(&response_data, "dict_0", "aaaa"); @@ -173,8 +200,7 @@ test_case_add_dict_string(void) rte_tel_data_add_dict_string(&response_data, "dict_2", "cccc"); rte_tel_data_add_dict_string(&response_data, "dict_3", "dddd"); - return TEST_OUTPUT("{\"/test\":{\"dict_0\":\"aaaa\",\"dict_1\":" - "\"bbbb\",\"dict_2\":\"cccc\",\"dict_3\":\"dddd\"}}"); + return CHECK_OUTPUT("{\"dict_0\":\"aaaa\",\"dict_1\":\"bbbb\",\"dict_2\":\"cccc\",\"dict_3\":\"dddd\"}"); } @@ -187,7 +213,6 @@ test_dict_with_array_string_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_STRING_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); rte_tel_data_add_array_string(child_data, "aaaa"); @@ -198,8 +223,7 @@ test_dict_with_array_string_values(void) rte_tel_data_add_dict_container(&response_data, "dict_1", child_data2, 0); - return TEST_OUTPUT("{\"/test\":{\"dict_0\":[\"aaaa\"],\"dict_1\":" - "[\"bbbb\"]}}"); + return CHECK_OUTPUT("{\"dict_0\":[\"aaaa\"],\"dict_1\":[\"bbbb\"]}"); } static int @@ -214,7 +238,6 @@ test_dict_with_dict_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_STRING_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); rte_tel_data_add_array_string(child_data, "aaaa"); @@ -226,8 +249,7 @@ test_dict_with_dict_values(void) rte_tel_data_add_dict_container(&response_data, "dict_of_dicts", dict_of_dicts, 0); - return TEST_OUTPUT("{\"/test\":{\"dict_of_dicts\":{\"dict_0\":" - "[\"aaaa\"],\"dict_1\":[\"bbbb\"]}}}"); + return CHECK_OUTPUT("{\"dict_of_dicts\":{\"dict_0\":[\"aaaa\"],\"dict_1\":[\"bbbb\"]}}"); } static int @@ -239,7 +261,6 @@ test_array_with_array_string_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_STRING_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_array(&response_data, RTE_TEL_CONTAINER); rte_tel_data_add_array_string(child_data, "aaaa"); @@ -248,18 +269,18 @@ test_array_with_array_string_values(void) rte_tel_data_add_array_container(&response_data, child_data, 0); rte_tel_data_add_array_container(&response_data, child_data2, 0); - return TEST_OUTPUT("{\"/test\":[[\"aaaa\"],[\"bbbb\"]]}"); + return CHECK_OUTPUT("[[\"aaaa\"],[\"bbbb\"]]"); } static int test_case_array_u64(void) { int i; - memset(&response_data, 0, sizeof(response_data)); + rte_tel_data_start_array(&response_data, RTE_TEL_U64_VAL); for (i = 0; i < 5; i++) rte_tel_data_add_array_u64(&response_data, i); - return TEST_OUTPUT("{\"/test\":[0,1,2,3,4]}"); + return CHECK_OUTPUT("[0,1,2,3,4]"); } static int @@ -268,15 +289,13 @@ test_case_add_dict_u64(void) int i = 0; char name_of_value[8]; - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); for (i = 0; i < 5; i++) { sprintf(name_of_value, "dict_%d", i); rte_tel_data_add_dict_u64(&response_data, name_of_value, i); } - return TEST_OUTPUT("{\"/test\":{\"dict_0\":0,\"dict_1\":1,\"dict_2\":2," - "\"dict_3\":3,\"dict_4\":4}}"); + return CHECK_OUTPUT("{\"dict_0\":0,\"dict_1\":1,\"dict_2\":2,\"dict_3\":3,\"dict_4\":4}"); } static int @@ -290,7 +309,6 @@ test_dict_with_array_u64_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_U64_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); for (i = 0; i < 10; i++) { @@ -303,8 +321,7 @@ test_dict_with_array_u64_values(void) rte_tel_data_add_dict_container(&response_data, "dict_1", child_data2, 0); - return TEST_OUTPUT("{\"/test\":{\"dict_0\":[0,1,2,3,4,5,6,7,8,9]," - "\"dict_1\":[0,1,2,3,4,5,6,7,8,9]}}"); + return CHECK_OUTPUT("{\"dict_0\":[0,1,2,3,4,5,6,7,8,9],\"dict_1\":[0,1,2,3,4,5,6,7,8,9]}"); } static int @@ -318,7 +335,6 @@ test_array_with_array_u64_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_U64_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_array(&response_data, RTE_TEL_CONTAINER); for (i = 0; i < 5; i++) { @@ -328,7 +344,7 @@ test_array_with_array_u64_values(void) rte_tel_data_add_array_container(&response_data, child_data, 0); rte_tel_data_add_array_container(&response_data, child_data2, 0); - return TEST_OUTPUT("{\"/test\":[[0,1,2,3,4],[0,1,2,3,4]]}"); + return CHECK_OUTPUT("[[0,1,2,3,4],[0,1,2,3,4]]"); } static int @@ -369,7 +385,7 @@ connect_to_socket(void) } static int -test_telemetry_data(void) +telemetry_data_autotest(void) { typedef int (*test_case)(void); unsigned int i = 0; @@ -378,7 +394,9 @@ test_telemetry_data(void) if (sock <= 0) return -1; - test_case test_cases[] = {test_case_array_string, + test_case test_cases[] = { + test_simple_string, + test_case_array_string, test_case_array_int, test_case_array_u64, test_case_add_dict_int, test_case_add_dict_u64, test_case_add_dict_string, @@ -390,8 +408,9 @@ test_telemetry_data(void) test_array_with_array_u64_values, test_array_with_array_string_values }; - rte_telemetry_register_cmd(REQUEST_CMD, test_cb, "Test"); + rte_telemetry_register_cmd(REQUEST_CMD, telemetry_test_cb, "Test"); for (i = 0; i < RTE_DIM(test_cases); i++) { + memset(&response_data, 0, sizeof(response_data)); if (test_cases[i]() != 0) { close(sock); return -1; @@ -401,4 +420,4 @@ test_telemetry_data(void) return 0; } -REGISTER_TEST_COMMAND(telemetry_data_autotest, test_telemetry_data); +REGISTER_TEST_COMMAND(telemetry_data_autotest, telemetry_data_autotest); From patchwork Fri Sep 9 09:35:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116138 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1EF7CA0540; Fri, 9 Sep 2022 11:36:52 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AC49142B96; Fri, 9 Sep 2022 11:35:57 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 39B7B42B7B for ; Fri, 9 Sep 2022 11:35:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716154; x=1694252154; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=g8UYN3JsOkvVvTeql4xctvdj+f5sfV8/ScRpaKgBzyo=; b=elj0V58k19NAvtC+1j7Qp9Uu/7IPFlhu95naOYXdAhroYldqEnIZuHuo 90ZMV85kx9uQxBo7LjgKa3mPuBcA6aNSY2bJtGKugnQTtIMLPM2fxE3eJ jCDbyPgzYFM6prsTbKx46WUkRBOs9e2z16qybZPq5S/OTnP5GLJYGqccP PUXBkpMwltssh0U58H+0TP2CeGOuDzvOZc4ZklgYzBCx3ZSkzKUFzK5la qXRIhzQsJd7xoV3yu9AuYmHywr/b9lg6Prq+Op0ckLNiNBiDqXiNH+7Oo xsd4r9cFIPj79lCezF/qZGMnMUGkz4XvI7T53mD3JPv0GWhkq7QmDO78h w==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437149" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437149" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996445" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:51 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 11/13] test/telemetry_data: add test cases for character escaping Date: Fri, 9 Sep 2022 10:35:21 +0100 Message-Id: <20220909093523.471727-12-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add in some basic unit tests to validate the character escaping being done on string data values, which tests end-to-end processing of those values beyond just the json-encoding steps tested by the "telemetry_json_autotest". Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- app/test/test_telemetry_data.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c index 9d174dc99f..69ca8b6c6f 100644 --- a/app/test/test_telemetry_data.c +++ b/app/test/test_telemetry_data.c @@ -347,6 +347,30 @@ test_array_with_array_u64_values(void) return CHECK_OUTPUT("[[0,1,2,3,4],[0,1,2,3,4]]"); } +static int +test_string_char_escaping(void) +{ + rte_tel_data_string(&response_data, "hello,\nworld\n"); + return CHECK_OUTPUT("\"hello,\\nworld\\n\""); +} + +static int +test_array_char_escaping(void) +{ + rte_tel_data_start_array(&response_data, RTE_TEL_STRING_VAL); + rte_tel_data_add_array_string(&response_data, "\\escape\r"); + rte_tel_data_add_array_string(&response_data, "characters\n"); + return CHECK_OUTPUT("[\"\\\\escape\\r\",\"characters\\n\"]"); +} + +static int +test_dict_char_escaping(void) +{ + rte_tel_data_start_dict(&response_data); + rte_tel_data_add_dict_string(&response_data, "name", "escaped\n\tvalue"); + return CHECK_OUTPUT("{\"name\":\"escaped\\n\\tvalue\"}"); +} + static int connect_to_socket(void) { @@ -406,7 +430,11 @@ telemetry_data_autotest(void) test_dict_with_dict_values, test_array_with_array_int_values, test_array_with_array_u64_values, - test_array_with_array_string_values }; + test_array_with_array_string_values, + test_string_char_escaping, + test_array_char_escaping, + test_dict_char_escaping, + }; rte_telemetry_register_cmd(REQUEST_CMD, telemetry_test_cb, "Test"); for (i = 0; i < RTE_DIM(test_cases); i++) { From patchwork Fri Sep 9 09:35:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116139 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 906B9A0540; Fri, 9 Sep 2022 11:36:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9B89E42B9B; Fri, 9 Sep 2022 11:35:58 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id D21DD42B83 for ; Fri, 9 Sep 2022 11:35:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716155; x=1694252155; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VQAGXAJSVEJmt//LQD1buGWpxsCyKtStxK8yQanehtU=; b=d16XN6GRgiffSUEvgHRnFLFNVsbQ0vz8SE3XlLY5cXpEmoqq2czrE2Js F9x9yHhANPIxD27qVG+y3rw1zdtzTbSKGb1rmu6il5jXPmNMvvaJW+V5k Rlj357NdxITs5a+e93342vX9/aqpRYdMSAO0GIRsmbxZg2TYBx3JX9PBK rgJCeApkDv6IvahM4Xxsnqq+/6NIRqQPmK0AlrfXxj44RCXxxtWFz+5/7 /TJEGddecVIYkdSPrGgJlvolfN9Deyi30/RX0mHa7+MhpmzLyrq3B6tfo iOSsp9G5LbQBTkYE7nyWZgQpRf5nnu8ifw70f91933hknns0alBzkzPdA g==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437156" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437156" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996453" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:53 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 12/13] telemetry: eliminate duplicate code for json output Date: Fri, 9 Sep 2022 10:35:22 +0100 Message-Id: <20220909093523.471727-13-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When preparing the json response to a telemetry socket query, the code for prefixing the command name, and appending the file "}" on the end of the response was duplicated for multiple reply types. Taking this code out of the switch statement reduces the duplication and makes the code more maintainable. For completeness of testing, add in a test case to validate the "null" response type - the only leg of the switch statment not already covered by an existing test case in the telemetry_data tests. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- app/test/test_telemetry_data.c | 7 +++++++ lib/telemetry/telemetry.c | 35 ++++++++++++---------------------- 2 files changed, 19 insertions(+), 23 deletions(-) diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c index 69ca8b6c6f..d92667a527 100644 --- a/app/test/test_telemetry_data.c +++ b/app/test/test_telemetry_data.c @@ -93,6 +93,12 @@ check_output(const char *func_name, const char *expected) return strncmp(expected, buf, sizeof(buf)); } +static int +test_null_return(void) +{ + return CHECK_OUTPUT("null"); +} + static int test_simple_string(void) { @@ -419,6 +425,7 @@ telemetry_data_autotest(void) return -1; test_case test_cases[] = { + test_null_return, test_simple_string, test_case_array_string, test_case_array_int, test_case_array_u64, diff --git a/lib/telemetry/telemetry.c b/lib/telemetry/telemetry.c index 03651e947d..cf60d27bd4 100644 --- a/lib/telemetry/telemetry.c +++ b/lib/telemetry/telemetry.c @@ -233,27 +233,22 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s) RTE_BUILD_BUG_ON(sizeof(out_buf) < MAX_CMD_LEN + RTE_TEL_MAX_SINGLE_STRING_LEN + 10); + + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", + MAX_CMD_LEN, cmd); + cb_data_buf = &out_buf[prefix_used]; + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + switch (d->type) { case RTE_TEL_NULL: - used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":null}", - MAX_CMD_LEN, cmd ? cmd : "none"); + used = strlcpy(cb_data_buf, "null", buf_len); break; - case RTE_TEL_STRING: - prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", - MAX_CMD_LEN, cmd); - cb_data_buf = &out_buf[prefix_used]; - buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + case RTE_TEL_STRING: used = rte_tel_json_str(cb_data_buf, buf_len, 0, d->data.str); - used += prefix_used; - used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); break; - case RTE_TEL_DICT: - prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", - MAX_CMD_LEN, cmd); - cb_data_buf = &out_buf[prefix_used]; - buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + case RTE_TEL_DICT: used = rte_tel_json_empty_obj(cb_data_buf, buf_len, 0); for (i = 0; i < d->data_len; i++) { const struct tel_dict_entry *v = &d->data.dict[i]; @@ -289,18 +284,12 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s) } } } - used += prefix_used; - used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); break; + case RTE_TEL_ARRAY_STRING: case RTE_TEL_ARRAY_INT: case RTE_TEL_ARRAY_U64: case RTE_TEL_ARRAY_CONTAINER: - prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", - MAX_CMD_LEN, cmd); - cb_data_buf = &out_buf[prefix_used]; - buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ - used = rte_tel_json_empty_array(cb_data_buf, buf_len, 0); for (i = 0; i < d->data_len; i++) if (d->type == RTE_TEL_ARRAY_STRING) @@ -328,10 +317,10 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s) if (!rec_data->keep) rte_tel_data_free(rec_data->data); } - used += prefix_used; - used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); break; } + used += prefix_used; + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); if (write(s, out_buf, used) < 0) perror("Error writing to socket"); } From patchwork Fri Sep 9 09:35:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 116140 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8449BA0540; Fri, 9 Sep 2022 11:37:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6C1FE42BA3; Fri, 9 Sep 2022 11:35:59 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id F165D42B91 for ; Fri, 9 Sep 2022 11:35:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662716156; x=1694252156; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ttJeebfNbU0R3tb1VJ8XmZ4ZDVQe9iMX6HX8upVLNJ0=; b=K6qOSR+dWVSObkdnkdXthFW9+S4eeFlOEHDgDKIIC2zSQt6sD7sIw3Jp MmgiwKjfvRAhoYx5wA1IgM9TcJI3QXwB+NR/4gXP9TcNX1+naHdLpuHCP 8IbQHtZif2mQNR54JZ7NK7JUpIZE3MVWapXvpNxxG+hHvXias6lVE5n/a t2ynppJaqDuRGK8gbxIQA3eR5+6vcq3x3sRWhKv2e1BnsRg2FX4NSvQiw A1Jen7njawlXfT6S/WSEcx/G7yfCCDeH+Mebe6Kn6UK1yNxuQ0x8zWDcA qmIb+VbzUFzPffoOeVJlLI7h8WQSAbH21SOIdAY34ZFBYnMmA7bXwRzdy A==; X-IronPort-AV: E=McAfee;i="6500,9779,10464"; a="297437157" X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="297437157" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2022 02:35:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,302,1654585200"; d="scan'208";a="740996457" Received: from silpixa00401385.ir.intel.com ([10.237.214.161]) by orsmga004.jf.intel.com with ESMTP; 09 Sep 2022 02:35:54 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , =?utf-8?q?Morten_Br=C3=B8rup?= Subject: [PATCH v3 13/13] telemetry: make help command more helpful Date: Fri, 9 Sep 2022 10:35:23 +0100 Message-Id: <20220909093523.471727-14-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220909093523.471727-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220909093523.471727-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The /help telemetry command prints out the help text for the given command passed in as parameter. However, entering /help without any parameters does not give any useful information as to the fact that you need to pass in a command to get help on. Update the command so it prints its own help text when called without any parameters. Signed-off-by: Bruce Richardson Acked-by: Ciara Power Acked-by: Morten Brørup --- lib/telemetry/telemetry.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/lib/telemetry/telemetry.c b/lib/telemetry/telemetry.c index cf60d27bd4..09febff0ae 100644 --- a/lib/telemetry/telemetry.c +++ b/lib/telemetry/telemetry.c @@ -139,15 +139,17 @@ command_help(const char *cmd __rte_unused, const char *params, struct rte_tel_data *d) { int i; + /* if no parameters return our own help text */ + const char *to_lookup = (params == NULL ? cmd : params); - if (!params) - return -1; rte_tel_data_start_dict(d); rte_spinlock_lock(&callback_sl); for (i = 0; i < num_callbacks; i++) - if (strcmp(params, callbacks[i].cmd) == 0) { - rte_tel_data_add_dict_string(d, params, - callbacks[i].help); + if (strcmp(to_lookup, callbacks[i].cmd) == 0) { + if (params == NULL) + rte_tel_data_string(d, callbacks[i].help); + else + rte_tel_data_add_dict_string(d, params, callbacks[i].help); break; } rte_spinlock_unlock(&callback_sl);