[v14,4/4] eal: add PMU support to tracing library

Message ID 20241011094944.3586051-5-tduszynski@marvell.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series add support for self monitoring |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/loongarch-compilation success Compilation OK
ci/loongarch-unit-testing success Unit Testing PASS
ci/github-robot: build fail github build: failed
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-marvell-Functional success Functional Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-broadcom-Performance success Performance Testing PASS
ci/Intel-compilation success Compilation OK
ci/iol-unit-amd64-testing fail Testing issues
ci/iol-compile-amd64-testing warning Testing issues
ci/iol-unit-arm64-testing fail Testing issues
ci/iol-compile-arm64-testing fail Testing issues
ci/intel-Testing success Testing PASS
ci/iol-sample-apps-testing success Testing PASS
ci/intel-Functional success Functional PASS

Commit Message

Tomasz Duszynski Oct. 11, 2024, 9:49 a.m. UTC
In order to profile app one needs to store significant amount of samples
somewhere for an analysis later on. Since trace library supports
storing data in a CTF format lets take advantage of that and add a
dedicated PMU tracepoint.

Signed-off-by: Tomasz Duszynski <tduszynski@marvell.com>
---
 app/test/test_trace_perf.c               |  4 ++
 doc/guides/prog_guide/profile_app.rst    |  5 ++
 doc/guides/prog_guide/trace_lib.rst      | 32 +++++++++++++
 lib/eal/common/eal_common_trace.c        |  7 ++-
 lib/eal/common/eal_common_trace_points.c |  5 ++
 lib/eal/include/rte_eal_trace.h          |  9 ++++
 lib/eal/meson.build                      |  2 +-
 lib/eal/version.map                      |  3 ++
 lib/pmu/rte_pmu.c                        | 61 ++++++++++++++++++++++++
 lib/pmu/rte_pmu.h                        | 18 +++++++
 lib/pmu/version.map                      |  1 +
 11 files changed, 145 insertions(+), 2 deletions(-)
  

Comments

David Marchand Oct. 11, 2024, 1:29 p.m. UTC | #1
On Fri, Oct 11, 2024 at 11:50 AM Tomasz Duszynski
<tduszynski@marvell.com> wrote:
> diff --git a/lib/eal/meson.build b/lib/eal/meson.build
> index e1d6c4cf17..2b1a1eb283 100644
> --- a/lib/eal/meson.build
> +++ b/lib/eal/meson.build
> @@ -14,7 +14,7 @@ subdir(exec_env)
>
>  subdir(arch_subdir)
>
> -deps += ['log', 'kvargs']
> +deps += ['log', 'kvargs', 'pmu']

Quick comment as I saw a failure in CI.

An optional library should not be a required dependency for EAL.
You need something like (untested):

if dpdk_conf.has('RTE_LIB_PMU')
    deps += ['pmu']
endif

>  if not is_windows
>      deps += ['telemetry']
>  endif
  
Tomasz Duszynski Oct. 15, 2024, 9:18 a.m. UTC | #2
>-----Original Message-----
>From: David Marchand <david.marchand@redhat.com>
>Sent: Friday, October 11, 2024 3:29 PM
>To: Tomasz Duszynski <tduszynski@marvell.com>
>Cc: Jerin Jacob <jerinj@marvell.com>; Sunil Kumar Kori <skori@marvell.com>; Tyler Retzlaff
><roretzla@linux.microsoft.com>; Ruifeng.Wang@arm.com; bruce.richardson@intel.com; dev@dpdk.org;
>konstantin.v.ananyev@yandex.ru; mattias.ronnblom@ericsson.com; mb@smartsharesystems.com;
>thomas@monjalon.net; zhoumin@loongson.cn; stephen@networkplumber.org
>Subject: [EXTERNAL] Re: [PATCH v14 4/4] eal: add PMU support to tracing library
>
>On Fri, Oct 11, 2024 at 11: 50 AM Tomasz Duszynski <tduszynski@ marvell. com> wrote: > diff --git
>a/lib/eal/meson. build b/lib/eal/meson. build > index e1d6c4cf17. . 2b1a1eb283 100644 > ---
>a/lib/eal/meson. build > +++ b/lib/eal/meson. build 
>On Fri, Oct 11, 2024 at 11:50 AM Tomasz Duszynski <tduszynski@marvell.com> wrote:
>> diff --git a/lib/eal/meson.build b/lib/eal/meson.build index
>> e1d6c4cf17..2b1a1eb283 100644
>> --- a/lib/eal/meson.build
>> +++ b/lib/eal/meson.build
>> @@ -14,7 +14,7 @@ subdir(exec_env)
>>
>>  subdir(arch_subdir)
>>
>> -deps += ['log', 'kvargs']
>> +deps += ['log', 'kvargs', 'pmu']
>
>Quick comment as I saw a failure in CI.
>
>An optional library should not be a required dependency for EAL.
>You need something like (untested):
>
>if dpdk_conf.has('RTE_LIB_PMU')
>    deps += ['pmu']
>endif
>

Okay, I'll double check. Thanks. 

>>  if not is_windows
>>      deps += ['telemetry']
>>  endif
>
>
>--
>David Marchand
  

Patch

diff --git a/app/test/test_trace_perf.c b/app/test/test_trace_perf.c
index 8257cc02be..37af2037b4 100644
--- a/app/test/test_trace_perf.c
+++ b/app/test/test_trace_perf.c
@@ -114,6 +114,8 @@  worker_fn_##func(void *arg) \
 #define GENERIC_DOUBLE rte_eal_trace_generic_double(3.66666)
 #define GENERIC_STR rte_eal_trace_generic_str("hello world")
 #define VOID_FP app_dpdk_test_fp()
+/* 0 corresponds first event passed via --trace= */
+#define READ_PMU rte_eal_trace_pmu_read(0)
 
 WORKER_DEFINE(GENERIC_VOID)
 WORKER_DEFINE(GENERIC_U64)
@@ -122,6 +124,7 @@  WORKER_DEFINE(GENERIC_FLOAT)
 WORKER_DEFINE(GENERIC_DOUBLE)
 WORKER_DEFINE(GENERIC_STR)
 WORKER_DEFINE(VOID_FP)
+WORKER_DEFINE(READ_PMU)
 
 static void
 run_test(const char *str, lcore_function_t f, struct test_data *data, size_t sz)
@@ -174,6 +177,7 @@  test_trace_perf(void)
 	run_test("double", worker_fn_GENERIC_DOUBLE, data, sz);
 	run_test("string", worker_fn_GENERIC_STR, data, sz);
 	run_test("void_fp", worker_fn_VOID_FP, data, sz);
+	run_test("read_pmu", worker_fn_READ_PMU, data, sz);
 
 	rte_free(data);
 	return TEST_SUCCESS;
diff --git a/doc/guides/prog_guide/profile_app.rst b/doc/guides/prog_guide/profile_app.rst
index 854c515a61..1ab6fb9eaa 100644
--- a/doc/guides/prog_guide/profile_app.rst
+++ b/doc/guides/prog_guide/profile_app.rst
@@ -36,6 +36,11 @@  As of now implementation imposes certain limitations:
 
 * Each EAL lcore measures same group of events
 
+Alternatively tracing library can be used which offers dedicated tracepoint
+``rte_eal_trace_pmu_event()``.
+
+Refer to :doc:`../prog_guide/trace_lib` for more details.
+
 
 Profiling on x86
 ----------------
diff --git a/doc/guides/prog_guide/trace_lib.rst b/doc/guides/prog_guide/trace_lib.rst
index d9b17abe90..378abccd72 100644
--- a/doc/guides/prog_guide/trace_lib.rst
+++ b/doc/guides/prog_guide/trace_lib.rst
@@ -46,6 +46,7 @@  DPDK tracing library features
   trace format and is compatible with ``LTTng``.
   For detailed information, refer to
   `Common Trace Format <https://diamon.org/ctf/>`_.
+- Support reading PMU events on ARM64 and x86-64 (Intel)
 
 How to add a tracepoint?
 ------------------------
@@ -139,6 +140,37 @@  the user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``.
 ``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using
 the ``enable_trace_fp`` option for meson build.
 
+PMU tracepoint
+--------------
+
+Performance monitoring unit (PMU) event values can be read from hardware
+registers using predefined ``rte_pmu_read`` tracepoint.
+
+Tracing is enabled via ``--trace`` EAL option by passing both expression
+matching PMU tracepoint name i.e ``lib.eal.pmu.read`` and expression
+``e=ev1[,ev2,...]`` matching particular events::
+
+    --trace='.*pmu.read\|e=cpu_cycles,l1d_cache'
+
+Event names are available under ``/sys/bus/event_source/devices/PMU/events``
+directory, where ``PMU`` is a placeholder for either a ``cpu`` or a directory
+containing ``cpus``.
+
+In contrary to other tracepoints this does not need any extra variables
+added to source files. Instead, caller passes index which follows the order of
+events specified via ``--trace`` parameter. In the following example index ``0``
+corresponds to ``cpu_cyclces`` while index ``1`` corresponds to ``l1d_cache``.
+
+.. code-block:: c
+
+ ...
+ rte_eal_trace_pmu_read(0);
+ rte_eal_trace_pmu_read(1);
+ ...
+
+PMU tracing support must be explicitly enabled using the ``enable_trace_fp``
+option for meson build.
+
 Event record mode
 -----------------
 
diff --git a/lib/eal/common/eal_common_trace.c b/lib/eal/common/eal_common_trace.c
index 918f49bf4f..ffeb822fb7 100644
--- a/lib/eal/common/eal_common_trace.c
+++ b/lib/eal/common/eal_common_trace.c
@@ -12,6 +12,7 @@ 
 #include <rte_errno.h>
 #include <rte_lcore.h>
 #include <rte_per_lcore.h>
+#include <rte_pmu.h>
 #include <rte_string_fns.h>
 
 #include "eal_trace.h"
@@ -72,8 +73,11 @@  eal_trace_init(void)
 		goto free_meta;
 
 	/* Apply global configurations */
-	STAILQ_FOREACH(arg, &trace.args, next)
+	STAILQ_FOREACH(arg, &trace.args, next) {
 		trace_args_apply(arg->val);
+		if (rte_pmu_init() == 0)
+			rte_pmu_add_events_by_pattern(arg->val);
+	}
 
 	rte_trace_mode_set(trace.mode);
 
@@ -89,6 +93,7 @@  eal_trace_init(void)
 void
 eal_trace_fini(void)
 {
+	rte_pmu_fini();
 	trace_mem_free();
 	trace_metadata_destroy();
 	eal_trace_args_free();
diff --git a/lib/eal/common/eal_common_trace_points.c b/lib/eal/common/eal_common_trace_points.c
index 0f1240ea3a..05f7ed83d5 100644
--- a/lib/eal/common/eal_common_trace_points.c
+++ b/lib/eal/common/eal_common_trace_points.c
@@ -100,3 +100,8 @@  RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_enable,
 	lib.eal.intr.enable)
 RTE_TRACE_POINT_REGISTER(rte_eal_trace_intr_disable,
 	lib.eal.intr.disable)
+
+#ifdef RTE_EXEC_ENV_LINUX
+RTE_TRACE_POINT_REGISTER(rte_eal_trace_pmu_read,
+	lib.eal.pmu.read)
+#endif
diff --git a/lib/eal/include/rte_eal_trace.h b/lib/eal/include/rte_eal_trace.h
index 9ad2112801..6c1ed8e8e3 100644
--- a/lib/eal/include/rte_eal_trace.h
+++ b/lib/eal/include/rte_eal_trace.h
@@ -11,6 +11,7 @@ 
  * API for EAL trace support
  */
 
+#include <rte_pmu.h>
 #include <rte_trace_point.h>
 
 #ifdef __cplusplus
@@ -127,6 +128,14 @@  RTE_TRACE_POINT(
 
 #define RTE_EAL_TRACE_GENERIC_FUNC rte_eal_trace_generic_func(__func__)
 
+RTE_TRACE_POINT_FP(
+	rte_eal_trace_pmu_read,
+	RTE_TRACE_POINT_ARGS(unsigned int index),
+	uint64_t val;
+	val = rte_pmu_read(index);
+	rte_trace_point_emit_u64(val);
+)
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/eal/meson.build b/lib/eal/meson.build
index e1d6c4cf17..2b1a1eb283 100644
--- a/lib/eal/meson.build
+++ b/lib/eal/meson.build
@@ -14,7 +14,7 @@  subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['log', 'kvargs']
+deps += ['log', 'kvargs', 'pmu']
 if not is_windows
     deps += ['telemetry']
 endif
diff --git a/lib/eal/version.map b/lib/eal/version.map
index e3ff412683..17c56f6bf7 100644
--- a/lib/eal/version.map
+++ b/lib/eal/version.map
@@ -396,6 +396,9 @@  EXPERIMENTAL {
 
 	# added in 24.03
 	rte_vfio_get_device_info; # WINDOWS_NO_EXPORT
+
+	# added in 24.11
+	__rte_eal_trace_pmu_read; # WINDOWS_NO_EXPORT
 };
 
 INTERNAL {
diff --git a/lib/pmu/rte_pmu.c b/lib/pmu/rte_pmu.c
index 3a861bca71..0fbff25c92 100644
--- a/lib/pmu/rte_pmu.c
+++ b/lib/pmu/rte_pmu.c
@@ -403,6 +403,67 @@  rte_pmu_add_event(const char *name)
 	return event->index;
 }
 
+static int
+add_events(const char *pattern)
+{
+	char *token, *copy;
+	int ret = 0;
+
+	copy = strdup(pattern);
+	if (copy == NULL)
+		return -ENOMEM;
+
+	token = strtok(copy, ",");
+	while (token) {
+		ret = rte_pmu_add_event(token);
+		if (ret < 0)
+			break;
+
+		token = strtok(NULL, ",");
+	}
+
+	free(copy);
+
+	return ret >= 0 ? 0 : ret;
+}
+
+int
+rte_pmu_add_events_by_pattern(const char *pattern)
+{
+	regmatch_t rmatch;
+	char buf[BUFSIZ];
+	unsigned int num;
+	regex_t reg;
+	int ret;
+
+	/* events are matched against occurrences of e=ev1[,ev2,..] pattern */
+	ret = regcomp(&reg, "e=([_[:alnum:]-],?)+", REG_EXTENDED);
+	if (ret)
+		return -EINVAL;
+
+	for (;;) {
+		if (regexec(&reg, pattern, 1, &rmatch, 0))
+			break;
+
+		num = rmatch.rm_eo - rmatch.rm_so;
+		if (num > sizeof(buf))
+			num = sizeof(buf);
+
+		/* skip e= pattern prefix */
+		memcpy(buf, pattern + rmatch.rm_so + 2, num - 2);
+		buf[num - 2] = '\0';
+		ret = add_events(buf);
+		if (ret)
+			break;
+
+		pattern += rmatch.rm_eo;
+	}
+
+	regfree(&reg);
+
+	return ret;
+}
+
 int
 rte_pmu_init(void)
 {
diff --git a/lib/pmu/rte_pmu.h b/lib/pmu/rte_pmu.h
index 248bb55024..26264b3b8f 100644
--- a/lib/pmu/rte_pmu.h
+++ b/lib/pmu/rte_pmu.h
@@ -175,6 +175,20 @@  __rte_experimental
 int
 rte_pmu_add_event(const char *name);
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Add events matching pattern to the group of enabled events.
+ *
+ * @param pattern
+ *   Pattern e=ev1[,ev2,...] matching events, where evX is a placeholder for an event listed under
+ *   /sys/bus/event_source/devices/pmu/events.
+ */
+__rte_experimental
+int
+rte_pmu_add_events_by_pattern(const char *pattern);
+
 /**
  * @warning
  * @b EXPERIMENTAL: this API may change without prior notice
@@ -221,6 +235,10 @@  static inline void rte_pmu_fini(void) { }
 __rte_experimental
 static inline int rte_pmu_add_event(const char *name __rte_unused) { return -ENOTSUP; }
 
+__rte_experimental
+static inline int
+rte_pmu_add_events_by_pattern(const char *pattern __rte_unused) { return -ENOTSUP; }
+
 __rte_experimental
 static inline uint64_t rte_pmu_read(unsigned int index __rte_unused) { return UINT64_MAX; }
 
diff --git a/lib/pmu/version.map b/lib/pmu/version.map
index d7c80ce4ce..0c98209b4a 100644
--- a/lib/pmu/version.map
+++ b/lib/pmu/version.map
@@ -6,6 +6,7 @@  EXPERIMENTAL {
 	per_lcore__event_group;
 	rte_pmu;
 	rte_pmu_add_event;
+	rte_pmu_add_events_by_pattern;
 	rte_pmu_fini;
 	rte_pmu_init;
 	rte_pmu_read;