mbox

[v11,0/4] add support for self monitoring

Message ID 20230216175502.3164820-1-tduszynski@marvell.com (mailing list archive)
Headers

Message

Tomasz Duszynski Feb. 16, 2023, 5:54 p.m. UTC
This series adds self monitoring support i.e allows to configure and
read performance measurement unit (PMU) counters in runtime without
using perf utility. This has certain advantages when application runs on
isolated cores running dedicated tasks.

Events can be read directly using rte_pmu_read() or using dedicated
tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be
stored inside CTF file.

By design, all enabled events are grouped together and the same group
is attached to lcores that use self monitoring funtionality.

Events are enabled by names, which need to be read from standard
location under sysfs i.e

/sys/bus/event_source/devices/PMU/events

where PMU is a core pmu i.e one measuring cpu events. As of today
raw events are not supported.

v11:
- skip fast test in case init fails
v10:
- check permissions before using counters
- do not use internal symbols in exported functions
- address review comments
v9:
- fix 'maybe-uninitialized' warning reported by CI
v8:
- just rebase series
v7:
- use per-lcore event group instead of global table index by lcore-id
- don't add pmu_autotest to fast tests because due to lack of suported on
  every arch
v6:
- move codebase to the separate library
- address review comments
v5:
- address review comments
- fix sign extension while reading pmu on x86
- fix regex mentioned in doc
- various minor changes/improvements here and there
v4:
- fix freeing mem detected by debug_autotest
v3:
- fix shared build
v2:
- fix problems reported by test build infra

Tomasz Duszynski (4):
  lib: add generic support for reading PMU events
  pmu: support reading ARM PMU events in runtime
  pmu: support reading Intel x86_64 PMU events in runtime
  eal: add PMU support to tracing library

 MAINTAINERS                              |   5 +
 app/test/meson.build                     |   2 +
 app/test/test_pmu.c                      |  68 +++
 app/test/test_trace_perf.c               |  10 +
 doc/api/doxy-api-index.md                |   3 +-
 doc/api/doxy-api.conf.in                 |   1 +
 doc/guides/prog_guide/profile_app.rst    |  17 +
 doc/guides/prog_guide/trace_lib.rst      |  32 ++
 doc/guides/rel_notes/release_23_03.rst   |   7 +
 lib/eal/common/eal_common_trace.c        |  13 +-
 lib/eal/common/eal_common_trace_points.c |   5 +
 lib/eal/include/rte_eal_trace.h          |  13 +
 lib/eal/meson.build                      |   3 +
 lib/eal/version.map                      |   1 +
 lib/meson.build                          |   1 +
 lib/pmu/meson.build                      |  21 +
 lib/pmu/pmu_arm64.c                      |  94 ++++
 lib/pmu/pmu_private.h                    |  32 ++
 lib/pmu/rte_pmu.c                        | 521 +++++++++++++++++++++++
 lib/pmu/rte_pmu.h                        | 232 ++++++++++
 lib/pmu/rte_pmu_pmc_arm64.h              |  30 ++
 lib/pmu/rte_pmu_pmc_x86_64.h             |  24 ++
 lib/pmu/version.map                      |  16 +
 23 files changed, 1149 insertions(+), 2 deletions(-)
 create mode 100644 app/test/test_pmu.c
 create mode 100644 lib/pmu/meson.build
 create mode 100644 lib/pmu/pmu_arm64.c
 create mode 100644 lib/pmu/pmu_private.h
 create mode 100644 lib/pmu/rte_pmu.c
 create mode 100644 lib/pmu/rte_pmu.h
 create mode 100644 lib/pmu/rte_pmu_pmc_arm64.h
 create mode 100644 lib/pmu/rte_pmu_pmc_x86_64.h
 create mode 100644 lib/pmu/version.map

--
2.34.1
  

Comments

Thomas Monjalon July 31, 2023, 12:33 p.m. UTC | #1
Ping for update
What is the status of this feature?


04/05/2023 10:02, David Marchand:
> Hello Tomasz,
> 
> On Thu, Feb 16, 2023 at 6:55 PM Tomasz Duszynski <tduszynski@marvell.com> wrote:
> >
> > This series adds self monitoring support i.e allows to configure and
> > read performance measurement unit (PMU) counters in runtime without
> > using perf utility. This has certain advantages when application runs on
> > isolated cores running dedicated tasks.
> >
> > Events can be read directly using rte_pmu_read() or using dedicated
> > tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be
> > stored inside CTF file.
> >
> > By design, all enabled events are grouped together and the same group
> > is attached to lcores that use self monitoring funtionality.
> >
> > Events are enabled by names, which need to be read from standard
> > location under sysfs i.e
> >
> > /sys/bus/event_source/devices/PMU/events
> >
> > where PMU is a core pmu i.e one measuring cpu events. As of today
> > raw events are not supported.
> >
> > Tomasz Duszynski (4):
> >   lib: add generic support for reading PMU events
> >   pmu: support reading ARM PMU events in runtime
> >   pmu: support reading Intel x86_64 PMU events in runtime
> >   eal: add PMU support to tracing library
> 
> There are still some pending comments on this series and it can't be
> merged until they get sorted out.
> 
> I noted two points :
> - Konstantin asked for better explanations in the implementation.
> - He also pointed out at using this feature with non EAL lcores.
> 
> Could you work on this so we can consider this series for v23.07?
  
Tomasz Duszynski Aug. 7, 2023, 8:11 a.m. UTC | #2
>-----Original Message-----
>From: Thomas Monjalon <thomas@monjalon.net>
>Sent: Monday, July 31, 2023 2:33 PM
>To: Tomasz Duszynski <tduszynski@marvell.com>
>Cc: dev@dpdk.org; roretzla@linux.microsoft.com; Ruifeng.Wang@arm.com; bruce.richardson@intel.com;
>Jerin Jacob Kollanukkaran <jerinj@marvell.com>; mattias.ronnblom@ericsson.com;
>mb@smartsharesystems.com; zhoumin@loongson.cn; Konstantin Ananyev
><konstantin.ananyev@huawei.com>; David Marchand <david.marchand@redhat.com>
>Subject: [EXT] Re: [PATCH v11 0/4] add support for self monitoring
>
>External Email
>
>----------------------------------------------------------------------
>Ping for update
>What is the status of this feature?
>
>

Hi Thomas, 

I'll re-spin the series soon. 

>04/05/2023 10:02, David Marchand:
>> Hello Tomasz,
>>
>> On Thu, Feb 16, 2023 at 6:55 PM Tomasz Duszynski <tduszynski@marvell.com> wrote:
>> >
>> > This series adds self monitoring support i.e allows to configure and
>> > read performance measurement unit (PMU) counters in runtime without
>> > using perf utility. This has certain advantages when application
>> > runs on isolated cores running dedicated tasks.
>> >
>> > Events can be read directly using rte_pmu_read() or using dedicated
>> > tracepoint rte_eal_trace_pmu_read(). The latter will cause events to
>> > be stored inside CTF file.
>> >
>> > By design, all enabled events are grouped together and the same
>> > group is attached to lcores that use self monitoring funtionality.
>> >
>> > Events are enabled by names, which need to be read from standard
>> > location under sysfs i.e
>> >
>> > /sys/bus/event_source/devices/PMU/events
>> >
>> > where PMU is a core pmu i.e one measuring cpu events. As of today
>> > raw events are not supported.
>> >
>> > Tomasz Duszynski (4):
>> >   lib: add generic support for reading PMU events
>> >   pmu: support reading ARM PMU events in runtime
>> >   pmu: support reading Intel x86_64 PMU events in runtime
>> >   eal: add PMU support to tracing library
>>
>> There are still some pending comments on this series and it can't be
>> merged until they get sorted out.
>>
>> I noted two points :
>> - Konstantin asked for better explanations in the implementation.
>> - He also pointed out at using this feature with non EAL lcores.
>>
>> Could you work on this so we can consider this series for v23.07?
>
>
>
>
  
David Marchand Sept. 21, 2023, 8:26 a.m. UTC | #3
Hello,

On Mon, Aug 7, 2023 at 10:12 AM Tomasz Duszynski <tduszynski@marvell.com> wrote:
> >Ping for update
> >What is the status of this feature?
>
> I'll re-spin the series soon.

-rc1 is getting closer.
Any update?
  
Morten Brørup Sept. 2, 2024, 2:48 p.m. UTC | #4
> From: David Marchand [mailto:david.marchand@redhat.com]
> Sent: Thursday, 21 September 2023 10.27
> 
> Hello,
> 
> On Mon, Aug 7, 2023 at 10:12 AM Tomasz Duszynski <tduszynski@marvell.com>
> wrote:
> > >Ping for update
> > >What is the status of this feature?
> >
> > I'll re-spin the series soon.
> 
> -rc1 is getting closer.
> Any update?

@Thomasz, @Jerin,

I expect this feature to be useful for profiling of systems deployed in production, and would like to see it go into DPDK.

What is its status?

-Morten
  
Tomasz Duszynski Sept. 5, 2024, 3:49 a.m. UTC | #5
[...]

> > From: David Marchand [mailto:david.marchand@redhat.com]
> > Sent: Thursday, 21 September 2023 10.27
> >
> > Hello,
> >
> > On Mon, Aug 7, 2023 at 10:12 AM Tomasz Duszynski
> > <tduszynski@marvell.com>
> > wrote:
> > > >Ping for update
> > > >What is the status of this feature?
> > >
> > > I'll re-spin the series soon.
> >
> > -rc1 is getting closer.
> > Any update?
> 
> @Thomasz, @Jerin,
> 
> I expect this feature to be useful for profiling of systems deployed in production, and would like
> to see it go into DPDK.
> 
> What is its status?
> 
> -Morten

Hi Morten, 

I was buried in other things and did not work on that so to speak. If there's still need 
for this feature, as you say, I'll respin this series for good. Hopefully new version 
will be available this month.
  
David Marchand Oct. 4, 2024, 11 a.m. UTC | #6
Hello Tomasz,

On Sat, Sep 28, 2024 at 12:07 AM Tomasz Duszynski
<tduszynski@marvell.com> wrote:
>
> This series adds self monitoring support i.e allows to configure and
> read performance measurement unit (PMU) counters in runtime without
> using perf utility. This has certain advantages when application runs on
> isolated cores running dedicated tasks.
>
> Events can be read directly using rte_pmu_read() or using dedicated
> tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be
> stored inside CTF file.
>
> By design, all enabled events are grouped together and the same group
> is attached to lcores that use self monitoring funtionality.
>
> Events are enabled by names, which need to be read from standard
> location under sysfs i.e
>
> /sys/bus/event_source/devices/PMU/events
>
> where PMU is a core pmu i.e one measuring cpu events. As of today
> raw events are not supported.
>
> v12:
> - rebase old series

It seems the series was rebased on next-net (I see a reference to some
RN entry in next-net).
Can you rebase on main so CI can test it?

Thanks.
  
Tomasz Duszynski Oct. 9, 2024, 12:45 p.m. UTC | #7
>Hello Tomasz,
>
>On Sat, Sep 28, 2024 at 12:07 AM Tomasz Duszynski <tduszynski@marvell.com> wrote:
>>
>> This series adds self monitoring support i.e allows to configure and
>> read performance measurement unit (PMU) counters in runtime without
>> using perf utility. This has certain advantages when application runs
>> on isolated cores running dedicated tasks.
>>
>> Events can be read directly using rte_pmu_read() or using dedicated
>> tracepoint rte_eal_trace_pmu_read(). The latter will cause events to
>> be stored inside CTF file.
>>
>> By design, all enabled events are grouped together and the same group
>> is attached to lcores that use self monitoring funtionality.
>>
>> Events are enabled by names, which need to be read from standard
>> location under sysfs i.e
>>
>> /sys/bus/event_source/devices/PMU/events
>>
>> where PMU is a core pmu i.e one measuring cpu events. As of today raw
>> events are not supported.
>>
>> v12:
>> - rebase old series
>
>It seems the series was rebased on next-net (I see a reference to some RN entry in next-net).
>Can you rebase on main so CI can test it?
>
>Thanks.

Missed your comment prior sending v13. I'll resend v14 rebased on main. 

>
>--
>David Marchand