[v11,0/4] add support for self monitoring
Message ID | 20230216175502.3164820-1-tduszynski@marvell.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8871041CB3; Thu, 16 Feb 2023 18:55:35 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 21F3542D1A; Thu, 16 Feb 2023 18:55:35 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 5CE3540EE3 for <dev@dpdk.org>; Thu, 16 Feb 2023 18:55:33 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 31GH3hKY007079; Thu, 16 Feb 2023 09:55:30 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=pfpt0220; bh=FTH4G8LayQjvO6LryfWUe0fe+zlDqVqWqSlnKq/rsi4=; b=YPi6/6Ty37CEYrMZvUwMVpHlr1BW7K6/LiHjz+6S/3F+fe8PDbqAGkrJzLG+cSrZLkvu kBJc9VOYvwewVLJAHPaNlp44aL0jmhZ8eaPSJbdnBNEiFnUXtINc802VQpcfbIkzYqD0 QFatRECySbRcZ+lkenHsMKFwFRgYmIg+EWt0esewdG8xwZPWm+m6MjdqzaIAJjATjum8 xNj0T7/n/1INOJSLmcVGFBpp7xehmm7+qnqM9SC4Nbj9u/dWqhJIpzd5QDjne+TbzEkj t2dLHMAWCJ0FwBgfkVhzuV9m6+owmaY8u/YbhXlM1zSxU2AMQTz8dr0zDiywTQJ4Z6cC 4w== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3nsg6wb0cp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 16 Feb 2023 09:55:30 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Thu, 16 Feb 2023 09:55:27 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Thu, 16 Feb 2023 09:55:27 -0800 Received: from cavium-DT10.. (unknown [10.28.34.39]) by maili.marvell.com (Postfix) with ESMTP id 797A13F7097; Thu, 16 Feb 2023 09:55:24 -0800 (PST) From: Tomasz Duszynski <tduszynski@marvell.com> To: <dev@dpdk.org> CC: <roretzla@linux.microsoft.com>, <Ruifeng.Wang@arm.com>, <bruce.richardson@intel.com>, <jerinj@marvell.com>, <mattias.ronnblom@ericsson.com>, <mb@smartsharesystems.com>, <thomas@monjalon.net>, <zhoumin@loongson.cn>, <david.marchand@redhat.com>, Tomasz Duszynski <tduszynski@marvell.com> Subject: [PATCH v11 0/4] add support for self monitoring Date: Thu, 16 Feb 2023 18:54:58 +0100 Message-ID: <20230216175502.3164820-1-tduszynski@marvell.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230213113156.385482-1-tduszynski@marvell.com> References: <20230213113156.385482-1-tduszynski@marvell.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Proofpoint-ORIG-GUID: xOWpd11DRx-wfzQIbdIIPCS62ZmXhY1s X-Proofpoint-GUID: xOWpd11DRx-wfzQIbdIIPCS62ZmXhY1s X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.170.22 definitions=2023-02-16_14,2023-02-16_01,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org |
Message
Tomasz Duszynski
Feb. 16, 2023, 5:54 p.m. UTC
This series adds self monitoring support i.e allows to configure and read performance measurement unit (PMU) counters in runtime without using perf utility. This has certain advantages when application runs on isolated cores running dedicated tasks. Events can be read directly using rte_pmu_read() or using dedicated tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be stored inside CTF file. By design, all enabled events are grouped together and the same group is attached to lcores that use self monitoring funtionality. Events are enabled by names, which need to be read from standard location under sysfs i.e /sys/bus/event_source/devices/PMU/events where PMU is a core pmu i.e one measuring cpu events. As of today raw events are not supported. v11: - skip fast test in case init fails v10: - check permissions before using counters - do not use internal symbols in exported functions - address review comments v9: - fix 'maybe-uninitialized' warning reported by CI v8: - just rebase series v7: - use per-lcore event group instead of global table index by lcore-id - don't add pmu_autotest to fast tests because due to lack of suported on every arch v6: - move codebase to the separate library - address review comments v5: - address review comments - fix sign extension while reading pmu on x86 - fix regex mentioned in doc - various minor changes/improvements here and there v4: - fix freeing mem detected by debug_autotest v3: - fix shared build v2: - fix problems reported by test build infra Tomasz Duszynski (4): lib: add generic support for reading PMU events pmu: support reading ARM PMU events in runtime pmu: support reading Intel x86_64 PMU events in runtime eal: add PMU support to tracing library MAINTAINERS | 5 + app/test/meson.build | 2 + app/test/test_pmu.c | 68 +++ app/test/test_trace_perf.c | 10 + doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + doc/guides/prog_guide/profile_app.rst | 17 + doc/guides/prog_guide/trace_lib.rst | 32 ++ doc/guides/rel_notes/release_23_03.rst | 7 + lib/eal/common/eal_common_trace.c | 13 +- lib/eal/common/eal_common_trace_points.c | 5 + lib/eal/include/rte_eal_trace.h | 13 + lib/eal/meson.build | 3 + lib/eal/version.map | 1 + lib/meson.build | 1 + lib/pmu/meson.build | 21 + lib/pmu/pmu_arm64.c | 94 ++++ lib/pmu/pmu_private.h | 32 ++ lib/pmu/rte_pmu.c | 521 +++++++++++++++++++++++ lib/pmu/rte_pmu.h | 232 ++++++++++ lib/pmu/rte_pmu_pmc_arm64.h | 30 ++ lib/pmu/rte_pmu_pmc_x86_64.h | 24 ++ lib/pmu/version.map | 16 + 23 files changed, 1149 insertions(+), 2 deletions(-) create mode 100644 app/test/test_pmu.c create mode 100644 lib/pmu/meson.build create mode 100644 lib/pmu/pmu_arm64.c create mode 100644 lib/pmu/pmu_private.h create mode 100644 lib/pmu/rte_pmu.c create mode 100644 lib/pmu/rte_pmu.h create mode 100644 lib/pmu/rte_pmu_pmc_arm64.h create mode 100644 lib/pmu/rte_pmu_pmc_x86_64.h create mode 100644 lib/pmu/version.map -- 2.34.1
Comments
Ping for update What is the status of this feature? 04/05/2023 10:02, David Marchand: > Hello Tomasz, > > On Thu, Feb 16, 2023 at 6:55 PM Tomasz Duszynski <tduszynski@marvell.com> wrote: > > > > This series adds self monitoring support i.e allows to configure and > > read performance measurement unit (PMU) counters in runtime without > > using perf utility. This has certain advantages when application runs on > > isolated cores running dedicated tasks. > > > > Events can be read directly using rte_pmu_read() or using dedicated > > tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be > > stored inside CTF file. > > > > By design, all enabled events are grouped together and the same group > > is attached to lcores that use self monitoring funtionality. > > > > Events are enabled by names, which need to be read from standard > > location under sysfs i.e > > > > /sys/bus/event_source/devices/PMU/events > > > > where PMU is a core pmu i.e one measuring cpu events. As of today > > raw events are not supported. > > > > Tomasz Duszynski (4): > > lib: add generic support for reading PMU events > > pmu: support reading ARM PMU events in runtime > > pmu: support reading Intel x86_64 PMU events in runtime > > eal: add PMU support to tracing library > > There are still some pending comments on this series and it can't be > merged until they get sorted out. > > I noted two points : > - Konstantin asked for better explanations in the implementation. > - He also pointed out at using this feature with non EAL lcores. > > Could you work on this so we can consider this series for v23.07?
>-----Original Message----- >From: Thomas Monjalon <thomas@monjalon.net> >Sent: Monday, July 31, 2023 2:33 PM >To: Tomasz Duszynski <tduszynski@marvell.com> >Cc: dev@dpdk.org; roretzla@linux.microsoft.com; Ruifeng.Wang@arm.com; bruce.richardson@intel.com; >Jerin Jacob Kollanukkaran <jerinj@marvell.com>; mattias.ronnblom@ericsson.com; >mb@smartsharesystems.com; zhoumin@loongson.cn; Konstantin Ananyev ><konstantin.ananyev@huawei.com>; David Marchand <david.marchand@redhat.com> >Subject: [EXT] Re: [PATCH v11 0/4] add support for self monitoring > >External Email > >---------------------------------------------------------------------- >Ping for update >What is the status of this feature? > > Hi Thomas, I'll re-spin the series soon. >04/05/2023 10:02, David Marchand: >> Hello Tomasz, >> >> On Thu, Feb 16, 2023 at 6:55 PM Tomasz Duszynski <tduszynski@marvell.com> wrote: >> > >> > This series adds self monitoring support i.e allows to configure and >> > read performance measurement unit (PMU) counters in runtime without >> > using perf utility. This has certain advantages when application >> > runs on isolated cores running dedicated tasks. >> > >> > Events can be read directly using rte_pmu_read() or using dedicated >> > tracepoint rte_eal_trace_pmu_read(). The latter will cause events to >> > be stored inside CTF file. >> > >> > By design, all enabled events are grouped together and the same >> > group is attached to lcores that use self monitoring funtionality. >> > >> > Events are enabled by names, which need to be read from standard >> > location under sysfs i.e >> > >> > /sys/bus/event_source/devices/PMU/events >> > >> > where PMU is a core pmu i.e one measuring cpu events. As of today >> > raw events are not supported. >> > >> > Tomasz Duszynski (4): >> > lib: add generic support for reading PMU events >> > pmu: support reading ARM PMU events in runtime >> > pmu: support reading Intel x86_64 PMU events in runtime >> > eal: add PMU support to tracing library >> >> There are still some pending comments on this series and it can't be >> merged until they get sorted out. >> >> I noted two points : >> - Konstantin asked for better explanations in the implementation. >> - He also pointed out at using this feature with non EAL lcores. >> >> Could you work on this so we can consider this series for v23.07? > > > >
Hello, On Mon, Aug 7, 2023 at 10:12 AM Tomasz Duszynski <tduszynski@marvell.com> wrote: > >Ping for update > >What is the status of this feature? > > I'll re-spin the series soon. -rc1 is getting closer. Any update?
> From: David Marchand [mailto:david.marchand@redhat.com] > Sent: Thursday, 21 September 2023 10.27 > > Hello, > > On Mon, Aug 7, 2023 at 10:12 AM Tomasz Duszynski <tduszynski@marvell.com> > wrote: > > >Ping for update > > >What is the status of this feature? > > > > I'll re-spin the series soon. > > -rc1 is getting closer. > Any update? @Thomasz, @Jerin, I expect this feature to be useful for profiling of systems deployed in production, and would like to see it go into DPDK. What is its status? -Morten
[...] > > From: David Marchand [mailto:david.marchand@redhat.com] > > Sent: Thursday, 21 September 2023 10.27 > > > > Hello, > > > > On Mon, Aug 7, 2023 at 10:12 AM Tomasz Duszynski > > <tduszynski@marvell.com> > > wrote: > > > >Ping for update > > > >What is the status of this feature? > > > > > > I'll re-spin the series soon. > > > > -rc1 is getting closer. > > Any update? > > @Thomasz, @Jerin, > > I expect this feature to be useful for profiling of systems deployed in production, and would like > to see it go into DPDK. > > What is its status? > > -Morten Hi Morten, I was buried in other things and did not work on that so to speak. If there's still need for this feature, as you say, I'll respin this series for good. Hopefully new version will be available this month.
Hello Tomasz, On Sat, Sep 28, 2024 at 12:07 AM Tomasz Duszynski <tduszynski@marvell.com> wrote: > > This series adds self monitoring support i.e allows to configure and > read performance measurement unit (PMU) counters in runtime without > using perf utility. This has certain advantages when application runs on > isolated cores running dedicated tasks. > > Events can be read directly using rte_pmu_read() or using dedicated > tracepoint rte_eal_trace_pmu_read(). The latter will cause events to be > stored inside CTF file. > > By design, all enabled events are grouped together and the same group > is attached to lcores that use self monitoring funtionality. > > Events are enabled by names, which need to be read from standard > location under sysfs i.e > > /sys/bus/event_source/devices/PMU/events > > where PMU is a core pmu i.e one measuring cpu events. As of today > raw events are not supported. > > v12: > - rebase old series It seems the series was rebased on next-net (I see a reference to some RN entry in next-net). Can you rebase on main so CI can test it? Thanks.
>Hello Tomasz, > >On Sat, Sep 28, 2024 at 12:07 AM Tomasz Duszynski <tduszynski@marvell.com> wrote: >> >> This series adds self monitoring support i.e allows to configure and >> read performance measurement unit (PMU) counters in runtime without >> using perf utility. This has certain advantages when application runs >> on isolated cores running dedicated tasks. >> >> Events can be read directly using rte_pmu_read() or using dedicated >> tracepoint rte_eal_trace_pmu_read(). The latter will cause events to >> be stored inside CTF file. >> >> By design, all enabled events are grouped together and the same group >> is attached to lcores that use self monitoring funtionality. >> >> Events are enabled by names, which need to be read from standard >> location under sysfs i.e >> >> /sys/bus/event_source/devices/PMU/events >> >> where PMU is a core pmu i.e one measuring cpu events. As of today raw >> events are not supported. >> >> v12: >> - rebase old series > >It seems the series was rebased on next-net (I see a reference to some RN entry in next-net). >Can you rebase on main so CI can test it? > >Thanks. Missed your comment prior sending v13. I'll resend v14 rebased on main. > >-- >David Marchand