From patchwork Tue Sep 30 23:25:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Monjalon X-Patchwork-Id: 157151 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 224C048877; Wed, 1 Oct 2025 01:38:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1FD9B40670; Wed, 1 Oct 2025 01:38:53 +0200 (CEST) Received: from fout-b1-smtp.messagingengine.com (fout-b1-smtp.messagingengine.com [202.12.124.144]) by mails.dpdk.org (Postfix) with ESMTP id A17B74065D for ; Wed, 1 Oct 2025 01:38:51 +0200 (CEST) Received: from phl-compute-11.internal (phl-compute-11.internal [10.202.2.51]) by mailfout.stl.internal (Postfix) with ESMTP id CA54E1D00135; Tue, 30 Sep 2025 19:38:50 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-11.internal (MEProxy); Tue, 30 Sep 2025 19:38:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm2; t=1759275530; x= 1759361930; bh=S8HZcsTbrH5lG1POubh+56/8oulOXfugHauOTy6xObA=; b=X xDNXRrkVHOI19k+IFjGV0U0IJUtvne7BQa8bbUZ8WPM2A4N9L9fhltJ6noXQtlqK k7w1PXJtfPXUyqFtyTUkbDd3UPorhRmlEvOE+gnmeTG/LYK7MnMo+Z4tOVqZ/RVs t5QnPv08j0cFK6FXB0EKZPNTHjtr96VxOftozUPeHzthMpV8OdmUJwDlS56AlR7A FslSaMgI/CSIp4DHOj5nde0EM/q8j5+Cu9z2aTTeDOsRmrnhZrCx/s0K4A0yJX6h 38gFCD7ibIMIxWfmcIy1QuHp6ptpczumeMaEU9E2Uc64dEr5uEllowkRsr2b18W2 SbD4rsOGksmJM5LeQIuwQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1759275530; x=1759361930; bh=S 8HZcsTbrH5lG1POubh+56/8oulOXfugHauOTy6xObA=; b=OSfOWzjf7/nDp1U2c bGxvzjPJwBlf9obrJzrctug80XCQ98P87n45yAG274b9uO5JgKIkzIMlHivT/MUA 3actjA0WUB7CkLNPxTwvPSo237rzFuqXqOLUNNI9kMsnR6u/KYJdtNOEQC3/zAna PYy0GBPhjuRIJMWHNvMpH+tRgcASfByU7fhQWMPwtjkRTk7G1zvJrja0okHkNSez QsJoDlEzPKOYagopFxNawfazEF4hcQySJvDF82zdl90wnTI0UBSJKTU4TL9j5tA4 Y4P/++qZ2fd2mNSl4NuvrGdNZzjEA1tczPALVc1wthDcRqAIJCTic4lr8i1b23fK WeyqA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdekudejjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecunecujfgurhephffvvefufffkofgjfhgggfestdekredtre dttdenucfhrhhomhepvfhhohhmrghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhho nhhjrghlohhnrdhnvghtqeenucggtffrrghtthgvrhhnpedvjefhudeghedvtdeijeeige etuedugfejueekieeltdfhteevkeehhfeilefhtdenucevlhhushhtvghrufhiiigvpedt necurfgrrhgrmhepmhgrihhlfhhrohhmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvg htpdhnsggprhgtphhtthhopeeipdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegu vghvseguphgukhdrohhrghdprhgtphhtthhopehshhhpvghrvghtiiesnhhvihguihgrrd gtohhmpdhrtghpthhtohepvhhirggthhgvshhlrghvohesnhhvihguihgrrdgtohhmpdhr tghpthhtohepsghruhgtvgdrrhhitghhrghrughsohhnsehinhhtvghlrdgtohhmpdhrtg hpthhtohepmhgssehsmhgrrhhtshhhrghrvghshihsthgvmhhsrdgtohhmpdhrtghpthht ohepshhtvghphhgvnhesnhgvthifohhrkhhplhhumhgsvghrrdhorhhg X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 30 Sep 2025 19:38:49 -0400 (EDT) From: Thomas Monjalon To: dev@dpdk.org Cc: shperetz@nvidia.com, viacheslavo@nvidia.com, bruce.richardson@intel.com, mb@smartsharesystems.com, stephen@networkplumber.org Subject: [PATCH v3 1/5] mbuf: move header include for logs Date: Wed, 1 Oct 2025 01:25:02 +0200 Message-ID: <20250930233828.3999565-2-thomas@monjalon.net> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250930233828.3999565-1-thomas@monjalon.net> References: <20250616072910.113042-1-shperetz@nvidia.com> <20250930233828.3999565-1-thomas@monjalon.net> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The header mbuf_log.h depends on rte_log.h, so it should include it. After this move, no need to include rte_log.h in addition of mbuf_log.h. Signed-off-by: Thomas Monjalon --- lib/mbuf/mbuf_log.h | 2 ++ lib/mbuf/rte_mbuf.c | 1 - 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/mbuf/mbuf_log.h b/lib/mbuf/mbuf_log.h index 8feaf20747..f163cbd71c 100644 --- a/lib/mbuf/mbuf_log.h +++ b/lib/mbuf/mbuf_log.h @@ -1,5 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause */ +#include + extern int mbuf_logtype; #define RTE_LOGTYPE_MBUF mbuf_logtype #define MBUF_LOG(level, ...) \ diff --git a/lib/mbuf/rte_mbuf.c b/lib/mbuf/rte_mbuf.c index 9e7731a8a2..22d7cfc475 100644 --- a/lib/mbuf/rte_mbuf.c +++ b/lib/mbuf/rte_mbuf.c @@ -12,7 +12,6 @@ #include #include #include -#include #include #include #include From patchwork Tue Sep 30 23:25:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Monjalon X-Patchwork-Id: 157152 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24D1548877; Wed, 1 Oct 2025 01:39:01 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 68B8240678; Wed, 1 Oct 2025 01:38:56 +0200 (CEST) Received: from fout-b1-smtp.messagingengine.com (fout-b1-smtp.messagingengine.com [202.12.124.144]) by mails.dpdk.org (Postfix) with ESMTP id 54EA340678 for ; Wed, 1 Oct 2025 01:38:55 +0200 (CEST) Received: from phl-compute-02.internal (phl-compute-02.internal [10.202.2.42]) by mailfout.stl.internal (Postfix) with ESMTP id ADC3A1D00135; Tue, 30 Sep 2025 19:38:54 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-02.internal (MEProxy); Tue, 30 Sep 2025 19:38:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm2; t=1759275534; x= 1759361934; bh=bQWa7c7fKPlMPm8wobX1vV5h0keSyp83EFAnlxcnEZQ=; b=L 9mNYZrUcxTsL1kXpDPJXgBIstuhnV/R6OtZsKdP2OvI12UQ+OhTB4sLqC5V478x5 l3iADmvzQLgu6h0IFbuohohxyfnQfbAlK0q2tHY2qgpgHnUXH7LJau8ZCYdW4oG/ 6Cv2iVO4HnYpgCHjSebUdL1sNdvyVn+j94RRxpyAHeoqryO/eTrpWqabQUFp5LXA h05TiNIoQrbiWQpQaqgm2GyQsWsUn4xnUw4L2K/x1aqnkaSXuNO3tnsMWEzg1kUB wfAVbZEWZblEAldjKRvvqvpzyWSH9A0u7tzq3ZfXaRdWjWP6o7mYH5YrvteSP588 sK+E0TUVI+NN2/PlOjmCw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1759275534; x=1759361934; bh=b QWa7c7fKPlMPm8wobX1vV5h0keSyp83EFAnlxcnEZQ=; b=XOINBPQOLlblhk1wD 2TqOxZKHMtYWZS4goabYY2oiMcjuHDAPkexck7oaNf05yYVeRhK/chLhPSdV6E3S 3cEZXwjxv0u40FTAvDynmuju9a9dHtLuKMgaiOY1mRpBDbbzj1BXA5t8Oe2g9f3j KUIuTTXw3M9XJ7+V1jNToOfO9mX4N6cPu4+9XC9Opg5hv47dWlObbo4cmi12i1Kg 0iKmObxPQ3uQDleu8Hinwn5IAiXPl0/5Jab4sy7G7AHj1tZJikTKDGZQzvx5UZ2N uV1/R04dZljydQH9tN68rJ2M3yQqnrgQdvgxR95CdZt3LNCEvE3psg+CvpE5JRk9 MKBLQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdekudejiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecunecujfgurhephffvvefufffkofgjfhgggfestdekredtre dttdenucfhrhhomhepvfhhohhmrghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhho nhhjrghlohhnrdhnvghtqeenucggtffrrghtthgvrhhnpeetueehvdevheettefhgeette ejueffkefgvdegkedttedtudeugeeugfevieegteenucffohhmrghinhepfihikhhiphgv ughirgdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtpdhnsggprhgtphhtthhopeei pdhmohguvgepshhmthhpohhuthdprhgtphhtthhopeguvghvseguphgukhdrohhrghdprh gtphhtthhopehshhhpvghrvghtiiesnhhvihguihgrrdgtohhmpdhrtghpthhtohepvhhi rggthhgvshhlrghvohesnhhvihguihgrrdgtohhmpdhrtghpthhtohepsghruhgtvgdrrh hitghhrghrughsohhnsehinhhtvghlrdgtohhmpdhrtghpthhtohepmhgssehsmhgrrhht shhhrghrvghshihsthgvmhhsrdgtohhmpdhrtghpthhtohepshhtvghphhgvnhesnhgvth ifohhrkhhplhhumhgsvghrrdhorhhg X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 30 Sep 2025 19:38:52 -0400 (EDT) From: Thomas Monjalon To: dev@dpdk.org Cc: shperetz@nvidia.com, viacheslavo@nvidia.com, bruce.richardson@intel.com, mb@smartsharesystems.com, stephen@networkplumber.org Subject: [PATCH v3 2/5] mbuf: record mbuf operations history Date: Wed, 1 Oct 2025 01:25:03 +0200 Message-ID: <20250930233828.3999565-3-thomas@monjalon.net> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250930233828.3999565-1-thomas@monjalon.net> References: <20250616072910.113042-1-shperetz@nvidia.com> <20250930233828.3999565-1-thomas@monjalon.net> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Shani Peretz This feature is designed to monitor the lifecycle of mbufs as they move between the application and the PMD. It will allow to track the operations and transitions of each mbuf throughout the system, helping in debugging and understanding objects flow. As a debug feature impacting the data path, it is disabled by default. The implementation uses a dynamic field to store a 64-bit atomic history value in each mbuf. Each operation is represented by a 4-bit value, allowing up to 16 operations to be tracked. Atomic operations ensure thread safety for cloned mbufs accessed by multiple lcores. The dynamic field is automatically initialized when the first mbuf pool is created. Some operations done in the mbuf library are marked. More operations from other libraries or the application can be marked. Signed-off-by: Shani Peretz Signed-off-by: Thomas Monjalon --- config/meson.build | 1 + lib/mbuf/mbuf_history.c | 214 ++++++++++++++++++++++++++++++++++++ lib/mbuf/meson.build | 2 + lib/mbuf/rte_mbuf.c | 6 +- lib/mbuf/rte_mbuf.h | 13 ++- lib/mbuf/rte_mbuf_dyn.h | 18 +++ lib/mbuf/rte_mbuf_history.h | 174 +++++++++++++++++++++++++++++ meson_options.txt | 2 + 8 files changed, 427 insertions(+), 3 deletions(-) create mode 100644 lib/mbuf/mbuf_history.c create mode 100644 lib/mbuf/rte_mbuf_history.h diff --git a/config/meson.build b/config/meson.build index 55497f0bf5..d1f21f3115 100644 --- a/config/meson.build +++ b/config/meson.build @@ -379,6 +379,7 @@ if get_option('mbuf_refcnt_atomic') dpdk_conf.set('RTE_MBUF_REFCNT_ATOMIC', true) endif dpdk_conf.set10('RTE_IOVA_IN_MBUF', get_option('enable_iova_as_pa')) +dpdk_conf.set10('RTE_MBUF_HISTORY_DEBUG', get_option('enable_mbuf_history')) compile_time_cpuflags = [] subdir(arch_subdir) diff --git a/lib/mbuf/mbuf_history.c b/lib/mbuf/mbuf_history.c new file mode 100644 index 0000000000..852cf0f89a --- /dev/null +++ b/lib/mbuf/mbuf_history.c @@ -0,0 +1,214 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 NVIDIA Corporation & Affiliates + */ + +#include +#include +#include +#include + +#include "rte_mbuf_history.h" +#include "rte_mbuf_dyn.h" +#include "rte_mbuf.h" +#include "mbuf_log.h" + +/* Dynamic field offset */ +RTE_EXPORT_SYMBOL(rte_mbuf_history_field_offset); +int rte_mbuf_history_field_offset = -1; + +#if RTE_MBUF_HISTORY_DEBUG + +#define HISTORY_LAST_MASK (RTE_BIT64(RTE_MBUF_HISTORY_BITS) - 1) + +/* Dynamic field definition for mbuf history */ +static const struct rte_mbuf_dynfield mbuf_dynfield_history = { + .name = RTE_MBUF_DYNFIELD_HISTORY_NAME, + .size = sizeof(rte_mbuf_history_t), + .align = RTE_ALIGN(sizeof(rte_mbuf_history_t), 8), +}; + +/* Context structure for statistics counting and history printing */ +struct count_and_print_ctx { + uint64_t *stats; + FILE *f; +}; + +static uint64_t +mbuf_history_get(const struct rte_mbuf *m) +{ + if (rte_mbuf_history_field_offset < 0 || m == NULL) + return 0; + + rte_mbuf_history_t *history = RTE_MBUF_DYNFIELD(m, + rte_mbuf_history_field_offset, rte_mbuf_history_t *); + + return rte_atomic_load_explicit(history, rte_memory_order_acquire); +} + +static int +mbuf_history_print(FILE *f, const struct rte_mbuf *m, uint64_t history) +{ + return fprintf(f, "mbuf %p: %016" PRIx64 "\n", m, history); +} + +static void +mbuf_history_count_stats_and_print(struct rte_mempool *mp __rte_unused, + void *opaque, void *obj, unsigned obj_idx __rte_unused) +{ + struct count_and_print_ctx *ctx = (struct count_and_print_ctx *)opaque; + struct rte_mbuf *m = (struct rte_mbuf *)obj; + uint64_t history, last_op; + + if (obj == NULL || ctx == NULL || ctx->stats == NULL || ctx->f == NULL) + return; + + history = mbuf_history_get(m); + /* Extract the most recent operation */ + last_op = history & HISTORY_LAST_MASK; + RTE_ASSERT(last_op < RTE_MBUF_HISTORY_OP_MAX); + + ctx->stats[last_op]++; + ctx->stats[RTE_MBUF_HISTORY_OP_MAX]++; /* total */ + + if (history != 0) + mbuf_history_print(ctx->f, m, history); +} + +static void +mbuf_history_get_stats(struct rte_mempool *mp, void *arg) +{ + FILE *f = (FILE *)arg; + uint64_t stats[RTE_MBUF_HISTORY_OP_MAX + 1] = {0}; + struct count_and_print_ctx ctx = { + .stats = stats, + .f = f + }; + + if (f == NULL) + return; + + /* Output mempool header */ + fprintf(f, "mempool <%s>@%p\n", mp->name, mp); + + /* Single pass: collect statistics and print mbuf history */ + rte_mempool_obj_iter(mp, mbuf_history_count_stats_and_print, &ctx); + + /* Calculate total allocated mbufs */ + uint64_t total_allocated = + stats[RTE_MBUF_HISTORY_OP_LIB_ALLOC] + + stats[RTE_MBUF_HISTORY_OP_PMD_ALLOC] + + stats[RTE_MBUF_HISTORY_OP_APP_ALLOC] + + stats[RTE_MBUF_HISTORY_OP_RX] + + stats[RTE_MBUF_HISTORY_OP_TX] + + stats[RTE_MBUF_HISTORY_OP_PREP_TX] + + stats[RTE_MBUF_HISTORY_OP_BUSY_TX] + + stats[RTE_MBUF_HISTORY_OP_ENQUEUE] + + stats[RTE_MBUF_HISTORY_OP_DEQUEUE]; + + /* Print statistics summary */ + fprintf(f, " populated = %u\n", mp->populated_size); + fprintf(f, " never allocated = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_NEVER]); + fprintf(f, " lib free = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_LIB_FREE]); + fprintf(f, " PMD free = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_PMD_FREE]); + fprintf(f, " app free = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_APP_FREE]); + fprintf(f, " allocated = %" PRIu64 "\n", total_allocated); + fprintf(f, " lib alloc = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_LIB_ALLOC]); + fprintf(f, " PMD alloc = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_PMD_ALLOC]); + fprintf(f, " app alloc = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_APP_ALLOC]); + fprintf(f, " Rx = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_RX]); + fprintf(f, " Tx = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_TX]); + fprintf(f, " prep Tx = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_PREP_TX]); + fprintf(f, " busy Tx = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_BUSY_TX]); + fprintf(f, " enqueue = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_ENQUEUE]); + fprintf(f, " dequeue = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_DEQUEUE]); + fprintf(f, " user defined 1 = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_USR1]); + fprintf(f, " user defined 2 = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_USR2]); + fprintf(f, " counted total = %" PRIu64 "\n", stats[RTE_MBUF_HISTORY_OP_MAX]); +} + +#endif /* RTE_MBUF_HISTORY_DEBUG */ + +RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_mbuf_history_dump, 25.11) +void rte_mbuf_history_dump(FILE *f, const struct rte_mbuf *m) +{ +#if !RTE_MBUF_HISTORY_DEBUG + RTE_SET_USED(f); + RTE_SET_USED(m); + MBUF_LOG(INFO, "mbuf history recorder is not enabled"); +#else + if (f == NULL) { + MBUF_LOG(ERR, "Invalid mbuf dump file."); + return; + } + if (m == NULL) { + fprintf(f, "ERROR: Invalid mbuf pointer\n"); + return; + } + if (rte_mbuf_history_field_offset < 0) { + fprintf(f, "WARNING: mbuf history not initialized. Call rte_mbuf_history_init() first.\n"); + return; + } + mbuf_history_print(f, m, mbuf_history_get(m)); +#endif +} + +RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_mbuf_history_dump_mempool, 25.11) +void rte_mbuf_history_dump_mempool(FILE *f, struct rte_mempool *mp) +{ +#if !RTE_MBUF_HISTORY_DEBUG + RTE_SET_USED(f); + RTE_SET_USED(mp); + MBUF_LOG(INFO, "mbuf history recorder is not enabled"); +#else + if (f == NULL) { + MBUF_LOG(ERR, "Invalid mbuf dump file."); + return; + } + if (mp == NULL) { + fprintf(f, "ERROR: Invalid mempool pointer\n"); + return; + } + if (rte_mbuf_history_field_offset < 0) { + fprintf(f, "WARNING: mbuf history not initialized. Call rte_mbuf_history_init() first.\n"); + return; + } + mbuf_history_get_stats(mp, f); +#endif +} + +RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_mbuf_history_dump_all, 25.11) +void rte_mbuf_history_dump_all(FILE *f) +{ +#if !RTE_MBUF_HISTORY_DEBUG + RTE_SET_USED(f); + MBUF_LOG(INFO, "mbuf history recorder is not enabled"); +#else + if (f == NULL) { + MBUF_LOG(ERR, "Invalid mbuf dump file."); + return; + } + fprintf(f, "mbuf history statistics:\n"); + if (rte_mbuf_history_field_offset < 0) { + fprintf(f, "WARNING: mbuf history not initialized. Call rte_mbuf_history_init() first.\n\n"); + return; + } + rte_mempool_walk(mbuf_history_get_stats, f); +#endif +} + +RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_mbuf_history_init, 25.11) +void rte_mbuf_history_init(void) +{ +#if RTE_MBUF_HISTORY_DEBUG + if (rte_mbuf_history_field_offset >= 0) { + /* already initialized */ + return; + } + + rte_mbuf_history_field_offset = rte_mbuf_dynfield_register(&mbuf_dynfield_history); + if (rte_mbuf_history_field_offset < 0) { + MBUF_LOG(ERR, "Failed to register mbuf history dynamic field: %s\n", + rte_strerror(rte_errno)); + } +#endif +} diff --git a/lib/mbuf/meson.build b/lib/mbuf/meson.build index 0435c5e628..a394db2b19 100644 --- a/lib/mbuf/meson.build +++ b/lib/mbuf/meson.build @@ -2,6 +2,7 @@ # Copyright(c) 2017 Intel Corporation sources = files( + 'mbuf_history.c', 'rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c', @@ -13,5 +14,6 @@ headers = files( 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h', 'rte_mbuf_dyn.h', + 'rte_mbuf_history.h', ) deps += ['mempool'] diff --git a/lib/mbuf/rte_mbuf.c b/lib/mbuf/rte_mbuf.c index 22d7cfc475..d96c2196a3 100644 --- a/lib/mbuf/rte_mbuf.c +++ b/lib/mbuf/rte_mbuf.c @@ -41,6 +41,8 @@ rte_pktmbuf_pool_init(struct rte_mempool *mp, void *opaque_arg) sizeof(struct rte_pktmbuf_pool_private)); RTE_ASSERT(mp->elt_size >= sizeof(struct rte_mbuf)); + rte_mbuf_history_init(); + /* if no structure is provided, assume no mbuf private area */ user_mbp_priv = opaque_arg; if (user_mbp_priv == NULL) { @@ -515,8 +517,10 @@ void rte_pktmbuf_free_bulk(struct rte_mbuf **mbufs, unsigned int count) } while (m != NULL); } - if (nb_pending > 0) + if (nb_pending > 0) { + rte_mbuf_history_mark_bulk(pending, nb_pending, RTE_MBUF_HISTORY_OP_LIB_FREE); rte_mempool_put_bulk(pending[0]->pool, (void **)pending, nb_pending); + } } /* Creates a shallow copy of mbuf */ diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h index 06ab7502a5..a6c6b21253 100644 --- a/lib/mbuf/rte_mbuf.h +++ b/lib/mbuf/rte_mbuf.h @@ -40,6 +40,7 @@ #include #include #include +#include #ifdef __cplusplus extern "C" { @@ -607,6 +608,7 @@ static inline struct rte_mbuf *rte_mbuf_raw_alloc(struct rte_mempool *mp) if (rte_mempool_get(mp, &ret.ptr) < 0) return NULL; __rte_mbuf_raw_sanity_check(ret.m); + rte_mbuf_history_mark(ret.m, RTE_MBUF_HISTORY_OP_LIB_ALLOC); return ret.m; } @@ -642,9 +644,12 @@ static __rte_always_inline int rte_mbuf_raw_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf **mbufs, unsigned int count) { int rc = rte_mempool_get_bulk(mp, (void **)mbufs, count); - if (likely(rc == 0)) - for (unsigned int idx = 0; idx < count; idx++) + if (likely(rc == 0)) { + for (unsigned int idx = 0; idx < count; idx++) { __rte_mbuf_raw_sanity_check(mbufs[idx]); + rte_mbuf_history_mark(mbufs[idx], RTE_MBUF_HISTORY_OP_LIB_ALLOC); + } + } return rc; } @@ -667,6 +672,7 @@ rte_mbuf_raw_free(struct rte_mbuf *m) { __rte_mbuf_raw_sanity_check(m); rte_mempool_put(m->pool, m); + rte_mbuf_history_mark(m, RTE_MBUF_HISTORY_OP_LIB_FREE); } /** @@ -701,6 +707,7 @@ rte_mbuf_raw_free_bulk(struct rte_mempool *mp, struct rte_mbuf **mbufs, unsigned RTE_ASSERT(m != NULL); RTE_ASSERT(m->pool == mp); __rte_mbuf_raw_sanity_check(m); + rte_mbuf_history_mark(mbufs[idx], RTE_MBUF_HISTORY_OP_LIB_FREE); } rte_mempool_put_bulk(mp, (void **)mbufs, count); @@ -1013,6 +1020,8 @@ static inline int rte_pktmbuf_alloc_bulk(struct rte_mempool *pool, if (unlikely(rc)) return rc; + rte_mbuf_history_mark_bulk(mbufs, count, RTE_MBUF_HISTORY_OP_LIB_ALLOC); + /* To understand duff's device on loop unwinding optimization, see * https://en.wikipedia.org/wiki/Duff's_device. * Here while() loop is used rather than do() while{} to avoid extra diff --git a/lib/mbuf/rte_mbuf_dyn.h b/lib/mbuf/rte_mbuf_dyn.h index 865c90f579..dc1d4f146a 100644 --- a/lib/mbuf/rte_mbuf_dyn.h +++ b/lib/mbuf/rte_mbuf_dyn.h @@ -69,6 +69,7 @@ #include #include +#include #ifdef __cplusplus extern "C" { @@ -240,6 +241,23 @@ void rte_mbuf_dyn_dump(FILE *out); * and parameters together. */ +/** + * The mbuf history dynamic field provides lifecycle tracking + * for mbuf objects through the system. + * It records a fixed set of predefined operations for debugging. + */ +#define RTE_MBUF_DYNFIELD_HISTORY_NAME "rte_mbuf_dynfield_history" + +/** + * Type for mbuf history dynamic field. + * + * Use atomic operations to support cloned mbufs + * accessed simultaneously by multiple lcores. + * + * The size is 64-bit for better performance on modern systems. + */ +typedef RTE_ATOMIC(uint64_t) rte_mbuf_history_t; + /* * The metadata dynamic field provides some extra packet information * to interact with RTE Flow engine. The metadata in sent mbufs can be diff --git a/lib/mbuf/rte_mbuf_history.h b/lib/mbuf/rte_mbuf_history.h new file mode 100644 index 0000000000..71019cf914 --- /dev/null +++ b/lib/mbuf/rte_mbuf_history.h @@ -0,0 +1,174 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 NVIDIA Corporation & Affiliates + */ + +#ifndef _RTE_MBUF_HISTORY_H_ +#define _RTE_MBUF_HISTORY_H_ + +/** + * @file + * + * These functions allow to track history of mbuf objects using a dynamic field. + * + * It tracks the lifecycle of mbuf objects through the system + * with a fixed set of predefined events to maintain performance. + * + * The history is stored as an atomic value (64-bit) in a dynamic field of the mbuf, + * with each event encoded in 4 bits, allowing up to 16 events to be tracked. + * Atomic operations ensure thread safety for cloned mbufs accessed by multiple lcores. + */ + +#include +#include + +#include + +/* Forward declaration to avoid circular dependency. */ +struct rte_mbuf; +struct rte_mempool; + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Number of bits for each history operation. + */ +#define RTE_MBUF_HISTORY_BITS 4 + +/** + * Maximum number of history operations that can be stored. + */ +#define RTE_MBUF_HISTORY_MAX (sizeof(rte_mbuf_history_t) * 8 / RTE_MBUF_HISTORY_BITS) + +/** + * History operation types. + */ +enum rte_mbuf_history_op { + RTE_MBUF_HISTORY_OP_NEVER = 0, /**< Initial state - never allocated */ + RTE_MBUF_HISTORY_OP_LIB_FREE = 1, /**< Freed back to pool */ + RTE_MBUF_HISTORY_OP_PMD_FREE = 2, /**< Freed by PMD */ + RTE_MBUF_HISTORY_OP_APP_FREE = 3, /**< Freed by application */ + RTE_MBUF_HISTORY_OP_LIB_ALLOC = 4, /**< Allocation in mbuf library */ + RTE_MBUF_HISTORY_OP_PMD_ALLOC = 5, /**< Allocated by PMD for Rx */ + RTE_MBUF_HISTORY_OP_APP_ALLOC = 6, /**< Allocated by application */ + RTE_MBUF_HISTORY_OP_RX = 7, /**< Received */ + RTE_MBUF_HISTORY_OP_TX = 8, /**< Sent */ + RTE_MBUF_HISTORY_OP_PREP_TX = 9, /**< Being prepared before Tx */ + RTE_MBUF_HISTORY_OP_BUSY_TX = 10, /**< Returned due to Tx busy */ + RTE_MBUF_HISTORY_OP_ENQUEUE = 11, /**< Enqueued for processing */ + RTE_MBUF_HISTORY_OP_DEQUEUE = 12, /**< Dequeued for processing */ + /* 13 - reserved for future */ + RTE_MBUF_HISTORY_OP_USR2 = 14, /**< Application-defined event 2 */ + RTE_MBUF_HISTORY_OP_USR1 = 15, /**< Application-defined event 1 */ + RTE_MBUF_HISTORY_OP_MAX = 16, /**< Maximum number of operation types */ +}; + +/** + * Global offset for the history dynamic field (set during initialization). + */ +extern int rte_mbuf_history_field_offset; + +/** + * Initialize the mbuf history system. + * + * This function registers the dynamic field for mbuf history tracking. + * It should be called once during application initialization. + * + * Note: This function is called by rte_pktmbuf_pool_create, + * so explicit invocation is usually not required. + */ +__rte_experimental +void rte_mbuf_history_init(void); + +/** + * Mark an mbuf with a history event. + * + * @param m + * Pointer to the mbuf. + * @param op + * The operation to record. + */ +static inline void rte_mbuf_history_mark(struct rte_mbuf *m, uint32_t op) +{ +#if !RTE_MBUF_HISTORY_DEBUG + RTE_SET_USED(m); + RTE_SET_USED(op); +#else + RTE_ASSERT(rte_mbuf_history_field_offset >= 0); + RTE_ASSERT(op < RTE_MBUF_HISTORY_OP_MAX); + if (unlikely (m == NULL)) + return; + + rte_mbuf_history_t *history_field = RTE_MBUF_DYNFIELD(m, + rte_mbuf_history_field_offset, rte_mbuf_history_t *); + uint64_t history = rte_atomic_load_explicit(history_field, rte_memory_order_acquire); + history = (history << RTE_MBUF_HISTORY_BITS) | op; + rte_atomic_store_explicit(history_field, history, rte_memory_order_release); +#endif +} + +/** + * Mark multiple mbufs with a history event. + * + * @param mbufs + * Array of mbuf pointers. + * @param n + * Number of mbufs to mark. + * @param op + * The operation to record. + */ +static inline void rte_mbuf_history_mark_bulk(struct rte_mbuf * const *mbufs, + uint32_t n, uint32_t op) +{ +#if !RTE_MBUF_HISTORY_DEBUG + RTE_SET_USED(mbufs); + RTE_SET_USED(n); + RTE_SET_USED(op); +#else + RTE_ASSERT(rte_mbuf_history_field_offset >= 0); + RTE_ASSERT(op < RTE_MBUF_HISTORY_OP_MAX); + if (unlikely (mbufs == NULL)) + return; + + while (n--) + rte_mbuf_history_mark(*mbufs++, op); +#endif +} + +/** + * Dump mbuf history for a single mbuf to a file. + * + * @param f + * File pointer to write the history to. + * @param m + * Pointer to the mbuf to dump history for. + */ +__rte_experimental +void rte_mbuf_history_dump(FILE *f, const struct rte_mbuf *m); + +/** + * Dump mbuf history statistics for a single mempool to a file. + * + * @param f + * File pointer to write the history statistics to. + * @param mp + * Pointer to the mempool to dump history for. + */ +__rte_experimental +void rte_mbuf_history_dump_mempool(FILE *f, struct rte_mempool *mp); + +/** + * Dump mbuf history statistics for all mempools to a file. + * + * @param f + * File pointer to write the history statistics to. + */ +__rte_experimental +void rte_mbuf_history_dump_all(FILE *f); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_MBUF_HISTORY_H_ */ diff --git a/meson_options.txt b/meson_options.txt index e28d24054c..5543a07583 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -26,6 +26,8 @@ option('enable_driver_sdk', type: 'boolean', value: false, description: 'Install headers to build drivers.') option('enable_libs', type: 'string', value: '', description: 'Comma-separated list of optional libraries to explicitly enable. [NOTE: mandatory libs are always enabled]') +option('enable_mbuf_history', type: 'boolean', value: false, description: + 'Track mbufs for debugging purpose. Default is false.') option('examples', type: 'string', value: '', description: 'Comma-separated list of examples to build by default') option('ibverbs_link', type: 'combo', choices : ['static', 'shared', 'dlopen'], value: 'shared', description: From patchwork Tue Sep 30 23:25:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Monjalon X-Patchwork-Id: 157153 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A0B8548877; Wed, 1 Oct 2025 01:39:09 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 17EE840689; Wed, 1 Oct 2025 01:39:01 +0200 (CEST) Received: from fhigh-b1-smtp.messagingengine.com (fhigh-b1-smtp.messagingengine.com [202.12.124.152]) by mails.dpdk.org (Postfix) with ESMTP id 4AB184065D for ; Wed, 1 Oct 2025 01:38:59 +0200 (CEST) Received: from phl-compute-01.internal (phl-compute-01.internal [10.202.2.41]) by mailfhigh.stl.internal (Postfix) with ESMTP id 847037A0149; Tue, 30 Sep 2025 19:38:58 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-01.internal (MEProxy); Tue, 30 Sep 2025 19:38:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm2; t=1759275538; x= 1759361938; bh=dnf+64shqECrfkw/Ad2frRjna936OVUnAZ1Y+KKlJi4=; b=Q zyt3SJdpAAh5vw4vDKde2B61KK5XquK+4Qa21dT+lbnH+w+9qB8S5anG7dD+nBrQ w2yNljnrE2XYgdo4F1EgCoFkv2cSVGPZadY2DkUAOX2uq5iR1//xrfEGnpDJvPQe dcvggYomBaEuB3XPO7YQU2sH+qrChEQrRgCQ7RULq34oYMwQC8I3IwZR1NENZgws qDeQhrW8ET+c1YT/ZsAEg1d2yAtprRr90QUred4lunIFzfDylY+F1rua2XfinRrQ RvTjuMxkYmO3sYyudqT1Sw7t0nGPGVNqzokPD92oDj26U6EupZkAKWW6qxvdBtnf ttim/LnMvFMlFBk9Hno1A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1759275538; x=1759361938; bh=d nf+64shqECrfkw/Ad2frRjna936OVUnAZ1Y+KKlJi4=; b=vkT4vDaqMv7xmXMCe NQwoOumkq/raGnVSlKxq5BcI059EJaAofFd/ruYlED+Si+C0y7vO0dIlu/FvuG22 11l/opN6kD5bUCOnzKRXoy4CVdpN3RLYm3W/A8a4wD/LnZBo/dW/mosScHyft5IW nU8hH0cLDTNB8XsqhhC5/5FXSEJvmaJ71dWIBNsg5yV8Qh5Whu5kjIO53yYzXqej fbZX2DETIiDNUvhaK+5st++aQyFvMXGt1NbYsrQCnb3wE+zJQqFqXXFfQxOclABd 4iZEY3RTUB26dj0V8N5kYkd8+851dDZ+j8tAgoTcDslj4E4C0uhw5cmSF2hkgqWu HE/Qw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdekudejiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefvhhhomhgrshcu ofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecuggftrf grthhtvghrnhepvdejhfdugeehvddtieejieegteeuudfgjeeukeeiledthfetveekhefh ieelhfdtnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh epthhhohhmrghssehmohhnjhgrlhhonhdrnhgvthdpnhgspghrtghpthhtohepjedpmhho uggvpehsmhhtphhouhhtpdhrtghpthhtohepuggvvhesughpughkrdhorhhgpdhrtghpth htohepshhhphgvrhgvthiisehnvhhiughirgdrtghomhdprhgtphhtthhopehvihgrtghh vghslhgrvhhosehnvhhiughirgdrtghomhdprhgtphhtthhopegsrhhutggvrdhrihgthh grrhgushhonhesihhnthgvlhdrtghomhdprhgtphhtthhopehmsgesshhmrghrthhshhgr rhgvshihshhtvghmshdrtghomhdprhgtphhtthhopehsthgvphhhvghnsehnvghtfihorh hkphhluhhmsggvrhdrohhrghdprhgtphhtthhopegrnhgurhgvfidrrhihsggthhgvnhhk ohesohhkthgvthhlrggsshdrrhhu X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 30 Sep 2025 19:38:56 -0400 (EDT) From: Thomas Monjalon To: dev@dpdk.org Cc: shperetz@nvidia.com, viacheslavo@nvidia.com, bruce.richardson@intel.com, mb@smartsharesystems.com, stephen@networkplumber.org, Andrew Rybchenko Subject: [PATCH v3 3/5] ethdev: mark mbufs in burst functions Date: Wed, 1 Oct 2025 01:25:04 +0200 Message-ID: <20250930233828.3999565-4-thomas@monjalon.net> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250930233828.3999565-1-thomas@monjalon.net> References: <20250616072910.113042-1-shperetz@nvidia.com> <20250930233828.3999565-1-thomas@monjalon.net> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Shani Peretz Mark the Rx/Tx steps in mbufs for debugging. It has no performance impact if mbuf history is disabled (by default). Signed-off-by: Shani Peretz --- lib/ethdev/rte_ethdev.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index d23c143eed..8a9683c5e9 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -6336,6 +6336,8 @@ rte_eth_rx_burst(uint16_t port_id, uint16_t queue_id, nb_rx = p->rx_pkt_burst(qd, rx_pkts, nb_pkts); + rte_mbuf_history_mark_bulk(rx_pkts, nb_rx, RTE_MBUF_HISTORY_OP_RX); + #ifdef RTE_ETHDEV_RXTX_CALLBACKS { void *cb; @@ -6688,8 +6690,19 @@ rte_eth_tx_burst(uint16_t port_id, uint16_t queue_id, } #endif +#if RTE_MBUF_HISTORY_DEBUG + uint16_t requested_pkts = nb_pkts; + rte_mbuf_history_mark_bulk(tx_pkts, nb_pkts, RTE_MBUF_HISTORY_OP_TX); +#endif + nb_pkts = p->tx_pkt_burst(qd, tx_pkts, nb_pkts); +#if RTE_MBUF_HISTORY_DEBUG + if (requested_pkts > nb_pkts) + rte_mbuf_history_mark_bulk(tx_pkts + nb_pkts, + requested_pkts - nb_pkts, RTE_MBUF_HISTORY_OP_BUSY_TX); +#endif + rte_ethdev_trace_tx_burst(port_id, queue_id, (void **)tx_pkts, nb_pkts); return nb_pkts; } @@ -6785,6 +6798,8 @@ rte_eth_tx_prepare(uint16_t port_id, uint16_t queue_id, } #endif + rte_mbuf_history_mark_bulk(tx_pkts, nb_pkts, RTE_MBUF_HISTORY_OP_PREP_TX); + return p->tx_pkt_prepare(qd, tx_pkts, nb_pkts); } From patchwork Tue Sep 30 23:25:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Monjalon X-Patchwork-Id: 157154 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2A30148877; Wed, 1 Oct 2025 01:39:15 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A887E40696; Wed, 1 Oct 2025 01:39:03 +0200 (CEST) Received: from fhigh-b1-smtp.messagingengine.com (fhigh-b1-smtp.messagingengine.com [202.12.124.152]) by mails.dpdk.org (Postfix) with ESMTP id 04462406BA for ; Wed, 1 Oct 2025 01:39:01 +0200 (CEST) Received: from phl-compute-11.internal (phl-compute-11.internal [10.202.2.51]) by mailfhigh.stl.internal (Postfix) with ESMTP id 536C27A0159; Tue, 30 Sep 2025 19:39:01 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-11.internal (MEProxy); Tue, 30 Sep 2025 19:39:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm2; t=1759275541; x= 1759361941; bh=P9DBDynpFVW5yQ8VWLAvw1yFkmx4Dpna0vf9DDkz+3w=; b=H hJ1qfYP3tB8bjz/IHtKMy6cE3SvgabclGHrLC+MJ7gAai0UIA85XhhpKVM1tWCjF fEqZEswi4RQU+qEJpYhgytI+KvlEXZsIpPYxKLyLwoRaGVIG9heoWCt+i3tJmfU5 wWpbN3Q6ALjzYhA+sTrmki2n4/A2jSat0Qk4QXvgPrKT5kOGerMh5D/ukjVbQpKe U8MqDugWRhaQIG0OkqEG0REs+MeHZTBdVwZOhzxZ0NZFXk6Ji7w3ALrcAJj+ighJ lRNlhwkJryvfKDekRCKNEXCFFWKd2Sqg+uXaqTYMCwjaM0O4Z5AC4ry1jrKTmf0F Synwev+tyVIIxFEGKxN2Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1759275541; x=1759361941; bh=P 9DBDynpFVW5yQ8VWLAvw1yFkmx4Dpna0vf9DDkz+3w=; b=Cxy0MwNfDhx06hAsz uClQwvFNhp+KeZ/xFFphpgAAronIddASXc5iW2QBFQVxEpx8CNNCvNpD+g5uPTVf IVHUcPFm6QHkR/DhaHYPxxNJzvKOuyDv510yWS5eDxQLrpL2h2Rw4z4kAJ2gCcoS Kvhjisc7ujr5c/TIol5mMLYBjJ6iJmIprQMadTxwxcHCH59+PKwe3DMyGBUwZH6Q fhyETaxTEplBI0TlHv09xWEopm8WjIL+QIU+49Q6sMiJ6rlyYiv6sXYYkrh/pw5w 7l/kNiVXQA2nc4RKzwC7RxuEaRmJVfXbWQ4zS4PA1TI5PJSARSVSGiwxo8ycVmCD jQAVA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdekudejjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefvhhhomhgrshcu ofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecuggftrf grthhtvghrnhepvdejhfdugeehvddtieejieegteeuudfgjeeukeeiledthfetveekhefh ieelhfdtnecuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomh epthhhohhmrghssehmohhnjhgrlhhonhdrnhgvthdpnhgspghrtghpthhtohepjedpmhho uggvpehsmhhtphhouhhtpdhrtghpthhtohepuggvvhesughpughkrdhorhhgpdhrtghpth htohepshhhphgvrhgvthiisehnvhhiughirgdrtghomhdprhgtphhtthhopehvihgrtghh vghslhgrvhhosehnvhhiughirgdrtghomhdprhgtphhtthhopegsrhhutggvrdhrihgthh grrhgushhonhesihhnthgvlhdrtghomhdprhgtphhtthhopehmsgesshhmrghrthhshhgr rhgvshihshhtvghmshdrtghomhdprhgtphhtthhopehsthgvphhhvghnsehnvghtfihorh hkphhluhhmsggvrhdrohhrghdprhgtphhtthhopegrmhgrnhdruggvvghprdhsihhnghhh sehinhhtvghlrdgtohhm X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 30 Sep 2025 19:38:59 -0400 (EDT) From: Thomas Monjalon To: dev@dpdk.org Cc: shperetz@nvidia.com, viacheslavo@nvidia.com, bruce.richardson@intel.com, mb@smartsharesystems.com, stephen@networkplumber.org, Aman Singh Subject: [PATCH v3 4/5] app/testpmd: add commands to dump mbuf history Date: Wed, 1 Oct 2025 01:25:05 +0200 Message-ID: <20250930233828.3999565-5-thomas@monjalon.net> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250930233828.3999565-1-thomas@monjalon.net> References: <20250616072910.113042-1-shperetz@nvidia.com> <20250930233828.3999565-1-thomas@monjalon.net> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Shani Peretz Dump the mbuf history to the console or to a file. The dump will contain: - Operation history for each mbuf - Summary and statistics about all mbufs Dump the history of all mbuf pools: testpmd> dump_mbuf_history_all Dump the history of one mbuf pool: testpmd> dump_mbuf_history_single_mempool [file_name] Dump the history of one mbuf: testpmd> dump_mbuf_history_single Signed-off-by: Shani Peretz --- app/test-pmd/cmdline.c | 246 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 245 insertions(+), 1 deletion(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 3731fba370..113db68f8c 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -40,6 +40,7 @@ #include #endif #include +#include #include #include @@ -296,6 +297,15 @@ static void cmd_help_long_parsed(void *parsed_result, "dump_log_types\n" " Dumps the log level for all the dpdk modules\n\n" + "dump_mbuf_history_all\n" + " Dumps the mbuf history for all mempools\n\n" + + "dump_mbuf_history_single \n" + " Dumps the history for a specific mbuf (use 0x
format)\n\n" + + "dump_mbuf_history_single_mempool [file_name]\n" + " Dumps the history for a specific mempool\n\n" + "show port (port_id) speed_lanes capabilities" " Show speed lanes capabilities of a port.\n\n" ); @@ -9177,6 +9187,8 @@ static void cmd_dump_parsed(void *parsed_result, #endif else if (!strcmp(res->dump, "dump_log_types")) rte_log_dump(stdout); + else if (!strcmp(res->dump, "dump_mbuf_history_all")) + rte_mbuf_history_dump_all(stdout); } static cmdline_parse_token_string_t cmd_dump_dump = @@ -9198,7 +9210,8 @@ cmd_dump_init(void) #ifndef RTE_EXEC_ENV_WINDOWS "dump_trace#" #endif - "dump_log_types"; + "dump_log_types#" + "dump_mbuf_history"; } static cmdline_parse_inst_t cmd_dump = { @@ -9260,6 +9273,232 @@ static cmdline_parse_inst_t cmd_dump_one = { }, }; +/* Dump mbuf history to file */ +struct cmd_dump_mbuf_to_file_result { + cmdline_fixed_string_t dump; + cmdline_fixed_string_t file; +}; + +static void cmd_dump_mbuf_to_file_parsed(void *parsed_result, + struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_dump_mbuf_to_file_result *res = parsed_result; + FILE *file = stdout; + char *file_name = res->file; + + if (strcmp(res->dump, "dump_mbuf_history_all")) { + cmdline_printf(cl, "Invalid dump type\n"); + return; + } + + if (file_name && strlen(file_name)) { + file = fopen(file_name, "w"); + if (!file) { + rte_mbuf_history_dump_all(stdout); + return; + } + } + rte_mbuf_history_dump_all(file); + printf("Flow dump finished\n"); + if (file_name && strlen(file_name)) + fclose(file); +} + +static cmdline_parse_token_string_t cmd_dump_mbuf_to_file_dump = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_to_file_result, dump, + "dump_mbuf_history_all"); + +static cmdline_parse_token_string_t cmd_dump_mbuf_to_file_file = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_to_file_result, file, NULL); + +static cmdline_parse_inst_t cmd_dump_mbuf_to_file = { + .f = cmd_dump_mbuf_to_file_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "dump_mbuf_history_all : Dump mbuf history to file", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dump_mbuf_to_file_dump, + (void *)&cmd_dump_mbuf_to_file_file, + NULL, + }, +}; + +/* Dump single mbuf history */ +struct cmd_dump_mbuf_history_single_result { + cmdline_fixed_string_t dump; + cmdline_fixed_string_t mbuf_ptr; +}; + +static void cmd_dump_mbuf_history_single_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_dump_mbuf_history_single_result *res = parsed_result; + struct rte_mbuf *mbuf; + uintptr_t mbuf_addr; + + /* Parse the mbuf pointer from hex string */ + if (sscanf(res->mbuf_ptr, "0x%" SCNxPTR, &mbuf_addr) != 1) { + printf("Invalid mbuf pointer format. Use 0x
\n"); + return; + } + + mbuf = (struct rte_mbuf *)mbuf_addr; + + printf("Dumping history for mbuf at %p:\n", mbuf); + rte_mbuf_history_dump(stdout, mbuf); +} + +static cmdline_parse_token_string_t cmd_dump_mbuf_history_single_dump = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_history_single_result, dump, + "dump_mbuf_history_single"); + +static cmdline_parse_token_string_t cmd_dump_mbuf_history_single_ptr = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_history_single_result, mbuf_ptr, + NULL); + +static cmdline_parse_inst_t cmd_dump_mbuf_history_single = { + .f = cmd_dump_mbuf_history_single_parsed, + .data = NULL, + .help_str = "dump_mbuf_history_single : Dump history for specific mbuf", + .tokens = { + (void *)&cmd_dump_mbuf_history_single_dump, + (void *)&cmd_dump_mbuf_history_single_ptr, + NULL, + }, +}; + +/* Dump single mempool history */ +struct cmd_dump_mbuf_history_single_mempool_result { + cmdline_fixed_string_t dump; + cmdline_fixed_string_t mempool_name; +}; + +static void cmd_dump_mbuf_history_single_mempool_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_dump_mbuf_history_single_mempool_result *res = parsed_result; + struct rte_mempool *mp; + + mp = rte_mempool_lookup(res->mempool_name); + if (mp == NULL) { + printf("Mempool '%s' not found\n", res->mempool_name); + return; + } + + printf("Dumping history for mempool '%s':\n", res->mempool_name); + + rte_mbuf_history_dump_mempool(stdout, mp); +} + +static cmdline_parse_token_string_t cmd_dump_mbuf_history_single_mempool_dump = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_history_single_mempool_result, + dump, "dump_mbuf_history_single_mempool"); + +static cmdline_parse_token_string_t cmd_dump_mbuf_history_single_mempool_name = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_history_single_mempool_result, + mempool_name, NULL); + +static cmdline_parse_inst_t cmd_dump_mbuf_history_single_mempool = { + .f = cmd_dump_mbuf_history_single_mempool_parsed, + .data = NULL, + .help_str = "dump_mbuf_history_single_mempool : Dump history for specific mempool", + .tokens = { + (void *)&cmd_dump_mbuf_history_single_mempool_dump, + (void *)&cmd_dump_mbuf_history_single_mempool_name, + NULL, + }, +}; + +/* Dump single mempool history to file only */ +struct cmd_dump_mbuf_history_single_mempool_to_file_result { + cmdline_fixed_string_t dump; + cmdline_fixed_string_t mempool_name; + cmdline_fixed_string_t file; +}; + +static void cmd_dump_mbuf_history_single_mempool_to_file_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_dump_mbuf_history_single_mempool_to_file_result *res = parsed_result; + struct rte_mempool *mp; + FILE *file = NULL; + char *file_name = res->file; + + mp = rte_mempool_lookup(res->mempool_name); + if (mp == NULL) { + printf("Mempool '%s' not found\n", res->mempool_name); + return; + } + + if (file_name && strlen(file_name)) { + file = fopen(file_name, "w"); + if (!file) { + printf("Failed to open file '%s'\n", file_name); + return; + } + } else { + printf("File name required for this command\n"); + return; + } + + printf("Dumping history for mempool '%s' to file '%s'...\n", res->mempool_name, file_name); + rte_mbuf_history_dump_mempool(file, mp); + printf("Mempool history dump finished\n"); + fclose(file); +} + +static cmdline_parse_token_string_t cmd_dump_mbuf_history_single_mempool_to_file_dump = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_history_single_mempool_to_file_result, + dump, "dump_mbuf_history_single_mempool"); + +static cmdline_parse_token_string_t cmd_dump_mbuf_history_single_mempool_to_file_name = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_history_single_mempool_to_file_result, + mempool_name, NULL); + +static cmdline_parse_token_string_t cmd_dump_mbuf_history_single_mempool_to_file_file = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_history_single_mempool_to_file_result, + file, NULL); + +static cmdline_parse_inst_t cmd_dump_mbuf_history_single_mempool_to_file = { + .f = cmd_dump_mbuf_history_single_mempool_to_file_parsed, + .data = NULL, + .help_str = "dump_mbuf_history_single_mempool : Dump history for specific mempool to file", + .tokens = { + (void *)&cmd_dump_mbuf_history_single_mempool_to_file_dump, + (void *)&cmd_dump_mbuf_history_single_mempool_to_file_name, + (void *)&cmd_dump_mbuf_history_single_mempool_to_file_file, + NULL, + }, +}; + +/* Dump all mempools history */ +struct cmd_dump_mbuf_history_all_result { + cmdline_fixed_string_t dump; +}; + +static void cmd_dump_mbuf_history_all_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + __rte_unused struct cmd_dump_mempool_history_all_result *res = parsed_result; + + printf("Dumping history for all mempools:\n"); + rte_mbuf_history_dump_all(stdout); +} + +static cmdline_parse_token_string_t cmd_dump_mbuf_history_all_dump = + TOKEN_STRING_INITIALIZER(struct cmd_dump_mbuf_history_all_result, dump, + "dump_mbuf_history_all"); + +static cmdline_parse_inst_t cmd_dump_mbuf_history_all = { + .f = cmd_dump_mbuf_history_all_parsed, + .data = NULL, + .help_str = "dump_mbuf_history_all: Dump history for all mempools", + .tokens = { + (void *)&cmd_dump_mbuf_history_all_dump, + NULL, + }, +}; + + /* *** Filters Control *** */ #define IPV4_ADDR_TO_UINT(ip_addr, ip) \ @@ -13999,6 +14238,11 @@ static cmdline_parse_ctx_t builtin_ctx[] = { &cmd_cleanup_txq_mbufs, &cmd_dump, &cmd_dump_one, + &cmd_dump_mbuf_to_file, + &cmd_dump_mbuf_history_single, + &cmd_dump_mbuf_history_single_mempool, + &cmd_dump_mbuf_history_single_mempool_to_file, + &cmd_dump_mbuf_history_all, &cmd_flow, &cmd_show_port_meter_cap, &cmd_add_port_meter_profile_srtcm, From patchwork Tue Sep 30 23:25:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Monjalon X-Patchwork-Id: 157155 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 70AF548877; Wed, 1 Oct 2025 01:39:20 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A830940A6C; Wed, 1 Oct 2025 01:39:05 +0200 (CEST) Received: from fhigh-b1-smtp.messagingengine.com (fhigh-b1-smtp.messagingengine.com [202.12.124.152]) by mails.dpdk.org (Postfix) with ESMTP id B2C0140A67 for ; Wed, 1 Oct 2025 01:39:04 +0200 (CEST) Received: from phl-compute-02.internal (phl-compute-02.internal [10.202.2.42]) by mailfhigh.stl.internal (Postfix) with ESMTP id 122817A0078; Tue, 30 Sep 2025 19:39:04 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-02.internal (MEProxy); Tue, 30 Sep 2025 19:39:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm2; t=1759275543; x= 1759361943; bh=Zp5sIjYM0WyW1GHhPsU5SZlnTs32ONHP5Pa3dCw6VhE=; b=h n+XBq+KjvTlT3zvwiBznSqAyLUYXBRHZSRSJnY4i4iVsIXzo4XEPXbt47g53sOIB hOZGg06RU3IapuSKeuWG9Co43sLAeKH28Ymyu2GmtDmeDadOrH0taPPiDAcfJ/Wk /q0JvtFjEnAwM40HBK7AKhC04kBBCl9KpXx8MWheYUaug1PgYhU4Gz3msp39QEf9 PsI8zXbZiwRnnAJ8pyDMoQ/HzPb228bT2Nw8rKVlt0CU7sDYO1MMsl5qdfKfQqTC jYw2oLlF4dESxm07mHYUjsqPsKdcoJqqDJnO5cgofXpzpQY0mUykAHralSRfVdab 279sm0plkgqva9pVQp+FA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1759275543; x=1759361943; bh=Z p5sIjYM0WyW1GHhPsU5SZlnTs32ONHP5Pa3dCw6VhE=; b=iHYrQeM3piY9Z3kWO jfEBVxGrOwB7B4Ld0Z5GlLTX1wCJnw1aEUOxqs4CIX5QxCbqHL7aGH78LiFZKz66 Rfom5T8WTbO1Hl1EuS1CDg9Li6Wih4Zhuh1byORb8VgEZRLiicH42MXcFq4WWA2l 2+TsMxtIFqzqK1vVGKUiYP0tHNfwNaGy6/407F/3a4EFCMVQvzG5zXJMJ4nT4YE/ +40P4e7Lu8GKzN9lSfu//ptCW6TXsaauZYqI7cOZPuVw142z5YQsJK0OdSQbAIH6 MWbnrEq8Pwe1XOSCeC7/H3b3+FJtcAj8tUv/SYUC2R+be7v1hBTHGLyg6cKBL8iu Pug8g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdekudejiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefvhhhomhgrshcu ofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecuggftrf grthhtvghrnhephfeludejhfeiteekhfegteevudelkeekvefhleehuefgfeelgeehgfeg udeitefhnecuffhomhgrihhnpeguohhtrghllhdrghhrohhuphenucevlhhushhtvghruf hiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehthhhomhgrshesmhhonhhjrghl ohhnrdhnvghtpdhnsggprhgtphhtthhopeejpdhmohguvgepshhmthhpohhuthdprhgtph htthhopeguvghvseguphgukhdrohhrghdprhgtphhtthhopehshhhpvghrvghtiiesnhhv ihguihgrrdgtohhmpdhrtghpthhtohepvhhirggthhgvshhlrghvohesnhhvihguihgrrd gtohhmpdhrtghpthhtohepsghruhgtvgdrrhhitghhrghrughsohhnsehinhhtvghlrdgt ohhmpdhrtghpthhtohepmhgssehsmhgrrhhtshhhrghrvghshihsthgvmhhsrdgtohhmpd hrtghpthhtohepshhtvghphhgvnhesnhgvthifohhrkhhplhhumhgsvghrrdhorhhgpdhr tghpthhtoheprhhjrghrrhihsehrvgguhhgrthdrtghomh X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 30 Sep 2025 19:39:02 -0400 (EDT) From: Thomas Monjalon To: dev@dpdk.org Cc: shperetz@nvidia.com, viacheslavo@nvidia.com, bruce.richardson@intel.com, mb@smartsharesystems.com, stephen@networkplumber.org, Robin Jarry Subject: [PATCH v3 5/5] usertools/mbuf: parse mbuf history dump Date: Wed, 1 Oct 2025 01:25:06 +0200 Message-ID: <20250930233828.3999565-6-thomas@monjalon.net> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250930233828.3999565-1-thomas@monjalon.net> References: <20250616072910.113042-1-shperetz@nvidia.com> <20250930233828.3999565-1-thomas@monjalon.net> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Shani Peretz Add a Python script that parses the history dump of mbufs generated by rte_mbuf_history_dump() and related functions, and presents it in a human-readable format. If an operation ID is repeated, such as in the case of a double free, it will be highlighted in red and listed at the end of the file. Signed-off-by: Shani Peretz --- usertools/dpdk-mbuf_history_parser.py | 173 ++++++++++++++++++++++++++ 1 file changed, 173 insertions(+) create mode 100755 usertools/dpdk-mbuf_history_parser.py diff --git a/usertools/dpdk-mbuf_history_parser.py b/usertools/dpdk-mbuf_history_parser.py new file mode 100755 index 0000000000..dfb02d99be --- /dev/null +++ b/usertools/dpdk-mbuf_history_parser.py @@ -0,0 +1,173 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2023 NVIDIA Corporation & Affiliates + +import sys +import re +import os +import enum + +RED = "\033[91m" +RESET = "\033[0m" +ENUM_PATTERN = r'enum\s+rte_mbuf_history_op\s*{([^}]+)}' +VALUE_PATTERN = r'([A-Z_]+)\s*=\s*(\d+),\s*(?:/\*\s*(.*?)\s*\*/)?' +HEADER_FILE = os.path.join( + os.path.dirname(os.path.dirname(__file__)), + 'lib/mbuf/rte_mbuf_history.h' +) + + +def print_history_sequence(address: str, sequence: list[str]): + max_op_width = max( + len(re.sub(r'\x1b\[[0-9;]*m', '', op)) for op in sequence + ) + op_width = max_op_width + for i in range(0, len(sequence), 4): + chunk = sequence[i:i + 4] + formatted_ops = [f"{op:<{op_width}}" for op in chunk] + line = "" + for j, op in enumerate(formatted_ops): + line += op + if j < len(formatted_ops) - 1: + line += " -> " + if i + 4 < len(sequence): + line += " ->" + print(f"mbuf {address}: " + line) + print() + + +def match_field(match: re.Match) -> tuple[int, str]: + name, value, _ = match.groups() + return (int(value), name.replace('RTE_MBUF_', '')) + + +class HistoryEnum: + def __init__(self, ops: enum.Enum): + self.ops = ops + + @staticmethod + def from_header(header_file: str) -> 'HistoryEnum': + with open(header_file, 'r') as f: + content = f.read() + + # Extract each enum value and its comment + enum_content = re.search(ENUM_PATTERN, content, re.DOTALL).group(1) + fields = map(match_field, re.finditer(VALUE_PATTERN, enum_content)) + fields = dict({v: k for k, v in fields}) + return HistoryEnum(enum.Enum('HistoryOps', fields)) + + +class HistoryLine: + def __init__(self, address: str, ops: list): + self.address = address + self.ops = ops + + def repeats(self) -> [list[str], str | None]: + repeated = None + sequence = [] + for idx, op in enumerate(self.ops): + if idx > 0 and op == self.ops[idx - 1] and op.name != 'NEVER': + sequence[-1] = f"{RED}{op.name}{RESET}" + sequence.append(f"{RED}{op.name}{RESET}") + repeated = op.name + else: + sequence.append(op.name) + return sequence, repeated + + +class HistoryMetrics: + def __init__(self, metrics: dict[str, int]): + self.metrics = metrics + + def max_name_width(self) -> int: + return max(len(name) for name in self.metrics.keys()) + + +class HistoryParser: + def __init__(self): + self.history_enum = HistoryEnum.from_header(HEADER_FILE) + + def parse( + self, dump_file: str + ) -> tuple[list[HistoryLine], 'HistoryMetrics']: + with open(dump_file, 'r') as f: + lines = [line for line in f.readlines() if line.strip()] + populated = next(line for line in lines if " populated=" in line) + metrics_start = lines.index(populated) + + history_lines = lines[3:metrics_start] + metrics_lines = lines[metrics_start:-1] + return ( + self._parse_history(history_lines), + self._parse_metrics(metrics_lines) + ) + + def _parse_metrics(self, lines: list[str]) -> HistoryMetrics: + metrics = {} + for line in lines: + key, value = line.split('=', 1) + metrics[key] = int(value) + return HistoryMetrics(metrics) + + def _parse_history(self, lines: list[str]) -> list[HistoryLine]: + # Parse the format "mbuf 0x1054b9980: 0000000000000065" + history_lines = [] + for line in lines: + address = line.split(':')[0].split('mbuf ')[1] + history = line.split(':')[1] + history_lines.append( + HistoryLine( + address=address, + ops=self._parse(int(history, 16)) + ) + ) + return history_lines + + def _parse(self, history: int) -> list[str]: + ops = [] + for _ in range(16): # 64 bits / 4 bits = 16 possible operations + op = history & 0xF # Extract lowest 4 bits + if op == 0: + break + ops.append(self.history_enum.ops(op)) + history >>= 4 + + ops.reverse() + return ops + + +def print_history_lines(history_lines: list[HistoryLine]): + lines = [ + (line.address, line.repeats()) for line in history_lines + ] + + for address, (sequence, _) in lines: + print_history_sequence(address, sequence) + + print("=== Violations ===") + for address, (sequence, repeated) in lines: + if repeated: + print(f"mbuf {address} has repeated ops: {RED}{repeated}{RESET}") + + +def print_metrics(metrics: HistoryMetrics): + print("=== Metrics Summary ===") + for name, value in metrics.metrics.items(): + print(f"{name + ':':<{metrics.max_name_width() + 2}} {value}") + + +def main(): + if len(sys.argv) != 2: + print("Usage: {} ".format(sys.argv[0])) + sys.exit(1) + + history_parser = HistoryParser() + history_lines, metrics = history_parser.parse(sys.argv[1]) + + print_history_lines(history_lines) + print() + print_metrics(metrics) + + +if __name__ == "__main__": + main()