From patchwork Mon Apr 1 20:36:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Jarry X-Patchwork-Id: 139017 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4072A43DAC; Mon, 1 Apr 2024 22:37:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C0C76402A3; Mon, 1 Apr 2024 22:37:33 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 8411B4029F for ; Mon, 1 Apr 2024 22:37:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712003850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hfP/j6l5vwI7HMdoLnDlMrD4y3oU601C66vfFNLNYSI=; b=YytlYhfx5Oa9q94EkdcuqPKkKI+XzRatOhEfpe2S1WoWwdbIe4hViZm5hRasJdzhDpt+nC J3YNRKvTcx+MBoJ0CnyiVsFkg1ZESTaZTeHGmJ59N2TW44K+/Bvd1g67nfZVKMF0Es6sp5 kWBXIOcfEQkxLpAA3Xuw5q1bKE8DAdE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-141-XCti7M1jMh6a1u1sEFY5AQ-1; Mon, 01 Apr 2024 16:37:28 -0400 X-MC-Unique: XCti7M1jMh6a1u1sEFY5AQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ED9A18007A6; Mon, 1 Apr 2024 20:37:27 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.208.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id 962F62166B31; Mon, 1 Apr 2024 20:37:26 +0000 (UTC) From: Robin Jarry To: dev@dpdk.org, Jerin Jacob , Kiran Kumar K , Nithin Dabilpuram , Zhirun Yan Subject: [PATCH v2] graph: avoid accessing graph list when getting stats Date: Mon, 1 Apr 2024 22:36:49 +0200 Message-ID: <20240401203647.1909165-3-rjarry@redhat.com> In-Reply-To: <20240325155303.770468-2-rjarry@redhat.com> References: <20240325155303.770468-2-rjarry@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In rte_graph_cluster_stats_get, the walk model of the first graph is checked to determine if multi-core dispatch specific counters should be updated or not. This global list is accessed without any locks. If the global list is modified by another thread while rte_graph_cluster_stats_get is called, it can result in undefined behaviour. Adding a lock would make it impossible to call rte_graph_cluster_stats_get in packet processing code paths. Avoid accessing the global list instead by storing a bool field in the private rte_graph_cluster_stats structure. Also update the default callback to avoid accessing the global list and use a different default callback depending on the graph model. Signed-off-by: Robin Jarry Acked-by: Kiran Kumar Kokkilagadda --- Notes: v2: * (kiran) removed unnecessary loop in stats_mem_init. lib/graph/graph_stats.c | 57 ++++++++++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 21 deletions(-) diff --git a/lib/graph/graph_stats.c b/lib/graph/graph_stats.c index 2fb808b21ec5..d71451a17b95 100644 --- a/lib/graph/graph_stats.c +++ b/lib/graph/graph_stats.c @@ -34,6 +34,7 @@ struct __rte_cache_aligned rte_graph_cluster_stats { uint32_t cluster_node_size; /* Size of struct cluster_node */ rte_node_t max_nodes; int socket_id; + bool dispatch; void *cookie; size_t sz; @@ -74,17 +75,16 @@ print_banner_dispatch(FILE *f) } static inline void -print_banner(FILE *f) +print_banner(FILE *f, bool dispatch) { - if (rte_graph_worker_model_get(STAILQ_FIRST(graph_list_head_get())->graph) == - RTE_GRAPH_MODEL_MCORE_DISPATCH) + if (dispatch) print_banner_dispatch(f); else print_banner_default(f); } static inline void -print_node(FILE *f, const struct rte_graph_cluster_node_stats *stat) +print_node(FILE *f, const struct rte_graph_cluster_node_stats *stat, bool dispatch) { double objs_per_call, objs_per_sec, cycles_per_call, ts_per_hz; const uint64_t prev_calls = stat->prev_calls; @@ -104,8 +104,7 @@ print_node(FILE *f, const struct rte_graph_cluster_node_stats *stat) objs_per_sec = ts_per_hz ? (objs - prev_objs) / ts_per_hz : 0; objs_per_sec /= 1000000; - if (rte_graph_worker_model_get(STAILQ_FIRST(graph_list_head_get())->graph) == - RTE_GRAPH_MODEL_MCORE_DISPATCH) { + if (dispatch) { fprintf(f, "|%-31s|%-15" PRIu64 "|%-15" PRIu64 "|%-15" PRIu64 "|%-15" PRIu64 "|%-15" PRIu64 @@ -123,20 +122,17 @@ print_node(FILE *f, const struct rte_graph_cluster_node_stats *stat) } static int -graph_cluster_stats_cb(bool is_first, bool is_last, void *cookie, +graph_cluster_stats_cb(bool dispatch, bool is_first, bool is_last, void *cookie, const struct rte_graph_cluster_node_stats *stat) { FILE *f = cookie; - int model; - - model = rte_graph_worker_model_get(STAILQ_FIRST(graph_list_head_get())->graph); if (unlikely(is_first)) - print_banner(f); + print_banner(f, dispatch); if (stat->objs) - print_node(f, stat); + print_node(f, stat, dispatch); if (unlikely(is_last)) { - if (model == RTE_GRAPH_MODEL_MCORE_DISPATCH) + if (dispatch) boarder_model_dispatch(); else boarder(); @@ -145,6 +141,20 @@ graph_cluster_stats_cb(bool is_first, bool is_last, void *cookie, return 0; }; +static int +graph_cluster_stats_cb_rtc(bool is_first, bool is_last, void *cookie, + const struct rte_graph_cluster_node_stats *stat) +{ + return graph_cluster_stats_cb(false, is_first, is_last, cookie, stat); +}; + +static int +graph_cluster_stats_cb_dispatch(bool is_first, bool is_last, void *cookie, + const struct rte_graph_cluster_node_stats *stat) +{ + return graph_cluster_stats_cb(true, is_first, is_last, cookie, stat); +}; + static struct rte_graph_cluster_stats * stats_mem_init(struct cluster *cluster, const struct rte_graph_cluster_stats_param *prm) @@ -157,8 +167,13 @@ stats_mem_init(struct cluster *cluster, /* Fix up callback */ fn = prm->fn; - if (fn == NULL) - fn = graph_cluster_stats_cb; + if (fn == NULL) { + const struct rte_graph *graph = cluster->graphs[0]->graph; + if (graph->model == RTE_GRAPH_MODEL_MCORE_DISPATCH) + fn = graph_cluster_stats_cb_dispatch; + else + fn = graph_cluster_stats_cb_rtc; + } cluster_node_size = sizeof(struct cluster_node); /* For a given cluster, max nodes will be the max number of graphs */ @@ -350,6 +365,8 @@ rte_graph_cluster_stats_create(const struct rte_graph_cluster_stats_param *prm) if (stats_mem_populate(&stats, graph_fp, graph_node)) goto realloc_fail; } + if (graph->graph->model == RTE_GRAPH_MODEL_MCORE_DISPATCH) + stats->dispatch = true; } /* Finally copy to hugepage memory to avoid pressure on rte_realloc */ @@ -375,20 +392,18 @@ rte_graph_cluster_stats_destroy(struct rte_graph_cluster_stats *stat) } static inline void -cluster_node_arregate_stats(struct cluster_node *cluster) +cluster_node_arregate_stats(struct cluster_node *cluster, bool dispatch) { uint64_t calls = 0, cycles = 0, objs = 0, realloc_count = 0; struct rte_graph_cluster_node_stats *stat = &cluster->stat; uint64_t sched_objs = 0, sched_fail = 0; struct rte_node *node; rte_node_t count; - int model; - model = rte_graph_worker_model_get(STAILQ_FIRST(graph_list_head_get())->graph); for (count = 0; count < cluster->nb_nodes; count++) { node = cluster->nodes[count]; - if (model == RTE_GRAPH_MODEL_MCORE_DISPATCH) { + if (dispatch) { sched_objs += node->dispatch.total_sched_objs; sched_fail += node->dispatch.total_sched_fail; } @@ -403,7 +418,7 @@ cluster_node_arregate_stats(struct cluster_node *cluster) stat->objs = objs; stat->cycles = cycles; - if (model == RTE_GRAPH_MODEL_MCORE_DISPATCH) { + if (dispatch) { stat->dispatch.sched_objs = sched_objs; stat->dispatch.sched_fail = sched_fail; } @@ -433,7 +448,7 @@ rte_graph_cluster_stats_get(struct rte_graph_cluster_stats *stat, bool skip_cb) cluster = stat->clusters; for (count = 0; count < stat->max_nodes; count++) { - cluster_node_arregate_stats(cluster); + cluster_node_arregate_stats(cluster, stat->dispatch); if (!skip_cb) rc = stat->fn(!count, (count == stat->max_nodes - 1), stat->cookie, &cluster->stat);