From patchwork Thu Nov 9 17:34:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134020 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3CBEA432E9; Thu, 9 Nov 2023 18:34:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0F0B640DF5; Thu, 9 Nov 2023 18:34:27 +0100 (CET) Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by mails.dpdk.org (Postfix) with ESMTP id 94C454021E for ; Thu, 9 Nov 2023 18:34:25 +0100 (CET) Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1cc394f4cdfso9448375ad.0 for ; Thu, 09 Nov 2023 09:34:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699551264; x=1700156064; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dzO7hXbQ2jqdDptnVyq6Ah0zFTtMVblIxZyUbi0KLfM=; b=DQBzfOriKDKIWO0AqfrQr+9nDyplwn1T2CazILniOsyDdXzeDEvmMgoo2Gbzl6BRc2 cBegj4vtVLHv2PkW2Zt7txajImx4gVWIJYqi0ZdNi0jXCQCgrYRvP8w8UkNAsuCnlXW7 7xMDmieQfbzDV3urJGj4vLeXcmOlOUsfgNFydyUqyf/deBzw1l2oQ43IvBR9HKdwXQOa QeM8Qn5Qs1FwD1O3KzrR4q4x1u7YBR55PTGPuzPOyuWHnGXKdCyTqrmuccTbTOThJ30t LQmUL64w5Lnx92CPVuJMD4ucinz49n8bCSW8dgAYoRqdAZDtBCQEgioFoYF4v3RKou1G roew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699551264; x=1700156064; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dzO7hXbQ2jqdDptnVyq6Ah0zFTtMVblIxZyUbi0KLfM=; b=ZI43dyEkdDQLmTScbHJaGbdId5STrvjJ66p7fCfho7NRyU4Xf7dPidLq1jG8+pDurW l5hqltjAYXdAs8sm8T4JhAOwErqi6iQtRqfrJW5bK7rOOu4PMLnsXKScMDbnFVsCOdld Tl65ZCetEaSQtbTitaA1oylASlR5XyYddWXu5d/E/Xf5UtHB97prX1A8C9vh1/RMHSV5 WKP9v7rwxutfMPlkiRRgw8n54SARd2YlEmqJ6wGIQy7bI6xQUaU50PDzD2cKZtBTctyF rKT3j7KxktoGi7Vzjz06daSR/VQno9kgkqR7fIQTDSAsfbB+q/8kfLtRetp60fYT+2As eDoA== X-Gm-Message-State: AOJu0Yw6awKE/DKr5yQMEFgMTLChY/VtNAmE24mb+uEYIGRsv5yhvC8i wWPLNBJycEKeYnAke+8f201GPb3CwEKWcMUwDZU= X-Google-Smtp-Source: AGHT+IE2Y/SYUKW0AaoXzyoEoFqKr9wIfyi4by0cPw0smmAnZ82U1DJAy7kvLlIa3RhByJ33Cqh+PQ== X-Received: by 2002:a17:902:ccca:b0:1ca:7086:f009 with SMTP id z10-20020a170902ccca00b001ca7086f009mr6796291ple.61.1699551264467; Thu, 09 Nov 2023 09:34:24 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f7-20020a170902ab8700b001c76fcccee8sm3787356plr.156.2023.11.09.09.34.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 09:34:23 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , =?utf-8?q?Morten_Br?= =?utf-8?q?=C3=B8rup?= , Reshma Pattan , Jianfeng Tan Subject: [PATCH v4 1/5] pdump: fix setting rte_errno on mp error Date: Thu, 9 Nov 2023 09:34:08 -0800 Message-Id: <20231109173412.108093-2-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109173412.108093-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109173412.108093-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The response from MP server sets err_value to negative on error. The convention for rte_errno is to use a positive value on error. This makes errors like duplicate registration show up with the correct error value. Fixes: 660098d61f57 ("pdump: use generic multi-process channel") Signed-off-by: Stephen Hemminger Acked-by: Morten Brørup --- lib/pdump/rte_pdump.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/pdump/rte_pdump.c b/lib/pdump/rte_pdump.c index 80b90c6f7d03..e94f49e21250 100644 --- a/lib/pdump/rte_pdump.c +++ b/lib/pdump/rte_pdump.c @@ -564,9 +564,10 @@ pdump_prepare_client_request(const char *device, uint16_t queue, if (rte_mp_request_sync(&mp_req, &mp_reply, &ts) == 0) { mp_rep = &mp_reply.msgs[0]; resp = (struct pdump_response *)mp_rep->param; - rte_errno = resp->err_value; - if (!resp->err_value) + if (resp->err_value == 0) ret = 0; + else + rte_errno = -resp->err_value; free(mp_reply.msgs); } From patchwork Thu Nov 9 17:34:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134021 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DE3E9432E9; Thu, 9 Nov 2023 18:34:35 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2AEB84021E; Thu, 9 Nov 2023 18:34:28 +0100 (CET) Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by mails.dpdk.org (Postfix) with ESMTP id B9B2940DF5 for ; Thu, 9 Nov 2023 18:34:26 +0100 (CET) Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1cc2575dfc7so9500885ad.1 for ; Thu, 09 Nov 2023 09:34:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699551266; x=1700156066; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=o2G/RbxCtbubJZD6MEODF5wEL/gqk9pJIsV2iv3V91Q=; b=gCT2vo9vEGkh+F1wl8YpRsTuh/GkKs7HpLw9wOkCCIz3CkzZ1TWBevpxjkEB0GFueb BravMqoP0JHeyWfo8nVR6XvliD+Na0UWhSgAtt72ui4zoc/KeNHhksjiGqevzhb7g2+A V2nYQMYLTRkr0ESkTedFRstpEMMAAXorA1G+o1DwqPplfRJV09sAG/txuUiw4fTsH3Ku Vn2J7an0LZFImmi8sqxHUQA+MtkpDmVKzEjwvwIJmXf4xSWYtcqmLLd6af7DHHGg/dDr tR8BYhLYtJX+7I77Dr2TZa2GAIyCROie1446tVYLrwSlva1j3l2EyMK+w/U3/D92Dep4 UGDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699551266; x=1700156066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o2G/RbxCtbubJZD6MEODF5wEL/gqk9pJIsV2iv3V91Q=; b=KSd4AAsCoFdbOYNxRl7OW88u9TqoQ1Wrwbmio7uqd02S0TxAINKCUADflJNVlgAWFL T1g3EMkcMpHWxNSoyoVn5myA8j1FOLowBSZFp7NL/U4H/hP3jmtEuHdj0PAGqQ/GFJb6 /XLCZLlBdKPTad54r6SVHRZ+42/hK5COuXOiIZgem/U1H0i0RK/+/O/9Yw2D73Mua3+v Lg/5NUHeH/2xRERiXSwCfXjHQDwtVPRJtWnd6aX387eWVWdDmXF9yyqC0kCPC3tZ3iH6 8Mm8RzsOKyUqHoW07fOdLCzDRVM5D7JDB2JxMwLm7fu3De8Bmtv4ibBxvXf5g39FzdP0 COHQ== X-Gm-Message-State: AOJu0YywMWpkcgOfJFzelTUUv3fzK7I3aaDvnxoiS90AVIovUUJrz9Mt KWhA99za4SD9l5wymvzX92WRrb+zAhYqvh/6AsE= X-Google-Smtp-Source: AGHT+IGYlnW+RlP3RorSnVVkqvohHeYrGV5gHyvzZEA4ZdXUAoSV50KltHAJk0m8fOWZgJjIxeU5Lg== X-Received: by 2002:a17:903:11cd:b0:1ca:362b:166c with SMTP id q13-20020a17090311cd00b001ca362b166cmr6266754plh.61.1699551265711; Thu, 09 Nov 2023 09:34:25 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f7-20020a170902ab8700b001c76fcccee8sm3787356plr.156.2023.11.09.09.34.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 09:34:24 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , Isaac Boukris , Reshma Pattan Subject: [PATCH v4 2/5] dumpcap: allow multiple invocations Date: Thu, 9 Nov 2023 09:34:09 -0800 Message-Id: <20231109173412.108093-3-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109173412.108093-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109173412.108093-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org If dumpcap is run twice with each instance pointing a different interface, it would fail because of overlap in ring a pool names. Fix by putting process id in the name. It is still not allowed to do multiple invocations on the same interface because only one callback is allowed and only one copy of mbuf is done. Dumpcap will fail with error in this case: pdump_prepare_client_request(): client request for pdump enable/disable failed EAL: Error - exiting with code: 1 Cause: Packet dump enable on 0:net_null0 failed File exists Fixes: cbb44143be74 ("app/dumpcap: add new packet capture application") Reported-by: Isaac Boukris Signed-off-by: Stephen Hemminger Reviewed-by: Morten Brørup --- app/dumpcap/main.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/app/dumpcap/main.c b/app/dumpcap/main.c index 64294bbfb3e6..74c754e272c5 100644 --- a/app/dumpcap/main.c +++ b/app/dumpcap/main.c @@ -44,7 +44,6 @@ #include #include -#define RING_NAME "capture-ring" #define MONITOR_INTERVAL (500 * 1000) #define MBUF_POOL_CACHE_SIZE 32 #define BURST_SIZE 32 @@ -647,6 +646,7 @@ static void dpdk_init(void) static struct rte_ring *create_ring(void) { struct rte_ring *ring; + char ring_name[RTE_RING_NAMESIZE]; size_t size, log2; /* Find next power of 2 >= size. */ @@ -660,28 +660,28 @@ static struct rte_ring *create_ring(void) ring_size = size; } - ring = rte_ring_lookup(RING_NAME); - if (ring == NULL) { - ring = rte_ring_create(RING_NAME, ring_size, - rte_socket_id(), 0); - if (ring == NULL) - rte_exit(EXIT_FAILURE, "Could not create ring :%s\n", - rte_strerror(rte_errno)); - } + /* Want one ring per invocation of program */ + snprintf(ring_name, sizeof(ring_name), + "dumpcap-%d", getpid()); + + ring = rte_ring_create(ring_name, ring_size, + rte_socket_id(), 0); + if (ring == NULL) + rte_exit(EXIT_FAILURE, "Could not create ring :%s\n", + rte_strerror(rte_errno)); + return ring; } static struct rte_mempool *create_mempool(void) { const struct interface *intf; - static const char pool_name[] = "capture_mbufs"; + char pool_name[RTE_MEMPOOL_NAMESIZE]; size_t num_mbufs = 2 * ring_size; struct rte_mempool *mp; uint32_t data_size = 128; - mp = rte_mempool_lookup(pool_name); - if (mp) - return mp; + snprintf(pool_name, sizeof(pool_name), "capture_%u", getpid()); /* Common pool so size mbuf for biggest snap length */ TAILQ_FOREACH(intf, &interfaces, next) { @@ -826,7 +826,7 @@ static void enable_pdump(struct rte_ring *r, struct rte_mempool *mp) rte_exit(EXIT_FAILURE, "Packet dump enable on %u:%s failed %s\n", intf->port, intf->name, - rte_strerror(-ret)); + rte_strerror(rte_errno)); } if (intf->opts.promisc_mode) { From patchwork Thu Nov 9 17:34:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134022 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A7CB9432E9; Thu, 9 Nov 2023 18:34:41 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3766340E13; Thu, 9 Nov 2023 18:34:31 +0100 (CET) Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by mails.dpdk.org (Postfix) with ESMTP id 797C440E2D for ; Thu, 9 Nov 2023 18:34:28 +0100 (CET) Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1cc0e78ec92so8659015ad.3 for ; Thu, 09 Nov 2023 09:34:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699551267; x=1700156067; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iYWUZOySddJX58RQBIhFVSEmPqSNnSarPr+n0v2kwpk=; b=q0kACsKPTszSjS6M8/ok7qsXDINlVoA0A9uLvbLv9rGkFMKr6tg/+nWuTlnGPX9h5o RB3cxXtOawcY/XAeZOu00WUbxyCi9gtXNC+0R6NRiJRCVbCCjImpNE2lb7heApL/Cisb Rgx1K12TPszoby3C0vkeyOmxz83sUz7d8n8Uy5fQIWEYFh02OZLehnvxZjAfmQgeU4QG lVxNNUVicFSdcSmLIi16pPrnC9PZqxSqq7Q7vhHuiQYjYdMoxUK297XEWohEi7Ykisr1 oZj8QUsSQQXkQsmZj7wAe3xKFaQKno2J1tAdgHaiy2yZTnvex8g2wAHSdvbFD1lTHqo+ JNLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699551267; x=1700156067; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iYWUZOySddJX58RQBIhFVSEmPqSNnSarPr+n0v2kwpk=; b=ieLR2wb0yy6fP33DyVlNk21MFWnWXu8tFQxTycvR5KWf4SQNaNMlHsUxoRTay5dqqg rNvvhCXriyfpbNHFY99q8Yt48bPsjuqpVhemy6rL3Bq9U6psE6pLrS9yhezaSXI0DTSL vh84wbfdjW3HKlpxGDoenCAbsDw948RP2sLAzYxvMK4rxdtMYIudQ8+1GHnCM8nctXtu T10NkQmxNJPRLhQSa6p7+aE2y5RLeNEMaAPvhV59jr+5ssfdQUi9AOVbwXKL25ufOZzU hs0DvNNQ2Te1R33Ns/3q1O4DUVXY4jEzBf1NEQpbpCyT6F//BCRvos2m6aD6oaiv4b6V bhpQ== X-Gm-Message-State: AOJu0Yywel85gciaoKerwOeszoIkzDUp5BwT3HcOxOKHqG3ijNwBbJEx UhnO8Yx4bj/p0TqQ/FF2DoLHU40hAHmXey5Q08o= X-Google-Smtp-Source: AGHT+IFrfT3L3OGTmwKCO/pf+89KaBnnrjm2L35uLtj9FTcA/5StWn8W1Yr/BjeP59aY2+K/nfDwcg== X-Received: by 2002:a17:902:daca:b0:1cc:51b8:8102 with SMTP id q10-20020a170902daca00b001cc51b88102mr5088139plx.2.1699551267159; Thu, 09 Nov 2023 09:34:27 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f7-20020a170902ab8700b001c76fcccee8sm3787356plr.156.2023.11.09.09.34.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 09:34:26 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , =?utf-8?q?Morten_Br?= =?utf-8?q?=C3=B8rup?= , Reshma Pattan , Jerin Jacob , Kiran Kumar K , Nithin Dabilpuram , Zhirun Yan , Quentin Armitage Subject: [PATCH v4 3/5] pcapng: modify timestamp calculation Date: Thu, 9 Nov 2023 09:34:10 -0800 Message-Id: <20231109173412.108093-4-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109173412.108093-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109173412.108093-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The computation of timestamp is best done in the part of pcapng library that is in secondary process. The secondary process is already doing a bunch of system calls which makes it not performance sensitive. This does change the rte_pcapng_copy() and rte_pcapng_write_stats() experimental API's. Simplify the computation of nanoseconds from TSC to a two step process which avoids numeric overflow issues. The previous code was not thread safe as well. Fixes: c882eb544842 ("pcapng: fix timestamp wrapping in output files") Signed-off-by: Stephen Hemminger Acked-by: Morten Brørup --- app/dumpcap/main.c | 25 +++------ app/test/test_pcapng.c | 4 +- lib/graph/graph_pcap.c | 2 +- lib/pcapng/rte_pcapng.c | 119 +++++++++++++++------------------------- lib/pcapng/rte_pcapng.h | 19 ++----- lib/pdump/rte_pdump.c | 4 +- 6 files changed, 61 insertions(+), 112 deletions(-) diff --git a/app/dumpcap/main.c b/app/dumpcap/main.c index 74c754e272c5..b5770875fab4 100644 --- a/app/dumpcap/main.c +++ b/app/dumpcap/main.c @@ -66,13 +66,13 @@ static bool print_stats; /* capture limit options */ static struct { - uint64_t duration; /* nanoseconds */ + time_t duration; /* seconds */ unsigned long packets; /* number of packets in file */ size_t size; /* file size (bytes) */ } stop; /* Running state */ -static uint64_t start_time, end_time; +static time_t start_time; static uint64_t packets_received; static size_t file_size; @@ -197,7 +197,7 @@ static void auto_stop(char *opt) if (*value == '\0' || *endp != '\0' || interval <= 0) rte_exit(EXIT_FAILURE, "Invalid duration \"%s\"\n", value); - stop.duration = NSEC_PER_SEC * interval; + stop.duration = interval; } else if (strcmp(opt, "filesize") == 0) { stop.size = get_uint(value, "filesize", 0) * 1024; } else if (strcmp(opt, "packets") == 0) { @@ -511,15 +511,6 @@ static void statistics_loop(void) } } -/* Return the time since 1/1/1970 in nanoseconds */ -static uint64_t create_timestamp(void) -{ - struct timespec now; - - clock_gettime(CLOCK_MONOTONIC, &now); - return rte_timespec_to_ns(&now); -} - static void cleanup_pdump_resources(void) { @@ -589,9 +580,8 @@ report_packet_stats(dumpcap_out_t out) ifdrop = pdump_stats.nombuf + pdump_stats.ringfull; if (use_pcapng) - rte_pcapng_write_stats(out.pcapng, intf->port, NULL, - start_time, end_time, - ifrecv, ifdrop); + rte_pcapng_write_stats(out.pcapng, intf->port, + ifrecv, ifdrop, NULL); if (ifrecv == 0) percent = 0; @@ -983,7 +973,7 @@ int main(int argc, char **argv) mp = create_mempool(); out = create_output(); - start_time = create_timestamp(); + start_time = time(NULL); enable_pdump(r, mp); if (!quiet) { @@ -1005,11 +995,10 @@ int main(int argc, char **argv) break; if (stop.duration != 0 && - create_timestamp() - start_time > stop.duration) + time(NULL) - start_time > stop.duration) break; } - end_time = create_timestamp(); disable_primary_monitor(); if (rte_eal_primary_proc_alive(NULL)) diff --git a/app/test/test_pcapng.c b/app/test/test_pcapng.c index b8429a02f160..55aa2cf93666 100644 --- a/app/test/test_pcapng.c +++ b/app/test/test_pcapng.c @@ -173,8 +173,8 @@ test_write_stats(void) ssize_t len; /* write a statistics block */ - len = rte_pcapng_write_stats(pcapng, port_id, - NULL, 0, 0, + len = rte_pcapng_write_stats(pcapng, port_id, NULL, + 0, 0, 0, NUM_PACKETS, 0); if (len <= 0) { fprintf(stderr, "Write of statistics failed\n"); diff --git a/lib/graph/graph_pcap.c b/lib/graph/graph_pcap.c index db722c375fa7..89525f1220ca 100644 --- a/lib/graph/graph_pcap.c +++ b/lib/graph/graph_pcap.c @@ -214,7 +214,7 @@ graph_pcap_dispatch(struct rte_graph *graph, mbuf = (struct rte_mbuf *)objs[i]; mc = rte_pcapng_copy(mbuf->port, 0, mbuf, pkt_mp, mbuf->pkt_len, - rte_get_tsc_cycles(), 0, buffer); + 0, buffer); if (mc == NULL) break; diff --git a/lib/pcapng/rte_pcapng.c b/lib/pcapng/rte_pcapng.c index 3c91fc77644a..13fd2b97fb80 100644 --- a/lib/pcapng/rte_pcapng.c +++ b/lib/pcapng/rte_pcapng.c @@ -36,22 +36,14 @@ /* Format of the capture file handle */ struct rte_pcapng { int outfd; /* output file */ - unsigned int ports; /* number of interfaces added */ + uint64_t offset_ns; /* ns since 1/1/1970 when initialized */ + uint64_t tsc_base; /* TSC when started */ /* DPDK port id to interface index in file */ uint32_t port_index[RTE_MAX_ETHPORTS]; }; -/* For converting TSC cycles to PCAPNG ns format */ -static struct pcapng_time { - uint64_t ns; - uint64_t cycles; - uint64_t tsc_hz; - struct rte_reciprocal_u64 tsc_hz_inverse; -} pcapng_time; - - #ifdef RTE_EXEC_ENV_WINDOWS /* * Windows does not have writev() call. @@ -102,56 +94,21 @@ static ssize_t writev(int fd, const struct iovec *iov, int iovcnt) #define if_indextoname(ifindex, ifname) NULL #endif -static inline void -pcapng_init(void) +/* Convert from TSC (CPU cycles) to nanoseconds */ +static uint64_t +pcapng_timestamp(const rte_pcapng_t *self, uint64_t cycles) { - struct timespec ts; + uint64_t delta, rem, secs, ns; + const uint64_t hz = rte_get_tsc_hz(); - pcapng_time.cycles = rte_get_tsc_cycles(); - clock_gettime(CLOCK_REALTIME, &ts); - pcapng_time.cycles = (pcapng_time.cycles + rte_get_tsc_cycles()) / 2; - pcapng_time.ns = rte_timespec_to_ns(&ts); - - pcapng_time.tsc_hz = rte_get_tsc_hz(); - pcapng_time.tsc_hz_inverse = rte_reciprocal_value_u64(pcapng_time.tsc_hz); -} + delta = cycles - self->tsc_base; -/* PCAPNG timestamps are in nanoseconds */ -static uint64_t pcapng_tsc_to_ns(uint64_t cycles) -{ - uint64_t delta, secs; - - if (!pcapng_time.tsc_hz) - pcapng_init(); - - /* In essence the calculation is: - * delta = (cycles - pcapng_time.cycles) * NSEC_PRE_SEC / rte_get_tsc_hz() - * but this overflows within 4 to 8 seconds depending on TSC frequency. - * Instead, if delta >= pcapng_time.tsc_hz: - * Increase pcapng_time.ns and pcapng_time.cycles by the number of - * whole seconds in delta and reduce delta accordingly. - * delta will therefore always lie in the interval [0, pcapng_time.tsc_hz), - * which will not overflow when multiplied by NSEC_PER_SEC provided the - * TSC frequency < approx 18.4GHz. - * - * Currently all TSCs operate below 5GHz. - */ - delta = cycles - pcapng_time.cycles; - if (unlikely(delta >= pcapng_time.tsc_hz)) { - if (likely(delta < pcapng_time.tsc_hz * 2)) { - delta -= pcapng_time.tsc_hz; - pcapng_time.cycles += pcapng_time.tsc_hz; - pcapng_time.ns += NSEC_PER_SEC; - } else { - secs = rte_reciprocal_divide_u64(delta, &pcapng_time.tsc_hz_inverse); - delta -= secs * pcapng_time.tsc_hz; - pcapng_time.cycles += secs * pcapng_time.tsc_hz; - pcapng_time.ns += secs * NSEC_PER_SEC; - } - } + /* Avoid numeric wraparound by computing seconds first */ + secs = delta / hz; + rem = delta % hz; + ns = (rem * NS_PER_S) / hz; - return pcapng_time.ns + rte_reciprocal_divide_u64(delta * NSEC_PER_SEC, - &pcapng_time.tsc_hz_inverse); + return secs * NS_PER_S + ns + self->offset_ns; } /* length of option including padding */ @@ -368,15 +325,15 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, */ ssize_t rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, - const char *comment, - uint64_t start_time, uint64_t end_time, - uint64_t ifrecv, uint64_t ifdrop) + uint64_t ifrecv, uint64_t ifdrop, + const char *comment) { struct pcapng_statistics *hdr; struct pcapng_option *opt; + uint64_t start_time = self->offset_ns; + uint64_t sample_time; uint32_t optlen, len; uint8_t *buf; - uint64_t ns; RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); @@ -386,10 +343,10 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, optlen += pcapng_optlen(sizeof(ifrecv)); if (ifdrop != UINT64_MAX) optlen += pcapng_optlen(sizeof(ifdrop)); + if (start_time != 0) optlen += pcapng_optlen(sizeof(start_time)); - if (end_time != 0) - optlen += pcapng_optlen(sizeof(end_time)); + if (comment) optlen += pcapng_optlen(strlen(comment)); if (optlen != 0) @@ -409,9 +366,6 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, if (start_time != 0) opt = pcapng_add_option(opt, PCAPNG_ISB_STARTTIME, &start_time, sizeof(start_time)); - if (end_time != 0) - opt = pcapng_add_option(opt, PCAPNG_ISB_ENDTIME, - &end_time, sizeof(end_time)); if (ifrecv != UINT64_MAX) opt = pcapng_add_option(opt, PCAPNG_ISB_IFRECV, &ifrecv, sizeof(ifrecv)); @@ -425,9 +379,9 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, hdr->block_length = len; hdr->interface_id = self->port_index[port_id]; - ns = pcapng_tsc_to_ns(rte_get_tsc_cycles()); - hdr->timestamp_hi = ns >> 32; - hdr->timestamp_lo = (uint32_t)ns; + sample_time = pcapng_timestamp(self, rte_get_tsc_cycles()); + hdr->timestamp_hi = sample_time >> 32; + hdr->timestamp_lo = (uint32_t)sample_time; /* clone block_length after option */ memcpy(opt, &len, sizeof(uint32_t)); @@ -520,23 +474,21 @@ struct rte_mbuf * rte_pcapng_copy(uint16_t port_id, uint32_t queue, const struct rte_mbuf *md, struct rte_mempool *mp, - uint32_t length, uint64_t cycles, + uint32_t length, enum rte_pcapng_direction direction, const char *comment) { struct pcapng_enhance_packet_block *epb; uint32_t orig_len, data_len, padding, flags; struct pcapng_option *opt; + uint64_t timestamp; uint16_t optlen; struct rte_mbuf *mc; - uint64_t ns; bool rss_hash; #ifdef RTE_LIBRTE_ETHDEV_DEBUG RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, NULL); #endif - ns = pcapng_tsc_to_ns(cycles); - orig_len = rte_pktmbuf_pkt_len(md); /* Take snapshot of the data */ @@ -641,8 +593,10 @@ rte_pcapng_copy(uint16_t port_id, uint32_t queue, /* Interface index is filled in later during write */ mc->port = port_id; - epb->timestamp_hi = ns >> 32; - epb->timestamp_lo = (uint32_t)ns; + /* Put timestamp in cycles here - adjust in packet write */ + timestamp = rte_get_tsc_cycles(); + epb->timestamp_hi = timestamp >> 32; + epb->timestamp_lo = (uint32_t)timestamp; epb->capture_length = data_len; epb->original_length = orig_len; @@ -668,6 +622,7 @@ rte_pcapng_write_packets(rte_pcapng_t *self, for (i = 0; i < nb_pkts; i++) { struct rte_mbuf *m = pkts[i]; struct pcapng_enhance_packet_block *epb; + uint64_t cycles, timestamp; /* sanity check that is really a pcapng mbuf */ epb = rte_pktmbuf_mtod(m, struct pcapng_enhance_packet_block *); @@ -684,6 +639,13 @@ rte_pcapng_write_packets(rte_pcapng_t *self, return -1; } + /* adjust timestamp recorded in packet */ + cycles = (uint64_t)epb->timestamp_hi << 32; + cycles += epb->timestamp_lo; + timestamp = pcapng_timestamp(self, cycles); + epb->timestamp_hi = timestamp >> 32; + epb->timestamp_lo = (uint32_t)timestamp; + /* * Handle case of highly fragmented and large burst size * Note: this assumes that max segments per mbuf < IOV_MAX @@ -725,6 +687,8 @@ rte_pcapng_fdopen(int fd, { unsigned int i; rte_pcapng_t *self; + struct timespec ts; + uint64_t cycles; self = malloc(sizeof(*self)); if (!self) { @@ -734,6 +698,13 @@ rte_pcapng_fdopen(int fd, self->outfd = fd; self->ports = 0; + + /* record start time in ns since 1/1/1970 */ + cycles = rte_get_tsc_cycles(); + clock_gettime(CLOCK_REALTIME, &ts); + self->tsc_base = (cycles + rte_get_tsc_cycles()) / 2; + self->offset_ns = rte_timespec_to_ns(&ts); + for (i = 0; i < RTE_MAX_ETHPORTS; i++) self->port_index[i] = UINT32_MAX; diff --git a/lib/pcapng/rte_pcapng.h b/lib/pcapng/rte_pcapng.h index d93cc9f73ad5..c40795c721de 100644 --- a/lib/pcapng/rte_pcapng.h +++ b/lib/pcapng/rte_pcapng.h @@ -121,8 +121,6 @@ enum rte_pcapng_direction { * @param length * The upper limit on bytes to copy. Passing UINT32_MAX * means all data (after offset). - * @param timestamp - * The timestamp in TSC cycles. * @param direction * The direction of the packer: receive, transmit or unknown. * @param comment @@ -136,7 +134,7 @@ __rte_experimental struct rte_mbuf * rte_pcapng_copy(uint16_t port_id, uint32_t queue, const struct rte_mbuf *m, struct rte_mempool *mp, - uint32_t length, uint64_t timestamp, + uint32_t length, enum rte_pcapng_direction direction, const char *comment); @@ -188,29 +186,22 @@ rte_pcapng_write_packets(rte_pcapng_t *self, * The handle to the packet capture file * @param port * The Ethernet port to report stats on. - * @param comment - * Optional comment to add to statistics. - * @param start_time - * The time when packet capture was started in nanoseconds. - * Optional: can be zero if not known. - * @param end_time - * The time when packet capture was stopped in nanoseconds. - * Optional: can be zero if not finished; * @param ifrecv * The number of packets received by capture. * Optional: use UINT64_MAX if not known. * @param ifdrop * The number of packets missed by the capture process. * Optional: use UINT64_MAX if not known. + * @param comment + * Optional comment to add to statistics. * @return * number of bytes written to file, -1 on failure to write file */ __rte_experimental ssize_t rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port, - const char *comment, - uint64_t start_time, uint64_t end_time, - uint64_t ifrecv, uint64_t ifdrop); + uint64_t ifrecv, uint64_t ifdrop, + const char *comment); #ifdef __cplusplus } diff --git a/lib/pdump/rte_pdump.c b/lib/pdump/rte_pdump.c index e94f49e21250..5a1ec14d7a18 100644 --- a/lib/pdump/rte_pdump.c +++ b/lib/pdump/rte_pdump.c @@ -90,7 +90,6 @@ pdump_copy(uint16_t port_id, uint16_t queue, int ring_enq; uint16_t d_pkts = 0; struct rte_mbuf *dup_bufs[nb_pkts]; - uint64_t ts; struct rte_ring *ring; struct rte_mempool *mp; struct rte_mbuf *p; @@ -99,7 +98,6 @@ pdump_copy(uint16_t port_id, uint16_t queue, if (cbs->filter) rte_bpf_exec_burst(cbs->filter, (void **)pkts, rcs, nb_pkts); - ts = rte_get_tsc_cycles(); ring = cbs->ring; mp = cbs->mp; for (i = 0; i < nb_pkts; i++) { @@ -122,7 +120,7 @@ pdump_copy(uint16_t port_id, uint16_t queue, if (cbs->ver == V2) p = rte_pcapng_copy(port_id, queue, pkts[i], mp, cbs->snaplen, - ts, direction, NULL); + direction, NULL); else p = rte_pktmbuf_copy(pkts[i], mp, 0, cbs->snaplen); From patchwork Thu Nov 9 17:34:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134023 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4AA1F432E9; Thu, 9 Nov 2023 18:34:49 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A8CB540ED8; Thu, 9 Nov 2023 18:34:32 +0100 (CET) Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by mails.dpdk.org (Postfix) with ESMTP id 57A8A40E64 for ; Thu, 9 Nov 2023 18:34:31 +0100 (CET) Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1cc3bb4c307so9849525ad.0 for ; Thu, 09 Nov 2023 09:34:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699551270; x=1700156070; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N2r/H5v/jMSX7aFlXgmw6wyhS0CLdclJuU10KJ1gaU0=; b=pJDh+AZEvmN67A9qyz1px1iigf+aswBZSoar+YXrpfwVdrUSDVnECtLFlJP0ZPSwYj 7gGHwErKj5yUUsadjetAQWGNem9bztgwigKCx+EgYYuUHWnXq+5wz3ECUGptpGZuhjpU J3kbIt5zgxCFg1M72auaTeeNGsS5gWzQa/dYgFkbAHZKhy2uwiSLQyxM8TQax0jBRsWU wS0h673Gy4fTci3i92S+OJz/WStvkDz4TTvZbSs6+8gTFhvXtV1BpFOhMmY/+OwlbNKS lx8MWfe+xzR+2TjeerIXXBy85LhsXyQKHqbgI0RL7Y04HGtk5l0RsziAGMyFYGIyRnRE UKug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699551270; x=1700156070; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N2r/H5v/jMSX7aFlXgmw6wyhS0CLdclJuU10KJ1gaU0=; b=iJ+gxJ+Jh6pWiHrLEQns8tOBwXW5zqp4AcrQsH4tSbaRHaP9wgyVg3Xo7M8gVqgRe2 Nv8VIYNXqBaBeOJasKZGzAFw4lppfOieAsjEVnRpd2bKb8b95Jc+waZ3xKxk8Nsxsg86 4PqubRpOPhMnti+hKLSamX4IKe1pMmcTCqk1+GM84/BLpXPhpXBK/P899KF36BNnvllw VelCoDvRptWXe/izarTl3E9+scBdJf9q2/Pxtf47ltgj+T8kDgKFdPT3ZCgnXPdYy3oa C+gYJzoLiGeDRaEjp8CwHd3+xVrSJY12Qc8Hqd4hDcnxq/KFVQjUfc082dyOiDhWARN0 IUDQ== X-Gm-Message-State: AOJu0YykOB+GSV+6KyVIHDIkC+lutbLDwMRozVHVVTnkqA4TWpNvrlu4 gWCq+rRMqJS0KO5JGrK3qIewXBJP0B0cj562cLg= X-Google-Smtp-Source: AGHT+IF1m7H1f1xku6SWupRD/UdGi6ZMBGnwrGNK/uqmiacnqn6A7yBkVl1/Lchijm9HLlnbmCrN5Q== X-Received: by 2002:a17:903:230d:b0:1cc:5f51:b1ed with SMTP id d13-20020a170903230d00b001cc5f51b1edmr6542255plh.47.1699551268811; Thu, 09 Nov 2023 09:34:28 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f7-20020a170902ab8700b001c76fcccee8sm3787356plr.156.2023.11.09.09.34.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 09:34:27 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , =?utf-8?q?Morten_Br?= =?utf-8?q?=C3=B8rup?= , Reshma Pattan Subject: [PATCH v4 4/5] pcapng: avoid using alloca() Date: Thu, 9 Nov 2023 09:34:11 -0800 Message-Id: <20231109173412.108093-5-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109173412.108093-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109173412.108093-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The function alloca() like VLA's has problems if the caller passes a large value. Instead use a fixed size buffer (2K) which will be more than sufficient for the info related blocks in the file. Add bounds checks as well. Signed-off-by: Stephen Hemminger Acked-by: Morten Brørup --- lib/pcapng/rte_pcapng.c | 37 ++++++++++++++++--------------------- 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/lib/pcapng/rte_pcapng.c b/lib/pcapng/rte_pcapng.c index 13fd2b97fb80..f74ec939a9f8 100644 --- a/lib/pcapng/rte_pcapng.c +++ b/lib/pcapng/rte_pcapng.c @@ -33,6 +33,9 @@ /* conversion from DPDK speed to PCAPNG */ #define PCAPNG_MBPS_SPEED 1000000ull +/* upper bound for section, stats and interface blocks */ +#define PCAPNG_BLKSIZ 2048 + /* Format of the capture file handle */ struct rte_pcapng { int outfd; /* output file */ @@ -140,9 +143,8 @@ pcapng_section_block(rte_pcapng_t *self, { struct pcapng_section_header *hdr; struct pcapng_option *opt; - void *buf; + uint8_t buf[PCAPNG_BLKSIZ]; uint32_t len; - ssize_t cc; len = sizeof(*hdr); if (hw) @@ -158,8 +160,7 @@ pcapng_section_block(rte_pcapng_t *self, len += pcapng_optlen(0); len += sizeof(uint32_t); - buf = calloc(1, len); - if (!buf) + if (len > sizeof(buf)) return -1; hdr = (struct pcapng_section_header *)buf; @@ -193,10 +194,7 @@ pcapng_section_block(rte_pcapng_t *self, /* clone block_length after option */ memcpy(opt, &hdr->block_length, sizeof(uint32_t)); - cc = write(self->outfd, buf, len); - free(buf); - - return cc; + return write(self->outfd, buf, len); } /* Write an interface block for a DPDK port */ @@ -213,7 +211,7 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, struct pcapng_option *opt; const uint8_t tsresol = 9; /* nanosecond resolution */ uint32_t len; - void *buf; + uint8_t buf[PCAPNG_BLKSIZ]; char ifname_buf[IF_NAMESIZE]; char ifhw[256]; uint64_t speed = 0; @@ -267,8 +265,7 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, len += pcapng_optlen(0); len += sizeof(uint32_t); - buf = alloca(len); - if (!buf) + if (len > sizeof(buf)) return -1; hdr = (struct pcapng_interface_block *)buf; @@ -296,17 +293,16 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, opt = pcapng_add_option(opt, PCAPNG_IFB_HARDWARE, ifhw, strlen(ifhw)); if (filter) { - /* Encoding is that the first octet indicates string vs BPF */ size_t len; - char *buf; len = strlen(filter) + 1; - buf = alloca(len); - *buf = '\0'; - memcpy(buf + 1, filter, len); + opt->code = PCAPNG_IFB_FILTER; + opt->length = len; + /* Encoding is that the first octet indicates string vs BPF */ + opt->data[0] = 0; + memcpy(opt->data + 1, filter, strlen(filter)); - opt = pcapng_add_option(opt, PCAPNG_IFB_FILTER, - buf, len); + opt = (struct pcapng_option *)((uint8_t *)opt + pcapng_optlen(len)); } opt = pcapng_add_option(opt, PCAPNG_OPT_END, NULL, 0); @@ -333,7 +329,7 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, uint64_t start_time = self->offset_ns; uint64_t sample_time; uint32_t optlen, len; - uint8_t *buf; + uint8_t buf[PCAPNG_BLKSIZ]; RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); @@ -353,8 +349,7 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, optlen += pcapng_optlen(0); len = sizeof(*hdr) + optlen + sizeof(uint32_t); - buf = alloca(len); - if (buf == NULL) + if (len > sizeof(buf)) return -1; hdr = (struct pcapng_statistics *)buf; From patchwork Thu Nov 9 17:34:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134024 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6062432E9; Thu, 9 Nov 2023 18:34:54 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C0F7240E6E; Thu, 9 Nov 2023 18:34:34 +0100 (CET) Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by mails.dpdk.org (Postfix) with ESMTP id 96CFB40EDF for ; Thu, 9 Nov 2023 18:34:33 +0100 (CET) Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1cc1e1e74beso10348145ad.1 for ; Thu, 09 Nov 2023 09:34:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699551272; x=1700156072; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZxBQOrc3DzlSpoSRdVgX3NKVNdfN6wt2OXsbYKLwbLc=; b=Q24Y34EUAppZkCD69G09upGxH7ys3tyPEF9i+knnfNZuPyf2Q6oIGhL0YGnVJy2739 v9ITNOhnBs+ry5zgRnV0ioHxmKbNnrcHfyderLwO8UIBi2SPzbWEUROwFahVlriYIy6n LXr8AMk5hB1ewtWXp2y9cURRioQAmNdVnlvt3YYfxXpe8yA5Fd+qhg9QPCWb5ucGrYHe EO1U6bkCC0/1tfxn+92yg2HH5+V8J+zcWuTro2Xzm0Swp8W9tJThH9NflZ7eb2JGpuMU 82upT2ORbmQl445FMklp7nCMWsAiUnaBbQmjoiDpswjzs6a4q8quoDGVl5DS06r9OHcj DjtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699551272; x=1700156072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZxBQOrc3DzlSpoSRdVgX3NKVNdfN6wt2OXsbYKLwbLc=; b=ToSE6Y7htt6g709Hb+716HfkFyCcjeNf9Lw4NAGyMEStqCPglVOVORseItIN5CuaPU SR2lhR0GretpGWCoxfQS5yTV7UDG5SCxfyopuL35ZkFNaXLSO8VikuXNGnTH7F8RNO0q NrTTimgjSAPt1IpmAtN5Q6aEWRzoJrcj0XkOiA+TVI/i4I23FOcJM9/ILGkgWJz9VPio Sv75b3Q7HcCU77fIs3GOLtSfEgUfMizRn4K7W+KIClu4LBBDqJ+yiMGcN9euBUzMDKit ddupuLOx0YHyKkE2LYHZzK6gYqkq3vCKHgH69jjcQv/y/DaMuuS92noIZtl0DwDF+kT9 uEjw== X-Gm-Message-State: AOJu0YwPAgeRHHajEX/mWS8OSIjpcq7vn8hK/g/+ti/7nRSe/0JmKOmb RhiczDiF+bxDMEOrQsi3M5PVx8RPrF2kHBKcMiU= X-Google-Smtp-Source: AGHT+IFddkROlR7QWf5yNEDxIFpjotjAfIrWEwjks+vacDuw/AwawXHqT+evtfE0MvM9RjLbJwFepA== X-Received: by 2002:a17:902:dacc:b0:1cc:4677:2f0e with SMTP id q12-20020a170902dacc00b001cc46772f0emr6835044plx.45.1699551272195; Thu, 09 Nov 2023 09:34:32 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f7-20020a170902ab8700b001c76fcccee8sm3787356plr.156.2023.11.09.09.34.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 09:34:29 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , Reshma Pattan Subject: [PATCH v4 5/5] test: cleanups to pcapng test Date: Thu, 9 Nov 2023 09:34:12 -0800 Message-Id: <20231109173412.108093-6-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109173412.108093-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109173412.108093-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Overhaul of the pcapng test: - promote it to be a fast test so it gets regularly run. - create null device and use i. - use UDP discard packets that are valid so that for debugging the resulting pcapng file can be looked at with wireshark. - do basic checks on resulting pcap file that lengths and timestamps are in range. - add test for interface options Signed-off-by: Stephen Hemminger --- app/test/meson.build | 2 +- app/test/test_pcapng.c | 418 +++++++++++++++++++++++++++-------------- 2 files changed, 282 insertions(+), 138 deletions(-) diff --git a/app/test/meson.build b/app/test/meson.build index 4183d66b0e9c..dcc93f4a43b4 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -128,7 +128,7 @@ source_file_deps = { 'test_metrics.c': ['metrics'], 'test_mp_secondary.c': ['hash', 'lpm'], 'test_net_ether.c': ['net'], - 'test_pcapng.c': ['ethdev', 'net', 'pcapng'], + 'test_pcapng.c': ['ethdev', 'net', 'pcapng', 'bus_vdev'], 'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'], 'test_pdump.c': ['pdump'] + sample_packet_forward_deps, 'test_per_lcore.c': [], diff --git a/app/test/test_pcapng.c b/app/test/test_pcapng.c index 55aa2cf93666..c973aa47d1f8 100644 --- a/app/test/test_pcapng.c +++ b/app/test/test_pcapng.c @@ -6,25 +6,34 @@ #include #include +#include #include #include +#include #include #include #include #include +#include +#include +#include +#include #include #include "test.h" -#define NUM_PACKETS 10 -#define DUMMY_MBUF_NUM 3 +#define PCAPNG_TEST_DEBUG 0 + +#define TOTAL_PACKETS 4096 +#define MAX_BURST 64 +#define MAX_GAP_US 100000 +#define DUMMY_MBUF_NUM 3 -static rte_pcapng_t *pcapng; static struct rte_mempool *mp; static const uint32_t pkt_len = 200; static uint16_t port_id; -static char file_name[] = "/tmp/pcapng_test_XXXXXX.pcapng"; +static const char null_dev[] = "net_null0"; /* first mbuf in the packet, should always be at offset 0 */ struct dummy_mbuf { @@ -61,6 +70,7 @@ mbuf1_prepare(struct dummy_mbuf *dm, uint32_t plen) struct { struct rte_ether_hdr eth; struct rte_ipv4_hdr ip; + struct rte_udp_hdr udp; } pkt = { .eth = { .dst_addr.addr_bytes = "\xff\xff\xff\xff\xff\xff", @@ -68,149 +78,201 @@ mbuf1_prepare(struct dummy_mbuf *dm, uint32_t plen) }, .ip = { .version_ihl = RTE_IPV4_VHL_DEF, - .total_length = rte_cpu_to_be_16(plen), - .time_to_live = IPDEFTTL, - .next_proto_id = IPPROTO_RAW, + .time_to_live = 1, + .next_proto_id = IPPROTO_UDP, .src_addr = rte_cpu_to_be_32(RTE_IPV4_LOOPBACK), .dst_addr = rte_cpu_to_be_32(RTE_IPV4_BROADCAST), - } + }, + .udp = { + .dst_port = rte_cpu_to_be_16(9), /* Discard port */ + }, }; memset(dm, 0, sizeof(*dm)); dummy_mbuf_prep(&dm->mb[0], dm->buf[0], sizeof(dm->buf[0]), plen); rte_eth_random_addr(pkt.eth.src_addr.addr_bytes); - memcpy(rte_pktmbuf_mtod(dm->mb, void *), &pkt, RTE_MIN(sizeof(pkt), plen)); + plen -= sizeof(struct rte_ether_hdr); + + pkt.ip.total_length = rte_cpu_to_be_16(plen); + pkt.ip.hdr_checksum = rte_ipv4_cksum(&pkt.ip); + + plen -= sizeof(struct rte_ipv4_hdr); + pkt.udp.src_port = rte_rand(); + pkt.udp.dgram_len = rte_cpu_to_be_16(plen); + + memcpy(rte_pktmbuf_mtod(dm->mb, void *), &pkt, sizeof(pkt)); } static int test_setup(void) { - int tmp_fd; - - port_id = rte_eth_find_next(0); - if (port_id >= RTE_MAX_ETHPORTS) { - fprintf(stderr, "No valid Ether port\n"); - return -1; - } + port_id = rte_eth_dev_count_avail(); - tmp_fd = mkstemps(file_name, strlen(".pcapng")); - if (tmp_fd == -1) { - perror("mkstemps() failure"); - return -1; - } - printf("pcapng: output file %s\n", file_name); - - /* open a test capture file */ - pcapng = rte_pcapng_fdopen(tmp_fd, NULL, NULL, "pcapng_test", NULL); - if (pcapng == NULL) { - fprintf(stderr, "rte_pcapng_fdopen failed\n"); - close(tmp_fd); - return -1; - } - - /* Add interface to the file */ - if (rte_pcapng_add_interface(pcapng, port_id, - NULL, NULL, NULL) != 0) { - fprintf(stderr, "can not add port %u\n", port_id); - return -1; + /* Make a dummy null device to snoop on */ + if (rte_vdev_init(null_dev, NULL) != 0) { + fprintf(stderr, "Failed to create vdev '%s'\n", null_dev); + goto fail; } /* Make a pool for cloned packets */ - mp = rte_pktmbuf_pool_create_by_ops("pcapng_test_pool", IOV_MAX + NUM_PACKETS, - 0, 0, - rte_pcapng_mbuf_size(pkt_len), + mp = rte_pktmbuf_pool_create_by_ops("pcapng_test_pool", + MAX_BURST, 0, 0, + rte_pcapng_mbuf_size(pkt_len) + 128, SOCKET_ID_ANY, "ring_mp_sc"); if (mp == NULL) { fprintf(stderr, "Cannot create mempool\n"); - return -1; + goto fail; } + return 0; + +fail: + rte_vdev_uninit(null_dev); + rte_mempool_free(mp); + return -1; } static int -test_write_packets(void) +fill_pcapng_file(rte_pcapng_t *pcapng, unsigned int num_packets) { - struct rte_mbuf *orig; - struct rte_mbuf *clones[NUM_PACKETS] = { }; struct dummy_mbuf mbfs; - unsigned int i; + struct rte_mbuf *orig; + unsigned int burst_size; + unsigned int count; ssize_t len; /* make a dummy packet */ mbuf1_prepare(&mbfs, pkt_len); - - /* clone them */ orig = &mbfs.mb[0]; - for (i = 0; i < NUM_PACKETS; i++) { - struct rte_mbuf *mc; - mc = rte_pcapng_copy(port_id, 0, orig, mp, pkt_len, - rte_get_tsc_cycles(), 0, NULL); - if (mc == NULL) { - fprintf(stderr, "Cannot copy packet\n"); + for (count = 0; count < num_packets; count += burst_size) { + struct rte_mbuf *clones[MAX_BURST]; + unsigned int i; + + /* put 1 .. MAX_BURST packets in one write call */ + burst_size = rte_rand_max(MAX_BURST) + 1; + for (i = 0; i < burst_size; i++) { + struct rte_mbuf *mc; + + mc = rte_pcapng_copy(port_id, 0, orig, mp, pkt_len, + RTE_PCAPNG_DIRECTION_IN, + NULL); + if (mc == NULL) { + fprintf(stderr, "Cannot copy packet\n"); + return -1; + } + clones[i] = mc; + } + + /* write it to capture file */ + len = rte_pcapng_write_packets(pcapng, clones, burst_size); + rte_pktmbuf_free_bulk(clones, burst_size); + + if (len <= 0) { + fprintf(stderr, "Write of packets failed: %s\n", + rte_strerror(rte_errno)); return -1; } - clones[i] = mc; + + /* Leave a small gap between packets to test for time wrap */ + usleep(rte_rand_max(MAX_GAP_US)); } - /* write it to capture file */ - len = rte_pcapng_write_packets(pcapng, clones, NUM_PACKETS); + return count; +} - rte_pktmbuf_free_bulk(clones, NUM_PACKETS); +static char * +fmt_time(char *buf, size_t size, uint64_t ts_ns) +{ + time_t sec; + size_t len; - if (len <= 0) { - fprintf(stderr, "Write of packets failed\n"); - return -1; - } + sec = ts_ns / NS_PER_S; + len = strftime(buf, size, "%X", localtime(&sec)); + snprintf(buf + len, size - len, ".%09lu", + (unsigned long)(ts_ns % NS_PER_S)); - return 0; + return buf; } -static int -test_write_stats(void) +/* Context for the pcap_loop callback */ +struct pkt_print_ctx { + pcap_t *pcap; + unsigned int count; + uint64_t start_ns; + uint64_t end_ns; +}; + +static void +print_packet(uint64_t ts_ns, const struct rte_ether_hdr *eh, size_t len) { - ssize_t len; + char tbuf[128], src[64], dst[64]; - /* write a statistics block */ - len = rte_pcapng_write_stats(pcapng, port_id, NULL, - 0, 0, 0, - NUM_PACKETS, 0); - if (len <= 0) { - fprintf(stderr, "Write of statistics failed\n"); - return -1; - } - return 0; + fmt_time(tbuf, sizeof(tbuf), ts_ns); + rte_ether_format_addr(dst, sizeof(dst), &eh->dst_addr); + rte_ether_format_addr(src, sizeof(src), &eh->src_addr); + printf("%s: %s -> %s type %x length %zu\n", + tbuf, src, dst, rte_be_to_cpu_16(eh->ether_type), len); } +/* Callback from pcap_loop used to validate packets in the file */ static void -pkt_print(u_char *user, const struct pcap_pkthdr *h, - const u_char *bytes) +parse_pcap_packet(u_char *user, const struct pcap_pkthdr *h, + const u_char *bytes) { - unsigned int *countp = (unsigned int *)user; + struct pkt_print_ctx *ctx = (struct pkt_print_ctx *)user; const struct rte_ether_hdr *eh; - struct tm *tm; - char tbuf[128], src[64], dst[64]; + const struct rte_ipv4_hdr *ip; + uint64_t ns; - tm = localtime(&h->ts.tv_sec); - if (tm == NULL) { - perror("localtime"); - return; + eh = (const struct rte_ether_hdr *)bytes; + ip = (const struct rte_ipv4_hdr *)(eh + 1); + + ctx->count += 1; + + /* The pcap library is misleading in reporting timestamp. + * packet header struct gives timestamp as a timeval (ie. usec); + * but the file is open in nanonsecond mode therefore + * the timestamp is really in timespec (ie. nanoseconds). + */ + ns = h->ts.tv_sec * NS_PER_S + h->ts.tv_usec; + if (ns < ctx->start_ns || ns > ctx->end_ns) { + char tstart[128], tend[128]; + + fmt_time(tstart, sizeof(tstart), ctx->start_ns); + fmt_time(tend, sizeof(tend), ctx->end_ns); + fprintf(stderr, "Timestamp out of range [%s .. %s]\n", + tstart, tend); + goto error; } - if (strftime(tbuf, sizeof(tbuf), "%X", tm) == 0) { - fprintf(stderr, "strftime returned 0!\n"); - return; + if (!rte_is_broadcast_ether_addr(&eh->dst_addr)) { + fprintf(stderr, "Destination is not broadcast\n"); + goto error; } - eh = (const struct rte_ether_hdr *)bytes; - rte_ether_format_addr(dst, sizeof(dst), &eh->dst_addr); - rte_ether_format_addr(src, sizeof(src), &eh->src_addr); - printf("%s.%06lu: %s -> %s type %x length %u\n", - tbuf, (unsigned long)h->ts.tv_usec, - src, dst, rte_be_to_cpu_16(eh->ether_type), h->len); + if (rte_ipv4_cksum(ip) != 0) { + fprintf(stderr, "Bad IPv4 checksum\n"); + goto error; + } + + return; /* packet is normal */ + +error: + print_packet(ns, eh, h->len); + + /* Stop parsing at first error */ + pcap_breakloop(ctx->pcap); +} - *countp += 1; +static uint64_t +current_timestamp(void) +{ + struct timespec ts; + + clock_gettime(CLOCK_REALTIME, &ts); + return rte_timespec_to_ns(&ts); } /* @@ -219,78 +281,162 @@ pkt_print(u_char *user, const struct pcap_pkthdr *h, * but that creates an unwanted dependency. */ static int -test_validate(void) +valid_pcapng_file(const char *file_name, uint64_t started, unsigned int expected) { char errbuf[PCAP_ERRBUF_SIZE]; - unsigned int count = 0; - pcap_t *pcap; + struct pkt_print_ctx ctx = { }; int ret; - pcap = pcap_open_offline(file_name, errbuf); - if (pcap == NULL) { + ctx.start_ns = started; + ctx.end_ns = current_timestamp(); + + ctx.pcap = pcap_open_offline_with_tstamp_precision(file_name, + PCAP_TSTAMP_PRECISION_NANO, + errbuf); + if (ctx.pcap == NULL) { fprintf(stderr, "pcap_open_offline('%s') failed: %s\n", file_name, errbuf); return -1; } - ret = pcap_loop(pcap, 0, pkt_print, (u_char *)&count); - if (ret == 0) - printf("Saw %u packets\n", count); - else + ret = pcap_loop(ctx.pcap, 0, parse_pcap_packet, (u_char *)&ctx); + if (ret != 0) { fprintf(stderr, "pcap_dispatch: failed: %s\n", - pcap_geterr(pcap)); - pcap_close(pcap); + pcap_geterr(ctx.pcap)); + } else if (ctx.count != expected) { + printf("Only %u packets, expected %u\n", + ctx.count, expected); + ret = -1; + } + + pcap_close(ctx.pcap); return ret; } static int -test_write_over_limit_iov_max(void) +test_add_interface(void) { - struct rte_mbuf *orig; - struct rte_mbuf *clones[IOV_MAX + NUM_PACKETS] = { }; - struct dummy_mbuf mbfs; - unsigned int i; - ssize_t len; + char file_name[] = "/tmp/pcapng_test_XXXXXX.pcapng"; + static rte_pcapng_t *pcapng; + int ret, tmp_fd; + uint64_t now = current_timestamp(); - /* make a dummy packet */ - mbuf1_prepare(&mbfs, pkt_len); + tmp_fd = mkstemps(file_name, strlen(".pcapng")); + if (tmp_fd == -1) { + perror("mkstemps() failure"); + goto fail; + } + printf("pcapng: output file %s\n", file_name); - /* clone them */ - orig = &mbfs.mb[0]; - for (i = 0; i < IOV_MAX + NUM_PACKETS; i++) { - struct rte_mbuf *mc; + /* open a test capture file */ + pcapng = rte_pcapng_fdopen(tmp_fd, NULL, NULL, "pcapng_addif", NULL); + if (pcapng == NULL) { + fprintf(stderr, "rte_pcapng_fdopen failed\n"); + close(tmp_fd); + goto fail; + } - mc = rte_pcapng_copy(port_id, 0, orig, mp, pkt_len, - rte_get_tsc_cycles(), 0, NULL); - if (mc == NULL) { - fprintf(stderr, "Cannot copy packet\n"); - return -1; - } - clones[i] = mc; + /* Add interface to the file */ + ret = rte_pcapng_add_interface(pcapng, port_id, + NULL, NULL, NULL); + if (ret < 0) { + fprintf(stderr, "can not add port %u\n", port_id); + goto fail; } - /* write it to capture file */ - len = rte_pcapng_write_packets(pcapng, clones, IOV_MAX + NUM_PACKETS); + /* Add interface with ifname and ifdescr */ + ret = rte_pcapng_add_interface(pcapng, port_id, + "myeth", "Some long description", NULL); + if (ret < 0) { + fprintf(stderr, "can not add port %u with ifname\n", port_id); + goto fail; + } + + /* Add interface with filter */ + ret = rte_pcapng_add_interface(pcapng, port_id, + NULL, NULL, "tcp port 8080"); + if (ret < 0) { + fprintf(stderr, "can not add port %u with filter\n", port_id); + goto fail; + } - rte_pktmbuf_free_bulk(clones, IOV_MAX + NUM_PACKETS); + rte_pcapng_close(pcapng); - if (len <= 0) { - fprintf(stderr, "Write of packets failed\n"); - return -1; + ret = valid_pcapng_file(file_name, now, 0); + /* if test fails want to investigate the file */ + if (ret == 0) + unlink(file_name); + + return ret; + +fail: + rte_pcapng_close(pcapng); + return -1; +} + +static int +test_write_packets(void) +{ + char file_name[] = "/tmp/pcapng_test_XXXXXX.pcapng"; + static rte_pcapng_t *pcapng; + int ret, tmp_fd, count; + uint64_t now = current_timestamp(); + + tmp_fd = mkstemps(file_name, strlen(".pcapng")); + if (tmp_fd == -1) { + perror("mkstemps() failure"); + goto fail; } + printf("pcapng: output file %s\n", file_name); - return 0; + /* open a test capture file */ + pcapng = rte_pcapng_fdopen(tmp_fd, NULL, NULL, "pcapng_test", NULL); + if (pcapng == NULL) { + fprintf(stderr, "rte_pcapng_fdopen failed\n"); + close(tmp_fd); + goto fail; + } + + /* Add interface to the file */ + ret = rte_pcapng_add_interface(pcapng, port_id, + NULL, NULL, NULL); + if (ret < 0) { + fprintf(stderr, "can not add port %u\n", port_id); + goto fail; + } + + count = fill_pcapng_file(pcapng, TOTAL_PACKETS); + if (count < 0) + goto fail; + + /* write a statistics block */ + ret = rte_pcapng_write_stats(pcapng, port_id, + count, 0, "end of test"); + if (ret <= 0) { + fprintf(stderr, "Write of statistics failed\n"); + goto fail; + } + + rte_pcapng_close(pcapng); + + ret = valid_pcapng_file(file_name, now, count); + /* if test fails want to investigate the file */ + if (ret == 0) + unlink(file_name); + + return ret; + +fail: + rte_pcapng_close(pcapng); + return -1; } static void test_cleanup(void) { rte_mempool_free(mp); - - if (pcapng) - rte_pcapng_close(pcapng); - + rte_vdev_uninit(null_dev); } static struct @@ -299,10 +445,8 @@ unit_test_suite test_pcapng_suite = { .teardown = test_cleanup, .suite_name = "Test Pcapng Unit Test Suite", .unit_test_cases = { + TEST_CASE(test_add_interface), TEST_CASE(test_write_packets), - TEST_CASE(test_write_stats), - TEST_CASE(test_validate), - TEST_CASE(test_write_over_limit_iov_max), TEST_CASES_END() } }; @@ -313,4 +457,4 @@ test_pcapng(void) return unit_test_suite_runner(&test_pcapng_suite); } -REGISTER_TEST_COMMAND(pcapng_autotest, test_pcapng); +REGISTER_FAST_TEST(pcapng_autotest, true, true, test_pcapng);