From patchwork Thu Nov 9 19:45:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134031 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 79846432EA; Thu, 9 Nov 2023 20:46:14 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7801B40DC9; Thu, 9 Nov 2023 20:46:10 +0100 (CET) Received: from mail-oi1-f173.google.com (mail-oi1-f173.google.com [209.85.167.173]) by mails.dpdk.org (Postfix) with ESMTP id 37DDD4026B for ; Thu, 9 Nov 2023 20:46:09 +0100 (CET) Received: by mail-oi1-f173.google.com with SMTP id 5614622812f47-3b565722c0eso705147b6e.2 for ; Thu, 09 Nov 2023 11:46:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699559168; x=1700163968; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dzO7hXbQ2jqdDptnVyq6Ah0zFTtMVblIxZyUbi0KLfM=; b=ZyDVDP5RkuFPz0tIjINQfeTyLSYT7uP9zXPjfyQFTDHw0Dv8uiykoZkSBOtZ9uZGaS 5OTwVbO4Hd3bvECugDyQ4RxSULjJrqnZgGdMs+nQx+5fHnDJfZgTc6UYT8JbDpKwVyHK G0ixfJYAZ83aLhX76nVsF1XXwcuH7J2as82wszkSIazlyYJ/SpOpiwuLa1BgTLnjl+wa U/hxhAqzRVV6nzOLc4fpHIB06OogdSz7er4WfPHH19kLRu61i6416TfhPjTD4q/1Lp1U AnzInstpMr+9r0kd3HsP2XMKetOf1ZR6E8aNhUwiAnLpeUg/6gZMngHM0b1nB/U5nowa mHwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699559168; x=1700163968; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dzO7hXbQ2jqdDptnVyq6Ah0zFTtMVblIxZyUbi0KLfM=; b=EHkhYHEEe3NazA7EmedylhC7rne173fkVruYVdjWzn2G5/9ApYZ/dqlFIhgPI1XwTr TZV9WrbprIUzTspdKRIDwtGLr0cI4oghFHI5xP+uc/2ZJ+hooNASR/ASaJsTccL4sftd slvVasvCroCtKE0Zqp4PD6xqwFlVlzqXh3H79+/DQNPOzk086FxhQVS81HFevFsGJsjL px2LDrRvpkxv2Ms4gYVTdPBSAuUKqpYqK4z+wo/7OED5RTzvbHx2Vu0hBt5XoBMTQqUU 9rdoIJF4+mLkSgtcpfTX+Qto5dQ93uxhzqhuI3nJVBN4SnJovgM/RSOl1TAVWK4BJ5pq OPGw== X-Gm-Message-State: AOJu0YxIpoR2tKF95qMcY7MR6v0SLofpUsUCEyJtUnPY1KfjIdsSCItz siERu89MF28R3GHgMQt0FKLjq8Biw7HBw/HA198= X-Google-Smtp-Source: AGHT+IGwVgyrh0+U/yKDCn5G87whYQLIFg1Jnk5u/7JSzgiZ+71wgx0omZjwtWWQ2JDXLTY2AfLvIg== X-Received: by 2002:a05:6808:2096:b0:3b5:ae0d:b208 with SMTP id s22-20020a056808209600b003b5ae0db208mr2985986oiw.6.1699559168095; Thu, 09 Nov 2023 11:46:08 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id j191-20020a6380c8000000b0059b782e8541sm5176101pgd.28.2023.11.09.11.46.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 11:46:07 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , =?utf-8?q?Morten_Br?= =?utf-8?q?=C3=B8rup?= , Reshma Pattan , Jianfeng Tan Subject: [PATCH v5 1/5] pdump: fix setting rte_errno on mp error Date: Thu, 9 Nov 2023 11:45:53 -0800 Message-Id: <20231109194557.111350-2-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109194557.111350-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109194557.111350-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The response from MP server sets err_value to negative on error. The convention for rte_errno is to use a positive value on error. This makes errors like duplicate registration show up with the correct error value. Fixes: 660098d61f57 ("pdump: use generic multi-process channel") Signed-off-by: Stephen Hemminger Acked-by: Morten Brørup --- lib/pdump/rte_pdump.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/pdump/rte_pdump.c b/lib/pdump/rte_pdump.c index 80b90c6f7d03..e94f49e21250 100644 --- a/lib/pdump/rte_pdump.c +++ b/lib/pdump/rte_pdump.c @@ -564,9 +564,10 @@ pdump_prepare_client_request(const char *device, uint16_t queue, if (rte_mp_request_sync(&mp_req, &mp_reply, &ts) == 0) { mp_rep = &mp_reply.msgs[0]; resp = (struct pdump_response *)mp_rep->param; - rte_errno = resp->err_value; - if (!resp->err_value) + if (resp->err_value == 0) ret = 0; + else + rte_errno = -resp->err_value; free(mp_reply.msgs); } From patchwork Thu Nov 9 19:45:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134032 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 69403432EA; Thu, 9 Nov 2023 20:46:20 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 95BBD40F1A; Thu, 9 Nov 2023 20:46:11 +0100 (CET) Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) by mails.dpdk.org (Postfix) with ESMTP id 1015240DC9 for ; Thu, 9 Nov 2023 20:46:10 +0100 (CET) Received: by mail-oi1-f174.google.com with SMTP id 5614622812f47-3b3e13fc1f7so742284b6e.0 for ; Thu, 09 Nov 2023 11:46:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699559169; x=1700163969; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U85fl2qShxBl2MNBjIkUZNLb6jkMS5sOCrwRRpEWPsg=; b=ZTMKCIcwy5/CQJjHzASpoLbAxn5MZJ73afNhUHcvV7EUxOGK7ACbrHRrLpRsDDk3Rr 6AaCnrTwLnRnI4zBnY82NGfFUjYdX962H2obK65AMJH5FE8vg07aE+Me5FLmJOxQn7yD p1YoA0b8u9KjaTb7/ETuSXdyrFZ6AsGPBba0N49aA+pvWnKpGeeMrp7NgXgsWWDRJ3nu IIW4+SN3/xW0AnQyXJTGgFRI97O4CLKO0D94OFh8Pa8JfQiKQ55tpyY5W1FOqy6m0CiP 1Yt0uOdLniQ7o+3fqqkhIWfldOYAMLAoJfI6r8h01D4oZ1W1il9cy4WPz2F9zlDDv6+r r17g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699559169; x=1700163969; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U85fl2qShxBl2MNBjIkUZNLb6jkMS5sOCrwRRpEWPsg=; b=PLD6wP3aPZkwZBRnQkZ/lfm9AO+kKXTwvg+1d0J/4LGUnMgA/S8gczF5YVsxLXRu6T R0E+pnnlS0w1CK1SdXV05QP7SeTB4aWUfJsIFMJeUX1XNlUONVLeiDaiRvS4coMxJEep 8scTQKQNlZllD+JiFL4qKG1GTge12c3AY0u5/1/bjalQHlHQFOSEJoDlrfWZdYUWKnX9 0xSRDnBPYVGXpAXXGO7ctPXgXU93QPK91Lz+VbuaTcytEMDA9et76nig8uF6fUZUdQxJ TQiaioocEn7BxsLWYAejvf1XEvIG9rXNW0a3zAoVVfpH26F8qtxGG+GS0NHsZtMDxrsq z1uQ== X-Gm-Message-State: AOJu0Yyr6X6gLxTheSgOw25oiQoJY12BTnA1Ey5wiwxUUOS4UdMFyNBl VT8K30ao5nAF5PwG6SKrWxQ4mR25EGaMa7gh1Ms= X-Google-Smtp-Source: AGHT+IF2D/KKUNQxApKaACSCsklsZlFIhOBtdae9tWXJwjpDzFjFXrhNZ2T83YSPlE9GZ4E57HxjxQ== X-Received: by 2002:a05:6808:1183:b0:3ae:16aa:8ba9 with SMTP id j3-20020a056808118300b003ae16aa8ba9mr3033139oil.30.1699559169077; Thu, 09 Nov 2023 11:46:09 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id j191-20020a6380c8000000b0059b782e8541sm5176101pgd.28.2023.11.09.11.46.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 11:46:08 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , Isaac Boukris , Reshma Pattan Subject: [PATCH v5 2/5] dumpcap: allow multiple invocations Date: Thu, 9 Nov 2023 11:45:54 -0800 Message-Id: <20231109194557.111350-3-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109194557.111350-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109194557.111350-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org If dumpcap is run twice with each instance pointing a different interface, it would fail because of overlap in ring a pool names. Fix by putting process id in the name. It is still not allowed to do multiple invocations on the same interface because only one callback is allowed and only one copy of mbuf is done. Dumpcap will fail with error in this case: pdump_prepare_client_request(): client request for pdump enable/disable failed EAL: Error - exiting with code: 1 Cause: Packet dump enable on 0:net_null0 failed File exists Fixes: cbb44143be74 ("app/dumpcap: add new packet capture application") Reported-by: Isaac Boukris Signed-off-by: Stephen Hemminger Reviewed-by: Morten Brørup --- app/dumpcap/main.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/app/dumpcap/main.c b/app/dumpcap/main.c index 64294bbfb3e6..efc60372d718 100644 --- a/app/dumpcap/main.c +++ b/app/dumpcap/main.c @@ -44,7 +44,6 @@ #include #include -#define RING_NAME "capture-ring" #define MONITOR_INTERVAL (500 * 1000) #define MBUF_POOL_CACHE_SIZE 32 #define BURST_SIZE 32 @@ -647,6 +646,7 @@ static void dpdk_init(void) static struct rte_ring *create_ring(void) { struct rte_ring *ring; + char ring_name[RTE_RING_NAMESIZE]; size_t size, log2; /* Find next power of 2 >= size. */ @@ -660,28 +660,28 @@ static struct rte_ring *create_ring(void) ring_size = size; } - ring = rte_ring_lookup(RING_NAME); - if (ring == NULL) { - ring = rte_ring_create(RING_NAME, ring_size, - rte_socket_id(), 0); - if (ring == NULL) - rte_exit(EXIT_FAILURE, "Could not create ring :%s\n", - rte_strerror(rte_errno)); - } + /* Want one ring per invocation of program */ + snprintf(ring_name, sizeof(ring_name), + "dumpcap-%d", getpid()); + + ring = rte_ring_create(ring_name, ring_size, + rte_socket_id(), 0); + if (ring == NULL) + rte_exit(EXIT_FAILURE, "Could not create ring :%s\n", + rte_strerror(rte_errno)); + return ring; } static struct rte_mempool *create_mempool(void) { const struct interface *intf; - static const char pool_name[] = "capture_mbufs"; + char pool_name[RTE_MEMPOOL_NAMESIZE]; size_t num_mbufs = 2 * ring_size; struct rte_mempool *mp; uint32_t data_size = 128; - mp = rte_mempool_lookup(pool_name); - if (mp) - return mp; + snprintf(pool_name, sizeof(pool_name), "capture_%d", getpid()); /* Common pool so size mbuf for biggest snap length */ TAILQ_FOREACH(intf, &interfaces, next) { @@ -826,7 +826,7 @@ static void enable_pdump(struct rte_ring *r, struct rte_mempool *mp) rte_exit(EXIT_FAILURE, "Packet dump enable on %u:%s failed %s\n", intf->port, intf->name, - rte_strerror(-ret)); + rte_strerror(rte_errno)); } if (intf->opts.promisc_mode) { From patchwork Thu Nov 9 19:45:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134033 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 567B7432EA; Thu, 9 Nov 2023 20:46:26 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D2805410D5; Thu, 9 Nov 2023 20:46:13 +0100 (CET) Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by mails.dpdk.org (Postfix) with ESMTP id 7B86440EE7 for ; Thu, 9 Nov 2023 20:46:11 +0100 (CET) Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6bd73395bceso1116512b3a.0 for ; Thu, 09 Nov 2023 11:46:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699559170; x=1700163970; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i5MOeIAdsNWL1yGlWWoIbS+scZG2FnE7YE8ftvn8AFk=; b=ANmh3Z3jtkLfuagLenaWKsa3Tbt8t3aKA3XT4GUs6DfOCfZOEVShlo4rYvJ4hHlpIO Ui73X+TazKuGpa6MGzyofGw+oSLMTjyGouPeOnrKcYD2+8qk40h9ffb/cqckQxYIbct3 OBCk+XlC5BAN4+qRl9IrD+nEnrS4FnGCSMI3g2ML7iHV1nuqx0Qnry91zyHdHN+V1Lje kNvCVVr5922WpKsz4kUBOV86ZkK79sMu+8N35uQorxThisWBS24YqMZYkmJOCeP5hN9D Tkv/eGy239Nojs32/UhkiO4cvMR2vgkT8o72dEUUN/9AwmspM//hKMkN3R0n5dLLlmWW uR8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699559170; x=1700163970; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i5MOeIAdsNWL1yGlWWoIbS+scZG2FnE7YE8ftvn8AFk=; b=EF0W2hA/GVDnU1QY0A081nYyinNRp0Gng5I+PIlFdOJaWlboNDCSeqwpMbQ282/H6F K09KnApnpx+NnGE8ngXLsa6x5Ii2/ZjXTzmN+6l/CCVpRrqNIHElaRoLw88lEoUINceL V3zYhJQwHBterp9vBQZ480JpYHiKySqa055Uf6zij3PwfuEgxQBL8BWjwjonOPyYx8U5 dO5ei63+7NK8plFi13yCu2MDWOd3Pe+0c8fONcLwX2aVrXILWkk7Zu0+atgmUkWR9bE/ x8kbMU+Eu14uq6adNo78oo8ejMG++LOPpfHc6XuNUniLSvKylNUJVLGDa2XmJTDS9cYi V5ZQ== X-Gm-Message-State: AOJu0YzcZ5RJWDh1S1+jItHXCvQl7fIOD2yQgpg30ELnd7Athrv0Drb4 XmPShphPEorVvU0KbBZBg61rAYTZ7dSZKC4Q/vw= X-Google-Smtp-Source: AGHT+IE2niMA6M9+rhgvmhKp7urcfp+TtYhJgK/9NUlNUjErprn4Y8dY36LssW/6d9dm16ztgQNYTg== X-Received: by 2002:a05:6a00:8c0c:b0:690:9a5a:e34e with SMTP id ih12-20020a056a008c0c00b006909a5ae34emr369316pfb.12.1699559170146; Thu, 09 Nov 2023 11:46:10 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id j191-20020a6380c8000000b0059b782e8541sm5176101pgd.28.2023.11.09.11.46.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 11:46:09 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , =?utf-8?q?Morten_Br?= =?utf-8?q?=C3=B8rup?= , Reshma Pattan , Jerin Jacob , Kiran Kumar K , Nithin Dabilpuram , Zhirun Yan , Quentin Armitage Subject: [PATCH v5 3/5] pcapng: modify timestamp calculation Date: Thu, 9 Nov 2023 11:45:55 -0800 Message-Id: <20231109194557.111350-4-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109194557.111350-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109194557.111350-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The computation of timestamp is best done in the part of pcapng library that is in secondary process. The secondary process is already doing a bunch of system calls which makes it not performance sensitive. This does change the rte_pcapng_copy() and rte_pcapng_write_stats() experimental API's. Simplify the computation of nanoseconds from TSC to a two step process which avoids numeric overflow issues. The previous code was not thread safe as well. Fixes: c882eb544842 ("pcapng: fix timestamp wrapping in output files") Signed-off-by: Stephen Hemminger Acked-by: Morten Brørup --- app/dumpcap/main.c | 25 +++------ app/test/test_pcapng.c | 4 +- lib/graph/graph_pcap.c | 2 +- lib/pcapng/rte_pcapng.c | 119 +++++++++++++++------------------------- lib/pcapng/rte_pcapng.h | 19 ++----- lib/pdump/rte_pdump.c | 4 +- 6 files changed, 61 insertions(+), 112 deletions(-) diff --git a/app/dumpcap/main.c b/app/dumpcap/main.c index efc60372d718..583bce80166c 100644 --- a/app/dumpcap/main.c +++ b/app/dumpcap/main.c @@ -66,13 +66,13 @@ static bool print_stats; /* capture limit options */ static struct { - uint64_t duration; /* nanoseconds */ + time_t duration; /* seconds */ unsigned long packets; /* number of packets in file */ size_t size; /* file size (bytes) */ } stop; /* Running state */ -static uint64_t start_time, end_time; +static time_t start_time; static uint64_t packets_received; static size_t file_size; @@ -197,7 +197,7 @@ static void auto_stop(char *opt) if (*value == '\0' || *endp != '\0' || interval <= 0) rte_exit(EXIT_FAILURE, "Invalid duration \"%s\"\n", value); - stop.duration = NSEC_PER_SEC * interval; + stop.duration = interval; } else if (strcmp(opt, "filesize") == 0) { stop.size = get_uint(value, "filesize", 0) * 1024; } else if (strcmp(opt, "packets") == 0) { @@ -511,15 +511,6 @@ static void statistics_loop(void) } } -/* Return the time since 1/1/1970 in nanoseconds */ -static uint64_t create_timestamp(void) -{ - struct timespec now; - - clock_gettime(CLOCK_MONOTONIC, &now); - return rte_timespec_to_ns(&now); -} - static void cleanup_pdump_resources(void) { @@ -589,9 +580,8 @@ report_packet_stats(dumpcap_out_t out) ifdrop = pdump_stats.nombuf + pdump_stats.ringfull; if (use_pcapng) - rte_pcapng_write_stats(out.pcapng, intf->port, NULL, - start_time, end_time, - ifrecv, ifdrop); + rte_pcapng_write_stats(out.pcapng, intf->port, + ifrecv, ifdrop, NULL); if (ifrecv == 0) percent = 0; @@ -983,7 +973,7 @@ int main(int argc, char **argv) mp = create_mempool(); out = create_output(); - start_time = create_timestamp(); + start_time = time(NULL); enable_pdump(r, mp); if (!quiet) { @@ -1005,11 +995,10 @@ int main(int argc, char **argv) break; if (stop.duration != 0 && - create_timestamp() - start_time > stop.duration) + time(NULL) - start_time > stop.duration) break; } - end_time = create_timestamp(); disable_primary_monitor(); if (rte_eal_primary_proc_alive(NULL)) diff --git a/app/test/test_pcapng.c b/app/test/test_pcapng.c index b8429a02f160..55aa2cf93666 100644 --- a/app/test/test_pcapng.c +++ b/app/test/test_pcapng.c @@ -173,8 +173,8 @@ test_write_stats(void) ssize_t len; /* write a statistics block */ - len = rte_pcapng_write_stats(pcapng, port_id, - NULL, 0, 0, + len = rte_pcapng_write_stats(pcapng, port_id, NULL, + 0, 0, 0, NUM_PACKETS, 0); if (len <= 0) { fprintf(stderr, "Write of statistics failed\n"); diff --git a/lib/graph/graph_pcap.c b/lib/graph/graph_pcap.c index db722c375fa7..89525f1220ca 100644 --- a/lib/graph/graph_pcap.c +++ b/lib/graph/graph_pcap.c @@ -214,7 +214,7 @@ graph_pcap_dispatch(struct rte_graph *graph, mbuf = (struct rte_mbuf *)objs[i]; mc = rte_pcapng_copy(mbuf->port, 0, mbuf, pkt_mp, mbuf->pkt_len, - rte_get_tsc_cycles(), 0, buffer); + 0, buffer); if (mc == NULL) break; diff --git a/lib/pcapng/rte_pcapng.c b/lib/pcapng/rte_pcapng.c index 3c91fc77644a..13fd2b97fb80 100644 --- a/lib/pcapng/rte_pcapng.c +++ b/lib/pcapng/rte_pcapng.c @@ -36,22 +36,14 @@ /* Format of the capture file handle */ struct rte_pcapng { int outfd; /* output file */ - unsigned int ports; /* number of interfaces added */ + uint64_t offset_ns; /* ns since 1/1/1970 when initialized */ + uint64_t tsc_base; /* TSC when started */ /* DPDK port id to interface index in file */ uint32_t port_index[RTE_MAX_ETHPORTS]; }; -/* For converting TSC cycles to PCAPNG ns format */ -static struct pcapng_time { - uint64_t ns; - uint64_t cycles; - uint64_t tsc_hz; - struct rte_reciprocal_u64 tsc_hz_inverse; -} pcapng_time; - - #ifdef RTE_EXEC_ENV_WINDOWS /* * Windows does not have writev() call. @@ -102,56 +94,21 @@ static ssize_t writev(int fd, const struct iovec *iov, int iovcnt) #define if_indextoname(ifindex, ifname) NULL #endif -static inline void -pcapng_init(void) +/* Convert from TSC (CPU cycles) to nanoseconds */ +static uint64_t +pcapng_timestamp(const rte_pcapng_t *self, uint64_t cycles) { - struct timespec ts; + uint64_t delta, rem, secs, ns; + const uint64_t hz = rte_get_tsc_hz(); - pcapng_time.cycles = rte_get_tsc_cycles(); - clock_gettime(CLOCK_REALTIME, &ts); - pcapng_time.cycles = (pcapng_time.cycles + rte_get_tsc_cycles()) / 2; - pcapng_time.ns = rte_timespec_to_ns(&ts); - - pcapng_time.tsc_hz = rte_get_tsc_hz(); - pcapng_time.tsc_hz_inverse = rte_reciprocal_value_u64(pcapng_time.tsc_hz); -} + delta = cycles - self->tsc_base; -/* PCAPNG timestamps are in nanoseconds */ -static uint64_t pcapng_tsc_to_ns(uint64_t cycles) -{ - uint64_t delta, secs; - - if (!pcapng_time.tsc_hz) - pcapng_init(); - - /* In essence the calculation is: - * delta = (cycles - pcapng_time.cycles) * NSEC_PRE_SEC / rte_get_tsc_hz() - * but this overflows within 4 to 8 seconds depending on TSC frequency. - * Instead, if delta >= pcapng_time.tsc_hz: - * Increase pcapng_time.ns and pcapng_time.cycles by the number of - * whole seconds in delta and reduce delta accordingly. - * delta will therefore always lie in the interval [0, pcapng_time.tsc_hz), - * which will not overflow when multiplied by NSEC_PER_SEC provided the - * TSC frequency < approx 18.4GHz. - * - * Currently all TSCs operate below 5GHz. - */ - delta = cycles - pcapng_time.cycles; - if (unlikely(delta >= pcapng_time.tsc_hz)) { - if (likely(delta < pcapng_time.tsc_hz * 2)) { - delta -= pcapng_time.tsc_hz; - pcapng_time.cycles += pcapng_time.tsc_hz; - pcapng_time.ns += NSEC_PER_SEC; - } else { - secs = rte_reciprocal_divide_u64(delta, &pcapng_time.tsc_hz_inverse); - delta -= secs * pcapng_time.tsc_hz; - pcapng_time.cycles += secs * pcapng_time.tsc_hz; - pcapng_time.ns += secs * NSEC_PER_SEC; - } - } + /* Avoid numeric wraparound by computing seconds first */ + secs = delta / hz; + rem = delta % hz; + ns = (rem * NS_PER_S) / hz; - return pcapng_time.ns + rte_reciprocal_divide_u64(delta * NSEC_PER_SEC, - &pcapng_time.tsc_hz_inverse); + return secs * NS_PER_S + ns + self->offset_ns; } /* length of option including padding */ @@ -368,15 +325,15 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, */ ssize_t rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, - const char *comment, - uint64_t start_time, uint64_t end_time, - uint64_t ifrecv, uint64_t ifdrop) + uint64_t ifrecv, uint64_t ifdrop, + const char *comment) { struct pcapng_statistics *hdr; struct pcapng_option *opt; + uint64_t start_time = self->offset_ns; + uint64_t sample_time; uint32_t optlen, len; uint8_t *buf; - uint64_t ns; RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); @@ -386,10 +343,10 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, optlen += pcapng_optlen(sizeof(ifrecv)); if (ifdrop != UINT64_MAX) optlen += pcapng_optlen(sizeof(ifdrop)); + if (start_time != 0) optlen += pcapng_optlen(sizeof(start_time)); - if (end_time != 0) - optlen += pcapng_optlen(sizeof(end_time)); + if (comment) optlen += pcapng_optlen(strlen(comment)); if (optlen != 0) @@ -409,9 +366,6 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, if (start_time != 0) opt = pcapng_add_option(opt, PCAPNG_ISB_STARTTIME, &start_time, sizeof(start_time)); - if (end_time != 0) - opt = pcapng_add_option(opt, PCAPNG_ISB_ENDTIME, - &end_time, sizeof(end_time)); if (ifrecv != UINT64_MAX) opt = pcapng_add_option(opt, PCAPNG_ISB_IFRECV, &ifrecv, sizeof(ifrecv)); @@ -425,9 +379,9 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, hdr->block_length = len; hdr->interface_id = self->port_index[port_id]; - ns = pcapng_tsc_to_ns(rte_get_tsc_cycles()); - hdr->timestamp_hi = ns >> 32; - hdr->timestamp_lo = (uint32_t)ns; + sample_time = pcapng_timestamp(self, rte_get_tsc_cycles()); + hdr->timestamp_hi = sample_time >> 32; + hdr->timestamp_lo = (uint32_t)sample_time; /* clone block_length after option */ memcpy(opt, &len, sizeof(uint32_t)); @@ -520,23 +474,21 @@ struct rte_mbuf * rte_pcapng_copy(uint16_t port_id, uint32_t queue, const struct rte_mbuf *md, struct rte_mempool *mp, - uint32_t length, uint64_t cycles, + uint32_t length, enum rte_pcapng_direction direction, const char *comment) { struct pcapng_enhance_packet_block *epb; uint32_t orig_len, data_len, padding, flags; struct pcapng_option *opt; + uint64_t timestamp; uint16_t optlen; struct rte_mbuf *mc; - uint64_t ns; bool rss_hash; #ifdef RTE_LIBRTE_ETHDEV_DEBUG RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, NULL); #endif - ns = pcapng_tsc_to_ns(cycles); - orig_len = rte_pktmbuf_pkt_len(md); /* Take snapshot of the data */ @@ -641,8 +593,10 @@ rte_pcapng_copy(uint16_t port_id, uint32_t queue, /* Interface index is filled in later during write */ mc->port = port_id; - epb->timestamp_hi = ns >> 32; - epb->timestamp_lo = (uint32_t)ns; + /* Put timestamp in cycles here - adjust in packet write */ + timestamp = rte_get_tsc_cycles(); + epb->timestamp_hi = timestamp >> 32; + epb->timestamp_lo = (uint32_t)timestamp; epb->capture_length = data_len; epb->original_length = orig_len; @@ -668,6 +622,7 @@ rte_pcapng_write_packets(rte_pcapng_t *self, for (i = 0; i < nb_pkts; i++) { struct rte_mbuf *m = pkts[i]; struct pcapng_enhance_packet_block *epb; + uint64_t cycles, timestamp; /* sanity check that is really a pcapng mbuf */ epb = rte_pktmbuf_mtod(m, struct pcapng_enhance_packet_block *); @@ -684,6 +639,13 @@ rte_pcapng_write_packets(rte_pcapng_t *self, return -1; } + /* adjust timestamp recorded in packet */ + cycles = (uint64_t)epb->timestamp_hi << 32; + cycles += epb->timestamp_lo; + timestamp = pcapng_timestamp(self, cycles); + epb->timestamp_hi = timestamp >> 32; + epb->timestamp_lo = (uint32_t)timestamp; + /* * Handle case of highly fragmented and large burst size * Note: this assumes that max segments per mbuf < IOV_MAX @@ -725,6 +687,8 @@ rte_pcapng_fdopen(int fd, { unsigned int i; rte_pcapng_t *self; + struct timespec ts; + uint64_t cycles; self = malloc(sizeof(*self)); if (!self) { @@ -734,6 +698,13 @@ rte_pcapng_fdopen(int fd, self->outfd = fd; self->ports = 0; + + /* record start time in ns since 1/1/1970 */ + cycles = rte_get_tsc_cycles(); + clock_gettime(CLOCK_REALTIME, &ts); + self->tsc_base = (cycles + rte_get_tsc_cycles()) / 2; + self->offset_ns = rte_timespec_to_ns(&ts); + for (i = 0; i < RTE_MAX_ETHPORTS; i++) self->port_index[i] = UINT32_MAX; diff --git a/lib/pcapng/rte_pcapng.h b/lib/pcapng/rte_pcapng.h index d93cc9f73ad5..c40795c721de 100644 --- a/lib/pcapng/rte_pcapng.h +++ b/lib/pcapng/rte_pcapng.h @@ -121,8 +121,6 @@ enum rte_pcapng_direction { * @param length * The upper limit on bytes to copy. Passing UINT32_MAX * means all data (after offset). - * @param timestamp - * The timestamp in TSC cycles. * @param direction * The direction of the packer: receive, transmit or unknown. * @param comment @@ -136,7 +134,7 @@ __rte_experimental struct rte_mbuf * rte_pcapng_copy(uint16_t port_id, uint32_t queue, const struct rte_mbuf *m, struct rte_mempool *mp, - uint32_t length, uint64_t timestamp, + uint32_t length, enum rte_pcapng_direction direction, const char *comment); @@ -188,29 +186,22 @@ rte_pcapng_write_packets(rte_pcapng_t *self, * The handle to the packet capture file * @param port * The Ethernet port to report stats on. - * @param comment - * Optional comment to add to statistics. - * @param start_time - * The time when packet capture was started in nanoseconds. - * Optional: can be zero if not known. - * @param end_time - * The time when packet capture was stopped in nanoseconds. - * Optional: can be zero if not finished; * @param ifrecv * The number of packets received by capture. * Optional: use UINT64_MAX if not known. * @param ifdrop * The number of packets missed by the capture process. * Optional: use UINT64_MAX if not known. + * @param comment + * Optional comment to add to statistics. * @return * number of bytes written to file, -1 on failure to write file */ __rte_experimental ssize_t rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port, - const char *comment, - uint64_t start_time, uint64_t end_time, - uint64_t ifrecv, uint64_t ifdrop); + uint64_t ifrecv, uint64_t ifdrop, + const char *comment); #ifdef __cplusplus } diff --git a/lib/pdump/rte_pdump.c b/lib/pdump/rte_pdump.c index e94f49e21250..5a1ec14d7a18 100644 --- a/lib/pdump/rte_pdump.c +++ b/lib/pdump/rte_pdump.c @@ -90,7 +90,6 @@ pdump_copy(uint16_t port_id, uint16_t queue, int ring_enq; uint16_t d_pkts = 0; struct rte_mbuf *dup_bufs[nb_pkts]; - uint64_t ts; struct rte_ring *ring; struct rte_mempool *mp; struct rte_mbuf *p; @@ -99,7 +98,6 @@ pdump_copy(uint16_t port_id, uint16_t queue, if (cbs->filter) rte_bpf_exec_burst(cbs->filter, (void **)pkts, rcs, nb_pkts); - ts = rte_get_tsc_cycles(); ring = cbs->ring; mp = cbs->mp; for (i = 0; i < nb_pkts; i++) { @@ -122,7 +120,7 @@ pdump_copy(uint16_t port_id, uint16_t queue, if (cbs->ver == V2) p = rte_pcapng_copy(port_id, queue, pkts[i], mp, cbs->snaplen, - ts, direction, NULL); + direction, NULL); else p = rte_pktmbuf_copy(pkts[i], mp, 0, cbs->snaplen); From patchwork Thu Nov 9 19:45:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134034 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6C0AD432EA; Thu, 9 Nov 2023 20:46:32 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F35C4410E6; Thu, 9 Nov 2023 20:46:14 +0100 (CET) Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) by mails.dpdk.org (Postfix) with ESMTP id 6772140FDE for ; Thu, 9 Nov 2023 20:46:12 +0100 (CET) Received: by mail-oi1-f176.google.com with SMTP id 5614622812f47-3b2ec5ee2e4so721381b6e.3 for ; Thu, 09 Nov 2023 11:46:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699559171; x=1700163971; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N2r/H5v/jMSX7aFlXgmw6wyhS0CLdclJuU10KJ1gaU0=; b=w2jkR+LQGS3Ok2XDK0nf51txb+rTLqIF6L1F5HcftH9lObpMYIZzlG5GgmXnMDVAwn hn/oCCYxtZYkR+FXp5BYOu6yiizwjmaIgt46bRI0U1VCS3ikUuJnhYuIhLDQDj++kDkW hGkdnBau/uNBBKp5XvSGfMN6y7FxUy+Cy7k9hJX6PPcIjm5BMd1TgBHDr5Hqes1QDc+Y yfGFXmG6It/awU6UZVeFBPAkmE+DAvs7kyz2FF0KvHbw6rdxTKXTPcuKQCjSfCpYHYhK nViMZQwfPmXGCXdHqQIsUOMuCGD0QbWzKwFLAEDtQXI5SiGWqSHBaku3kZjGhVQpdCKz xMrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699559171; x=1700163971; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N2r/H5v/jMSX7aFlXgmw6wyhS0CLdclJuU10KJ1gaU0=; b=cusWWGhxomcQBVMjwdBewwZbQ5KVuhWgEdX03Tah8roRxOlTzko+E3p+h3LXOSG4mx UohVoGOOn8W15WrcOojXaLmY/XVkOhJg9N6uQGwnznTZYuBfCULpF0s0cDeLjWC1HHS5 3t3yBN4mJuWOkzDlwfofwJZFuXz2haKHzHAydahF5EhPzebhYDkPcw3PR99EEXzkBS1f d/R/v4kHCEpnPFeUW6axtSB0CtKzDR8nUWbZY/XYlChTVQRLPtbUjSVJAiL94gUXRs/d eNJhuBKqtZPa7CJXnvHEw4hgvLm8dlSeiH435uKVC6P4T7xr6SlZuj1PaDjtUg1L1iaX +mFw== X-Gm-Message-State: AOJu0YyAUXrMaKd7VaK/rmGN/TtS+BmPaaI6mdwwZKqDWBU+G4UffNtx U8dkutbfKW8+74jlCrKC3boIx9nczSj6Tzgq1ow= X-Google-Smtp-Source: AGHT+IFauCYlcgxMWspov9IFoxi1G5lIjcCsttMqllkftbv4cewyLY/hwtaO9AT8YDyi/GG9Np1keA== X-Received: by 2002:a05:6808:292:b0:3a6:febf:e1 with SMTP id z18-20020a056808029200b003a6febf00e1mr2671180oic.55.1699559171489; Thu, 09 Nov 2023 11:46:11 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id j191-20020a6380c8000000b0059b782e8541sm5176101pgd.28.2023.11.09.11.46.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 11:46:10 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , =?utf-8?q?Morten_Br?= =?utf-8?q?=C3=B8rup?= , Reshma Pattan Subject: [PATCH v5 4/5] pcapng: avoid using alloca() Date: Thu, 9 Nov 2023 11:45:56 -0800 Message-Id: <20231109194557.111350-5-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109194557.111350-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109194557.111350-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The function alloca() like VLA's has problems if the caller passes a large value. Instead use a fixed size buffer (2K) which will be more than sufficient for the info related blocks in the file. Add bounds checks as well. Signed-off-by: Stephen Hemminger Acked-by: Morten Brørup --- lib/pcapng/rte_pcapng.c | 37 ++++++++++++++++--------------------- 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/lib/pcapng/rte_pcapng.c b/lib/pcapng/rte_pcapng.c index 13fd2b97fb80..f74ec939a9f8 100644 --- a/lib/pcapng/rte_pcapng.c +++ b/lib/pcapng/rte_pcapng.c @@ -33,6 +33,9 @@ /* conversion from DPDK speed to PCAPNG */ #define PCAPNG_MBPS_SPEED 1000000ull +/* upper bound for section, stats and interface blocks */ +#define PCAPNG_BLKSIZ 2048 + /* Format of the capture file handle */ struct rte_pcapng { int outfd; /* output file */ @@ -140,9 +143,8 @@ pcapng_section_block(rte_pcapng_t *self, { struct pcapng_section_header *hdr; struct pcapng_option *opt; - void *buf; + uint8_t buf[PCAPNG_BLKSIZ]; uint32_t len; - ssize_t cc; len = sizeof(*hdr); if (hw) @@ -158,8 +160,7 @@ pcapng_section_block(rte_pcapng_t *self, len += pcapng_optlen(0); len += sizeof(uint32_t); - buf = calloc(1, len); - if (!buf) + if (len > sizeof(buf)) return -1; hdr = (struct pcapng_section_header *)buf; @@ -193,10 +194,7 @@ pcapng_section_block(rte_pcapng_t *self, /* clone block_length after option */ memcpy(opt, &hdr->block_length, sizeof(uint32_t)); - cc = write(self->outfd, buf, len); - free(buf); - - return cc; + return write(self->outfd, buf, len); } /* Write an interface block for a DPDK port */ @@ -213,7 +211,7 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, struct pcapng_option *opt; const uint8_t tsresol = 9; /* nanosecond resolution */ uint32_t len; - void *buf; + uint8_t buf[PCAPNG_BLKSIZ]; char ifname_buf[IF_NAMESIZE]; char ifhw[256]; uint64_t speed = 0; @@ -267,8 +265,7 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, len += pcapng_optlen(0); len += sizeof(uint32_t); - buf = alloca(len); - if (!buf) + if (len > sizeof(buf)) return -1; hdr = (struct pcapng_interface_block *)buf; @@ -296,17 +293,16 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, opt = pcapng_add_option(opt, PCAPNG_IFB_HARDWARE, ifhw, strlen(ifhw)); if (filter) { - /* Encoding is that the first octet indicates string vs BPF */ size_t len; - char *buf; len = strlen(filter) + 1; - buf = alloca(len); - *buf = '\0'; - memcpy(buf + 1, filter, len); + opt->code = PCAPNG_IFB_FILTER; + opt->length = len; + /* Encoding is that the first octet indicates string vs BPF */ + opt->data[0] = 0; + memcpy(opt->data + 1, filter, strlen(filter)); - opt = pcapng_add_option(opt, PCAPNG_IFB_FILTER, - buf, len); + opt = (struct pcapng_option *)((uint8_t *)opt + pcapng_optlen(len)); } opt = pcapng_add_option(opt, PCAPNG_OPT_END, NULL, 0); @@ -333,7 +329,7 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, uint64_t start_time = self->offset_ns; uint64_t sample_time; uint32_t optlen, len; - uint8_t *buf; + uint8_t buf[PCAPNG_BLKSIZ]; RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); @@ -353,8 +349,7 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, optlen += pcapng_optlen(0); len = sizeof(*hdr) + optlen + sizeof(uint32_t); - buf = alloca(len); - if (buf == NULL) + if (len > sizeof(buf)) return -1; hdr = (struct pcapng_statistics *)buf; From patchwork Thu Nov 9 19:45:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134035 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C5223432EA; Thu, 9 Nov 2023 20:46:40 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 832104113C; Thu, 9 Nov 2023 20:46:16 +0100 (CET) Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by mails.dpdk.org (Postfix) with ESMTP id CC025410D0 for ; Thu, 9 Nov 2023 20:46:13 +0100 (CET) Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-6c39ad730aaso1153617b3a.0 for ; Thu, 09 Nov 2023 11:46:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699559172; x=1700163972; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZxBQOrc3DzlSpoSRdVgX3NKVNdfN6wt2OXsbYKLwbLc=; b=p+pC29TirzIGozZYdpSBtTjeRc25t9iC+nuZwLe8Tiog9GvG4axwmhi1K7RB54sGNT elpLoD5vtKk8wUgb5DEQUsP6tf9W2zTp2AJEWJmGf+ToVJUjEkvnTjYaC2X+SCTzp9LX OidsS6qJ1xn2tNy/T8wM8kLYiLJcpXKKv4Thb5yGVzgLqI0sVHdkrio2JBVSqIaUSmL+ E8KoGk09IkRMkIrX9QdXtkqnOcnDqRF2x2WHkH7bIx+9T5nuleLXpG2k1Hd1yNol1Ca5 hOTt33V+SuxYAf6EcizXyIixXr65QRDVlHC24svhtfCsdGuJardwv7/pLVFNYA01TIu2 KYGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699559172; x=1700163972; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZxBQOrc3DzlSpoSRdVgX3NKVNdfN6wt2OXsbYKLwbLc=; b=wHljtvsYNRJ/BI6sYyDE1OyH4Seg6SqfdnDTrdJUUZ5gaYrCXxw5ZiuQ2psIPONn28 RPOOgGstQTQjX7IHXVnIg1ltYtyTJua2/UiH5K9omibxGulDNDWJ1TG7wKTBn1yWenRn lnmaZg4jbKZBT4Slz+bCPfQ8KIRu5qt2cj0A2MZhBiDS5x6y7wP1PmFinrPVK0FCl9p0 HPi0d1dh5nEZRxagnHwNir5qi1ter7+QuJ1H2Drf0jyScqrX/ceZsJY7jj8WkIizADja zXZsDFWXQwzxcr0I/DAD473ObQvcyHRI2fWTysfnU5U6dBiPV6fRpR3EUeKa4xFk4jBO l1Zg== X-Gm-Message-State: AOJu0YzF9safg/79BVnDyX4611r04myYMk1WgqM48De/qWagfXbeGw5V ZN+ZVjxXNpU6VM5bWtlzUw8xWWP5XT9n7Kl9IjI= X-Google-Smtp-Source: AGHT+IHvP/euqi+52f/0phr96Wruvw+tOgczAYEJ9qByXeDzv2dELo5LoENk7KDz8W3NoCwCJjgWdg== X-Received: by 2002:a05:6a00:1d07:b0:6c4:b182:f3d9 with SMTP id a7-20020a056a001d0700b006c4b182f3d9mr2455474pfx.21.1699559172366; Thu, 09 Nov 2023 11:46:12 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id j191-20020a6380c8000000b0059b782e8541sm5176101pgd.28.2023.11.09.11.46.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 11:46:11 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , Reshma Pattan Subject: [PATCH v5 5/5] test: cleanups to pcapng test Date: Thu, 9 Nov 2023 11:45:57 -0800 Message-Id: <20231109194557.111350-6-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231109194557.111350-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231109194557.111350-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Overhaul of the pcapng test: - promote it to be a fast test so it gets regularly run. - create null device and use i. - use UDP discard packets that are valid so that for debugging the resulting pcapng file can be looked at with wireshark. - do basic checks on resulting pcap file that lengths and timestamps are in range. - add test for interface options Signed-off-by: Stephen Hemminger --- app/test/meson.build | 2 +- app/test/test_pcapng.c | 418 +++++++++++++++++++++++++++-------------- 2 files changed, 282 insertions(+), 138 deletions(-) diff --git a/app/test/meson.build b/app/test/meson.build index 4183d66b0e9c..dcc93f4a43b4 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -128,7 +128,7 @@ source_file_deps = { 'test_metrics.c': ['metrics'], 'test_mp_secondary.c': ['hash', 'lpm'], 'test_net_ether.c': ['net'], - 'test_pcapng.c': ['ethdev', 'net', 'pcapng'], + 'test_pcapng.c': ['ethdev', 'net', 'pcapng', 'bus_vdev'], 'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'], 'test_pdump.c': ['pdump'] + sample_packet_forward_deps, 'test_per_lcore.c': [], diff --git a/app/test/test_pcapng.c b/app/test/test_pcapng.c index 55aa2cf93666..c973aa47d1f8 100644 --- a/app/test/test_pcapng.c +++ b/app/test/test_pcapng.c @@ -6,25 +6,34 @@ #include #include +#include #include #include +#include #include #include #include #include +#include +#include +#include +#include #include #include "test.h" -#define NUM_PACKETS 10 -#define DUMMY_MBUF_NUM 3 +#define PCAPNG_TEST_DEBUG 0 + +#define TOTAL_PACKETS 4096 +#define MAX_BURST 64 +#define MAX_GAP_US 100000 +#define DUMMY_MBUF_NUM 3 -static rte_pcapng_t *pcapng; static struct rte_mempool *mp; static const uint32_t pkt_len = 200; static uint16_t port_id; -static char file_name[] = "/tmp/pcapng_test_XXXXXX.pcapng"; +static const char null_dev[] = "net_null0"; /* first mbuf in the packet, should always be at offset 0 */ struct dummy_mbuf { @@ -61,6 +70,7 @@ mbuf1_prepare(struct dummy_mbuf *dm, uint32_t plen) struct { struct rte_ether_hdr eth; struct rte_ipv4_hdr ip; + struct rte_udp_hdr udp; } pkt = { .eth = { .dst_addr.addr_bytes = "\xff\xff\xff\xff\xff\xff", @@ -68,149 +78,201 @@ mbuf1_prepare(struct dummy_mbuf *dm, uint32_t plen) }, .ip = { .version_ihl = RTE_IPV4_VHL_DEF, - .total_length = rte_cpu_to_be_16(plen), - .time_to_live = IPDEFTTL, - .next_proto_id = IPPROTO_RAW, + .time_to_live = 1, + .next_proto_id = IPPROTO_UDP, .src_addr = rte_cpu_to_be_32(RTE_IPV4_LOOPBACK), .dst_addr = rte_cpu_to_be_32(RTE_IPV4_BROADCAST), - } + }, + .udp = { + .dst_port = rte_cpu_to_be_16(9), /* Discard port */ + }, }; memset(dm, 0, sizeof(*dm)); dummy_mbuf_prep(&dm->mb[0], dm->buf[0], sizeof(dm->buf[0]), plen); rte_eth_random_addr(pkt.eth.src_addr.addr_bytes); - memcpy(rte_pktmbuf_mtod(dm->mb, void *), &pkt, RTE_MIN(sizeof(pkt), plen)); + plen -= sizeof(struct rte_ether_hdr); + + pkt.ip.total_length = rte_cpu_to_be_16(plen); + pkt.ip.hdr_checksum = rte_ipv4_cksum(&pkt.ip); + + plen -= sizeof(struct rte_ipv4_hdr); + pkt.udp.src_port = rte_rand(); + pkt.udp.dgram_len = rte_cpu_to_be_16(plen); + + memcpy(rte_pktmbuf_mtod(dm->mb, void *), &pkt, sizeof(pkt)); } static int test_setup(void) { - int tmp_fd; - - port_id = rte_eth_find_next(0); - if (port_id >= RTE_MAX_ETHPORTS) { - fprintf(stderr, "No valid Ether port\n"); - return -1; - } + port_id = rte_eth_dev_count_avail(); - tmp_fd = mkstemps(file_name, strlen(".pcapng")); - if (tmp_fd == -1) { - perror("mkstemps() failure"); - return -1; - } - printf("pcapng: output file %s\n", file_name); - - /* open a test capture file */ - pcapng = rte_pcapng_fdopen(tmp_fd, NULL, NULL, "pcapng_test", NULL); - if (pcapng == NULL) { - fprintf(stderr, "rte_pcapng_fdopen failed\n"); - close(tmp_fd); - return -1; - } - - /* Add interface to the file */ - if (rte_pcapng_add_interface(pcapng, port_id, - NULL, NULL, NULL) != 0) { - fprintf(stderr, "can not add port %u\n", port_id); - return -1; + /* Make a dummy null device to snoop on */ + if (rte_vdev_init(null_dev, NULL) != 0) { + fprintf(stderr, "Failed to create vdev '%s'\n", null_dev); + goto fail; } /* Make a pool for cloned packets */ - mp = rte_pktmbuf_pool_create_by_ops("pcapng_test_pool", IOV_MAX + NUM_PACKETS, - 0, 0, - rte_pcapng_mbuf_size(pkt_len), + mp = rte_pktmbuf_pool_create_by_ops("pcapng_test_pool", + MAX_BURST, 0, 0, + rte_pcapng_mbuf_size(pkt_len) + 128, SOCKET_ID_ANY, "ring_mp_sc"); if (mp == NULL) { fprintf(stderr, "Cannot create mempool\n"); - return -1; + goto fail; } + return 0; + +fail: + rte_vdev_uninit(null_dev); + rte_mempool_free(mp); + return -1; } static int -test_write_packets(void) +fill_pcapng_file(rte_pcapng_t *pcapng, unsigned int num_packets) { - struct rte_mbuf *orig; - struct rte_mbuf *clones[NUM_PACKETS] = { }; struct dummy_mbuf mbfs; - unsigned int i; + struct rte_mbuf *orig; + unsigned int burst_size; + unsigned int count; ssize_t len; /* make a dummy packet */ mbuf1_prepare(&mbfs, pkt_len); - - /* clone them */ orig = &mbfs.mb[0]; - for (i = 0; i < NUM_PACKETS; i++) { - struct rte_mbuf *mc; - mc = rte_pcapng_copy(port_id, 0, orig, mp, pkt_len, - rte_get_tsc_cycles(), 0, NULL); - if (mc == NULL) { - fprintf(stderr, "Cannot copy packet\n"); + for (count = 0; count < num_packets; count += burst_size) { + struct rte_mbuf *clones[MAX_BURST]; + unsigned int i; + + /* put 1 .. MAX_BURST packets in one write call */ + burst_size = rte_rand_max(MAX_BURST) + 1; + for (i = 0; i < burst_size; i++) { + struct rte_mbuf *mc; + + mc = rte_pcapng_copy(port_id, 0, orig, mp, pkt_len, + RTE_PCAPNG_DIRECTION_IN, + NULL); + if (mc == NULL) { + fprintf(stderr, "Cannot copy packet\n"); + return -1; + } + clones[i] = mc; + } + + /* write it to capture file */ + len = rte_pcapng_write_packets(pcapng, clones, burst_size); + rte_pktmbuf_free_bulk(clones, burst_size); + + if (len <= 0) { + fprintf(stderr, "Write of packets failed: %s\n", + rte_strerror(rte_errno)); return -1; } - clones[i] = mc; + + /* Leave a small gap between packets to test for time wrap */ + usleep(rte_rand_max(MAX_GAP_US)); } - /* write it to capture file */ - len = rte_pcapng_write_packets(pcapng, clones, NUM_PACKETS); + return count; +} - rte_pktmbuf_free_bulk(clones, NUM_PACKETS); +static char * +fmt_time(char *buf, size_t size, uint64_t ts_ns) +{ + time_t sec; + size_t len; - if (len <= 0) { - fprintf(stderr, "Write of packets failed\n"); - return -1; - } + sec = ts_ns / NS_PER_S; + len = strftime(buf, size, "%X", localtime(&sec)); + snprintf(buf + len, size - len, ".%09lu", + (unsigned long)(ts_ns % NS_PER_S)); - return 0; + return buf; } -static int -test_write_stats(void) +/* Context for the pcap_loop callback */ +struct pkt_print_ctx { + pcap_t *pcap; + unsigned int count; + uint64_t start_ns; + uint64_t end_ns; +}; + +static void +print_packet(uint64_t ts_ns, const struct rte_ether_hdr *eh, size_t len) { - ssize_t len; + char tbuf[128], src[64], dst[64]; - /* write a statistics block */ - len = rte_pcapng_write_stats(pcapng, port_id, NULL, - 0, 0, 0, - NUM_PACKETS, 0); - if (len <= 0) { - fprintf(stderr, "Write of statistics failed\n"); - return -1; - } - return 0; + fmt_time(tbuf, sizeof(tbuf), ts_ns); + rte_ether_format_addr(dst, sizeof(dst), &eh->dst_addr); + rte_ether_format_addr(src, sizeof(src), &eh->src_addr); + printf("%s: %s -> %s type %x length %zu\n", + tbuf, src, dst, rte_be_to_cpu_16(eh->ether_type), len); } +/* Callback from pcap_loop used to validate packets in the file */ static void -pkt_print(u_char *user, const struct pcap_pkthdr *h, - const u_char *bytes) +parse_pcap_packet(u_char *user, const struct pcap_pkthdr *h, + const u_char *bytes) { - unsigned int *countp = (unsigned int *)user; + struct pkt_print_ctx *ctx = (struct pkt_print_ctx *)user; const struct rte_ether_hdr *eh; - struct tm *tm; - char tbuf[128], src[64], dst[64]; + const struct rte_ipv4_hdr *ip; + uint64_t ns; - tm = localtime(&h->ts.tv_sec); - if (tm == NULL) { - perror("localtime"); - return; + eh = (const struct rte_ether_hdr *)bytes; + ip = (const struct rte_ipv4_hdr *)(eh + 1); + + ctx->count += 1; + + /* The pcap library is misleading in reporting timestamp. + * packet header struct gives timestamp as a timeval (ie. usec); + * but the file is open in nanonsecond mode therefore + * the timestamp is really in timespec (ie. nanoseconds). + */ + ns = h->ts.tv_sec * NS_PER_S + h->ts.tv_usec; + if (ns < ctx->start_ns || ns > ctx->end_ns) { + char tstart[128], tend[128]; + + fmt_time(tstart, sizeof(tstart), ctx->start_ns); + fmt_time(tend, sizeof(tend), ctx->end_ns); + fprintf(stderr, "Timestamp out of range [%s .. %s]\n", + tstart, tend); + goto error; } - if (strftime(tbuf, sizeof(tbuf), "%X", tm) == 0) { - fprintf(stderr, "strftime returned 0!\n"); - return; + if (!rte_is_broadcast_ether_addr(&eh->dst_addr)) { + fprintf(stderr, "Destination is not broadcast\n"); + goto error; } - eh = (const struct rte_ether_hdr *)bytes; - rte_ether_format_addr(dst, sizeof(dst), &eh->dst_addr); - rte_ether_format_addr(src, sizeof(src), &eh->src_addr); - printf("%s.%06lu: %s -> %s type %x length %u\n", - tbuf, (unsigned long)h->ts.tv_usec, - src, dst, rte_be_to_cpu_16(eh->ether_type), h->len); + if (rte_ipv4_cksum(ip) != 0) { + fprintf(stderr, "Bad IPv4 checksum\n"); + goto error; + } + + return; /* packet is normal */ + +error: + print_packet(ns, eh, h->len); + + /* Stop parsing at first error */ + pcap_breakloop(ctx->pcap); +} - *countp += 1; +static uint64_t +current_timestamp(void) +{ + struct timespec ts; + + clock_gettime(CLOCK_REALTIME, &ts); + return rte_timespec_to_ns(&ts); } /* @@ -219,78 +281,162 @@ pkt_print(u_char *user, const struct pcap_pkthdr *h, * but that creates an unwanted dependency. */ static int -test_validate(void) +valid_pcapng_file(const char *file_name, uint64_t started, unsigned int expected) { char errbuf[PCAP_ERRBUF_SIZE]; - unsigned int count = 0; - pcap_t *pcap; + struct pkt_print_ctx ctx = { }; int ret; - pcap = pcap_open_offline(file_name, errbuf); - if (pcap == NULL) { + ctx.start_ns = started; + ctx.end_ns = current_timestamp(); + + ctx.pcap = pcap_open_offline_with_tstamp_precision(file_name, + PCAP_TSTAMP_PRECISION_NANO, + errbuf); + if (ctx.pcap == NULL) { fprintf(stderr, "pcap_open_offline('%s') failed: %s\n", file_name, errbuf); return -1; } - ret = pcap_loop(pcap, 0, pkt_print, (u_char *)&count); - if (ret == 0) - printf("Saw %u packets\n", count); - else + ret = pcap_loop(ctx.pcap, 0, parse_pcap_packet, (u_char *)&ctx); + if (ret != 0) { fprintf(stderr, "pcap_dispatch: failed: %s\n", - pcap_geterr(pcap)); - pcap_close(pcap); + pcap_geterr(ctx.pcap)); + } else if (ctx.count != expected) { + printf("Only %u packets, expected %u\n", + ctx.count, expected); + ret = -1; + } + + pcap_close(ctx.pcap); return ret; } static int -test_write_over_limit_iov_max(void) +test_add_interface(void) { - struct rte_mbuf *orig; - struct rte_mbuf *clones[IOV_MAX + NUM_PACKETS] = { }; - struct dummy_mbuf mbfs; - unsigned int i; - ssize_t len; + char file_name[] = "/tmp/pcapng_test_XXXXXX.pcapng"; + static rte_pcapng_t *pcapng; + int ret, tmp_fd; + uint64_t now = current_timestamp(); - /* make a dummy packet */ - mbuf1_prepare(&mbfs, pkt_len); + tmp_fd = mkstemps(file_name, strlen(".pcapng")); + if (tmp_fd == -1) { + perror("mkstemps() failure"); + goto fail; + } + printf("pcapng: output file %s\n", file_name); - /* clone them */ - orig = &mbfs.mb[0]; - for (i = 0; i < IOV_MAX + NUM_PACKETS; i++) { - struct rte_mbuf *mc; + /* open a test capture file */ + pcapng = rte_pcapng_fdopen(tmp_fd, NULL, NULL, "pcapng_addif", NULL); + if (pcapng == NULL) { + fprintf(stderr, "rte_pcapng_fdopen failed\n"); + close(tmp_fd); + goto fail; + } - mc = rte_pcapng_copy(port_id, 0, orig, mp, pkt_len, - rte_get_tsc_cycles(), 0, NULL); - if (mc == NULL) { - fprintf(stderr, "Cannot copy packet\n"); - return -1; - } - clones[i] = mc; + /* Add interface to the file */ + ret = rte_pcapng_add_interface(pcapng, port_id, + NULL, NULL, NULL); + if (ret < 0) { + fprintf(stderr, "can not add port %u\n", port_id); + goto fail; } - /* write it to capture file */ - len = rte_pcapng_write_packets(pcapng, clones, IOV_MAX + NUM_PACKETS); + /* Add interface with ifname and ifdescr */ + ret = rte_pcapng_add_interface(pcapng, port_id, + "myeth", "Some long description", NULL); + if (ret < 0) { + fprintf(stderr, "can not add port %u with ifname\n", port_id); + goto fail; + } + + /* Add interface with filter */ + ret = rte_pcapng_add_interface(pcapng, port_id, + NULL, NULL, "tcp port 8080"); + if (ret < 0) { + fprintf(stderr, "can not add port %u with filter\n", port_id); + goto fail; + } - rte_pktmbuf_free_bulk(clones, IOV_MAX + NUM_PACKETS); + rte_pcapng_close(pcapng); - if (len <= 0) { - fprintf(stderr, "Write of packets failed\n"); - return -1; + ret = valid_pcapng_file(file_name, now, 0); + /* if test fails want to investigate the file */ + if (ret == 0) + unlink(file_name); + + return ret; + +fail: + rte_pcapng_close(pcapng); + return -1; +} + +static int +test_write_packets(void) +{ + char file_name[] = "/tmp/pcapng_test_XXXXXX.pcapng"; + static rte_pcapng_t *pcapng; + int ret, tmp_fd, count; + uint64_t now = current_timestamp(); + + tmp_fd = mkstemps(file_name, strlen(".pcapng")); + if (tmp_fd == -1) { + perror("mkstemps() failure"); + goto fail; } + printf("pcapng: output file %s\n", file_name); - return 0; + /* open a test capture file */ + pcapng = rte_pcapng_fdopen(tmp_fd, NULL, NULL, "pcapng_test", NULL); + if (pcapng == NULL) { + fprintf(stderr, "rte_pcapng_fdopen failed\n"); + close(tmp_fd); + goto fail; + } + + /* Add interface to the file */ + ret = rte_pcapng_add_interface(pcapng, port_id, + NULL, NULL, NULL); + if (ret < 0) { + fprintf(stderr, "can not add port %u\n", port_id); + goto fail; + } + + count = fill_pcapng_file(pcapng, TOTAL_PACKETS); + if (count < 0) + goto fail; + + /* write a statistics block */ + ret = rte_pcapng_write_stats(pcapng, port_id, + count, 0, "end of test"); + if (ret <= 0) { + fprintf(stderr, "Write of statistics failed\n"); + goto fail; + } + + rte_pcapng_close(pcapng); + + ret = valid_pcapng_file(file_name, now, count); + /* if test fails want to investigate the file */ + if (ret == 0) + unlink(file_name); + + return ret; + +fail: + rte_pcapng_close(pcapng); + return -1; } static void test_cleanup(void) { rte_mempool_free(mp); - - if (pcapng) - rte_pcapng_close(pcapng); - + rte_vdev_uninit(null_dev); } static struct @@ -299,10 +445,8 @@ unit_test_suite test_pcapng_suite = { .teardown = test_cleanup, .suite_name = "Test Pcapng Unit Test Suite", .unit_test_cases = { + TEST_CASE(test_add_interface), TEST_CASE(test_write_packets), - TEST_CASE(test_write_stats), - TEST_CASE(test_validate), - TEST_CASE(test_write_over_limit_iov_max), TEST_CASES_END() } }; @@ -313,4 +457,4 @@ test_pcapng(void) return unit_test_suite_runner(&test_pcapng_suite); } -REGISTER_TEST_COMMAND(pcapng_autotest, test_pcapng); +REGISTER_FAST_TEST(pcapng_autotest, true, true, test_pcapng);