From patchwork Wed Nov 8 18:35:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 133998 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 312C7432D9; Wed, 8 Nov 2023 19:36:19 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DC38342D80; Wed, 8 Nov 2023 19:36:11 +0100 (CET) Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by mails.dpdk.org (Postfix) with ESMTP id 52044402A7 for ; Wed, 8 Nov 2023 19:36:10 +0100 (CET) Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2802b744e52so881652a91.0 for ; Wed, 08 Nov 2023 10:36:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699468569; x=1700073369; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rYJvVipozp0Uok+XahB6RG6LMWbULBCJYi4hvG4xsnU=; b=xnSsfwXvGKezDL7Qt3VUBdJQAk1LNbkw+i+qswUA1h8UHAs4o8bgiKvUSMieBQaawQ AAjF+BMlmOpF1JEjwxjLuyDJ/VfxefEhGuRwh/7cD6wP8X8pU+5OOm46ofzgGiwvrsYe 8ZQcMnfWZK8zkpVfyzxGRn3TVJ4WWZvhFaFxeWDILSpTPhodJz6IUa++wNUiNBzxCKC6 TCMfptYUfMnJAKw7UiPGj/6qLPhcR3dIKyqjG5ZxPx77HxBx6jJPMjXyAs1+XG+bY1po m+2VxxiJVscOPzoZalVSuGkpfGwKRQcPVvfEPKJxMUpf2/nRIH4TlKAqzLKcdGlibRQr 7qkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699468569; x=1700073369; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rYJvVipozp0Uok+XahB6RG6LMWbULBCJYi4hvG4xsnU=; b=i7DZ3Asg/uqhLQTlkhoKmlnd+19CpXKaTxeHUoJFDu8bK8Y6uTRJAPuJdQeyA5TcfC gyrLY3e7KAVQwyBrUFH/ld4ZEeLY9a15K3olx/+jytp7NTSDfhUEFoOyz6hmezGNWumS vSs0FztERF6+N+Un7YW7mGNWhXieuZWyBY0Gj6AhWIGp3L5/Nn6kesXTNvW30hdQmWkE eVH8sjRmtWF+jN01F5kjaTO76ZADGqT/TnBMSIVH8EbCIMKlGN8G1tgGq7G+piVy+1Hg iyDE9s08r4oLsQYZAWuwksQe3Yqf50HdxvJ6PaGYcsK8E/Jy4temIEeiC4ztB9dRpWfS +gXQ== X-Gm-Message-State: AOJu0Yy9KtsxR0shgeVxvwG5YfERH7Ny9OcT2FoHg6xBUeNnCwu5gyEe rccVlMvI7UxlfFyXPD8f5aXwcVcEMIQdfz3JxRw= X-Google-Smtp-Source: AGHT+IE2/TFev2ubgp6d8YVJbWVmFGOlkyVpk0WuIWfSide3t4LcYayPtTd/Y49JHaWdAs78cpxiAQ== X-Received: by 2002:a17:90a:7003:b0:280:a4a1:5d03 with SMTP id f3-20020a17090a700300b00280a4a15d03mr9626928pjk.4.1699468568963; Wed, 08 Nov 2023 10:36:08 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id p25-20020a635b19000000b005b944b20f34sm3402522pgb.85.2023.11.08.10.36.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Nov 2023 10:36:08 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 1/5] pdump: fix setting rte_errno on mp error Date: Wed, 8 Nov 2023 10:35:53 -0800 Message-Id: <20231108183557.381955-2-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231108183557.381955-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231108183557.381955-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The response from MP server sets err_value to negative on error. The convention for rte_errno is to use a positive value on error. This makes errors like duplicate registration show up with the correct error value. Fixes: 660098d61f57 ("pdump: use generic multi-process channel") Signed-off-by: Stephen Hemminger Acked-by: Morten Brørup --- lib/pdump/rte_pdump.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/pdump/rte_pdump.c b/lib/pdump/rte_pdump.c index 80b90c6f7d03..e94f49e21250 100644 --- a/lib/pdump/rte_pdump.c +++ b/lib/pdump/rte_pdump.c @@ -564,9 +564,10 @@ pdump_prepare_client_request(const char *device, uint16_t queue, if (rte_mp_request_sync(&mp_req, &mp_reply, &ts) == 0) { mp_rep = &mp_reply.msgs[0]; resp = (struct pdump_response *)mp_rep->param; - rte_errno = resp->err_value; - if (!resp->err_value) + if (resp->err_value == 0) ret = 0; + else + rte_errno = -resp->err_value; free(mp_reply.msgs); } From patchwork Wed Nov 8 18:35:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 133999 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9280F432D9; Wed, 8 Nov 2023 19:36:26 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 225D3427DB; Wed, 8 Nov 2023 19:36:14 +0100 (CET) Received: from mail-oo1-f49.google.com (mail-oo1-f49.google.com [209.85.161.49]) by mails.dpdk.org (Postfix) with ESMTP id 6380941133 for ; Wed, 8 Nov 2023 19:36:11 +0100 (CET) Received: by mail-oo1-f49.google.com with SMTP id 006d021491bc7-586ba7cdb6bso3903631eaf.2 for ; Wed, 08 Nov 2023 10:36:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699468570; x=1700073370; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=omUtTH7y7VOw30b39TLXxrqp0tKrYGDXycHE1fkYinQ=; b=MwZXWJ1SPo8Oxd0YfMOpYOdcpcqVwsOQnJJwqXOXl2ITA+XD8D22RGphhb+z9C3IsX 0y9Ahp9REP2qjfrfCxP+MHFMxBIM35Y4aRYMrkRRMNzCqu5KMIitWzGxrezu5s7MDTOR ghRV3qnP8F81CRFUtsdlDfW2By2jNB4PACkOjW4y9Cl6fDyBbAlWy24UBxzz1YdAK13s DUaqnKqYD4NdqemU2RSqAtQ4M9CZgjVp5/7OQ8OQN9Bd3DXMoiB4gjJDRMwq3RFI3J84 5hZh/zISN6LtNOYMfQPQ1HzqscQP/+rsw8DBGgnF/HJwhNPfhgOOW1j0rAQxjvE6YpKk Ugsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699468570; x=1700073370; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=omUtTH7y7VOw30b39TLXxrqp0tKrYGDXycHE1fkYinQ=; b=KvlL06njj0V4wZdyoC0Bzn3M3alyy1twESDITNFWI6EjKre/GZT4V7p4bfSWJKKyB8 YPuctvvC7AswOe2sfhql9fRbsxWJNNcnOd8uQCj+UaRxGs3vrwhZcJB2u4q+IQnoXIQd qre/yu7nRl505Wg2+G7t0p7bfOZF2WZw3qvZBpRPEzW/AeQpYuwUCcEplpNfsQIHL8xv R2OJ8FxKBgbuvEGidAulmKf15yxWG9Et4/LCjd2IUjWmqxW1Q+YEUp1soEqaJyOWXrd2 OsYlPJzey4CB4+kZ0IGbujCiisGYSPJk1ow5NBwKxi+50duJpNtIwH4hil1BsvwBa6xt ULLA== X-Gm-Message-State: AOJu0Yxttf+FLhGqIRxlCegJh2CWmzMUSDDO0UpSlPinTivbMelYbxKj SF35VajlmNvMR0Az9V/FMD7Sf3VhYaVR3ILW3ok= X-Google-Smtp-Source: AGHT+IGNFcqWmts0rv5nxqbzpR6TJ51U0GqJZPVW36tl0GPKgpquwSBEYmlrUPj1fGffuOnJIppHzA== X-Received: by 2002:a05:6358:3a1b:b0:168:e69c:66f5 with SMTP id g27-20020a0563583a1b00b00168e69c66f5mr2201069rwe.15.1699468570241; Wed, 08 Nov 2023 10:36:10 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id p25-20020a635b19000000b005b944b20f34sm3402522pgb.85.2023.11.08.10.36.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Nov 2023 10:36:09 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , Isaac Boukris Subject: [PATCH v3 2/5] dumpcap: allow multiple invocations Date: Wed, 8 Nov 2023 10:35:54 -0800 Message-Id: <20231108183557.381955-3-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231108183557.381955-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231108183557.381955-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org If dumpcap is run twice with each instance pointing a different interface, it would fail because of overlap in ring a pool names. Fix by putting process id in the name. Fixes: cbb44143be74 ("app/dumpcap: add new packet capture application") Reported-by: Isaac Boukris Signed-off-by: Stephen Hemminger Reviewed-by: Morten Brørup --- app/dumpcap/main.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/app/dumpcap/main.c b/app/dumpcap/main.c index 64294bbfb3e6..37754fd06f4f 100644 --- a/app/dumpcap/main.c +++ b/app/dumpcap/main.c @@ -44,7 +44,6 @@ #include #include -#define RING_NAME "capture-ring" #define MONITOR_INTERVAL (500 * 1000) #define MBUF_POOL_CACHE_SIZE 32 #define BURST_SIZE 32 @@ -647,6 +646,7 @@ static void dpdk_init(void) static struct rte_ring *create_ring(void) { struct rte_ring *ring; + char ring_name[RTE_RING_NAMESIZE]; size_t size, log2; /* Find next power of 2 >= size. */ @@ -660,28 +660,28 @@ static struct rte_ring *create_ring(void) ring_size = size; } - ring = rte_ring_lookup(RING_NAME); - if (ring == NULL) { - ring = rte_ring_create(RING_NAME, ring_size, - rte_socket_id(), 0); - if (ring == NULL) - rte_exit(EXIT_FAILURE, "Could not create ring :%s\n", - rte_strerror(rte_errno)); - } + /* Want one ring per invocation of program */ + snprintf(ring_name, sizeof(ring_name), + "dumpcap-%u", getpid()); + + ring = rte_ring_create(ring_name, ring_size, + rte_socket_id(), 0); + if (ring == NULL) + rte_exit(EXIT_FAILURE, "Could not create ring :%s\n", + rte_strerror(rte_errno)); + return ring; } static struct rte_mempool *create_mempool(void) { const struct interface *intf; - static const char pool_name[] = "capture_mbufs"; + char pool_name[RTE_MEMPOOL_NAMESIZE]; size_t num_mbufs = 2 * ring_size; struct rte_mempool *mp; uint32_t data_size = 128; - mp = rte_mempool_lookup(pool_name); - if (mp) - return mp; + snprintf(pool_name, sizeof(pool_name), "capture_%u", getpid()); /* Common pool so size mbuf for biggest snap length */ TAILQ_FOREACH(intf, &interfaces, next) { @@ -826,7 +826,7 @@ static void enable_pdump(struct rte_ring *r, struct rte_mempool *mp) rte_exit(EXIT_FAILURE, "Packet dump enable on %u:%s failed %s\n", intf->port, intf->name, - rte_strerror(-ret)); + rte_strerror(rte_errno)); } if (intf->opts.promisc_mode) { From patchwork Wed Nov 8 18:35:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134000 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1C6B6432D9; Wed, 8 Nov 2023 19:36:34 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4CC3542DB4; Wed, 8 Nov 2023 19:36:15 +0100 (CET) Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by mails.dpdk.org (Postfix) with ESMTP id A6ECF42D97 for ; Wed, 8 Nov 2023 19:36:12 +0100 (CET) Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-6c320a821c4so6410889b3a.2 for ; Wed, 08 Nov 2023 10:36:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699468571; x=1700073371; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZiCUYLYm7oTmeqbelviwZ3U94Lx9nxBTNPSXBrfe+0U=; b=g0xwuzZxDFe42pMXZh9P4Hgr7QxAUwkZOKKiV/+AUPMcihwHEppyxp3E+z7ZROuyB/ XSm0Jn2bO7bZtj7WjF02L5h58m3/31yBHiZNVVlXZy/AYA0VNsM1M+Haw01qqZ00eC3U 8Wxl9vxnOso2HobO8zd/P9ZyZX1BsqK26LXPpkiyRCuDlGcuu342sS/TaManX0010SbS KCnJYFjLIDTbtdLiF9wgX4bcYNyd/Lhr9JmHZTAno+74XclLJreMzUeoDkzzOqnDnW3q FNg8NGmuotxORgGGGChO/F2Mzo/xPxX2dbfLYsDiTvwZA3z+S5KIxSnwO1fNc52Pqqn/ EwrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699468571; x=1700073371; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZiCUYLYm7oTmeqbelviwZ3U94Lx9nxBTNPSXBrfe+0U=; b=T3FCy7RqDBouhAAEOzVBfeUjjnMsJVCf41+Ckv9M2OGKdG5/0t4tJCWbWXx9SVYuEv xSTu3NJFCS83eKqeIqm17FS7Xb3MWruqYfatFJLp5coog4FsQ2CmRcBv48tNOtnFB5Hp AadFy2wdvNeTFhZzJG8Vi+MmjDck0ISwcBVdQASthMrBBg9WF/QwjB69VRRK5CXjdHUh B+jKU5XBsA+GDIXPtVQPc0u+t1sQvJhy1d2Xc0ccX9AKxPsoekYJuONqQUUTbMCQKomJ x+Cf8EUybBTDKrLgPyy7BzAPpsxRzyzO8ZDMh9MgSdEyYOSweESptoRnZ/IBOZvZx98o nwhA== X-Gm-Message-State: AOJu0YyKaeEIyAx3KuGp4XEQsNqNtx3Rrg45hrIzMLAC1CFnUWqkL3lO SX9fLc/nrjnZuOHPNoUeioznCBk3U41sCN7CK+M= X-Google-Smtp-Source: AGHT+IEhQOqyPW0a215EFcxuafzbL98wJyaOFFlgFL/52GmKFSm7lta0Ms2+Kr5pMFaRTI5phuC9Uw== X-Received: by 2002:a05:6a21:18d:b0:181:1d71:7e27 with SMTP id le13-20020a056a21018d00b001811d717e27mr3331062pzb.43.1699468571375; Wed, 08 Nov 2023 10:36:11 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id p25-20020a635b19000000b005b944b20f34sm3402522pgb.85.2023.11.08.10.36.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Nov 2023 10:36:10 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 3/5] pcapng: modify timestamp calculation Date: Wed, 8 Nov 2023 10:35:55 -0800 Message-Id: <20231108183557.381955-4-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231108183557.381955-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231108183557.381955-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The computation of timestamp is best done in the part of pcapng library that is in secondary process. The secondary process is already doing a bunch of system calls which makes it not performance sensitive. Simplify the computation of nanoseconds from TSC to a two step process which avoids numeric overflow issues. The previous code was not thread safe as well. Fixes: c882eb544842 ("pcapng: fix timestamp wrapping in output files") Signed-off-by: Stephen Hemminger Acked-by: Morten Brørup --- app/dumpcap/main.c | 25 +++------ app/test/test_pcapng.c | 4 +- lib/graph/graph_pcap.c | 2 +- lib/pcapng/rte_pcapng.c | 119 +++++++++++++++------------------------- lib/pcapng/rte_pcapng.h | 19 ++----- lib/pdump/rte_pdump.c | 4 +- 6 files changed, 61 insertions(+), 112 deletions(-) diff --git a/app/dumpcap/main.c b/app/dumpcap/main.c index 37754fd06f4f..764dac6c37c0 100644 --- a/app/dumpcap/main.c +++ b/app/dumpcap/main.c @@ -66,13 +66,13 @@ static bool print_stats; /* capture limit options */ static struct { - uint64_t duration; /* nanoseconds */ + time_t duration; /* seconds */ unsigned long packets; /* number of packets in file */ size_t size; /* file size (bytes) */ } stop; /* Running state */ -static uint64_t start_time, end_time; +static time_t start_time; static uint64_t packets_received; static size_t file_size; @@ -197,7 +197,7 @@ static void auto_stop(char *opt) if (*value == '\0' || *endp != '\0' || interval <= 0) rte_exit(EXIT_FAILURE, "Invalid duration \"%s\"\n", value); - stop.duration = NSEC_PER_SEC * interval; + stop.duration = interval; } else if (strcmp(opt, "filesize") == 0) { stop.size = get_uint(value, "filesize", 0) * 1024; } else if (strcmp(opt, "packets") == 0) { @@ -511,15 +511,6 @@ static void statistics_loop(void) } } -/* Return the time since 1/1/1970 in nanoseconds */ -static uint64_t create_timestamp(void) -{ - struct timespec now; - - clock_gettime(CLOCK_MONOTONIC, &now); - return rte_timespec_to_ns(&now); -} - static void cleanup_pdump_resources(void) { @@ -589,9 +580,8 @@ report_packet_stats(dumpcap_out_t out) ifdrop = pdump_stats.nombuf + pdump_stats.ringfull; if (use_pcapng) - rte_pcapng_write_stats(out.pcapng, intf->port, NULL, - start_time, end_time, - ifrecv, ifdrop); + rte_pcapng_write_stats(out.pcapng, intf->port, + ifrecv, ifdrop, NULL); if (ifrecv == 0) percent = 0; @@ -983,7 +973,7 @@ int main(int argc, char **argv) mp = create_mempool(); out = create_output(); - start_time = create_timestamp(); + start_time = time(NULL); enable_pdump(r, mp); if (!quiet) { @@ -1005,11 +995,10 @@ int main(int argc, char **argv) break; if (stop.duration != 0 && - create_timestamp() - start_time > stop.duration) + time(NULL) - start_time > stop.duration) break; } - end_time = create_timestamp(); disable_primary_monitor(); if (rte_eal_primary_proc_alive(NULL)) diff --git a/app/test/test_pcapng.c b/app/test/test_pcapng.c index b8429a02f160..55aa2cf93666 100644 --- a/app/test/test_pcapng.c +++ b/app/test/test_pcapng.c @@ -173,8 +173,8 @@ test_write_stats(void) ssize_t len; /* write a statistics block */ - len = rte_pcapng_write_stats(pcapng, port_id, - NULL, 0, 0, + len = rte_pcapng_write_stats(pcapng, port_id, NULL, + 0, 0, 0, NUM_PACKETS, 0); if (len <= 0) { fprintf(stderr, "Write of statistics failed\n"); diff --git a/lib/graph/graph_pcap.c b/lib/graph/graph_pcap.c index db722c375fa7..89525f1220ca 100644 --- a/lib/graph/graph_pcap.c +++ b/lib/graph/graph_pcap.c @@ -214,7 +214,7 @@ graph_pcap_dispatch(struct rte_graph *graph, mbuf = (struct rte_mbuf *)objs[i]; mc = rte_pcapng_copy(mbuf->port, 0, mbuf, pkt_mp, mbuf->pkt_len, - rte_get_tsc_cycles(), 0, buffer); + 0, buffer); if (mc == NULL) break; diff --git a/lib/pcapng/rte_pcapng.c b/lib/pcapng/rte_pcapng.c index 3c91fc77644a..13fd2b97fb80 100644 --- a/lib/pcapng/rte_pcapng.c +++ b/lib/pcapng/rte_pcapng.c @@ -36,22 +36,14 @@ /* Format of the capture file handle */ struct rte_pcapng { int outfd; /* output file */ - unsigned int ports; /* number of interfaces added */ + uint64_t offset_ns; /* ns since 1/1/1970 when initialized */ + uint64_t tsc_base; /* TSC when started */ /* DPDK port id to interface index in file */ uint32_t port_index[RTE_MAX_ETHPORTS]; }; -/* For converting TSC cycles to PCAPNG ns format */ -static struct pcapng_time { - uint64_t ns; - uint64_t cycles; - uint64_t tsc_hz; - struct rte_reciprocal_u64 tsc_hz_inverse; -} pcapng_time; - - #ifdef RTE_EXEC_ENV_WINDOWS /* * Windows does not have writev() call. @@ -102,56 +94,21 @@ static ssize_t writev(int fd, const struct iovec *iov, int iovcnt) #define if_indextoname(ifindex, ifname) NULL #endif -static inline void -pcapng_init(void) +/* Convert from TSC (CPU cycles) to nanoseconds */ +static uint64_t +pcapng_timestamp(const rte_pcapng_t *self, uint64_t cycles) { - struct timespec ts; + uint64_t delta, rem, secs, ns; + const uint64_t hz = rte_get_tsc_hz(); - pcapng_time.cycles = rte_get_tsc_cycles(); - clock_gettime(CLOCK_REALTIME, &ts); - pcapng_time.cycles = (pcapng_time.cycles + rte_get_tsc_cycles()) / 2; - pcapng_time.ns = rte_timespec_to_ns(&ts); - - pcapng_time.tsc_hz = rte_get_tsc_hz(); - pcapng_time.tsc_hz_inverse = rte_reciprocal_value_u64(pcapng_time.tsc_hz); -} + delta = cycles - self->tsc_base; -/* PCAPNG timestamps are in nanoseconds */ -static uint64_t pcapng_tsc_to_ns(uint64_t cycles) -{ - uint64_t delta, secs; - - if (!pcapng_time.tsc_hz) - pcapng_init(); - - /* In essence the calculation is: - * delta = (cycles - pcapng_time.cycles) * NSEC_PRE_SEC / rte_get_tsc_hz() - * but this overflows within 4 to 8 seconds depending on TSC frequency. - * Instead, if delta >= pcapng_time.tsc_hz: - * Increase pcapng_time.ns and pcapng_time.cycles by the number of - * whole seconds in delta and reduce delta accordingly. - * delta will therefore always lie in the interval [0, pcapng_time.tsc_hz), - * which will not overflow when multiplied by NSEC_PER_SEC provided the - * TSC frequency < approx 18.4GHz. - * - * Currently all TSCs operate below 5GHz. - */ - delta = cycles - pcapng_time.cycles; - if (unlikely(delta >= pcapng_time.tsc_hz)) { - if (likely(delta < pcapng_time.tsc_hz * 2)) { - delta -= pcapng_time.tsc_hz; - pcapng_time.cycles += pcapng_time.tsc_hz; - pcapng_time.ns += NSEC_PER_SEC; - } else { - secs = rte_reciprocal_divide_u64(delta, &pcapng_time.tsc_hz_inverse); - delta -= secs * pcapng_time.tsc_hz; - pcapng_time.cycles += secs * pcapng_time.tsc_hz; - pcapng_time.ns += secs * NSEC_PER_SEC; - } - } + /* Avoid numeric wraparound by computing seconds first */ + secs = delta / hz; + rem = delta % hz; + ns = (rem * NS_PER_S) / hz; - return pcapng_time.ns + rte_reciprocal_divide_u64(delta * NSEC_PER_SEC, - &pcapng_time.tsc_hz_inverse); + return secs * NS_PER_S + ns + self->offset_ns; } /* length of option including padding */ @@ -368,15 +325,15 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, */ ssize_t rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, - const char *comment, - uint64_t start_time, uint64_t end_time, - uint64_t ifrecv, uint64_t ifdrop) + uint64_t ifrecv, uint64_t ifdrop, + const char *comment) { struct pcapng_statistics *hdr; struct pcapng_option *opt; + uint64_t start_time = self->offset_ns; + uint64_t sample_time; uint32_t optlen, len; uint8_t *buf; - uint64_t ns; RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); @@ -386,10 +343,10 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, optlen += pcapng_optlen(sizeof(ifrecv)); if (ifdrop != UINT64_MAX) optlen += pcapng_optlen(sizeof(ifdrop)); + if (start_time != 0) optlen += pcapng_optlen(sizeof(start_time)); - if (end_time != 0) - optlen += pcapng_optlen(sizeof(end_time)); + if (comment) optlen += pcapng_optlen(strlen(comment)); if (optlen != 0) @@ -409,9 +366,6 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, if (start_time != 0) opt = pcapng_add_option(opt, PCAPNG_ISB_STARTTIME, &start_time, sizeof(start_time)); - if (end_time != 0) - opt = pcapng_add_option(opt, PCAPNG_ISB_ENDTIME, - &end_time, sizeof(end_time)); if (ifrecv != UINT64_MAX) opt = pcapng_add_option(opt, PCAPNG_ISB_IFRECV, &ifrecv, sizeof(ifrecv)); @@ -425,9 +379,9 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, hdr->block_length = len; hdr->interface_id = self->port_index[port_id]; - ns = pcapng_tsc_to_ns(rte_get_tsc_cycles()); - hdr->timestamp_hi = ns >> 32; - hdr->timestamp_lo = (uint32_t)ns; + sample_time = pcapng_timestamp(self, rte_get_tsc_cycles()); + hdr->timestamp_hi = sample_time >> 32; + hdr->timestamp_lo = (uint32_t)sample_time; /* clone block_length after option */ memcpy(opt, &len, sizeof(uint32_t)); @@ -520,23 +474,21 @@ struct rte_mbuf * rte_pcapng_copy(uint16_t port_id, uint32_t queue, const struct rte_mbuf *md, struct rte_mempool *mp, - uint32_t length, uint64_t cycles, + uint32_t length, enum rte_pcapng_direction direction, const char *comment) { struct pcapng_enhance_packet_block *epb; uint32_t orig_len, data_len, padding, flags; struct pcapng_option *opt; + uint64_t timestamp; uint16_t optlen; struct rte_mbuf *mc; - uint64_t ns; bool rss_hash; #ifdef RTE_LIBRTE_ETHDEV_DEBUG RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, NULL); #endif - ns = pcapng_tsc_to_ns(cycles); - orig_len = rte_pktmbuf_pkt_len(md); /* Take snapshot of the data */ @@ -641,8 +593,10 @@ rte_pcapng_copy(uint16_t port_id, uint32_t queue, /* Interface index is filled in later during write */ mc->port = port_id; - epb->timestamp_hi = ns >> 32; - epb->timestamp_lo = (uint32_t)ns; + /* Put timestamp in cycles here - adjust in packet write */ + timestamp = rte_get_tsc_cycles(); + epb->timestamp_hi = timestamp >> 32; + epb->timestamp_lo = (uint32_t)timestamp; epb->capture_length = data_len; epb->original_length = orig_len; @@ -668,6 +622,7 @@ rte_pcapng_write_packets(rte_pcapng_t *self, for (i = 0; i < nb_pkts; i++) { struct rte_mbuf *m = pkts[i]; struct pcapng_enhance_packet_block *epb; + uint64_t cycles, timestamp; /* sanity check that is really a pcapng mbuf */ epb = rte_pktmbuf_mtod(m, struct pcapng_enhance_packet_block *); @@ -684,6 +639,13 @@ rte_pcapng_write_packets(rte_pcapng_t *self, return -1; } + /* adjust timestamp recorded in packet */ + cycles = (uint64_t)epb->timestamp_hi << 32; + cycles += epb->timestamp_lo; + timestamp = pcapng_timestamp(self, cycles); + epb->timestamp_hi = timestamp >> 32; + epb->timestamp_lo = (uint32_t)timestamp; + /* * Handle case of highly fragmented and large burst size * Note: this assumes that max segments per mbuf < IOV_MAX @@ -725,6 +687,8 @@ rte_pcapng_fdopen(int fd, { unsigned int i; rte_pcapng_t *self; + struct timespec ts; + uint64_t cycles; self = malloc(sizeof(*self)); if (!self) { @@ -734,6 +698,13 @@ rte_pcapng_fdopen(int fd, self->outfd = fd; self->ports = 0; + + /* record start time in ns since 1/1/1970 */ + cycles = rte_get_tsc_cycles(); + clock_gettime(CLOCK_REALTIME, &ts); + self->tsc_base = (cycles + rte_get_tsc_cycles()) / 2; + self->offset_ns = rte_timespec_to_ns(&ts); + for (i = 0; i < RTE_MAX_ETHPORTS; i++) self->port_index[i] = UINT32_MAX; diff --git a/lib/pcapng/rte_pcapng.h b/lib/pcapng/rte_pcapng.h index d93cc9f73ad5..c40795c721de 100644 --- a/lib/pcapng/rte_pcapng.h +++ b/lib/pcapng/rte_pcapng.h @@ -121,8 +121,6 @@ enum rte_pcapng_direction { * @param length * The upper limit on bytes to copy. Passing UINT32_MAX * means all data (after offset). - * @param timestamp - * The timestamp in TSC cycles. * @param direction * The direction of the packer: receive, transmit or unknown. * @param comment @@ -136,7 +134,7 @@ __rte_experimental struct rte_mbuf * rte_pcapng_copy(uint16_t port_id, uint32_t queue, const struct rte_mbuf *m, struct rte_mempool *mp, - uint32_t length, uint64_t timestamp, + uint32_t length, enum rte_pcapng_direction direction, const char *comment); @@ -188,29 +186,22 @@ rte_pcapng_write_packets(rte_pcapng_t *self, * The handle to the packet capture file * @param port * The Ethernet port to report stats on. - * @param comment - * Optional comment to add to statistics. - * @param start_time - * The time when packet capture was started in nanoseconds. - * Optional: can be zero if not known. - * @param end_time - * The time when packet capture was stopped in nanoseconds. - * Optional: can be zero if not finished; * @param ifrecv * The number of packets received by capture. * Optional: use UINT64_MAX if not known. * @param ifdrop * The number of packets missed by the capture process. * Optional: use UINT64_MAX if not known. + * @param comment + * Optional comment to add to statistics. * @return * number of bytes written to file, -1 on failure to write file */ __rte_experimental ssize_t rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port, - const char *comment, - uint64_t start_time, uint64_t end_time, - uint64_t ifrecv, uint64_t ifdrop); + uint64_t ifrecv, uint64_t ifdrop, + const char *comment); #ifdef __cplusplus } diff --git a/lib/pdump/rte_pdump.c b/lib/pdump/rte_pdump.c index e94f49e21250..5a1ec14d7a18 100644 --- a/lib/pdump/rte_pdump.c +++ b/lib/pdump/rte_pdump.c @@ -90,7 +90,6 @@ pdump_copy(uint16_t port_id, uint16_t queue, int ring_enq; uint16_t d_pkts = 0; struct rte_mbuf *dup_bufs[nb_pkts]; - uint64_t ts; struct rte_ring *ring; struct rte_mempool *mp; struct rte_mbuf *p; @@ -99,7 +98,6 @@ pdump_copy(uint16_t port_id, uint16_t queue, if (cbs->filter) rte_bpf_exec_burst(cbs->filter, (void **)pkts, rcs, nb_pkts); - ts = rte_get_tsc_cycles(); ring = cbs->ring; mp = cbs->mp; for (i = 0; i < nb_pkts; i++) { @@ -122,7 +120,7 @@ pdump_copy(uint16_t port_id, uint16_t queue, if (cbs->ver == V2) p = rte_pcapng_copy(port_id, queue, pkts[i], mp, cbs->snaplen, - ts, direction, NULL); + direction, NULL); else p = rte_pktmbuf_copy(pkts[i], mp, 0, cbs->snaplen); From patchwork Wed Nov 8 18:35:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134001 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8AE7A432D9; Wed, 8 Nov 2023 19:36:42 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8D0D942DD1; Wed, 8 Nov 2023 19:36:16 +0100 (CET) Received: from mail-oa1-f53.google.com (mail-oa1-f53.google.com [209.85.160.53]) by mails.dpdk.org (Postfix) with ESMTP id 8A04941156 for ; Wed, 8 Nov 2023 19:36:13 +0100 (CET) Received: by mail-oa1-f53.google.com with SMTP id 586e51a60fabf-1ea48ef2cbfso4338036fac.2 for ; Wed, 08 Nov 2023 10:36:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699468572; x=1700073372; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Vsmc5T89G0SYjUfKbvLN3pvhlouN8b2pD5fYyKZ0XzU=; b=aYxZcdMWLmid76jgjWpnVIgJBjfL/JJ/fB3bMBNdjfjiyTHRlx174okOS2Hreb+dRN X6d7zTtMmFqiui9KQS6rfKTq8ku7gvW9G5P2vwlcRgQp4nrSsBwHXhObHcWxpnwQMu6w i0KE5oW2t+N5HnFSF8z4skxC+t8sr7fHJomm7UI3TlK8TvhZyTUFThjnsCnjEIiNMIqL cfBN+WQx4du0ZPzFXm29f3P9HWGcMq5EmHiryh6S5Th48UFtb/NbyDU9jUDUD7Negsy7 qMkQcPO1lMR9Ib0b524fNYib/y7/5Nblc6ZRyalaH7BATo4QWlmbCf6BEa/f4UL9OZKR 3P4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699468572; x=1700073372; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vsmc5T89G0SYjUfKbvLN3pvhlouN8b2pD5fYyKZ0XzU=; b=RpKxXSbenlEG6gF5Cwn2ubTo7Naw4DfWyHyEXhoHO+eTPeL8I88IVPbq9RgH7IHNpw P0tmNzUU53pD7OKAKk3T4akrYCkZi65OTm0in1BUfyyTGwbM2qf53KZs2S/RHJCEjoEy G79LU4VJeWFRhJbGSgMfD/NWWJDV2yQdK0WaraKD2w9a/gAMQb8MMfCq+CFCf+3Ri0mL JdN0H3dx7xugiRXq47YKMiRdaiomzmuzjnwc3bzWn6sfv4270q4/PYnUsw+7642ggYKe uXSVXpl0cq565yd+kKGR4S1MbH6TrUpmrcy4AAtYtrY75GZIw+71NzPXQaF/zzfhz46h jc7Q== X-Gm-Message-State: AOJu0Yxeskl+fOCjUZNvgop7yNUkFVhp+ze76TNeJ/kMngtpgimcfFm3 2srnJ3eU3yz/85fzpIRVzpt4dSiODo4MwSsNpJ4= X-Google-Smtp-Source: AGHT+IGmd9aF3GyMSMsPN8/QZzYf4kJ0Y1KZPnQeK0VWpTDHO1GPixIxIS+KmGy/exapLoaGII3pFw== X-Received: by 2002:a05:6871:60c:b0:1ea:60c2:9fcd with SMTP id w12-20020a056871060c00b001ea60c29fcdmr2960175oan.47.1699468572582; Wed, 08 Nov 2023 10:36:12 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id p25-20020a635b19000000b005b944b20f34sm3402522pgb.85.2023.11.08.10.36.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Nov 2023 10:36:11 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 4/5] pcapng: avoid using alloca() Date: Wed, 8 Nov 2023 10:35:56 -0800 Message-Id: <20231108183557.381955-5-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231108183557.381955-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231108183557.381955-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The function alloca() like VLA's has problems if the caller passes a large value. Instead use a fixed size buffer (4K) which will be more than sufficient for the info related blocks in the file. Add bounds checks as well. Signed-off-by: Stephen Hemminger Acked-by: Morten Brørup --- lib/pcapng/rte_pcapng.c | 34 +++++++++++++--------------------- 1 file changed, 13 insertions(+), 21 deletions(-) diff --git a/lib/pcapng/rte_pcapng.c b/lib/pcapng/rte_pcapng.c index 13fd2b97fb80..67f74d31aa32 100644 --- a/lib/pcapng/rte_pcapng.c +++ b/lib/pcapng/rte_pcapng.c @@ -140,9 +140,8 @@ pcapng_section_block(rte_pcapng_t *self, { struct pcapng_section_header *hdr; struct pcapng_option *opt; - void *buf; + uint8_t buf[BUFSIZ]; uint32_t len; - ssize_t cc; len = sizeof(*hdr); if (hw) @@ -158,8 +157,7 @@ pcapng_section_block(rte_pcapng_t *self, len += pcapng_optlen(0); len += sizeof(uint32_t); - buf = calloc(1, len); - if (!buf) + if (len > sizeof(buf)) return -1; hdr = (struct pcapng_section_header *)buf; @@ -193,10 +191,7 @@ pcapng_section_block(rte_pcapng_t *self, /* clone block_length after option */ memcpy(opt, &hdr->block_length, sizeof(uint32_t)); - cc = write(self->outfd, buf, len); - free(buf); - - return cc; + return write(self->outfd, buf, len); } /* Write an interface block for a DPDK port */ @@ -213,7 +208,7 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, struct pcapng_option *opt; const uint8_t tsresol = 9; /* nanosecond resolution */ uint32_t len; - void *buf; + uint8_t buf[BUFSIZ]; char ifname_buf[IF_NAMESIZE]; char ifhw[256]; uint64_t speed = 0; @@ -267,8 +262,7 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, len += pcapng_optlen(0); len += sizeof(uint32_t); - buf = alloca(len); - if (!buf) + if (len > sizeof(buf)) return -1; hdr = (struct pcapng_interface_block *)buf; @@ -296,17 +290,16 @@ rte_pcapng_add_interface(rte_pcapng_t *self, uint16_t port, opt = pcapng_add_option(opt, PCAPNG_IFB_HARDWARE, ifhw, strlen(ifhw)); if (filter) { - /* Encoding is that the first octet indicates string vs BPF */ size_t len; - char *buf; len = strlen(filter) + 1; - buf = alloca(len); - *buf = '\0'; - memcpy(buf + 1, filter, len); + opt->code = PCAPNG_IFB_FILTER; + opt->length = len; + /* Encoding is that the first octet indicates string vs BPF */ + opt->data[0] = 0; + memcpy(opt->data + 1, filter, strlen(filter)); - opt = pcapng_add_option(opt, PCAPNG_IFB_FILTER, - buf, len); + opt = (struct pcapng_option *)((uint8_t *)opt + pcapng_optlen(len)); } opt = pcapng_add_option(opt, PCAPNG_OPT_END, NULL, 0); @@ -333,7 +326,7 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, uint64_t start_time = self->offset_ns; uint64_t sample_time; uint32_t optlen, len; - uint8_t *buf; + uint8_t buf[BUFSIZ]; RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); @@ -353,8 +346,7 @@ rte_pcapng_write_stats(rte_pcapng_t *self, uint16_t port_id, optlen += pcapng_optlen(0); len = sizeof(*hdr) + optlen + sizeof(uint32_t); - buf = alloca(len); - if (buf == NULL) + if (len > sizeof(buf)) return -1; hdr = (struct pcapng_statistics *)buf; From patchwork Wed Nov 8 18:35:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 134002 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E9A09432D9; Wed, 8 Nov 2023 19:36:49 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AD37642DDE; Wed, 8 Nov 2023 19:36:17 +0100 (CET) Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) by mails.dpdk.org (Postfix) with ESMTP id B58E742DAC for ; Wed, 8 Nov 2023 19:36:14 +0100 (CET) Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-586753b0ab0so3809479eaf.0 for ; Wed, 08 Nov 2023 10:36:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1699468574; x=1700073374; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZxBQOrc3DzlSpoSRdVgX3NKVNdfN6wt2OXsbYKLwbLc=; b=JcTFEjGNP1KT86pax71+HeVj5ITdm57Cdte5l2BoYxnquQXh9mYEp0fhALLV5TmXn6 ouMA7RZl9HpR4VL1vN0jZ+bi3fm6+I1E1weu+EY9RT1drVpuyopqU9u2RGP3bzsL+3Vo c3tjObPAQRLBIUhAgEIZH3ag3W369bgBSzqJiEUybQFdUIeS2udaH911psEGbadk8k73 uJKSl1MTlRGD0gWVeLu3QXQeoFoBqNad1oy7ph0yrJppB+D3dVO0frBF0QA4V+dKHgH2 PlrZk/nanhKfIQagkXu+YpaRDtwoKkdwEzdcWiGqlUWciStUNLGeO6X1Vf5wwnLDh7UI qi2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699468574; x=1700073374; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZxBQOrc3DzlSpoSRdVgX3NKVNdfN6wt2OXsbYKLwbLc=; b=TNypumlfEpZxQQknDwHYkaNlYUaZpB8dpCo6ubig7R/Yrdk2288pVyQ8w1Ao+ZBIi7 Ok7SFlj7LiCaKSG40OEZocsK2l4Om4KGgL8kZDQiQH41rkKy9We1Is7ck+/t7GPjmPJU qReFcmZHT+RAEmfECAhOqDHyAD3CVG0W8BVxfNLQTod+80vUXBA4nw7UInRZiS76Va5U KmXzeroIHAHghLsZ0XzHyBHynDI+QyIAiKyAxH44xIFatjs1Pq7IQJGqcGw8doJozRrP kif2QAvm4yTo7+XyFfm1BiUHlKZtgMgRFebIvfIQbHLHV4PqfuJEAcAKsEO7vI4Wu2JT tiLA== X-Gm-Message-State: AOJu0Ywj9Bikus0u3dmYMYkSH7+rmQXQj/9ytwheOYsaOXmYtJ5A8EB8 GjiK1msUlIq2hY6RAIs4vhPak9LhDP9lQn6bj5w= X-Google-Smtp-Source: AGHT+IFsiyASUHX5mKgdeRNPg7G5dPRf1/HM62SpxwWYD49khJ6+9a5rZns3qVBWu6gtPO/0dfZx3w== X-Received: by 2002:a05:6358:63a7:b0:168:e5ee:8fbd with SMTP id k39-20020a05635863a700b00168e5ee8fbdmr2963212rwh.27.1699468573716; Wed, 08 Nov 2023 10:36:13 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id p25-20020a635b19000000b005b944b20f34sm3402522pgb.85.2023.11.08.10.36.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Nov 2023 10:36:12 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 5/5] test: cleanups to pcapng test Date: Wed, 8 Nov 2023 10:35:57 -0800 Message-Id: <20231108183557.381955-6-stephen@networkplumber.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231108183557.381955-1-stephen@networkplumber.org> References: <20230921042349.104150-1-stephen@networkplumber.org> <20231108183557.381955-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Overhaul of the pcapng test: - promote it to be a fast test so it gets regularly run. - create null device and use i. - use UDP discard packets that are valid so that for debugging the resulting pcapng file can be looked at with wireshark. - do basic checks on resulting pcap file that lengths and timestamps are in range. - add test for interface options Signed-off-by: Stephen Hemminger --- app/test/meson.build | 2 +- app/test/test_pcapng.c | 418 +++++++++++++++++++++++++++-------------- 2 files changed, 282 insertions(+), 138 deletions(-) diff --git a/app/test/meson.build b/app/test/meson.build index 4183d66b0e9c..dcc93f4a43b4 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -128,7 +128,7 @@ source_file_deps = { 'test_metrics.c': ['metrics'], 'test_mp_secondary.c': ['hash', 'lpm'], 'test_net_ether.c': ['net'], - 'test_pcapng.c': ['ethdev', 'net', 'pcapng'], + 'test_pcapng.c': ['ethdev', 'net', 'pcapng', 'bus_vdev'], 'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'], 'test_pdump.c': ['pdump'] + sample_packet_forward_deps, 'test_per_lcore.c': [], diff --git a/app/test/test_pcapng.c b/app/test/test_pcapng.c index 55aa2cf93666..c973aa47d1f8 100644 --- a/app/test/test_pcapng.c +++ b/app/test/test_pcapng.c @@ -6,25 +6,34 @@ #include #include +#include #include #include +#include #include #include #include #include +#include +#include +#include +#include #include #include "test.h" -#define NUM_PACKETS 10 -#define DUMMY_MBUF_NUM 3 +#define PCAPNG_TEST_DEBUG 0 + +#define TOTAL_PACKETS 4096 +#define MAX_BURST 64 +#define MAX_GAP_US 100000 +#define DUMMY_MBUF_NUM 3 -static rte_pcapng_t *pcapng; static struct rte_mempool *mp; static const uint32_t pkt_len = 200; static uint16_t port_id; -static char file_name[] = "/tmp/pcapng_test_XXXXXX.pcapng"; +static const char null_dev[] = "net_null0"; /* first mbuf in the packet, should always be at offset 0 */ struct dummy_mbuf { @@ -61,6 +70,7 @@ mbuf1_prepare(struct dummy_mbuf *dm, uint32_t plen) struct { struct rte_ether_hdr eth; struct rte_ipv4_hdr ip; + struct rte_udp_hdr udp; } pkt = { .eth = { .dst_addr.addr_bytes = "\xff\xff\xff\xff\xff\xff", @@ -68,149 +78,201 @@ mbuf1_prepare(struct dummy_mbuf *dm, uint32_t plen) }, .ip = { .version_ihl = RTE_IPV4_VHL_DEF, - .total_length = rte_cpu_to_be_16(plen), - .time_to_live = IPDEFTTL, - .next_proto_id = IPPROTO_RAW, + .time_to_live = 1, + .next_proto_id = IPPROTO_UDP, .src_addr = rte_cpu_to_be_32(RTE_IPV4_LOOPBACK), .dst_addr = rte_cpu_to_be_32(RTE_IPV4_BROADCAST), - } + }, + .udp = { + .dst_port = rte_cpu_to_be_16(9), /* Discard port */ + }, }; memset(dm, 0, sizeof(*dm)); dummy_mbuf_prep(&dm->mb[0], dm->buf[0], sizeof(dm->buf[0]), plen); rte_eth_random_addr(pkt.eth.src_addr.addr_bytes); - memcpy(rte_pktmbuf_mtod(dm->mb, void *), &pkt, RTE_MIN(sizeof(pkt), plen)); + plen -= sizeof(struct rte_ether_hdr); + + pkt.ip.total_length = rte_cpu_to_be_16(plen); + pkt.ip.hdr_checksum = rte_ipv4_cksum(&pkt.ip); + + plen -= sizeof(struct rte_ipv4_hdr); + pkt.udp.src_port = rte_rand(); + pkt.udp.dgram_len = rte_cpu_to_be_16(plen); + + memcpy(rte_pktmbuf_mtod(dm->mb, void *), &pkt, sizeof(pkt)); } static int test_setup(void) { - int tmp_fd; - - port_id = rte_eth_find_next(0); - if (port_id >= RTE_MAX_ETHPORTS) { - fprintf(stderr, "No valid Ether port\n"); - return -1; - } + port_id = rte_eth_dev_count_avail(); - tmp_fd = mkstemps(file_name, strlen(".pcapng")); - if (tmp_fd == -1) { - perror("mkstemps() failure"); - return -1; - } - printf("pcapng: output file %s\n", file_name); - - /* open a test capture file */ - pcapng = rte_pcapng_fdopen(tmp_fd, NULL, NULL, "pcapng_test", NULL); - if (pcapng == NULL) { - fprintf(stderr, "rte_pcapng_fdopen failed\n"); - close(tmp_fd); - return -1; - } - - /* Add interface to the file */ - if (rte_pcapng_add_interface(pcapng, port_id, - NULL, NULL, NULL) != 0) { - fprintf(stderr, "can not add port %u\n", port_id); - return -1; + /* Make a dummy null device to snoop on */ + if (rte_vdev_init(null_dev, NULL) != 0) { + fprintf(stderr, "Failed to create vdev '%s'\n", null_dev); + goto fail; } /* Make a pool for cloned packets */ - mp = rte_pktmbuf_pool_create_by_ops("pcapng_test_pool", IOV_MAX + NUM_PACKETS, - 0, 0, - rte_pcapng_mbuf_size(pkt_len), + mp = rte_pktmbuf_pool_create_by_ops("pcapng_test_pool", + MAX_BURST, 0, 0, + rte_pcapng_mbuf_size(pkt_len) + 128, SOCKET_ID_ANY, "ring_mp_sc"); if (mp == NULL) { fprintf(stderr, "Cannot create mempool\n"); - return -1; + goto fail; } + return 0; + +fail: + rte_vdev_uninit(null_dev); + rte_mempool_free(mp); + return -1; } static int -test_write_packets(void) +fill_pcapng_file(rte_pcapng_t *pcapng, unsigned int num_packets) { - struct rte_mbuf *orig; - struct rte_mbuf *clones[NUM_PACKETS] = { }; struct dummy_mbuf mbfs; - unsigned int i; + struct rte_mbuf *orig; + unsigned int burst_size; + unsigned int count; ssize_t len; /* make a dummy packet */ mbuf1_prepare(&mbfs, pkt_len); - - /* clone them */ orig = &mbfs.mb[0]; - for (i = 0; i < NUM_PACKETS; i++) { - struct rte_mbuf *mc; - mc = rte_pcapng_copy(port_id, 0, orig, mp, pkt_len, - rte_get_tsc_cycles(), 0, NULL); - if (mc == NULL) { - fprintf(stderr, "Cannot copy packet\n"); + for (count = 0; count < num_packets; count += burst_size) { + struct rte_mbuf *clones[MAX_BURST]; + unsigned int i; + + /* put 1 .. MAX_BURST packets in one write call */ + burst_size = rte_rand_max(MAX_BURST) + 1; + for (i = 0; i < burst_size; i++) { + struct rte_mbuf *mc; + + mc = rte_pcapng_copy(port_id, 0, orig, mp, pkt_len, + RTE_PCAPNG_DIRECTION_IN, + NULL); + if (mc == NULL) { + fprintf(stderr, "Cannot copy packet\n"); + return -1; + } + clones[i] = mc; + } + + /* write it to capture file */ + len = rte_pcapng_write_packets(pcapng, clones, burst_size); + rte_pktmbuf_free_bulk(clones, burst_size); + + if (len <= 0) { + fprintf(stderr, "Write of packets failed: %s\n", + rte_strerror(rte_errno)); return -1; } - clones[i] = mc; + + /* Leave a small gap between packets to test for time wrap */ + usleep(rte_rand_max(MAX_GAP_US)); } - /* write it to capture file */ - len = rte_pcapng_write_packets(pcapng, clones, NUM_PACKETS); + return count; +} - rte_pktmbuf_free_bulk(clones, NUM_PACKETS); +static char * +fmt_time(char *buf, size_t size, uint64_t ts_ns) +{ + time_t sec; + size_t len; - if (len <= 0) { - fprintf(stderr, "Write of packets failed\n"); - return -1; - } + sec = ts_ns / NS_PER_S; + len = strftime(buf, size, "%X", localtime(&sec)); + snprintf(buf + len, size - len, ".%09lu", + (unsigned long)(ts_ns % NS_PER_S)); - return 0; + return buf; } -static int -test_write_stats(void) +/* Context for the pcap_loop callback */ +struct pkt_print_ctx { + pcap_t *pcap; + unsigned int count; + uint64_t start_ns; + uint64_t end_ns; +}; + +static void +print_packet(uint64_t ts_ns, const struct rte_ether_hdr *eh, size_t len) { - ssize_t len; + char tbuf[128], src[64], dst[64]; - /* write a statistics block */ - len = rte_pcapng_write_stats(pcapng, port_id, NULL, - 0, 0, 0, - NUM_PACKETS, 0); - if (len <= 0) { - fprintf(stderr, "Write of statistics failed\n"); - return -1; - } - return 0; + fmt_time(tbuf, sizeof(tbuf), ts_ns); + rte_ether_format_addr(dst, sizeof(dst), &eh->dst_addr); + rte_ether_format_addr(src, sizeof(src), &eh->src_addr); + printf("%s: %s -> %s type %x length %zu\n", + tbuf, src, dst, rte_be_to_cpu_16(eh->ether_type), len); } +/* Callback from pcap_loop used to validate packets in the file */ static void -pkt_print(u_char *user, const struct pcap_pkthdr *h, - const u_char *bytes) +parse_pcap_packet(u_char *user, const struct pcap_pkthdr *h, + const u_char *bytes) { - unsigned int *countp = (unsigned int *)user; + struct pkt_print_ctx *ctx = (struct pkt_print_ctx *)user; const struct rte_ether_hdr *eh; - struct tm *tm; - char tbuf[128], src[64], dst[64]; + const struct rte_ipv4_hdr *ip; + uint64_t ns; - tm = localtime(&h->ts.tv_sec); - if (tm == NULL) { - perror("localtime"); - return; + eh = (const struct rte_ether_hdr *)bytes; + ip = (const struct rte_ipv4_hdr *)(eh + 1); + + ctx->count += 1; + + /* The pcap library is misleading in reporting timestamp. + * packet header struct gives timestamp as a timeval (ie. usec); + * but the file is open in nanonsecond mode therefore + * the timestamp is really in timespec (ie. nanoseconds). + */ + ns = h->ts.tv_sec * NS_PER_S + h->ts.tv_usec; + if (ns < ctx->start_ns || ns > ctx->end_ns) { + char tstart[128], tend[128]; + + fmt_time(tstart, sizeof(tstart), ctx->start_ns); + fmt_time(tend, sizeof(tend), ctx->end_ns); + fprintf(stderr, "Timestamp out of range [%s .. %s]\n", + tstart, tend); + goto error; } - if (strftime(tbuf, sizeof(tbuf), "%X", tm) == 0) { - fprintf(stderr, "strftime returned 0!\n"); - return; + if (!rte_is_broadcast_ether_addr(&eh->dst_addr)) { + fprintf(stderr, "Destination is not broadcast\n"); + goto error; } - eh = (const struct rte_ether_hdr *)bytes; - rte_ether_format_addr(dst, sizeof(dst), &eh->dst_addr); - rte_ether_format_addr(src, sizeof(src), &eh->src_addr); - printf("%s.%06lu: %s -> %s type %x length %u\n", - tbuf, (unsigned long)h->ts.tv_usec, - src, dst, rte_be_to_cpu_16(eh->ether_type), h->len); + if (rte_ipv4_cksum(ip) != 0) { + fprintf(stderr, "Bad IPv4 checksum\n"); + goto error; + } + + return; /* packet is normal */ + +error: + print_packet(ns, eh, h->len); + + /* Stop parsing at first error */ + pcap_breakloop(ctx->pcap); +} - *countp += 1; +static uint64_t +current_timestamp(void) +{ + struct timespec ts; + + clock_gettime(CLOCK_REALTIME, &ts); + return rte_timespec_to_ns(&ts); } /* @@ -219,78 +281,162 @@ pkt_print(u_char *user, const struct pcap_pkthdr *h, * but that creates an unwanted dependency. */ static int -test_validate(void) +valid_pcapng_file(const char *file_name, uint64_t started, unsigned int expected) { char errbuf[PCAP_ERRBUF_SIZE]; - unsigned int count = 0; - pcap_t *pcap; + struct pkt_print_ctx ctx = { }; int ret; - pcap = pcap_open_offline(file_name, errbuf); - if (pcap == NULL) { + ctx.start_ns = started; + ctx.end_ns = current_timestamp(); + + ctx.pcap = pcap_open_offline_with_tstamp_precision(file_name, + PCAP_TSTAMP_PRECISION_NANO, + errbuf); + if (ctx.pcap == NULL) { fprintf(stderr, "pcap_open_offline('%s') failed: %s\n", file_name, errbuf); return -1; } - ret = pcap_loop(pcap, 0, pkt_print, (u_char *)&count); - if (ret == 0) - printf("Saw %u packets\n", count); - else + ret = pcap_loop(ctx.pcap, 0, parse_pcap_packet, (u_char *)&ctx); + if (ret != 0) { fprintf(stderr, "pcap_dispatch: failed: %s\n", - pcap_geterr(pcap)); - pcap_close(pcap); + pcap_geterr(ctx.pcap)); + } else if (ctx.count != expected) { + printf("Only %u packets, expected %u\n", + ctx.count, expected); + ret = -1; + } + + pcap_close(ctx.pcap); return ret; } static int -test_write_over_limit_iov_max(void) +test_add_interface(void) { - struct rte_mbuf *orig; - struct rte_mbuf *clones[IOV_MAX + NUM_PACKETS] = { }; - struct dummy_mbuf mbfs; - unsigned int i; - ssize_t len; + char file_name[] = "/tmp/pcapng_test_XXXXXX.pcapng"; + static rte_pcapng_t *pcapng; + int ret, tmp_fd; + uint64_t now = current_timestamp(); - /* make a dummy packet */ - mbuf1_prepare(&mbfs, pkt_len); + tmp_fd = mkstemps(file_name, strlen(".pcapng")); + if (tmp_fd == -1) { + perror("mkstemps() failure"); + goto fail; + } + printf("pcapng: output file %s\n", file_name); - /* clone them */ - orig = &mbfs.mb[0]; - for (i = 0; i < IOV_MAX + NUM_PACKETS; i++) { - struct rte_mbuf *mc; + /* open a test capture file */ + pcapng = rte_pcapng_fdopen(tmp_fd, NULL, NULL, "pcapng_addif", NULL); + if (pcapng == NULL) { + fprintf(stderr, "rte_pcapng_fdopen failed\n"); + close(tmp_fd); + goto fail; + } - mc = rte_pcapng_copy(port_id, 0, orig, mp, pkt_len, - rte_get_tsc_cycles(), 0, NULL); - if (mc == NULL) { - fprintf(stderr, "Cannot copy packet\n"); - return -1; - } - clones[i] = mc; + /* Add interface to the file */ + ret = rte_pcapng_add_interface(pcapng, port_id, + NULL, NULL, NULL); + if (ret < 0) { + fprintf(stderr, "can not add port %u\n", port_id); + goto fail; } - /* write it to capture file */ - len = rte_pcapng_write_packets(pcapng, clones, IOV_MAX + NUM_PACKETS); + /* Add interface with ifname and ifdescr */ + ret = rte_pcapng_add_interface(pcapng, port_id, + "myeth", "Some long description", NULL); + if (ret < 0) { + fprintf(stderr, "can not add port %u with ifname\n", port_id); + goto fail; + } + + /* Add interface with filter */ + ret = rte_pcapng_add_interface(pcapng, port_id, + NULL, NULL, "tcp port 8080"); + if (ret < 0) { + fprintf(stderr, "can not add port %u with filter\n", port_id); + goto fail; + } - rte_pktmbuf_free_bulk(clones, IOV_MAX + NUM_PACKETS); + rte_pcapng_close(pcapng); - if (len <= 0) { - fprintf(stderr, "Write of packets failed\n"); - return -1; + ret = valid_pcapng_file(file_name, now, 0); + /* if test fails want to investigate the file */ + if (ret == 0) + unlink(file_name); + + return ret; + +fail: + rte_pcapng_close(pcapng); + return -1; +} + +static int +test_write_packets(void) +{ + char file_name[] = "/tmp/pcapng_test_XXXXXX.pcapng"; + static rte_pcapng_t *pcapng; + int ret, tmp_fd, count; + uint64_t now = current_timestamp(); + + tmp_fd = mkstemps(file_name, strlen(".pcapng")); + if (tmp_fd == -1) { + perror("mkstemps() failure"); + goto fail; } + printf("pcapng: output file %s\n", file_name); - return 0; + /* open a test capture file */ + pcapng = rte_pcapng_fdopen(tmp_fd, NULL, NULL, "pcapng_test", NULL); + if (pcapng == NULL) { + fprintf(stderr, "rte_pcapng_fdopen failed\n"); + close(tmp_fd); + goto fail; + } + + /* Add interface to the file */ + ret = rte_pcapng_add_interface(pcapng, port_id, + NULL, NULL, NULL); + if (ret < 0) { + fprintf(stderr, "can not add port %u\n", port_id); + goto fail; + } + + count = fill_pcapng_file(pcapng, TOTAL_PACKETS); + if (count < 0) + goto fail; + + /* write a statistics block */ + ret = rte_pcapng_write_stats(pcapng, port_id, + count, 0, "end of test"); + if (ret <= 0) { + fprintf(stderr, "Write of statistics failed\n"); + goto fail; + } + + rte_pcapng_close(pcapng); + + ret = valid_pcapng_file(file_name, now, count); + /* if test fails want to investigate the file */ + if (ret == 0) + unlink(file_name); + + return ret; + +fail: + rte_pcapng_close(pcapng); + return -1; } static void test_cleanup(void) { rte_mempool_free(mp); - - if (pcapng) - rte_pcapng_close(pcapng); - + rte_vdev_uninit(null_dev); } static struct @@ -299,10 +445,8 @@ unit_test_suite test_pcapng_suite = { .teardown = test_cleanup, .suite_name = "Test Pcapng Unit Test Suite", .unit_test_cases = { + TEST_CASE(test_add_interface), TEST_CASE(test_write_packets), - TEST_CASE(test_write_stats), - TEST_CASE(test_validate), - TEST_CASE(test_write_over_limit_iov_max), TEST_CASES_END() } }; @@ -313,4 +457,4 @@ test_pcapng(void) return unit_test_suite_runner(&test_pcapng_suite); } -REGISTER_TEST_COMMAND(pcapng_autotest, test_pcapng); +REGISTER_FAST_TEST(pcapng_autotest, true, true, test_pcapng);