From patchwork Thu Feb 8 17:41:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 136550 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0A43F43AD9; Thu, 8 Feb 2024 18:51:09 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2864042DF9; Thu, 8 Feb 2024 18:51:04 +0100 (CET) Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by mails.dpdk.org (Postfix) with ESMTP id C9F0940273 for ; Thu, 8 Feb 2024 18:51:02 +0100 (CET) Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-5d8b276979aso1536891a12.2 for ; Thu, 08 Feb 2024 09:51:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1707414662; x=1708019462; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qqrc3dYAP1STL4K2E19XdAgGAiiB9sbm8sOysngF+vA=; b=BpvHORGi2xQ7zP29prt4F/iT9ooSZ2oMVL+gBDwc2wTVTsmlWegIvTDCKMOI+7GxiH wtVHSRkcXtsCSiTxy0kAc/8+dKnWFYk6DLzFpwGXgso5b0dR4uSoUXZCzNpIywk4uhIK 4qiH6xbEuhrF/I/GpOwag0xv24t87N9iymSGtaGSL8Jql3SbRhi8RoQimworz2ndQtEd 5Z9ENqfD4Qnz7Fry1x8bGFv/Ik6HcZ434isnUpZYppWEbk3nxhmQptacZRb6lsQCKUy8 or5KfTFtLL+hDurBLjH4IQGzLaXEG+3EbB9/8jTAo4H6j5TaxkV9zAEbCyMvxzZm2mtM 5wHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707414662; x=1708019462; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qqrc3dYAP1STL4K2E19XdAgGAiiB9sbm8sOysngF+vA=; b=vIoXEmQd0hV09JfuocPNO1D4gy3jh7FjqR8lDwj53jGAnNfXBZUNra3TShkxq7I1ZX f/6pAiHUkuCKBaWOES+B7rkzrnsogdGfE4vaqHD5eth70MeAF8BVnESh1EYJLpIYlVg6 85p7aldibvnH1dFO7NZ+UgGHnFToJp0GteVVfVpYy3WS/RIzNMo3fblWkA12ySd3B3ye W6yDoNDGma3DWtwfaZRqr/AOSTlauObUkEIpMSMpmW/LXjJ59zMM+UrpbTo2024xki45 rZ4ZqRA+wYPgfDZOEOWK3kGmKlw1Hhu7hk503Hq7daMdUDsG5xGMhNwrOd5XnqFn4ra0 b8/A== X-Gm-Message-State: AOJu0YzxBay8lDF1tMW1A8ALMS7zQ5pDINTXwzWo0iVN2zjLmLSLcQwm 3BnwZKz9RnjpqcxpcxE4MWQcChb2vcSVZ9lwDucy+hN/+KjFqpp3DCULQm6Yzfh+A+CGAAA8msa HaJM= X-Google-Smtp-Source: AGHT+IH6YELQglJ+0/i88/auW17pxBsIBEDcUY+zIeyjI0b4sg2672Z3cXje0xGfBlhwfiophCqpkg== X-Received: by 2002:a05:6a21:3489:b0:19e:376d:eff5 with SMTP id yo9-20020a056a21348900b0019e376deff5mr299589pzb.43.1707414661929; Thu, 08 Feb 2024 09:51:01 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f19-20020a056a00229300b006dbda7bcf3csm5030pfe.83.2024.02.08.09.51.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 09:51:01 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 1/7] net/tap: remove unused RSS hash types Date: Thu, 8 Feb 2024 09:41:25 -0800 Message-ID: <20240208175051.326550-2-stephen@networkplumber.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240208175051.326550-1-stephen@networkplumber.org> References: <20240130034925.44869-1-stephen@networkplumber.org> <20240208175051.326550-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The driver doesn't support these other hash types, and there is no reason to implement these in future. Signed-off-by: Stephen Hemminger --- drivers/net/tap/tap_rss.h | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/net/tap/tap_rss.h b/drivers/net/tap/tap_rss.h index dff46a012f94..8766ffc244f6 100644 --- a/drivers/net/tap/tap_rss.h +++ b/drivers/net/tap/tap_rss.h @@ -21,12 +21,6 @@ enum hash_field { HASH_FIELD_IPV4_L3_L4, /* IPv4 src/dst addr + L4 src/dst ports */ HASH_FIELD_IPV6_L3, /* IPv6 src/dst addr */ HASH_FIELD_IPV6_L3_L4, /* IPv6 src/dst addr + L4 src/dst ports */ - HASH_FIELD_L2_SRC, /* Ethernet src addr */ - HASH_FIELD_L2_DST, /* Ethernet dst addr */ - HASH_FIELD_L3_SRC, /* L3 src addr */ - HASH_FIELD_L3_DST, /* L3 dst addr */ - HASH_FIELD_L4_SRC, /* TCP/UDP src ports */ - HASH_FIELD_L4_DST, /* TCP/UDP dst ports */ }; struct rss_key { From patchwork Thu Feb 8 17:41:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 136551 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AE53843AE1; Thu, 8 Feb 2024 18:51:15 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5020642E29; Thu, 8 Feb 2024 18:51:05 +0100 (CET) Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by mails.dpdk.org (Postfix) with ESMTP id ACEDA42D7B for ; Thu, 8 Feb 2024 18:51:03 +0100 (CET) Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-6e04ea51984so69085b3a.0 for ; Thu, 08 Feb 2024 09:51:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1707414663; x=1708019463; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=klwclTwCg/MXy6mC0AM+H8Mght/ByiOBM7aYphZ4FYI=; b=d+9YQtsd3R72Q2uiMmgYRL08+9TxI+r5dgp1Pr9UEY32Vj9r0vxg7jW9T9j5s4nrtp MhhZSn2jOAmSlpxigE3Or3y2VbhM4VC8+e5b3VaPrQi3lqylPzD0Efa6VQpHQLH7SH48 kfArD7ZuaWFF2quX2Ls8DU4MtvgNKXuxdoqIQeJfEGk3yeNhFcDvQZYAmxL4TJ+HegeE bNFgK4r7RTJS1QhGOcj8NIMXixAyu9Pdy9jJQKxOZDALtQpeXRU/JbB67L6W43o3BsqU HAUMzOABrXNMKLSHmIlfw49Jnoa5BHFNhq/jJ7fxqiW91pru8F07JJ0yc6xv7/pSb6wG 4q4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707414663; x=1708019463; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=klwclTwCg/MXy6mC0AM+H8Mght/ByiOBM7aYphZ4FYI=; b=CeVy3+w/CjU6WvtDazEzeO9drEM5ptzKRDq6k6TdD31P+py7cOQEdU3Zg+9a5QORb6 6v1aVu5xlZtjgCvuwlS7GAuyJ6f+8q1d8s1aHDVF72Mf7jJAZED4zV5LGTJFRG6H0bwY Z33kHsz2/JBaVMAVT34Njh/KignoetuwgtxmTfpa/BmIaAcdxPGFBQf4I1qXcLmVnREt ikfhOBxwB8uFFlrUXJ388PFQRWNe/JOqSSEDpMTk4HAkrAokgGBmMbFKjpUH+gHqDGxR bIvPDMQ8rmm8akDL0hhjmOnAXhQJ7HO8+gzeIxS8TvGRKbqmb9GJmUetRBteGOMv0wu9 1xcw== X-Gm-Message-State: AOJu0Yy1ePYRvfNZ0rnULHUcBLxVijhLKJc5dOpeCDq2fKFfZISHHy0Z 3RaG2GYzqWQuiHWEfaYPHP1cZR39SUiW+LmBQvS5oTCtNZ8a419wVH+toaVXFcJDSv+oZpiM9U4 TRm8= X-Google-Smtp-Source: AGHT+IGl7rl+9mwjDA8+8rJOJi7CzjaJL8liGDSNHubl0JGUWSWFEf/fwHcU+soUzDG9uEbDxMO85w== X-Received: by 2002:aa7:86c5:0:b0:6e0:528c:54e7 with SMTP id h5-20020aa786c5000000b006e0528c54e7mr6448616pfo.9.1707414662699; Thu, 08 Feb 2024 09:51:02 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f19-20020a056a00229300b006dbda7bcf3csm5030pfe.83.2024.02.08.09.51.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 09:51:02 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 2/7] net/tap: validate and setup parameters for BPF RSS Date: Thu, 8 Feb 2024 09:41:26 -0800 Message-ID: <20240208175051.326550-3-stephen@networkplumber.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240208175051.326550-1-stephen@networkplumber.org> References: <20240130034925.44869-1-stephen@networkplumber.org> <20240208175051.326550-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The flow RSS support via BPF was not using the key, or hash type parameters. Which is good because they were never properly setup. Fix the setup and validate the flow parameters, the BPF side gets fixed later. Signed-off-by: Stephen Hemminger --- drivers/net/tap/tap_bpf_insns.h | 16 ++++---- drivers/net/tap/tap_flow.c | 65 ++++++++++++++++++++++++++++++--- drivers/net/tap/tap_rss.h | 5 +-- 3 files changed, 70 insertions(+), 16 deletions(-) diff --git a/drivers/net/tap/tap_bpf_insns.h b/drivers/net/tap/tap_bpf_insns.h index 53fa76c4e6b0..ee26cf885ed7 100644 --- a/drivers/net/tap/tap_bpf_insns.h +++ b/drivers/net/tap/tap_bpf_insns.h @@ -1709,13 +1709,13 @@ static struct bpf_insn l3_l4_hash_insns[] = { {0x57, 1, 0, 0, 0x00000001}, {0x15, 1, 0, 1, 0x00000000}, {0xa7, 3, 0, 0, 0xfe0fee15}, - {0x71, 1, 0, 201, 0x00000000}, + {0x71, 1, 0, 45, 0x00000000}, {0x67, 1, 0, 0, 0x00000008}, - {0x71, 2, 0, 200, 0x00000000}, + {0x71, 2, 0, 44, 0x00000000}, {0x4f, 1, 2, 0, 0x00000000}, - {0x71, 2, 0, 202, 0x00000000}, + {0x71, 2, 0, 46, 0x00000000}, {0x67, 2, 0, 0, 0x00000010}, - {0x71, 4, 0, 203, 0x00000000}, + {0x71, 4, 0, 47, 0x00000000}, {0x67, 4, 0, 0, 0x00000018}, {0x4f, 4, 2, 0, 0x00000000}, {0x4f, 4, 1, 0, 0x00000000}, @@ -1725,13 +1725,13 @@ static struct bpf_insn l3_l4_hash_insns[] = { {0x57, 3, 0, 0, 0x0000000f}, {0x67, 3, 0, 0, 0x00000002}, {0x0f, 0, 3, 0, 0x00000000}, - {0x71, 1, 0, 137, 0x00000000}, + {0x71, 1, 0, 49, 0x00000000}, {0x67, 1, 0, 0, 0x00000008}, - {0x71, 2, 0, 136, 0x00000000}, + {0x71, 2, 0, 48, 0x00000000}, {0x4f, 1, 2, 0, 0x00000000}, - {0x71, 2, 0, 138, 0x00000000}, + {0x71, 2, 0, 50, 0x00000000}, {0x67, 2, 0, 0, 0x00000010}, - {0x71, 3, 0, 139, 0x00000000}, + {0x71, 3, 0, 51, 0x00000000}, {0x67, 3, 0, 0, 0x00000018}, {0x4f, 3, 2, 0, 0x00000000}, {0x4f, 3, 1, 0, 0x00000000}, diff --git a/drivers/net/tap/tap_flow.c b/drivers/net/tap/tap_flow.c index ed4d42f92f9f..cd49aa51c8b0 100644 --- a/drivers/net/tap/tap_flow.c +++ b/drivers/net/tap/tap_flow.c @@ -11,8 +11,10 @@ #include #include +#include #include #include + #include #include #include @@ -2053,6 +2055,21 @@ static int bpf_rss_key(enum bpf_rss_key_e cmd, __u32 *key_idx) return err; } + +/* Default RSS hash key also used by mlx devices */ +static const uint8_t rss_hash_default_key[] = { + 0x2c, 0xc6, 0x81, 0xd1, + 0x5b, 0xdb, 0xf4, 0xf7, + 0xfc, 0xa2, 0x83, 0x19, + 0xdb, 0x1a, 0x3e, 0x94, + 0x6b, 0x9e, 0x38, 0xd9, + 0x2c, 0x9c, 0x03, 0xd1, + 0xad, 0x99, 0x44, 0xa7, + 0xd9, 0x56, 0x3d, 0x59, + 0x06, 0x3c, 0x25, 0xf3, + 0xfc, 0x1f, 0xdc, 0x2a, +}; + /** * Add RSS hash calculations and queue selection * @@ -2071,11 +2088,11 @@ static int rss_add_actions(struct rte_flow *flow, struct pmd_internals *pmd, const struct rte_flow_action_rss *rss, struct rte_flow_error *error) { - /* 4096 is the maximum number of instructions for a BPF program */ + struct rss_key rss_entry = { }; + const uint8_t *key_in; + uint32_t hash_type = 0; unsigned int i; int err; - struct rss_key rss_entry = { .hash_fields = 0, - .key_size = 0 }; /* Check supported RSS features */ if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT) @@ -2087,6 +2104,41 @@ static int rss_add_actions(struct rte_flow *flow, struct pmd_internals *pmd, (error, ENOTSUP, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, "a nonzero RSS encapsulation level is not supported"); + if (rss->queue_num == 0 || rss->queue_num >= TAP_MAX_QUEUES) + return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "invalid number of queues"); + + /* allow RSS key_len 0 in case of NULL (default) RSS key. */ + if (rss->key_len == 0) { + if (rss->key != NULL) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + &rss->key_len, "RSS hash key length 0"); + key_in = rss_hash_default_key; + } else { + if (rss->key_len != TAP_RSS_HASH_KEY_SIZE) + return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "RSS hash invalid key length"); + if (rss->key == NULL) + return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "RSS hash key is NULL"); + key_in = rss->key; + } + + if (rss->types & TAP_RSS_HF_MASK) + return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "RSS hash type not supported"); + + if (rss->types & (RTE_ETH_RSS_NONFRAG_IPV4_UDP | RTE_ETH_RSS_NONFRAG_IPV4_TCP)) + hash_type |= RTE_BIT32(HASH_FIELD_IPV4_L3_L4); + else if (rss->types & (RTE_ETH_RSS_IPV4 | RTE_ETH_RSS_FRAG_IPV4)) + hash_type |= RTE_BIT32(HASH_FIELD_IPV4_L3); + + if (rss->types & (RTE_ETH_RSS_NONFRAG_IPV6_UDP | RTE_ETH_RSS_NONFRAG_IPV6_TCP)) + hash_type |= RTE_BIT32(HASH_FIELD_IPV6_L3_L4); + else if (rss->types & (RTE_ETH_RSS_IPV6 | RTE_ETH_RSS_FRAG_IPV6 | RTE_ETH_RSS_IPV6_EX)) + hash_type |= RTE_BIT32(HASH_FIELD_IPV6_L3); + /* Get a new map key for a new RSS rule */ err = bpf_rss_key(KEY_CMD_GET, &flow->key_idx); if (err < 0) { @@ -2101,8 +2153,11 @@ static int rss_add_actions(struct rte_flow *flow, struct pmd_internals *pmd, rss_entry.nb_queues = rss->queue_num; for (i = 0; i < rss->queue_num; i++) rss_entry.queues[i] = rss->queue[i]; - rss_entry.hash_fields = - (1 << HASH_FIELD_IPV4_L3_L4) | (1 << HASH_FIELD_IPV6_L3_L4); + + rss_entry.hash_fields = hash_type; + rte_convert_rss_key((const uint32_t *)key_in, (uint32_t *)rss_entry.key, + TAP_RSS_HASH_KEY_SIZE); + /* Add this RSS entry to map */ err = tap_flow_bpf_update_rss_elem(pmd->map_fd, diff --git a/drivers/net/tap/tap_rss.h b/drivers/net/tap/tap_rss.h index 8766ffc244f6..6009be7031b0 100644 --- a/drivers/net/tap/tap_rss.h +++ b/drivers/net/tap/tap_rss.h @@ -24,11 +24,10 @@ enum hash_field { }; struct rss_key { - __u8 key[128]; __u32 hash_fields; - __u32 key_size; - __u32 queues[TAP_MAX_QUEUES]; + __u8 key[TAP_RSS_HASH_KEY_SIZE]; __u32 nb_queues; + __u32 queues[TAP_MAX_QUEUES]; } __attribute__((packed)); #endif /* _TAP_RSS_H_ */ From patchwork Thu Feb 8 17:41:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 136552 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9226A43AE1; Thu, 8 Feb 2024 18:51:24 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D767242E4D; Thu, 8 Feb 2024 18:51:06 +0100 (CET) Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) by mails.dpdk.org (Postfix) with ESMTP id 5829242E04 for ; Thu, 8 Feb 2024 18:51:04 +0100 (CET) Received: by mail-pg1-f179.google.com with SMTP id 41be03b00d2f7-5ca29c131ebso10774a12.0 for ; Thu, 08 Feb 2024 09:51:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1707414663; x=1708019463; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mP60+k5ZPeJeBS5jRMFk72yyTGUxHj9BuzlrQpIYSFE=; b=sGubokjAFtMv3POfLf2uohj8DKq749GOvPVDJ1dWtc9oOq9hDJhxW6UMxqtJFjLTP6 f1H9nGegj3PXXx0IlLn7VNetdi50wF4+RputM73RDsa+UYELHYeWjegrXqh1ji2rBe6T FHwqdMP+7/C3wDKn8KyxmzoKl/TvjpkHX0uYj4lCkXaLyWAhNK57hdoxx47mIGtHivUJ lQQ/36r5KtoLtYCT+Zr1s39L6ehyL2R0wQaexRtmd4Rgsmj6DYR8OWiViaybwq6pGu2m 8tsu8lXhsmItQe0XAyljr/PEsTp2/FXDYjbP9wG32/KsJaP5/EYXcv9BsDNGDth0gn1F 4JlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707414663; x=1708019463; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mP60+k5ZPeJeBS5jRMFk72yyTGUxHj9BuzlrQpIYSFE=; b=vl1Bu9/fXQ2GU1QI+lTowE8GUlVzJNISJdnkmBheFS5i6z25niZ8EpRH6vWnc9KkN8 NmYnVPWMe5lsEOLAx7j1PW7dbPqQf5prKki24dYZHqVp32y7mPhfuvSKxUBX0eh6bvDR VDCNz9ba/n9+ILmE688LMycouKEFnFzpzODMtMcDev8tzFroo5Lf0nd0H4gwqxGNaRRj cv02VGZ6eJ7rvwTv36rC75qPmPXWJ+3og2e69Y3Zkh8TDAG+Jr0S9JBVW5zrJF0DND/P P9neyUESLj/IZnghL9V1+M4x9Mz0biPjc2luNHgH2PfT1ZBYwwkYyuQWaVEXuq4ZbsBY QKnA== X-Gm-Message-State: AOJu0Yyevpg8liTKb2JRpE1H5OQxa6XAnO3IHyt6o5U6y2R8oRllRbd3 BHnk/S93dbsJzkBpIKXdixobBGYn1Rla5nlzUs2b6dbanhrm/PcqKDQCKyphBMPj9VJteylr9tp s4N0= X-Google-Smtp-Source: AGHT+IGo5jDa8ezZXPxygnj6yIo4Na3PpgezcWRhe05JlzHSLfSaB5xVULfl1wzSEB7q7POWZXCNWg== X-Received: by 2002:a05:6a21:610:b0:19e:99ce:6f99 with SMTP id ll16-20020a056a21061000b0019e99ce6f99mr248546pzb.44.1707414663476; Thu, 08 Feb 2024 09:51:03 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f19-20020a056a00229300b006dbda7bcf3csm5030pfe.83.2024.02.08.09.51.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 09:51:03 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 3/7] tap: stop "vendoring" linux bpf headers Date: Thu, 8 Feb 2024 09:41:27 -0800 Message-ID: <20240208175051.326550-4-stephen@networkplumber.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240208175051.326550-1-stephen@networkplumber.org> References: <20240130034925.44869-1-stephen@networkplumber.org> <20240208175051.326550-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The proper place for finding bpf structures and functions is in linux/bpf.h. The original version was trying to workaround the case where the build environment was running on old pre BPF version of Glibc, but the target environment had BPF. This is not a supportable build method, and not how rest of DPDK works. Having own private (and divergent) version headers leads to future problems when BPF definitions evolve. Since DPDK officially supports only LTS or later kernel there is no need for the #ifdef workarounds in the TAP flow code. Cloning headers leads to problems and no longer needed. Signed-off-by: Stephen Hemminger --- drivers/net/tap/bpf/bpf_extract.py | 1 - drivers/net/tap/tap_bpf.h | 121 ----------------------------- drivers/net/tap/tap_bpf_api.c | 20 +++-- drivers/net/tap/tap_bpf_insns.h | 1 - drivers/net/tap/tap_flow.c | 89 --------------------- 5 files changed, 13 insertions(+), 219 deletions(-) delete mode 100644 drivers/net/tap/tap_bpf.h diff --git a/drivers/net/tap/bpf/bpf_extract.py b/drivers/net/tap/bpf/bpf_extract.py index b630c42b809f..73c4dafe4eca 100644 --- a/drivers/net/tap/bpf/bpf_extract.py +++ b/drivers/net/tap/bpf/bpf_extract.py @@ -65,7 +65,6 @@ def write_header(out, source): print(f' * Auto-generated from {source}', file=out) print(" * This not the original source file. Do NOT edit it.", file=out) print(" */\n", file=out) - print("#include ", file=out) def main(): diff --git a/drivers/net/tap/tap_bpf.h b/drivers/net/tap/tap_bpf.h deleted file mode 100644 index 0d38bc111fe0..000000000000 --- a/drivers/net/tap/tap_bpf.h +++ /dev/null @@ -1,121 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 - * Copyright 2017 Mellanox Technologies, Ltd - */ - -#ifndef __TAP_BPF_H__ -#define __TAP_BPF_H__ - -#include - -/* Do not #include since eBPF must compile on different - * distros which may include partial definitions for eBPF (while the - * kernel itself may support eBPF). Instead define here all that is needed - */ - -/* BPF_MAP_UPDATE_ELEM command flags */ -#define BPF_ANY 0 /* create a new element or update an existing */ - -/* BPF architecture instruction struct */ -struct bpf_insn { - __u8 code; - __u8 dst_reg:4; - __u8 src_reg:4; - __s16 off; - __s32 imm; /* immediate value */ -}; - -/* BPF program types */ -enum bpf_prog_type { - BPF_PROG_TYPE_UNSPEC, - BPF_PROG_TYPE_SOCKET_FILTER, - BPF_PROG_TYPE_KPROBE, - BPF_PROG_TYPE_SCHED_CLS, - BPF_PROG_TYPE_SCHED_ACT, -}; - -/* BPF commands types */ -enum bpf_cmd { - BPF_MAP_CREATE, - BPF_MAP_LOOKUP_ELEM, - BPF_MAP_UPDATE_ELEM, - BPF_MAP_DELETE_ELEM, - BPF_MAP_GET_NEXT_KEY, - BPF_PROG_LOAD, -}; - -/* BPF maps types */ -enum bpf_map_type { - BPF_MAP_TYPE_UNSPEC, - BPF_MAP_TYPE_HASH, -}; - -/* union of anonymous structs used with TAP BPF commands */ -union bpf_attr { - /* BPF_MAP_CREATE command */ - struct { - __u32 map_type; - __u32 key_size; - __u32 value_size; - __u32 max_entries; - __u32 map_flags; - __u32 inner_map_fd; - }; - - /* BPF_MAP_UPDATE_ELEM, BPF_MAP_DELETE_ELEM commands */ - struct { - __u32 map_fd; - __aligned_u64 key; - union { - __aligned_u64 value; - __aligned_u64 next_key; - }; - __u64 flags; - }; - - /* BPF_PROG_LOAD command */ - struct { - __u32 prog_type; - __u32 insn_cnt; - __aligned_u64 insns; - __aligned_u64 license; - __u32 log_level; - __u32 log_size; - __aligned_u64 log_buf; - __u32 kern_version; - __u32 prog_flags; - }; -} __rte_aligned(8); - -#ifndef __NR_bpf -# if defined(__i386__) -# define __NR_bpf 357 -# elif defined(__x86_64__) -# define __NR_bpf 321 -# elif defined(__arm__) -# define __NR_bpf 386 -# elif defined(__aarch64__) -# define __NR_bpf 280 -# elif defined(__sparc__) -# define __NR_bpf 349 -# elif defined(__s390__) -# define __NR_bpf 351 -# elif defined(__powerpc__) -# define __NR_bpf 361 -# elif defined(__riscv) -# define __NR_bpf 280 -# elif defined(__loongarch__) -# define __NR_bpf 280 -# else -# error __NR_bpf not defined -# endif -#endif - -enum { - BPF_MAP_ID_KEY, - BPF_MAP_ID_SIMPLE, -}; - -static int bpf_load(enum bpf_prog_type type, const struct bpf_insn *insns, - size_t insns_cnt, const char *license); - -#endif /* __TAP_BPF_H__ */ diff --git a/drivers/net/tap/tap_bpf_api.c b/drivers/net/tap/tap_bpf_api.c index 15283f8917ed..9e05e2ddf19b 100644 --- a/drivers/net/tap/tap_bpf_api.c +++ b/drivers/net/tap/tap_bpf_api.c @@ -2,19 +2,19 @@ * Copyright 2017 Mellanox Technologies, Ltd */ -#include -#include #include -#include +#include +#include -#include -#include #include #include -#include -#include + #include + +static int bpf_load(enum bpf_prog_type type, const struct bpf_insn *insns, + size_t insns_cnt, const char *license); + /** * Load BPF program (section cls_q) into the kernel and return a bpf fd * @@ -89,7 +89,13 @@ static inline __u64 ptr_to_u64(const void *ptr) static inline int sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr, unsigned int size) { +#ifdef __NR_bpf return syscall(__NR_bpf, cmd, attr, size); +#else + TAP_LOG(ERR, "No bpf syscall, kernel headers too old?\n"); + errno = ENOSYS; + return -1; +#endif } /** diff --git a/drivers/net/tap/tap_bpf_insns.h b/drivers/net/tap/tap_bpf_insns.h index ee26cf885ed7..b59b7e9141fd 100644 --- a/drivers/net/tap/tap_bpf_insns.h +++ b/drivers/net/tap/tap_bpf_insns.h @@ -3,7 +3,6 @@ * This not the original source file. Do NOT edit it. */ -#include static struct bpf_insn cls_q_insns[] = { {0x61, 2, 1, 52, 0x00000000}, diff --git a/drivers/net/tap/tap_flow.c b/drivers/net/tap/tap_flow.c index cd49aa51c8b0..94436af55ce8 100644 --- a/drivers/net/tap/tap_flow.c +++ b/drivers/net/tap/tap_flow.c @@ -20,95 +20,6 @@ #include #include -#ifndef HAVE_TC_FLOWER -/* - * For kernels < 4.2, this enum is not defined. Runtime checks will be made to - * avoid sending TC messages the kernel cannot understand. - */ -enum { - TCA_FLOWER_UNSPEC, - TCA_FLOWER_CLASSID, - TCA_FLOWER_INDEV, - TCA_FLOWER_ACT, - TCA_FLOWER_KEY_ETH_DST, /* ETH_ALEN */ - TCA_FLOWER_KEY_ETH_DST_MASK, /* ETH_ALEN */ - TCA_FLOWER_KEY_ETH_SRC, /* ETH_ALEN */ - TCA_FLOWER_KEY_ETH_SRC_MASK, /* ETH_ALEN */ - TCA_FLOWER_KEY_ETH_TYPE, /* be16 */ - TCA_FLOWER_KEY_IP_PROTO, /* u8 */ - TCA_FLOWER_KEY_IPV4_SRC, /* be32 */ - TCA_FLOWER_KEY_IPV4_SRC_MASK, /* be32 */ - TCA_FLOWER_KEY_IPV4_DST, /* be32 */ - TCA_FLOWER_KEY_IPV4_DST_MASK, /* be32 */ - TCA_FLOWER_KEY_IPV6_SRC, /* struct in6_addr */ - TCA_FLOWER_KEY_IPV6_SRC_MASK, /* struct in6_addr */ - TCA_FLOWER_KEY_IPV6_DST, /* struct in6_addr */ - TCA_FLOWER_KEY_IPV6_DST_MASK, /* struct in6_addr */ - TCA_FLOWER_KEY_TCP_SRC, /* be16 */ - TCA_FLOWER_KEY_TCP_DST, /* be16 */ - TCA_FLOWER_KEY_UDP_SRC, /* be16 */ - TCA_FLOWER_KEY_UDP_DST, /* be16 */ -}; -#endif -#ifndef HAVE_TC_VLAN_ID -enum { - /* TCA_FLOWER_FLAGS, */ - TCA_FLOWER_KEY_VLAN_ID = TCA_FLOWER_KEY_UDP_DST + 2, /* be16 */ - TCA_FLOWER_KEY_VLAN_PRIO, /* u8 */ - TCA_FLOWER_KEY_VLAN_ETH_TYPE, /* be16 */ -}; -#endif -/* - * For kernels < 4.2 BPF related enums may not be defined. - * Runtime checks will be carried out to gracefully report on TC messages that - * are rejected by the kernel. Rejection reasons may be due to: - * 1. enum is not defined - * 2. enum is defined but kernel is not configured to support BPF system calls, - * BPF classifications or BPF actions. - */ -#ifndef HAVE_TC_BPF -enum { - TCA_BPF_UNSPEC, - TCA_BPF_ACT, - TCA_BPF_POLICE, - TCA_BPF_CLASSID, - TCA_BPF_OPS_LEN, - TCA_BPF_OPS, -}; -#endif -#ifndef HAVE_TC_BPF_FD -enum { - TCA_BPF_FD = TCA_BPF_OPS + 1, - TCA_BPF_NAME, -}; -#endif -#ifndef HAVE_TC_ACT_BPF -#define tc_gen \ - __u32 index; \ - __u32 capab; \ - int action; \ - int refcnt; \ - int bindcnt - -struct tc_act_bpf { - tc_gen; -}; - -enum { - TCA_ACT_BPF_UNSPEC, - TCA_ACT_BPF_TM, - TCA_ACT_BPF_PARMS, - TCA_ACT_BPF_OPS_LEN, - TCA_ACT_BPF_OPS, -}; - -#endif -#ifndef HAVE_TC_ACT_BPF_FD -enum { - TCA_ACT_BPF_FD = TCA_ACT_BPF_OPS + 1, - TCA_ACT_BPF_NAME, -}; -#endif /* RSS key management */ enum bpf_rss_key_e { From patchwork Thu Feb 8 17:41:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 136553 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9578743AE1; Thu, 8 Feb 2024 18:51:31 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 116F342E5E; Thu, 8 Feb 2024 18:51:08 +0100 (CET) Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by mails.dpdk.org (Postfix) with ESMTP id 65C7242E2F for ; Thu, 8 Feb 2024 18:51:05 +0100 (CET) Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6e053b272b0so64505b3a.1 for ; Thu, 08 Feb 2024 09:51:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1707414664; x=1708019464; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7Gx7j0Tm+E5EGVxQHJmo1BaRxu4F0CgpHFYnPF9VzD4=; b=HPaZ4khlOG7OG719DYvj+pY5Krjv13UeXbAjLqzRHj55vRiK9w3zsaAasMQ1Hd9u/J eFZPBEXTrnpvKZsbjQ/aS+gszHXlq4ynxpUHeI5HN/vGC7KOef2688upcd4eZXZqqo8y +bhIKaSKyYp+GduIiTky7/pJHiTiTeJrwT1WkgIyVmDmzQjrhwEw1fRaxKJjW6eAQZ8H q5/nywyGpolKkf/qG96bZzL0a+pTUvR0kKwtMUretYaLqhCrlAvTYnmhms/nYY0eNqnC 4LpTWQX7PWBV40wPQRxwuKB+N5X32s2EkvKCRRxeEBnLI49v8nuY8CfOwvl+clL1W+F9 eUHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707414664; x=1708019464; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7Gx7j0Tm+E5EGVxQHJmo1BaRxu4F0CgpHFYnPF9VzD4=; b=CIilrGHQfFp2TIlnFb7dpBIwQwXu9dliW4023vMue7FDiMRs2O1R6hLmnsxqindVpK RIPvwU2TxjPHqhreU3wRQVujc3VpSOCj3JIXTGJxPuGSdQKcSNOhw048gGvEiVIKJZJ8 IXzfow+Wxg2SYELsqfKFy54KqRRtfenlliL/PTwCQECZUQ1x9UX9CMsy5HN16npfB+6X S+fcalB/u091jSRPCSNvFxoCIAkzUHa6vojaTqQQvOBZlXzXjwpbryGImO7arlkzJ8NJ OJNpqbcDkwyMypXW7rSQmlFZH2AEX+wjbtoinx0WPibvYjtRADw4gOHSpk/C3tn2l6Jg jxZQ== X-Gm-Message-State: AOJu0YwAPc6lUv5nPnyHH6gkYPYkE+1ced4qHs0qPAvT4iCusy4St5sP YU+VvFZI9TlhqQyzd22o6e73o2D2qjG+AOsb4pWDqXhRNbIRe1cPzPFcuG3LovaNUiL7EoyqNa7 Ghzo= X-Google-Smtp-Source: AGHT+IENt2JlRmJY8GhJR4qNQKEOK6waA8uA4Wl0fD6n+GL0azY5d1DJdybnuaSOYwJK4MstJ4qVFQ== X-Received: by 2002:a05:6a20:a104:b0:19e:4c43:3aa1 with SMTP id q4-20020a056a20a10400b0019e4c433aa1mr353378pzk.39.1707414664443; Thu, 08 Feb 2024 09:51:04 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f19-20020a056a00229300b006dbda7bcf3csm5030pfe.83.2024.02.08.09.51.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 09:51:04 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 4/7] net/tap: rewrite the RSS BPF program Date: Thu, 8 Feb 2024 09:41:28 -0800 Message-ID: <20240208175051.326550-5-stephen@networkplumber.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240208175051.326550-1-stephen@networkplumber.org> References: <20240130034925.44869-1-stephen@networkplumber.org> <20240208175051.326550-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Rewrite the BPF program used to do queue based RSS. Important changes: - uses newer BPF map format BTF - accepts key as parameter rather than constant default - can do L3 or L4 hashing - supports IPv4 options - supports IPv6 extension headers - restructured for readability The usage of BPF is different as well: - the incoming configuration is looked up based on class parameters rather than patching the BPF. - the resulting queue is placed in skb rather than requiring a second pass through classifier step. Note: This version only works with later patch to enable it on the DPDK driver side. It is submitted as an incremental patch to allow for easier review. Bisection still works because the old instruction are still present for now. Signed-off-by: Stephen Hemminger --- .gitignore | 3 - drivers/net/tap/bpf/Makefile | 19 -- drivers/net/tap/bpf/README | 12 ++ drivers/net/tap/bpf/bpf_api.h | 276 -------------------------- drivers/net/tap/bpf/bpf_elf.h | 53 ----- drivers/net/tap/bpf/bpf_extract.py | 85 -------- drivers/net/tap/bpf/meson.build | 81 ++++++++ drivers/net/tap/bpf/tap_bpf_program.c | 255 ------------------------ drivers/net/tap/bpf/tap_rss.c | 272 +++++++++++++++++++++++++ 9 files changed, 365 insertions(+), 691 deletions(-) delete mode 100644 drivers/net/tap/bpf/Makefile create mode 100644 drivers/net/tap/bpf/README delete mode 100644 drivers/net/tap/bpf/bpf_api.h delete mode 100644 drivers/net/tap/bpf/bpf_elf.h delete mode 100644 drivers/net/tap/bpf/bpf_extract.py create mode 100644 drivers/net/tap/bpf/meson.build delete mode 100644 drivers/net/tap/bpf/tap_bpf_program.c create mode 100644 drivers/net/tap/bpf/tap_rss.c diff --git a/.gitignore b/.gitignore index 3f444dcace2e..01a47a760660 100644 --- a/.gitignore +++ b/.gitignore @@ -36,9 +36,6 @@ TAGS # ignore python bytecode files *.pyc -# ignore BPF programs -drivers/net/tap/bpf/tap_bpf_program.o - # DTS results dts/output diff --git a/drivers/net/tap/bpf/Makefile b/drivers/net/tap/bpf/Makefile deleted file mode 100644 index 9efeeb1bc704..000000000000 --- a/drivers/net/tap/bpf/Makefile +++ /dev/null @@ -1,19 +0,0 @@ -# SPDX-License-Identifier: BSD-3-Clause -# This file is not built as part of normal DPDK build. -# It is used to generate the eBPF code for TAP RSS. - -CLANG=clang -CLANG_OPTS=-O2 -TARGET=../tap_bpf_insns.h - -all: $(TARGET) - -clean: - rm tap_bpf_program.o $(TARGET) - -tap_bpf_program.o: tap_bpf_program.c - $(CLANG) $(CLANG_OPTS) -emit-llvm -c $< -o - | \ - llc -march=bpf -filetype=obj -o $@ - -$(TARGET): tap_bpf_program.o - python3 bpf_extract.py -stap_bpf_program.c -o $@ $< diff --git a/drivers/net/tap/bpf/README b/drivers/net/tap/bpf/README new file mode 100644 index 000000000000..960a10da73b8 --- /dev/null +++ b/drivers/net/tap/bpf/README @@ -0,0 +1,12 @@ +This is the BPF program used to implement the RSS across queues +flow action. It works like the skbedit tc filter but instead of mapping +to only one queues, it maps to multiple queues based on RSS hash. + +This version is built the BPF Compile Once — Run Everywhere (CO-RE) +framework and uses libbpf and bpftool. + +Limitations +- requires libbpf version XX or later +- rebuilding the BPF requires clang and bpftool +- only Toeplitz hash with standard 40 byte key is supported +- the number of queues per RSS action is limited to 16 diff --git a/drivers/net/tap/bpf/bpf_api.h b/drivers/net/tap/bpf/bpf_api.h deleted file mode 100644 index 2638a8a4ac9a..000000000000 --- a/drivers/net/tap/bpf/bpf_api.h +++ /dev/null @@ -1,276 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 or BSD-3-Clause */ - -#ifndef __BPF_API__ -#define __BPF_API__ - -/* Note: - * - * This file can be included into eBPF kernel programs. It contains - * a couple of useful helper functions, map/section ABI (bpf_elf.h), - * misc macros and some eBPF specific LLVM built-ins. - */ - -#include - -#include -#include -#include - -#include - -#include "bpf_elf.h" - -/** libbpf pin type. */ -enum libbpf_pin_type { - LIBBPF_PIN_NONE, - /* PIN_BY_NAME: pin maps by name (in /sys/fs/bpf by default) */ - LIBBPF_PIN_BY_NAME, -}; - -/** Type helper macros. */ - -#define __uint(name, val) int (*name)[val] -#define __type(name, val) typeof(val) *name -#define __array(name, val) typeof(val) *name[] - -/** Misc macros. */ - -#ifndef __stringify -# define __stringify(X) #X -#endif - -#ifndef __maybe_unused -# define __maybe_unused __attribute__((__unused__)) -#endif - -#ifndef offsetof -# define offsetof(TYPE, MEMBER) __builtin_offsetof(TYPE, MEMBER) -#endif - -#ifndef likely -# define likely(X) __builtin_expect(!!(X), 1) -#endif - -#ifndef unlikely -# define unlikely(X) __builtin_expect(!!(X), 0) -#endif - -#ifndef htons -# define htons(X) __constant_htons((X)) -#endif - -#ifndef ntohs -# define ntohs(X) __constant_ntohs((X)) -#endif - -#ifndef htonl -# define htonl(X) __constant_htonl((X)) -#endif - -#ifndef ntohl -# define ntohl(X) __constant_ntohl((X)) -#endif - -#ifndef __inline__ -# define __inline__ __attribute__((always_inline)) -#endif - -/** Section helper macros. */ - -#ifndef __section -# define __section(NAME) \ - __attribute__((section(NAME), used)) -#endif - -#ifndef __section_tail -# define __section_tail(ID, KEY) \ - __section(__stringify(ID) "/" __stringify(KEY)) -#endif - -#ifndef __section_xdp_entry -# define __section_xdp_entry \ - __section(ELF_SECTION_PROG) -#endif - -#ifndef __section_cls_entry -# define __section_cls_entry \ - __section(ELF_SECTION_CLASSIFIER) -#endif - -#ifndef __section_act_entry -# define __section_act_entry \ - __section(ELF_SECTION_ACTION) -#endif - -#ifndef __section_lwt_entry -# define __section_lwt_entry \ - __section(ELF_SECTION_PROG) -#endif - -#ifndef __section_license -# define __section_license \ - __section(ELF_SECTION_LICENSE) -#endif - -#ifndef __section_maps -# define __section_maps \ - __section(ELF_SECTION_MAPS) -#endif - -/** Declaration helper macros. */ - -#ifndef BPF_LICENSE -# define BPF_LICENSE(NAME) \ - char ____license[] __section_license = NAME -#endif - -/** Classifier helper */ - -#ifndef BPF_H_DEFAULT -# define BPF_H_DEFAULT -1 -#endif - -/** BPF helper functions for tc. Individual flags are in linux/bpf.h */ - -#ifndef __BPF_FUNC -# define __BPF_FUNC(NAME, ...) \ - (* NAME)(__VA_ARGS__) __maybe_unused -#endif - -#ifndef BPF_FUNC -# define BPF_FUNC(NAME, ...) \ - __BPF_FUNC(NAME, __VA_ARGS__) = (void *) BPF_FUNC_##NAME -#endif - -/* Map access/manipulation */ -static void *BPF_FUNC(map_lookup_elem, void *map, const void *key); -static int BPF_FUNC(map_update_elem, void *map, const void *key, - const void *value, uint32_t flags); -static int BPF_FUNC(map_delete_elem, void *map, const void *key); - -/* Time access */ -static uint64_t BPF_FUNC(ktime_get_ns); - -/* Debugging */ - -/* FIXME: __attribute__ ((format(printf, 1, 3))) not possible unless - * llvm bug https://llvm.org/bugs/show_bug.cgi?id=26243 gets resolved. - * It would require ____fmt to be made const, which generates a reloc - * entry (non-map). - */ -static void BPF_FUNC(trace_printk, const char *fmt, int fmt_size, ...); - -#ifndef printt -# define printt(fmt, ...) \ - ({ \ - char ____fmt[] = fmt; \ - trace_printk(____fmt, sizeof(____fmt), ##__VA_ARGS__); \ - }) -#endif - -/* Random numbers */ -static uint32_t BPF_FUNC(get_prandom_u32); - -/* Tail calls */ -static void BPF_FUNC(tail_call, struct __sk_buff *skb, void *map, - uint32_t index); - -/* System helpers */ -static uint32_t BPF_FUNC(get_smp_processor_id); -static uint32_t BPF_FUNC(get_numa_node_id); - -/* Packet misc meta data */ -static uint32_t BPF_FUNC(get_cgroup_classid, struct __sk_buff *skb); -static int BPF_FUNC(skb_under_cgroup, void *map, uint32_t index); - -static uint32_t BPF_FUNC(get_route_realm, struct __sk_buff *skb); -static uint32_t BPF_FUNC(get_hash_recalc, struct __sk_buff *skb); -static uint32_t BPF_FUNC(set_hash_invalid, struct __sk_buff *skb); - -/* Packet redirection */ -static int BPF_FUNC(redirect, int ifindex, uint32_t flags); -static int BPF_FUNC(clone_redirect, struct __sk_buff *skb, int ifindex, - uint32_t flags); - -/* Packet manipulation */ -static int BPF_FUNC(skb_load_bytes, struct __sk_buff *skb, uint32_t off, - void *to, uint32_t len); -static int BPF_FUNC(skb_store_bytes, struct __sk_buff *skb, uint32_t off, - const void *from, uint32_t len, uint32_t flags); - -static int BPF_FUNC(l3_csum_replace, struct __sk_buff *skb, uint32_t off, - uint32_t from, uint32_t to, uint32_t flags); -static int BPF_FUNC(l4_csum_replace, struct __sk_buff *skb, uint32_t off, - uint32_t from, uint32_t to, uint32_t flags); -static int BPF_FUNC(csum_diff, const void *from, uint32_t from_size, - const void *to, uint32_t to_size, uint32_t seed); -static int BPF_FUNC(csum_update, struct __sk_buff *skb, uint32_t wsum); - -static int BPF_FUNC(skb_change_type, struct __sk_buff *skb, uint32_t type); -static int BPF_FUNC(skb_change_proto, struct __sk_buff *skb, uint32_t proto, - uint32_t flags); -static int BPF_FUNC(skb_change_tail, struct __sk_buff *skb, uint32_t nlen, - uint32_t flags); - -static int BPF_FUNC(skb_pull_data, struct __sk_buff *skb, uint32_t len); - -/* Event notification */ -static int __BPF_FUNC(skb_event_output, struct __sk_buff *skb, void *map, - uint64_t index, const void *data, uint32_t size) = - (void *) BPF_FUNC_perf_event_output; - -/* Packet vlan encap/decap */ -static int BPF_FUNC(skb_vlan_push, struct __sk_buff *skb, uint16_t proto, - uint16_t vlan_tci); -static int BPF_FUNC(skb_vlan_pop, struct __sk_buff *skb); - -/* Packet tunnel encap/decap */ -static int BPF_FUNC(skb_get_tunnel_key, struct __sk_buff *skb, - struct bpf_tunnel_key *to, uint32_t size, uint32_t flags); -static int BPF_FUNC(skb_set_tunnel_key, struct __sk_buff *skb, - const struct bpf_tunnel_key *from, uint32_t size, - uint32_t flags); - -static int BPF_FUNC(skb_get_tunnel_opt, struct __sk_buff *skb, - void *to, uint32_t size); -static int BPF_FUNC(skb_set_tunnel_opt, struct __sk_buff *skb, - const void *from, uint32_t size); - -/** LLVM built-ins, mem*() routines work for constant size */ - -#ifndef lock_xadd -# define lock_xadd(ptr, val) ((void) __sync_fetch_and_add(ptr, val)) -#endif - -#ifndef memset -# define memset(s, c, n) __builtin_memset((s), (c), (n)) -#endif - -#ifndef memcpy -# define memcpy(d, s, n) __builtin_memcpy((d), (s), (n)) -#endif - -#ifndef memmove -# define memmove(d, s, n) __builtin_memmove((d), (s), (n)) -#endif - -/* FIXME: __builtin_memcmp() is not yet fully usable unless llvm bug - * https://llvm.org/bugs/show_bug.cgi?id=26218 gets resolved. Also - * this one would generate a reloc entry (non-map), otherwise. - */ -#if 0 -#ifndef memcmp -# define memcmp(a, b, n) __builtin_memcmp((a), (b), (n)) -#endif -#endif - -unsigned long long load_byte(void *skb, unsigned long long off) - asm ("llvm.bpf.load.byte"); - -unsigned long long load_half(void *skb, unsigned long long off) - asm ("llvm.bpf.load.half"); - -unsigned long long load_word(void *skb, unsigned long long off) - asm ("llvm.bpf.load.word"); - -#endif /* __BPF_API__ */ diff --git a/drivers/net/tap/bpf/bpf_elf.h b/drivers/net/tap/bpf/bpf_elf.h deleted file mode 100644 index ea8a11c95c0f..000000000000 --- a/drivers/net/tap/bpf/bpf_elf.h +++ /dev/null @@ -1,53 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 or BSD-3-Clause */ -#ifndef __BPF_ELF__ -#define __BPF_ELF__ - -#include - -/* Note: - * - * Below ELF section names and bpf_elf_map structure definition - * are not (!) kernel ABI. It's rather a "contract" between the - * application and the BPF loader in tc. For compatibility, the - * section names should stay as-is. Introduction of aliases, if - * needed, are a possibility, though. - */ - -/* ELF section names, etc */ -#define ELF_SECTION_LICENSE "license" -#define ELF_SECTION_MAPS "maps" -#define ELF_SECTION_PROG "prog" -#define ELF_SECTION_CLASSIFIER "classifier" -#define ELF_SECTION_ACTION "action" - -#define ELF_MAX_MAPS 64 -#define ELF_MAX_LICENSE_LEN 128 - -/* Object pinning settings */ -#define PIN_NONE 0 -#define PIN_OBJECT_NS 1 -#define PIN_GLOBAL_NS 2 - -/* ELF map definition */ -struct bpf_elf_map { - __u32 type; - __u32 size_key; - __u32 size_value; - __u32 max_elem; - __u32 flags; - __u32 id; - __u32 pinning; - __u32 inner_id; - __u32 inner_idx; -}; - -#define BPF_ANNOTATE_KV_PAIR(name, type_key, type_val) \ - struct ____btf_map_##name { \ - type_key key; \ - type_val value; \ - }; \ - struct ____btf_map_##name \ - __attribute__ ((section(".maps." #name), used)) \ - ____btf_map_##name = { } - -#endif /* __BPF_ELF__ */ diff --git a/drivers/net/tap/bpf/bpf_extract.py b/drivers/net/tap/bpf/bpf_extract.py deleted file mode 100644 index 73c4dafe4eca..000000000000 --- a/drivers/net/tap/bpf/bpf_extract.py +++ /dev/null @@ -1,85 +0,0 @@ -#!/usr/bin/env python3 -# SPDX-License-Identifier: BSD-3-Clause -# Copyright (c) 2023 Stephen Hemminger - -import argparse -import sys -import struct -from tempfile import TemporaryFile -from elftools.elf.elffile import ELFFile - - -def load_sections(elffile): - """Get sections of interest from ELF""" - result = [] - parts = [("cls_q", "cls_q_insns"), ("l3_l4", "l3_l4_hash_insns")] - for name, tag in parts: - section = elffile.get_section_by_name(name) - if section: - insns = struct.iter_unpack('> 4 - dst = bpf[1] & 0xf - off = bpf[2] - imm = bpf[3] - print(f'\t{{{code:#04x}, {dst:4d}, {src:4d}, {off:8d}, {imm:#010x}}},', - file=out) - print('};', file=out) - - -def parse_args(): - """Parse command line arguments""" - parser = argparse.ArgumentParser() - parser.add_argument('-s', - '--source', - type=str, - help="original source file") - parser.add_argument('-o', '--out', type=str, help="output C file path") - parser.add_argument("file", - nargs='+', - help="object file path or '-' for stdin") - return parser.parse_args() - - -def open_input(path): - """Open the file or stdin""" - if path == "-": - temp = TemporaryFile() - temp.write(sys.stdin.buffer.read()) - return temp - return open(path, 'rb') - - -def write_header(out, source): - """Write file intro header""" - print("/* SPDX-License-Identifier: BSD-3-Clause", file=out) - if source: - print(f' * Auto-generated from {source}', file=out) - print(" * This not the original source file. Do NOT edit it.", file=out) - print(" */\n", file=out) - - -def main(): - '''program main function''' - args = parse_args() - - with open(args.out, 'w', - encoding="utf-8") if args.out else sys.stdout as out: - write_header(out, args.source) - for path in args.file: - elffile = ELFFile(open_input(path)) - sections = load_sections(elffile) - for name, insns in sections: - dump_section(name, insns, out) - - -if __name__ == "__main__": - main() diff --git a/drivers/net/tap/bpf/meson.build b/drivers/net/tap/bpf/meson.build new file mode 100644 index 000000000000..f2c03a19fd4d --- /dev/null +++ b/drivers/net/tap/bpf/meson.build @@ -0,0 +1,81 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2024 Stephen Hemminger + +enable_tap_rss = false + +libbpf = dependency('libbpf', required: false, method: 'pkg-config') +if not libbpf.found() + message('net/tap: no RSS support missing libbpf') + subdir_done() +endif + +# Debian install this in /usr/sbin which is not in $PATH +bpftool = find_program('bpftool', '/usr/sbin/bpftool', required: false, version: '>= 5.6.0') +if not bpftool.found() + message('net/tap: no RSS support missing bpftool') + subdir_done() +endif + +clang_supports_bpf = false +clang = find_program('clang', required: false) +if clang.found() + clang_supports_bpf = run_command(clang, '-target', 'bpf', '--print-supported-cpus', + check: false).returncode() == 0 +endif + +if not clang_supports_bpf + message('net/tap: no RSS support missing clang BPF') + subdir_done() +endif + +enable_tap_rss = true + +libbpf_include_dir = libbpf.get_variable(pkgconfig : 'includedir') + +# The include files and others include +# but is not defined for multi-lib environment target. +# Workaround by using include directoriy from the host build environment. +machine_name = run_command('uname', '-m').stdout().strip() +march_include_dir = '/usr/include/' + machine_name + '-linux-gnu' + +clang_flags = [ + '-O2', + '-Wall', + '-Wextra', + '-target', + 'bpf', + '-g', + '-c', +] + +bpf_o_cmd = [ + clang, + clang_flags, + '-idirafter', + libbpf_include_dir, + '-idirafter', + march_include_dir, + '@INPUT@', + '-o', + '@OUTPUT@' +] + +skel_h_cmd = [ + bpftool, + 'gen', + 'skeleton', + '@INPUT@' +] + +tap_rss_o = custom_target( + 'tap_rss.bpf.o', + input: 'tap_rss.c', + output: 'tap_rss.o', + command: bpf_o_cmd) + +tap_rss_skel_h = custom_target( + 'tap_rss.skel.h', + input: tap_rss_o, + output: 'tap_rss.skel.h', + command: skel_h_cmd, + capture: true) diff --git a/drivers/net/tap/bpf/tap_bpf_program.c b/drivers/net/tap/bpf/tap_bpf_program.c deleted file mode 100644 index f05aed021c30..000000000000 --- a/drivers/net/tap/bpf/tap_bpf_program.c +++ /dev/null @@ -1,255 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 - * Copyright 2017 Mellanox Technologies, Ltd - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "bpf_api.h" -#include "bpf_elf.h" -#include "../tap_rss.h" - -/** Create IPv4 address */ -#define IPv4(a, b, c, d) ((__u32)(((a) & 0xff) << 24) | \ - (((b) & 0xff) << 16) | \ - (((c) & 0xff) << 8) | \ - ((d) & 0xff)) - -#define PORT(a, b) ((__u16)(((a) & 0xff) << 8) | \ - ((b) & 0xff)) - -/* - * The queue number is offset by a unique QUEUE_OFFSET, to distinguish - * packets that have gone through this rule (skb->cb[1] != 0) from others. - */ -#define QUEUE_OFFSET 0x7cafe800 -#define PIN_GLOBAL_NS 2 - -#define KEY_IDX 0 -#define BPF_MAP_ID_KEY 1 - -struct vlan_hdr { - __be16 proto; - __be16 tci; -}; - -struct bpf_elf_map __attribute__((section("maps"), used)) -map_keys = { - .type = BPF_MAP_TYPE_HASH, - .id = BPF_MAP_ID_KEY, - .size_key = sizeof(__u32), - .size_value = sizeof(struct rss_key), - .max_elem = 256, - .pinning = PIN_GLOBAL_NS, -}; - -__section("cls_q") int -match_q(struct __sk_buff *skb) -{ - __u32 queue = skb->cb[1]; - /* queue is set by tap_flow_bpf_cls_q() before load */ - volatile __u32 q = 0xdeadbeef; - __u32 match_queue = QUEUE_OFFSET + q; - - /* printt("match_q$i() queue = %d\n", queue); */ - - if (queue != match_queue) - return TC_ACT_OK; - - /* queue match */ - skb->cb[1] = 0; - return TC_ACT_UNSPEC; -} - - -struct ipv4_l3_l4_tuple { - __u32 src_addr; - __u32 dst_addr; - __u16 dport; - __u16 sport; -} __attribute__((packed)); - -struct ipv6_l3_l4_tuple { - __u8 src_addr[16]; - __u8 dst_addr[16]; - __u16 dport; - __u16 sport; -} __attribute__((packed)); - -static const __u8 def_rss_key[TAP_RSS_HASH_KEY_SIZE] = { - 0xd1, 0x81, 0xc6, 0x2c, - 0xf7, 0xf4, 0xdb, 0x5b, - 0x19, 0x83, 0xa2, 0xfc, - 0x94, 0x3e, 0x1a, 0xdb, - 0xd9, 0x38, 0x9e, 0x6b, - 0xd1, 0x03, 0x9c, 0x2c, - 0xa7, 0x44, 0x99, 0xad, - 0x59, 0x3d, 0x56, 0xd9, - 0xf3, 0x25, 0x3c, 0x06, - 0x2a, 0xdc, 0x1f, 0xfc, -}; - -static __u32 __attribute__((always_inline)) -rte_softrss_be(const __u32 *input_tuple, const uint8_t *rss_key, - __u8 input_len) -{ - __u32 i, j, hash = 0; -#pragma unroll - for (j = 0; j < input_len; j++) { -#pragma unroll - for (i = 0; i < 32; i++) { - if (input_tuple[j] & (1U << (31 - i))) { - hash ^= ((const __u32 *)def_rss_key)[j] << i | - (__u32)((uint64_t) - (((const __u32 *)def_rss_key)[j + 1]) - >> (32 - i)); - } - } - } - return hash; -} - -static int __attribute__((always_inline)) -rss_l3_l4(struct __sk_buff *skb) -{ - void *data_end = (void *)(long)skb->data_end; - void *data = (void *)(long)skb->data; - __u16 proto = (__u16)skb->protocol; - __u32 key_idx = 0xdeadbeef; - __u32 hash; - struct rss_key *rsskey; - __u64 off = ETH_HLEN; - int j; - __u8 *key = 0; - __u32 len; - __u32 queue = 0; - bool mf = 0; - __u16 frag_off = 0; - - rsskey = map_lookup_elem(&map_keys, &key_idx); - if (!rsskey) { - printt("hash(): rss key is not configured\n"); - return TC_ACT_OK; - } - key = (__u8 *)rsskey->key; - - /* Get correct proto for 802.1ad */ - if (skb->vlan_present && skb->vlan_proto == htons(ETH_P_8021AD)) { - if (data + ETH_ALEN * 2 + sizeof(struct vlan_hdr) + - sizeof(proto) > data_end) - return TC_ACT_OK; - proto = *(__u16 *)(data + ETH_ALEN * 2 + - sizeof(struct vlan_hdr)); - off += sizeof(struct vlan_hdr); - } - - if (proto == htons(ETH_P_IP)) { - if (data + off + sizeof(struct iphdr) + sizeof(__u32) - > data_end) - return TC_ACT_OK; - - __u8 *src_dst_addr = data + off + offsetof(struct iphdr, saddr); - __u8 *frag_off_addr = data + off + offsetof(struct iphdr, frag_off); - __u8 *prot_addr = data + off + offsetof(struct iphdr, protocol); - __u8 *src_dst_port = data + off + sizeof(struct iphdr); - struct ipv4_l3_l4_tuple v4_tuple = { - .src_addr = IPv4(*(src_dst_addr + 0), - *(src_dst_addr + 1), - *(src_dst_addr + 2), - *(src_dst_addr + 3)), - .dst_addr = IPv4(*(src_dst_addr + 4), - *(src_dst_addr + 5), - *(src_dst_addr + 6), - *(src_dst_addr + 7)), - .sport = 0, - .dport = 0, - }; - /** Fetch the L4-payer port numbers only in-case of TCP/UDP - ** and also if the packet is not fragmented. Since fragmented - ** chunks do not have L4 TCP/UDP header. - **/ - if (*prot_addr == IPPROTO_UDP || *prot_addr == IPPROTO_TCP) { - frag_off = PORT(*(frag_off_addr + 0), - *(frag_off_addr + 1)); - mf = frag_off & 0x2000; - frag_off = frag_off & 0x1fff; - if (mf == 0 && frag_off == 0) { - v4_tuple.sport = PORT(*(src_dst_port + 0), - *(src_dst_port + 1)); - v4_tuple.dport = PORT(*(src_dst_port + 2), - *(src_dst_port + 3)); - } - } - __u8 input_len = sizeof(v4_tuple) / sizeof(__u32); - if (rsskey->hash_fields & (1 << HASH_FIELD_IPV4_L3)) - input_len--; - hash = rte_softrss_be((__u32 *)&v4_tuple, key, 3); - } else if (proto == htons(ETH_P_IPV6)) { - if (data + off + sizeof(struct ipv6hdr) + - sizeof(__u32) > data_end) - return TC_ACT_OK; - __u8 *src_dst_addr = data + off + - offsetof(struct ipv6hdr, saddr); - __u8 *src_dst_port = data + off + - sizeof(struct ipv6hdr); - __u8 *next_hdr = data + off + - offsetof(struct ipv6hdr, nexthdr); - - struct ipv6_l3_l4_tuple v6_tuple; - for (j = 0; j < 4; j++) - *((uint32_t *)&v6_tuple.src_addr + j) = - __builtin_bswap32(*((uint32_t *) - src_dst_addr + j)); - for (j = 0; j < 4; j++) - *((uint32_t *)&v6_tuple.dst_addr + j) = - __builtin_bswap32(*((uint32_t *) - src_dst_addr + 4 + j)); - - /** Fetch the L4 header port-numbers only if next-header - * is TCP/UDP **/ - if (*next_hdr == IPPROTO_UDP || *next_hdr == IPPROTO_TCP) { - v6_tuple.sport = PORT(*(src_dst_port + 0), - *(src_dst_port + 1)); - v6_tuple.dport = PORT(*(src_dst_port + 2), - *(src_dst_port + 3)); - } else { - v6_tuple.sport = 0; - v6_tuple.dport = 0; - } - - __u8 input_len = sizeof(v6_tuple) / sizeof(__u32); - if (rsskey->hash_fields & (1 << HASH_FIELD_IPV6_L3)) - input_len--; - hash = rte_softrss_be((__u32 *)&v6_tuple, key, 9); - } else { - return TC_ACT_PIPE; - } - - queue = rsskey->queues[(hash % rsskey->nb_queues) & - (TAP_MAX_QUEUES - 1)]; - skb->cb[1] = QUEUE_OFFSET + queue; - /* printt(">>>>> rss_l3_l4 hash=0x%x queue=%u\n", hash, queue); */ - - return TC_ACT_RECLASSIFY; -} - -#define RSS(L) \ - __section(#L) int \ - L ## _hash(struct __sk_buff *skb) \ - { \ - return rss_ ## L (skb); \ - } - -RSS(l3_l4) - -BPF_LICENSE("Dual BSD/GPL"); diff --git a/drivers/net/tap/bpf/tap_rss.c b/drivers/net/tap/bpf/tap_rss.c new file mode 100644 index 000000000000..1abd18cb606e --- /dev/null +++ b/drivers/net/tap/bpf/tap_rss.c @@ -0,0 +1,272 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 + * Copyright 2017 Mellanox Technologies, Ltd + */ + +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "../tap_rss.h" + +/* + * This map provides configuration information about flows + * which need BPF RSS. + * + * The hash is indexed by the tc_index. + */ +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(__u16)); + __uint(value_size, sizeof(struct rss_key)); + __uint(max_entries, TAP_MAX_QUEUES); +} rss_map SEC(".maps"); + + +#define IP_MF 0x2000 /** IP header Flags **/ +#define IP_OFFSET 0x1FFF /** IP header fragment offset **/ + +/* + * Compute Toeplitz hash over the input tuple. + * This is same as rte_softrss_be in lib/hash + * but loop needs to be setup to match BPF restrictions. + */ +static __u32 __attribute__((always_inline)) +softrss_be(const __u32 *input_tuple, __u32 input_len, const __u32 *key) +{ + __u32 i, j, hash = 0; + +#pragma unroll + for (j = 0; j < input_len; j++) { +#pragma unroll + for (i = 0; i < 32; i++) { + if (input_tuple[j] & (1U << (31 - i))) + hash ^= key[j] << i | key[j + 1] >> (32 - i); + } + } + return hash; +} + +/* Compute RSS hash for IPv4 packet. + * return in 0 if RSS not specified + */ +static __u32 __attribute__((always_inline)) +parse_ipv4(const struct __sk_buff *skb, __u32 hash_type, const __u32 *key) +{ + struct iphdr iph; + __u32 off = 0; + + if (bpf_skb_load_bytes_relative(skb, off, &iph, sizeof(iph), BPF_HDR_START_NET)) + return 0; /* no IP header present */ + + struct { + __u32 src_addr; + __u32 dst_addr; + __u16 dport; + __u16 sport; + } v4_tuple = { + .src_addr = bpf_ntohl(iph.saddr), + .dst_addr = bpf_ntohl(iph.daddr), + }; + + /* If only calculating L3 hash, do it now */ + if (hash_type & (1 << HASH_FIELD_IPV4_L3)) + return softrss_be((__u32 *)&v4_tuple, sizeof(v4_tuple) / sizeof(__u32) - 1, key); + + /* No L4 if packet is a fragmented */ + if ((iph.frag_off & bpf_htons(IP_MF | IP_OFFSET)) != 0) + return 0; + + /* Do RSS on UDP or TCP ports */ + if (iph.protocol == IPPROTO_UDP || iph.protocol == IPPROTO_TCP) { + __u16 src_dst_port[2]; + + off += iph.ihl * 4; + if (bpf_skb_load_bytes_relative(skb, off, &src_dst_port, sizeof(src_dst_port), + BPF_HDR_START_NET)) + return 0; /* TCP or UDP header missing */ + + v4_tuple.sport = bpf_ntohs(src_dst_port[0]); + v4_tuple.dport = bpf_ntohs(src_dst_port[1]); + return softrss_be((__u32 *)&v4_tuple, sizeof(v4_tuple) / sizeof(__u32), key); + } + + /* Other protocol */ + return 0; +} + +/* parse ipv6 extended headers, update offset and return next proto. + * returns next proto on success, -1 on malformed header + */ +static int __attribute__((always_inline)) +skip_ip6_ext(__u16 proto, const struct __sk_buff *skb, __u32 *off, int *frag) +{ + struct ext_hdr { + __u8 next_hdr; + __u8 len; + } xh; + unsigned int i; + + *frag = 0; + +#define MAX_EXT_HDRS 5 +#pragma unroll + for (i = 0; i < MAX_EXT_HDRS; i++) { + switch (proto) { + case IPPROTO_HOPOPTS: + case IPPROTO_ROUTING: + case IPPROTO_DSTOPTS: + if (bpf_skb_load_bytes_relative(skb, *off, &xh, sizeof(xh), + BPF_HDR_START_NET)) + return -1; + + *off += (xh.len + 1) * 8; + proto = xh.next_hdr; + break; + case IPPROTO_FRAGMENT: + if (bpf_skb_load_bytes_relative(skb, *off, &xh, sizeof(xh), + BPF_HDR_START_NET)) + return -1; + + *off += 8; + proto = xh.next_hdr; + *frag = 1; + return proto; /* this is always the last ext hdr */ + default: + return proto; + } + } + + /* too many extension headers give up */ + return -1; +} + +static __u32 __attribute__((always_inline)) +parse_ipv6(const struct __sk_buff *skb, __u32 hash_type, const __u32 *key) +{ + struct { + __u32 src_addr[4]; + __u32 dst_addr[4]; + __u16 dport; + __u16 sport; + } v6_tuple = { }; + struct ipv6hdr ip6h; + __u32 off = 0, j; + int proto, frag; + + if (bpf_skb_load_bytes_relative(skb, off, &ip6h, sizeof(ip6h), BPF_HDR_START_NET)) + return 0; + +#pragma unroll + for (j = 0; j < 4; j++) { + v6_tuple.src_addr[j] = bpf_ntohl(ip6h.saddr.in6_u.u6_addr32[j]); + v6_tuple.dst_addr[j] = bpf_ntohl(ip6h.daddr.in6_u.u6_addr32[j]); + } + + if (hash_type & (1 << HASH_FIELD_IPV6_L3)) + return softrss_be((__u32 *)&v6_tuple, sizeof(v6_tuple) / sizeof(__u32) - 1, key); + + off += sizeof(ip6h); + proto = skip_ip6_ext(ip6h.nexthdr, skb, &off, &frag); + if (proto < 0) + return 0; + + if (frag) + return 0; + + /* Do RSS on UDP or TCP ports */ + if (proto == IPPROTO_UDP || proto == IPPROTO_TCP) { + __u16 src_dst_port[2]; + + if (bpf_skb_load_bytes_relative(skb, off, &src_dst_port, sizeof(src_dst_port), + BPF_HDR_START_NET)) + return 0; + + v6_tuple.sport = bpf_ntohs(src_dst_port[0]); + v6_tuple.dport = bpf_ntohs(src_dst_port[1]); + + return softrss_be((__u32 *)&v6_tuple, sizeof(v6_tuple) / sizeof(__u32), key); + } + + return 0; +} + +/* + * Compute RSS hash for packets. + * Returns 0 if no hash is possible. + */ +static __u32 __attribute__((always_inline)) +calculate_rss_hash(const struct __sk_buff *skb, const struct rss_key *rsskey) +{ + const __u32 *key = (const __u32 *)rsskey->key; + + if (skb->protocol == bpf_htons(ETH_P_IP)) + return parse_ipv4(skb, rsskey->hash_fields, key); + else if (skb->protocol == bpf_htons(ETH_P_IPV6)) + return parse_ipv6(skb, rsskey->hash_fields, key); + else + return 0; +} + +/* scale value to be into range [0, n), assumes val is large */ +static __u32 __attribute__((always_inline)) +reciprocal_scale(__u32 val, __u32 n) +{ + return (__u32)(((__u64)val * n) >> 32); +} + +/* layout of qdisc skb cb (from sch_generic.h) */ +struct qdisc_skb_cb { + struct { + unsigned int pkt_len; + __u16 dev_queue_mapping; + __u16 tc_classid; + }; +#define QDISC_CB_PRIV_LEN 20 + unsigned char data[QDISC_CB_PRIV_LEN]; +}; + +/* + * When this BPF program is run by tc from the filter classifier, + * it is able to read skb metadata and packet data. + * + * For packets where RSS is not possible, then just return TC_ACT_OK. + * When RSS is desired, change the skb->queue_mapping and set TC_ACT_PIPE + * to continue processing. + * + * This should be BPF_PROG_TYPE_SCHED_ACT so section needs to be "action" + */ +SEC("action") int +rss_flow_action(struct __sk_buff *skb) +{ + const struct rss_key *rsskey; + __u16 classid; + __u32 hash; + + /* TC layer puts the BPF_CLASSID into the skb cb area */ + classid = ((const struct qdisc_skb_cb *)skb->cb)->tc_classid; + + /* Lookup RSS configuration for that BPF class */ + rsskey = bpf_map_lookup_elem(&rss_map, &classid); + if (rsskey == NULL) { + bpf_printk("hash(): rss not configured"); + return TC_ACT_OK; + } + + hash = calculate_rss_hash(skb, rsskey); + bpf_printk("hash %u\n", hash); + if (hash) { + /* Fold hash to the number of queues configured */ + skb->queue_mapping = reciprocal_scale(hash, rsskey->nb_queues); + bpf_printk("queue %u\n", skb->queue_mapping); + return TC_ACT_PIPE; + } + return TC_ACT_OK; +} + +char _license[] SEC("license") = "Dual BSD/GPL"; From patchwork Thu Feb 8 17:41:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 136554 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AED2343AE1; Thu, 8 Feb 2024 18:51:42 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 91C0842E65; Thu, 8 Feb 2024 18:51:09 +0100 (CET) Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by mails.dpdk.org (Postfix) with ESMTP id 4DF9942E2F for ; Thu, 8 Feb 2024 18:51:06 +0100 (CET) Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6da202aa138so61280b3a.2 for ; Thu, 08 Feb 2024 09:51:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1707414665; x=1708019465; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5fLbfOoPWFp8+7e+bVUmtPAhNL+m2IvM3AhwA0UFGoM=; b=nNHm0pX8ITvSvhBeRPUh09YWj+vNGYR99xppm64sicd+pkgUHP88S+IPFA6pUPfzjc njMrOloEHbsecfGP4A+WKzNPALOpCujpKvpS9QwyyXYARFaxfmM7Fq3SDk/4Yym/wRmz OIOkIkBK3Dv/1WQW8XI4xcH+8tJ49ETNR1NTxqCgJ08UWLJVGqhX6lpINfNoImVS6lwN G+wSHb7qQLq9bZiPpNuorMWtK8a9AxvkG7ggJaG0Taxc9E2oJXE5R32Y61thAW9gLHlv Atp/FiA5XTZY08qBIEqk8j3vCQx3D4YuDhvggS57mOE5BnAbKYGHQ+rFL98chLEMopRA JURw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707414665; x=1708019465; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5fLbfOoPWFp8+7e+bVUmtPAhNL+m2IvM3AhwA0UFGoM=; b=f2jgkazYqvPeQ1ezKdLQPqRYUO+n5IKPEnLYYicfB+VicQ0NnHXdUIOsKtSxfPojSE LSfuhjZjJmrTnORGbS2phIJQYQhM/tRU5RkcOK9nkxwJEmGaLCOXABsTEn7R7qH/C8zg 2GePKMEGWr8nRzu8Rv/RZH3duib5RJcXWrUmJ9seqWieIVtN0pnHnQlUgPHwdO72X+uq eISWgZbLiwQriKZELp6zer26kdafBcCAKmlF6xqCziPXjMJvTwoA/nG4DrcGeh+g6ORy BQwAhS3iiRyl8QkxcQhSulkIvioCa5J57NlPtvAE53NiUeWVclfb+a7nV0hWNFL3jrrx IjaQ== X-Gm-Message-State: AOJu0Yxs3sUXEZsa2dMp9Eud/eS7LYpEbF0JwRox61tNxJUkbvDNX9PX 8hqHco3gyvSanVikwZLhinTUHKZ1GX3EvXPWRuYYkMtWClf66tL6GnHsKsQ1ngklZaWZ/2WsbEu 06wM= X-Google-Smtp-Source: AGHT+IH1xjot0vHE3ekGDXMKt6KAk0tgW0Afd+ucHPdXDfH3mwavmR+EV6HGN4YG7hxh/twt8LySxw== X-Received: by 2002:a05:6a20:258c:b0:19c:aaee:bb1b with SMTP id k12-20020a056a20258c00b0019caaeebb1bmr386585pzd.7.1707414665241; Thu, 08 Feb 2024 09:51:05 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f19-20020a056a00229300b006dbda7bcf3csm5030pfe.83.2024.02.08.09.51.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 09:51:04 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 5/7] net/tap: use libbpf to load new BPF program Date: Thu, 8 Feb 2024 09:41:29 -0800 Message-ID: <20240208175051.326550-6-stephen@networkplumber.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240208175051.326550-1-stephen@networkplumber.org> References: <20240130034925.44869-1-stephen@networkplumber.org> <20240208175051.326550-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org There were multiple issues in the RSS queue support in the TAP driver. This required extensive rework of the BPF support. Change the BPF loading to use bpftool to create a skeleton header file, and load with libbpf. The BPF is always compiled from source so less chance that source and instructions diverge. Also resolves issue where libbpf and source get out of sync. The program is only loaded once, so if multiple rules are created only one BPF program is loaded in kernel. The new BPF program only needs a single action. No need for action and re-classification step. It alsow fixes the missing bits from the original. - supports setting RSS key per flow - level of hash can be L3 or L3/L4. Signed-off-by: Stephen Hemminger --- drivers/net/tap/bpf/tap_bpf_program.o | Bin 0 -> 28080 bytes drivers/net/tap/meson.build | 26 +- drivers/net/tap/rte_eth_tap.c | 2 + drivers/net/tap/rte_eth_tap.h | 9 +- drivers/net/tap/tap_flow.c | 381 +++++++------------------- drivers/net/tap/tap_flow.h | 11 +- drivers/net/tap/tap_rss.h | 3 + drivers/net/tap/tap_tcmsgs.h | 4 +- 8 files changed, 113 insertions(+), 323 deletions(-) create mode 100644 drivers/net/tap/bpf/tap_bpf_program.o index 5099ccdff11b..ad51b6bbbb7c 100644 diff --git a/drivers/net/tap/bpf/tap_bpf_program.o b/drivers/net/tap/bpf/tap_bpf_program.o new file mode 100644 index 0000000000000000000000000000000000000000..db016eaadefc63e0bbc45b3115f497a465438d33 GIT binary patch literal 28080 zcmbtc4Rl>qnY~G0+W>_|BoeE_McN@Cl-$3UvKaIassx~TUg(Zzk<#m{m2FHQoj1rW=GRzEt@uvbx!3wj_x!?ER_kGkZSo7AR zci+A5Is4rG&N=U#oAD zhx_k-lCSEXyZbeVw-C{_F+ZP*>w<~zUcD5@5bz2*UKkhrm0RllA?QAg>c#16c;##m z+)ki!a1OdY+UP;oz!~}R$_4(|jGIvHnUVWf?###mcp8*t*Ng+Y_2x8UUB zeS_fA89RL&-1RFiTXKmLmtMZIXU&X>Du2nU@G{4*SRHn+nX#kFuUxe%bo`|~SB7h5 zj92;Iu2sE`Z#i-<`0bwGGSx9bu{~b@{og|3JWFZvJ?+K6ie}$MbwE zN95O?3)Q_FYpxpe?Q5Fy!@7_E2mH})_tq|)a?O+m9H%$u*YC;4NyprLW!1-B**`eV z7wEr!AAfABuM6G+w6MO(-)H~zc`*HJeX*c2&tDcSog4(t?_3;%LxW)cteig=-6;f| ze|CPpu#neZxjsL4@4ri`e`bEZ6xX|U^&f^H@LGz4yYu-c_)}cx?oOP)8x`QS6bI)& z|1f_VRPt^cxcb^bZKEUi>z+cL@6RrmN3VPEXujUHw}k7Ppu5*5=yy(9cb?Qk@KT>Q zb;i9?ckPw>jR&Mlr0&`&^`>|A{!HpP@uH{XW zYVE&!qSReE$;*Ko?>-MYRxZ!^O?cnVM-a>7yd3jf&edCz_2T+12uA9m(N`cCsry{G zT`2z~f|0tIc{pGI+ zrhn~pe}jAKFRvgN&UJ77sZGWjC7^&O- zaxH?9y6rEoAsDIK{_+}v>0j$U1@=6@yoz8r*S+2!f|0ts)jPUol2z>#9 zk-E=C=o=7B{|#?NtM<@eP}vCxrgD~-U;lY@YRb!O-)ZoTGS?)n6Q7(Kl58?;^&*DXab z=DJ2%&%-Vl?b9IjFC!SE7o`4P1Y`7isecr~7`;yFze7M;FJoO@yU$-E7_FC-ejkF+ zdZ_ee2tezNN*_l6T5nML_Yi>A3rg=s09vnC`hy5S>vc+BhX9mbM*k7_KLY_My`=RN z0VqAxdM^S{ItGTU|4{^>bPNosZ$tn}N8gqDsR%&n=&Mq{2>~b_Z>`i1MF3LA?JNCw z1R!qjCK)H*fXVbzSdT7X(khq=e%(977!Y2lM?})1-R8 zpL6?E_gXk zGe2H|R$_bT*W-TQb$HIZ`5doOb-u=Re&40_Q0ji)rS(Rs`+b+z8>H^{U0N?l-S4}! zUN3dG@2Y6MPU?Q&rSx*`^9c9n_FKAqN$GxnrOJm&_xmfQW8UJ=Bh>HrS4zjcC3L^P zQaa`>q5J)n(lKud-R-Y*`!R0`-S4kt`7-*q(A~aDl`m=C@1tb-Q0smlC3QD%iTeFM zO6us4Quq5PsUtI~`+bzu(Y2-S_ED<+7$Q>l`zTQ!x36@!f0E^K`%3rwB~d;!y5BDe zU9D$uFMhuybT=Ml``mu1i=OvXW|47}<3qtq%AF1O#lDgmjNZri~qP*MxsP?;gLFj(}Bg*6U zmG1UEvOI2I>3*Lh%7;ex`y8SBc|rL5eU8xmydZS9&*@t0N?{z*{?$Eic3rh;iW}9K zThLG4e&-30C7jFa{JbEpbNe3N&$!QdKQHh+je7k4M?7y+`y77Wc9L_=_meo3s`D83 zPkx`H^-$`5f1~xof2co>+-$>of z2SWGz8>zebKKwLX}E0F18I2N?p; zdRfi?mm(0Ymz4e(0?>M>^zR@5t?Tu{1qeXvdVTN%1fX@jK3IhSw650&&maJ;>-E7S z2tes&^mpzb=nr2(07@@u{e1{P>7mvK5P;I%d?4!oBLYymSsy%&0F>_L15th(0#Le} z4}|_11fX;`9|--O2tew%eWkY}0IB2lmEMm4q#hdmX#^m3KOYGH2N8hO?fPIH0_a-n zo(kyCeZ-*F2k$@-Z)AN?L?C-xA1p&4^6znvBY^62&^NF?xCepA=lE!5DCggc5s1;% z`rsi1Vsy1WxCMb2U9AshAqbGvQ2 zt?Tu{#Rx#_dVTN^0?@i%AKZ!nw650&M<4*L>-E9kBLJnB(a*%^*Ow50(o0%zM*vC> zwcdvSly24sI}m`<&H7*~0#LeHACwV*(#`rHKmbZN>w`K3Aa&fny8lTCKgX0iD*I@qrmc6YHu0jxRWPR`=0@>U8U;=^2^}(YE;LWTL zHX;x?zafrL&cCN15TmR0!8!zDbhSR1fk2F|)(07aFuGbFT#5jUuGR;?L;zYZtNH(K z1funl($7Z#S`U?e9|F+2ULPz&09x1UgU1no*7f?}UId_Zy*^li0JN^x2R}mqN-v|o zi_fpmBLJnBv>qb>rH5MYK>$iO>w}*n0HvGt!EFdY>1KVbQM%|0g2=spIyQ{z(KN_0Z@CA^@q|^}#FzAa%PwI1K@Gt#!5TeSL5nf_NkAgCh~h z-qr_aArQGf*n|My%=%zH0+I6@njOl|mpc%M(bf9kcm!f}wLbV50x`N;AN&PD7+tLo z{)_;OuGR;?MF3hatNH(N1funl((ge4S`U@pg8;Oy*9SjE09x1UgWC~+*7f?}C1+Jlpbom69FjQtPj420F-Xl2i*uj>1KVf z4FM?KtPgHN07^IOgF_I2)N%Xj{*OZdQpfEp{VD_?_0Z^lLI6^?>x1VIfYj~!;1>v> zYj7rN-TU){ZzG5|vOYKmf$VL4a2EoptoC-nCS-jc0;sMJh?n1A!IdTXcZ_=wh|z1` zo8addsP|$7W3FqI^^74Hqc=!>J%TZMLFyp_F?zk!Pe2exuao*9g3x+d{k>)ig3)?O z>C+H|)m7mpmod(LO%onD7}n+ zFZA~z0Hv3-K7as}9%_9j0#Le}7exI}ApoVjc|qt8AONMic|quF5rER&ydd;_5P;I% zydd-g5rEWj`%0gM0HluFSNiD)Kz6b$G9k;Li??(Vq z$L%ZqY6S4QzbBxQmvQ5jCR~iyQampTaok2no~JfV3W669!150vG`yIX-_rnHs{Z{F z*FS?mj2=q;5d>m%^*yRP5Qx#$_o&{FK#Z=wM|CX%F}nI5)oTdE=<0h^uOb+&mur8Y z#r^F@AX+aeeHQ}IdZ_fB2tezY55)Ieb|3()V?Gf2KOz9FV?Gf2AOg@j<^!QmMF3jI zd?57c2tes&^e3SoiU5>e()#-mfYL*)UyA^g?&bqg|EmZ<>25v{`d<-%(%pO@^uHhg zrMvk+=r17vrMvk+=)Xq*QpfEp{Rsphb=q(TFeKaoo|1=`_@$L_agoEv;DdI z9f@mqf8p5t9F1^KI_J;yr@{RE&mjK$O(Cc&P`^9gaT5N)%gsmboApa^bnTno>R8lQ zmwyqe=AR4vdzJqB*k4!mpOob1sK@&s?oWgHZ_STW!MopYbr(}lxZkn3JFfKQ`tA9_ zb1p|u=aG2MwT)jVU*5GWJn^iPk3Dt48O*ugM7iHXx!+8=-%GjQRJq?%x!+v5-&?ug zV!2miehBKW`cTkt^?r5k*FCPi`P1QC9H495!969)Tbz>{=zGTb+28)R15zRQqyMdw zy=s#G#gk^w-p`#+%I{lCU|k~m^+A7qA!wwGSiQ-07F@sHwDSRco;ZHMT)&F0@6CT+ z;V$y88~x973c*fu{WhOtAsCi#2ui$$_tVusX83vJJ7M_s{=5)0xPBpCCy{T9;mhRP zVfazc{p1@Oehc}Q3|~*aiw*yM@+}+w zdH-{^La@Q`N0IMV!}s~~La@v5KlJZkA!zpRL+utWBj0wzf17+e4ga+NIwty$;g2NW zVZ*N`-%-OKLB8XLznFX*^RInY8~P>kZ8iLV`uC|2EHL~rvy z{dpnSZul>f?@q)2k8g1yXz}k)wfVv4$+yGszvOXe_=CxJ$nY1E?}*_?$al=}=acV* z;fKk$!5?eweB8eug`i~kL&lL-(kZKlJBVDAExm;ZuslScbDPsr+K5Epj0+*3M^r?ihE5Ka6}k48N3odkudN`3@Pr$DbF1 z5yRg}zGH^}6^-8s!-wSC;NQ1e`!6Nm7Q-Jxz8!`?pL}}_e=E-$hJT#r4a3hQ-!a2K z=5sFu6NW#Cd>itgr&e#^4DxL;{4M0$VR-!H8?QppYxwo#J7oBOrExl9_yfpy%`%^_!eGx`gUO5OTLQ@znjOA z;ZyS6VEC(O-q>pRpV0W-VfX>^^*^ehwz$Lh$v4b@Hd$4EN8_ku_=J2H8~z&dEgL=} z-wlSpjC{8m{u|`G!|=bQapeD}1JzarK^yso^t&&Xo}+P8GW-$byV&rH$hU0xuaoZv z!{_VcLa^2F8_9Qv;djtD3TR)%?QACB(D462z9qwdlg962!(TwYWy61-d^Z^WQ5r{E z4Sz8CP8j|o@@=GjCAYtcd|M6wA@c1s{Eg&0VEFAcUWN_-4)Pr}{KGU(#|^(9`R+3O zN#xs1&)c~DpCsRQ!~eU#pb&H#{uAUoVEBLLb%o(4lkceEPbc4T!+(~1cNzXqG>)3z z#x2i}F}0`N@E@V}bQ*p$jo$&o|AofUu;K3`-%-QAK;vlK@b4ktU54)`-{ymL`+r2f z?S@}LzMY1@i+l$R|6eqYh7Erd`HmX?a`GKFd<*%i2Lotw5Bavp2L|B3Ouik4f11Wo zui&SP^ z@Y`toP8j|`@@;sBZvRhcoVFOgo_sqDKbL%a4PPeTA;bTRx5IkX@Yj&q0cyQhVLcc0mI)*zQcxJLcXJh-$=gWhW|Be=GTR z8vaQdM+1hxnS6&0|1gcCQNvFm-*LmALB6{T|2gsvL(zD^)5y2O@Sh;xUc*;t{0aE%}ZZ{%IOV zV}=i@JrjoSr1mr%uG{|=YEO&dpRIS-6@m`Ke~o;54gVC4qank;o%1*RBJv$G{I|$= z!tfW8Z^KO8{x6VktKol6N7zo^UGGj90&xe}hw8~!IWew+VJxBss+ zj@k`>68Ux-{xjq|VE7Ywo;3V5b-$uTj zhF?Lx1BU-T`3@W2|4|U*Xw>lclJB_TchY#-W%w5IZEn%+zl?lKhW~-T49~9&zl?l` z4FAoBJhu_UKQ}q&#|+;}z7vLDL%t11dQko6S2T`V41X}Sr^E2`sXe`hzn$7MWcVF4 zjz$dsMe-dp{5XxH3Bxy%Z^Kd5rh9M>`L-DTtK{2Z__N8k*YG#*>#D=w`waik)SMqN z{B+LW@E<1M3B&JCz76m3&944)+P-=HEr!qkx09IX4L?TXx7YA*CEp>#KS<+u#PE~I zcg*nbrTd*Q{B`8paCEil9{h~PZ;RotCEo>x|It4A{SFxZMH)xLhM&vxq~UMidD8Ih z)AIU*V{|(&C*RQUcav|)@GsCfT5R~^$+v9yKkk?NZZQ08^4)6qCFHxq@OP1KFk83( zC4X87Lc@QHd`pIZhQ`Zc!yih%WyAN7?*_wfCf}`w?e>x(LTBm%HN`C!QEBnlmRq78zs=QU(cwm_fv# zol7&w7^Dmm1~G$(K^v8BV^Cy}F-RFC3}OZmgCbYMpvWL&kTOUZ#0(+^8CS!g$RJ~o zGDsN23?c?8SHqylAY+g+NEpNnA_fUp!=T6@V~{dP7{m-B1~FH|pvWL&kTOUZ#0;V; z;A$cQMFtszltIEEW)LxGYvtk$G6pGwgh9+8Vi32sQ>Sg?PD>!0Lm*|4Fo+pM4BELg zgN#ASAYl+Qh#0g{={5#M1{s5tLBb$r5HTonH4KUjG6pGwgh9+8Vvun)42ldg1}TGt zLChdxka9H)iVQLaDT9PT%phWra5W5y3^E2OgM>lMAYu@6H4KUjG6pGwgh9+8ssgSi zB2Z+IF-RFC3}OZmgSJ*K&LCruGDsLi42qbJS6tk+y35VF(d?*5JvZZ?OCXy=AZ3s+ zh#5o-+PO4?j6upEVGuKj7_?F8HU>oo8H1ET!XRc4F(`6142ldg1}TGtLChdxka0B( ziVQLaDT9PT%phWray1N!3^E2OgM>lMAYzbkH4KUjG6pGwgh9+8Vi0pR42ldg1}TGt zLChek0)L6JemAZ3s+h#5o-id+qYB7=-U${=A7 zGl&>uTn&RFgN#ASAYl+Qh!~_?4TBmq-^f;ZwW zOdw+>Wsoq48AJ@)sZbk(B7=-U${=A7Gl&?paWxEz3^E2OgM>lMAYxGDY8Vt5WDHUU z34@qH#318p7!(;~3{nOOgP1|YAmwTp6d7a;QU(cwm_fuK;c6HZ8DtDn1_^_hLBt^D zY8Vt5WDHUU34@qHR0Uj3M4-qZV~{dP7{m-B25qfeoI%DQWsoq48AJg8-`9sxL2Y+i zt8K#tIc_|l#yRA-?SgXW?mxc`7ko)z32Qs%oONFV+go|wZ*v{~@fDxo_(>3ZDnCls z@V=aj*zz}kaE}&Bd;uN%`2OkKJsxmv=kkrjyQA~B8Fm))3wZxem*;zUKEcsFrrA^Z ztEoIcC*UIPTKRei&(9oZ9`^+6sl0nmAz$2XuIG4&4!h?I@-<537f2GA@1N qO!nlsk_fd); internals->nlsk_fd = -1; + tap_flow_bpf_destroy(internals); } for (i = 0; i < RTE_PMD_TAP_MAX_QUEUES; i++) { @@ -1959,6 +1960,7 @@ eth_dev_tap_create(struct rte_vdev_device *vdev, const char *tap_name, strlcpy(pmd->name, tap_name, sizeof(pmd->name)); pmd->type = type; pmd->ka_fd = -1; + pmd->rss = NULL; pmd->nlsk_fd = -1; pmd->gso_ctx_mp = NULL; diff --git a/drivers/net/tap/rte_eth_tap.h b/drivers/net/tap/rte_eth_tap.h index 5ac93f93e961..0cf2b30bb03b 100644 --- a/drivers/net/tap/rte_eth_tap.h +++ b/drivers/net/tap/rte_eth_tap.h @@ -79,12 +79,11 @@ struct pmd_internals { int flow_isolate; /* 1 if flow isolation is enabled */ int flower_support; /* 1 if kernel supports, else 0 */ int flower_vlan_support; /* 1 if kernel supports, else 0 */ - int rss_enabled; /* 1 if RSS is enabled, else 0 */ int persist; /* 1 if keep link up, else 0 */ - /* implicit rules set when RSS is enabled */ - int map_fd; /* BPF RSS map fd */ - int bpf_fd[RTE_PMD_TAP_MAX_QUEUES];/* List of bpf fds per queue */ - LIST_HEAD(tap_rss_flows, rte_flow) rss_flows; + + struct tap_rss *rss; /* BPF program */ + uint16_t bpf_flowid; /* next BPF class id */ + LIST_HEAD(tap_flows, rte_flow) flows; /* rte_flow rules */ /* implicit rte_flow rules set when a remote device is active */ LIST_HEAD(tap_implicit_flows, rte_flow) implicit_flows; diff --git a/drivers/net/tap/tap_flow.c b/drivers/net/tap/tap_flow.c index 94436af55ce8..bc817bdb4fd3 100644 --- a/drivers/net/tap/tap_flow.c +++ b/drivers/net/tap/tap_flow.c @@ -16,24 +16,16 @@ #include #include -#include #include #include - -/* RSS key management */ -enum bpf_rss_key_e { - KEY_CMD_GET = 1, - KEY_CMD_RELEASE, - KEY_CMD_INIT, - KEY_CMD_DEINIT, -}; - -enum key_status_e { - KEY_STAT_UNSPEC, - KEY_STAT_USED, - KEY_STAT_AVAILABLE, -}; +#ifdef HAVE_BPF_RSS +/* Workaround for warning in bpftool generated skeleton code */ +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wcast-qual" +#include "tap_rss.skel.h" +#pragma GCC diagnostic pop +#endif #define ISOLATE_HANDLE 1 #define REMOTE_PROMISCUOUS_HANDLE 2 @@ -41,8 +33,7 @@ enum key_status_e { struct rte_flow { LIST_ENTRY(rte_flow) next; /* Pointer to the next rte_flow structure */ struct rte_flow *remote_flow; /* associated remote flow */ - int bpf_fd[SEC_MAX]; /* list of bfs fds per ELF section */ - uint32_t key_idx; /* RSS rule key index into BPF map */ + uint16_t flowid; struct nlmsg msg; }; @@ -72,6 +63,7 @@ struct action_data { } skbedit; struct bpf { struct tc_act_bpf bpf; + uint16_t classid; int bpf_fd; const char *annotation; } bpf; @@ -112,13 +104,12 @@ tap_flow_isolate(struct rte_eth_dev *dev, int set, struct rte_flow_error *error); -static int bpf_rss_key(enum bpf_rss_key_e cmd, __u32 *key_idx); -static int rss_enable(struct pmd_internals *pmd, - const struct rte_flow_attr *attr, - struct rte_flow_error *error); +#ifdef HAVE_BPF_RSS +static int rss_enable(struct pmd_internals *pmd, struct rte_flow_error *error); static int rss_add_actions(struct rte_flow *flow, struct pmd_internals *pmd, const struct rte_flow_action_rss *rss, struct rte_flow_error *error); +#endif static const struct rte_flow_ops tap_flow_ops = { .validate = tap_flow_validate, @@ -865,6 +856,9 @@ add_action(struct rte_flow *flow, size_t *act_index, struct action_data *adata) tap_nlattr_add(&msg->nh, TCA_ACT_BPF_PARMS, sizeof(adata->bpf.bpf), &adata->bpf.bpf); + tap_nlattr_add(&msg->nh, TCA_BPF_CLASSID, + sizeof(adata->bpf.classid), + &adata->bpf.classid); } else { return -1; } @@ -1101,8 +1095,7 @@ priv_flow_process(struct pmd_internals *pmd, }, }; - err = add_actions(flow, 1, &adata, - TCA_FLOWER_ACT); + err = add_actions(flow, 1, &adata, TCA_FLOWER_ACT); } } else if (actions->type == RTE_FLOW_ACTION_TYPE_QUEUE) { const struct rte_flow_action_queue *queue = @@ -1129,6 +1122,7 @@ priv_flow_process(struct pmd_internals *pmd, err = add_actions(flow, 1, &adata, TCA_FLOWER_ACT); } +#ifdef HAVE_BPF_RSS } else if (actions->type == RTE_FLOW_ACTION_TYPE_RSS) { const struct rte_flow_action_rss *rss = (const struct rte_flow_action_rss *) @@ -1137,13 +1131,14 @@ priv_flow_process(struct pmd_internals *pmd, if (action++) goto exit_action_not_supported; - if (!pmd->rss_enabled) { - err = rss_enable(pmd, attr, error); + if (pmd->rss == NULL) { + err = rss_enable(pmd, error); if (err) goto exit_action_not_supported; } if (flow) err = rss_add_actions(flow, pmd, rss, error); +#endif } else { goto exit_action_not_supported; } @@ -1239,26 +1234,17 @@ tap_flow_set_handle(struct rte_flow *flow) * */ static void -tap_flow_free(struct pmd_internals *pmd, struct rte_flow *flow) +tap_flow_free(struct pmd_internals *pmd __rte_unused, struct rte_flow *flow) { - int i; - if (!flow) return; - if (pmd->rss_enabled) { - /* Close flow BPF file descriptors */ - for (i = 0; i < SEC_MAX; i++) - if (flow->bpf_fd[i] != 0) { - close(flow->bpf_fd[i]); - flow->bpf_fd[i] = 0; - } - - /* Release the map key for this RSS rule */ - bpf_rss_key(KEY_CMD_RELEASE, &flow->key_idx); - flow->key_idx = 0; - } - +#ifdef HAVE_BPF_RSS + struct tap_rss *rss = pmd->rss; + if (rss) + bpf_map__delete_elem(rss->maps.rss_map, &flow->flowid, + sizeof(flow->flowid), 0); +#endif /* Free flow allocated memory */ rte_free(flow); } @@ -1725,14 +1711,18 @@ tap_flow_implicit_flush(struct pmd_internals *pmd, struct rte_flow_error *error) return 0; } -#define MAX_RSS_KEYS 256 -#define KEY_IDX_OFFSET (3 * MAX_RSS_KEYS) -#define SEC_NAME_CLS_Q "cls_q" - -static const char *sec_name[SEC_MAX] = { - [SEC_L3_L4] = "l3_l4", -}; +/** + * Cleanup when device is closed + */ +void tap_flow_bpf_destroy(struct pmd_internals *pmd __rte_unused) +{ +#ifdef HAVE_BPF_RSS + tap_rss__destroy(pmd->rss); + pmd->rss = NULL; +#endif +} +#ifdef HAVE_BPF_RSS /** * Enable RSS on tap: create TC rules for queuing. * @@ -1747,226 +1737,62 @@ static const char *sec_name[SEC_MAX] = { * * @return 0 on success, negative value on failure. */ -static int rss_enable(struct pmd_internals *pmd, - const struct rte_flow_attr *attr, - struct rte_flow_error *error) +static int rss_enable(struct pmd_internals *pmd, struct rte_flow_error *error) { - struct rte_flow *rss_flow = NULL; - struct nlmsg *msg = NULL; - /* 4096 is the maximum number of instructions for a BPF program */ - char annotation[64]; - int i; - int err = 0; - - /* unlimit locked memory */ - struct rlimit memlock_limit = { - .rlim_cur = RLIM_INFINITY, - .rlim_max = RLIM_INFINITY, - }; - setrlimit(RLIMIT_MEMLOCK, &memlock_limit); - - /* Get a new map key for a new RSS rule */ - err = bpf_rss_key(KEY_CMD_INIT, NULL); - if (err < 0) { - rte_flow_error_set( - error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, - "Failed to initialize BPF RSS keys"); - - return -1; - } - - /* - * Create BPF RSS MAP - */ - pmd->map_fd = tap_flow_bpf_rss_map_create(sizeof(__u32), /* key size */ - sizeof(struct rss_key), - MAX_RSS_KEYS); - if (pmd->map_fd < 0) { - TAP_LOG(ERR, - "Failed to create BPF map (%d): %s", - errno, strerror(errno)); - rte_flow_error_set( - error, ENOTSUP, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, - "Kernel too old or not configured " - "to support BPF maps"); + int err; - return -ENOTSUP; + /* Load the BPF program (defined in tap_bpf.h from skeleton) */ + pmd->rss = tap_rss__open_and_load(); + if (pmd->rss == NULL) { + TAP_LOG(ERR, "Failed to load BPF object: %s", strerror(errno)); + rte_flow_error_set(error, errno, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, + "BPF object could not be loaded"); + return -errno; } - /* - * Add a rule per queue to match reclassified packets and direct them to - * the correct queue. - */ - for (i = 0; i < pmd->dev->data->nb_rx_queues; i++) { - pmd->bpf_fd[i] = tap_flow_bpf_cls_q(i); - if (pmd->bpf_fd[i] < 0) { - TAP_LOG(ERR, - "Failed to load BPF section %s for queue %d", - SEC_NAME_CLS_Q, i); - rte_flow_error_set( - error, ENOTSUP, RTE_FLOW_ERROR_TYPE_HANDLE, - NULL, - "Kernel too old or not configured " - "to support BPF programs loading"); - - return -ENOTSUP; - } - - rss_flow = rte_zmalloc(__func__, sizeof(struct rte_flow), 0); - if (!rss_flow) { - TAP_LOG(ERR, - "Cannot allocate memory for rte_flow"); - return -1; - } - msg = &rss_flow->msg; - tc_init_msg(msg, pmd->if_index, RTM_NEWTFILTER, NLM_F_REQUEST | - NLM_F_ACK | NLM_F_EXCL | NLM_F_CREATE); - msg->t.tcm_info = TC_H_MAKE(0, htons(ETH_P_ALL)); - tap_flow_set_handle(rss_flow); - uint16_t group = attr->group << GROUP_SHIFT; - uint16_t prio = group | (i + PRIORITY_OFFSET); - msg->t.tcm_info = TC_H_MAKE(prio << 16, msg->t.tcm_info); - msg->t.tcm_parent = TC_H_MAKE(MULTIQ_MAJOR_HANDLE, 0); - - tap_nlattr_add(&msg->nh, TCA_KIND, sizeof("bpf"), "bpf"); - if (tap_nlattr_nested_start(msg, TCA_OPTIONS) < 0) - return -1; - tap_nlattr_add32(&msg->nh, TCA_BPF_FD, pmd->bpf_fd[i]); - snprintf(annotation, sizeof(annotation), "[%s%d]", - SEC_NAME_CLS_Q, i); - tap_nlattr_add(&msg->nh, TCA_BPF_NAME, strlen(annotation) + 1, - annotation); - /* Actions */ - { - struct action_data adata = { - .id = "skbedit", - .skbedit = { - .skbedit = { - .action = TC_ACT_PIPE, - }, - .queue = i, - }, - }; - if (add_actions(rss_flow, 1, &adata, TCA_BPF_ACT) < 0) - return -1; - } - tap_nlattr_nested_finish(msg); /* nested TCA_OPTIONS */ - - /* Netlink message is now ready to be sent */ - if (tap_nl_send(pmd->nlsk_fd, &msg->nh) < 0) - return -1; - err = tap_nl_recv_ack(pmd->nlsk_fd); - if (err < 0) { - TAP_LOG(ERR, - "Kernel refused TC filter rule creation (%d): %s", - errno, strerror(errno)); - return err; - } - LIST_INSERT_HEAD(&pmd->rss_flows, rss_flow, next); + /* Attach the maps defined in BPF program */ + err = tap_rss__attach(pmd->rss); + if (err < 0) { + TAP_LOG(ERR, "Failed to attach BPF object: %d", err); + rte_flow_error_set(error, -err, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, + "BPF object could not be attached"); + tap_flow_bpf_destroy(pmd); + return err; } - pmd->rss_enabled = 1; - return err; + return 0; } -/** - * Manage bpf RSS keys repository with operations: init, get, release - * - * @param[in] cmd - * Command on RSS keys: init, get, release - * - * @param[in, out] key_idx - * Pointer to RSS Key index (out for get command, in for release command) - * - * @return -1 if couldn't get, release or init the RSS keys, 0 otherwise. - */ -static int bpf_rss_key(enum bpf_rss_key_e cmd, __u32 *key_idx) -{ - __u32 i; - int err = 0; - static __u32 num_used_keys; - static __u32 rss_keys[MAX_RSS_KEYS] = {KEY_STAT_UNSPEC}; - static __u32 rss_keys_initialized; - __u32 key; - - switch (cmd) { - case KEY_CMD_GET: - if (!rss_keys_initialized) { - err = -1; - break; - } - - if (num_used_keys == RTE_DIM(rss_keys)) { - err = -1; - break; - } - - *key_idx = num_used_keys % RTE_DIM(rss_keys); - while (rss_keys[*key_idx] == KEY_STAT_USED) - *key_idx = (*key_idx + 1) % RTE_DIM(rss_keys); - - rss_keys[*key_idx] = KEY_STAT_USED; - - /* - * Add an offset to key_idx in order to handle a case of - * RSS and non RSS flows mixture. - * If a non RSS flow is destroyed it has an eBPF map - * index 0 (initialized on flow creation) and might - * unintentionally remove RSS entry 0 from eBPF map. - * To avoid this issue, add an offset to the real index - * during a KEY_CMD_GET operation and subtract this offset - * during a KEY_CMD_RELEASE operation in order to restore - * the real index. - */ - *key_idx += KEY_IDX_OFFSET; - num_used_keys++; - break; - - case KEY_CMD_RELEASE: - if (!rss_keys_initialized) - break; - - /* - * Subtract offset to restore real key index - * If a non RSS flow is falsely trying to release map - * entry 0 - the offset subtraction will calculate the real - * map index as an out-of-range value and the release operation - * will be silently ignored. - */ - key = *key_idx - KEY_IDX_OFFSET; - if (key >= RTE_DIM(rss_keys)) - break; - if (rss_keys[key] == KEY_STAT_USED) { - rss_keys[key] = KEY_STAT_AVAILABLE; - num_used_keys--; - } - break; - - case KEY_CMD_INIT: - for (i = 0; i < RTE_DIM(rss_keys); i++) - rss_keys[i] = KEY_STAT_AVAILABLE; +/* Choose next flow id to use for BPF action */ +static int tap_rss_flow_assign(struct pmd_internals *pmd, uint16_t *flow_id) +{ + struct rte_flow *flow; + uint16_t id; - rss_keys_initialized = 1; - num_used_keys = 0; - break; + id = pmd->bpf_flowid; - case KEY_CMD_DEINIT: - for (i = 0; i < RTE_DIM(rss_keys); i++) - rss_keys[i] = KEY_STAT_UNSPEC; +next_id: + /* Skip 0xffff and 0 as id's */ + if (++id == UINT16_MAX) + id = 1; - rss_keys_initialized = 0; - num_used_keys = 0; - break; + /* Wrapped around, all id's have been used */ + if (id == pmd->bpf_flowid) + return -1; - default: - break; + /* Make sure this id has not been used already */ + for (flow = LIST_FIRST(&pmd->flows); flow; flow = LIST_NEXT(flow, next)) { + if (flow->flowid == id) + goto next_id; } - return err; + /* Record starting point for next time */ + pmd->bpf_flowid = id; + *flow_id = id; + return 0; } - /* Default RSS hash key also used by mlx devices */ static const uint8_t rss_hash_default_key[] = { 0x2c, 0xc6, 0x81, 0xd1, @@ -2050,34 +1876,34 @@ static int rss_add_actions(struct rte_flow *flow, struct pmd_internals *pmd, else if (rss->types & (RTE_ETH_RSS_IPV6 | RTE_ETH_RSS_FRAG_IPV6 | RTE_ETH_RSS_IPV6_EX)) hash_type |= RTE_BIT32(HASH_FIELD_IPV6_L3); - /* Get a new map key for a new RSS rule */ - err = bpf_rss_key(KEY_CMD_GET, &flow->key_idx); + + /* Choose new flow id, which is used as index into the BPF map */ + err = tap_rss_flow_assign(pmd, &flow->flowid); if (err < 0) { rte_flow_error_set( error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, - "Failed to get BPF RSS key"); + "Failed to get BPF flowid"); return -1; } - /* Update RSS map entry with queues */ - rss_entry.nb_queues = rss->queue_num; - for (i = 0; i < rss->queue_num; i++) - rss_entry.queues[i] = rss->queue[i]; - rss_entry.hash_fields = hash_type; rte_convert_rss_key((const uint32_t *)key_in, (uint32_t *)rss_entry.key, TAP_RSS_HASH_KEY_SIZE); + /* Update RSS map entry with queues */ + rss_entry.nb_queues = rss->queue_num; + for (i = 0; i < rss->queue_num; i++) + rss_entry.queues[i] = rss->queue[i]; /* Add this RSS entry to map */ - err = tap_flow_bpf_update_rss_elem(pmd->map_fd, - &flow->key_idx, &rss_entry); - + err = bpf_map__update_elem(pmd->rss->maps.rss_map, + &flow->flowid, sizeof(uint16_t), + &rss_entry, sizeof(rss_entry), 0); if (err) { TAP_LOG(ERR, "Failed to update BPF map entry #%u (%d): %s", - flow->key_idx, errno, strerror(errno)); + flow->flowid, errno, strerror(errno)); rte_flow_error_set( error, ENOTSUP, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, "Kernel too old or not configured " @@ -2086,33 +1912,16 @@ static int rss_add_actions(struct rte_flow *flow, struct pmd_internals *pmd, return -ENOTSUP; } - - /* - * Load bpf rules to calculate hash for this key_idx - */ - - flow->bpf_fd[SEC_L3_L4] = - tap_flow_bpf_calc_l3_l4_hash(flow->key_idx, pmd->map_fd); - if (flow->bpf_fd[SEC_L3_L4] < 0) { - TAP_LOG(ERR, - "Failed to load BPF section %s (%d): %s", - sec_name[SEC_L3_L4], errno, strerror(errno)); - rte_flow_error_set( - error, ENOTSUP, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, - "Kernel too old or not configured " - "to support BPF program loading"); - - return -ENOTSUP; - } - /* Actions */ { + const struct bpf_program *rss_prog = pmd->rss->progs.rss_flow_action; struct action_data adata[] = { { .id = "bpf", .bpf = { - .bpf_fd = flow->bpf_fd[SEC_L3_L4], - .annotation = sec_name[SEC_L3_L4], + .annotation = "tap_rss", + .classid = flow->flowid, + .bpf_fd = bpf_program__fd(rss_prog), .bpf = { .action = TC_ACT_PIPE, }, @@ -2120,13 +1929,13 @@ static int rss_add_actions(struct rte_flow *flow, struct pmd_internals *pmd, }, }; - if (add_actions(flow, RTE_DIM(adata), adata, - TCA_FLOWER_ACT) < 0) + if (add_actions(flow, 1, adata, TCA_FLOWER_ACT) < 0) return -1; } return 0; } +#endif /** * Get rte_flow operations. diff --git a/drivers/net/tap/tap_flow.h b/drivers/net/tap/tap_flow.h index 240fbc3dfaef..41f9833619a1 100644 --- a/drivers/net/tap/tap_flow.h +++ b/drivers/net/tap/tap_flow.h @@ -9,7 +9,6 @@ #include #include #include -#include /** * In TC, priority 0 means we require the kernel to allocate one for us. @@ -41,10 +40,6 @@ enum implicit_rule_index { TAP_REMOTE_MAX_IDX, }; -enum bpf_fd_idx { - SEC_L3_L4, - SEC_MAX, -}; int tap_dev_flow_ops_get(struct rte_eth_dev *dev, const struct rte_flow_ops **ops); @@ -57,10 +52,6 @@ int tap_flow_implicit_destroy(struct pmd_internals *pmd, int tap_flow_implicit_flush(struct pmd_internals *pmd, struct rte_flow_error *error); -int tap_flow_bpf_cls_q(__u32 queue_idx); -int tap_flow_bpf_calc_l3_l4_hash(__u32 key_idx, int map_fd); -int tap_flow_bpf_rss_map_create(unsigned int key_size, unsigned int value_size, - unsigned int max_entries); -int tap_flow_bpf_update_rss_elem(int fd, void *key, void *value); +void tap_flow_bpf_destroy(struct pmd_internals *pmd); #endif /* _TAP_FLOW_H_ */ diff --git a/drivers/net/tap/tap_rss.h b/drivers/net/tap/tap_rss.h index 6009be7031b0..51b7ff0d007e 100644 --- a/drivers/net/tap/tap_rss.h +++ b/drivers/net/tap/tap_rss.h @@ -9,6 +9,9 @@ #define TAP_MAX_QUEUES 16 #endif +/* Size of the map from BPF classid to queue table */ +#define TAP_RSS_MAX TAP_MAX_QUEUES + /* Fixed RSS hash key size in bytes. */ #define TAP_RSS_HASH_KEY_SIZE 40 diff --git a/drivers/net/tap/tap_tcmsgs.h b/drivers/net/tap/tap_tcmsgs.h index a64cb29d6fa8..00a0f22e3108 100644 --- a/drivers/net/tap/tap_tcmsgs.h +++ b/drivers/net/tap/tap_tcmsgs.h @@ -6,7 +6,6 @@ #ifndef _TAP_TCMSGS_H_ #define _TAP_TCMSGS_H_ -#include #include #include #include @@ -14,9 +13,8 @@ #include #include #include -#ifdef HAVE_TC_ACT_BPF #include -#endif + #include #include From patchwork Thu Feb 8 17:41:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 136555 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 12BE243AE1; Thu, 8 Feb 2024 18:51:54 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 13C7E42E72; Thu, 8 Feb 2024 18:51:11 +0100 (CET) Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by mails.dpdk.org (Postfix) with ESMTP id 5BC6C42E51 for ; Thu, 8 Feb 2024 18:51:07 +0100 (CET) Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-5d8b887bb0cso1599966a12.2 for ; Thu, 08 Feb 2024 09:51:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1707414666; x=1708019466; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=o9zVh2JaP1PuX1Dc2fWKSZELHWK4ylCWL2/2UJmDRV0=; b=ij36YGlpC5Z4aANXkcsC1l0H7oLyeJbDoXX4nwDeoxPZgwSnJyvgNTa7eaHScgMO0u ws/jL9TMCOWglfhIBmE5Hu7ZJgihXT5U/33ZV+8ZPyFI2fewwFwCBluV8Tc4+KjIBDRz id1TPL38a/KsjY3IiCIA0H1Hci8cbDfUzsZHTtWVjcg1pQ5KW7mbsKlbmt52XRdXOUdm ZIzM0C/aG3E1mzvwwdYQ1IrqiQo/vxu4dHyNlpKEgkPgqtvS1s/Y4m5Q+kGMie5zkyQL 6mw7xZ5AMAXDRk4BG9AAYQu6pXXYsu+8TICMsUetrodjr/FClKuHC1lFl8qBGQ7B1/sM WIZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707414666; x=1708019466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o9zVh2JaP1PuX1Dc2fWKSZELHWK4ylCWL2/2UJmDRV0=; b=khOKlmxSkVC4nU3GqDXZFXRnZZZhAHre7/nsFiizLypmcT6E97TjZfh+IxLDr61A9M v+qsBNxqpLFUZny4EULQSNLUt/CVZ3MV8qppbgZJKVmWzvCueKwJ/HV99Ik6ONdn8fC5 bNfD2gdgj9fHmuwCJcKt3j6pFi+9ABsyaMOBdCAkN0kOIDdTt+DeLZQ0kq4vA7vtfVoH XbTuAvlMMTXOerkcNZBaJ6+lGLsIkFq+mek4fgR2AEYM4o+dj2Tsxl0L2bvLgjhgOjJK qydPkRdS6b5cqbMeP34Vt4/hqE3he2dcdZPqUso0TuEH6ZO69Q4kEppR9JTG1U2tMtWb XkNQ== X-Gm-Message-State: AOJu0YyGO98RQfVRyprdPHRJv2gfpMxArGiF7At9iOd4t16suO88jFOF MxtlXyu6uqT65PJfhf2x/QDMkZau1PQYQt26RI+wPc7onJ5jCkrV2yH4eCDO+3DYMK9uker+yfv KuqU= X-Google-Smtp-Source: AGHT+IFVxblPtJkJXFkgW5nj9ffhHadylbPWJ7Rs56h2hjT/wSnOLWrV4dwrpgu8VISXKbJmBn0N0g== X-Received: by 2002:a05:6a21:3289:b0:19e:aaec:833a with SMTP id yt9-20020a056a21328900b0019eaaec833amr325862pzb.55.1707414666147; Thu, 08 Feb 2024 09:51:06 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f19-20020a056a00229300b006dbda7bcf3csm5030pfe.83.2024.02.08.09.51.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 09:51:05 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 6/7] net/tap: remove no longer used files Date: Thu, 8 Feb 2024 09:41:30 -0800 Message-ID: <20240208175051.326550-7-stephen@networkplumber.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240208175051.326550-1-stephen@networkplumber.org> References: <20240130034925.44869-1-stephen@networkplumber.org> <20240208175051.326550-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The BPF api was replaced by use of libbpf. And the BPF instruction header was replaced by the skeleton. Signed-off-by: Stephen Hemminger --- drivers/net/tap/tap_bpf_api.c | 196 ---- drivers/net/tap/tap_bpf_insns.h | 1742 ------------------------------- 2 files changed, 1938 deletions(-) delete mode 100644 drivers/net/tap/tap_bpf_api.c delete mode 100644 drivers/net/tap/tap_bpf_insns.h diff --git a/drivers/net/tap/tap_bpf_api.c b/drivers/net/tap/tap_bpf_api.c deleted file mode 100644 index 9e05e2ddf19b..000000000000 --- a/drivers/net/tap/tap_bpf_api.c +++ /dev/null @@ -1,196 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright 2017 Mellanox Technologies, Ltd - */ - -#include -#include -#include - -#include -#include - -#include - - -static int bpf_load(enum bpf_prog_type type, const struct bpf_insn *insns, - size_t insns_cnt, const char *license); - -/** - * Load BPF program (section cls_q) into the kernel and return a bpf fd - * - * @param queue_idx - * Queue index matching packet cb - * - * @return - * -1 if the BPF program couldn't be loaded. An fd (int) otherwise. - */ -int tap_flow_bpf_cls_q(__u32 queue_idx) -{ - cls_q_insns[1].imm = queue_idx; - - return bpf_load(BPF_PROG_TYPE_SCHED_CLS, - (struct bpf_insn *)cls_q_insns, - RTE_DIM(cls_q_insns), - "Dual BSD/GPL"); -} - -/** - * Load BPF program (section l3_l4) into the kernel and return a bpf fd. - * - * @param[in] key_idx - * RSS MAP key index - * - * @param[in] map_fd - * BPF RSS map file descriptor - * - * @return - * -1 if the BPF program couldn't be loaded. An fd (int) otherwise. - */ -int tap_flow_bpf_calc_l3_l4_hash(__u32 key_idx, int map_fd) -{ - l3_l4_hash_insns[4].imm = key_idx; - l3_l4_hash_insns[9].imm = map_fd; - - return bpf_load(BPF_PROG_TYPE_SCHED_ACT, - (struct bpf_insn *)l3_l4_hash_insns, - RTE_DIM(l3_l4_hash_insns), - "Dual BSD/GPL"); -} - -/** - * Helper function to convert a pointer to unsigned 64 bits - * - * @param[in] ptr - * pointer to address - * - * @return - * 64 bit unsigned long type of pointer address - */ -static inline __u64 ptr_to_u64(const void *ptr) -{ - return (__u64)(unsigned long)ptr; -} - -/** - * Call BPF system call - * - * @param[in] cmd - * BPF command for program loading, map creation, map entry update, etc - * - * @param[in] attr - * System call attributes relevant to system call command - * - * @param[in] size - * size of attr parameter - * - * @return - * -1 if BPF system call failed, 0 otherwise - */ -static inline int sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr, - unsigned int size) -{ -#ifdef __NR_bpf - return syscall(__NR_bpf, cmd, attr, size); -#else - TAP_LOG(ERR, "No bpf syscall, kernel headers too old?\n"); - errno = ENOSYS; - return -1; -#endif -} - -/** - * Load BPF instructions to kernel - * - * @param[in] type - * BPF program type: classifier or action - * - * @param[in] insns - * Array of BPF instructions (equivalent to BPF instructions) - * - * @param[in] insns_cnt - * Number of BPF instructions (size of array) - * - * @param[in] license - * License string that must be acknowledged by the kernel - * - * @return - * -1 if the BPF program couldn't be loaded, fd (file descriptor) otherwise - */ -static int bpf_load(enum bpf_prog_type type, - const struct bpf_insn *insns, - size_t insns_cnt, - const char *license) -{ - union bpf_attr attr = {}; - - bzero(&attr, sizeof(attr)); - attr.prog_type = type; - attr.insn_cnt = (__u32)insns_cnt; - attr.insns = ptr_to_u64(insns); - attr.license = ptr_to_u64(license); - attr.log_buf = ptr_to_u64(NULL); - attr.log_level = 0; - attr.kern_version = 0; - - return sys_bpf(BPF_PROG_LOAD, &attr, sizeof(attr)); -} - -/** - * Create BPF map for RSS rules - * - * @param[in] key_size - * map RSS key size - * - * @param[in] value_size - * Map RSS value size - * - * @param[in] max_entries - * Map max number of RSS entries (limit on max RSS rules) - * - * @return - * -1 if BPF map couldn't be created, map fd otherwise - */ -int tap_flow_bpf_rss_map_create(unsigned int key_size, - unsigned int value_size, - unsigned int max_entries) -{ - union bpf_attr attr = {}; - - bzero(&attr, sizeof(attr)); - attr.map_type = BPF_MAP_TYPE_HASH; - attr.key_size = key_size; - attr.value_size = value_size; - attr.max_entries = max_entries; - - return sys_bpf(BPF_MAP_CREATE, &attr, sizeof(attr)); -} - -/** - * Update RSS entry in BPF map - * - * @param[in] fd - * RSS map fd - * - * @param[in] key - * Pointer to RSS key whose entry is updated - * - * @param[in] value - * Pointer to RSS new updated value - * - * @return - * -1 if RSS entry failed to be updated, 0 otherwise - */ -int tap_flow_bpf_update_rss_elem(int fd, void *key, void *value) -{ - union bpf_attr attr = {}; - - bzero(&attr, sizeof(attr)); - - attr.map_type = BPF_MAP_TYPE_HASH; - attr.map_fd = fd; - attr.key = ptr_to_u64(key); - attr.value = ptr_to_u64(value); - attr.flags = BPF_ANY; - - return sys_bpf(BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr)); -} diff --git a/drivers/net/tap/tap_bpf_insns.h b/drivers/net/tap/tap_bpf_insns.h deleted file mode 100644 index b59b7e9141fd..000000000000 --- a/drivers/net/tap/tap_bpf_insns.h +++ /dev/null @@ -1,1742 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Auto-generated from tap_bpf_program.c - * This not the original source file. Do NOT edit it. - */ - - -static struct bpf_insn cls_q_insns[] = { - {0x61, 2, 1, 52, 0x00000000}, - {0x18, 3, 0, 0, 0xdeadbeef}, - {0x00, 0, 0, 0, 0x00000000}, - {0x63, 10, 3, -4, 0x00000000}, - {0xb7, 0, 0, 0, 0x00000000}, - {0x61, 3, 10, -4, 0x00000000}, - {0x07, 3, 0, 0, 0x7cafe800}, - {0x67, 3, 0, 0, 0x00000020}, - {0x77, 3, 0, 0, 0x00000020}, - {0x5d, 2, 3, 4, 0x00000000}, - {0xb7, 2, 0, 0, 0x00000000}, - {0x63, 1, 2, 52, 0x00000000}, - {0x18, 0, 0, 0, 0xffffffff}, - {0x00, 0, 0, 0, 0x00000000}, - {0x95, 0, 0, 0, 0x00000000}, -}; - -static struct bpf_insn l3_l4_hash_insns[] = { - {0xbf, 7, 1, 0, 0x00000000}, - {0x61, 6, 7, 16, 0x00000000}, - {0x61, 8, 7, 76, 0x00000000}, - {0x61, 9, 7, 80, 0x00000000}, - {0x18, 1, 0, 0, 0xdeadbeef}, - {0x00, 0, 0, 0, 0x00000000}, - {0x63, 10, 1, -4, 0x00000000}, - {0xbf, 2, 10, 0, 0x00000000}, - {0x07, 2, 0, 0, 0xfffffffc}, - {0x18, 1, 0, 0, 0x00000000}, - {0x00, 0, 0, 0, 0x00000000}, - {0x85, 0, 0, 0, 0x00000001}, - {0x55, 0, 0, 21, 0x00000000}, - {0xb7, 1, 0, 0, 0x00000a64}, - {0x6b, 10, 1, -16, 0x00000000}, - {0x18, 1, 0, 0, 0x69666e6f}, - {0x00, 0, 0, 0, 0x65727567}, - {0x7b, 10, 1, -24, 0x00000000}, - {0x18, 1, 0, 0, 0x6e207369}, - {0x00, 0, 0, 0, 0x6320746f}, - {0x7b, 10, 1, -32, 0x00000000}, - {0x18, 1, 0, 0, 0x20737372}, - {0x00, 0, 0, 0, 0x2079656b}, - {0x7b, 10, 1, -40, 0x00000000}, - {0x18, 1, 0, 0, 0x68736168}, - {0x00, 0, 0, 0, 0x203a2928}, - {0x7b, 10, 1, -48, 0x00000000}, - {0xb7, 7, 0, 0, 0x00000000}, - {0x73, 10, 7, -14, 0x00000000}, - {0xbf, 1, 10, 0, 0x00000000}, - {0x07, 1, 0, 0, 0xffffffd0}, - {0xb7, 2, 0, 0, 0x00000023}, - {0x85, 0, 0, 0, 0x00000006}, - {0x05, 0, 0, 1680, 0x00000000}, - {0xb7, 1, 0, 0, 0x0000000e}, - {0x61, 2, 7, 20, 0x00000000}, - {0x15, 2, 0, 10, 0x00000000}, - {0x61, 2, 7, 28, 0x00000000}, - {0x55, 2, 0, 8, 0x0000a888}, - {0xbf, 2, 7, 0, 0x00000000}, - {0xb7, 7, 0, 0, 0x00000000}, - {0xbf, 1, 8, 0, 0x00000000}, - {0x07, 1, 0, 0, 0x00000012}, - {0x2d, 1, 9, 1670, 0x00000000}, - {0xb7, 1, 0, 0, 0x00000012}, - {0x69, 6, 8, 16, 0x00000000}, - {0xbf, 7, 2, 0, 0x00000000}, - {0x57, 6, 0, 0, 0x0000ffff}, - {0x7b, 10, 7, -56, 0x00000000}, - {0x15, 6, 0, 443, 0x0000dd86}, - {0xb7, 7, 0, 0, 0x00000003}, - {0x55, 6, 0, 1662, 0x00000008}, - {0x0f, 8, 1, 0, 0x00000000}, - {0xb7, 7, 0, 0, 0x00000000}, - {0xbf, 1, 8, 0, 0x00000000}, - {0x07, 1, 0, 0, 0x00000018}, - {0x2d, 1, 9, 1657, 0x00000000}, - {0xb7, 1, 0, 0, 0x00000000}, - {0x71, 3, 8, 12, 0x00000000}, - {0x71, 2, 8, 9, 0x00000000}, - {0x15, 2, 0, 1, 0x00000011}, - {0x55, 2, 0, 21, 0x00000006}, - {0x71, 2, 8, 7, 0x00000000}, - {0x71, 4, 8, 6, 0x00000000}, - {0xbf, 5, 4, 0, 0x00000000}, - {0x67, 5, 0, 0, 0x00000008}, - {0x57, 5, 0, 0, 0x00001f00}, - {0x4f, 5, 2, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000020}, - {0x4f, 4, 5, 0, 0x00000000}, - {0x55, 4, 0, 12, 0x00000000}, - {0xbf, 2, 8, 0, 0x00000000}, - {0x07, 2, 0, 0, 0x00000014}, - {0x71, 4, 2, 0, 0x00000000}, - {0x67, 4, 0, 0, 0x00000018}, - {0x71, 1, 2, 1, 0x00000000}, - {0x67, 1, 0, 0, 0x00000010}, - {0x4f, 1, 4, 0, 0x00000000}, - {0x71, 4, 2, 3, 0x00000000}, - {0x4f, 1, 4, 0, 0x00000000}, - {0x71, 2, 2, 2, 0x00000000}, - {0x67, 2, 0, 0, 0x00000008}, - {0x4f, 1, 2, 0, 0x00000000}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x67, 4, 0, 0, 0x00000038}, - {0xc7, 4, 0, 0, 0x00000038}, - {0xb7, 2, 0, 0, 0x00000000}, - {0x65, 4, 0, 1, 0xffffffff}, - {0xb7, 7, 0, 0, 0x2cc681d1}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000040}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x598d03a2}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000020}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xb31a0745}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000010}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x66340e8a}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000008}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xcc681d15}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000004}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x98d03a2b}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000002}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x31a07456}, - {0x71, 4, 8, 13, 0x00000000}, - {0x57, 3, 0, 0, 0x00000001}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x6340e8ad}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x67, 3, 0, 0, 0x00000038}, - {0xc7, 3, 0, 0, 0x00000038}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xa7, 5, 0, 0, 0xc681d15b}, - {0x6d, 2, 3, 1, 0x00000000}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000040}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x8d03a2b7}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000020}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x1a07456f}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000010}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x340e8ade}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000008}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x681d15bd}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000004}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xd03a2b7b}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000002}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xa07456f6}, - {0x71, 3, 8, 14, 0x00000000}, - {0x57, 4, 0, 0, 0x00000001}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x40e8aded}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x67, 4, 0, 0, 0x00000038}, - {0xc7, 4, 0, 0, 0x00000038}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xa7, 7, 0, 0, 0x81d15bdb}, - {0x6d, 2, 4, 1, 0x00000000}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000040}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x03a2b7b7}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000020}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x07456f6f}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000010}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x0e8adedf}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000008}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x1d15bdbf}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000004}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x3a2b7b7e}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000002}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x7456f6fd}, - {0x71, 4, 8, 15, 0x00000000}, - {0x57, 3, 0, 0, 0x00000001}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xe8adedfa}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x67, 3, 0, 0, 0x00000038}, - {0xc7, 3, 0, 0, 0x00000038}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xa7, 5, 0, 0, 0xd15bdbf4}, - {0x6d, 2, 3, 1, 0x00000000}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000040}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xa2b7b7e9}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000020}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x456f6fd3}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000010}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x8adedfa7}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000008}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x15bdbf4f}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000004}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x2b7b7e9e}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000002}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x56f6fd3d}, - {0x71, 3, 8, 16, 0x00000000}, - {0x57, 4, 0, 0, 0x00000001}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xadedfa7b}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x67, 4, 0, 0, 0x00000038}, - {0xc7, 4, 0, 0, 0x00000038}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xa7, 7, 0, 0, 0x5bdbf4f7}, - {0x6d, 2, 4, 1, 0x00000000}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000040}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xb7b7e9ef}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000020}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x6f6fd3df}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000010}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xdedfa7bf}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000008}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xbdbf4f7f}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000004}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x7b7e9eff}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000002}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xf6fd3dff}, - {0x71, 4, 8, 17, 0x00000000}, - {0x57, 3, 0, 0, 0x00000001}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xedfa7bfe}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x67, 3, 0, 0, 0x00000038}, - {0xc7, 3, 0, 0, 0x00000038}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xa7, 5, 0, 0, 0xdbf4f7fc}, - {0x6d, 2, 3, 1, 0x00000000}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000040}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xb7e9eff9}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000020}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x6fd3dff2}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000010}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xdfa7bfe5}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000008}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xbf4f7fca}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000004}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x7e9eff94}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000002}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xfd3dff28}, - {0x71, 3, 8, 18, 0x00000000}, - {0x57, 4, 0, 0, 0x00000001}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xfa7bfe51}, - {0xbf, 6, 3, 0, 0x00000000}, - {0x67, 6, 0, 0, 0x00000038}, - {0xc7, 6, 0, 0, 0x00000038}, - {0xbf, 4, 5, 0, 0x00000000}, - {0xa7, 4, 0, 0, 0xf4f7fca2}, - {0x6d, 2, 6, 1, 0x00000000}, - {0xbf, 4, 5, 0, 0x00000000}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000040}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xe9eff945}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000020}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xd3dff28a}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000010}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xa7bfe514}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000008}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x4f7fca28}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000004}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x9eff9450}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000002}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x3dff28a0}, - {0x71, 5, 8, 19, 0x00000000}, - {0x57, 3, 0, 0, 0x00000001}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x7bfe5141}, - {0xbf, 3, 5, 0, 0x00000000}, - {0x67, 3, 0, 0, 0x00000038}, - {0xc7, 3, 0, 0, 0x00000038}, - {0xbf, 7, 4, 0, 0x00000000}, - {0xa7, 7, 0, 0, 0xf7fca283}, - {0x6d, 2, 3, 1, 0x00000000}, - {0xbf, 7, 4, 0, 0x00000000}, - {0xbf, 3, 5, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000040}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xeff94506}, - {0xbf, 3, 5, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000020}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xdff28a0c}, - {0xbf, 3, 5, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000010}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xbfe51418}, - {0xbf, 3, 5, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000008}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x7fca2831}, - {0xbf, 3, 5, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000004}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xff945063}, - {0xbf, 3, 5, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000002}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xff28a0c6}, - {0x57, 5, 0, 0, 0x00000001}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xfe51418c}, - {0xbf, 4, 1, 0, 0x00000000}, - {0x67, 4, 0, 0, 0x00000020}, - {0xc7, 4, 0, 0, 0x00000020}, - {0xbf, 3, 7, 0, 0x00000000}, - {0xa7, 3, 0, 0, 0xfca28319}, - {0x6d, 2, 4, 1, 0x00000000}, - {0xbf, 3, 7, 0, 0x00000000}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x40000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xf9450633}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x20000000}, - {0x79, 6, 10, -56, 0x00000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xf28a0c67}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x10000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xe51418ce}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x08000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xca28319d}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x04000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x9450633b}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x02000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x28a0c676}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x01000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x51418ced}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00800000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xa28319db}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00400000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x450633b6}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00200000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x8a0c676c}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00100000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x1418ced8}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00080000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x28319db1}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00040000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x50633b63}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00020000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xa0c676c6}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00010000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x418ced8d}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00008000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x8319db1a}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00004000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x0633b634}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00002000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x0c676c68}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00001000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x18ced8d1}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000800}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x319db1a3}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000400}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x633b6347}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000200}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xc676c68f}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000100}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x8ced8d1f}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000080}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x19db1a3e}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000040}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x33b6347d}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000020}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x676c68fa}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000010}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xced8d1f4}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000008}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x9db1a3e9}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000004}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x3b6347d2}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000002}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x76c68fa5}, - {0x57, 1, 0, 0, 0x00000001}, - {0x15, 1, 0, 1194, 0x00000000}, - {0xa7, 3, 0, 0, 0xed8d1f4a}, - {0x05, 0, 0, 1192, 0x00000000}, - {0x0f, 8, 1, 0, 0x00000000}, - {0xb7, 7, 0, 0, 0x00000000}, - {0xbf, 1, 8, 0, 0x00000000}, - {0x07, 1, 0, 0, 0x0000002c}, - {0x2d, 1, 9, 1216, 0x00000000}, - {0x61, 2, 8, 8, 0x00000000}, - {0xdc, 2, 0, 0, 0x00000040}, - {0xc7, 2, 0, 0, 0x00000020}, - {0x71, 3, 8, 6, 0x00000000}, - {0x15, 3, 0, 2, 0x00000011}, - {0xb7, 1, 0, 0, 0x00000000}, - {0x55, 3, 0, 12, 0x00000006}, - {0xbf, 3, 8, 0, 0x00000000}, - {0x07, 3, 0, 0, 0x00000028}, - {0x71, 4, 3, 0, 0x00000000}, - {0x67, 4, 0, 0, 0x00000018}, - {0x71, 1, 3, 1, 0x00000000}, - {0x67, 1, 0, 0, 0x00000010}, - {0x4f, 1, 4, 0, 0x00000000}, - {0x71, 4, 3, 3, 0x00000000}, - {0x4f, 1, 4, 0, 0x00000000}, - {0x71, 3, 3, 2, 0x00000000}, - {0x67, 3, 0, 0, 0x00000008}, - {0x4f, 1, 3, 0, 0x00000000}, - {0xbf, 4, 2, 0, 0x00000000}, - {0x77, 4, 0, 0, 0x0000001f}, - {0x57, 4, 0, 0, 0x2cc681d1}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x40000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x598d03a2}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x20000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xb31a0745}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x10000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x66340e8a}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x08000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xcc681d15}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x04000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x98d03a2b}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x02000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x31a07456}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x01000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x6340e8ad}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00800000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xc681d15b}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00400000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x8d03a2b7}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00200000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x1a07456f}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00100000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x340e8ade}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00080000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x681d15bd}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00040000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xd03a2b7b}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00020000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xa07456f6}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00010000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x40e8aded}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00008000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x81d15bdb}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00004000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x03a2b7b7}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00002000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x07456f6f}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00001000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x0e8adedf}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000800}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x1d15bdbf}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000400}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x3a2b7b7e}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000200}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x7456f6fd}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000100}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xe8adedfa}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000080}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xd15bdbf4}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000040}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xa2b7b7e9}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000020}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x456f6fd3}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000010}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x8adedfa7}, - {0xbf, 3, 2, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000008}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x15bdbf4f}, - {0x61, 3, 8, 12, 0x00000000}, - {0xbf, 5, 2, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000004}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x2b7b7e9e}, - {0xdc, 3, 0, 0, 0x00000040}, - {0xbf, 5, 2, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000002}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x56f6fd3d}, - {0xc7, 3, 0, 0, 0x00000020}, - {0x57, 2, 0, 0, 0x00000001}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xadedfa7b}, - {0xb7, 2, 0, 0, 0x00000000}, - {0xbf, 5, 4, 0, 0x00000000}, - {0xa7, 5, 0, 0, 0x5bdbf4f7}, - {0x6d, 2, 3, 1, 0x00000000}, - {0xbf, 5, 4, 0, 0x00000000}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x40000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xb7b7e9ef}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x20000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x6f6fd3df}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x10000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xdedfa7bf}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x08000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xbdbf4f7f}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x04000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x7b7e9eff}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x02000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xf6fd3dff}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x01000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xedfa7bfe}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00800000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xdbf4f7fc}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00400000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xb7e9eff9}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00200000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x6fd3dff2}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00100000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xdfa7bfe5}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00080000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xbf4f7fca}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00040000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x7e9eff94}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00020000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xfd3dff28}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00010000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xfa7bfe51}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00008000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xf4f7fca2}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00004000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xe9eff945}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00002000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xd3dff28a}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00001000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xa7bfe514}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000800}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x4f7fca28}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000400}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x9eff9450}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000200}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x3dff28a0}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000100}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x7bfe5141}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000080}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xf7fca283}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000040}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xeff94506}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000020}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xdff28a0c}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000010}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xbfe51418}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000008}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x7fca2831}, - {0x61, 4, 8, 16, 0x00000000}, - {0xbf, 6, 3, 0, 0x00000000}, - {0x57, 6, 0, 0, 0x00000004}, - {0x15, 6, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xff945063}, - {0xdc, 4, 0, 0, 0x00000040}, - {0xbf, 6, 3, 0, 0x00000000}, - {0x57, 6, 0, 0, 0x00000002}, - {0x15, 6, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xff28a0c6}, - {0xc7, 4, 0, 0, 0x00000020}, - {0x57, 3, 0, 0, 0x00000001}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xfe51418c}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xa7, 7, 0, 0, 0xfca28319}, - {0x6d, 2, 4, 1, 0x00000000}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x40000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xf9450633}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x20000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xf28a0c67}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x10000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xe51418ce}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x08000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xca28319d}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x04000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x9450633b}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x02000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x28a0c676}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x01000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x51418ced}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00800000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xa28319db}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00400000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x450633b6}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00200000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x8a0c676c}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00100000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x1418ced8}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00080000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x28319db1}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00040000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x50633b63}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00020000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xa0c676c6}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00010000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x418ced8d}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00008000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x8319db1a}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00004000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x0633b634}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00002000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x0c676c68}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00001000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x18ced8d1}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000800}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x319db1a3}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000400}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x633b6347}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000200}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xc676c68f}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000100}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x8ced8d1f}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000080}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x19db1a3e}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000040}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x33b6347d}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000020}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x676c68fa}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000010}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xced8d1f4}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000008}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x9db1a3e9}, - {0x61, 3, 8, 20, 0x00000000}, - {0xbf, 5, 4, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000004}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x3b6347d2}, - {0xdc, 3, 0, 0, 0x00000040}, - {0xbf, 5, 4, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000002}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x76c68fa5}, - {0xc7, 3, 0, 0, 0x00000020}, - {0x57, 4, 0, 0, 0x00000001}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xed8d1f4a}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xa7, 5, 0, 0, 0xdb1a3e94}, - {0x6d, 2, 3, 1, 0x00000000}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x40000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xb6347d28}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x20000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x6c68fa51}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x10000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xd8d1f4a3}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x08000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xb1a3e946}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x04000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x6347d28d}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x02000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xc68fa51a}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x01000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x8d1f4a35}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00800000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x1a3e946b}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00400000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x347d28d7}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00200000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x68fa51ae}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00100000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xd1f4a35c}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00080000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xa3e946b9}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00040000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x47d28d73}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00020000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x8fa51ae7}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00010000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x1f4a35cf}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00008000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x3e946b9e}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00004000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x7d28d73c}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00002000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xfa51ae78}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00001000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xf4a35cf1}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000800}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xe946b9e3}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000400}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xd28d73c7}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000200}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xa51ae78e}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000100}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x4a35cf1c}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000080}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x946b9e38}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000040}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x28d73c71}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000020}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x51ae78e3}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000010}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xa35cf1c6}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000008}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x46b9e38d}, - {0x61, 4, 8, 24, 0x00000000}, - {0xbf, 6, 3, 0, 0x00000000}, - {0x57, 6, 0, 0, 0x00000004}, - {0x15, 6, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x8d73c71b}, - {0xdc, 4, 0, 0, 0x00000040}, - {0xbf, 6, 3, 0, 0x00000000}, - {0x57, 6, 0, 0, 0x00000002}, - {0x15, 6, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x1ae78e36}, - {0xc7, 4, 0, 0, 0x00000020}, - {0x57, 3, 0, 0, 0x00000001}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x35cf1c6c}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xa7, 7, 0, 0, 0x6b9e38d9}, - {0x6d, 2, 4, 1, 0x00000000}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x40000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xd73c71b2}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x20000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xae78e364}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x10000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x5cf1c6c9}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x08000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xb9e38d92}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x04000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x73c71b25}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x02000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xe78e364b}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x01000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xcf1c6c96}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00800000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x9e38d92c}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00400000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x3c71b259}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00200000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x78e364b2}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00100000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xf1c6c964}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00080000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xe38d92c9}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00040000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xc71b2593}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00020000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x8e364b27}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00010000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x1c6c964e}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00008000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x38d92c9c}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00004000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x71b25938}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00002000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xe364b270}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00001000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xc6c964e0}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000800}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x8d92c9c0}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000400}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x1b259380}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000200}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x364b2700}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000100}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x6c964e01}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000080}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xd92c9c03}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000040}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xb2593807}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000020}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x64b2700f}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000010}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xc964e01e}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000008}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x92c9c03d}, - {0x61, 3, 8, 28, 0x00000000}, - {0xbf, 5, 4, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000004}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x2593807a}, - {0xdc, 3, 0, 0, 0x00000040}, - {0xbf, 5, 4, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000002}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x4b2700f4}, - {0xc7, 3, 0, 0, 0x00000020}, - {0x57, 4, 0, 0, 0x00000001}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x964e01e8}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xa7, 5, 0, 0, 0x2c9c03d1}, - {0x6d, 2, 3, 1, 0x00000000}, - {0xbf, 5, 7, 0, 0x00000000}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x40000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x593807a3}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x20000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xb2700f46}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x10000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x64e01e8d}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x08000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xc9c03d1a}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x04000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x93807a35}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x02000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x2700f46b}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x01000000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x4e01e8d6}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00800000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x9c03d1ad}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00400000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x3807a35b}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00200000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x700f46b6}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00100000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xe01e8d6c}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00080000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xc03d1ad9}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00040000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x807a35b3}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00020000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x00f46b66}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00010000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x01e8d6cc}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00008000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x03d1ad99}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00004000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x07a35b32}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00002000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x0f46b665}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00001000}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x1e8d6cca}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000800}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x3d1ad994}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000400}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x7a35b328}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000200}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xf46b6651}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000100}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xe8d6cca2}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000080}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xd1ad9944}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000040}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xa35b3289}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000020}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x46b66512}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000010}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x8d6cca25}, - {0xbf, 4, 3, 0, 0x00000000}, - {0x57, 4, 0, 0, 0x00000008}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x1ad9944a}, - {0x61, 4, 8, 32, 0x00000000}, - {0xbf, 6, 3, 0, 0x00000000}, - {0x57, 6, 0, 0, 0x00000004}, - {0x15, 6, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x35b32894}, - {0xdc, 4, 0, 0, 0x00000040}, - {0xbf, 6, 3, 0, 0x00000000}, - {0x57, 6, 0, 0, 0x00000002}, - {0x15, 6, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0x6b665129}, - {0xc7, 4, 0, 0, 0x00000020}, - {0x57, 3, 0, 0, 0x00000001}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 5, 0, 0, 0xd6cca253}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xa7, 7, 0, 0, 0xad9944a7}, - {0x6d, 2, 4, 1, 0x00000000}, - {0xbf, 7, 5, 0, 0x00000000}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x40000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x5b32894f}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x20000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xb665129f}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x10000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x6cca253e}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x08000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xd9944a7d}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x04000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xb32894fb}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x02000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x665129f6}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x01000000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xcca253ec}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00800000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x9944a7d9}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00400000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x32894fb2}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00200000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x65129f65}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00100000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xca253eca}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00080000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x944a7d95}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00040000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x2894fb2a}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00020000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x5129f655}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00010000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xa253ecab}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00008000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x44a7d956}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00004000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x894fb2ac}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00002000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x129f6558}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00001000}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x253ecab1}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000800}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x4a7d9563}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000400}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x94fb2ac7}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000200}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x29f6558f}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000100}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x53ecab1e}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000080}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xa7d9563d}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000040}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x4fb2ac7a}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000020}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x9f6558f5}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000010}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x3ecab1ea}, - {0xbf, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x00000008}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0x7d9563d5}, - {0x61, 3, 8, 36, 0x00000000}, - {0xbf, 5, 4, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000004}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xfb2ac7ab}, - {0xdc, 3, 0, 0, 0x00000040}, - {0xbf, 5, 4, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000002}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xf6558f56}, - {0xc7, 3, 0, 0, 0x00000020}, - {0x57, 4, 0, 0, 0x00000001}, - {0x15, 4, 0, 1, 0x00000000}, - {0xa7, 7, 0, 0, 0xecab1eac}, - {0xbf, 4, 7, 0, 0x00000000}, - {0xa7, 4, 0, 0, 0xd9563d59}, - {0x6d, 2, 3, 1, 0x00000000}, - {0xbf, 4, 7, 0, 0x00000000}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x40000000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xb2ac7ab2}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x20000000}, - {0x79, 6, 10, -56, 0x00000000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x6558f564}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x10000000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xcab1eac8}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x08000000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x9563d590}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x04000000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x2ac7ab20}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x02000000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x558f5641}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x01000000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xab1eac83}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00800000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x563d5906}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00400000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xac7ab20c}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00200000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x58f56418}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00100000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xb1eac831}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00080000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x63d59063}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00040000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xc7ab20c7}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00020000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x8f56418f}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00010000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x1eac831e}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00008000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x3d59063c}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00004000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x7ab20c78}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00002000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xf56418f0}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00001000}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xeac831e1}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000800}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xd59063c2}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000400}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xab20c784}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000200}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x56418f09}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000100}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xac831e12}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000080}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x59063c25}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000040}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xb20c784b}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000020}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x6418f097}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000010}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0xc831e12f}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000008}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x9063c25f}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000004}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x20c784be}, - {0xbf, 5, 3, 0, 0x00000000}, - {0x57, 5, 0, 0, 0x00000002}, - {0x15, 5, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x418f097c}, - {0x57, 3, 0, 0, 0x00000001}, - {0x15, 3, 0, 1, 0x00000000}, - {0xa7, 4, 0, 0, 0x831e12f9}, - {0xbf, 5, 1, 0, 0x00000000}, - {0x67, 5, 0, 0, 0x00000020}, - {0xc7, 5, 0, 0, 0x00000020}, - {0xbf, 3, 4, 0, 0x00000000}, - {0xa7, 3, 0, 0, 0x063c25f3}, - {0x6d, 2, 5, 1, 0x00000000}, - {0xbf, 3, 4, 0, 0x00000000}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x40000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x0c784be7}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x20000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x18f097cf}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x10000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x31e12f9f}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x08000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x63c25f3f}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x04000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xc784be7f}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x02000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x8f097cff}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x01000000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x1e12f9fe}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00800000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x3c25f3fc}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00400000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x784be7f8}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00200000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xf097cff0}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00100000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xe12f9fe0}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00080000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xc25f3fc1}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00040000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x84be7f83}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00020000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x097cff07}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00010000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x12f9fe0f}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00008000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x25f3fc1f}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00004000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x4be7f83f}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00002000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x97cff07f}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00001000}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x2f9fe0fe}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000800}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x5f3fc1fd}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000400}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xbe7f83fb}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000200}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x7cff07f7}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000100}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xf9fe0fee}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000080}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xf3fc1fdc}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000040}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xe7f83fb8}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000020}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xcff07f70}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000010}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x9fe0fee1}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000008}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x3fc1fdc2}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000004}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0x7f83fb85}, - {0xbf, 2, 1, 0, 0x00000000}, - {0x57, 2, 0, 0, 0x00000002}, - {0x15, 2, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xff07f70a}, - {0x57, 1, 0, 0, 0x00000001}, - {0x15, 1, 0, 1, 0x00000000}, - {0xa7, 3, 0, 0, 0xfe0fee15}, - {0x71, 1, 0, 45, 0x00000000}, - {0x67, 1, 0, 0, 0x00000008}, - {0x71, 2, 0, 44, 0x00000000}, - {0x4f, 1, 2, 0, 0x00000000}, - {0x71, 2, 0, 46, 0x00000000}, - {0x67, 2, 0, 0, 0x00000010}, - {0x71, 4, 0, 47, 0x00000000}, - {0x67, 4, 0, 0, 0x00000018}, - {0x4f, 4, 2, 0, 0x00000000}, - {0x4f, 4, 1, 0, 0x00000000}, - {0x67, 3, 0, 0, 0x00000020}, - {0x77, 3, 0, 0, 0x00000020}, - {0x9f, 3, 4, 0, 0x00000000}, - {0x57, 3, 0, 0, 0x0000000f}, - {0x67, 3, 0, 0, 0x00000002}, - {0x0f, 0, 3, 0, 0x00000000}, - {0x71, 1, 0, 49, 0x00000000}, - {0x67, 1, 0, 0, 0x00000008}, - {0x71, 2, 0, 48, 0x00000000}, - {0x4f, 1, 2, 0, 0x00000000}, - {0x71, 2, 0, 50, 0x00000000}, - {0x67, 2, 0, 0, 0x00000010}, - {0x71, 3, 0, 51, 0x00000000}, - {0x67, 3, 0, 0, 0x00000018}, - {0x4f, 3, 2, 0, 0x00000000}, - {0x4f, 3, 1, 0, 0x00000000}, - {0x07, 3, 0, 0, 0x7cafe800}, - {0x63, 6, 3, 52, 0x00000000}, - {0xb7, 7, 0, 0, 0x00000001}, - {0xbf, 0, 7, 0, 0x00000000}, - {0x95, 0, 0, 0, 0x00000000}, -}; From patchwork Thu Feb 8 17:41:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Hemminger X-Patchwork-Id: 136556 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1AA7343AE1; Thu, 8 Feb 2024 18:52:01 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2ABEF42E76; Thu, 8 Feb 2024 18:51:12 +0100 (CET) Received: from mail-oi1-f171.google.com (mail-oi1-f171.google.com [209.85.167.171]) by mails.dpdk.org (Postfix) with ESMTP id D37C542E59 for ; Thu, 8 Feb 2024 18:51:07 +0100 (CET) Received: by mail-oi1-f171.google.com with SMTP id 5614622812f47-3bfd27f6dc0so511b6e.1 for ; Thu, 08 Feb 2024 09:51:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1707414667; x=1708019467; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GzI3jCIABWsauwkYS1usLzNZOxVjowFgorEUcM3fRJU=; b=fpSDL+YelSaFq82HTnK8UlqVNHaNh4NplCo2iiVr8ur/f4Zmu0fwnPMG4u2MZ+O4Vk TJrLT0zZuqcv+T8kcDNlOPB5ApQPJao5TH7EN7xeJlzkE1Bpf0XwNIjDOLpzOFyK8fHz lTsQZz1w7osU0qrTa+YrdCuSfhbp9aWYzEhBLkauEh3Oj6VjNMsBH6IDDWRgcB6QOYku +Z22XWho3UMbQYi/QDlMHJh/iEOCsuZ/o8gjfRaSYNEv+tCiCxzw6HDuJNPhaNIX1294 7oiIwbhRcltCnh9ZsmaSkcchc6oO68NhDMddUTzS3HlkiULB0cQ7i9LgqQ1mZxq4UVkt DLUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707414667; x=1708019467; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GzI3jCIABWsauwkYS1usLzNZOxVjowFgorEUcM3fRJU=; b=F43kk8MRvW8csDeBwWwFbHopYJqkDE0eBdS6DFTOx463YMcJjci/PUJWhttZIKeLLe GoeViMp8ezzgAuNOcU27y5OkhFHcHTZgkx3SE2w1XSjTNgaYZW5AAQNDIwfonAXQqdMy P9FKm/+dahszRVTTaEfgsILVl6CIUi/E+qW2x4lZo7dQR5mR/FWk8PgbayrA5VkgLlV+ +K7cq7uR2ZsB9MSS/s80olFkKbvtAADq0YMbQqlIZhViZCs1Elxa0VdZDCZjRmpT6wXA rxTMZw1LFNDJEuJgn0vdWPb1D8ah6tf3Bwu/IP154FyA1o79pim1Ygdl6NQsR4c/e03p Af1Q== X-Gm-Message-State: AOJu0YxpZLnAdXb6HkvpU/bMPj/ps45Nx0bKn1GxU5XEWvjx9oZZKs/2 oe+ZKlmA0Fu7+VhgClCe8yZ9OYV+f7sWcg3jfS3uKx53mSk1CxVgYG/qDBcV1TJL0hK9VrFJiG+ x5RM= X-Google-Smtp-Source: AGHT+IHgiXdyR5alp5KIoqlCO8EaSVutMx/cTXEwGRZ58Jh0vbNh3zpyc82I18UWrG00yok3nkyTHw== X-Received: by 2002:a05:6359:230a:b0:176:71c3:80db with SMTP id lk10-20020a056359230a00b0017671c380dbmr7342857rwb.16.1707414667018; Thu, 08 Feb 2024 09:51:07 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id f19-20020a056a00229300b006dbda7bcf3csm5030pfe.83.2024.02.08.09.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 09:51:06 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Subject: [PATCH v3 7/7] MAINTAINERS: add maintainer for TAP device Date: Thu, 8 Feb 2024 09:41:31 -0800 Message-ID: <20240208175051.326550-8-stephen@networkplumber.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240208175051.326550-1-stephen@networkplumber.org> References: <20240130034925.44869-1-stephen@networkplumber.org> <20240208175051.326550-1-stephen@networkplumber.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add myself as maintainer for TAP device. Signed-off-by: Stephen Hemminger --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index 5fb3a73f840e..92d27e97aa9e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1015,6 +1015,7 @@ F: doc/guides/nics/pcap_ring.rst F: doc/guides/nics/features/pcap.ini Tap PMD +M: Stephen Hemminger F: drivers/net/tap/ F: doc/guides/nics/tap.rst F: doc/guides/nics/features/tap.ini