From patchwork Fri Apr 17 07:22:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 68733 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2A388A058A; Fri, 17 Apr 2020 09:23:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EB1D01DCE5; Fri, 17 Apr 2020 09:23:25 +0200 (CEST) Received: from mail-pg1-f194.google.com (mail-pg1-f194.google.com [209.85.215.194]) by dpdk.org (Postfix) with ESMTP id BFDF31DCAE for ; Fri, 17 Apr 2020 09:23:24 +0200 (CEST) Received: by mail-pg1-f194.google.com with SMTP id o15so130948pgi.1 for ; Fri, 17 Apr 2020 00:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Hu/GBFxH1GswcPkBIY05SZ34S0YUMcJRmWUq62h29JU=; b=lpFN/ACRKPWP23S45mXa00fQLvelE80FEEB9mDJvw9SsgMXHhs1CyQV17Vb5GMYjNN OH2dSxQrho2ulpiAWSdHf0FmtJNqz3c6YedShH7ekfmuWRMlbZviasoXa0kAbdl7oWSj dTPO1K1yWHBdUyFopXflC+WSv7Fla6TEw25+DMtSHIf///6Fc/6nrOr8KkIy0M8/dS/T hjsdIuLZDhWwnh076fKNgzc/BF2resLU6aTZ3syVsrtd4AeI00mMR3OmhVJGQ3EuiB0c +iIskmj3D2bjb953SYNupUaaVB+ZT8PTF9VsdmnDOQCEEqKvg+/2EgMOlF4BW4P6U4m8 rEPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Hu/GBFxH1GswcPkBIY05SZ34S0YUMcJRmWUq62h29JU=; b=VtetOZkG9KPUhXn8/AZuAlj6xt5iugzd2fZudKzd3vZZF8Iv2cxVyDIshFV2q6O8Ed raDUvfQUzTATGh559RBBJOJ+4aJRUUX1xjR4+vk+ulPLCuPatQ/qIACnNPPY5qRgnAc2 uiwgXpgYbsR35zjDuo5ro/yFiZyO7EmJsmcWqU6hMPBtXuMvQF4tcGAtldb4St6nsvO/ TSNIdxag4ro9MuFVS0apg6/nshkSNmR+vZ6Yvq5McFqCn5tMzSBRmZL3BZJ60JZ9pXok Qs/7uAczUg/JwOpcLVpxT7MDeu0wh47UGc6sYIEieLZkF7ykcBnna2g9rI+0WDT4Ju8y LW5g== X-Gm-Message-State: AGi0PuaYKgi+6YoywIi0OAt6q4kQaul8ZHSEvGQXZvaYT+S33hkMNAXI OqVhfDdAikxoOpFAa4P4vX0= X-Google-Smtp-Source: APiQypKpa1QhN/pWV0r2knGwKs7WPp7GaEbgmAineC+QGQo5/x/Lo4SvsJM8tp1bNO88ALk4AyvR7A== X-Received: by 2002:a63:6fc6:: with SMTP id k189mr1662882pgc.175.1587108203702; Fri, 17 Apr 2020 00:23:23 -0700 (PDT) Received: from hyd1588t430.marvell.com ([115.113.156.2]) by smtp.gmail.com with ESMTPSA id v4sm18750983pfb.31.2020.04.17.00.23.19 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Apr 2020 00:23:23 -0700 (PDT) From: Nithin Dabilpuram To: thomas@monjalon.net, ferruh.yigit@intel.com, arybchenko@solarflare.com, orika@mellanox.com, cristian.dumitrescu@intel.com, anatoly.burakov@intel.com, John McNamara , Marko Kovacevic , Olivier Matz Cc: dev@dpdk.org, jerinj@marvell.com, kkanas@marvell.com, Nithin Dabilpuram Date: Fri, 17 Apr 2020 12:52:51 +0530 Message-Id: <20200417072254.11455-1-nithind1988@gmail.com> X-Mailer: git-send-email 2.8.4 Subject: [dpdk-dev] [PATCH 1/3] mbuf: add Tx offloads for packet marking X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Nithin Dabilpuram Introduce PKT_TX_MARK_IP_DSCP, PKT_TX_MARK_IP_ECN and PKT_TX_MARK_VLAN_DEI Tx offload flags to support packet marking. When packet marking feature in Traffic manager is enabled, application has to the use the three new flags to indicate to PMD on whether packet marking needs to be enabled on the specific mbuf or not. By setting the three flags, it is assumed by PMD that application has already verified the applicability of marking on that specific packet and PMD need not perform further checks as per RFC. Signed-off-by: Krzysztof Kanas Signed-off-by: Nithin Dabilpuram --- doc/guides/nics/features.rst | 14 ++++++++++++++ lib/librte_mbuf/rte_mbuf.c | 6 ++++++ lib/librte_mbuf/rte_mbuf_core.h | 36 ++++++++++++++++++++++++++++++++++-- 3 files changed, 54 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst index edd21c4..bc978fb 100644 --- a/doc/guides/nics/features.rst +++ b/doc/guides/nics/features.rst @@ -913,6 +913,20 @@ Supports to get Rx/Tx packet burst mode information. * **[implements] eth_dev_ops**: ``rx_burst_mode_get``, ``tx_burst_mode_get``. * **[related] API**: ``rte_eth_rx_burst_mode_get()``, ``rte_eth_tx_burst_mode_get()``. +.. _nic_features_traffic_manager_packet_marking_offload: + +Traffic Manager Packet marking offload +-------------------------------------- + +Supports enabling a packet marking offload specific mbuf. + +* **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_MARK_IP_DSCP``, + ``mbuf.ol_flags:PKT_TX_MARK_IP_ECN``, ``mbuf.ol_flags:PKT_TX_MARK_VLAN_DEI``, + ``mbuf.ol_flags:PKT_TX_IPV4``, ``mbuf.ol_flags:PKT_TX_IPV6``. +* **[uses] mbuf**: ``mbuf.l2_len``. +* **[related] API**: ``rte_tm_mark_ip_dscp()``, ``rte_tm_mark_ip_ecn()``, + ``rte_tm_mark_vlan_dei()``. + .. _nic_features_other: Other dev ops not represented by a Feature diff --git a/lib/librte_mbuf/rte_mbuf.c b/lib/librte_mbuf/rte_mbuf.c index cd5794d..5c6896d 100644 --- a/lib/librte_mbuf/rte_mbuf.c +++ b/lib/librte_mbuf/rte_mbuf.c @@ -880,6 +880,9 @@ const char *rte_get_tx_ol_flag_name(uint64_t mask) case PKT_TX_SEC_OFFLOAD: return "PKT_TX_SEC_OFFLOAD"; case PKT_TX_UDP_SEG: return "PKT_TX_UDP_SEG"; case PKT_TX_OUTER_UDP_CKSUM: return "PKT_TX_OUTER_UDP_CKSUM"; + case PKT_TX_MARK_VLAN_DEI: return "PKT_TX_MARK_VLAN_DEI"; + case PKT_TX_MARK_IP_DSCP: return "PKT_TX_MARK_IP_DSCP"; + case PKT_TX_MARK_IP_ECN: return "PKT_TX_MARK_IP_ECN"; default: return NULL; } } @@ -916,6 +919,9 @@ rte_get_tx_ol_flag_list(uint64_t mask, char *buf, size_t buflen) { PKT_TX_SEC_OFFLOAD, PKT_TX_SEC_OFFLOAD, NULL }, { PKT_TX_UDP_SEG, PKT_TX_UDP_SEG, NULL }, { PKT_TX_OUTER_UDP_CKSUM, PKT_TX_OUTER_UDP_CKSUM, NULL }, + { PKT_TX_MARK_VLAN_DEI, PKT_TX_MARK_VLAN_DEI, NULL }, + { PKT_TX_MARK_IP_DSCP, PKT_TX_MARK_IP_DSCP, NULL }, + { PKT_TX_MARK_IP_ECN, PKT_TX_MARK_IP_ECN, NULL }, }; const char *name; unsigned int i; diff --git a/lib/librte_mbuf/rte_mbuf_core.h b/lib/librte_mbuf/rte_mbuf_core.h index b9a59c8..d9f1290 100644 --- a/lib/librte_mbuf/rte_mbuf_core.h +++ b/lib/librte_mbuf/rte_mbuf_core.h @@ -187,11 +187,40 @@ extern "C" { /* add new RX flags here, don't forget to update PKT_FIRST_FREE */ #define PKT_FIRST_FREE (1ULL << 23) -#define PKT_LAST_FREE (1ULL << 40) +#define PKT_LAST_FREE (1ULL << 37) /* add new TX flags here, don't forget to update PKT_LAST_FREE */ /** + * Packet marking offload flags. These flags indicated what kind + * of packet marking needs to be applied on a given mbuf when + * appropriate Traffic Manager configuration is in place. + * When user set's these flags on a mbuf, below assumptions are made + * 1) When PKT_TX_MARK_VLAN_DEI is set, + * a) PMD assumes pkt to be a 802.1q packet. + * b) Application should also set mbuf.l2_len where 802.1Q header is + * at (mbuf.l2_len - 6) offset. + * 2) When PKT_TX_MARK_IP_DSCP is set, + * a) Application should also set either PKT_TX_IPV4 or PKT_TX_IPV6 + * to indicate whether if it is IPv4 packet or IPv6 packet + * for DSCP marking. It should also set PKT_TX_IP_CKSUM if it is + * IPv4 pkt. + * b) Application should also set mbuf.l2_len that indicates + * start offset of L3 header. + * 3) When PKT_TX_MARK_IP_ECN is set, + * a) Application should also set either PKT_TX_IPV4 or PKT_TX_IPV6. + * It should also set PKT_TX_IP_CKSUM if it is IPv4 pkt. + * b) PMD will assume pkt L4 protocol is either TCP or SCTP and + * ECN is set to 2'b01 or 2'b10 as per RFC 3168 and hence HW + * can mark the packet for a configured color. + * c) Application should also set mbuf.l2_len that indicates + * start offset of L3 header. + */ +#define PKT_TX_MARK_VLAN_DEI (1ULL << 38) +#define PKT_TX_MARK_IP_DSCP (1ULL << 39) +#define PKT_TX_MARK_IP_ECN (1ULL << 40) + +/** * Outer UDP checksum offload flag. This flag is used for enabling * outer UDP checksum in PMD. To use outer UDP checksum, the user needs to * 1) Enable the following in mbuf, @@ -384,7 +413,10 @@ extern "C" { PKT_TX_MACSEC | \ PKT_TX_SEC_OFFLOAD | \ PKT_TX_UDP_SEG | \ - PKT_TX_OUTER_UDP_CKSUM) + PKT_TX_OUTER_UDP_CKSUM | \ + PKT_TX_MARK_VLAN_DEI | \ + PKT_TX_MARK_IP_DSCP | \ + PKT_TX_MARK_IP_ECN) /** * Mbuf having an external buffer attached. shinfo in mbuf must be filled. From patchwork Fri Apr 17 07:22:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 68734 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 521E8A058A; Fri, 17 Apr 2020 09:23:37 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CEBFA1DD27; Fri, 17 Apr 2020 09:23:30 +0200 (CEST) Received: from mail-pj1-f66.google.com (mail-pj1-f66.google.com [209.85.216.66]) by dpdk.org (Postfix) with ESMTP id 0B0641DD21 for ; Fri, 17 Apr 2020 09:23:29 +0200 (CEST) Received: by mail-pj1-f66.google.com with SMTP id a7so498721pju.2 for ; Fri, 17 Apr 2020 00:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UVcT3F831WVaWX4eiJgnLMmsThtm8zIHpln3Jmr91gc=; b=ZxXg3L55L3nDd4VUvFY7DGiuy6Vg2HPwcHdbGFebOpZvODVBbg+YjA7sxPSS9NxjOe 0zzGUAk4erG6FW0m+CSF0o2p6Dbr7SBZY1ZFL0RI4C90geI8/Xv4G+H2ZtqowRNkIVtd 6UcBL+IoRtOqpiC8ybH1hx07QTPtIgSHGgQLMtKiI/2O2V/D2dKklVCVKcoPxQjoTItd JupQOM2J2syn0znIDZyup3hld/UdO147kTAxePVjQkSbYN0IRRa3IY8EmnaC+fHH9HVo TuFUk6KJizv2GIfbUEemFbW3waWZXyXl0s40Kyt+UBJif9CO/N1NnhiJy0XSv7DZUq+e DKKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UVcT3F831WVaWX4eiJgnLMmsThtm8zIHpln3Jmr91gc=; b=akA6thcdJoWxpNDG5cXQbNHBJVP1GkpJWgE/kTOep2fBJnfk/fDeNWcq0mHyWI0OS0 Yr0FoSY7bcXa4+LxwpMfVCcJYFSoUeoOsraC8A4p66Hy4SNPsAQUn8pf2LZn8hmnIqbI 4Vt/K117WB1qlYuqenyMGRfZfc8X/zINpSpeUaBlrbHo2KDAqufgNR/DNDnEL4zyGb6F JVGwkPKzcrGji/Oh2bXoohWYioJmLIQMs8jKCq3bkhFW3Xq4QU4VTSIOOveLzv0kyvOI pbcv517fDgaUXnSbBeluVZPPROIlq2EnMw3bdn7IFJiNEobNetYRK8qJC0God4kjcK5Y 1Mhg== X-Gm-Message-State: AGi0PuZEVM4zwP5ef85UsqPMo21XcrZVSsEvvNhBFBe3ojcwjdKkgnoN +LKe0QfE8RMBd/5a3y79qek= X-Google-Smtp-Source: APiQypIcWshkUG+t09Bvqg/eMz6JVe7X9xhrbSZnHH7fEnI/AjeyN0WDfG6PQ8asOgIfWCsf6geZkA== X-Received: by 2002:a17:902:8543:: with SMTP id d3mr2149835plo.314.1587108207950; Fri, 17 Apr 2020 00:23:27 -0700 (PDT) Received: from hyd1588t430.marvell.com ([115.113.156.2]) by smtp.gmail.com with ESMTPSA id v4sm18750983pfb.31.2020.04.17.00.23.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Apr 2020 00:23:27 -0700 (PDT) From: Nithin Dabilpuram To: thomas@monjalon.net, ferruh.yigit@intel.com, arybchenko@solarflare.com, orika@mellanox.com, cristian.dumitrescu@intel.com, anatoly.burakov@intel.com, Jerin Jacob , Nithin Dabilpuram , Kiran Kumar K Cc: dev@dpdk.org, kkanas@marvell.com Date: Fri, 17 Apr 2020 12:52:52 +0530 Message-Id: <20200417072254.11455-2-nithind1988@gmail.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20200417072254.11455-1-nithind1988@gmail.com> References: <20200417072254.11455-1-nithind1988@gmail.com> Subject: [dpdk-dev] [PATCH 2/3] net/octeontx2: add tm packet marking cb X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Krzysztof Kanas Add tm packet marking enable/disable callbacks for VLAN DEI, IP DSCP and IP ECN marking to support marking in VLAN, IPv4 and IPv6 packets. Signed-off-by: Krzysztof Kanas Signed-off-by: Nithin Dabilpuram --- Depends-on:series-9450 drivers/net/octeontx2/otx2_ethdev.c | 12 +- drivers/net/octeontx2/otx2_ethdev.h | 5 + drivers/net/octeontx2/otx2_tm.c | 472 +++++++++++++++++++++++++++++++++++- drivers/net/octeontx2/otx2_tm.h | 31 +++ drivers/net/octeontx2/otx2_tx.h | 9 +- 5 files changed, 514 insertions(+), 15 deletions(-) diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index 3116e5c..1c7c465 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -655,8 +655,8 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) return flags; } -static uint16_t -nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +uint16_t +otx2_nix_tx_offload_flags(struct rte_eth_dev *eth_dev) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); uint64_t conf = dev->tx_offloads; @@ -974,6 +974,10 @@ otx2_nix_form_default_desc(struct otx2_eth_txq *txq) send_mem->alg = NIX_SENDMEMALG_SETTSTMP; send_mem->addr = txq->dev->tstamp.tx_tstamp_iova; } + + /* Update mark format */ + txq->markfmt_en = txq->dev->tm_markfmt_en; + sg = (union nix_send_sg_s *)&txq->cmd[4]; } else { send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; @@ -1663,7 +1667,7 @@ otx2_nix_configure(struct rte_eth_dev *eth_dev) dev->rx_offloads = rxmode->offloads; dev->tx_offloads = txmode->offloads; dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); - dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + dev->tx_offload_flags |= otx2_nix_tx_offload_flags(eth_dev); dev->rss_info.rss_grps = NIX_RSS_GRPS; nb_rxq = RTE_MAX(data->nb_rx_queues, 1); @@ -1829,7 +1833,7 @@ otx2_nix_configure(struct rte_eth_dev *eth_dev) nix_lf_free(dev); fail_offloads: dev->rx_offload_flags &= ~nix_rx_offload_flags(eth_dev); - dev->tx_offload_flags &= ~nix_tx_offload_flags(eth_dev); + dev->tx_offload_flags &= ~otx2_nix_tx_offload_flags(eth_dev); fail_configure: dev->configured = 0; return rc; diff --git a/drivers/net/octeontx2/otx2_ethdev.h b/drivers/net/octeontx2/otx2_ethdev.h index 0fbf68b..b34842a 100644 --- a/drivers/net/octeontx2/otx2_ethdev.h +++ b/drivers/net/octeontx2/otx2_ethdev.h @@ -304,6 +304,9 @@ struct otx2_eth_dev { uint16_t txschq_list[NIX_TXSCH_LVL_CNT][MAX_TXSCHQ_PER_FUNC]; /* Contiguous queues */ uint16_t txschq_contig_list[NIX_TXSCH_LVL_CNT][MAX_TXSCHQ_PER_FUNC]; + uint64_t tm_markfmt_en; + uint8_t tm_markfmt_null; + uint8_t tm_markfmt[NIX_TM_MARK_MAX][NIX_TM_MARK_COLOR_MAX]; uint16_t otx2_tm_root_lvl; uint16_t link_cfg_lvl; uint16_t tm_flags; @@ -347,6 +350,7 @@ struct otx2_eth_txq { rte_iova_t fc_iova; uint16_t sqes_per_sqb_log2; int16_t nb_sqb_bufs_adj; + uint64_t markfmt_en; RTE_MARKER slow_path_start; uint16_t nb_sqb_bufs; uint16_t sq; @@ -570,6 +574,7 @@ int otx2_ethdev_parse_devargs(struct rte_devargs *devargs, void otx2_eth_set_rx_function(struct rte_eth_dev *eth_dev); void otx2_eth_set_tx_function(struct rte_eth_dev *eth_dev); void otx2_nix_form_default_desc(struct otx2_eth_txq *txq); +uint16_t otx2_nix_tx_offload_flags(struct rte_eth_dev *eth_dev); /* Timesync - PTP routines */ int otx2_nix_timesync_enable(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/octeontx2/otx2_tm.c b/drivers/net/octeontx2/otx2_tm.c index e94a276..f11b828 100644 --- a/drivers/net/octeontx2/otx2_tm.c +++ b/drivers/net/octeontx2/otx2_tm.c @@ -20,6 +20,30 @@ enum otx2_tm_node_level { OTX2_TM_LVL_MAX, }; +static const uint8_t y_mask_val[NIX_TM_MARK_MAX][2] = { + [NIX_TM_MARK_VLAN_DEI] = { 0x0, 0x8 }, + [NIX_TM_MARK_IPV4_DSCP] = { 0x1, 0x2 }, + [NIX_TM_MARK_IPV4_ECN] = { 0x0, 0xc }, + [NIX_TM_MARK_IPV6_DSCP] = { 0x1, 0x2 }, + [NIX_TM_MARK_IPV6_ECN] = { 0x0, 0x3 }, +}; + +static const uint8_t r_mask_val[NIX_TM_MARK_MAX][2] = { + [NIX_TM_MARK_VLAN_DEI] = { 0x0, 0x8 }, + [NIX_TM_MARK_IPV4_DSCP] = { 0x0, 0x3 }, + [NIX_TM_MARK_IPV4_ECN] = { 0x0, 0xc }, + [NIX_TM_MARK_IPV6_DSCP] = { 0x0, 0x3 }, + [NIX_TM_MARK_IPV6_ECN] = { 0x0, 0x3 }, +}; + +static const uint8_t mark_off[NIX_TM_MARK_MAX] = { + [NIX_TM_MARK_VLAN_DEI] = 0x3, /* Byte 14 Bit[4:1] */ + [NIX_TM_MARK_IPV4_DSCP] = 0x1, /* Byte 1 Bit[6:3] */ + [NIX_TM_MARK_IPV4_ECN] = 0x6, /* Byte 1 Bit[1:0], Byte 2 Bit[7:6] */ + [NIX_TM_MARK_IPV6_DSCP] = 0x5, /* Byte 0 Bit[2:0], Byte 1 Bit[7] */ + [NIX_TM_MARK_IPV6_ECN] = 0x0, /* Byte 1 Bit[7:4] */ +}; + static inline uint64_t shaper2regval(struct shaper_params *shaper) { @@ -375,10 +399,10 @@ prepare_tm_shaper_reg(struct otx2_nix_tm_node *tm_node, otx2_tm_dbg("Shaper config node %s(%u) lvl %u id %u, pir %" PRIu64 "(%" PRIu64 "B), cir %" PRIu64 "(%" PRIu64 "B)" - "adjust 0x%" PRIx64 "(pktmode %u) (%p)", + "adjust 0x%" PRIx64 "(pktmode %u) red_alg %x (%p)", nix_hwlvl2str(tm_node->hw_lvl), schq, tm_node->lvl, tm_node->id, pir.rate, pir.burst, cir.rate, cir.burst, - adjust, tm_node->pkt_mode, tm_node); + adjust, tm_node->pkt_mode, tm_node->red_algo, tm_node); switch (tm_node->hw_lvl) { case NIX_TXSCH_LVL_SMQ: @@ -478,6 +502,46 @@ prepare_tm_shaper_reg(struct otx2_nix_tm_node *tm_node, } static uint8_t +prepare_tm_shaper_red_algo(struct otx2_nix_tm_node *tm_node, + volatile uint64_t *reg, volatile uint64_t *regval, + volatile uint64_t *regval_mask) +{ + uint32_t schq = tm_node->hw_id; + uint8_t k = 0; + + otx2_tm_dbg("Shaper read alg node %s(%u) lvl %u id %u, red_alg %x (%p)", + nix_hwlvl2str(tm_node->hw_lvl), schq, tm_node->lvl, + tm_node->id, tm_node->red_algo, tm_node); + + /* Configure just RED algo */ + regval[k] = ((uint64_t)tm_node->red_algo << 9); + regval_mask[k] = ~(BIT_ULL(10) | BIT_ULL(9)); + + switch (tm_node->hw_lvl) { + case NIX_TXSCH_LVL_SMQ: + reg[k] = NIX_AF_MDQX_SHAPE(schq); + k++; + break; + case NIX_TXSCH_LVL_TL4: + reg[k] = NIX_AF_TL4X_SHAPE(schq); + k++; + break; + case NIX_TXSCH_LVL_TL3: + reg[k] = NIX_AF_TL3X_SHAPE(schq); + k++; + break; + case NIX_TXSCH_LVL_TL2: + reg[k] = NIX_AF_TL2X_SHAPE(schq); + k++; + break; + default: + break; + } + + return k; +} + +static uint8_t prepare_tm_sw_xoff(struct otx2_nix_tm_node *tm_node, bool enable, volatile uint64_t *reg, volatile uint64_t *regval) { @@ -1939,10 +2003,16 @@ otx2_nix_tm_capa_get(struct rte_eth_dev *eth_dev, RTE_TM_STATS_N_PKTS_RED_DROPPED | RTE_TM_STATS_N_BYTES_RED_DROPPED; - for (i = 0; i < RTE_COLORS; i++) { - cap->mark_vlan_dei_supported[i] = false; - cap->mark_ip_ecn_tcp_supported[i] = false; - cap->mark_ip_dscp_supported[i] = false; + cap->mark_vlan_dei_supported[RTE_COLOR_GREEN] = false; + cap->mark_ip_ecn_tcp_supported[RTE_COLOR_GREEN] = false; + cap->mark_ip_ecn_sctp_supported[RTE_COLOR_GREEN] = false; + cap->mark_ip_dscp_supported[RTE_COLOR_GREEN] = false; + + for (i = RTE_COLOR_YELLOW; i < RTE_COLORS; i++) { + cap->mark_vlan_dei_supported[i] = true; + cap->mark_ip_ecn_tcp_supported[i] = true; + cap->mark_ip_ecn_sctp_supported[i] = true; + cap->mark_ip_dscp_supported[i] = true; } return 0; @@ -2831,6 +2901,317 @@ otx2_nix_tm_node_stats_read(struct rte_eth_dev *eth_dev, uint32_t node_id, return rc; } +static void +nix_tm_mark_queue_reconfig(struct rte_eth_dev *eth_dev, bool enable) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + struct otx2_eth_txq *txq; + int i, nb_txq; + + nb_txq = RTE_MIN(dev->configured_nb_tx_qs, eth_dev->data->nb_tx_queues); + + if (enable) { + dev->tx_offload_flags |= NIX_TX_OFFLOAD_MARK_F; + /* Enable L3 and OL3 CKSUM for marking IP DSCP and ECN */ + if (dev->tm_flags & NIX_TM_MARK_IP_ECN_EN || + dev->tm_flags & NIX_TM_MARK_IP_DSCP_EN) + dev->tx_offload_flags |= + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | + NIX_TX_OFFLOAD_OL3_OL4_CSUM_F); + otx2_tm_dbg("Enabling TM marking with tx_offload_flags=0x%04x" + " markfmt_en=0x%" PRIx64, dev->tx_offload_flags, + dev->tm_markfmt_en); + } else { + /* Reset to old flags excluding MARK */ + dev->tx_offload_flags = otx2_nix_tx_offload_flags(eth_dev); + otx2_tm_dbg("Disabling TM marking with tx_offload_flags=0x%04x" + " markfmt_en=0x%" PRIx64, dev->tx_offload_flags, + dev->tm_markfmt_en); + } + + for (i = 0; i < nb_txq; i++) { + txq = eth_dev->data->tx_queues[i]; + otx2_nix_form_default_desc(txq); + } + rte_wmb(); +} + +/* Only called while device is stopped */ +static int +nix_tm_update_red_algo(struct otx2_eth_dev *dev, bool red_send, + struct rte_tm_error *error) +{ + struct otx2_mbox *mbox = dev->mbox; + struct otx2_nix_tm_node *tm_node; + struct nix_txschq_config *req; + uint8_t k; + int rc; + + TAILQ_FOREACH(tm_node, &dev->node_list, node) { + /* Skip leaf nodes */ + if (nix_tm_is_leaf(dev, tm_node->lvl)) + continue; + + if (tm_node->hw_lvl == NIX_TXSCH_LVL_TL1) + continue; + + /* Skip if no update of red_algo is needed */ + if ((red_send && (tm_node->red_algo == NIX_REDALG_SEND)) || + (!red_send && (tm_node->red_algo != NIX_REDALG_SEND))) + continue; + + /* Update Red algo */ + if (red_send) + tm_node->red_algo = NIX_REDALG_SEND; + else if (tm_node->flags & NIX_TM_NODE_RED_DISCARD) + tm_node->red_algo = NIX_REDALG_DISCARD; + else + tm_node->red_algo = NIX_REDALG_STD; + + /* Update txschq config */ + req = otx2_mbox_alloc_msg_nix_txschq_cfg(mbox); + req->lvl = tm_node->hw_lvl; + k = prepare_tm_shaper_red_algo(tm_node, req->reg, + req->regval, req->regval_mask); + req->num_regs = k; + + rc = send_tm_reqval(mbox, req, error); + if (rc) + return rc; + } + return 0; +} + +/* Return's true if queue reconfig is needed */ +static bool +nix_tm_update_markfmt(struct otx2_eth_dev *dev, + enum otx2_nix_tm_mark type, + int mark_yellow, int mark_red) +{ + uint64_t new_markfmt, old_markfmt; + uint8_t *tm_markfmt; + uint8_t en_shift; + uint64_t mask; + + if (type >= NIX_TM_MARK_MAX) + return 0; + + /* Pre-allocated mark formats for type:color combinations */ + tm_markfmt = dev->tm_markfmt[type]; + + if (!mark_yellow && !mark_red) { + /* Null format to disable */ + new_markfmt = dev->tm_markfmt_null; + } else { + /* Marking enabled with combination of yellow and red */ + if (mark_yellow && mark_red) + new_markfmt = tm_markfmt[NIX_TM_MARK_COLOR_Y_R]; + else if (mark_yellow) + new_markfmt = tm_markfmt[NIX_TM_MARK_COLOR_Y]; + else + new_markfmt = tm_markfmt[NIX_TM_MARK_COLOR_R]; + } + + mask = 0xFFull; + /* Format of fast path markfmt + * ipv6_ecn[8]:ipv4_ecn[8]:ipv6_dscp[8]:ipv4_dscp[8]:vlan_dei[16] + * fmt[7] = ptr offset for IPv4/IPv6 on l2_len. + * fmt[6:0] = markfmt idx. + */ + switch (type) { + case NIX_TM_MARK_VLAN_DEI: + en_shift = NIX_TM_MARK_VLAN_DEI_SHIFT; + mask = 0xFFFFull; + new_markfmt |= new_markfmt << 8; + break; + case NIX_TM_MARK_IPV4_DSCP: + new_markfmt |= BIT_ULL(7); + en_shift = NIX_TM_MARK_IPV4_DSCP_SHIFT; + break; + case NIX_TM_MARK_IPV4_ECN: + new_markfmt |= BIT_ULL(7); + en_shift = NIX_TM_MARK_IPV4_ECN_SHIFT; + break; + case NIX_TM_MARK_IPV6_DSCP: + en_shift = NIX_TM_MARK_IPV6_DSCP_SHIFT; + break; + case NIX_TM_MARK_IPV6_ECN: + new_markfmt |= BIT_ULL(7); + en_shift = NIX_TM_MARK_IPV6_ECN_SHIFT; + break; + default: + return 0; + } + + /* Skip if same as old config */ + old_markfmt = (dev->tm_markfmt_en >> en_shift) & mask; + if (old_markfmt == new_markfmt) + return false; + + /* Need queue reconfig */ + dev->tm_markfmt_en &= ~(mask << en_shift); + dev->tm_markfmt_en |= (new_markfmt << en_shift); + return true; +} + +static int +otx2_nix_tm_mark_vlan_dei(struct rte_eth_dev *eth_dev, int mark_green, + int mark_yellow, int mark_red, + struct rte_tm_error *error) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + bool need_reconfig, marking; + int rc; + + if (mark_green) { + error->type = RTE_TM_ERROR_TYPE_CAPABILITIES; + error->message = "Green VLAN marking not supported"; + return -EINVAL; + } + + if (eth_dev->data->dev_started) { + error->type = RTE_TM_ERROR_TYPE_CAPABILITIES; + error->message = "VLAN DEI mark for running ports not " + "supported"; + return -EBUSY; + } + + if (!(dev->tm_flags & NIX_TM_COMMITTED)) { + error->type = RTE_TM_ERROR_TYPE_UNSPECIFIED; + error->message = "hierarchy doesn't exist"; + return -EINVAL; + } + + otx2_tm_dbg("VLAN DEI marking update yellow=%u red=%u", + mark_yellow, mark_red); + + need_reconfig = nix_tm_update_markfmt(dev, NIX_TM_MARK_VLAN_DEI, + mark_yellow, mark_red); + if (!need_reconfig) + return 0; + + if (!mark_yellow && !mark_red) + dev->tm_flags &= ~NIX_TM_MARK_VLAN_DEI_EN; + else + dev->tm_flags |= NIX_TM_MARK_VLAN_DEI_EN; + + /* Update red algo for change in mark_red */ + rc = nix_tm_update_red_algo(dev, !!mark_red, error); + if (rc) + return rc; + + /* Update red algo */ + marking = !!(dev->tm_flags & NIX_TM_MARK_EN_MASK); + nix_tm_mark_queue_reconfig(eth_dev, marking); + return 0; +} + +static int +otx2_nix_tm_mark_ip_ecn(struct rte_eth_dev *eth_dev, int mark_green, + int mark_yellow, int mark_red, + struct rte_tm_error *error) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + bool need_reconfig, marking; + int rc; + + if (mark_green) { + error->type = RTE_TM_ERROR_TYPE_CAPABILITIES; + error->message = "Green IP ECN marking not supported"; + return -EINVAL; + } + + if (eth_dev->data->dev_started) { + error->type = RTE_TM_ERROR_TYPE_CAPABILITIES; + error->message = "IP ECN mark for running ports not " + "supported"; + return -EBUSY; + } + + if (!(dev->tm_flags & NIX_TM_COMMITTED)) { + error->type = RTE_TM_ERROR_TYPE_UNSPECIFIED; + error->message = "hierarchy doesn't exist"; + return -EINVAL; + } + + otx2_tm_dbg("IP ECN marking update yellow=%u red=%u", + mark_yellow, mark_red); + + need_reconfig = nix_tm_update_markfmt(dev, NIX_TM_MARK_IPV4_ECN, + mark_yellow, mark_red); + need_reconfig |= nix_tm_update_markfmt(dev, NIX_TM_MARK_IPV6_ECN, + mark_yellow, mark_red); + if (!need_reconfig) + return 0; + + if (!mark_yellow && !mark_red) + dev->tm_flags &= ~NIX_TM_MARK_IP_ECN_EN; + else + dev->tm_flags |= NIX_TM_MARK_IP_ECN_EN; + + /* Update red algo for change in mark_red */ + rc = nix_tm_update_red_algo(dev, !!mark_red, error); + if (rc) + return rc; + + marking = !!(dev->tm_flags & NIX_TM_MARK_EN_MASK); + nix_tm_mark_queue_reconfig(eth_dev, marking); + return 0; +} + +static int +otx2_nix_tm_mark_ip_dscp(struct rte_eth_dev *eth_dev, int mark_green, + int mark_yellow, int mark_red, + struct rte_tm_error *error) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + bool need_reconfig, marking; + int rc; + + if (mark_green) { + error->type = RTE_TM_ERROR_TYPE_CAPABILITIES; + error->message = "Green IP DSCP marking not supported"; + return -EINVAL; + } + + if (eth_dev->data->dev_started) { + error->type = RTE_TM_ERROR_TYPE_CAPABILITIES; + error->message = "IP DSCP mark for running ports not " + "supported"; + return -EBUSY; + } + + if (!(dev->tm_flags & NIX_TM_COMMITTED)) { + error->type = RTE_TM_ERROR_TYPE_UNSPECIFIED; + error->message = "hierarchy doesn't exist"; + return -EINVAL; + } + + otx2_tm_dbg("IP DSCP marking update yellow=%u red=%u", + mark_yellow, mark_red); + + need_reconfig = nix_tm_update_markfmt(dev, NIX_TM_MARK_IPV4_DSCP, + mark_yellow, mark_red); + need_reconfig |= nix_tm_update_markfmt(dev, NIX_TM_MARK_IPV6_DSCP, + mark_yellow, mark_red); + if (!need_reconfig) + return 0; + + if (!mark_yellow && !mark_red) + dev->tm_flags &= ~NIX_TM_MARK_IP_DSCP_EN; + else + dev->tm_flags |= NIX_TM_MARK_IP_DSCP_EN; + + /* Update red algo for change in mark_red */ + rc = nix_tm_update_red_algo(dev, !!mark_red, error); + if (rc) + return rc; + + marking = !!(dev->tm_flags & NIX_TM_MARK_EN_MASK); + nix_tm_mark_queue_reconfig(eth_dev, marking); + return 0; +} + const struct rte_tm_ops otx2_tm_ops = { .node_type_get = otx2_nix_tm_node_type_get, @@ -2850,9 +3231,80 @@ const struct rte_tm_ops otx2_tm_ops = { .node_shaper_update = otx2_nix_tm_node_shaper_update, .node_parent_update = otx2_nix_tm_node_parent_update, .node_stats_read = otx2_nix_tm_node_stats_read, + + .mark_vlan_dei = otx2_nix_tm_mark_vlan_dei, + .mark_ip_ecn = otx2_nix_tm_mark_ip_ecn, + .mark_ip_dscp = otx2_nix_tm_mark_ip_dscp, }; static int +nix_tm_mark_init(struct otx2_eth_dev *dev) +{ + struct nix_mark_format_cfg_rsp *rsp; + struct otx2_mbox *mbox = dev->mbox; + struct nix_mark_format_cfg *req; + int rc, i, j; + + /* Check for supported revisions */ + if (otx2_dev_is_95xx_Ax(dev) || + otx2_dev_is_96xx_Ax(dev)) + return 0; + + /* Null mark format */ + req = otx2_mbox_alloc_msg_nix_mark_format_cfg(mbox); + rc = otx2_mbox_process_msg(mbox, (void *)&rsp); + if (rc) { + otx2_err("TM failed to alloc null mark format, rc=%d", rc); + return rc; + } + + dev->tm_markfmt_null = rsp->mark_format_idx; + + /* Alloc vlan, dscp, ecn mark formats */ + for (i = 0; i < NIX_TM_MARK_MAX; i++) { + for (j = 0; j < NIX_TM_MARK_COLOR_MAX; j++) { + req = otx2_mbox_alloc_msg_nix_mark_format_cfg(mbox); + req->offset = mark_off[i]; + + switch (j) { + case NIX_TM_MARK_COLOR_Y: + req->y_mask = y_mask_val[i][0]; + req->y_val = y_mask_val[i][1]; + break; + case NIX_TM_MARK_COLOR_R: + req->r_mask = r_mask_val[i][0]; + req->r_val = r_mask_val[i][1]; + break; + case NIX_TM_MARK_COLOR_Y_R: + req->y_mask = y_mask_val[i][0]; + req->y_val = y_mask_val[i][1]; + req->r_mask = r_mask_val[i][0]; + req->r_val = r_mask_val[i][1]; + break; + } + + rc = otx2_mbox_process_msg(mbox, (void *)&rsp); + if (rc) { + otx2_err("TM failed to alloc mark fmt " + "type %u color %u, rc=%d", i, j, rc); + return rc; + } + + dev->tm_markfmt[i][j] = rsp->mark_format_idx; + otx2_tm_dbg("Mark type: %u, Mark Color:%u, id:%u\n", + i, j, dev->tm_markfmt[i][j]); + } + } + /* Update null mark format as default */ + nix_tm_update_markfmt(dev, NIX_TM_MARK_VLAN_DEI, 0, 0); + nix_tm_update_markfmt(dev, NIX_TM_MARK_IPV4_DSCP, 0, 0); + nix_tm_update_markfmt(dev, NIX_TM_MARK_IPV4_ECN, 0, 0); + nix_tm_update_markfmt(dev, NIX_TM_MARK_IPV6_DSCP, 0, 0); + nix_tm_update_markfmt(dev, NIX_TM_MARK_IPV6_ECN, 0, 0); + return 0; +} + +static int nix_tm_prepare_default_tree(struct rte_eth_dev *eth_dev) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); @@ -2963,7 +3415,7 @@ void otx2_nix_tm_conf_init(struct rte_eth_dev *eth_dev) int otx2_nix_tm_init_default(struct rte_eth_dev *eth_dev) { struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); - struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); uint16_t sq_cnt = eth_dev->data->nb_tx_queues; int rc; @@ -2985,6 +3437,10 @@ int otx2_nix_tm_init_default(struct rte_eth_dev *eth_dev) if (pci_dev->max_vfs) dev->tm_flags |= NIX_TM_TL1_NO_SP; + rc = nix_tm_mark_init(dev); + if (rc != 0) + return rc; + rc = nix_tm_prepare_default_tree(eth_dev); if (rc != 0) return rc; @@ -2992,8 +3448,8 @@ int otx2_nix_tm_init_default(struct rte_eth_dev *eth_dev) rc = nix_tm_alloc_resources(eth_dev, false); if (rc != 0) return rc; + dev->tm_leaf_cnt = sq_cnt; - return 0; } diff --git a/drivers/net/octeontx2/otx2_tm.h b/drivers/net/octeontx2/otx2_tm.h index db44d48..afc796a 100644 --- a/drivers/net/octeontx2/otx2_tm.h +++ b/drivers/net/octeontx2/otx2_tm.h @@ -13,6 +13,37 @@ #define NIX_TM_COMMITTED BIT_ULL(1) #define NIX_TM_RATE_LIMIT_TREE BIT_ULL(2) #define NIX_TM_TL1_NO_SP BIT_ULL(3) +#define NIX_TM_MARK_IP_DSCP_EN BIT_ULL(4) +#define NIX_TM_MARK_IP_ECN_EN BIT_ULL(5) +#define NIX_TM_MARK_VLAN_DEI_EN BIT_ULL(6) + + +#define NIX_TM_MARK_EN_MASK (NIX_TM_MARK_IP_DSCP_EN | \ + NIX_TM_MARK_IP_ECN_EN | \ + NIX_TM_MARK_VLAN_DEI_EN) + +/* Shift positions for struct otx2_eth_dev::tm_markfmt_en field */ +#define NIX_TM_MARK_VLAN_DEI_SHIFT 0 /* Leave 16b for VLAN for FP logic */ +#define NIX_TM_MARK_IPV4_DSCP_SHIFT 16 +#define NIX_TM_MARK_IPV6_DSCP_SHIFT 24 +#define NIX_TM_MARK_IPV4_ECN_SHIFT 32 +#define NIX_TM_MARK_IPV6_ECN_SHIFT 40 + +enum otx2_nix_tm_mark { + NIX_TM_MARK_VLAN_DEI, + NIX_TM_MARK_IPV4_DSCP, + NIX_TM_MARK_IPV4_ECN, + NIX_TM_MARK_IPV6_DSCP, + NIX_TM_MARK_IPV6_ECN, + NIX_TM_MARK_MAX +}; + +enum otx2_nix_tm_mark_color { + NIX_TM_MARK_COLOR_Y, + NIX_TM_MARK_COLOR_R, + NIX_TM_MARK_COLOR_Y_R, + NIX_TM_MARK_COLOR_MAX +}; struct otx2_eth_dev; diff --git a/drivers/net/octeontx2/otx2_tx.h b/drivers/net/octeontx2/otx2_tx.h index 3c43170..928e4ea 100644 --- a/drivers/net/octeontx2/otx2_tx.h +++ b/drivers/net/octeontx2/otx2_tx.h @@ -13,6 +13,7 @@ #define NIX_TX_OFFLOAD_TSTAMP_F BIT(4) #define NIX_TX_OFFLOAD_TSO_F BIT(5) #define NIX_TX_OFFLOAD_SECURITY_F BIT(6) +#define NIX_TX_OFFLOAD_MARK_F BIT(7) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -22,11 +23,12 @@ #define NIX_TX_NEED_SEND_HDR_W1 \ (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ - NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F | \ + NIX_TX_OFFLOAD_MARK_F) #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | \ - NIX_TX_OFFLOAD_TSO_F) + NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_MARK_F) #define NIX_UDP_TUN_BITMASK \ ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ @@ -42,7 +44,8 @@ static __rte_always_inline int otx2_nix_tx_ext_subs(const uint16_t flags) { return (flags & NIX_TX_OFFLOAD_TSTAMP_F) ? 2 : - ((flags & (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? + ((flags & (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F | + NIX_TX_OFFLOAD_MARK_F)) ? 1 : 0); } From patchwork Fri Apr 17 07:22:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 68735 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 11532A058A; Fri, 17 Apr 2020 09:23:48 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2D6F11DD41; Fri, 17 Apr 2020 09:23:36 +0200 (CEST) Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by dpdk.org (Postfix) with ESMTP id D9BEA1DCAE for ; Fri, 17 Apr 2020 09:23:33 +0200 (CEST) Received: by mail-pf1-f181.google.com with SMTP id d1so677959pfh.1 for ; Fri, 17 Apr 2020 00:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XIf8Zz3Sm9uiCJyoDaKv81Q5Vx+RwjMwoL8/4dIT9Cw=; b=NijmYWhIFYvJT6Bw0svcbz/CXOJx8X1X8NzoOt9dzG8aMEvvCC9KYgVZbBswVh7pXc jZE93a1JWG9jylYNkM+6NttTO5UxDcyxcMRaM3yMSAw11m5YRBCWA9lHjxtnc7u8VR8o Lz5Ep689H4/cUnp4uT9epIFT0Tc6UdocIy2Sdz7IS0rI577bNdTDismVA9B2NyDly1ta W+MxnpIarP5RZVdsvCpH5az6mtZ8YlomS/g4l0RWE/a8zVROmAFOVgTfbuyNEi2hAhJo ZHOmHRBjkprMbKpUe6h/xxwZh8ksgBHoOJAFtO7TAXSfCerqTr6A5e6aDTm+9hEAcIQ7 M97Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XIf8Zz3Sm9uiCJyoDaKv81Q5Vx+RwjMwoL8/4dIT9Cw=; b=ukBoFViDQqTOc2Vv9BpewZ8a9+mdBrsz5+dtvzZGUp1oyHgGkHC66kppomsZYy/evE TyLSvhTwAHUvkrOMVFNpDdIQDBk9wm9pU2fd0I5CiJV8TCjkQarRcKetzHL/JZoX2dW8 W4TsB96ESVs7PTgTQSDJ9OQNvc+lca3QASzYQ4cxAGy03+ISMnazIRShpQTenr/pWjAm WvR4xJBLqZELez+Jn2gu+R5l8TacU/x22OVg0VJYQ6MS0lyJDV7ontDZetkohshXC/ha aMw0+ZXdapPVqT8+LDRw6IMoY1EHh3u6uENOHsdYtowt+G2txQUzQu/eoU9YlXtkR3Ch e+zA== X-Gm-Message-State: AGi0PubCDmOsPjctEd1CF2XPL7DgHCynhbjkCEny0oEUsesSwe+yIHNJ VMxWG3o0WYtMhI+8B8KeonU= X-Google-Smtp-Source: APiQypIBL1tv5NZP5dRSvg64YIvcqvKLZh6Apq9rwgiweZIQHH9FaKpUyjr6tMoRxRuu2kEdH3tu4g== X-Received: by 2002:aa7:8f29:: with SMTP id y9mr1968947pfr.19.1587108212297; Fri, 17 Apr 2020 00:23:32 -0700 (PDT) Received: from hyd1588t430.marvell.com ([115.113.156.2]) by smtp.gmail.com with ESMTPSA id v4sm18750983pfb.31.2020.04.17.00.23.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Apr 2020 00:23:31 -0700 (PDT) From: Nithin Dabilpuram To: thomas@monjalon.net, ferruh.yigit@intel.com, arybchenko@solarflare.com, orika@mellanox.com, cristian.dumitrescu@intel.com, anatoly.burakov@intel.com, Pavan Nikhilesh , Jerin Jacob , Nithin Dabilpuram , Kiran Kumar K Cc: dev@dpdk.org, kkanas@marvell.com Date: Fri, 17 Apr 2020 12:52:53 +0530 Message-Id: <20200417072254.11455-3-nithind1988@gmail.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20200417072254.11455-1-nithind1988@gmail.com> References: <20200417072254.11455-1-nithind1988@gmail.com> Subject: [dpdk-dev] [PATCH 3/3] net/octeontx2: add Tx packet marking offload support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Nithin Dabilpuram This patch adds Tx packet marking offload support for VLAN DEI, IP ECN, and IP DSCP. Signed-off-by: Krzysztof Kanas Signed-off-by: Nithin Dabilpuram --- Depends-on:series-9450 drivers/event/octeontx2/otx2_evdev.c | 28 +- drivers/event/octeontx2/otx2_evdev.h | 2 +- drivers/event/octeontx2/otx2_worker.c | 4 +- drivers/event/octeontx2/otx2_worker.h | 4 +- drivers/event/octeontx2/otx2_worker_dual.c | 4 +- drivers/net/octeontx2/otx2_tx.c | 48 ++- drivers/net/octeontx2/otx2_tx.h | 580 ++++++++++++++++++++++------- 7 files changed, 503 insertions(+), 167 deletions(-) diff --git a/drivers/event/octeontx2/otx2_evdev.c b/drivers/event/octeontx2/otx2_evdev.c index d20213d..33cba09 100644 --- a/drivers/event/octeontx2/otx2_evdev.c +++ b/drivers/event/octeontx2/otx2_evdev.c @@ -178,36 +178,36 @@ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC /* Tx modes */ const event_tx_adapter_enqueue - ssogws_tx_adptr_enq[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = \ + ssogws_tx_adptr_enq[2][2][2][2][2][2][2][2] = { +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_tx_adptr_enq_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T }; const event_tx_adapter_enqueue - ssogws_tx_adptr_enq_seg[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = \ + ssogws_tx_adptr_enq_seg[2][2][2][2][2][2][2][2] = { +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_tx_adptr_enq_seg_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T }; const event_tx_adapter_enqueue - ssogws_dual_tx_adptr_enq[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = \ + ssogws_dual_tx_adptr_enq[2][2][2][2][2][2][2][2] = { +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_dual_tx_adptr_enq_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T }; const event_tx_adapter_enqueue - ssogws_dual_tx_adptr_enq_seg[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = \ + ssogws_dual_tx_adptr_enq_seg[2][2][2][2][2][2][2][2] = { +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_dual_tx_adptr_enq_seg_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T @@ -294,6 +294,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC if (dev->tx_offloads & NIX_TX_MULTI_SEG_F) { /* [SEC] [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ event_dev->txa_enqueue = ssogws_tx_adptr_enq_seg + [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MARK_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] @@ -303,6 +304,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->tx_offloads & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; } else { event_dev->txa_enqueue = ssogws_tx_adptr_enq + [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MARK_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] @@ -446,6 +448,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC if (dev->tx_offloads & NIX_TX_MULTI_SEG_F) { /* [SEC] [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ event_dev->txa_enqueue = ssogws_dual_tx_adptr_enq_seg + [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MARK_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] @@ -460,6 +463,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; } else { event_dev->txa_enqueue = ssogws_dual_tx_adptr_enq + [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MARK_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] diff --git a/drivers/event/octeontx2/otx2_evdev.h b/drivers/event/octeontx2/otx2_evdev.h index 3b47782..73eee8f 100644 --- a/drivers/event/octeontx2/otx2_evdev.h +++ b/drivers/event/octeontx2/otx2_evdev.h @@ -335,7 +335,7 @@ uint16_t otx2_ssogws_dual_deq_seg_timeout_burst_ ##name(void *port, \ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t otx2_ssogws_tx_adptr_enq_ ## name(void *port, struct rte_event ev[],\ uint16_t nb_events); \ uint16_t otx2_ssogws_tx_adptr_enq_seg_ ## name(void *port, \ diff --git a/drivers/event/octeontx2/otx2_worker.c b/drivers/event/octeontx2/otx2_worker.c index 8bec59e..cf73fd0 100644 --- a/drivers/event/octeontx2/otx2_worker.c +++ b/drivers/event/octeontx2/otx2_worker.c @@ -267,7 +267,7 @@ otx2_ssogws_enq_fwd_burst(void *port, const struct rte_event ev[], return 1; } -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_tx_adptr_enq_ ## name(void *port, struct rte_event ev[], \ uint16_t nb_events) \ @@ -281,7 +281,7 @@ otx2_ssogws_tx_adptr_enq_ ## name(void *port, struct rte_event ev[], \ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_tx_adptr_enq_seg_ ## name(void *port, struct rte_event ev[],\ uint16_t nb_events) \ diff --git a/drivers/event/octeontx2/otx2_worker.h b/drivers/event/octeontx2/otx2_worker.h index 5f5aa87..5a59493 100644 --- a/drivers/event/octeontx2/otx2_worker.h +++ b/drivers/event/octeontx2/otx2_worker.h @@ -270,8 +270,10 @@ static __rte_always_inline void otx2_ssogws_prepare_pkt(const struct otx2_eth_txq *txq, struct rte_mbuf *m, uint64_t *cmd, const uint32_t flags) { + const uint64_t markfmt_en = txq->markfmt_en; + otx2_lmt_mov(cmd, txq->cmd, otx2_nix_tx_ext_subs(flags)); - otx2_nix_xmit_prepare(m, cmd, flags); + otx2_nix_xmit_prepare(m, cmd, flags, markfmt_en); } static __rte_always_inline uint16_t diff --git a/drivers/event/octeontx2/otx2_worker_dual.c b/drivers/event/octeontx2/otx2_worker_dual.c index 3cba09c..5f3acc3 100644 --- a/drivers/event/octeontx2/otx2_worker_dual.c +++ b/drivers/event/octeontx2/otx2_worker_dual.c @@ -307,7 +307,7 @@ otx2_ssogws_dual_deq_seg_timeout_burst_ ##name(void *port, \ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_dual_tx_adptr_enq_ ## name(void *port, \ struct rte_event ev[], \ @@ -324,7 +324,7 @@ otx2_ssogws_dual_tx_adptr_enq_ ## name(void *port, \ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_dual_tx_adptr_enq_seg_ ## name(void *port, \ struct rte_event ev[], \ diff --git a/drivers/net/octeontx2/otx2_tx.c b/drivers/net/octeontx2/otx2_tx.c index 4f2036c..638dd9d 100644 --- a/drivers/net/octeontx2/otx2_tx.c +++ b/drivers/net/octeontx2/otx2_tx.c @@ -27,6 +27,7 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, struct otx2_eth_txq *txq = tx_queue; uint16_t i; const rte_iova_t io_addr = txq->io_addr; void *lmt_addr = txq->lmt_addr; + uint64_t markfmt_en = 0; NIX_XMIT_FC_OR_RETURN(txq, pkts); @@ -38,11 +39,14 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, otx2_nix_xmit_prepare_tso(tx_pkts[i], flags); } + if (flags & NIX_TX_OFFLOAD_MARK_F) + markfmt_en = txq->markfmt_en; + /* Lets commit any changes in the packet */ rte_cio_wmb(); for (i = 0; i < pkts; i++) { - otx2_nix_xmit_prepare(tx_pkts[i], cmd, flags); + otx2_nix_xmit_prepare(tx_pkts[i], cmd, flags, markfmt_en); /* Passing no of segdw as 4: HDR + EXT + SG + SMEM */ otx2_nix_xmit_prepare_tstamp(cmd, &txq->cmd[0], tx_pkts[i]->ol_flags, 4, flags); @@ -62,6 +66,7 @@ nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, struct otx2_eth_txq *txq = tx_queue; uint64_t i; const rte_iova_t io_addr = txq->io_addr; void *lmt_addr = txq->lmt_addr; + uint64_t markfmt_en = 0; uint16_t segdw; NIX_XMIT_FC_OR_RETURN(txq, pkts); @@ -74,11 +79,14 @@ nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, otx2_nix_xmit_prepare_tso(tx_pkts[i], flags); } + if (flags & NIX_TX_OFFLOAD_MARK_F) + markfmt_en = txq->markfmt_en; + /* Lets commit any changes in the packet */ rte_cio_wmb(); for (i = 0; i < pkts; i++) { - otx2_nix_xmit_prepare(tx_pkts[i], cmd, flags); + otx2_nix_xmit_prepare(tx_pkts[i], cmd, flags, markfmt_en); segdw = otx2_nix_prepare_mseg(tx_pkts[i], cmd, flags); otx2_nix_xmit_prepare_tstamp(cmd, &txq->cmd[0], tx_pkts[i]->ol_flags, segdw, @@ -946,7 +954,7 @@ nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, } #endif -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_xmit_pkts_ ## name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -963,7 +971,7 @@ otx2_nix_xmit_pkts_ ## name(void *tx_queue, \ NIX_TX_FASTPATH_MODES #undef T -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_xmit_pkts_mseg_ ## name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -981,17 +989,18 @@ otx2_nix_xmit_pkts_mseg_ ## name(void *tx_queue, \ NIX_TX_FASTPATH_MODES #undef T -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_xmit_pkts_vec_ ## name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ { \ uint64_t cmd[sz]; \ \ - /* VLAN, TSTMP, TSO is not supported by vec */ \ + /* VLAN, TSTMP, TSO, MARK is not supported by vec */ \ if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ (flags) & NIX_TX_OFFLOAD_TSTAMP_F || \ - (flags) & NIX_TX_OFFLOAD_TSO_F) \ + (flags) & NIX_TX_OFFLOAD_TSO_F || \ + (flags) & NIX_TX_OFFLOAD_MARK_F) \ return 0; \ return nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, (flags)); \ } @@ -1001,12 +1010,13 @@ NIX_TX_FASTPATH_MODES static inline void pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2][2][2]) + const eth_tx_burst_t tx_burst[2][2][2][2][2][2][2][2]) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - /* [SEC] [TSTMP] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + /* [MARK] [SEC] [TSTMP] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MARK_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] @@ -1021,25 +1031,25 @@ otx2_eth_set_tx_function(struct rte_eth_dev *eth_dev) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_ ## name, + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2][2][2] = { +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_ ## name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_mseg_ ## name, + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2][2][2] = { +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_mseg_ ## name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_vec_ ## name, + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2][2][2] = { +#define T(name, f7, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_vec_ ## name, NIX_TX_FASTPATH_MODES #undef T @@ -1048,7 +1058,7 @@ NIX_TX_FASTPATH_MODES if (dev->scalar_ena || (dev->tx_offload_flags & (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | - NIX_TX_OFFLOAD_TSO_F))) + NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_MARK_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); else pick_tx_func(eth_dev, nix_eth_tx_vec_burst); diff --git a/drivers/net/octeontx2/otx2_tx.h b/drivers/net/octeontx2/otx2_tx.h index 928e4ea..aba0607 100644 --- a/drivers/net/octeontx2/otx2_tx.h +++ b/drivers/net/octeontx2/otx2_tx.h @@ -200,7 +200,8 @@ otx2_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) } static __rte_always_inline void -otx2_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +otx2_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, + const uint16_t flags, const uint64_t markfmt_en) { struct nix_send_ext_s *send_hdr_ext; struct nix_send_hdr_s *send_hdr; @@ -353,6 +354,23 @@ otx2_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) } } + if (flags & NIX_TX_OFFLOAD_MARK_F) { + /* Pick the first among the following */ + uint8_t mark_offload = ffs((ol_flags & + (PKT_TX_MARK_VLAN_DEI | + PKT_TX_MARK_IP_ECN | + PKT_TX_MARK_IP_DSCP)) >> 38); + uint8_t ipv6 = !!(ol_flags & PKT_TX_IPV6); + uint16_t markfmt = (markfmt_en >> + ((mark_offload - 1) << 4)); + markfmt = (markfmt >> (ipv6 << 3)) & 0xFF; + send_hdr_ext->w0.mark_en = !!mark_offload; + send_hdr_ext->w0.markform = markfmt & 0x7F; + send_hdr_ext->w0.markptr = (mark_offload == 1) ? + m->l2_len - 4 : /* VLAN DEI */ + m->l2_len + (markfmt >> 7); /* IPv4v6 */ + } + if (flags & NIX_TX_NEED_SEND_HDR_W1) send_hdr->w1.u = w1.u; @@ -475,273 +493,575 @@ otx2_nix_xmit_mseg_one(uint64_t *cmd, void *lmt_addr, #define TSP_F NIX_TX_OFFLOAD_TSTAMP_F #define TSO_F NIX_TX_OFFLOAD_TSO_F #define TX_SEC_F NIX_TX_OFFLOAD_SECURITY_F +#define TX_MARK_F NIX_TX_OFFLOAD_MARK_F -/* [SEC] [TSO] [TSTMP] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +/* [MARK] [SEC] [TSO] [TSTMP] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ #define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 0, 0, 4, \ +T(no_offload, 0, 0, 0, 0, 0, 0, 0, 0, 4, \ NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 0, 0, 1, 4, \ +T(l3l4csum, 0, 0, 0, 0, 0, 0, 0, 1, 4, \ L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 0, 0, 1, 0, 4, \ +T(ol3ol4csum, 0, 0, 0, 0, 0, 0, 1, 0, 4, \ OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 0, 1, 1, 4, \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 0, 0, 1, 1, 4, \ OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 0, 0, 1, 0, 0, 6, \ +T(vlan, 0, 0, 0, 0, 0, 1, 0, 0, 6, \ VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 0, 0, 1, 0, 1, 6, \ +T(vlan_l3l4csum, 0, 0, 0, 0, 0, 1, 0, 1, 6, \ VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 0, 0, 1, 1, 0, 6, \ +T(vlan_ol3ol4csum, 0, 0, 0, 0, 0, 1, 1, 0, 6, \ VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 1, 6, \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 0, 0, 1, 1, 1, 6, \ VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 0, 0, 1, 0, 0, 0, 4, \ +T(noff, 0, 0, 0, 0, 1, 0, 0, 0, 4, \ NOFF_F) \ -T(noff_l3l4csum, 0, 0, 0, 1, 0, 0, 1, 4, \ +T(noff_l3l4csum, 0, 0, 0, 0, 1, 0, 0, 1, 4, \ NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 0, 0, 1, 0, 1, 0, 4, \ +T(noff_ol3ol4csum, 0, 0, 0, 0, 1, 0, 1, 0, 4, \ NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 0, 1, 1, 4, \ +T(noff_ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 0, 1, 1, 4, \ NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 0, 0, 1, 1, 0, 0, 6, \ +T(noff_vlan, 0, 0, 0, 0, 1, 1, 0, 0, 6, \ NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 0, 0, 1, 1, 0, 1, 6, \ +T(noff_vlan_l3l4csum, 0, 0, 0, 0, 1, 1, 0, 1, 6, \ NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 0, 0, 1, 1, 1, 0, 6, \ +T(noff_vlan_ol3ol4csum, 0, 0, 0, 0, 1, 1, 1, 0, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 1, 6, \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 1, 1, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts, 0, 0, 1, 0, 0, 0, 0, 8, \ +T(ts, 0, 0, 0, 1, 0, 0, 0, 0, 8, \ TSP_F) \ -T(ts_l3l4csum, 0, 0, 1, 0, 0, 0, 1, 8, \ +T(ts_l3l4csum, 0, 0, 0, 1, 0, 0, 0, 1, 8, \ TSP_F | L3L4CSUM_F) \ -T(ts_ol3ol4csum, 0, 0, 1, 0, 0, 1, 0, 8, \ +T(ts_ol3ol4csum, 0, 0, 0, 1, 0, 0, 1, 0, 8, \ TSP_F | OL3OL4CSUM_F) \ -T(ts_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 0, 1, 1, 8, \ +T(ts_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 0, 0, 1, 1, 8, \ TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_vlan, 0, 0, 1, 0, 1, 0, 0, 8, \ +T(ts_vlan, 0, 0, 0, 1, 0, 1, 0, 0, 8, \ TSP_F | VLAN_F) \ -T(ts_vlan_l3l4csum, 0, 0, 1, 0, 1, 0, 1, 8, \ +T(ts_vlan_l3l4csum, 0, 0, 0, 1, 0, 1, 0, 1, 8, \ TSP_F | VLAN_F | L3L4CSUM_F) \ -T(ts_vlan_ol3ol4csum, 0, 0, 1, 0, 1, 1, 0, 8, \ +T(ts_vlan_ol3ol4csum, 0, 0, 0, 1, 0, 1, 1, 0, 8, \ TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 1, 8, \ +T(ts_vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 0, 1, 1, 1, 8, \ TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_noff, 0, 0, 1, 1, 0, 0, 0, 8, \ +T(ts_noff, 0, 0, 0, 1, 1, 0, 0, 0, 8, \ TSP_F | NOFF_F) \ -T(ts_noff_l3l4csum, 0, 0, 1, 1, 0, 0, 1, 8, \ +T(ts_noff_l3l4csum, 0, 0, 0, 1, 1, 0, 0, 1, 8, \ TSP_F | NOFF_F | L3L4CSUM_F) \ -T(ts_noff_ol3ol4csum, 0, 0, 1, 1, 0, 1, 0, 8, \ +T(ts_noff_ol3ol4csum, 0, 0, 0, 1, 1, 0, 1, 0, 8, \ TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(ts_noff_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 0, 1, 1, 8, \ +T(ts_noff_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 0, 1, 1, 8, \ TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_noff_vlan, 0, 0, 1, 1, 1, 0, 0, 8, \ +T(ts_noff_vlan, 0, 0, 0, 1, 1, 1, 0, 0, 8, \ TSP_F | NOFF_F | VLAN_F) \ -T(ts_noff_vlan_l3l4csum, 0, 0, 1, 1, 1, 0, 1, 8, \ +T(ts_noff_vlan_l3l4csum, 0, 0, 0, 1, 1, 1, 0, 1, 8, \ TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(ts_noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 1, 0, 8, \ +T(ts_noff_vlan_ol3ol4csum, 0, 0, 0, 1, 1, 1, 1, 0, 8, \ TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 1, 8, \ +T(ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 1, 1, 8, \ TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ \ -T(tso, 0, 1, 0, 0, 0, 0, 0, 6, \ +T(tso, 0, 0, 1, 0, 0, 0, 0, 0, 6, \ TSO_F) \ -T(tso_l3l4csum, 0, 1, 0, 0, 0, 0, 1, 6, \ +T(tso_l3l4csum, 0, 0, 1, 0, 0, 0, 0, 1, 6, \ TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 0, 1, 0, 0, 0, 1, 0, 6, \ +T(tso_ol3ol4csum, 0, 0, 1, 0, 0, 0, 1, 0, 6, \ TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 0, 1, 1, 6, \ +T(tso_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 0, 0, 1, 1, 6, \ TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 0, 1, 0, 0, 1, 0, 0, 6, \ +T(tso_vlan, 0, 0, 1, 0, 0, 1, 0, 0, 6, \ TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 0, 1, 0, 0, 1, 0, 1, 6, \ +T(tso_vlan_l3l4csum, 0, 0, 1, 0, 0, 1, 0, 1, 6, \ TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 0, 1, 0, 0, 1, 1, 0, 6, \ +T(tso_vlan_ol3ol4csum, 0, 0, 1, 0, 0, 1, 1, 0, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 1, 6, \ +T(tso_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 0, 1, 1, 1, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 0, 1, 0, 1, 0, 0, 0, 6, \ +T(tso_noff, 0, 0, 1, 0, 1, 0, 0, 0, 6, \ TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 0, 1, 0, 1, 0, 0, 1, 6, \ +T(tso_noff_l3l4csum, 0, 0, 1, 0, 1, 0, 0, 1, 6, \ TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 0, 1, 0, 1, 0, 1, 0, 6, \ +T(tso_noff_ol3ol4csum, 0, 0, 1, 0, 1, 0, 1, 0, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 0, 1, 1, 6, \ +T(tso_noff_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 0, 1, 1, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 0, 1, 0, 1, 1, 0, 0, 6, \ +T(tso_noff_vlan, 0, 0, 1, 0, 1, 1, 0, 0, 6, \ TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 0, 1, 0, 1, 1, 0, 1, 6, \ +T(tso_noff_vlan_l3l4csum, 0, 0, 1, 0, 1, 1, 0, 1, 6, \ TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 1, 0, 6, \ +T(tso_noff_vlan_ol3ol4csum, 0, 0, 1, 0, 1, 1, 1, 0, 6, \ TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 1, 6, \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 1, 1, 6, \ TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts, 0, 1, 1, 0, 0, 0, 0, 8, \ +T(tso_ts, 0, 0, 1, 1, 0, 0, 0, 0, 8, \ TSO_F | TSP_F) \ -T(tso_ts_l3l4csum, 0, 1, 1, 0, 0, 0, 1, 8, \ +T(tso_ts_l3l4csum, 0, 0, 1, 1, 0, 0, 0, 1, 8, \ TSO_F | TSP_F | L3L4CSUM_F) \ -T(tso_ts_ol3ol4csum, 0, 1, 1, 0, 0, 1, 0, 8, \ +T(tso_ts_ol3ol4csum, 0, 0, 1, 1, 0, 0, 1, 0, 8, \ TSO_F | TSP_F | OL3OL4CSUM_F) \ -T(tso_ts_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 0, 1, 1, 8, \ +T(tso_ts_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 0, 0, 1, 1, 8, \ TSO_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts_vlan, 0, 1, 1, 0, 1, 0, 0, 8, \ +T(tso_ts_vlan, 0, 0, 1, 1, 0, 1, 0, 0, 8, \ TSO_F | TSP_F | VLAN_F) \ -T(tso_ts_vlan_l3l4csum, 0, 1, 1, 0, 1, 0, 1, 8, \ +T(tso_ts_vlan_l3l4csum, 0, 0, 1, 1, 0, 1, 0, 1, 8, \ TSO_F | TSP_F | VLAN_F | L3L4CSUM_F) \ -T(tso_ts_vlan_ol3ol4csum, 0, 1, 1, 0, 1, 1, 0, 8, \ +T(tso_ts_vlan_ol3ol4csum, 0, 0, 1, 1, 0, 1, 1, 0, 8, \ TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_ts_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 1, 8, \ +T(tso_ts_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 0, 1, 1, 1, 8, \ TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts_noff, 0, 1, 1, 1, 0, 0, 0, 8, \ +T(tso_ts_noff, 0, 0, 1, 1, 1, 0, 0, 0, 8, \ TSO_F | TSP_F | NOFF_F) \ -T(tso_ts_noff_l3l4csum, 0, 1, 1, 1, 0, 0, 1, 8, \ +T(tso_ts_noff_l3l4csum, 0, 0, 1, 1, 1, 0, 0, 1, 8, \ TSO_F | TSP_F | NOFF_F | L3L4CSUM_F) \ -T(tso_ts_noff_ol3ol4csum, 0, 1, 1, 1, 0, 1, 0, 8, \ +T(tso_ts_noff_ol3ol4csum, 0, 0, 1, 1, 1, 0, 1, 0, 8, \ TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_ts_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 0, 1, 1, 8, \ +T(tso_ts_noff_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 0, 1, 1, 8, \ TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts_noff_vlan, 0, 1, 1, 1, 1, 0, 0, 8, \ +T(tso_ts_noff_vlan, 0, 0, 1, 1, 1, 1, 0, 0, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F) \ -T(tso_ts_noff_vlan_l3l4csum, 0, 1, 1, 1, 1, 0, 1, 8, \ +T(tso_ts_noff_vlan_l3l4csum, 0, 0, 1, 1, 1, 1, 0, 1, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_ts_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 1, 0, 8, \ +T(tso_ts_noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 1, 1, 0, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 1, 8, \ +T(tso_ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 1, 1, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ L3L4CSUM_F) \ -T(sec, 1, 0, 0, 0, 0, 0, 0, 8, \ +T(sec, 0, 1, 0, 0, 0, 0, 0, 0, 8, \ TX_SEC_F) \ -T(sec_l3l4csum, 1, 0, 0, 0, 0, 0, 1, 8, \ +T(sec_l3l4csum, 0, 1, 0, 0, 0, 0, 0, 1, 8, \ TX_SEC_F | L3L4CSUM_F) \ -T(sec_ol3ol4csum, 1, 0, 0, 0, 0, 1, 0, 8, \ +T(sec_ol3ol4csum, 0, 1, 0, 0, 0, 0, 1, 0, 8, \ TX_SEC_F | OL3OL4CSUM_F) \ -T(sec_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 0, 1, 1, 8, \ +T(sec_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 0, 0, 1, 1, 8, \ TX_SEC_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_vlan, 1, 0, 0, 0, 1, 0, 0, 8, \ +T(sec_vlan, 0, 1, 0, 0, 0, 1, 0, 0, 8, \ TX_SEC_F | VLAN_F) \ -T(sec_vlan_l3l4csum, 1, 0, 0, 0, 1, 0, 1, 8, \ +T(sec_vlan_l3l4csum, 0, 1, 0, 0, 0, 1, 0, 1, 8, \ TX_SEC_F | VLAN_F | L3L4CSUM_F) \ -T(sec_vlan_ol3ol4csum, 1, 0, 0, 0, 1, 1, 0, 8, \ +T(sec_vlan_ol3ol4csum, 0, 1, 0, 0, 0, 1, 1, 0, 8, \ TX_SEC_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 1, 8, \ +T(sec_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 0, 1, 1, 1, 8, \ TX_SEC_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_noff, 1, 0, 0, 1, 0, 0, 0, 8, \ +T(sec_noff, 0, 1, 0, 0, 1, 0, 0, 0, 8, \ TX_SEC_F | NOFF_F) \ -T(sec_noff_l3l4csum, 1, 0, 0, 1, 0, 0, 1, 8, \ +T(sec_noff_l3l4csum, 0, 1, 0, 0, 1, 0, 0, 1, 8, \ TX_SEC_F | NOFF_F | L3L4CSUM_F) \ -T(sec_noff_ol3ol4csum, 1, 0, 0, 1, 0, 1, 0, 8, \ +T(sec_noff_ol3ol4csum, 0, 1, 0, 0, 1, 0, 1, 0, 8, \ TX_SEC_F | NOFF_F | OL3OL4CSUM_F) \ -T(sec_noff_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 0, 1, 1, 8, \ +T(sec_noff_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 0, 1, 1, 8, \ TX_SEC_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_noff_vlan, 1, 0, 0, 1, 1, 0, 0, 8, \ +T(sec_noff_vlan, 0, 1, 0, 0, 1, 1, 0, 0, 8, \ TX_SEC_F | NOFF_F | VLAN_F) \ -T(sec_noff_vlan_l3l4csum, 1, 0, 0, 1, 1, 0, 1, 8, \ +T(sec_noff_vlan_l3l4csum, 0, 1, 0, 0, 1, 1, 0, 1, 8, \ TX_SEC_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(sec_noff_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 1, 0, 8, \ +T(sec_noff_vlan_ol3ol4csum, 0, 1, 0, 0, 1, 1, 1, 0, 8, \ TX_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 1, 8, \ +T(sec_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 1, 1, 8, \ TX_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_ts, 1, 0, 1, 0, 0, 0, 0, 8, \ +T(sec_ts, 0, 1, 0, 1, 0, 0, 0, 0, 8, \ TX_SEC_F | TSP_F) \ -T(sec_ts_l3l4csum, 1, 0, 1, 0, 0, 0, 1, 8, \ +T(sec_ts_l3l4csum, 0, 1, 0, 1, 0, 0, 0, 1, 8, \ TX_SEC_F | TSP_F | L3L4CSUM_F) \ -T(sec_ts_ol3ol4csum, 1, 0, 1, 0, 0, 1, 0, 8, \ +T(sec_ts_ol3ol4csum, 0, 1, 0, 1, 0, 0, 1, 0, 8, \ TX_SEC_F | TSP_F | OL3OL4CSUM_F) \ -T(sec_ts_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 0, 1, 1, 8, \ +T(sec_ts_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 0, 0, 1, 1, 8, \ TX_SEC_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_ts_vlan, 1, 0, 1, 0, 1, 0, 0, 8, \ +T(sec_ts_vlan, 0, 1, 0, 1, 0, 1, 0, 0, 8, \ TX_SEC_F | TSP_F | VLAN_F) \ -T(sec_ts_vlan_l3l4csum, 1, 0, 1, 0, 1, 0, 1, 8, \ +T(sec_ts_vlan_l3l4csum, 0, 1, 0, 1, 0, 1, 0, 1, 8, \ TX_SEC_F | TSP_F | VLAN_F | L3L4CSUM_F) \ -T(sec_ts_vlan_ol3ol4csum, 1, 0, 1, 0, 1, 1, 0, 8, \ +T(sec_ts_vlan_ol3ol4csum, 0, 1, 0, 1, 0, 1, 1, 0, 8, \ TX_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 1, 8, \ +T(sec_ts_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 0, 1, 1, 1, 8, \ TX_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_ts_noff, 1, 0, 1, 1, 0, 0, 0, 8, \ +T(sec_ts_noff, 0, 1, 0, 1, 1, 0, 0, 0, 8, \ TX_SEC_F | TSP_F | NOFF_F) \ -T(sec_ts_noff_l3l4csum, 1, 0, 1, 1, 0, 0, 1, 8, \ +T(sec_ts_noff_l3l4csum, 0, 1, 0, 1, 1, 0, 0, 1, 8, \ TX_SEC_F | TSP_F | NOFF_F | L3L4CSUM_F) \ -T(sec_ts_noff_ol3ol4csum, 1, 0, 1, 1, 0, 1, 0, 8, \ +T(sec_ts_noff_ol3ol4csum, 0, 1, 0, 1, 1, 0, 1, 0, 8, \ TX_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(sec_ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 0, 1, 1, 8, \ +T(sec_ts_noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 0, 1, 1, 8, \ TX_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_ts_noff_vlan, 1, 0, 1, 1, 1, 0, 0, 8, \ +T(sec_ts_noff_vlan, 0, 1, 0, 1, 1, 1, 0, 0, 8, \ TX_SEC_F | TSP_F | NOFF_F | VLAN_F) \ -T(sec_ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 1, 0, 1, 8, \ +T(sec_ts_noff_vlan_l3l4csum, 0, 1, 0, 1, 1, 1, 0, 1, 8, \ TX_SEC_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(sec_ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 1, 0, 8, \ +T(sec_ts_noff_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 1, 1, 0, 8, \ TX_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 1, 8, \ +T(sec_ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 1, 1, 8, \ TX_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ L3L4CSUM_F) \ -T(sec_tso, 1, 1, 0, 0, 0, 0, 0, 8, \ +T(sec_tso, 0, 1, 1, 0, 0, 0, 0, 0, 8, \ TX_SEC_F | TSO_F) \ -T(sec_tso_l3l4csum, 1, 1, 0, 0, 0, 0, 1, 8, \ +T(sec_tso_l3l4csum, 0, 1, 1, 0, 0, 0, 0, 1, 8, \ TX_SEC_F | TSO_F | L3L4CSUM_F) \ -T(sec_tso_ol3ol4csum, 1, 1, 0, 0, 0, 1, 0, 8, \ +T(sec_tso_ol3ol4csum, 0, 1, 1, 0, 0, 0, 1, 0, 8, \ TX_SEC_F | TSO_F | OL3OL4CSUM_F) \ -T(sec_tso_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 0, 1, 1, 8, \ +T(sec_tso_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 0, 0, 1, 1, 8, \ TX_SEC_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_tso_vlan, 1, 1, 0, 0, 1, 0, 0, 8, \ +T(sec_tso_vlan, 0, 1, 1, 0, 0, 1, 0, 0, 8, \ TX_SEC_F | TSO_F | VLAN_F) \ -T(sec_tso_vlan_l3l4csum, 1, 1, 0, 0, 1, 0, 1, 8, \ +T(sec_tso_vlan_l3l4csum, 0, 1, 1, 0, 0, 1, 0, 1, 8, \ TX_SEC_F | TSO_F | VLAN_F | L3L4CSUM_F) \ -T(sec_tso_vlan_ol3ol4csum, 1, 1, 0, 0, 1, 1, 0, 8, \ +T(sec_tso_vlan_ol3ol4csum, 0, 1, 1, 0, 0, 1, 1, 0, 8, \ TX_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 1, 8, \ +T(sec_tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 0, 1, 1, 1, 8, \ TX_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_tso_noff, 1, 1, 0, 1, 0, 0, 0, 8, \ +T(sec_tso_noff, 0, 1, 1, 0, 1, 0, 0, 0, 8, \ TX_SEC_F | TSO_F | NOFF_F) \ -T(sec_tso_noff_l3l4csum, 1, 1, 0, 1, 0, 0, 1, 8, \ +T(sec_tso_noff_l3l4csum, 0, 1, 1, 0, 1, 0, 0, 1, 8, \ TX_SEC_F | TSO_F | NOFF_F | L3L4CSUM_F) \ -T(sec_tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 1, 0, 8, \ +T(sec_tso_noff_ol3ol4csum, 0, 1, 1, 0, 1, 0, 1, 0, 8, \ TX_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(sec_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 0, 1, 1, 8, \ +T(sec_tso_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 0, 1, 1, 8, \ TX_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_tso_noff_vlan, 1, 1, 0, 1, 1, 0, 0, 8, \ +T(sec_tso_noff_vlan, 0, 1, 1, 0, 1, 1, 0, 0, 8, \ TX_SEC_F | TSO_F | NOFF_F | VLAN_F) \ -T(sec_tso_noff_vlan_l3l4csum, 1, 1, 0, 1, 1, 0, 1, 8, \ +T(sec_tso_noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 1, 0, 1, 8, \ TX_SEC_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(sec_tso_noff_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 1, 0, 8, \ +T(sec_tso_noff_vlan_ol3ol4csum, 0, 1, 1, 0, 1, 1, 1, 0, 8, \ TX_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ T(sec_tso_noff_vlan_ol3ol4csum_l3l4csum, \ - 1, 1, 0, 1, 1, 1, 1, 8, \ + 0, 1, 1, 0, 1, 1, 1, 1, 8, \ TX_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ L3L4CSUM_F) \ -T(sec_tso_ts, 1, 1, 1, 0, 0, 0, 0, 8, \ +T(sec_tso_ts, 0, 1, 1, 1, 0, 0, 0, 0, 8, \ TX_SEC_F | TSO_F | TSP_F) \ -T(sec_tso_ts_l3l4csum, 1, 1, 1, 0, 0, 0, 1, 8, \ +T(sec_tso_ts_l3l4csum, 0, 1, 1, 1, 0, 0, 0, 1, 8, \ TX_SEC_F | TSO_F | TSP_F | L3L4CSUM_F) \ -T(sec_tso_ts_ol3ol4csum, 1, 1, 1, 0, 0, 1, 0, 8, \ +T(sec_tso_ts_ol3ol4csum, 0, 1, 1, 1, 0, 0, 1, 0, 8, \ TX_SEC_F | TSO_F | TSP_F | OL3OL4CSUM_F) \ -T(sec_tso_ts_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 0, 1, 1, 8, \ +T(sec_tso_ts_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 0, 0, 1, 1, 8, \ TX_SEC_F | TSO_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_tso_ts_vlan, 1, 1, 1, 0, 1, 0, 0, 8, \ +T(sec_tso_ts_vlan, 0, 1, 1, 1, 0, 1, 0, 0, 8, \ TX_SEC_F | TSO_F | TSP_F | VLAN_F) \ -T(sec_tso_ts_vlan_l3l4csum, 1, 1, 1, 0, 1, 0, 1, 8, \ +T(sec_tso_ts_vlan_l3l4csum, 0, 1, 1, 1, 0, 1, 0, 1, 8, \ TX_SEC_F | TSO_F | TSP_F | VLAN_F | L3L4CSUM_F) \ -T(sec_tso_ts_vlan_ol3ol4csum, 1, 1, 1, 0, 1, 1, 0, 8, \ +T(sec_tso_ts_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 1, 1, 0, 8, \ TX_SEC_F | TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_tso_ts_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 1, 8, \ +T(sec_tso_ts_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 0, 1, 1, 1, 8, \ TX_SEC_F | TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F | \ L3L4CSUM_F) \ -T(sec_tso_ts_noff, 1, 1, 1, 1, 0, 0, 0, 8, \ +T(sec_tso_ts_noff, 0, 1, 1, 1, 1, 0, 0, 0, 8, \ TX_SEC_F | TSO_F | TSP_F | NOFF_F) \ -T(sec_tso_ts_noff_l3l4csum, 1, 1, 1, 1, 0, 0, 1, 8, \ +T(sec_tso_ts_noff_l3l4csum, 0, 1, 1, 1, 1, 0, 0, 1, 8, \ TX_SEC_F | TSO_F | TSP_F | NOFF_F | L3L4CSUM_F) \ -T(sec_tso_ts_noff_ol3ol4csum, 1, 1, 1, 1, 0, 1, 0, 8, \ +T(sec_tso_ts_noff_ol3ol4csum, 0, 1, 1, 1, 1, 0, 1, 0, 8, \ TX_SEC_F | TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(sec_tso_ts_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 0, 1, 1, 8, \ +T(sec_tso_ts_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 0, 1, 1, 8, \ TX_SEC_F | TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F | \ L3L4CSUM_F) \ -T(sec_tso_ts_noff_vlan, 1, 1, 1, 1, 1, 0, 0, 8, \ +T(sec_tso_ts_noff_vlan, 0, 1, 1, 1, 1, 1, 0, 0, 8, \ TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F) \ -T(sec_tso_ts_noff_vlan_l3l4csum, 1, 1, 1, 1, 1, 0, 1, 8, \ +T(sec_tso_ts_noff_vlan_l3l4csum, 0, 1, 1, 1, 1, 1, 0, 1, 8, \ TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F)\ -T(sec_tso_ts_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 1, 0, 8, \ +T(sec_tso_ts_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 1, 1, 0, 8, \ TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F | \ OL3OL4CSUM_F) \ T(sec_tso_ts_noff_vlan_ol3ol4csum_l3l4csum, \ - 1, 1, 1, 1, 1, 1, 1, 8, \ + 0, 1, 1, 1, 1, 1, 1, 1, 8, \ TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark, 1, 0, 0, 0, 0, 0, 0, 0, 6, \ + TX_MARK_F) \ +T(mark_l3l4csum, 1, 0, 0, 0, 0, 0, 0, 1, 6, \ + TX_MARK_F | L3L4CSUM_F) \ +T(mark_ol3ol4csum, 1, 0, 0, 0, 0, 0, 1, 0, 6, \ + TX_MARK_F | OL3OL4CSUM_F) \ +T(mark_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 0, 0, 1, 1, 6, \ + TX_MARK_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_vlan, 1, 0, 0, 0, 0, 1, 0, 0, 6, \ + TX_MARK_F | VLAN_F) \ +T(mark_vlan_l3l4csum, 1, 0, 0, 0, 0, 1, 0, 1, 6, \ + TX_MARK_F | VLAN_F | L3L4CSUM_F) \ +T(mark_vlan_ol3ol4csum, 1, 0, 0, 0, 0, 1, 1, 0, 6, \ + TX_MARK_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 0, 1, 1, 1, 6, \ + TX_MARK_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_noff, 1, 0, 0, 0, 1, 0, 0, 0, 6, \ + TX_MARK_F | NOFF_F) \ +T(mark_noff_l3l4csum, 1, 0, 0, 0, 1, 0, 0, 1, 6, \ + TX_MARK_F | NOFF_F | L3L4CSUM_F) \ +T(mark_noff_ol3ol4csum, 1, 0, 0, 0, 1, 0, 1, 0, 6, \ + TX_MARK_F | NOFF_F | OL3OL4CSUM_F) \ +T(mark_noff_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 0, 1, 1, 6, \ + TX_MARK_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_noff_vlan, 1, 0, 0, 0, 1, 1, 0, 0, 6, \ + TX_MARK_F | NOFF_F | VLAN_F) \ +T(mark_noff_vlan_l3l4csum, 1, 0, 0, 0, 1, 1, 0, 1, 6, \ + TX_MARK_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(mark_noff_vlan_ol3ol4csum, 1, 0, 0, 0, 1, 1, 1, 0, 6, \ + TX_MARK_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 1, 1, 6, \ + TX_MARK_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F)\ +T(mark_ts, 1, 0, 0, 1, 0, 0, 0, 0, 8, \ + TX_MARK_F | TSP_F) \ +T(mark_ts_l3l4csum, 1, 0, 0, 1, 0, 0, 0, 1, 8, \ + TX_MARK_F | TSP_F | L3L4CSUM_F) \ +T(mark_ts_ol3ol4csum, 1, 0, 0, 1, 0, 0, 1, 0, 8, \ + TX_MARK_F | TSP_F | OL3OL4CSUM_F) \ +T(mark_ts_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 0, 0, 1, 1, 8, \ + TX_MARK_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_ts_vlan, 1, 0, 0, 1, 0, 1, 0, 0, 8, \ + TX_MARK_F | TSP_F | VLAN_F) \ +T(mark_ts_vlan_l3l4csum, 1, 0, 0, 1, 0, 1, 0, 1, 8, \ + TX_MARK_F | TSP_F | VLAN_F | L3L4CSUM_F) \ +T(mark_ts_vlan_ol3ol4csum, 1, 0, 0, 1, 0, 1, 1, 0, 8, \ + TX_MARK_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 0, 1, 1, 1, 8, \ + TX_MARK_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_ts_noff, 1, 0, 0, 1, 1, 0, 0, 0, 8, \ + TX_MARK_F | TSP_F | NOFF_F) \ +T(mark_ts_noff_l3l4csum, 1, 0, 0, 1, 1, 0, 0, 1, 8, \ + TX_MARK_F | TSP_F | NOFF_F | L3L4CSUM_F) \ +T(mark_ts_noff_ol3ol4csum, 1, 0, 0, 1, 1, 0, 1, 0, 8, \ + TX_MARK_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(mark_ts_noff_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 0, 1, 1, 8, \ + TX_MARK_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_ts_noff_vlan, 1, 0, 0, 1, 1, 1, 0, 0, 8, \ + TX_MARK_F | TSP_F | NOFF_F | VLAN_F) \ +T(mark_ts_noff_vlan_l3l4csum, 1, 0, 0, 1, 1, 1, 0, 1, 8, \ + TX_MARK_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(mark_ts_noff_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 1, 1, 0, 8, \ + TX_MARK_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_ts_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 0, 0, 1, 1, 1, 1, 1, 8, \ + TX_MARK_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_tso, 1, 0, 1, 0, 0, 0, 0, 0, 6, \ + TX_MARK_F | TSO_F) \ +T(mark_tso_l3l4csum, 1, 0, 1, 0, 0, 0, 0, 1, 6, \ + TX_MARK_F | TSO_F | L3L4CSUM_F) \ +T(mark_tso_ol3ol4csum, 1, 0, 1, 0, 0, 0, 1, 0, 6, \ + TX_MARK_F | TSO_F | OL3OL4CSUM_F) \ +T(mark_tso_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 0, 0, 1, 1, 6, \ + TX_MARK_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_tso_vlan, 1, 0, 1, 0, 0, 1, 0, 0, 6, \ + TX_MARK_F | TSO_F | VLAN_F) \ +T(mark_tso_vlan_l3l4csum, 1, 0, 1, 0, 0, 1, 0, 1, 6, \ + TX_MARK_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(mark_tso_vlan_ol3ol4csum, 1, 0, 1, 0, 0, 1, 1, 0, 6, \ + TX_MARK_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 0, 1, 1, 1, 6, \ + TX_MARK_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_tso_noff, 1, 0, 1, 0, 1, 0, 0, 0, 6, \ + TX_MARK_F | TSO_F | NOFF_F) \ +T(mark_tso_noff_l3l4csum, 1, 0, 1, 0, 1, 0, 0, 1, 6, \ + TX_MARK_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(mark_tso_noff_ol3ol4csum, 1, 0, 1, 0, 1, 0, 1, 0, 6, \ + TX_MARK_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(mark_tso_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 0, 1, 1, 6, \ + TX_MARK_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_tso_noff_vlan, 1, 0, 1, 0, 1, 1, 0, 0, 6, \ + TX_MARK_F | TSO_F | NOFF_F | VLAN_F) \ +T(mark_tso_noff_vlan_l3l4csum, 1, 0, 1, 0, 1, 1, 0, 1, 6, \ + TX_MARK_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(mark_tso_noff_vlan_ol3ol4csum, 1, 0, 1, 0, 1, 1, 1, 0, 6, \ + TX_MARK_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_tso_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 0, 1, 0, 1, 1, 1, 1, 6, \ + TX_MARK_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_tso_ts, 1, 0, 1, 1, 0, 0, 0, 0, 8, \ + TX_MARK_F | TSO_F | TSP_F) \ +T(mark_tso_ts_l3l4csum, 1, 0, 1, 1, 0, 0, 0, 1, 8, \ + TX_MARK_F | TSO_F | TSP_F | L3L4CSUM_F) \ +T(mark_tso_ts_ol3ol4csum, 1, 0, 1, 1, 0, 0, 1, 0, 8, \ + TX_MARK_F | TSO_F | TSP_F | OL3OL4CSUM_F) \ +T(mark_tso_ts_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 0, 0, 1, 1, 8, \ + TX_MARK_F | TSO_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_tso_ts_vlan, 1, 0, 1, 1, 0, 1, 0, 0, 8, \ + TX_MARK_F | TSO_F | TSP_F | VLAN_F) \ +T(mark_tso_ts_vlan_l3l4csum, 1, 0, 1, 1, 0, 1, 0, 1, 8, \ + TX_MARK_F | TSO_F | TSP_F | VLAN_F | L3L4CSUM_F) \ +T(mark_tso_ts_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 1, 1, 0, 8, \ + TX_MARK_F | TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_tso_ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 0, 1, 1, 1, 8, \ + TX_MARK_F | TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_tso_ts_noff, 1, 0, 1, 1, 1, 0, 0, 0, 8, \ + TX_MARK_F | TSO_F | TSP_F | NOFF_F) \ +T(mark_tso_ts_noff_l3l4csum, 1, 0, 1, 1, 1, 0, 0, 1, 8, \ + TX_MARK_F | TSO_F | TSP_F | NOFF_F | L3L4CSUM_F) \ +T(mark_tso_ts_noff_ol3ol4csum, 1, 0, 1, 1, 1, 0, 1, 0, 8, \ + TX_MARK_F | TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(mark_tso_ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 0, 1, 1, 8, \ + TX_MARK_F | TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_tso_ts_noff_vlan, 1, 0, 1, 1, 1, 1, 0, 0, 8, \ + TX_MARK_F | TSO_F | TSP_F | NOFF_F | VLAN_F) \ +T(mark_tso_ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 1, 1, 0, 1, 8, \ + TX_MARK_F | TSO_F | TSP_F | NOFF_F | VLAN_F | \ + L3L4CSUM_F) \ +T(mark_tso_ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 1, 1, 0, 8, \ + TX_MARK_F | TSO_F | TSP_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F) \ +T(mark_tso_ts_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 0, 1, 1, 1, 1, 1, 1, 8, \ + TX_MARK_F | TSO_F | TSP_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_sec, 1, 1, 0, 0, 0, 0, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F) \ +T(mark_sec_l3l4csum, 1, 1, 0, 0, 0, 0, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | L3L4CSUM_F) \ +T(mark_sec_ol3ol4csum, 1, 1, 0, 0, 0, 0, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | OL3OL4CSUM_F) \ +T(mark_sec_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 0, 0, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_sec_vlan, 1, 1, 0, 0, 0, 1, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | VLAN_F) \ +T(mark_sec_vlan_l3l4csum, 1, 1, 0, 0, 0, 1, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | VLAN_F | L3L4CSUM_F) \ +T(mark_sec_vlan_ol3ol4csum, 1, 1, 0, 0, 0, 1, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_sec_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 0, 1, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_noff, 1, 1, 0, 0, 1, 0, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | NOFF_F) \ +T(mark_sec_noff_l3l4csum, 1, 1, 0, 0, 1, 0, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | NOFF_F | L3L4CSUM_F) \ +T(mark_sec_noff_ol3ol4csum, 1, 1, 0, 0, 1, 0, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | NOFF_F | OL3OL4CSUM_F) \ +T(mark_sec_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 0, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | NOFF_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_noff_vlan, 1, 1, 0, 0, 1, 1, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | NOFF_F | VLAN_F) \ +T(mark_sec_noff_vlan_l3l4csum, 1, 1, 0, 0, 1, 1, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(mark_sec_noff_vlan_ol3ol4csum, 1, 1, 0, 0, 1, 1, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_sec_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 0, 0, 1, 1, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_ts, 1, 1, 0, 1, 0, 0, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F) \ +T(mark_sec_ts_l3l4csum, 1, 1, 0, 1, 0, 0, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | L3L4CSUM_F) \ +T(mark_sec_ts_ol3ol4csum, 1, 1, 0, 1, 0, 0, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | OL3OL4CSUM_F) \ +T(mark_sec_ts_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 0, 0, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_ts_vlan, 1, 1, 0, 1, 0, 1, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | VLAN_F) \ +T(mark_sec_ts_vlan_l3l4csum, 1, 1, 0, 1, 0, 1, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | VLAN_F | L3L4CSUM_F) \ +T(mark_sec_ts_vlan_ol3ol4csum, 1, 1, 0, 1, 0, 1, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_sec_ts_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 0, 1, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_ts_noff, 1, 1, 0, 1, 1, 0, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | NOFF_F) \ +T(mark_sec_ts_noff_l3l4csum, 1, 1, 0, 1, 1, 0, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | NOFF_F | L3L4CSUM_F) \ +T(mark_sec_ts_noff_ol3ol4csum, 1, 1, 0, 1, 1, 0, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(mark_sec_ts_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 0, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_ts_noff_vlan, 1, 1, 0, 1, 1, 1, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | NOFF_F | VLAN_F) \ +T(mark_sec_ts_noff_vlan_l3l4csum, 1, 1, 0, 1, 1, 1, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | NOFF_F | VLAN_F | \ + L3L4CSUM_F) \ +T(mark_sec_ts_noff_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 1, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F) \ +T(mark_sec_ts_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 0, 1, 1, 1, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSP_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_sec_tso, 1, 1, 1, 0, 0, 0, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F) \ +T(mark_sec_tso_l3l4csum, 1, 1, 1, 0, 0, 0, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | L3L4CSUM_F) \ +T(mark_sec_tso_ol3ol4csum, 1, 1, 1, 0, 0, 0, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | OL3OL4CSUM_F) \ +T(mark_sec_tso_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 0, 0, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_tso_vlan, 1, 1, 1, 0, 0, 1, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | VLAN_F) \ +T(mark_sec_tso_vlan_l3l4csum, 1, 1, 1, 0, 0, 1, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(mark_sec_tso_vlan_ol3ol4csum, 1, 1, 1, 0, 0, 1, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(mark_sec_tso_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 1, 0, 0, 1, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_tso_noff, 1, 1, 1, 0, 1, 0, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | NOFF_F) \ +T(mark_sec_tso_noff_l3l4csum, 1, 1, 1, 0, 1, 0, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(mark_sec_tso_noff_ol3ol4csum, 1, 1, 1, 0, 1, 0, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(mark_sec_tso_noff_ol3ol4csum_l3l4csum, \ + 1, 1, 1, 0, 1, 0, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_tso_noff_vlan, 1, 1, 1, 0, 1, 1, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | NOFF_F | VLAN_F) \ +T(mark_sec_tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 1, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | NOFF_F | VLAN_F | \ + L3L4CSUM_F) \ +T(mark_sec_tso_noff_vlan_ol3ol4csum, 1, 1, 1, 0, 1, 1, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F) \ +T(mark_sec_tso_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 1, 0, 1, 1, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_sec_tso_ts, 1, 1, 1, 1, 0, 0, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F) \ +T(mark_sec_tso_ts_l3l4csum, 1, 1, 1, 1, 0, 0, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | L3L4CSUM_F) \ +T(mark_sec_tso_ts_ol3ol4csum, 1, 1, 1, 1, 0, 0, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | OL3OL4CSUM_F) \ +T(mark_sec_tso_ts_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 0, 0, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(mark_sec_tso_ts_vlan, 1, 1, 1, 1, 0, 1, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | VLAN_F) \ +T(mark_sec_tso_ts_vlan_l3l4csum, 1, 1, 1, 1, 0, 1, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | VLAN_F | \ + L3L4CSUM_F) \ +T(mark_sec_tso_ts_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 1, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | VLAN_F | \ + OL3OL4CSUM_F) \ +T(mark_sec_tso_ts_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 1, 1, 0, 1, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | VLAN_F | \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_sec_tso_ts_noff, 1, 1, 1, 1, 1, 0, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | NOFF_F) \ +T(mark_sec_tso_ts_noff_l3l4csum, 1, 1, 1, 1, 1, 0, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | NOFF_F | \ + L3L4CSUM_F) \ +T(mark_sec_tso_ts_noff_ol3ol4csum, 1, 1, 1, 1, 1, 0, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | NOFF_F | \ + OL3OL4CSUM_F) \ +T(mark_sec_tso_ts_noff_ol3ol4csum_l3l4csum, \ + 1, 1, 1, 1, 1, 0, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | NOFF_F | \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(mark_sec_tso_ts_noff_vlan, 1, 1, 1, 1, 1, 1, 0, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F) \ +T(mark_sec_tso_ts_noff_vlan_l3l4csum, 1, 1, 1, 1, 1, 1, 0, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F |\ + L3L4CSUM_F) \ +T(mark_sec_tso_ts_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 1, 1, 0, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F |\ + OL3OL4CSUM_F) \ +T(mark_sec_tso_ts_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 1, 1, 1, 1, 1, 1, 8, \ + TX_MARK_F | TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F |\ OL3OL4CSUM_F | L3L4CSUM_F) + + #endif /* __OTX2_TX_H__ */