From patchwork Mon Dec 13 08:22:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 105101 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4582BA00BE; Mon, 13 Dec 2021 09:23:47 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DCB7341226; Mon, 13 Dec 2021 09:23:10 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id D8960411AE for ; Mon, 13 Dec 2021 09:23:08 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 1BD1ZF0T027372; Mon, 13 Dec 2021 00:23:08 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Gsq6TjMYJ/sE2PdyajSnxqU1QtCDagZk4ZzwbuRs/yw=; b=iuBxCRi7VBkkl3OJ0hjcor+Goo8Bpz8jA7zOWgflbU8SuCf9CaJzRI/hYZEXiSBqh54/ r9g0Zrx39Mv3pWg6yOGXW+ZdxKcRivRiw9sRHy4BPpOopE1ztETT15Fq13lz6hBaB+Vu I5/kye9A52wVV2iI468i/3hsQIqsqrr9QrKkSoYrqLCIiMMpPCBi+hx4MXpGsn6Gxktf yexvxaWZtxuHLa9ONcyp5UmaLguYyL+88fn2k7mNDc5YkFWbZY8EvRu+5b7A2z+LWVNg Hm9B8gUfsjbw5bmqwao+Q5qywjHWqeZvciaIaiPpLtmkyFa3u+D396tGihXIjqZREkyL 1g== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3cwvmys3r6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 13 Dec 2021 00:23:07 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 13 Dec 2021 00:23:06 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 13 Dec 2021 00:23:06 -0800 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id 74E4C3F7050; Mon, 13 Dec 2021 00:23:03 -0800 (PST) From: To: , , , Pavan Nikhilesh , Shijith Thotton , Nithin Dabilpuram , "Kiran Kumar K" , Sunil Kumar Kori , Satha Rao CC: Subject: [PATCH 8/8] net/cnxk: add CN10K template Tx functions to build Date: Mon, 13 Dec 2021 13:52:25 +0530 Message-ID: <20211213082226.3646-8-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211213082226.3646-1-pbhagavatula@marvell.com> References: <20211213082226.3646-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: iBzoobo_MkedwpvqOACI5PdZfCDLevHa X-Proofpoint-ORIG-GUID: iBzoobo_MkedwpvqOACI5PdZfCDLevHa X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2021-12-13_03,2021-12-10_01,2021-12-02_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Pavan Nikhilesh Add CN10K segregated Tx and event Tx template functions to build, add macros to make future modifications simpler. Signed-off-by: Pavan Nikhilesh --- drivers/event/cnxk/cn10k_eventdev.c | 19 +- drivers/event/cnxk/cn10k_worker.h | 36 +- drivers/event/cnxk/cn10k_worker_tx_enq.c | 23 - drivers/event/cnxk/cn10k_worker_tx_enq_seg.c | 23 - drivers/event/cnxk/meson.build | 21 +- drivers/net/cnxk/cn10k_tx.c | 90 ---- drivers/net/cnxk/cn10k_tx.h | 522 +++++++++---------- drivers/net/cnxk/cn10k_tx_mseg.c | 26 - drivers/net/cnxk/cn10k_tx_select.c | 63 +++ drivers/net/cnxk/cn10k_tx_vec.c | 25 - drivers/net/cnxk/cn10k_tx_vec_mseg.c | 24 - drivers/net/cnxk/meson.build | 41 +- 12 files changed, 413 insertions(+), 500 deletions(-) delete mode 100644 drivers/event/cnxk/cn10k_worker_tx_enq.c delete mode 100644 drivers/event/cnxk/cn10k_worker_tx_enq_seg.c delete mode 100644 drivers/net/cnxk/cn10k_tx.c delete mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c create mode 100644 drivers/net/cnxk/cn10k_tx_select.c delete mode 100644 drivers/net/cnxk/cn10k_tx_vec.c delete mode 100644 drivers/net/cnxk/cn10k_tx_vec_mseg.c -- 2.17.1 diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c index 02f3d8235d..b56426960a 100644 --- a/drivers/event/cnxk/cn10k_eventdev.c +++ b/drivers/event/cnxk/cn10k_eventdev.c @@ -10,14 +10,7 @@ deq_op = deq_ops[dev->rx_offloads & (NIX_RX_OFFLOAD_MAX - 1)] #define CN10K_SET_EVDEV_ENQ_OP(dev, enq_op, enq_ops) \ - (enq_op = \ - enq_ops[!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] \ - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] \ - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] \ - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MBUF_NOFF_F)] \ - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_VLAN_QINQ_F)] \ - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] \ - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]) + enq_op = enq_ops[dev->tx_offloads & (NIX_TX_OFFLOAD_MAX - 1)] static uint32_t cn10k_sso_gw_mode_wdata(struct cnxk_sso_evdev *dev) @@ -390,17 +383,15 @@ cn10k_sso_fp_fns_set(struct rte_eventdev *event_dev) /* Tx modes */ const event_tx_adapter_enqueue_t - sso_hws_tx_adptr_enq[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_sso_hws_tx_adptr_enq_##name, + sso_hws_tx_adptr_enq[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn10k_sso_hws_tx_adptr_enq_##name, NIX_TX_FASTPATH_MODES #undef T }; const event_tx_adapter_enqueue_t - sso_hws_tx_adptr_enq_seg[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_sso_hws_tx_adptr_enq_seg_##name, + sso_hws_tx_adptr_enq_seg[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn10k_sso_hws_tx_adptr_enq_seg_##name, NIX_TX_FASTPATH_MODES #undef T }; diff --git a/drivers/event/cnxk/cn10k_worker.h b/drivers/event/cnxk/cn10k_worker.h index 160b90aa27..78d029baaa 100644 --- a/drivers/event/cnxk/cn10k_worker.h +++ b/drivers/event/cnxk/cn10k_worker.h @@ -613,17 +613,43 @@ cn10k_sso_hws_event_tx(struct cn10k_sso_hws *ws, struct rte_event *ev, return 1; } -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, sz, flags) \ uint16_t __rte_hot cn10k_sso_hws_tx_adptr_enq_##name( \ void *port, struct rte_event ev[], uint16_t nb_events); \ uint16_t __rte_hot cn10k_sso_hws_tx_adptr_enq_seg_##name( \ - void *port, struct rte_event ev[], uint16_t nb_events); \ - uint16_t __rte_hot cn10k_sso_hws_dual_tx_adptr_enq_##name( \ - void *port, struct rte_event ev[], uint16_t nb_events); \ - uint16_t __rte_hot cn10k_sso_hws_dual_tx_adptr_enq_seg_##name( \ void *port, struct rte_event ev[], uint16_t nb_events); NIX_TX_FASTPATH_MODES #undef T +#define SSO_TX(fn, sz, flags) \ + uint16_t __rte_hot fn(void *port, struct rte_event ev[], \ + uint16_t nb_events) \ + { \ + struct cn10k_sso_hws *ws = port; \ + uint64_t cmd[sz]; \ + \ + RTE_SET_USED(nb_events); \ + return cn10k_sso_hws_event_tx( \ + ws, &ev[0], cmd, \ + (const uint64_t(*)[RTE_MAX_QUEUES_PER_PORT]) & \ + ws->tx_adptr_data, \ + flags); \ + } + +#define SSO_TX_SEG(fn, sz, flags) \ + uint16_t __rte_hot fn(void *port, struct rte_event ev[], \ + uint16_t nb_events) \ + { \ + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ + struct cn10k_sso_hws *ws = port; \ + \ + RTE_SET_USED(nb_events); \ + return cn10k_sso_hws_event_tx( \ + ws, &ev[0], cmd, \ + (const uint64_t(*)[RTE_MAX_QUEUES_PER_PORT]) & \ + ws->tx_adptr_data, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + #endif diff --git a/drivers/event/cnxk/cn10k_worker_tx_enq.c b/drivers/event/cnxk/cn10k_worker_tx_enq.c deleted file mode 100644 index f14c7fc223..0000000000 --- a/drivers/event/cnxk/cn10k_worker_tx_enq.c +++ /dev/null @@ -1,23 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(C) 2021 Marvell. - */ - -#include "cn10k_worker.h" - -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - uint16_t __rte_hot cn10k_sso_hws_tx_adptr_enq_##name( \ - void *port, struct rte_event ev[], uint16_t nb_events) \ - { \ - struct cn10k_sso_hws *ws = port; \ - uint64_t cmd[sz]; \ - \ - RTE_SET_USED(nb_events); \ - return cn10k_sso_hws_event_tx( \ - ws, &ev[0], cmd, \ - (const uint64_t(*)[RTE_MAX_QUEUES_PER_PORT]) & \ - ws->tx_adptr_data, \ - flags); \ - } - -NIX_TX_FASTPATH_MODES -#undef T diff --git a/drivers/event/cnxk/cn10k_worker_tx_enq_seg.c b/drivers/event/cnxk/cn10k_worker_tx_enq_seg.c deleted file mode 100644 index 2ea61e5d88..0000000000 --- a/drivers/event/cnxk/cn10k_worker_tx_enq_seg.c +++ /dev/null @@ -1,23 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(C) 2021 Marvell. - */ - -#include "cn10k_worker.h" - -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - uint16_t __rte_hot cn10k_sso_hws_tx_adptr_enq_seg_##name( \ - void *port, struct rte_event ev[], uint16_t nb_events) \ - { \ - uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ - struct cn10k_sso_hws *ws = port; \ - \ - RTE_SET_USED(nb_events); \ - return cn10k_sso_hws_event_tx( \ - ws, &ev[0], cmd, \ - (const uint64_t(*)[RTE_MAX_QUEUES_PER_PORT]) & \ - ws->tx_adptr_data, \ - (flags) | NIX_TX_MULTI_SEG_F); \ - } - -NIX_TX_FASTPATH_MODES -#undef T diff --git a/drivers/event/cnxk/meson.build b/drivers/event/cnxk/meson.build index ac452fe451..b27bae7b12 100644 --- a/drivers/event/cnxk/meson.build +++ b/drivers/event/cnxk/meson.build @@ -13,8 +13,6 @@ sources = files( 'cn9k_worker.c', 'cn10k_eventdev.c', 'cn10k_worker.c', - 'cn10k_worker_tx_enq.c', - 'cn10k_worker_tx_enq_seg.c', 'cnxk_eventdev.c', 'cnxk_eventdev_adptr.c', 'cnxk_eventdev_selftest.c', @@ -454,6 +452,25 @@ sources += files( 'deq/cn10k/deq_112_127_ca_tmo_seg_burst.c', ) +sources += files( + 'tx/cn10k/tx_0_15.c', + 'tx/cn10k/tx_16_31.c', + 'tx/cn10k/tx_32_47.c', + 'tx/cn10k/tx_48_63.c', + 'tx/cn10k/tx_64_79.c', + 'tx/cn10k/tx_80_95.c', + 'tx/cn10k/tx_96_111.c', + 'tx/cn10k/tx_112_127.c', + 'tx/cn10k/tx_0_15_seg.c', + 'tx/cn10k/tx_16_31_seg.c', + 'tx/cn10k/tx_32_47_seg.c', + 'tx/cn10k/tx_48_63_seg.c', + 'tx/cn10k/tx_64_79_seg.c', + 'tx/cn10k/tx_80_95_seg.c', + 'tx/cn10k/tx_96_111_seg.c', + 'tx/cn10k/tx_112_127_seg.c', +) + extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] foreach flag: extra_flags if cc.has_argument(flag) diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c deleted file mode 100644 index 5e6c5ee111..0000000000 --- a/drivers/net/cnxk/cn10k_tx.c +++ /dev/null @@ -1,90 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(C) 2021 Marvell. - */ - -#include "cn10k_ethdev.h" -#include "cn10k_tx.h" - -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ - void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ - { \ - uint64_t cmd[sz]; \ - \ - /* For TSO inner checksum is a must */ \ - if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ - !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ - return 0; \ - return cn10k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, \ - 0, flags); \ - } - -NIX_TX_FASTPATH_MODES -#undef T - -static inline void -pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2][2][2]) -{ - struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - - /* [SEC] [TSP] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ - eth_dev->tx_pkt_burst = tx_burst - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_SECURITY_F)] - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; -} - -void -cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) -{ - struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, - - NIX_TX_FASTPATH_MODES -#undef T - }; - - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, - - NIX_TX_FASTPATH_MODES -#undef T - }; - - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, - - NIX_TX_FASTPATH_MODES -#undef T - }; - - const eth_tx_burst_t nix_eth_tx_vec_burst_mseg[2][2][2][2][2][2][2] = { -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_mseg_##name, - - NIX_TX_FASTPATH_MODES -#undef T - }; - - if (dev->scalar_ena) { - pick_tx_func(eth_dev, nix_eth_tx_burst); - if (dev->tx_offloads & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) - pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); - } else { - pick_tx_func(eth_dev, nix_eth_tx_vec_burst); - if (dev->tx_offloads & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) - pick_tx_func(eth_dev, nix_eth_tx_vec_burst_mseg); - } - - rte_mb(); -} diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 873e1871f9..6de8b18b47 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -16,6 +16,7 @@ #define NIX_TX_OFFLOAD_TSO_F BIT(4) #define NIX_TX_OFFLOAD_TSTAMP_F BIT(5) #define NIX_TX_OFFLOAD_SECURITY_F BIT(6) +#define NIX_TX_OFFLOAD_MAX (NIX_TX_OFFLOAD_SECURITY_F << 1) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -2675,279 +2676,272 @@ cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, #define T_SEC_F NIX_TX_OFFLOAD_SECURITY_F /* [T_SEC_F] [TSP] [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ -#define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 0, 0, 4, \ - NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 0, 0, 1, 4, \ - L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 0, 0, 1, 0, 4, \ - OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 0, 1, 1, 4, \ - OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 0, 0, 1, 0, 0, 6, \ - VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 0, 0, 1, 0, 1, 6, \ - VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 0, 0, 1, 1, 0, 6, \ - VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 1, 6, \ - VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 0, 0, 1, 0, 0, 0, 4, \ - NOFF_F) \ -T(noff_l3l4csum, 0, 0, 0, 1, 0, 0, 1, 4, \ - NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 0, 0, 1, 0, 1, 0, 4, \ - NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 0, 1, 1, 4, \ - NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 0, 0, 1, 1, 0, 0, 6, \ - NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 0, 0, 1, 1, 0, 1, 6, \ - NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 0, 0, 1, 1, 1, 0, 6, \ - NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 1, 6, \ - NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso, 0, 0, 1, 0, 0, 0, 0, 6, \ - TSO_F) \ -T(tso_l3l4csum, 0, 0, 1, 0, 0, 0, 1, 6, \ - TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 0, 0, 1, 0, 0, 1, 0, 6, \ - TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 0, 1, 1, 6, \ - TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 0, 0, 1, 0, 1, 0, 0, 6, \ - TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 0, 0, 1, 0, 1, 0, 1, 6, \ - TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 0, 0, 1, 0, 1, 1, 0, 6, \ - TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 1, 6, \ - TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 0, 0, 1, 1, 0, 0, 0, 6, \ - TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 0, 0, 1, 1, 0, 0, 1, 6, \ - TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 0, 0, 1, 1, 0, 1, 0, 6, \ - TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 0, 1, 1, 6, \ - TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 0, 0, 1, 1, 1, 0, 0, 6, \ - TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 0, 0, 1, 1, 1, 0, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 1, 0, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts, 0, 1, 0, 0, 0, 0, 0, 8, \ - TSP_F) \ -T(ts_l3l4csum, 0, 1, 0, 0, 0, 0, 1, 8, \ - TSP_F | L3L4CSUM_F) \ -T(ts_ol3ol4csum, 0, 1, 0, 0, 0, 1, 0, 8, \ - TSP_F | OL3OL4CSUM_F) \ -T(ts_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 0, 1, 1, 8, \ - TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_vlan, 0, 1, 0, 0, 1, 0, 0, 8, \ - TSP_F | VLAN_F) \ -T(ts_vlan_l3l4csum, 0, 1, 0, 0, 1, 0, 1, 8, \ - TSP_F | VLAN_F | L3L4CSUM_F) \ -T(ts_vlan_ol3ol4csum, 0, 1, 0, 0, 1, 1, 0, 8, \ - TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 1, 8, \ - TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_noff, 0, 1, 0, 1, 0, 0, 0, 8, \ - TSP_F | NOFF_F) \ -T(ts_noff_l3l4csum, 0, 1, 0, 1, 0, 0, 1, 8, \ - TSP_F | NOFF_F | L3L4CSUM_F) \ -T(ts_noff_ol3ol4csum, 0, 1, 0, 1, 0, 1, 0, 8, \ - TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(ts_noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 0, 1, 1, 8, \ - TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_noff_vlan, 0, 1, 0, 1, 1, 0, 0, 8, \ - TSP_F | NOFF_F | VLAN_F) \ -T(ts_noff_vlan_l3l4csum, 0, 1, 0, 1, 1, 0, 1, 8, \ - TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(ts_noff_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 1, 0, 8, \ - TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 1, 8, \ - TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_tso, 0, 1, 1, 0, 0, 0, 0, 8, \ - TSP_F | TSO_F) \ -T(ts_tso_l3l4csum, 0, 1, 1, 0, 0, 0, 1, 8, \ - TSP_F | TSO_F | L3L4CSUM_F) \ -T(ts_tso_ol3ol4csum, 0, 1, 1, 0, 0, 1, 0, 8, \ - TSP_F | TSO_F | OL3OL4CSUM_F) \ -T(ts_tso_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 0, 1, 1, 8, \ - TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_tso_vlan, 0, 1, 1, 0, 1, 0, 0, 8, \ - TSP_F | TSO_F | VLAN_F) \ -T(ts_tso_vlan_l3l4csum, 0, 1, 1, 0, 1, 0, 1, 8, \ - TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ -T(ts_tso_vlan_ol3ol4csum, 0, 1, 1, 0, 1, 1, 0, 8, \ - TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 1, 8, \ - TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_tso_noff, 0, 1, 1, 1, 0, 0, 0, 8, \ - TSP_F | TSO_F | NOFF_F) \ -T(ts_tso_noff_l3l4csum, 0, 1, 1, 1, 0, 0, 1, 8, \ - TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ -T(ts_tso_noff_ol3ol4csum, 0, 1, 1, 1, 0, 1, 0, 8, \ - TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(ts_tso_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 0, 1, 1, 8, \ - TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_tso_noff_vlan, 0, 1, 1, 1, 1, 0, 0, 8, \ - TSP_F | TSO_F | NOFF_F | VLAN_F) \ -T(ts_tso_noff_vlan_l3l4csum, 0, 1, 1, 1, 1, 0, 1, 8, \ - TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(ts_tso_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 1, 0, 8, \ - TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 1, 8, \ - TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F)\ -T(sec, 1, 0, 0, 0, 0, 0, 0, 4, \ - T_SEC_F) \ -T(sec_l3l4csum, 1, 0, 0, 0, 0, 0, 1, 4, \ - T_SEC_F | L3L4CSUM_F) \ -T(sec_ol3ol4csum, 1, 0, 0, 0, 0, 1, 0, 4, \ - T_SEC_F | OL3OL4CSUM_F) \ -T(sec_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 0, 1, 1, 4, \ - T_SEC_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_vlan, 1, 0, 0, 0, 1, 0, 0, 6, \ - T_SEC_F | VLAN_F) \ -T(sec_vlan_l3l4csum, 1, 0, 0, 0, 1, 0, 1, 6, \ - T_SEC_F | VLAN_F | L3L4CSUM_F) \ -T(sec_vlan_ol3ol4csum, 1, 0, 0, 0, 1, 1, 0, 6, \ - T_SEC_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 1, 6, \ - T_SEC_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_noff, 1, 0, 0, 1, 0, 0, 0, 4, \ - T_SEC_F | NOFF_F) \ -T(sec_noff_l3l4csum, 1, 0, 0, 1, 0, 0, 1, 4, \ - T_SEC_F | NOFF_F | L3L4CSUM_F) \ -T(sec_noff_ol3ol4csum, 1, 0, 0, 1, 0, 1, 0, 4, \ - T_SEC_F | NOFF_F | OL3OL4CSUM_F) \ -T(sec_noff_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 0, 1, 1, 4, \ - T_SEC_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_noff_vlan, 1, 0, 0, 1, 1, 0, 0, 6, \ - T_SEC_F | NOFF_F | VLAN_F) \ -T(sec_noff_vlan_l3l4csum, 1, 0, 0, 1, 1, 0, 1, 6, \ - T_SEC_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(sec_noff_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 1, 0, 6, \ - T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 1, 6, \ - T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_tso, 1, 0, 1, 0, 0, 0, 0, 6, \ - T_SEC_F | TSO_F) \ -T(sec_tso_l3l4csum, 1, 0, 1, 0, 0, 0, 1, 6, \ - T_SEC_F | TSO_F | L3L4CSUM_F) \ -T(sec_tso_ol3ol4csum, 1, 0, 1, 0, 0, 1, 0, 6, \ - T_SEC_F | TSO_F | OL3OL4CSUM_F) \ -T(sec_tso_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 0, 1, 1, 6, \ - T_SEC_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_tso_vlan, 1, 0, 1, 0, 1, 0, 0, 6, \ - T_SEC_F | TSO_F | VLAN_F) \ -T(sec_tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 0, 1, 6, \ - T_SEC_F | TSO_F | VLAN_F | L3L4CSUM_F) \ -T(sec_tso_vlan_ol3ol4csum, 1, 0, 1, 0, 1, 1, 0, 6, \ - T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 1, 6, \ - T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_tso_noff, 1, 0, 1, 1, 0, 0, 0, 6, \ - T_SEC_F | TSO_F | NOFF_F) \ -T(sec_tso_noff_l3l4csum, 1, 0, 1, 1, 0, 0, 1, 6, \ - T_SEC_F | TSO_F | NOFF_F | L3L4CSUM_F) \ -T(sec_tso_noff_ol3ol4csum, 1, 0, 1, 1, 0, 1, 0, 6, \ - T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(sec_tso_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 0, 1, 1, 6, \ - T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_tso_noff_vlan, 1, 0, 1, 1, 1, 0, 0, 6, \ - T_SEC_F | TSO_F | NOFF_F | VLAN_F) \ -T(sec_tso_noff_vlan_l3l4csum, 1, 0, 1, 1, 1, 0, 1, 6, \ - T_SEC_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(sec_tso_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 1, 0, 6, \ - T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 1, 6, \ - T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F)\ -T(sec_ts, 1, 1, 0, 0, 0, 0, 0, 8, \ - T_SEC_F | TSP_F) \ -T(sec_ts_l3l4csum, 1, 1, 0, 0, 0, 0, 1, 8, \ - T_SEC_F | TSP_F | L3L4CSUM_F) \ -T(sec_ts_ol3ol4csum, 1, 1, 0, 0, 0, 1, 0, 8, \ - T_SEC_F | TSP_F | OL3OL4CSUM_F) \ -T(sec_ts_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 0, 1, 1, 8, \ - T_SEC_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_ts_vlan, 1, 1, 0, 0, 1, 0, 0, 8, \ - T_SEC_F | TSP_F | VLAN_F) \ -T(sec_ts_vlan_l3l4csum, 1, 1, 0, 0, 1, 0, 1, 8, \ - T_SEC_F | TSP_F | VLAN_F | L3L4CSUM_F) \ -T(sec_ts_vlan_ol3ol4csum, 1, 1, 0, 0, 1, 1, 0, 8, \ - T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_ts_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 1, 8, \ - T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_ts_noff, 1, 1, 0, 1, 0, 0, 0, 8, \ - T_SEC_F | TSP_F | NOFF_F) \ -T(sec_ts_noff_l3l4csum, 1, 1, 0, 1, 0, 0, 1, 8, \ - T_SEC_F | TSP_F | NOFF_F | L3L4CSUM_F) \ -T(sec_ts_noff_ol3ol4csum, 1, 1, 0, 1, 0, 1, 0, 8, \ - T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(sec_ts_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 0, 1, 1, 8, \ - T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_ts_noff_vlan, 1, 1, 0, 1, 1, 0, 0, 8, \ - T_SEC_F | TSP_F | NOFF_F | VLAN_F) \ -T(sec_ts_noff_vlan_l3l4csum, 1, 1, 0, 1, 1, 0, 1, 8, \ - T_SEC_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(sec_ts_noff_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 1, 0, 8, \ - T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 1, 8, \ - T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F)\ -T(sec_ts_tso, 1, 1, 1, 0, 0, 0, 0, 8, \ - T_SEC_F | TSP_F | TSO_F) \ -T(sec_ts_tso_l3l4csum, 1, 1, 1, 0, 0, 0, 1, 8, \ - T_SEC_F | TSP_F | TSO_F | L3L4CSUM_F) \ -T(sec_ts_tso_ol3ol4csum, 1, 1, 1, 0, 0, 1, 0, 8, \ - T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F) \ -T(sec_ts_tso_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 0, 1, 1, 8, \ - T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_ts_tso_vlan, 1, 1, 1, 0, 1, 0, 0, 8, \ - T_SEC_F | TSP_F | TSO_F | VLAN_F) \ -T(sec_ts_tso_vlan_l3l4csum, 1, 1, 1, 0, 1, 0, 1, 8, \ - T_SEC_F | TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ -T(sec_ts_tso_vlan_ol3ol4csum, 1, 1, 1, 0, 1, 1, 0, 8, \ - T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(sec_ts_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 1, 8, \ - T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(sec_ts_tso_noff, 1, 1, 1, 1, 0, 0, 0, 8, \ - T_SEC_F | TSP_F | TSO_F | NOFF_F) \ -T(sec_ts_tso_noff_l3l4csum, 1, 1, 1, 1, 0, 0, 1, 8, \ - T_SEC_F | TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ -T(sec_ts_tso_noff_ol3ol4csum, 1, 1, 1, 1, 0, 1, 0, 8, \ - T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(sec_ts_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 0, 1, 1, 8, \ - T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F)\ -T(sec_ts_tso_noff_vlan, 1, 1, 1, 1, 1, 0, 0, 8, \ - T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F) \ -T(sec_ts_tso_noff_vlan_l3l4csum, 1, 1, 1, 1, 1, 0, 1, 8, \ - T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(sec_ts_tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 1, 0, 8, \ - T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F)\ -T(sec_ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 1, 8, \ - T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ - L3L4CSUM_F) - -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define NIX_TX_FASTPATH_MODES_0_15 \ + T(no_offload, 4, NIX_TX_OFFLOAD_NONE) \ + T(l3l4csum, 4, L3L4CSUM_F) \ + T(ol3ol4csum, 4, OL3OL4CSUM_F) \ + T(ol3ol4csum_l3l4csum, 4, OL3OL4CSUM_F | L3L4CSUM_F) \ + T(vlan, 6, VLAN_F) \ + T(vlan_l3l4csum, 6, VLAN_F | L3L4CSUM_F) \ + T(vlan_ol3ol4csum, 6, VLAN_F | OL3OL4CSUM_F) \ + T(vlan_ol3ol4csum_l3l4csum, 6, VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(noff, 4, NOFF_F) \ + T(noff_l3l4csum, 4, NOFF_F | L3L4CSUM_F) \ + T(noff_ol3ol4csum, 4, NOFF_F | OL3OL4CSUM_F) \ + T(noff_ol3ol4csum_l3l4csum, 4, NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(noff_vlan, 6, NOFF_F | VLAN_F) \ + T(noff_vlan_l3l4csum, 6, NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(noff_vlan_ol3ol4csum, 6, NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(noff_vlan_ol3ol4csum_l3l4csum, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_16_31 \ + T(tso, 6, TSO_F) \ + T(tso_l3l4csum, 6, TSO_F | L3L4CSUM_F) \ + T(tso_ol3ol4csum, 6, TSO_F | OL3OL4CSUM_F) \ + T(tso_ol3ol4csum_l3l4csum, 6, TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(tso_vlan, 6, TSO_F | VLAN_F) \ + T(tso_vlan_l3l4csum, 6, TSO_F | VLAN_F | L3L4CSUM_F) \ + T(tso_vlan_ol3ol4csum, 6, TSO_F | VLAN_F | OL3OL4CSUM_F) \ + T(tso_vlan_ol3ol4csum_l3l4csum, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(tso_noff, 6, TSO_F | NOFF_F) \ + T(tso_noff_l3l4csum, 6, TSO_F | NOFF_F | L3L4CSUM_F) \ + T(tso_noff_ol3ol4csum, 6, TSO_F | NOFF_F | OL3OL4CSUM_F) \ + T(tso_noff_ol3ol4csum_l3l4csum, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(tso_noff_vlan, 6, TSO_F | NOFF_F | VLAN_F) \ + T(tso_noff_vlan_l3l4csum, 6, TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(tso_noff_vlan_ol3ol4csum, 6, TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(tso_noff_vlan_ol3ol4csum_l3l4csum, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_32_47 \ + T(ts, 8, TSP_F) \ + T(ts_l3l4csum, 8, TSP_F | L3L4CSUM_F) \ + T(ts_ol3ol4csum, 8, TSP_F | OL3OL4CSUM_F) \ + T(ts_ol3ol4csum_l3l4csum, 8, TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_vlan, 8, TSP_F | VLAN_F) \ + T(ts_vlan_l3l4csum, 8, TSP_F | VLAN_F | L3L4CSUM_F) \ + T(ts_vlan_ol3ol4csum, 8, TSP_F | VLAN_F | OL3OL4CSUM_F) \ + T(ts_vlan_ol3ol4csum_l3l4csum, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_noff, 8, TSP_F | NOFF_F) \ + T(ts_noff_l3l4csum, 8, TSP_F | NOFF_F | L3L4CSUM_F) \ + T(ts_noff_ol3ol4csum, 8, TSP_F | NOFF_F | OL3OL4CSUM_F) \ + T(ts_noff_ol3ol4csum_l3l4csum, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_noff_vlan, 8, TSP_F | NOFF_F | VLAN_F) \ + T(ts_noff_vlan_l3l4csum, 8, TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(ts_noff_vlan_ol3ol4csum, 8, TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(ts_noff_vlan_ol3ol4csum_l3l4csum, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_48_63 \ + T(ts_tso, 8, TSP_F | TSO_F) \ + T(ts_tso_l3l4csum, 8, TSP_F | TSO_F | L3L4CSUM_F) \ + T(ts_tso_ol3ol4csum, 8, TSP_F | TSO_F | OL3OL4CSUM_F) \ + T(ts_tso_ol3ol4csum_l3l4csum, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_tso_vlan, 8, TSP_F | TSO_F | VLAN_F) \ + T(ts_tso_vlan_l3l4csum, 8, TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ + T(ts_tso_vlan_ol3ol4csum, 8, TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ + T(ts_tso_vlan_ol3ol4csum_l3l4csum, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_tso_noff, 8, TSP_F | TSO_F | NOFF_F) \ + T(ts_tso_noff_l3l4csum, 8, TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ + T(ts_tso_noff_ol3ol4csum, 8, TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ + T(ts_tso_noff_ol3ol4csum_l3l4csum, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_tso_noff_vlan, 8, TSP_F | TSO_F | NOFF_F | VLAN_F) \ + T(ts_tso_noff_vlan_l3l4csum, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(ts_tso_noff_vlan_ol3ol4csum, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_64_79 \ + T(sec, 4, T_SEC_F) \ + T(sec_l3l4csum, 4, T_SEC_F | L3L4CSUM_F) \ + T(sec_ol3ol4csum, 4, T_SEC_F | OL3OL4CSUM_F) \ + T(sec_ol3ol4csum_l3l4csum, 4, T_SEC_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_vlan, 6, T_SEC_F | VLAN_F) \ + T(sec_vlan_l3l4csum, 6, T_SEC_F | VLAN_F | L3L4CSUM_F) \ + T(sec_vlan_ol3ol4csum, 6, T_SEC_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_vlan_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_noff, 4, T_SEC_F | NOFF_F) \ + T(sec_noff_l3l4csum, 4, T_SEC_F | NOFF_F | L3L4CSUM_F) \ + T(sec_noff_ol3ol4csum, 4, T_SEC_F | NOFF_F | OL3OL4CSUM_F) \ + T(sec_noff_ol3ol4csum_l3l4csum, 4, \ + T_SEC_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_noff_vlan, 6, T_SEC_F | NOFF_F | VLAN_F) \ + T(sec_noff_vlan_l3l4csum, 6, T_SEC_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(sec_noff_vlan_ol3ol4csum, 6, \ + T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_noff_vlan_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_80_95 \ + T(sec_tso, 6, T_SEC_F | TSO_F) \ + T(sec_tso_l3l4csum, 6, T_SEC_F | TSO_F | L3L4CSUM_F) \ + T(sec_tso_ol3ol4csum, 6, T_SEC_F | TSO_F | OL3OL4CSUM_F) \ + T(sec_tso_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_tso_vlan, 6, T_SEC_F | TSO_F | VLAN_F) \ + T(sec_tso_vlan_l3l4csum, 6, T_SEC_F | TSO_F | VLAN_F | L3L4CSUM_F) \ + T(sec_tso_vlan_ol3ol4csum, 6, T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_tso_vlan_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_tso_noff, 6, T_SEC_F | TSO_F | NOFF_F) \ + T(sec_tso_noff_l3l4csum, 6, T_SEC_F | TSO_F | NOFF_F | L3L4CSUM_F) \ + T(sec_tso_noff_ol3ol4csum, 6, T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ + T(sec_tso_noff_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_tso_noff_vlan, 6, T_SEC_F | TSO_F | NOFF_F | VLAN_F) \ + T(sec_tso_noff_vlan_l3l4csum, 6, \ + T_SEC_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(sec_tso_noff_vlan_ol3ol4csum, 6, \ + T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_tso_noff_vlan_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_96_111 \ + T(sec_ts, 8, T_SEC_F | TSP_F) \ + T(sec_ts_l3l4csum, 8, T_SEC_F | TSP_F | L3L4CSUM_F) \ + T(sec_ts_ol3ol4csum, 8, T_SEC_F | TSP_F | OL3OL4CSUM_F) \ + T(sec_ts_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_vlan, 8, T_SEC_F | TSP_F | VLAN_F) \ + T(sec_ts_vlan_l3l4csum, 8, T_SEC_F | TSP_F | VLAN_F | L3L4CSUM_F) \ + T(sec_ts_vlan_ol3ol4csum, 8, T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_ts_vlan_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_noff, 8, T_SEC_F | TSP_F | NOFF_F) \ + T(sec_ts_noff_l3l4csum, 8, T_SEC_F | TSP_F | NOFF_F | L3L4CSUM_F) \ + T(sec_ts_noff_ol3ol4csum, 8, T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ + T(sec_ts_noff_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_noff_vlan, 8, T_SEC_F | TSP_F | NOFF_F | VLAN_F) \ + T(sec_ts_noff_vlan_l3l4csum, 8, \ + T_SEC_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(sec_ts_noff_vlan_ol3ol4csum, 8, \ + T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_ts_noff_vlan_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_112_127 \ + T(sec_ts_tso, 8, T_SEC_F | TSP_F | TSO_F) \ + T(sec_ts_tso_l3l4csum, 8, T_SEC_F | TSP_F | TSO_F | L3L4CSUM_F) \ + T(sec_ts_tso_ol3ol4csum, 8, T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F) \ + T(sec_ts_tso_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_tso_vlan, 8, T_SEC_F | TSP_F | TSO_F | VLAN_F) \ + T(sec_ts_tso_vlan_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ + T(sec_ts_tso_vlan_ol3ol4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_ts_tso_vlan_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_tso_noff, 8, T_SEC_F | TSP_F | TSO_F | NOFF_F) \ + T(sec_ts_tso_noff_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ + T(sec_ts_tso_noff_ol3ol4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ + T(sec_ts_tso_noff_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_tso_noff_vlan, 8, T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F) \ + T(sec_ts_tso_noff_vlan_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(sec_ts_tso_noff_vlan_ol3ol4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES \ + NIX_TX_FASTPATH_MODES_0_15 \ + NIX_TX_FASTPATH_MODES_16_31 \ + NIX_TX_FASTPATH_MODES_32_47 \ + NIX_TX_FASTPATH_MODES_48_63 \ + NIX_TX_FASTPATH_MODES_64_79 \ + NIX_TX_FASTPATH_MODES_80_95 \ + NIX_TX_FASTPATH_MODES_96_111 \ + NIX_TX_FASTPATH_MODES_112_127 + +#define T(name, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ - \ + \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_mseg_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ - \ + \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_vec_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ - \ + \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_vec_mseg_##name( \ - void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES #undef T +#define NIX_TX_XMIT(fn, sz, flags) \ + uint16_t __rte_noinline __rte_hot fn( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, 0, \ + flags); \ + } + +#define NIX_TX_XMIT_MSEG(fn, sz, flags) \ + uint16_t __rte_noinline __rte_hot fn( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + 0, \ + flags | NIX_TX_MULTI_SEG_F); \ + } + +#define NIX_TX_XMIT_VEC(fn, sz, flags) \ + uint16_t __rte_noinline __rte_hot fn( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, \ + cmd, 0, (flags)); \ + } + +#define NIX_TX_XMIT_VEC_MSEG(fn, sz, flags) \ + uint16_t __rte_noinline __rte_hot fn( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts_vector( \ + tx_queue, tx_pkts, pkts, cmd, 0, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + #endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/cn10k_tx_mseg.c b/drivers/net/cnxk/cn10k_tx_mseg.c deleted file mode 100644 index 2b834095cf..0000000000 --- a/drivers/net/cnxk/cn10k_tx_mseg.c +++ /dev/null @@ -1,26 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(C) 2021 Marvell. - */ - -#include "cn10k_ethdev.h" -#include "cn10k_tx.h" - -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - uint16_t __rte_noinline __rte_hot \ - cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ - struct rte_mbuf **tx_pkts, \ - uint16_t pkts) \ - { \ - uint64_t cmd[(sz)]; \ - \ - /* For TSO inner checksum is a must */ \ - if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ - !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ - return 0; \ - return cn10k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ - 0, (flags) \ - | NIX_TX_MULTI_SEG_F); \ - } - -NIX_TX_FASTPATH_MODES -#undef T diff --git a/drivers/net/cnxk/cn10k_tx_select.c b/drivers/net/cnxk/cn10k_tx_select.c new file mode 100644 index 0000000000..bd1fba06b9 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx_select.c @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[NIX_TX_OFFLOAD_MAX]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [SEC] [TSP] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = + tx_burst[dev->tx_offload_flags & (NIX_TX_OFFLOAD_MAX - 1)]; +} + +void +cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + const eth_tx_burst_t nix_eth_tx_burst[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn10k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + const eth_tx_burst_t nix_eth_tx_burst_mseg[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn10k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + const eth_tx_burst_t nix_eth_tx_vec_burst[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn10k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + const eth_tx_burst_t nix_eth_tx_vec_burst_mseg[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn10k_nix_xmit_pkts_vec_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena) { + pick_tx_func(eth_dev, nix_eth_tx_burst); + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); + } else { + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_vec_burst_mseg); + } + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_tx_vec.c b/drivers/net/cnxk/cn10k_tx_vec.c deleted file mode 100644 index 2789b13d60..0000000000 --- a/drivers/net/cnxk/cn10k_tx_vec.c +++ /dev/null @@ -1,25 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(C) 2021 Marvell. - */ - -#include "cn10k_ethdev.h" -#include "cn10k_tx.h" - -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - uint16_t __rte_noinline __rte_hot \ - cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ - struct rte_mbuf **tx_pkts, \ - uint16_t pkts) \ - { \ - uint64_t cmd[sz]; \ - \ - /* For TSO inner checksum is a must */ \ - if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ - !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ - return 0; \ - return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd,\ - 0, (flags)); \ - } - -NIX_TX_FASTPATH_MODES -#undef T diff --git a/drivers/net/cnxk/cn10k_tx_vec_mseg.c b/drivers/net/cnxk/cn10k_tx_vec_mseg.c deleted file mode 100644 index 98000df101..0000000000 --- a/drivers/net/cnxk/cn10k_tx_vec_mseg.c +++ /dev/null @@ -1,24 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(C) 2021 Marvell. - */ - -#include "cn10k_ethdev.h" -#include "cn10k_tx.h" - -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ - uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_vec_mseg_##name( \ - void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ - { \ - uint64_t cmd[sz]; \ - \ - /* For TSO inner checksum is a must */ \ - if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ - !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ - return 0; \ - return cn10k_nix_xmit_pkts_vector( \ - tx_queue, tx_pkts, pkts, cmd, 0, \ - (flags) | NIX_TX_MULTI_SEG_F); \ - } - -NIX_TX_FASTPATH_MODES -#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 1fd388a1f0..375c75d1c7 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -107,11 +107,8 @@ sources += files( 'cn10k_ethdev.c', 'cn10k_ethdev_sec.c', 'cn10k_rte_flow.c', - 'cn10k_tx.c', - 'cn10k_tx_mseg.c', - 'cn10k_tx_vec.c', - 'cn10k_tx_vec_mseg.c', 'cn10k_rx_select.c', + 'cn10k_tx_select.c', ) sources += files( @@ -149,6 +146,42 @@ sources += files( 'rx/cn10k/rx_112_127_vec_mseg.c', ) +sources += files( + 'tx/cn10k/tx_0_15.c', + 'tx/cn10k/tx_16_31.c', + 'tx/cn10k/tx_32_47.c', + 'tx/cn10k/tx_48_63.c', + 'tx/cn10k/tx_64_79.c', + 'tx/cn10k/tx_80_95.c', + 'tx/cn10k/tx_96_111.c', + 'tx/cn10k/tx_112_127.c', + 'tx/cn10k/tx_0_15_mseg.c', + 'tx/cn10k/tx_16_31_mseg.c', + 'tx/cn10k/tx_32_47_mseg.c', + 'tx/cn10k/tx_48_63_mseg.c', + 'tx/cn10k/tx_64_79_mseg.c', + 'tx/cn10k/tx_80_95_mseg.c', + 'tx/cn10k/tx_96_111_mseg.c', + 'tx/cn10k/tx_112_127_mseg.c', + 'tx/cn10k/tx_0_15_vec.c', + 'tx/cn10k/tx_16_31_vec.c', + 'tx/cn10k/tx_32_47_vec.c', + 'tx/cn10k/tx_48_63_vec.c', + 'tx/cn10k/tx_64_79_vec.c', + 'tx/cn10k/tx_80_95_vec.c', + 'tx/cn10k/tx_96_111_vec.c', + 'tx/cn10k/tx_112_127_vec.c', + 'tx/cn10k/tx_0_15_vec_mseg.c', + 'tx/cn10k/tx_16_31_vec_mseg.c', + 'tx/cn10k/tx_32_47_vec_mseg.c', + 'tx/cn10k/tx_48_63_vec_mseg.c', + 'tx/cn10k/tx_64_79_vec_mseg.c', + 'tx/cn10k/tx_80_95_vec_mseg.c', + 'tx/cn10k/tx_96_111_vec_mseg.c', + 'tx/cn10k/tx_112_127_vec_mseg.c', +) + + deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk']