From patchwork Fri Jul 2 21:14:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 95232 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 997AFA0C3F; Fri, 2 Jul 2021 23:15:07 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D350E41445; Fri, 2 Jul 2021 23:14:39 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 5D25141434 for ; Fri, 2 Jul 2021 23:14:38 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 162LB9tQ019824 for ; Fri, 2 Jul 2021 14:14:37 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=iV8Qhw/yk+e+kEQyHgFMXV9X9H6MVML5GUKum5C7740=; b=O0E2bU2TunnS/UpS9XWaUKbRMXLMa6IbQj6SglyGsO7+UXaWkz8thj8c2locH9TyKZYu LHfVXc+usdJWObUnrGSDIowk1GLMeHSEvOZtruZCgbJpr5Tu6bSzMXBr0CpuHDHudpB2 CchH1u4Bbws85lCiHX+h8GNcI1Zehg3fED4bSDXR7VFZGlCK8X4GsSjXYOz8Cbjcnbdw asaVJRPrXvW1dU4ppYhzoWIfxh8uOewfCyPTZiyrBRJUYsKx4RZkDWBno486pRL63WAk Vue86KAOPG4iHCC9oiaqLMf5aJHasS91nRc+p2s3n6paUMWovjRYHHJbzdjOygoYCeQm Ag== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 39j3y41d6h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 02 Jul 2021 14:14:37 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 2 Jul 2021 14:14:36 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 2 Jul 2021 14:14:35 -0700 Received: from BG-LT7430.marvell.com (BG-LT7430.marvell.com [10.28.177.176]) by maili.marvell.com (Postfix) with ESMTP id 2385F5B6937; Fri, 2 Jul 2021 14:14:32 -0700 (PDT) From: To: , Pavan Nikhilesh , "Shijith Thotton" , Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao CC: Date: Sat, 3 Jul 2021 02:44:08 +0530 Message-ID: <20210702211408.777-7-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210702211408.777-1-pbhagavatula@marvell.com> References: <20210629080109.581-1-pbhagavatula@marvell.com> <20210702211408.777-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: qP46W0Rc7E7tVQCeBYWaCyJWxzowtTfw X-Proofpoint-GUID: qP46W0Rc7E7tVQCeBYWaCyJWxzowtTfw X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-07-02_10:2021-07-02, 2021-07-02 signatures=0 Subject: [dpdk-dev] [PATCH v6 7/7] event/cnxk: add Tx event vector fastpath X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Add Tx event vector fastpath, integrate event vector Tx routine into Tx burst. Signed-off-by: Pavan Nikhilesh --- doc/guides/eventdevs/cnxk.rst | 1 + doc/guides/rel_notes/release_21_08.rst | 2 +- drivers/common/cnxk/roc_sso.h | 23 ++++++ drivers/event/cnxk/cn10k_eventdev.c | 3 +- drivers/event/cnxk/cn10k_worker.h | 104 +++++++++++++++++++++++-- drivers/event/cnxk/cn9k_worker.h | 4 +- drivers/event/cnxk/cnxk_worker.h | 22 ------ drivers/net/cnxk/cn10k_tx.c | 2 +- drivers/net/cnxk/cn10k_tx.h | 52 +++++++++---- drivers/net/cnxk/cn10k_tx_mseg.c | 3 +- drivers/net/cnxk/cn10k_tx_vec.c | 2 +- drivers/net/cnxk/cn10k_tx_vec_mseg.c | 2 +- 12 files changed, 167 insertions(+), 53 deletions(-) diff --git a/doc/guides/eventdevs/cnxk.rst b/doc/guides/eventdevs/cnxk.rst index 0297cd3d5..53560d383 100644 --- a/doc/guides/eventdevs/cnxk.rst +++ b/doc/guides/eventdevs/cnxk.rst @@ -47,6 +47,7 @@ Features of the OCTEON cnxk SSO PMD are: - Full Rx/Tx offload support defined through ethdev queue configuration. - HW managed event vectorization on CN10K for packets enqueued from ethdev to eventdev configurable per each Rx queue in Rx adapter. +- Event vector transmission via Tx adapter. Prerequisites and Compilation procedure --------------------------------------- diff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst index 11ccc9bcb..9e49cb27d 100644 --- a/doc/guides/rel_notes/release_21_08.rst +++ b/doc/guides/rel_notes/release_21_08.rst @@ -64,7 +64,7 @@ New Features * Added Rx/Tx adapter support for event/cnxk when the ethernet device requested is net/cnxk. - * Add support for event vectorization for Rx adapter. + * Add support for event vectorization for Rx/Tx adapter. Removed Items diff --git a/drivers/common/cnxk/roc_sso.h b/drivers/common/cnxk/roc_sso.h index a6030e7d8..316c6ccd5 100644 --- a/drivers/common/cnxk/roc_sso.h +++ b/drivers/common/cnxk/roc_sso.h @@ -44,6 +44,29 @@ struct roc_sso { uint8_t reserved[ROC_SSO_MEM_SZ] __plt_cache_aligned; } __plt_cache_aligned; +static __rte_always_inline void +roc_sso_hws_head_wait(uintptr_t tag_op) +{ +#ifdef RTE_ARCH_ARM64 + uint64_t tag; + + asm volatile(PLT_CPU_FEATURE_PREAMBLE + " ldr %[tag], [%[tag_op]] \n" + " tbnz %[tag], 35, done%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldr %[tag], [%[tag_op]] \n" + " tbz %[tag], 35, rty%= \n" + "done%=: \n" + : [tag] "=&r"(tag) + : [tag_op] "r"(tag_op)); +#else + /* Wait for the SWTAG/SWTAG_FULL operation */ + while (!(plt_read64(tag_op) & BIT_ULL(35))) + ; +#endif +} + /* SSO device initialization */ int __roc_api roc_sso_dev_init(struct roc_sso *roc_sso); int __roc_api roc_sso_dev_fini(struct roc_sso *roc_sso); diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c index e85fa4785..6f37c5bd2 100644 --- a/drivers/event/cnxk/cn10k_eventdev.c +++ b/drivers/event/cnxk/cn10k_eventdev.c @@ -782,7 +782,8 @@ cn10k_sso_tx_adapter_caps_get(const struct rte_eventdev *dev, if (ret) *caps = 0; else - *caps = RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT; + *caps = RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT | + RTE_EVENT_ETH_TX_ADAPTER_CAP_EVENT_VECTOR; return 0; } diff --git a/drivers/event/cnxk/cn10k_worker.h b/drivers/event/cnxk/cn10k_worker.h index 7a48a6b17..9cc099206 100644 --- a/drivers/event/cnxk/cn10k_worker.h +++ b/drivers/event/cnxk/cn10k_worker.h @@ -308,29 +308,120 @@ uint16_t __rte_hot cn10k_sso_hws_enq_fwd_burst(void *port, NIX_RX_FASTPATH_MODES #undef R -static __rte_always_inline const struct cn10k_eth_txq * +static __rte_always_inline struct cn10k_eth_txq * cn10k_sso_hws_xtract_meta(struct rte_mbuf *m, const uint64_t txq_data[][RTE_MAX_QUEUES_PER_PORT]) { - return (const struct cn10k_eth_txq *) + return (struct cn10k_eth_txq *) txq_data[m->port][rte_event_eth_tx_adapter_txq_get(m)]; } +static __rte_always_inline void +cn10k_sso_vwqe_split_tx(struct rte_mbuf **mbufs, uint16_t nb_mbufs, + uint64_t *cmd, uint16_t lmt_id, uintptr_t lmt_addr, + uint8_t sched_type, uintptr_t base, + const uint64_t txq_data[][RTE_MAX_QUEUES_PER_PORT], + const uint32_t flags) +{ + uint16_t port[4], queue[4]; + struct cn10k_eth_txq *txq; + uint16_t i, j; + uintptr_t pa; + + for (i = 0; i < nb_mbufs; i += 4) { + port[0] = mbufs[i]->port; + port[1] = mbufs[i + 1]->port; + port[2] = mbufs[i + 2]->port; + port[3] = mbufs[i + 3]->port; + + queue[0] = rte_event_eth_tx_adapter_txq_get(mbufs[i]); + queue[1] = rte_event_eth_tx_adapter_txq_get(mbufs[i + 1]); + queue[2] = rte_event_eth_tx_adapter_txq_get(mbufs[i + 2]); + queue[3] = rte_event_eth_tx_adapter_txq_get(mbufs[i + 3]); + + if (((port[0] ^ port[1]) & (port[2] ^ port[3])) || + ((queue[0] ^ queue[1]) & (queue[2] ^ queue[3]))) { + + for (j = 0; j < 4; j++) { + struct rte_mbuf *m = mbufs[i + j]; + + txq = (struct cn10k_eth_txq *) + txq_data[port[j]][queue[j]]; + cn10k_nix_tx_skeleton(txq, cmd, flags); + /* Perform header writes before barrier + * for TSO + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(m, flags); + + cn10k_nix_xmit_prepare(m, cmd, lmt_addr, flags, + txq->lso_tun_fmt); + if (flags & NIX_TX_MULTI_SEG_F) { + const uint16_t segdw = + cn10k_nix_prepare_mseg( + m, (uint64_t *)lmt_addr, + flags); + pa = txq->io_addr | ((segdw - 1) << 4); + } else { + pa = txq->io_addr | + (cn10k_nix_tx_ext_subs(flags) + 1) + << 4; + } + if (!sched_type) + roc_sso_hws_head_wait(base + + SSOW_LF_GWS_TAG); + + roc_lmt_submit_steorl(lmt_id, pa); + } + } else { + txq = (struct cn10k_eth_txq *) + txq_data[port[0]][queue[0]]; + cn10k_nix_xmit_pkts_vector(txq, &mbufs[i], 4, cmd, base + + SSOW_LF_GWS_TAG, + flags | NIX_TX_VWQE_F); + } + } +} + static __rte_always_inline uint16_t cn10k_sso_hws_event_tx(struct cn10k_sso_hws *ws, struct rte_event *ev, uint64_t *cmd, const uint64_t txq_data[][RTE_MAX_QUEUES_PER_PORT], const uint32_t flags) { - const struct cn10k_eth_txq *txq; - struct rte_mbuf *m = ev->mbuf; - uint16_t ref_cnt = m->refcnt; + struct cn10k_eth_txq *txq; + struct rte_mbuf *m; uintptr_t lmt_addr; + uint16_t ref_cnt; uint16_t lmt_id; uintptr_t pa; lmt_addr = ws->lmt_base; ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + + if (ev->event_type & RTE_EVENT_TYPE_VECTOR) { + struct rte_mbuf **mbufs = ev->vec->mbufs; + uint64_t meta = *(uint64_t *)ev->vec; + + if (meta & BIT(31)) { + txq = (struct cn10k_eth_txq *) + txq_data[meta >> 32][meta >> 48]; + + cn10k_nix_xmit_pkts_vector( + txq, mbufs, meta & 0xFFFF, cmd, + ws->tx_base + SSOW_LF_GWS_TAG, + flags | NIX_TX_VWQE_F); + } else { + cn10k_sso_vwqe_split_tx( + mbufs, meta & 0xFFFF, cmd, lmt_id, lmt_addr, + ev->sched_type, ws->tx_base, txq_data, flags); + } + rte_mempool_put(rte_mempool_from_obj(ev->vec), ev->vec); + return (meta & 0xFFFF); + } + + m = ev->mbuf; + ref_cnt = m->refcnt; txq = cn10k_sso_hws_xtract_meta(m, txq_data); cn10k_nix_tx_skeleton(txq, cmd, flags); /* Perform header writes before barrier for TSO */ @@ -346,7 +437,7 @@ cn10k_sso_hws_event_tx(struct cn10k_sso_hws *ws, struct rte_event *ev, pa = txq->io_addr | (cn10k_nix_tx_ext_subs(flags) + 1) << 4; } if (!ev->sched_type) - cnxk_sso_hws_head_wait(ws->tx_base + SSOW_LF_GWS_TAG); + roc_sso_hws_head_wait(ws->tx_base + SSOW_LF_GWS_TAG); roc_lmt_submit_steorl(lmt_id, pa); @@ -357,7 +448,6 @@ cn10k_sso_hws_event_tx(struct cn10k_sso_hws *ws, struct rte_event *ev, cnxk_sso_hws_swtag_flush(ws->tx_base + SSOW_LF_GWS_TAG, ws->tx_base + SSOW_LF_GWS_OP_SWTAG_FLUSH); - return 1; } diff --git a/drivers/event/cnxk/cn9k_worker.h b/drivers/event/cnxk/cn9k_worker.h index 3f9751211..cc1e14195 100644 --- a/drivers/event/cnxk/cn9k_worker.h +++ b/drivers/event/cnxk/cn9k_worker.h @@ -466,7 +466,7 @@ cn9k_sso_hws_event_tx(uint64_t base, struct rte_event *ev, uint64_t *cmd, const uint16_t segdw = cn9k_nix_prepare_mseg(m, cmd, flags); if (!CNXK_TT_FROM_EVENT(ev->event)) { cn9k_nix_xmit_mseg_prep_lmt(cmd, txq->lmt_addr, segdw); - cnxk_sso_hws_head_wait(base + SSOW_LF_GWS_TAG); + roc_sso_hws_head_wait(base + SSOW_LF_GWS_TAG); cn9k_sso_txq_fc_wait(txq); if (cn9k_nix_xmit_submit_lmt(txq->io_addr) == 0) cn9k_nix_xmit_mseg_one(cmd, txq->lmt_addr, @@ -478,7 +478,7 @@ cn9k_sso_hws_event_tx(uint64_t base, struct rte_event *ev, uint64_t *cmd, } else { if (!CNXK_TT_FROM_EVENT(ev->event)) { cn9k_nix_xmit_prep_lmt(cmd, txq->lmt_addr, flags); - cnxk_sso_hws_head_wait(base + SSOW_LF_GWS_TAG); + roc_sso_hws_head_wait(base + SSOW_LF_GWS_TAG); cn9k_sso_txq_fc_wait(txq); if (cn9k_nix_xmit_submit_lmt(txq->io_addr) == 0) cn9k_nix_xmit_one(cmd, txq->lmt_addr, diff --git a/drivers/event/cnxk/cnxk_worker.h b/drivers/event/cnxk/cnxk_worker.h index 7891b749d..9f9ceab8a 100644 --- a/drivers/event/cnxk/cnxk_worker.h +++ b/drivers/event/cnxk/cnxk_worker.h @@ -75,26 +75,4 @@ cnxk_sso_hws_swtag_wait(uintptr_t tag_op) #endif } -static __rte_always_inline void -cnxk_sso_hws_head_wait(uintptr_t tag_op) -{ -#ifdef RTE_ARCH_ARM64 - uint64_t tag; - - asm volatile(" ldr %[tag], [%[tag_op]] \n" - " tbnz %[tag], 35, done%= \n" - " sevl \n" - "rty%=: wfe \n" - " ldr %[tag], [%[tag_op]] \n" - " tbz %[tag], 35, rty%= \n" - "done%=: \n" - : [tag] "=&r"(tag) - : [tag_op] "r"(tag_op)); -#else - /* Wait for the HEAD to be set */ - while (!(plt_read64(tag_op) & BIT_ULL(35))) - ; -#endif -} - #endif diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index 1f30bab59..0e1276c60 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -16,7 +16,7 @@ !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ return 0; \ return cn10k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, \ - flags); \ + 0, flags); \ } NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index eb148b8e7..f75cae07a 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -18,6 +18,7 @@ * Defining it from backwards to denote its been * not used as offload flags to pick function */ +#define NIX_TX_VWQE_F BIT(14) #define NIX_TX_MULTI_SEG_F BIT(15) #define NIX_TX_NEED_SEND_HDR_W1 \ @@ -519,7 +520,7 @@ cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) static __rte_always_inline uint16_t cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, - uint64_t *cmd, const uint16_t flags) + uint64_t *cmd, uintptr_t base, const uint16_t flags) { struct cn10k_eth_txq *txq = tx_queue; const rte_iova_t io_addr = txq->io_addr; @@ -528,14 +529,15 @@ cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, uint64_t lso_tun_fmt; uint64_t data; - NIX_XMIT_FC_OR_RETURN(txq, pkts); + if (!(flags & NIX_TX_VWQE_F)) { + NIX_XMIT_FC_OR_RETURN(txq, pkts); + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + } /* Get cmd skeleton */ cn10k_nix_tx_skeleton(txq, cmd, flags); - /* Reduce the cached count */ - txq->fc_cache_pkts -= pkts; - if (flags & NIX_TX_OFFLOAD_TSO_F) lso_tun_fmt = txq->lso_tun_fmt; @@ -558,6 +560,9 @@ cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); } + if (flags & NIX_TX_VWQE_F) + roc_sso_hws_head_wait(base); + /* Trigger LMTST */ if (burst > 16) { data = cn10k_nix_tx_steor_data(flags); @@ -604,7 +609,8 @@ cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, static __rte_always_inline uint16_t cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, - uint16_t pkts, uint64_t *cmd, const uint16_t flags) + uint16_t pkts, uint64_t *cmd, uintptr_t base, + const uint16_t flags) { struct cn10k_eth_txq *txq = tx_queue; uintptr_t pa0, pa1, lmt_addr = txq->lmt_base; @@ -652,6 +658,9 @@ cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, shft += 3; } + if (flags & NIX_TX_VWQE_F) + roc_sso_hws_head_wait(base); + data0 = (uint64_t)data128; data1 = (uint64_t)(data128 >> 64); /* Make data0 similar to data1 */ @@ -984,7 +993,8 @@ cn10k_nix_prep_lmt_mseg_vector(struct rte_mbuf **mbufs, uint64x2_t *cmd0, static __rte_always_inline uint16_t cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, - uint16_t pkts, uint64_t *cmd, const uint16_t flags) + uint16_t pkts, uint64_t *cmd, uintptr_t base, + const uint16_t flags) { uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; @@ -1013,13 +1023,17 @@ cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, uint64_t data[2]; } wd; - NIX_XMIT_FC_OR_RETURN(txq, pkts); - - scalar = pkts & (NIX_DESCS_PER_LOOP - 1); - pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + if (!(flags & NIX_TX_VWQE_F)) { + NIX_XMIT_FC_OR_RETURN(txq, pkts); + scalar = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + } else { + scalar = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + } - /* Reduce the cached count */ - txq->fc_cache_pkts -= pkts; /* Perform header writes before barrier for TSO */ if (flags & NIX_TX_OFFLOAD_TSO_F) { for (i = 0; i < pkts; i++) @@ -1973,6 +1987,9 @@ cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, if (flags & NIX_TX_MULTI_SEG_F) wd.data[0] >>= 16; + if (flags & NIX_TX_VWQE_F) + roc_sso_hws_head_wait(base); + /* Trigger LMTST */ if (lnum > 16) { if (!(flags & NIX_TX_MULTI_SEG_F)) @@ -2029,10 +2046,11 @@ cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, if (unlikely(scalar)) { if (flags & NIX_TX_MULTI_SEG_F) pkts += cn10k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, - scalar, cmd, flags); + scalar, cmd, base, + flags); else pkts += cn10k_nix_xmit_pkts(tx_queue, tx_pkts, scalar, - cmd, flags); + cmd, base, flags); } return pkts; @@ -2041,13 +2059,15 @@ cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, #else static __rte_always_inline uint16_t cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, - uint16_t pkts, uint64_t *cmd, const uint16_t flags) + uint16_t pkts, uint64_t *cmd, uintptr_t base, + const uint16_t flags) { RTE_SET_USED(tx_queue); RTE_SET_USED(tx_pkts); RTE_SET_USED(pkts); RTE_SET_USED(cmd); RTE_SET_USED(flags); + RTE_SET_USED(base); return 0; } #endif diff --git a/drivers/net/cnxk/cn10k_tx_mseg.c b/drivers/net/cnxk/cn10k_tx_mseg.c index 33f675472..4ea4c8a4e 100644 --- a/drivers/net/cnxk/cn10k_tx_mseg.c +++ b/drivers/net/cnxk/cn10k_tx_mseg.c @@ -18,7 +18,8 @@ !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ return 0; \ return cn10k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ - (flags) | NIX_TX_MULTI_SEG_F); \ + 0, (flags) \ + | NIX_TX_MULTI_SEG_F); \ } NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_tx_vec.c b/drivers/net/cnxk/cn10k_tx_vec.c index 34e373750..a0350496a 100644 --- a/drivers/net/cnxk/cn10k_tx_vec.c +++ b/drivers/net/cnxk/cn10k_tx_vec.c @@ -18,7 +18,7 @@ !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ return 0; \ return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd,\ - (flags)); \ + 0, (flags)); \ } NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_tx_vec_mseg.c b/drivers/net/cnxk/cn10k_tx_vec_mseg.c index 1fad81dba..7f98f79b9 100644 --- a/drivers/net/cnxk/cn10k_tx_vec_mseg.c +++ b/drivers/net/cnxk/cn10k_tx_vec_mseg.c @@ -16,7 +16,7 @@ !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ return 0; \ return cn10k_nix_xmit_pkts_vector( \ - tx_queue, tx_pkts, pkts, cmd, \ + tx_queue, tx_pkts, pkts, cmd, 0, \ (flags) | NIX_TX_MULTI_SEG_F); \ }