From patchwork Mon Jul 16 09:26:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh X-Patchwork-Id: 43095 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9E24523C; Mon, 16 Jul 2018 11:27:26 +0200 (CEST) Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0070.outbound.protection.outlook.com [104.47.33.70]) by dpdk.org (Postfix) with ESMTP id C1348235 for ; Mon, 16 Jul 2018 11:27:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yDz92a7GbeIwnU2kZAZi63fRDIvebIHdJYvJ52yIKfY=; b=k97dtX4E0C3dOTyK7l4kFfbu/dww61QgH5Fs+we2wfwi0ieqP5CJ5Cr9TXytPmhoHZaBV9WQS0TkLY9eC3yDDg2O9u0UEFi/fX/gHb0BuwNUwyrkgR7JAeHVu6YjOar/kCbzbwuo0PTYUoHgSViyNxAmjeQ7BqlinC5dFJmyy00= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Pavan.Bhagavatula@cavium.com; Received: from localhost.localdomain (12.108.191.226) by MWHPR07MB3472.namprd07.prod.outlook.com (2603:10b6:301:63::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.952.18; Mon, 16 Jul 2018 09:27:19 +0000 From: Pavan Nikhilesh To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, rkudurumalla@caviumnetworks.com, ferruh.yigit@intel.com Cc: dev@dpdk.org, Pavan Nikhilesh Date: Mon, 16 Jul 2018 14:56:56 +0530 Message-Id: <20180716092657.17072-1-pbhagavatula@caviumnetworks.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180701164637.978-1-pbhagavatula@caviumnetworks.com> References: <20180701164637.978-1-pbhagavatula@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [12.108.191.226] X-ClientProxiedBy: DM6PR08CA0007.namprd08.prod.outlook.com (2603:10b6:5:80::20) To MWHPR07MB3472.namprd07.prod.outlook.com (2603:10b6:301:63::23) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 67f972cc-dc55-4ee4-f686-08d5eafe5596 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(5600053)(711020)(2017052603328)(7153060)(7193020); SRVR:MWHPR07MB3472; X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3472; 3:S2bEFRR/hwkdE5Let8VJpIap8ISe/XKNT5SUWQSbQxDAZAkuAd2014Ld8025GSH4pO6JdNRkm8r6VICNI6I/QfHr2qugFsdvk85tgQyvxcZYogBTm+d1hKnZoFeh6egIB9xIcrvk5WrZqqwGRPAJ9T0XDOFZpR1gzsjpL1OtjI7ItjPcJxWhEB85JE30Y+Xkmpq44O7ZaBE8t9lrduPhKWKZRh51rMLKZfAFYwDznWfsqMepvbpZiVtCOa/pgQa8; 25:EJC1qNRQHkL2RS9ZJuw2Va8ilwoZZrFjLlJPatlk+Gfm1tFjlqOc83GmXmD8h4DLPgzKrEKiP93pMb+1zXLjfMKTD1gpg6c1aotMwfkbYeZFuTi0YJ/97jvbH669sd3NaBnrGn42p8za5AVEJwcaGy9UoytcuLAAIFNfUut0nMTaWSB7x81qtFiKMgzM60ulZoQ7s8AGxC3rVkpMfx3Tgy/vdsnDNoIKXjGJwCRG2i/cJZNldKf9Oq4UzmQBiPC93ntOIt2pPFPu57VJguYPAxRXX92H3ZW8bbSr94LMim4Qd1MXzczzkbG5PySNthsAjBYniWyRZ42r2XWteZl9zg==; 31:Y2KC7lXM1d9lHZtVxP4tfkpdmyElns78LN2n0yucAIqwZU9c8fuYqZzcGU4WSamZAvoOp0pnQ+BkZTxpKlcQckh8/hh6zrl6uP4JWAHP7L1BUwx8nLCaXy5NPOegEOcyWHqxQqaiPmQ+SSxNLkjfSoHNQjA3zYi1bWpDA3cguwUsvYnhFZBcJsEXtTjpX3mTmvKB/D0GSvQvM865kQfzL44nsT5+8ot7JF0js6A29a4= X-MS-TrafficTypeDiagnostic: MWHPR07MB3472: X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3472; 20:8ViTCVkUZ5bLFNuLuns+ejbniolvVBI2wbE/JV4Uxgza3a7IMEy2MR7YC4AqNGwwbUeVBQHRKmGGC8xGRd0dMuBu+Om5VVdHHDzZFU3cCGBVK6uskA4P/ubqycRMZ4A8yDYkzxS/ZTnN/VMBTfFXWTvAmGlU60BrNwd3zgl3XHjwr7hOV54bWAfIX81Kj6uxKfws35MWFrG0r1Baxk/nOO6Vl2NPZCBXrT4LKJoHedGsk2m4wnWMQFHYGQWsfNf7V+Iln2f34pwdZiMT+rMtjSx8JCD9jQsE6PBs+KFjv7pjGY02Xk1zHfwXLR6WdzLDvRFFVF3JQdXMcKyIJ852latdinm9LY7Vq6ZLSy2DnumWMaSSFnaBTmKErdPXJiLfd/l6+12ipq0+sTCfBwjS1qKc4Y6/AEl4VWoFWaeQAZDR1TFIxzsLeGAk3CKgFnHkQUoXLOg+eT4t4hNWNHYlwH60WpxzvsH1ff1M3Sbdth1D8x78Wdrg6loWFgoBO4b8cu92eiKOH4xJ5/uzqB1j9gZBw5qmbj0081QG/4hZz0K7EqwjzjEf4/pWKJwuC2hoCJBYx9PaOekrtFCObbXzdrf0NG+S0XJiO16oEiXZe/4=; 4:Dr1j9n8qPttzrG4rqCJyOAYG9CcwmTo7AJVQUMcnI8FmdJykIy7utnvG6qwxNniq5pM3SkQ5Ea3ZH5kl1M4ZZzc+hVncYtBXIJ7786OaEQfMvw2qQm71PFYt4g5rgSQmzwr/m8D+5jPO665PZJaq8dCTpOEs8/GujrQ7o+KDy/wHC2hu8nwTFk3dOq4EIOgbZZB4TD1Cp1WnIX3xkB2nawJLCDOZx5XfC9AQM1EESd2aT2pCqlvIbdcdXeGObN+qjIgqRlTLmhQqk5NCVmk8Ng== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(10201501046)(3002001)(3231311)(944501410)(52105095)(93006095)(149027)(150027)(6041310)(20161123564045)(20161123558120)(20161123562045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016); SRVR:MWHPR07MB3472; BCL:0; PCL:0; RULEID:; SRVR:MWHPR07MB3472; X-Forefront-PRVS: 073515755F X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(396003)(366004)(39860400002)(376002)(346002)(136003)(199004)(189003)(6116002)(106356001)(6506007)(42882007)(81166006)(81156014)(8936002)(72206003)(478600001)(386003)(3846002)(8676002)(16526019)(47776003)(1857600001)(105586002)(6512007)(66066001)(52116002)(1076002)(446003)(6486002)(51416003)(26005)(76176011)(97736004)(107886003)(53936002)(5660300001)(7736002)(68736007)(2906002)(6666003)(25786009)(956004)(16586007)(2616005)(476003)(14444005)(11346002)(50226002)(486006)(36756003)(316002)(305945005)(50466002)(4326008)(48376002)(42262002); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR07MB3472; H:localhost.localdomain; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR07MB3472; 23:bux0mCwoNy0RUlob96rM7KjUlCYUtS3VH+ldh1Ugp?= TquF5E1H8/zcSQguYX9Ic68uGzhi90BxJNQMqhd9XxYkkdbdc5cJN/+yY7ogFHf9+oDGBKabzCNrAFhErkeJ4YWne+PGJixyhBnqRvNDEBftjXG99euRE7uNc6mVMHVAUbE5qXD8xQaQSwpA6uypAMe0hybX2TRtg5BxGWiEQBatYCuYLgIzDG+JOKSiZbYxfHa1fG3JhXYCUi7Sq0fPCM1gZXdIn7vUxSUKSJzsolGqlGIjjpkkNjAUCGvxtr2BbkvMQ5f2nCK0v+GTdCcm6wU0mcZXjn4s2bqnY6/uHyMstk0pzarmFcVf5ltUgAD8UtH+dMrB3cnPOxnzvt7JAwjxe2xo1DoyjgdMv8MxinHDPS6FMDO3WgBXPPn8D/Xugl+UznamKHYGjd1E+zsSInmISlogAwp+yOINE/C+C4X9gfsjvqXKDpTUPDE5+JMDayyIji/267Lkgqjc+NRmCJjoeLH3HeVykTNnZ8Yq7FRgW07VWABeJw6N4HfF4bjQGYS373gJRM+O8rDYBHtZPqyKgnraaczhjny0SCdk5LrjKbl5EX12QZMUkFKAedWZZjmvwC4v5Sn+xhqT8PF53DLutJZxI1XJ321v5tfGQsL/fWwGPVn5uTLjRegiM90rw14Oo6/4hzA12QHLL41ftX4wu/PRSXqjqYvuQggsgaFq4eHlAcnrzlUlApBbo+/hdINBKT5zXOyFrxOVaqbR0qgdNKGakgELLhUJceVYe59ZZbi1+/YANcpiDq+loioaYnuDDBgLgegZQUO+z+QmGMwFSxYeQJ51vL6JxgvMm3wz43svVmyaR0/uTpth6U9gLxoED93azFw6VMk/onOjm2cAShAp9HKhPr8qXdaLbk5CG1pH8on8FZ8VjR/zH4e7/ktBmTTXxdTeyAJsma0oyqfKYWJqf+QNTKSr4ZHnv51qAS/1piwurDPnHi/8Zdl9KaoJCAgNtDGVDDv/w6ymvDE/ZZBG4fPWmEw4gv0zjZBybg15CjlRpltdZHlr7X7kHk5dcs1i0sYzK5QW050bxFI7bnDhQvJ0eOyvVtKtLS5uYyzeX626F9l0zO/b1UxrQIMPcfxVyxIy7pcaa3aOJ06P+Oju3ERkQVn6q9QioWf6/ROBYHaEbS6LBp8fh1LhOHtEioyItYTFseMN8bCzYy/GUgl+FH2NWJ9xC6wyzgZPqGsjvvH+JAbb781E6qIk8jY5WQ420P2nouId/9OJTCj X-Microsoft-Antispam-Message-Info: Dps9wnOn+dQmzWwqVt2P2WOhtIjG4XW0/6h4DCkrVP4+8/113zvTUSu0mbCz7srmrzJOOYFp+aGPiN7JiKl5L/0FSL/Qx4EAaTSh2Rj/ZU80x2oOEjNs9RItjwEugu+/48xmUXk0avxwAD4KM3oj9v+qLXwRVFMAktjCdTXPZVfJUYhSgNEkvpglCbUR+AbxMmMdfeAj3ES3Nl1mwfRC7V72Ubg/uRCKYXYv+qmKQRXyHRIQPoUx7ZVQvZOY6B0BpwY5I1mcZMCxQgF1Z94EfS68jQsIeIv3XnULdITIzcCo0dimluMLhwtjO+yKQYbXi5m7yQBx+n1nIhYiXCbkFyqpiMRZoHgSPp96/UGJIvI= X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3472; 6:gpfQdkag1bajhkOiroRd+rxKzPxZFJ0Gvq8qES1Cpmd+ydnERSjFaUcOUFwWHA5u58hZmcxkI5EeXILiLJlsRe248TPqv/8BAVHT3Tx8LNvvOr5O8JhHgr5+l9wcOaSv879wZXwTV0KNGTYUKR3y0Ugcql3psYDZSWKeqE68W4Mg+rrLA6NfGlyCEP/n0bgX+nMH1PJKoty+guy8qiH96AQJbVNZSu07KRcqbYtkVHnqmgcB9OieAPQG7WKWS8wGCFInmlvX5F6R3nyC6kfndki3qrGzqDLnRMKS1IAkq9JedWBCWwz0NPb4LovCOkvjObUrYspK8A8vMF+wTI6RLB8AZqRpDw7F2Cjs7ZhZggrvsH/PyBtgNyrwDpIDkCLt8Kpp1vesriYpidOSoCrJVlHb0xjjk+Xua9bBo8hvM8zcUE+JUHObwEl677fRb+YWmd/aGDlJt0Wp8eD+kGw5hg==; 5:hMT5uKmfrTA406mqgkouu5FuBY238u3pjOmeAsmIFIqlgn3bNdtw21nUdR+z2/hs8fgkhPiLLJ9B6kkW36QW1rRHghl55xEVCLEaGq+/KgkXK10A5nwyDC2srlIpZWLZQzYx64h7IbUlvNRrb1CqSdIjLClkVvRcRZczbPM5k0A=; 24:GXiI2FLMdUMKQqplPzY1oil9clgj8BNnAd7emZPhlZaxrXsRyND/7HN1ABIA1MNRdSEENh4qjQ+AoBgeJkWc/vR+Zwt8do1nHMilUWacavk= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3472; 7:ZnyPAVpMkIXiFdAuqsm5X5zjANO/6nW7QPdgbYJg6IUXFx59sVIu2bVhpfXQV5h+1VgBpDYKdJ25STAOF0qXKTA/rfTdMTL9f8yMjhgut8kbib3EI6DqdihztWZ6hBDzOUAa35t/x8ehKg7hhqcq5kqNdCv4Nm+YS4WDGOG95m3kOX2sVI+K67JNLZE30znbLf4B9eTZlprvZdJWY+QTsFXwI1nngJvtZOWy2bzTP+SZELg/QU6+bnn47jCAL9Dy X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jul 2018 09:27:19.4155 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 67f972cc-dc55-4ee4-f686-08d5eafe5596 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR07MB3472 Subject: [dpdk-dev] [PATCH v2 1/2] net/thunderx: enable Rx checksum offload X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add L3/L4 Rx checksum offload and update capabilities. Signed-off-by: Pavan Nikhilesh --- v2 Changes: - Add Rx checksum offload support for l3fwd. drivers/net/thunderx/nicvf_ethdev.c | 33 ++++++++----- drivers/net/thunderx/nicvf_ethdev.h | 1 + drivers/net/thunderx/nicvf_rxtx.c | 73 +++++++++++++++++++++++++---- drivers/net/thunderx/nicvf_rxtx.h | 15 ++++-- drivers/net/thunderx/nicvf_struct.h | 27 ++++++----- 5 files changed, 113 insertions(+), 36 deletions(-) -- 2.18.0 diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 76fed9f99..8fd52e1c3 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -357,11 +357,9 @@ nicvf_dev_supported_ptypes_get(struct rte_eth_dev *dev) } memcpy((char *)ptypes + copied, &ptypes_end, sizeof(ptypes_end)); - if (dev->rx_pkt_burst == nicvf_recv_pkts || - dev->rx_pkt_burst == nicvf_recv_pkts_multiseg) - return ptypes; - return NULL; + /* All Ptypes are supported in all Rx functions. */ + return ptypes; } static void @@ -918,13 +916,18 @@ nicvf_set_tx_function(struct rte_eth_dev *dev) static void nicvf_set_rx_function(struct rte_eth_dev *dev) { - if (dev->data->scattered_rx) { - PMD_DRV_LOG(DEBUG, "Using multi-segment rx callback"); - dev->rx_pkt_burst = nicvf_recv_pkts_multiseg; - } else { - PMD_DRV_LOG(DEBUG, "Using single-segment rx callback"); - dev->rx_pkt_burst = nicvf_recv_pkts; - } + struct nicvf *nic = nicvf_pmd_priv(dev); + + const eth_rx_burst_t rx_burst_func[2][2] = { + /* [NORMAL/SCATTER] [NO_CKSUM/CKSUM] */ + [0][0] = nicvf_recv_pkts_no_offload, + [0][1] = nicvf_recv_pkts_cksum, + [1][0] = nicvf_recv_pkts_multiseg_no_offload, + [1][1] = nicvf_recv_pkts_multiseg_cksum, + }; + + dev->rx_pkt_burst = + rx_burst_func[dev->data->scattered_rx][nic->offload_cksum]; } static int @@ -1245,6 +1248,9 @@ nicvf_rxq_mbuf_setup(struct nicvf_rxq *rxq) offsetof(struct rte_mbuf, data_off) != 4); RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - offsetof(struct rte_mbuf, data_off) != 6); + RTE_BUILD_BUG_ON(offsetof(struct nicvf_rxq, rxq_fastpath_data_end) - + offsetof(struct nicvf_rxq, + rxq_fastpath_data_start) > 128); mb_def.nb_segs = 1; mb_def.data_off = RTE_PKTMBUF_HEADROOM + (nic->skip_bytes); mb_def.port = rxq->port_id; @@ -1745,7 +1751,7 @@ nicvf_dev_start(struct rte_eth_dev *dev) return ret; } - /* Configure callbacks based on scatter mode */ + /* Configure callbacks based on offloads */ nicvf_set_tx_function(dev); nicvf_set_rx_function(dev); @@ -1964,6 +1970,9 @@ nicvf_dev_configure(struct rte_eth_dev *dev) } } + if (rxmode->offloads & DEV_RX_OFFLOAD_CHECKSUM) + nic->offload_cksum = 1; + PMD_INIT_LOG(DEBUG, "Configured ethdev port%d hwcap=0x%" PRIx64, dev->data->port_id, nicvf_hw_cap(nic)); diff --git a/drivers/net/thunderx/nicvf_ethdev.h b/drivers/net/thunderx/nicvf_ethdev.h index 9af508803..ae440fef2 100644 --- a/drivers/net/thunderx/nicvf_ethdev.h +++ b/drivers/net/thunderx/nicvf_ethdev.h @@ -38,6 +38,7 @@ DEV_TX_OFFLOAD_MULTI_SEGS) #define NICVF_RX_OFFLOAD_CAPA ( \ + DEV_RX_OFFLOAD_CHECKSUM | \ DEV_RX_OFFLOAD_VLAN_STRIP | \ DEV_RX_OFFLOAD_CRC_STRIP | \ DEV_RX_OFFLOAD_JUMBO_FRAME | \ diff --git a/drivers/net/thunderx/nicvf_rxtx.c b/drivers/net/thunderx/nicvf_rxtx.c index 72305d9d2..fa4ee824a 100644 --- a/drivers/net/thunderx/nicvf_rxtx.c +++ b/drivers/net/thunderx/nicvf_rxtx.c @@ -327,6 +327,20 @@ nicvf_rx_classify_pkt(cqe_rx_word0_t cqe_rx_w0) return ptype_table[cqe_rx_w0.l3_type][cqe_rx_w0.l4_type]; } +static inline uint64_t __hot +nicvf_set_olflags(const cqe_rx_word0_t cqe_rx_w0) +{ + static const uint64_t flag_table[3] __rte_cache_aligned = { + PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD, + PKT_RX_IP_CKSUM_BAD | PKT_RX_L4_CKSUM_UNKNOWN, + PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD, + }; + + const uint8_t idx = (cqe_rx_w0.err_opcode == CQE_RX_ERR_L4_CHK) << 1 | + (cqe_rx_w0.err_opcode == CQE_RX_ERR_IP_CHK); + return flag_table[idx]; +} + static inline int __hot nicvf_fill_rbdr(struct nicvf_rxq *rxq, int to_fill) { @@ -385,11 +399,13 @@ nicvf_rx_offload(cqe_rx_word0_t cqe_rx_w0, cqe_rx_word2_t cqe_rx_w2, if (likely(cqe_rx_w0.rss_alg)) { pkt->hash.rss = cqe_rx_w2.rss_tag; pkt->ol_flags |= PKT_RX_RSS_HASH; + } } -uint16_t __hot -nicvf_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) +static __rte_always_inline uint16_t +nicvf_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts, + const uint32_t flag) { uint32_t i, to_process; struct cqe_rx_t *cqe_rx; @@ -420,7 +436,11 @@ nicvf_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rb0_ptr = *((uint64_t *)cqe_rx + rbptr_offset); pkt = (struct rte_mbuf *)nicvf_mbuff_phy2virt (rb0_ptr - cqe_rx_w1.align_pad, mbuf_phys_off); - pkt->ol_flags = 0; + + if (flag & NICVF_RX_OFFLOAD_NONE) + pkt->ol_flags = 0; + if (flag & NICVF_RX_OFFLOAD_CKSUM) + pkt->ol_flags = nicvf_set_olflags(cqe_rx_w0); pkt->data_len = cqe_rx_w3.rb0_sz; pkt->pkt_len = cqe_rx_w3.rb0_sz; pkt->packet_type = nicvf_rx_classify_pkt(cqe_rx_w0); @@ -445,11 +465,27 @@ nicvf_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) return to_process; } -static inline uint16_t __hot +uint16_t __hot +nicvf_recv_pkts_no_offload(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + return nicvf_recv_pkts(rx_queue, rx_pkts, nb_pkts, + NICVF_RX_OFFLOAD_NONE); +} + +uint16_t __hot +nicvf_recv_pkts_cksum(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + return nicvf_recv_pkts(rx_queue, rx_pkts, nb_pkts, + NICVF_RX_OFFLOAD_CKSUM); +} + +static __rte_always_inline uint16_t __hot nicvf_process_cq_mseg_entry(struct cqe_rx_t *cqe_rx, uint64_t mbuf_phys_off, struct rte_mbuf **rx_pkt, uint8_t rbptr_offset, - uint64_t mbuf_init) + uint64_t mbuf_init, const uint32_t flag) { struct rte_mbuf *pkt, *seg, *prev; cqe_rx_word0_t cqe_rx_w0; @@ -467,12 +503,15 @@ nicvf_process_cq_mseg_entry(struct cqe_rx_t *cqe_rx, pkt = (struct rte_mbuf *)nicvf_mbuff_phy2virt (rb_ptr[0] - cqe_rx_w1.align_pad, mbuf_phys_off); - pkt->ol_flags = 0; pkt->pkt_len = cqe_rx_w1.pkt_len; pkt->data_len = rb_sz[nicvf_frag_num(0)]; nicvf_mbuff_init_mseg_update( pkt, mbuf_init, cqe_rx_w1.align_pad, nb_segs); pkt->packet_type = nicvf_rx_classify_pkt(cqe_rx_w0); + if (flag & NICVF_RX_OFFLOAD_NONE) + pkt->ol_flags = 0; + if (flag & NICVF_RX_OFFLOAD_CKSUM) + pkt->ol_flags = nicvf_set_olflags(cqe_rx_w0); nicvf_rx_offload(cqe_rx_w0, cqe_rx_w2, pkt); *rx_pkt = pkt; @@ -491,9 +530,9 @@ nicvf_process_cq_mseg_entry(struct cqe_rx_t *cqe_rx, return nb_segs; } -uint16_t __hot +static __rte_always_inline uint16_t __hot nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, - uint16_t nb_pkts) + uint16_t nb_pkts, const uint32_t flag) { union cq_entry_t *cq_entry; struct cqe_rx_t *cqe_rx; @@ -515,7 +554,7 @@ nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, cq_entry = &desc[cqe_head]; cqe_rx = (struct cqe_rx_t *)cq_entry; nb_segs = nicvf_process_cq_mseg_entry(cqe_rx, mbuf_phys_off, - rx_pkts + i, rbptr_offset, mbuf_init); + rx_pkts + i, rbptr_offset, mbuf_init, flag); buffers_consumed += nb_segs; cqe_head = (cqe_head + 1) & cqe_mask; nicvf_prefetch_store_keep(rx_pkts[i]); @@ -535,6 +574,22 @@ nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, return to_process; } +uint16_t __hot +nicvf_recv_pkts_multiseg_no_offload(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + return nicvf_recv_pkts_multiseg(rx_queue, rx_pkts, nb_pkts, + NICVF_RX_OFFLOAD_NONE); +} + +uint16_t __hot +nicvf_recv_pkts_multiseg_cksum(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + return nicvf_recv_pkts_multiseg(rx_queue, rx_pkts, nb_pkts, + NICVF_RX_OFFLOAD_CKSUM); +} + uint32_t nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx) { diff --git a/drivers/net/thunderx/nicvf_rxtx.h b/drivers/net/thunderx/nicvf_rxtx.h index 8bdd582ed..72daffb60 100644 --- a/drivers/net/thunderx/nicvf_rxtx.h +++ b/drivers/net/thunderx/nicvf_rxtx.h @@ -8,6 +8,9 @@ #include #include +#define NICVF_RX_OFFLOAD_NONE 0x1 +#define NICVF_RX_OFFLOAD_CKSUM 0x2 + #define NICVF_TX_OFFLOAD_MASK (PKT_TX_IP_CKSUM | PKT_TX_L4_MASK) #ifndef __hot @@ -86,9 +89,15 @@ nicvf_mbuff_init_mseg_update(struct rte_mbuf *pkt, const uint64_t mbuf_init, uint32_t nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx); uint32_t nicvf_dev_rbdr_refill(struct rte_eth_dev *dev, uint16_t queue_idx); -uint16_t nicvf_recv_pkts(void *rxq, struct rte_mbuf **rx_pkts, uint16_t pkts); -uint16_t nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, - uint16_t nb_pkts); +uint16_t nicvf_recv_pkts_no_offload(void *rxq, struct rte_mbuf **rx_pkts, + uint16_t pkts); +uint16_t nicvf_recv_pkts_cksum(void *rxq, struct rte_mbuf **rx_pkts, + uint16_t pkts); + +uint16_t nicvf_recv_pkts_multiseg_no_offload(void *rx_queue, + struct rte_mbuf **rx_pkts, uint16_t nb_pkts); +uint16_t nicvf_recv_pkts_multiseg_cksum(void *rx_queue, + struct rte_mbuf **rx_pkts, uint16_t nb_pkts); uint16_t nicvf_xmit_pkts(void *txq, struct rte_mbuf **tx_pkts, uint16_t pkts); uint16_t nicvf_xmit_pkts_multiseg(void *txq, struct rte_mbuf **tx_pkts, diff --git a/drivers/net/thunderx/nicvf_struct.h b/drivers/net/thunderx/nicvf_struct.h index cf98f7c1a..a770e6b54 100644 --- a/drivers/net/thunderx/nicvf_struct.h +++ b/drivers/net/thunderx/nicvf_struct.h @@ -55,25 +55,27 @@ union mbuf_initializer { }; struct nicvf_rxq { + MARKER rxq_fastpath_data_start; + uint8_t rbptr_offset; + uint16_t rx_free_thresh; + uint32_t head; + uint32_t qlen_mask; + int32_t recv_buffers; + int32_t available_space; uint64_t mbuf_phys_off; uintptr_t cq_status; uintptr_t cq_door; - union mbuf_initializer mbuf_initializer; - nicvf_iova_addr_t phys; - union cq_entry_t *desc; struct nicvf_rbdr *shared_rbdr; - struct nicvf *nic; struct rte_mempool *pool; - uint32_t head; - uint32_t qlen_mask; - int32_t available_space; - int32_t recv_buffers; - uint16_t rx_free_thresh; - uint16_t queue_id; - uint16_t precharge_cnt; + union cq_entry_t *desc; + union mbuf_initializer mbuf_initializer; + MARKER rxq_fastpath_data_end; uint8_t rx_drop_en; + uint16_t precharge_cnt; uint16_t port_id; - uint8_t rbptr_offset; + uint16_t queue_id; + struct nicvf *nic; + nicvf_iova_addr_t phys; } __rte_cache_aligned; struct nicvf { @@ -85,6 +87,7 @@ struct nicvf { bool loopback_supported; bool pf_acked:1; bool pf_nacked:1; + bool offload_cksum:1; uint64_t hwcap; uint8_t link_up; uint8_t duplex;