From patchwork Sat Sep 29 10:30:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Igor Russkikh X-Patchwork-Id: 45674 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 57F441B205; Sat, 29 Sep 2018 12:32:01 +0200 (CEST) Received: from NAM05-DM3-obe.outbound.protection.outlook.com (mail-eopbgr730046.outbound.protection.outlook.com [40.107.73.46]) by dpdk.org (Postfix) with ESMTP id CE1291B1D3 for ; Sat, 29 Sep 2018 12:31:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=AQUANTIA1COM.onmicrosoft.com; s=selector1-aquantia-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+WFaZP/A4N5LIhjUjciyIiZPGZ5/MNISr4LF1bxgPpE=; b=g5YN12WgqZxGNvo1ByN/xZi287X4jstMBwNfESk0xbZBibH/ZiRoS519p/wSPEy8+/7iHtG2UQyT74Nt4OIqpwjSqk8azZzB0wwbiep/TUmQHdYkEVWbyrKuyDLxjQaKkp/rGF0Q18HKkpx7NVCA5Lj0O4npuVZofZNq5imfJRE= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Igor.Russkikh@aquantia.com; Received: from ubuntubox.rdc.aquantia.com (95.79.108.179) by BLUPR0701MB1650.namprd07.prod.outlook.com (2a01:111:e400:58c6::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1185.22; Sat, 29 Sep 2018 10:31:46 +0000 From: Igor Russkikh To: dev@dpdk.org Cc: pavel.belous@aquantia.com, igor.russkikh@aquantia.com, Pavel Belous Date: Sat, 29 Sep 2018 13:30:24 +0300 Message-Id: <88d51a926b48f55695fab80c120913ee88bd6122.1538215990.git.igor.russkikh@aquantia.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [95.79.108.179] X-ClientProxiedBy: VI1P193CA0009.EURP193.PROD.OUTLOOK.COM (2603:10a6:800:bd::19) To BLUPR0701MB1650.namprd07.prod.outlook.com (2a01:111:e400:58c6::20) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 18e177b1-43cb-4fa1-1df6-08d625f6c563 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534165)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:BLUPR0701MB1650; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1650; 3:epc27eaxoOF69xNWo+3aYEziMkDLesxCuW96O4bAfIUU89vSBuupkHhhfDBV7/Q3AuGLLhEgIBSXcHUyjGGzdPpD4knLv8mIGrhl5bZvdqh90Erw+/neixKB2QyxlxXRj0q9J0yi3dlRqmSsbfO6w5q0x3XDNkd5xQ3ewxiRTISQDiYcEj59YwCiGnK2fPqNMskBkENTdu9YmMf0oBkCiHZwI4/TixWO5++9vC4KOtljgLGyyi4ZN36c8khCE3Qd; 25:jGxmGj/gXNDQMOjKvY8yfc6DPWC2R3QNgsHedzJgllCcOajSI+waZTrx8lRD74qfFPQrJHfKg4cgl+CwFgpltJXn5j4h19epIWnt+xJrdLWuddE0k0BxQlBLnsjFwisRzoB12G3+WUtE/dqq0nAPJ5vHRnEOIhHwXqP2E3F21uvFyJsNS4mRj0UQMU3dGMqkXU9yo8H0nUQlJMzdYKPWGkHydsX5j4y40a8AwUoSX1vVfwpEb8TLO89WNvpeM14dspTCHDM6E3I+4k05FkRg3TDhYE2yxmzKOKhBU8kKMh1AVJCgwrPGqYKprC4mzsRYCKmb0+UWgP0XeLkYWfibKA==; 31:ztZ/Gz2ThgUkI4/BL/jXJimy3PidX6Uq3rPNHbxQAMz/un6wrgl0lWwrrGiKjznQhL2OFhNndJy4lFepNerEEhfBaLNVEY0uNGZblFzW178ePYh7UdhH84B/WTDBJH4lN3k2WnKKTtgk+8muYZT6eQl6tkRDIzUQfugEKHPJZDlURxlLZRqFaLmQ2Gx7iZDYjuZ3mO4prYSSkhMG/2x0isfn1PxEF7zCloLoaJI4mn0= X-MS-TrafficTypeDiagnostic: BLUPR0701MB1650: X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1650; 20:eDp4D7M691F0GADr7hU/uWwD2kX2EfDsqQduQwvnlfYRKGY7HeaUf3Ax/2lXuxs4gbBDfmTNcduKYUnhFU6BEsAHezROS3GsM4Enw5pCa7ovspZVvJrSeoPA2BM14JfcKRdSSUsy+op9+3dfI+iaDH80v1D9GeCwSrVjwUuqjj+p8DBe5GJb/aLtHSD3MRz1fVEu2IJOw2bYu6zPv6w70AJYOv1cfxWFSm9e4PCQIhHr60RnPwpDGAQrxKCjkIG33sM5WcDGL8sQOLSE+uIROsC7UpNbO+FSYYIFV2aOGtg/PFBPg4OGpdWdlu9jtYMwsOGBIrMxUKWBHT8jhaJEi0H0nsFikvYP//Qchr1rAYyIJgHB68LWEFADDZrMX7qvniAqMcj9bLZH9U0A7+S+/MmzvI1lliITNFBLTvNBTZutFzBimzde/LGx3AAoSa1HwyL7hfoT7m03CvbECOLU1brgL3TsTvmLLN1d7yLbFU51MXKyhYyRFyz4q+qGABDV; 4:yx3ujsunvMbos8Bm/zG88cXtlLqHNxBUCz0YLgR8V5R8WvHtKHLB25fy/H0K/trf6qAiQsW4WMENsImsS+TAI7ho5DYcMoUR6OTEceaTozGPKJGK/SKe3t5mKkBoW61H2fTAgfDBeDRPA15SKeKm3qcA0p2cnx6W28cL+FJtLEjF0G/hAuZChCphCX4xdnwKOZgNrXHsBz+t+e1f/2C6qZC1FkqZ4a3yG5B7kL1bIwTdqa10wLrOMHUxWupFmtxrOfiwtnlLJPbHjL8LU7O5+w== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(3231355)(944501410)(52105095)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123562045)(20161123560045)(201708071742011)(7699051); SRVR:BLUPR0701MB1650; BCL:0; PCL:0; RULEID:; SRVR:BLUPR0701MB1650; X-Forefront-PRVS: 0810818DA0 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(346002)(136003)(396003)(376002)(39850400004)(366004)(189003)(199004)(16526019)(14444005)(446003)(76176011)(2906002)(6916009)(6666003)(97736004)(8676002)(81166006)(81156014)(36756003)(4326008)(316002)(8936002)(50226002)(34290500001)(86362001)(5660300001)(68736007)(16586007)(118296001)(72206003)(3846002)(6116002)(53936002)(386003)(105586002)(6486002)(47776003)(66066001)(486006)(25786009)(305945005)(50466002)(44832011)(956004)(2616005)(26005)(107886003)(52116002)(51416003)(7696005)(186003)(476003)(478600001)(2361001)(7736002)(2351001)(48376002)(106356001)(11346002)(309714004); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR0701MB1650; H:ubuntubox.rdc.aquantia.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: aquantia.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BLUPR0701MB1650; 23:J08d+tSdiJwTd+hpQJ/19D7Lf/bxkVpPkmVaix6?= 43EpI2s4jHy4dBkGPrIUW6T0cMEUwrKVr5h1i08Il9TCmT816KdUdqW+y7p72NUJcVmzDuLQMNlXtLTE8fhnE+WSKW9rwAZekQxV2T43v0wRLh4O52h//MEF1x8NPlbpn6KeFfecXZghU2B8hQhQ+ub899TuzWbAYSY/Iy/x+anSy1G73Y8k/SK3TygFlAs5bobsEWNsyDpjGYVDZ7vj08fYbCB3MM+brta0ruuldwvV9tHhykAa+mMcdGA/G/7pEWizFU4gLaZyiREZsAuhAtlojm1g0aPGnVPVXFfKqmJOAFiR6CooIqtdb8+Bpo2KHryLzu6RbCZfmYGL8NME8U/XOeo3YsikNaZ11WWLKtWr9yYu/UYtuh5Td3qMjfgXI/31bd6slidySmO4JssG/oNNgczRVMktnPLEhfdMXYUs5BW1S77BBI8Nfv6HkvBdm2Zg67vRrhhnnuywD+r3fTn0GIkQ8zqETq+gA4GYloDe6tFtYXD8ut4SJZNAUTfoDvwM+AeZ+NWB3PXnnlyCNxDX7abDCxkP7l7R+m86VVRRP+WASNpkfBg0SfMHlkmE5BfykR36h69i60pUWlkNLQZX9Q+iP9mbEi3XvLJWmSIMuHYEsAxdpKHv/SdP/RbksDebCQ/q/TsoQwDpjvHfItmM5tvqHhGSW54sU93oAiJ9TtWgvO555pI8NxJwz5uL0kBkcgt/L4FAKvyfJyMpyK42I1+XfX3XrLduFR65ljRRdu81DK/9ojtWVKOlBNysOMOYUSej7mVm8rp3j8J106KvOrtotvDllnnALjufax5Ko+nAhD/2w2meLz9daW2ItUdV0iOKivJXvz0ah0Iejus/B9o0qrQo2tF/8+kSVDD7PH1t+MHE7Vsv8jkwQ0easHNwfH9hm9PIhnyncYJwPNWNpg58dd8o2yS+F/KMwEerE7Q817kwyOEulJgylvgLZgeg1DwYU2soU7cLVmOGMfGLMzPf+OrYtXvpNMIheKGjJqEX+MiddCgjF8Hizh4OqsijSYBfHJ1FT+XhBRGHk4CwpwdbmXiy8VqJj/q82JxAR+rVynw21tJj19Ri9UkVg4Uc+OjxZL7N6jThosPzRw8WoaGhroRSEJbld1T87GLZGVU/Y2vgM5u/4/DFdURFeU8zq+x1yUCuDVFxC/gYnTAli7pRubuxrmRmYfY5jBuEB6qSanYVbp/MKoME2ceiyQ/2r/y5Fu8lGgIFRXhvHJhfg3ubEwdU0+Vr9W5kstL0MBHsa+IJqKPdmRdCy7wiV6EPwfX6H9Ub0n3wSC4cHTHel X-Microsoft-Antispam-Message-Info: RUSnC8F9p/Wc1YWGPrs01nh+rhJsXZ1DDASrbE+Y8TKR+R0bZDw1iGDcrIL2NDeNG/7cs0V8dJ6Axu1z5KjKAbyzC0l6X2LkKkMqziLAbVWY318dn6OSAUYPkAFvkGLPhnboSznqZKNCu0IJYkbUv75jQn0aF9GeT+J73RL2qb4lX4YS3qTH0w2MC0osDwWIvYmPNgALxobNPn113uH9Sll/5N+QGw/AS8CoiEnPMknHSYzeESdCelanaAXgtJ4Yx4OAUks+O0CaWcyeZKSBAO9HRq5CdRLmVmUC42yQUsPLqyaKqwI/eG04IoM5nAO3oZ8LBJPEPljKWnMqRLkS1ICNVH5gjVdTNEHaN1YDfVQ= X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1650; 6:I42WE1SW+rdwwtgBadqgjYEIJ7UyOrA0hf4tKRjhIkQRBrK+HDJzwCHxhB+GkyyFSXvh3lt3TqUrFa4gVtUErvjxo2Grya42mxjneHL37+EA/9tr8YPTxHi8xFECIFjKAaT4Cb4RY8wdQzifCwJQsdzkEt4zGnsaHFR8LILVObxj2KpkebFmETBpo5cr7NCRrnPYrlVr3I4m12uM4hRTkyuiqT9aykdLRHfHl+1jW4ISZOSqK/+4QDzt7TnyM/mTSgH/XIAwHYbWtooDZ+9Zluvr2Q54mhfG4+1/OKQfIQOa4YBhXCSsdrgkYeyjwKL74buh6xsKAqKOHfbasvzRGrr9gR5SyOuCXGnZeAoaKmVlsH7+/6g6TvwjZ6A7Zhph75s0uXy1LnKGWXheQUXes7Q7ZktWLqqLA4Nb3yQqq5IRIFmSVHF1oJ1Vv+4imuGVh/OiNpA0rhk4y0U41dOaMw==; 5:0kDyNzxyXaxsJTFYR57SiKs4Iuqy2oDSV8UwPXlhF2Iiq7vtvpYC5cK+a42AzU+3T4Wtk6DaHAKF83YZTWeKRTIofwyT2Ri/YlXJgtMwCfVv+U75szc9tG7unbZQSLs0sCWasOJjnsIk5QWYCL3WOUEe6FLluBCBEKScdZa6W7E=; 7:Gs7z2uD7deyjBanEQjMHXse61ml3KWFiRNcHtNLrqRUHB+hYN6FQrfeSOhSf/8vEtjjuwZtrA9HXLmpOG2kDX3mEaRXIov26133lOXEeKCwdcGO5cYKPKsl/f5YA22+qeNnB96Qz3kzdRVaq0HDMU62qsZF0XFnj4VZ94gzQQPnJ7kBR361SwNfDXa3sDSQTMZl8H38anDjvjOxcZVP1XmwVr6ma8bEQj8K/5ONYUGj644knSMcu1laRP60sXxnO SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: aquantia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2018 10:31:46.2509 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 18e177b1-43cb-4fa1-1df6-08d625f6c563 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 83e2e134-991c-4ede-8ced-34d47e38e6b1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0701MB1650 Subject: [dpdk-dev] [PATCH v3 10/22] net/atlantic: TX side structures and implementation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavel Belous Signed-off-by: Igor Russkikh Signed-off-by: Pavel Belous --- drivers/net/atlantic/atl_ethdev.c | 28 ++ drivers/net/atlantic/atl_ethdev.h | 7 + drivers/net/atlantic/atl_rxtx.c | 530 +++++++++++++++++++++++++++++++++++++- 3 files changed, 556 insertions(+), 9 deletions(-) diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c index cdcfc5ec02c2..6a00277c3c8e 100644 --- a/drivers/net/atlantic/atl_ethdev.c +++ b/drivers/net/atlantic/atl_ethdev.c @@ -79,12 +79,27 @@ static struct rte_pci_driver rte_atl_pmd = { | DEV_RX_OFFLOAD_TCP_CKSUM \ | DEV_RX_OFFLOAD_JUMBO_FRAME) +#define ATL_TX_OFFLOADS (DEV_TX_OFFLOAD_VLAN_INSERT \ + | DEV_TX_OFFLOAD_IPV4_CKSUM \ + | DEV_TX_OFFLOAD_UDP_CKSUM \ + | DEV_TX_OFFLOAD_TCP_CKSUM \ + | DEV_TX_OFFLOAD_TCP_TSO \ + | DEV_TX_OFFLOAD_MULTI_SEGS) + static const struct rte_eth_desc_lim rx_desc_lim = { .nb_max = ATL_MAX_RING_DESC, .nb_min = ATL_MIN_RING_DESC, .nb_align = ATL_RXD_ALIGN, }; +static const struct rte_eth_desc_lim tx_desc_lim = { + .nb_max = ATL_MAX_RING_DESC, + .nb_min = ATL_MIN_RING_DESC, + .nb_align = ATL_TXD_ALIGN, + .nb_seg_max = ATL_TX_MAX_SEG, + .nb_mtu_seg_max = ATL_TX_MAX_SEG, +}; + static const struct eth_dev_ops atl_eth_dev_ops = { .dev_configure = atl_dev_configure, .dev_start = atl_dev_start, @@ -101,6 +116,11 @@ static const struct eth_dev_ops atl_eth_dev_ops = { .rx_queue_stop = atl_rx_queue_stop, .rx_queue_setup = atl_rx_queue_setup, .rx_queue_release = atl_rx_queue_release, + + .tx_queue_start = atl_tx_queue_start, + .tx_queue_stop = atl_tx_queue_stop, + .tx_queue_setup = atl_tx_queue_setup, + .tx_queue_release = atl_tx_queue_release, }; static inline int32_t @@ -369,11 +389,19 @@ atl_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->rx_offload_capa = ATL_RX_OFFLOADS; + dev_info->tx_offload_capa = ATL_TX_OFFLOADS; + + dev_info->default_rxconf = (struct rte_eth_rxconf) { .rx_free_thresh = ATL_DEFAULT_RX_FREE_THRESH, }; + dev_info->default_txconf = (struct rte_eth_txconf) { + .tx_free_thresh = ATL_DEFAULT_TX_FREE_THRESH, + }; + dev_info->rx_desc_lim = rx_desc_lim; + dev_info->tx_desc_lim = tx_desc_lim; } static const uint32_t * diff --git a/drivers/net/atlantic/atl_ethdev.h b/drivers/net/atlantic/atl_ethdev.h index acb3066c189f..cafe37cdf963 100644 --- a/drivers/net/atlantic/atl_ethdev.h +++ b/drivers/net/atlantic/atl_ethdev.h @@ -28,12 +28,17 @@ struct atl_adapter { * RX/TX function prototypes */ void atl_rx_queue_release(void *rxq); +void atl_tx_queue_release(void *txq); int atl_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, uint16_t nb_rx_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mb_pool); +int atl_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, + uint16_t nb_tx_desc, unsigned int socket_id, + const struct rte_eth_txconf *tx_conf); + int atl_rx_init(struct rte_eth_dev *dev); int atl_tx_init(struct rte_eth_dev *dev); @@ -44,6 +49,8 @@ void atl_free_queues(struct rte_eth_dev *dev); int atl_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id); int atl_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id); +int atl_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id); +int atl_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id); uint16_t atl_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); diff --git a/drivers/net/atlantic/atl_rxtx.c b/drivers/net/atlantic/atl_rxtx.c index 0ce41aaf1780..172d5fb232f2 100644 --- a/drivers/net/atlantic/atl_rxtx.c +++ b/drivers/net/atlantic/atl_rxtx.c @@ -4,6 +4,7 @@ #include #include +#include #include "atl_ethdev.h" #include "atl_hw_regs.h" @@ -13,6 +14,20 @@ #include "hw_atl/hw_atl_b0.h" #include "hw_atl/hw_atl_b0_internal.h" +#define ATL_TX_CKSUM_OFFLOAD_MASK ( \ + PKT_TX_IP_CKSUM | \ + PKT_TX_L4_MASK | \ + PKT_TX_TCP_SEG) + +#define ATL_TX_OFFLOAD_MASK ( \ + PKT_TX_VLAN_PKT | \ + PKT_TX_IP_CKSUM | \ + PKT_TX_L4_MASK | \ + PKT_TX_TCP_SEG) + +#define ATL_TX_OFFLOAD_NOTSUP_MASK \ + (PKT_TX_OFFLOAD_MASK ^ ATL_TX_OFFLOAD_MASK) + /** * Structure associated with each descriptor of the RX ring of a RX queue. */ @@ -21,6 +36,15 @@ struct atl_rx_entry { }; /** + * Structure associated with each descriptor of the TX ring of a TX queue. + */ +struct atl_tx_entry { + struct rte_mbuf *mbuf; + uint16_t next_id; + uint16_t last_id; +}; + +/** * Structure associated with each RX queue. */ struct atl_rx_queue { @@ -39,6 +63,22 @@ struct atl_rx_queue { bool l4_csum_enabled; }; +/** + * Structure associated with each TX queue. + */ +struct atl_tx_queue { + struct hw_atl_txd_s *hw_ring; + uint64_t hw_ring_phys_addr; + struct atl_tx_entry *sw_ring; + uint16_t nb_tx_desc; + uint16_t tx_tail; + uint16_t tx_head; + uint16_t queue_id; + uint16_t port_id; + uint16_t tx_free_thresh; + uint16_t tx_free; +}; + static inline void atl_reset_rx_queue(struct atl_rx_queue *rxq) { @@ -143,13 +183,141 @@ atl_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, return 0; } +static inline void +atl_reset_tx_queue(struct atl_tx_queue *txq) +{ + struct atl_tx_entry *tx_entry; + union hw_atl_txc_s *txc; + uint16_t i; + + PMD_INIT_FUNC_TRACE(); + + if (!txq) { + PMD_DRV_LOG(ERR, "Pointer to txq is NULL"); + return; + } + + tx_entry = txq->sw_ring; + + for (i = 0; i < txq->nb_tx_desc; i++) { + txc = (union hw_atl_txc_s *)&txq->hw_ring[i]; + txc->flags1 = 0; + txc->flags2 = 2; + } + + for (i = 0; i < txq->nb_tx_desc; i++) { + txq->hw_ring[i].dd = 1; + tx_entry[i].mbuf = NULL; + } + + txq->tx_tail = 0; + txq->tx_head = 0; + txq->tx_free = txq->nb_tx_desc - 1; +} + int -atl_tx_init(struct rte_eth_dev *eth_dev __rte_unused) +atl_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, + uint16_t nb_tx_desc, unsigned int socket_id, + const struct rte_eth_txconf *tx_conf) { + struct atl_tx_queue *txq; + const struct rte_memzone *mz; + + PMD_INIT_FUNC_TRACE(); + + /* make sure a valid number of descriptors have been requested */ + if (nb_tx_desc < AQ_HW_MIN_TX_RING_SIZE || + nb_tx_desc > AQ_HW_MAX_TX_RING_SIZE) { + PMD_INIT_LOG(ERR, "Number of Tx descriptors must be " + "less than or equal to %d, " + "greater than or equal to %d", AQ_HW_MAX_TX_RING_SIZE, + AQ_HW_MIN_TX_RING_SIZE); + return -EINVAL; + } + + /* + * if this queue existed already, free the associated memory. The + * queue cannot be reused in case we need to allocate memory on + * different socket than was previously used. + */ + if (dev->data->tx_queues[tx_queue_id] != NULL) { + atl_tx_queue_release(dev->data->tx_queues[tx_queue_id]); + dev->data->tx_queues[tx_queue_id] = NULL; + } + + /* allocate memory for the queue structure */ + txq = rte_zmalloc_socket("atlantic Tx queue", sizeof(*txq), + RTE_CACHE_LINE_SIZE, socket_id); + if (txq == NULL) { + PMD_INIT_LOG(ERR, "Cannot allocate queue structure"); + return -ENOMEM; + } + + /* setup queue */ + txq->nb_tx_desc = nb_tx_desc; + txq->port_id = dev->data->port_id; + txq->queue_id = tx_queue_id; + txq->tx_free_thresh = tx_conf->tx_free_thresh; + + + /* allocate memory for the software ring */ + txq->sw_ring = rte_zmalloc_socket("atlantic sw tx ring", + nb_tx_desc * sizeof(struct atl_tx_entry), + RTE_CACHE_LINE_SIZE, socket_id); + if (txq->sw_ring == NULL) { + PMD_INIT_LOG(ERR, "Cannot allocate software ring"); + rte_free(txq); + return -ENOMEM; + } + + /* + * allocate memory for the hardware descriptor ring. A memzone large + * enough to hold the maximum ring size is requested to allow for + * resizing in later calls to the queue setup function. + */ + mz = rte_eth_dma_zone_reserve(dev, "tx hw_ring", tx_queue_id, + HW_ATL_B0_MAX_TXD * sizeof(struct hw_atl_txd_s), + 128, socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Cannot allocate hardware ring"); + rte_free(txq->sw_ring); + rte_free(txq); + return -ENOMEM; + } + txq->hw_ring = mz->addr; + txq->hw_ring_phys_addr = mz->iova; + + atl_reset_tx_queue(txq); + + dev->data->tx_queues[tx_queue_id] = txq; return 0; } int +atl_tx_init(struct rte_eth_dev *eth_dev) +{ + struct aq_hw_s *hw = ATL_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); + struct atl_tx_queue *txq; + uint64_t base_addr = 0; + int i = 0; + int err = 0; + + PMD_INIT_FUNC_TRACE(); + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + txq = eth_dev->data->tx_queues[i]; + base_addr = txq->hw_ring_phys_addr; + + err = hw_atl_b0_hw_ring_tx_init(hw, base_addr, + txq->queue_id, + txq->nb_tx_desc, 0, + txq->port_id); + } + + return err; +} + +int atl_rx_init(struct rte_eth_dev *eth_dev) { struct aq_hw_s *hw = ATL_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); @@ -313,15 +481,78 @@ atl_rx_queue_release(void *rx_queue) } } -uint16_t -atl_prep_pkts(void *tx_queue __rte_unused, - struct rte_mbuf **tx_pkts __rte_unused, - uint16_t nb_pkts __rte_unused) +static void +atl_tx_queue_release_mbufs(struct atl_tx_queue *txq) { + int i; + + PMD_INIT_FUNC_TRACE(); + + if (txq->sw_ring != NULL) { + for (i = 0; i < txq->nb_tx_desc; i++) { + if (txq->sw_ring[i].mbuf != NULL) { + rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf); + txq->sw_ring[i].mbuf = NULL; + } + } + } +} + +int +atl_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id) +{ + struct aq_hw_s *hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private); + + PMD_INIT_FUNC_TRACE(); + + if (tx_queue_id < dev->data->nb_tx_queues) { + hw_atl_b0_hw_ring_tx_start(hw, tx_queue_id); + + rte_wmb(); + hw_atl_b0_hw_tx_ring_tail_update(hw, 0, tx_queue_id); + dev->data->tx_queue_state[tx_queue_id] = + RTE_ETH_QUEUE_STATE_STARTED; + } else { + return -1; + } + + return 0; +} + +int +atl_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id) +{ + struct aq_hw_s *hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private); + struct atl_tx_queue *txq; + + PMD_INIT_FUNC_TRACE(); + + txq = dev->data->tx_queues[tx_queue_id]; + + hw_atl_b0_hw_ring_tx_stop(hw, tx_queue_id); + + atl_tx_queue_release_mbufs(txq); + atl_reset_tx_queue(txq); + dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED; + return 0; } void +atl_tx_queue_release(void *tx_queue) +{ + PMD_INIT_FUNC_TRACE(); + + if (tx_queue != NULL) { + struct atl_tx_queue *txq = (struct atl_tx_queue *)tx_queue; + + atl_tx_queue_release_mbufs(txq); + rte_free(txq->sw_ring); + rte_free(txq); + } +} + +void atl_free_queues(struct rte_eth_dev *dev) { unsigned int i; @@ -333,6 +564,12 @@ atl_free_queues(struct rte_eth_dev *dev) dev->data->rx_queues[i] = 0; } dev->data->nb_rx_queues = 0; + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + atl_tx_queue_release(dev->data->tx_queues[i]); + dev->data->tx_queues[i] = 0; + } + dev->data->nb_tx_queues = 0; } int @@ -342,6 +579,13 @@ atl_start_queues(struct rte_eth_dev *dev) PMD_INIT_FUNC_TRACE(); + for (i = 0; i < dev->data->nb_tx_queues; i++) { + if (atl_tx_queue_start(dev, i) != 0) { + PMD_DRV_LOG(ERR, "Start Tx queue %d failed", i); + return -1; + } + } + for (i = 0; i < dev->data->nb_rx_queues; i++) { if (atl_rx_queue_start(dev, i) != 0) { PMD_DRV_LOG(ERR, "Start Rx queue %d failed", i); @@ -359,6 +603,13 @@ atl_stop_queues(struct rte_eth_dev *dev) PMD_INIT_FUNC_TRACE(); + for (i = 0; i < dev->data->nb_tx_queues; i++) { + if (atl_tx_queue_stop(dev, i) != 0) { + PMD_DRV_LOG(ERR, "Stop Tx queue %d failed", i); + return -1; + } + } + for (i = 0; i < dev->data->nb_rx_queues; i++) { if (atl_rx_queue_stop(dev, i) != 0) { PMD_DRV_LOG(ERR, "Stop Rx queue %d failed", i); @@ -369,6 +620,47 @@ atl_stop_queues(struct rte_eth_dev *dev) return 0; } +uint16_t +atl_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + int i, ret; + uint64_t ol_flags; + struct rte_mbuf *m; + + PMD_INIT_FUNC_TRACE(); + + for (i = 0; i < nb_pkts; i++) { + m = tx_pkts[i]; + ol_flags = m->ol_flags; + + if (m->nb_segs > AQ_HW_MAX_SEGS_SIZE) { + rte_errno = -EINVAL; + return i; + } + + if (ol_flags & ATL_TX_OFFLOAD_NOTSUP_MASK) { + rte_errno = -ENOTSUP; + return i; + } + +#ifdef RTE_LIBRTE_ETHDEV_DEBUG + ret = rte_validate_tx_offload(m); + if (ret != 0) { + rte_errno = ret; + return i; + } +#endif + ret = rte_net_intel_cksum_prepare(m); + if (ret != 0) { + rte_errno = ret; + return i; + } + } + + return i; +} + static uint64_t atl_desc_to_offload_flags(struct atl_rx_queue *rxq, struct hw_atl_rxd_wb_s *rxd_wb) @@ -637,12 +929,232 @@ atl_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) return nb_rx; } +static void +atl_xmit_cleanup(struct atl_tx_queue *txq) +{ + struct atl_tx_entry *sw_ring; + struct hw_atl_txd_s *txd; + int to_clean = 0; + + PMD_INIT_FUNC_TRACE(); + + if (txq != NULL) { + sw_ring = txq->sw_ring; + int head = txq->tx_head; + int cnt; + int i; + + for (i = 0, cnt = head; ; i++) { + txd = &txq->hw_ring[cnt]; + + if (txd->dd) + to_clean++; + + cnt = (cnt + 1) % txq->nb_tx_desc; + if (cnt == txq->tx_tail) + break; + } + + if (to_clean == 0) + return; + + while (to_clean) { + txd = &txq->hw_ring[head]; + + struct atl_tx_entry *rx_entry = &sw_ring[head]; + + if (rx_entry->mbuf) { + rte_pktmbuf_free_seg(rx_entry->mbuf); + rx_entry->mbuf = NULL; + } + + if (txd->dd) + to_clean--; + + txd->buf_addr = 0; + txd->flags = 0; + + head = (head + 1) % txq->nb_tx_desc; + txq->tx_free++; + } + + txq->tx_head = head; + } +} + +static int +atl_tso_setup(struct rte_mbuf *tx_pkt, union hw_atl_txc_s *txc) +{ + uint32_t tx_cmd = 0; + uint64_t ol_flags = tx_pkt->ol_flags; + + PMD_INIT_FUNC_TRACE(); + + if (ol_flags & PKT_TX_TCP_SEG) { + PMD_DRV_LOG(DEBUG, "xmit TSO pkt"); + + tx_cmd |= tx_desc_cmd_lso | tx_desc_cmd_l4cs; + + txc->cmd = 0x4; + + if (ol_flags & PKT_TX_IPV6) + txc->cmd |= 0x2; + + txc->l2_len = tx_pkt->l2_len; + txc->l3_len = tx_pkt->l3_len; + txc->l4_len = tx_pkt->l4_len; + + txc->mss_len = tx_pkt->tso_segsz; + } + + if (ol_flags & PKT_TX_VLAN_PKT) { + tx_cmd |= tx_desc_cmd_vlan; + txc->vlan_tag = tx_pkt->vlan_tci; + } + + if (tx_cmd) { + txc->type = tx_desc_type_ctx; + txc->idx = 0; + } + + return tx_cmd; +} + +static inline void +atl_setup_csum_offload(struct rte_mbuf *mbuf, struct hw_atl_txd_s *txd, + uint32_t tx_cmd) +{ + txd->cmd |= tx_desc_cmd_fcs; + txd->cmd |= (mbuf->ol_flags & PKT_TX_IP_CKSUM) ? tx_desc_cmd_ipv4 : 0; + /* L4 csum requested */ + txd->cmd |= (mbuf->ol_flags & PKT_TX_L4_MASK) ? tx_desc_cmd_l4cs : 0; + txd->cmd |= tx_cmd; +} + +static inline void +atl_xmit_pkt(struct aq_hw_s *hw, struct atl_tx_queue *txq, + struct rte_mbuf *tx_pkt) +{ + uint32_t pay_len = 0; + int tail = 0; + struct atl_tx_entry *tx_entry; + uint64_t buf_dma_addr; + struct rte_mbuf *m_seg; + union hw_atl_txc_s *txc = NULL; + struct hw_atl_txd_s *txd = NULL; + u32 tx_cmd = 0U; + int desc_count = 0; + + PMD_INIT_FUNC_TRACE(); + + tail = txq->tx_tail; + + txc = (union hw_atl_txc_s *)&txq->hw_ring[tail]; + + txc->flags1 = 0U; + txc->flags2 = 0U; + + tx_cmd = atl_tso_setup(tx_pkt, txc); + + if (tx_cmd) { + /* We've consumed the first desc, adjust counters */ + tail = (tail + 1) % txq->nb_tx_desc; + txq->tx_tail = tail; + txq->tx_free -= 1; + + txd = &txq->hw_ring[tail]; + txd->flags = 0U; + } else { + txd = (struct hw_atl_txd_s *)txc; + } + + txd->ct_en = !!tx_cmd; + + txd->type = tx_desc_type_desc; + + atl_setup_csum_offload(tx_pkt, txd, tx_cmd); + + if (tx_cmd) + txd->ct_idx = 0; + + pay_len = tx_pkt->pkt_len; + + txd->pay_len = pay_len; + + for (m_seg = tx_pkt; m_seg; m_seg = m_seg->next) { + if (desc_count > 0) { + txd = &txq->hw_ring[tail]; + txd->flags = 0U; + } + + buf_dma_addr = rte_mbuf_data_iova(m_seg); + txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr); + + txd->type = tx_desc_type_desc; + txd->len = m_seg->data_len; + txd->pay_len = pay_len; + + /* Store mbuf for freeing later */ + tx_entry = &txq->sw_ring[tail]; + + if (tx_entry->mbuf) + rte_pktmbuf_free_seg(tx_entry->mbuf); + tx_entry->mbuf = m_seg; + + tail = (tail + 1) % txq->nb_tx_desc; + + desc_count++; + } + + // Last descriptor requires EOP and WB + txd->eop = 1U; + txd->cmd |= tx_desc_cmd_wb; + + hw_atl_b0_hw_tx_ring_tail_update(hw, tail, txq->queue_id); + + txq->tx_tail = tail; + + txq->tx_free -= desc_count; +} uint16_t -atl_xmit_pkts(void *tx_queue __rte_unused, - struct rte_mbuf **tx_pkts __rte_unused, - uint16_t nb_pkts __rte_unused) +atl_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) { - return 0; + struct rte_eth_dev *dev = NULL; + struct aq_hw_s *hw = NULL; + struct atl_tx_queue *txq = tx_queue; + struct rte_mbuf *tx_pkt; + uint16_t nb_tx; + + dev = &rte_eth_devices[txq->port_id]; + hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private); + + PMD_TX_LOG(DEBUG, "txq%d pkts: %d tx_free=%d tx_tail=%d tx_head=%d", + txq->queue_id, nb_pkts, txq->tx_free, + txq->tx_tail, txq->tx_head); + + for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) { + tx_pkt = *tx_pkts++; + + /* Clean Tx queue if needed */ + if (txq->tx_free < txq->tx_free_thresh) + atl_xmit_cleanup(txq); + + /* Check if we have enough free descriptors */ + if (txq->tx_free < tx_pkt->nb_segs) + break; + + /* check mbuf is valid */ + if ((tx_pkt->nb_segs == 0) || + ((tx_pkt->nb_segs > 1) && (tx_pkt->next == NULL))) + break; + + /* Send the packet */ + atl_xmit_pkt(hw, txq, tx_pkt); + } + + PMD_TX_LOG(DEBUG, "atl_xmit_pkts %d transmitted", nb_tx); + + return nb_tx; }