From patchwork Mon Aug 17 17:49:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 75582 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 43E33A034C; Mon, 17 Aug 2020 19:49:45 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4AA184CA7; Mon, 17 Aug 2020 19:49:44 +0200 (CEST) Received: from hqnvemgate26.nvidia.com (hqnvemgate26.nvidia.com [216.228.121.65]) by dpdk.org (Postfix) with ESMTP id 128EA4C99 for ; Mon, 17 Aug 2020 19:49:41 +0200 (CEST) Received: from hqpgpgate102.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Mon, 17 Aug 2020 10:49:27 -0700 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate102.nvidia.com (PGP Universal service); Mon, 17 Aug 2020 10:49:41 -0700 X-PGP-Universal: processed; by hqpgpgate102.nvidia.com on Mon, 17 Aug 2020 10:49:41 -0700 Received: from HQMAIL101.nvidia.com (172.20.187.10) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 17 Aug 2020 17:49:40 +0000 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.106) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Mon, 17 Aug 2020 17:49:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K1IaxXfKS8FUsNb4RHes5783rsefmqHkmA0xQp2MlrXcBIGUynycslvnTHX13DI5aZDDpBoj7vaPA12Cy8x5EFyYP0XT8VDgUlOxet1h4J4EI5aPYS9wd3smFBss+pqaW0ZCeUsOpOVBFs1X0jEbPDK++7WCG5oSQobQOWl0oOK8Mq7r4AkFVTK8dyCEnVVnf1cdVt8oXp3muhon4HZ71sSTijzvtdlqO6JDrtZo48/wHCQVaTnc95Pz2YdX6cQAy6J5puUJIUXv44pPfoHAOrQ2qGw4Bvwucz4IdJDD3uly0Z796LdbfV+k4DDW7YRctd10VyTtqPI5q6Mcj6nUPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Wc32dHwSeIgRig19xjA79w4HpiOfGgxMtGlEOReOfG0=; b=XPbnXYcnNGtXKQEwXyhOXqH3fEiClwCB/UcZvt63WEXvYgZrPfJ1u7M2TiGvoOWwW418W1+UuC+LUH97TSoQtwhwQKj0b5KA8q59GFstUKBe+8MNML2NVTYb4zNQhmvTQvYwUGA91R7ewP7fb5VPA7NApf10Zy9FaKDYLBG70/OHTHo+61GzgzpCSvF6/xUlu00fgtBnBmmRMFma6+d+bOpLUej0MpqNJJR47ZuEKekWMYYP6yd97sJfV9+ccdLAPDYDVQGZB/uah+uYCS2+9lQNmmbOnrM+mA+tvtwsUAKciAPU27kXIRkUQlYbtNeS350L7naP56APjPRhzM7daw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MWHPR12MB1360.namprd12.prod.outlook.com (2603:10b6:300:12::7) by MWHPR12MB1694.namprd12.prod.outlook.com (2603:10b6:301:11::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3283.18; Mon, 17 Aug 2020 17:49:39 +0000 Received: from MWHPR12MB1360.namprd12.prod.outlook.com ([fe80::711e:ec6f:ba28:d3d0]) by MWHPR12MB1360.namprd12.prod.outlook.com ([fe80::711e:ec6f:ba28:d3d0%5]) with mapi id 15.20.3283.027; Mon, 17 Aug 2020 17:49:38 +0000 From: Slava Ovsiienko To: "dev@dpdk.org" CC: Thomas Monjalon , "stephen@networkplumber.org" , "ferruh.yigit@intel.com" , Shahaf Shuler , "olivier.matz@6wind.com" , "jerinjacobk@gmail.com" , "maxime.coquelin@redhat.com" , "david.marchand@redhat.com" , "arybchenko@solarflare.com" , Asaf Penso Thread-Topic: [RFC] ethdev: introduce Rx buffer split Thread-Index: AdZ0vX26iflBD6P5QjC0iZ6rubHz2w== Date: Mon, 17 Aug 2020 17:49:38 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [95.164.10.10] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0d05c18c-33d6-4aaf-96a4-08d842d5e9bd x-ms-traffictypediagnostic: MWHPR12MB1694: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:551; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 9dFMMEFGaQOXlFXSMLjyNV/tAV5HBqXAEFqVsObOZh+Q5mqKbnLcS3iQdfceVsTlmF1bYvRd494rd71HMt6Sg6btPlb7wHHx7c3AyI95ygRYA0LoWqU52vok4HvuB4EsGUsvyCTZpf28b5JNPc0dz7F6zr/vhdRCVnuYMOfbSpEbGmdpGY3pKBXM5UkhLSu5uou15YQYZ7QSflZSbI8syW4srmMaddrtzMKnQCsP3PRienwP2HDHfvrslgnKhExLU2gJUaQ94XI8oXrs8pmlj2IzJjTK+plVj6dY3v1KNxb0kDNt0jUhx8Za3Lh1w1+Ym63Ma4IhwP4kWT3Y6fPJwQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR12MB1360.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(346002)(366004)(396003)(376002)(30864003)(66446008)(76116006)(66476007)(64756008)(66946007)(316002)(52536014)(5660300002)(66556008)(8676002)(55016002)(54906003)(71200400001)(33656002)(83380400001)(86362001)(4326008)(26005)(6916009)(6506007)(478600001)(107886003)(8936002)(53546011)(7696005)(2906002)(9686003)(186003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: T9zllH/BO6gfNmZ5EXmQ5edHusHkGyZcfwYlBWSJsLPwDQ9p3T4MXTcUljRIXVOWDmiJMugJqjm49wSN7dwtZE/ctF0OS3+RyLURk7LTG34yEqbXh3W8yI5JYNj/tqq87DRj3c3ZQxw9tW8SSDEMQ3fuoGE3Ej4gqLZOAjOXv/ouxob2b/TBtgoqHNT9ZxpVn0aPQkAnWrpMV3XzergnNkZPG5nmahbvLAtRiTfSK0YgLU5MXmru7TH8nf2bHexjvjsle8BYnCoRCeATBurAkKR7Rw84lZvZhMBg4jaZE7M8Am+2eIT14AklfWwA29Y+szr2SHADBuKGva1BYz+Hu4i+aA/jNUEF611pcezE63owE4IQLZt833W7VFHBN1qS8MomUagJUWRW8e2IxC3qd6s0IdMQ8yE2wDH+GKPJIlUwExStbzz/Hc9QUF8r5dmwaOddlUBnh9+LXjF8NUTtkDgdbTAiINIFEvAFiRKobUDcHBVBGJ/6rCEwuHMyyjIwwnvLV+HnlT1nzDfiKbcqoDhB11bR4JW+w5yBm2DpKjZGSowr2XAxtQKjRdVPDFccDsMThqRQqnbm+vLzHwltKOsE2FePT9vACRSOeJ8g+6Y84MBWaIFoWDSuOxFruXLEiJliyFZg+vkPOdToc1QJ1Q== MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MWHPR12MB1360.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0d05c18c-33d6-4aaf-96a4-08d842d5e9bd X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Aug 2020 17:49:38.7949 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: lr1uQhatGPtAu2ODr+oIbVV7gco7f/XqCLpILuXF0CblIknDbELNyrNCL7gzpVuG+/RQwEy518oa7uER7aRjgg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1694 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1597686567; bh=Wc32dHwSeIgRig19xjA79w4HpiOfGgxMtGlEOReOfG0=; h=X-PGP-Universal:ARC-Seal:ARC-Message-Signature: ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic: Thread-Index:Date:Message-ID:Accept-Language:Content-Language: X-MS-Has-Attach:X-MS-TNEF-Correlator:authentication-results: x-originating-ip:x-ms-publictraffictype: x-ms-office365-filtering-correlation-id:x-ms-traffictypediagnostic: x-ld-processed:x-ms-exchange-transport-forked: x-microsoft-antispam-prvs:x-ms-oob-tlc-oobclassifiers: x-ms-exchange-senderadcheck:x-microsoft-antispam: x-microsoft-antispam-message-info:x-forefront-antispam-report: x-ms-exchange-antispam-messagedata:Content-Type: Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=fSlEbwwJH26qhd7vCXLI9kDiz6vbJSBiG9lj+5AkxrK7/QsMssromSC9q+rPB7mDZ ZS7T4ubcqnnQ6vnpzfWXgbQmVugogggai+HfUi9PTKiXDFuoJ6QzL3UHlSvtUMH5ju rKJXkZzs95atKBKcChTi55H+IOYRVzYfWQGl/nswaz4jtl//vWCI72vj/++w5NEPwX 7FCyovsKpdvpHMvdv2JVxd2QVGRurK1WcHWOEZNK5ReJe1RnS9zfVH8Zr87yPQFFEr SAbFyl/9+jwP/NgX0EizP23VzkhnXipdu/VrguMND9ctBh7AcBiyAVuCci4PjTZxrp VeSwnQS1sbGwA== Subject: [dpdk-dev] [RFC] ethdev: introduce Rx buffer split X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From 7f7052d8b85ff3ff7011bd844b6d3169c6e51923 Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Mon, 17 Aug 2020 16:57:43 +0000 Subject: [RFC] ethdev: introduce Rx buffer split The DPDK datapath in the transmit direction is very flexible. An application can build the multisegment packet and manages almost all data aspects - the memory pools where segments are allocated from, the segment lengths, the memory attributes like external buffers, registered for DMA, etc. In the receiving direction, the datapath is much less flexible, an application can only specify the memory pool to configure the receiving queue and nothing more. In order to extend receiving datapath capabilities it is proposed to add the way to provide extended infoirmation how to split the packets being received. The following structure is introduced to specify the Rx packet segment: struct rte_eth_rxseg { struct rte_mempool *mp; /* memory pools to allocate segment from */ uint16_t length; /* segment maximal data length */ uint16_t offset; /* data offset from beginning of mbuf data buffer */ uint32_t reserved; /* reserved field */ }; The new routine rte_eth_rx_queue_setup_ex() is introduced to setup the given Rx queue using the new extended Rx packet segment description: int rte_eth_rx_queue_setup_ex(uint16_t port_id, uint16_t rx_queue_id, uint16_t nb_rx_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, const struct rte_eth_rxseg *rx_seg, uint16_t n_seg) This routine presents the two new parameters: rx_seg - pointer the array of segment descriptions, each element describes the memory pool, maximal data length, initial data offset from the beginning of data buffer in mbuf n_seg - number of elements in the array The new offload flag DEV_RX_OFFLOAD_BUFFER_SPLIT in device capabilities is introduced to present the way for PMD to report to application about supporting Rx packet split to configurable segments. Prior invoking the rte_eth_rx_queue_setup_ex() routine application should check DEV_RX_OFFLOAD_BUFFER_SPLIT flag. If the Rx queue is configured with new routine the packets being received will be split into multiple segments pushed to the mbufs with specified attributes. The PMD will allocate the first mbuf from the pool specified in the first segment descriptor and puts the data staring at specified offeset in the allocated mbuf data buffer. If packet length exceeds the specified segment length the next mbuf will be allocated according to the next segment descriptor (if any) and data will be put in its data buffer at specified offset and not exceeding specified length. If there is no next descriptor the next mbuf will be allocated and filled in the same way (from the same pool and with the same buffer offset/length) as the current one. For example, let's suppose we configured the Rx queue with the following segments: seg0 - pool0, len0=14B, off0=RTE_PKTMBUF_HEADROOM seg1 - pool1, len1=20B, off1=0B seg2 - pool2, len2=20B, off2=0B seg3 - pool3, len3=512B, off3=0B The packet 46 bytes long will look like the following: seg0 - 14B long @ RTE_PKTMBUF_HEADROOM in mbuf from pool0 seg1 - 20B long @ 0 in mbuf from pool1 seg2 - 12B long @ 0 in mbuf from pool2 The packet 1500 bytes long will look like the following: seg0 - 14B @ RTE_PKTMBUF_HEADROOM in mbuf from pool0 seg1 - 20B @ 0 in mbuf from pool1 seg2 - 20B @ 0 in mbuf from pool2 seg3 - 512B @ 0 in mbuf from pool3 seg4 - 512B @ 0 in mbuf from pool3 seg5 - 422B @ 0 in mbuf from pool3 The offload DEV_RX_OFFLOAD_SCATTER must be present and configured to support new buffer spllit feature (if n_seg is greater than one). The new approach would allow splitting the ingress packets into multiple parts pushed to the memory with different attributes. For example, the packet headers can be pushed to the embedded data buffers within mbufs and the application data into the external buffers attached to mbufs allocated from the different memory pools. The memory attributes for the split parts may differ either - for example the application data may be pushed into the external memory located on the dedicated physical device, say GPU or NVMe. This would improve the DPDK receiving datapath flexibility with preserving compatibility with existing API. Also, the proposed segment description might be used to specify Rx packet split for some other features. For example, provide the way to specify the extra memory pool for the Header Split feature of some Intel PMD. Signed-off-by: Viacheslav Ovsiienko --- lib/librte_ethdev/rte_ethdev.c | 166 ++++++++++++++++++++++++++++++++++++ lib/librte_ethdev/rte_ethdev.h | 15 ++++ lib/librte_ethdev/rte_ethdev_core.h | 10 +++ 3 files changed, 191 insertions(+) diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 7858ad5..638e42d 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -1933,6 +1933,172 @@ struct rte_eth_dev * } int +rte_eth_rx_queue_setup_ex(uint16_t port_id, uint16_t rx_queue_id, + uint16_t nb_rx_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + const struct rte_eth_rxseg *rx_seg, uint16_t n_seg) +{ + int ret; + uint16_t seg_idx; + uint32_t mbp_buf_size; + struct rte_eth_dev *dev; + struct rte_eth_dev_info dev_info; + struct rte_eth_rxconf local_conf; + void **rxq; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); + + dev = &rte_eth_devices[port_id]; + if (rx_queue_id >= dev->data->nb_rx_queues) { + RTE_ETHDEV_LOG(ERR, "Invalid RX queue_id=%u\n", rx_queue_id); + return -EINVAL; + } + + if (rx_seg == NULL) { + RTE_ETHDEV_LOG(ERR, "Invalid null description pointer\n"); + return -EINVAL; + } + + if (n_seg == 0) { + RTE_ETHDEV_LOG(ERR, "Invalid zero description number\n"); + return -EINVAL; + } + + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_setup_ex, -ENOTSUP); + + /* + * Check the size of the mbuf data buffer. + * This value must be provided in the private data of the memory pool. + * First check that the memory pool has a valid private data. + */ + ret = rte_eth_dev_info_get(port_id, &dev_info); + if (ret != 0) + return ret; + + for (seg_idx = 0; seg_idx < n_seg; seg_idx++) { + struct rte_mempool *mp = rx_seg[seg_idx].mp; + + if (mp->private_data_size < sizeof(struct rte_pktmbuf_pool_private)) { + RTE_ETHDEV_LOG(ERR, "%s private_data_size %d < %d\n", + mp->name, (int)mp->private_data_size, + (int)sizeof(struct rte_pktmbuf_pool_private)); + return -ENOSPC; + } + + mbp_buf_size = rte_pktmbuf_data_room_size(mp); + if (mbp_buf_size < rx_seg[seg_idx].length + rx_seg[seg_idx].offset) { + RTE_ETHDEV_LOG(ERR, + "%s mbuf_data_room_size %d < %d" + " (segment length=%d + segment offset=%d)\n", + mp->name, (int)mbp_buf_size, + (int)(rx_seg[seg_idx].length + rx_seg[seg_idx].offset), + (int)rx_seg[seg_idx].length, + (int)rx_seg[seg_idx].offset); + return -EINVAL; + } + } + + /* Use default specified by driver, if nb_rx_desc is zero */ + if (nb_rx_desc == 0) { + nb_rx_desc = dev_info.default_rxportconf.ring_size; + /* If driver default is also zero, fall back on EAL default */ + if (nb_rx_desc == 0) + nb_rx_desc = RTE_ETH_DEV_FALLBACK_RX_RINGSIZE; + } + + if (nb_rx_desc > dev_info.rx_desc_lim.nb_max || + nb_rx_desc < dev_info.rx_desc_lim.nb_min || + nb_rx_desc % dev_info.rx_desc_lim.nb_align != 0) { + + RTE_ETHDEV_LOG(ERR, + "Invalid value for nb_rx_desc(=%hu), should be: <= %hu, >= %hu, and a product of %hu\n", + nb_rx_desc, dev_info.rx_desc_lim.nb_max, + dev_info.rx_desc_lim.nb_min, + dev_info.rx_desc_lim.nb_align); + return -EINVAL; + } + + if (dev->data->dev_started && + !(dev_info.dev_capa & + RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP)) + return -EBUSY; + + if (dev->data->dev_started && + (dev->data->rx_queue_state[rx_queue_id] != + RTE_ETH_QUEUE_STATE_STOPPED)) + return -EBUSY; + + rxq = dev->data->rx_queues; + if (rxq[rx_queue_id]) { + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_release, + -ENOTSUP); + (*dev->dev_ops->rx_queue_release)(rxq[rx_queue_id]); + rxq[rx_queue_id] = NULL; + } + + if (rx_conf == NULL) + rx_conf = &dev_info.default_rxconf; + + local_conf = *rx_conf; + + /* + * If an offloading has already been enabled in + * rte_eth_dev_configure(), it has been enabled on all queues, + * so there is no need to enable it in this queue again. + * The local_conf.offloads input to underlying PMD only carries + * those offloadings which are only enabled on this queue and + * not enabled on all queues. + */ + local_conf.offloads &= ~dev->data->dev_conf.rxmode.offloads; + + /* + * New added offloadings for this queue are those not enabled in + * rte_eth_dev_configure() and they must be per-queue type. + * A pure per-port offloading can't be enabled on a queue while + * disabled on another queue. A pure per-port offloading can't + * be enabled for any queue as new added one if it hasn't been + * enabled in rte_eth_dev_configure(). + */ + if ((local_conf.offloads & dev_info.rx_queue_offload_capa) != + local_conf.offloads) { + RTE_ETHDEV_LOG(ERR, + "Ethdev port_id=%d rx_queue_id=%d, new added offloads 0x%"PRIx64" must be " + "within per-queue offload capabilities 0x%"PRIx64" in %s()\n", + port_id, rx_queue_id, local_conf.offloads, + dev_info.rx_queue_offload_capa, + __func__); + return -EINVAL; + } + + /* + * If LRO is enabled, check that the maximum aggregated packet + * size is supported by the configured device. + */ + if (local_conf.offloads & DEV_RX_OFFLOAD_TCP_LRO) { + if (dev->data->dev_conf.rxmode.max_lro_pkt_size == 0) + dev->data->dev_conf.rxmode.max_lro_pkt_size = + dev->data->dev_conf.rxmode.max_rx_pkt_len; + int ret = check_lro_pkt_size(port_id, + dev->data->dev_conf.rxmode.max_lro_pkt_size, + dev->data->dev_conf.rxmode.max_rx_pkt_len, + dev_info.max_lro_pkt_size); + if (ret != 0) + return ret; + } + + ret = (*dev->dev_ops->rx_queue_setup_ex)(dev, rx_queue_id, nb_rx_desc, + socket_id, &local_conf, + rx_seg, n_seg); + if (!ret) { + if (!dev->data->min_rx_buf_size || + dev->data->min_rx_buf_size > mbp_buf_size) + dev->data->min_rx_buf_size = mbp_buf_size; + } + + return eth_err(port_id, ret); +} + +int rte_eth_rx_hairpin_queue_setup(uint16_t port_id, uint16_t rx_queue_id, uint16_t nb_rx_desc, const struct rte_eth_hairpin_conf *conf) diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h index 70295d7..701264a 100644 --- a/lib/librte_ethdev/rte_ethdev.h +++ b/lib/librte_ethdev/rte_ethdev.h @@ -938,6 +938,16 @@ struct rte_eth_txmode { }; /** + * A structure used to configure an RX packet segment to split. + */ +struct rte_eth_rxseg { + struct rte_mempool *mp; /**< Memory pools to allocate segment from */ + uint16_t length; /**< Segment maximal data length */ + uint16_t offset; /**< Data offset from beggining of mbuf data buffer */ + uint32_t reserved; /**< Reserved field */ +}; + +/** * A structure used to configure an RX ring of an Ethernet port. */ struct rte_eth_rxconf { @@ -1988,6 +1998,11 @@ int rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mb_pool); +int rte_eth_rx_queue_setup_ex(uint16_t port_id, uint16_t rx_queue_id, + uint16_t nb_rx_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + const struct rte_eth_rxseg *rx_seg, uint16_t n_seg); + /** * @warning * @b EXPERIMENTAL: this API may change, or be removed, without prior notice diff --git a/lib/librte_ethdev/rte_ethdev_core.h b/lib/librte_ethdev/rte_ethdev_core.h index 32407dd..27018de 100644 --- a/lib/librte_ethdev/rte_ethdev_core.h +++ b/lib/librte_ethdev/rte_ethdev_core.h @@ -265,6 +265,15 @@ typedef int (*eth_rx_queue_setup_t)(struct rte_eth_dev *dev, struct rte_mempool *mb_pool); /**< @internal Set up a receive queue of an Ethernet device. */ +typedef int (*eth_rx_queue_setup_ex_t)(struct rte_eth_dev *dev, + uint16_t rx_queue_id, + uint16_t nb_rx_desc, + unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + const struct rte_eth_rxseg *rx_seg, + uint16_t n_seg); +/**< @internal Set up a receive queue of an Ethernet device. */ + typedef int (*eth_tx_queue_setup_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, uint16_t nb_tx_desc, @@ -659,6 +668,7 @@ struct eth_dev_ops { eth_queue_start_t tx_queue_start;/**< Start TX for a queue. */ eth_queue_stop_t tx_queue_stop; /**< Stop TX for a queue. */ eth_rx_queue_setup_t rx_queue_setup;/**< Set up device RX queue. */ + eth_rx_queue_setup_ex_t rx_queue_setup_ex;/**< Set up device RX queue. */ eth_queue_release_t rx_queue_release; /**< Release RX queue. */ eth_rx_queue_count_t rx_queue_count; /**< Get the number of used RX descriptors. */ From patchwork Mon Oct 12 16:19:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 80382 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A9E70A04B6; Mon, 12 Oct 2020 18:20:30 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 91D481D941; Mon, 12 Oct 2020 18:19:59 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 4B42A1D92F for ; Mon, 12 Oct 2020 18:19:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@nvidia.com) with SMTP; 12 Oct 2020 19:19:45 +0300 Received: from nvidia.com (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 09CGJjKJ017440; Mon, 12 Oct 2020 19:19:45 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: thomasm@monjalon.net, stephen@networkplumber.org, ferruh.yigit@intel.com, olivier.matz@6wind.com, jerinjacobk@gmail.com, maxime.coquelin@redhat.com, david.marchand@redhat.com, arybchenko@solarflare.com Date: Mon, 12 Oct 2020 16:19:38 +0000 Message-Id: <1602519585-5194-3-git-send-email-viacheslavo@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> References: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> Subject: [dpdk-dev] [PATCH v3 2/9] app/testpmd: add multiple pools per core creation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The command line parameter --mbuf-size is updated, it can handle the multiple values like the following: --mbuf-size=2176,512,768,4096 specifying the creation the extra memory pools with the requested mbuf data buffer sizes. If some buffer split feature is engaged the extra memory pools can be used to configure the Rx queues with rte_the_dev_rx_queue_setup_ex(). The extra pools are created with requested sizes, and pool names are assigned with appended index: mbuf_pool_socket_%socket_%index. Index zero is used to specify the first mandatory pool to maintain compatibility with existing code. Signed-off-by: Viacheslav Ovsiienko --- app/test-pmd/bpf_cmd.c | 4 +-- app/test-pmd/cmdline.c | 2 +- app/test-pmd/config.c | 6 ++-- app/test-pmd/parameters.c | 24 +++++++++---- app/test-pmd/testpmd.c | 63 +++++++++++++++++++---------------- app/test-pmd/testpmd.h | 24 ++++++++++--- doc/guides/testpmd_app_ug/run_app.rst | 7 ++-- 7 files changed, 83 insertions(+), 47 deletions(-) diff --git a/app/test-pmd/bpf_cmd.c b/app/test-pmd/bpf_cmd.c index 16e3c3b..0a1a178 100644 --- a/app/test-pmd/bpf_cmd.c +++ b/app/test-pmd/bpf_cmd.c @@ -69,7 +69,7 @@ struct cmd_bpf_ld_result { *flags = RTE_BPF_ETH_F_NONE; arg->type = RTE_BPF_ARG_PTR; - arg->size = mbuf_data_size; + arg->size = mbuf_data_size[0]; for (i = 0; str[i] != 0; i++) { v = toupper(str[i]); @@ -78,7 +78,7 @@ struct cmd_bpf_ld_result { else if (v == 'M') { arg->type = RTE_BPF_ARG_PTR_MBUF; arg->size = sizeof(struct rte_mbuf); - arg->buf_size = mbuf_data_size; + arg->buf_size = mbuf_data_size[0]; } else if (v == '-') continue; else diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 273fb1a..a585cf0 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -2907,7 +2907,7 @@ struct cmd_setup_rxtx_queue { if (!numa_support || socket_id == NUMA_NO_CONFIG) socket_id = port->socket_id; - mp = mbuf_pool_find(socket_id); + mp = mbuf_pool_find(socket_id, 0); if (mp == NULL) { printf("Failed to setup RX queue: " "No mempool allocation" diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index 8ccd989..4405abc 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -647,7 +647,7 @@ static int bus_match_all(const struct rte_bus *bus, const void *data) printf("\nConnect to socket: %u", port->socket_id); if (port_numa[port_id] != NUMA_NO_CONFIG) { - mp = mbuf_pool_find(port_numa[port_id]); + mp = mbuf_pool_find(port_numa[port_id], 0); if (mp) printf("\nmemory allocation on the socket: %d", port_numa[port_id]); @@ -3309,9 +3309,9 @@ struct igb_ring_desc_16_bytes { */ tx_pkt_len = 0; for (i = 0; i < nb_segs; i++) { - if (seg_lengths[i] > (unsigned) mbuf_data_size) { + if (seg_lengths[i] > mbuf_data_size[0]) { printf("length[%u]=%u > mbuf_data_size=%u - give up\n", - i, seg_lengths[i], (unsigned) mbuf_data_size); + i, seg_lengths[i], mbuf_data_size[0]); return; } tx_pkt_len = (uint16_t)(tx_pkt_len + seg_lengths[i]); diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 1ead595..1f40d73 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -106,7 +106,9 @@ "(flag: 1 for RX; 2 for TX; 3 for RX and TX).\n"); printf(" --socket-num=N: set socket from which all memory is allocated " "in NUMA mode.\n"); - printf(" --mbuf-size=N: set the data size of mbuf to N bytes.\n"); + printf(" --mbuf-size=N,[N1[,..Nn]: set the data size of mbuf to " + "N bytes. If multiple numbers are specified the extra pools " + "will be created to receive with packet split features\n"); printf(" --total-num-mbufs=N: set the number of mbufs to be allocated " "in mbuf pools.\n"); printf(" --max-pkt-len=N: set the maximum size of packet to N bytes.\n"); @@ -890,12 +892,22 @@ } } if (!strcmp(lgopts[opt_idx].name, "mbuf-size")) { - n = atoi(optarg); - if (n > 0 && n <= 0xFFFF) - mbuf_data_size = (uint16_t) n; - else + unsigned int mb_sz[MAX_SEGS_BUFFER_SPLIT]; + unsigned int nb_segs, i; + + nb_segs = parse_item_list(optarg, "mbuf-size", + MAX_SEGS_BUFFER_SPLIT, mb_sz, 0); + if (nb_segs <= 0) rte_exit(EXIT_FAILURE, - "mbuf-size should be > 0 and < 65536\n"); + "bad mbuf-size\n"); + for (i = 0; i < nb_segs; i++) { + if (mb_sz[i] <= 0 || mb_sz[i] > 0xFFFF) + rte_exit(EXIT_FAILURE, + "mbuf-size should be " + "> 0 and < 65536\n"); + mbuf_data_size[i] = (uint16_t) mb_sz[i]; + } + mbuf_data_size_n = nb_segs; } if (!strcmp(lgopts[opt_idx].name, "total-num-mbufs")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index ccba71c..ec66060 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -186,7 +186,7 @@ struct fwd_engine * fwd_engines[] = { NULL, }; -struct rte_mempool *mempools[RTE_MAX_NUMA_NODES]; +struct rte_mempool *mempools[RTE_MAX_NUMA_NODES * MAX_SEGS_BUFFER_SPLIT]; uint16_t mempool_flags; struct fwd_config cur_fwd_config; @@ -195,7 +195,10 @@ struct fwd_engine * fwd_engines[] = { uint32_t burst_tx_delay_time = BURST_TX_WAIT_US; uint32_t burst_tx_retry_num = BURST_TX_RETRIES; -uint16_t mbuf_data_size = DEFAULT_MBUF_DATA_SIZE; /**< Mbuf data space size. */ +uint32_t mbuf_data_size_n = 1; /* Number of specified mbuf sizes. */ +uint16_t mbuf_data_size[MAX_SEGS_BUFFER_SPLIT] = { + DEFAULT_MBUF_DATA_SIZE +}; /**< Mbuf data space size. */ uint32_t param_total_num_mbufs = 0; /**< number of mbufs in all pools - if * specified on command-line. */ uint16_t stats_period; /**< Period to show statistics (disabled by default) */ @@ -955,14 +958,14 @@ struct extmem_param { */ static struct rte_mempool * mbuf_pool_create(uint16_t mbuf_seg_size, unsigned nb_mbuf, - unsigned int socket_id) + unsigned int socket_id, unsigned int size_idx) { char pool_name[RTE_MEMPOOL_NAMESIZE]; struct rte_mempool *rte_mp = NULL; uint32_t mb_size; mb_size = sizeof(struct rte_mbuf) + mbuf_seg_size; - mbuf_poolname_build(socket_id, pool_name, sizeof(pool_name)); + mbuf_poolname_build(socket_id, pool_name, sizeof(pool_name), size_idx); TESTPMD_LOG(INFO, "create a new mbuf pool <%s>: n=%u, size=%u, socket=%u\n", @@ -1485,8 +1488,8 @@ struct extmem_param { port->dev_info.rx_desc_lim.nb_mtu_seg_max; if ((data_size + RTE_PKTMBUF_HEADROOM) > - mbuf_data_size) { - mbuf_data_size = data_size + + mbuf_data_size[0]) { + mbuf_data_size[0] = data_size + RTE_PKTMBUF_HEADROOM; warning = 1; } @@ -1494,9 +1497,9 @@ struct extmem_param { } if (warning) - TESTPMD_LOG(WARNING, "Configured mbuf size %hu\n", - mbuf_data_size); - + TESTPMD_LOG(WARNING, + "Configured mbuf size of the first segment %hu\n", + mbuf_data_size[0]); /* * Create pools of mbuf. * If NUMA support is disabled, create a single pool of mbuf in @@ -1516,21 +1519,23 @@ struct extmem_param { } if (numa_support) { - uint8_t i; + uint8_t i, j; for (i = 0; i < num_sockets; i++) - mempools[i] = mbuf_pool_create(mbuf_data_size, - nb_mbuf_per_pool, - socket_ids[i]); + for (j = 0; j < mbuf_data_size_n; j++) + mempools[i * MAX_SEGS_BUFFER_SPLIT + j] = + mbuf_pool_create(mbuf_data_size[j], + nb_mbuf_per_pool, + socket_ids[i], 0); } else { - if (socket_num == UMA_NO_CONFIG) - mempools[0] = mbuf_pool_create(mbuf_data_size, - nb_mbuf_per_pool, 0); - else - mempools[socket_num] = mbuf_pool_create - (mbuf_data_size, - nb_mbuf_per_pool, - socket_num); + uint8_t i; + + for (i = 0; i < mbuf_data_size_n; i++) + mempools[i] = mbuf_pool_create + (mbuf_data_size[i], + nb_mbuf_per_pool, + socket_num == UMA_NO_CONFIG ? + 0 : socket_num, 0); } init_port_config(); @@ -1542,10 +1547,10 @@ struct extmem_param { */ for (lc_id = 0; lc_id < nb_lcores; lc_id++) { mbp = mbuf_pool_find( - rte_lcore_to_socket_id(fwd_lcores_cpuids[lc_id])); + rte_lcore_to_socket_id(fwd_lcores_cpuids[lc_id]), 0); if (mbp == NULL) - mbp = mbuf_pool_find(0); + mbp = mbuf_pool_find(0, 0); fwd_lcores[lc_id]->mbp = mbp; /* initialize GSO context */ fwd_lcores[lc_id]->gso_ctx.direct_pool = mbp; @@ -2498,7 +2503,8 @@ struct extmem_param { if ((numa_support) && (rxring_numa[pi] != NUMA_NO_CONFIG)) { struct rte_mempool * mp = - mbuf_pool_find(rxring_numa[pi]); + mbuf_pool_find + (rxring_numa[pi], 0); if (mp == NULL) { printf("Failed to setup RX queue:" "No mempool allocation" @@ -2514,7 +2520,8 @@ struct extmem_param { mp); } else { struct rte_mempool *mp = - mbuf_pool_find(port->socket_id); + mbuf_pool_find + (port->socket_id, 0); if (mp == NULL) { printf("Failed to setup RX queue:" "No mempool allocation" @@ -2909,13 +2916,13 @@ struct extmem_param { pmd_test_exit(void) { portid_t pt_id; + unsigned int i; int ret; - int i; if (test_done == 0) stop_packet_forwarding(); - for (i = 0 ; i < RTE_MAX_NUMA_NODES ; i++) { + for (i = 0 ; i < RTE_DIM(mempools) ; i++) { if (mempools[i]) { if (mp_alloc_type == MP_ALLOC_ANON) rte_mempool_mem_iter(mempools[i], dma_unmap_cb, @@ -2959,7 +2966,7 @@ struct extmem_param { return; } } - for (i = 0 ; i < RTE_MAX_NUMA_NODES ; i++) { + for (i = 0 ; i < RTE_DIM(mempools) ; i++) { if (mempools[i]) rte_mempool_free(mempools[i]); } diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 227b694..e56a89c 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -42,6 +42,13 @@ */ #define RTE_MAX_SEGS_PER_PKT 255 /**< nb_segs is a 8-bit unsigned char. */ +/* + * The maximum number of segments per packet is used to configure + * buffer split feature, also specifies the maximum amount of + * optional Rx pools to allocate mbufs to split. + */ +#define MAX_SEGS_BUFFER_SPLIT 8 /**< nb_segs is a 8-bit unsigned char. */ + #define MAX_PKT_BURST 512 #define DEF_PKT_BURST 32 @@ -393,7 +400,9 @@ struct queue_stats_mappings { extern uint8_t dcb_config; extern uint8_t dcb_test; -extern uint16_t mbuf_data_size; /**< Mbuf data space size. */ +extern uint32_t mbuf_data_size_n; +extern uint16_t mbuf_data_size[MAX_SEGS_BUFFER_SPLIT]; +/**< Mbuf data space size. */ extern uint32_t param_total_num_mbufs; extern uint16_t stats_period; @@ -604,17 +613,22 @@ struct mplsoudp_decap_conf { /* Mbuf Pools */ static inline void -mbuf_poolname_build(unsigned int sock_id, char* mp_name, int name_size) +mbuf_poolname_build(unsigned int sock_id, char *mp_name, + int name_size, unsigned int idx) { - snprintf(mp_name, name_size, "mbuf_pool_socket_%u", sock_id); + if (!idx) + snprintf(mp_name, name_size, "mbuf_pool_socket_%u", sock_id); + else + snprintf(mp_name, name_size, "mbuf_pool_socket_%u_%u", + sock_id, idx); } static inline struct rte_mempool * -mbuf_pool_find(unsigned int sock_id) +mbuf_pool_find(unsigned int sock_id, unsigned int idx) { char pool_name[RTE_MEMPOOL_NAMESIZE]; - mbuf_poolname_build(sock_id, pool_name, sizeof(pool_name)); + mbuf_poolname_build(sock_id, pool_name, sizeof(pool_name), idx); return rte_mempool_lookup((const char *)pool_name); } diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst index e2539f6..2d5a263 100644 --- a/doc/guides/testpmd_app_ug/run_app.rst +++ b/doc/guides/testpmd_app_ug/run_app.rst @@ -107,9 +107,12 @@ The command line options are: Set the socket from which all memory is allocated in NUMA mode, where 0 <= N < number of sockets on the board. -* ``--mbuf-size=N`` +* ``--mbuf-size=N[,N1[,...Nn]`` - Set the data size of the mbufs used to N bytes, where N < 65536. The default value is 2048. + Set the data size of the mbufs used to N bytes, where N < 65536. + The default value is 2048. If multiple mbuf-size values are specified the + extra memory pools will be created for allocating mbufs to receive packets + with buffer splittling features. * ``--total-num-mbufs=N`` From patchwork Mon Oct 12 16:19:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 80380 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 69B82A04B6; Mon, 12 Oct 2020 18:19:56 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8B1B51D927; Mon, 12 Oct 2020 18:19:53 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 48EC11D92B for ; Mon, 12 Oct 2020 18:19:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@nvidia.com) with SMTP; 12 Oct 2020 19:19:45 +0300 Received: from nvidia.com (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 09CGJjKK017440; Mon, 12 Oct 2020 19:19:45 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: thomasm@monjalon.net, stephen@networkplumber.org, ferruh.yigit@intel.com, olivier.matz@6wind.com, jerinjacobk@gmail.com, maxime.coquelin@redhat.com, david.marchand@redhat.com, arybchenko@solarflare.com Date: Mon, 12 Oct 2020 16:19:39 +0000 Message-Id: <1602519585-5194-4-git-send-email-viacheslavo@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> References: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> Subject: [dpdk-dev] [PATCH v3 3/9] app/testpmd: add buffer split offload configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch add support for RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT providing per queue configuration for this offload. Signed-off-by: Viacheslav Ovsiienko --- app/test-pmd/cmdline.c | 21 +++++++++++---------- app/test-pmd/config.c | 9 +++++++++ 2 files changed, 20 insertions(+), 10 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index a585cf0..fa71039 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -883,16 +883,16 @@ static void cmd_help_long_parsed(void *parsed_result, "port config rx_offload vlan_strip|" "ipv4_cksum|udp_cksum|tcp_cksum|tcp_lro|qinq_strip|" "outer_ipv4_cksum|macsec_strip|header_split|" - "vlan_filter|vlan_extend|jumbo_frame|" - "scatter|timestamp|security|keep_crc on|off\n" + "vlan_filter|vlan_extend|jumbo_frame|scatter|" + "buffer_split|timestamp|security|keep_crc on|off\n" " Enable or disable a per port Rx offloading" " on all Rx queues of a port\n\n" "port (port_id) rxq (queue_id) rx_offload vlan_strip|" "ipv4_cksum|udp_cksum|tcp_cksum|tcp_lro|qinq_strip|" "outer_ipv4_cksum|macsec_strip|header_split|" - "vlan_filter|vlan_extend|jumbo_frame|" - "scatter|timestamp|security|keep_crc on|off\n" + "vlan_filter|vlan_extend|jumbo_frame|scatter|" + "buffer_split|timestamp|security|keep_crc on|off\n" " Enable or disable a per queue Rx offloading" " only on a specific Rx queue\n\n" @@ -18417,7 +18417,8 @@ struct cmd_config_per_port_rx_offload_result { offload, "vlan_strip#ipv4_cksum#udp_cksum#tcp_cksum#tcp_lro#" "qinq_strip#outer_ipv4_cksum#macsec_strip#" "header_split#vlan_filter#vlan_extend#jumbo_frame#" - "scatter#timestamp#security#keep_crc#rss_hash"); + "scatter#buffer_split#timestamp#security#" + "keep_crc#rss_hash"); cmdline_parse_token_string_t cmd_config_per_port_rx_offload_result_on_off = TOKEN_STRING_INITIALIZER (struct cmd_config_per_port_rx_offload_result, @@ -18497,8 +18498,8 @@ struct cmd_config_per_port_rx_offload_result { .help_str = "port config rx_offload vlan_strip|ipv4_cksum|" "udp_cksum|tcp_cksum|tcp_lro|qinq_strip|outer_ipv4_cksum|" "macsec_strip|header_split|vlan_filter|vlan_extend|" - "jumbo_frame|scatter|timestamp|security|keep_crc|rss_hash " - "on|off", + "jumbo_frame|scatter|buffer_split|timestamp|security|" + "keep_crc|rss_hash on|off", .tokens = { (void *)&cmd_config_per_port_rx_offload_result_port, (void *)&cmd_config_per_port_rx_offload_result_config, @@ -18547,7 +18548,7 @@ struct cmd_config_per_queue_rx_offload_result { offload, "vlan_strip#ipv4_cksum#udp_cksum#tcp_cksum#tcp_lro#" "qinq_strip#outer_ipv4_cksum#macsec_strip#" "header_split#vlan_filter#vlan_extend#jumbo_frame#" - "scatter#timestamp#security#keep_crc"); + "scatter#buffer_split#timestamp#security#keep_crc"); cmdline_parse_token_string_t cmd_config_per_queue_rx_offload_result_on_off = TOKEN_STRING_INITIALIZER (struct cmd_config_per_queue_rx_offload_result, @@ -18603,8 +18604,8 @@ struct cmd_config_per_queue_rx_offload_result { "vlan_strip|ipv4_cksum|" "udp_cksum|tcp_cksum|tcp_lro|qinq_strip|outer_ipv4_cksum|" "macsec_strip|header_split|vlan_filter|vlan_extend|" - "jumbo_frame|scatter|timestamp|security|keep_crc " - "on|off", + "jumbo_frame|scatter|buffer_split|timestamp|security|" + "keep_crc on|off", .tokens = { (void *)&cmd_config_per_queue_rx_offload_result_port, (void *)&cmd_config_per_queue_rx_offload_result_port_id, diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index 4405abc..6af8ea9 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -1049,6 +1049,15 @@ static int bus_match_all(const struct rte_bus *bus, const void *data) printf("off\n"); } + if (dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) { + printf("RX offload buffer split: "); + if (ports[port_id].dev_conf.rxmode.offloads & + RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) + printf("on\n"); + else + printf("off\n"); + } + if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_VLAN_INSERT) { printf("VLAN insert: "); if (ports[port_id].dev_conf.txmode.offloads & From patchwork Mon Oct 12 16:19:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 80385 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C420FA04B6; Mon, 12 Oct 2020 18:21:37 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5B64E1D955; Mon, 12 Oct 2020 18:20:04 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 5E9941D93A for ; Mon, 12 Oct 2020 18:19:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@nvidia.com) with SMTP; 12 Oct 2020 19:19:46 +0300 Received: from nvidia.com (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 09CGJjKN017440; Mon, 12 Oct 2020 19:19:45 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: thomasm@monjalon.net, stephen@networkplumber.org, ferruh.yigit@intel.com, olivier.matz@6wind.com, jerinjacobk@gmail.com, maxime.coquelin@redhat.com, david.marchand@redhat.com, arybchenko@solarflare.com Date: Mon, 12 Oct 2020 16:19:42 +0000 Message-Id: <1602519585-5194-7-git-send-email-viacheslavo@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> References: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> Subject: [dpdk-dev] [PATCH v3 6/9] net/mlx5: add extended Rx queue setup routine X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The routine to provide Rx queue setup with specifying extended receiving buffer description is added. It allows application to specify desired segment lengths, data position offsets in the buffer and dedicated memory pool for each segment. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 2 + drivers/net/mlx5/mlx5.h | 3 ++ drivers/net/mlx5/mlx5_rxq.c | 91 +++++++++++++++++++++++++++++++++++----- drivers/net/mlx5/mlx5_rxtx.h | 10 ++++- 4 files changed, 95 insertions(+), 11 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 487714f..0e85489 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -2495,6 +2495,7 @@ .dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get, .vlan_filter_set = mlx5_vlan_filter_set, .rx_queue_setup = mlx5_rx_queue_setup, + .rxseg_queue_setup = mlx5_rxseg_queue_setup, .rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup, .tx_queue_setup = mlx5_tx_queue_setup, .tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup, @@ -2578,6 +2579,7 @@ .dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get, .vlan_filter_set = mlx5_vlan_filter_set, .rx_queue_setup = mlx5_rx_queue_setup, + .rxseg_queue_setup = mlx5_rxseg_queue_setup, .rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup, .tx_queue_setup = mlx5_tx_queue_setup, .tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup, diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 87d3c15..bfc0812 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -162,6 +162,9 @@ struct mlx5_stats_ctrl { /* Maximal size of aggregated LRO packet. */ #define MLX5_MAX_LRO_SIZE (UINT8_MAX * MLX5_LRO_SEG_CHUNK_SIZE) +/* Maximal number of segments to split. */ +#define MLX5_MAX_RXQ_NSEG (1u << MLX5_MAX_LOG_RQ_SEGS) + /* LRO configurations structure. */ struct mlx5_lro_config { uint32_t supported:1; /* Whether LRO is supported. */ diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index f1d8373..42818d8 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -390,6 +390,7 @@ struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_dev_config *config = &priv->config; uint64_t offloads = (DEV_RX_OFFLOAD_SCATTER | + RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT | DEV_RX_OFFLOAD_TIMESTAMP | DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_RSS_HASH); @@ -715,16 +716,20 @@ * NUMA socket on which memory must be allocated. * @param[in] conf * Thresholds parameters. - * @param mp - * Memory pool for buffer allocations. + * @param rx_seg + * Pointer the array of segment descriptions, each element + * describes the memory pool, maximal data length, initial + * data offset from the beginning of data buffer in mbuf + * @param n_seg + * Number of elements in the segment descriptions array * * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, - unsigned int socket, const struct rte_eth_rxconf *conf, - struct rte_mempool *mp) +mlx5_rxseg_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, + unsigned int socket, const struct rte_eth_rxconf *conf, + const struct rte_eth_rxseg *rx_seg, uint16_t n_seg) { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_rxq_data *rxq = (*priv->rxqs)[idx]; @@ -732,10 +737,43 @@ container_of(rxq, struct mlx5_rxq_ctrl, rxq); int res; + if (!n_seg || !rx_seg) { + DRV_LOG(ERR, "port %u queue index %u invalid " + "split description", + dev->data->port_id, idx); + rte_errno = EINVAL; + return -rte_errno; + } + if (n_seg > 1) { + uint64_t offloads = conf->offloads | + dev->data->dev_conf.rxmode.offloads; + + if (!(offloads & DEV_RX_OFFLOAD_SCATTER)) { + DRV_LOG(ERR, "port %u queue index %u split " + "configuration requires scattering", + dev->data->port_id, idx); + rte_errno = ENOSPC; + return -rte_errno; + } + if (!(offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) { + DRV_LOG(ERR, "port %u queue index %u split " + "offload not configured", + dev->data->port_id, idx); + rte_errno = ENOSPC; + return -rte_errno; + } + if (n_seg > MLX5_MAX_RXQ_NSEG) { + DRV_LOG(ERR, "port %u queue index %u too many " + "segments %u to split", + dev->data->port_id, idx, n_seg); + rte_errno = EOVERFLOW; + return -rte_errno; + } + } res = mlx5_rx_queue_pre_setup(dev, idx, &desc); if (res) return res; - rxq_ctrl = mlx5_rxq_new(dev, idx, desc, socket, conf, mp); + rxq_ctrl = mlx5_rxq_new(dev, idx, desc, socket, conf, rx_seg, n_seg); if (!rxq_ctrl) { DRV_LOG(ERR, "port %u unable to allocate queue index %u", dev->data->port_id, idx); @@ -756,6 +794,39 @@ * RX queue index. * @param desc * Number of descriptors to configure in queue. + * @param socket + * NUMA socket on which memory must be allocated. + * @param[in] conf + * Thresholds parameters. + * @param mp + * Memory pool for buffer allocations. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, + unsigned int socket, const struct rte_eth_rxconf *conf, + struct rte_mempool *mp) +{ + struct rte_eth_rxseg rx_seg = { + .mp = mp, + /* + * All other fields are zeroed, zero segment length + * means the pool buffer size should be used by PMD. + */ + }; + return mlx5_rxseg_queue_setup(dev, idx, desc, socket, conf, &rx_seg, 1); +} + +/** + * + * @param dev + * Pointer to Ethernet device structure. + * @param idx + * RX queue index. + * @param desc + * Number of descriptors to configure in queue. * @param hairpin_conf * Hairpin configuration parameters. * @@ -1328,11 +1399,11 @@ struct mlx5_rxq_ctrl * mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, unsigned int socket, const struct rte_eth_rxconf *conf, - struct rte_mempool *mp) + const struct rte_eth_rxseg *rx_seg, uint16_t n_seg) { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_rxq_ctrl *tmpl; - unsigned int mb_len = rte_pktmbuf_data_room_size(mp); + unsigned int mb_len = rte_pktmbuf_data_room_size(rx_seg[0].mp); unsigned int mprq_stride_nums; unsigned int mprq_stride_size; unsigned int mprq_stride_cap; @@ -1346,7 +1417,7 @@ struct mlx5_rxq_ctrl * uint64_t offloads = conf->offloads | dev->data->dev_conf.rxmode.offloads; unsigned int lro_on_queue = !!(offloads & DEV_RX_OFFLOAD_TCP_LRO); - const int mprq_en = mlx5_check_mprq_support(dev) > 0; + const int mprq_en = mlx5_check_mprq_support(dev) > 0 && n_seg == 1; unsigned int max_rx_pkt_len = lro_on_queue ? dev->data->dev_conf.rxmode.max_lro_pkt_size : dev->data->dev_conf.rxmode.max_rx_pkt_len; @@ -1531,7 +1602,7 @@ struct mlx5_rxq_ctrl * (!!(dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS)); tmpl->rxq.port_id = dev->data->port_id; tmpl->priv = priv; - tmpl->rxq.mp = mp; + tmpl->rxq.mp = rx_seg[0].mp; tmpl->rxq.elts_n = log2above(desc); tmpl->rxq.rq_repl_thresh = MLX5_VPMD_RXQ_RPLNSH_THRESH(1 << tmpl->rxq.elts_n); diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 674296e..f103a30 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -150,6 +150,9 @@ struct mlx5_rxq_data { rte_spinlock_t *uar_lock_cq; /* CQ (UAR) access lock required for 32bit implementations */ #endif + struct rte_eth_rxseg rxseg[MLX5_MAX_RXQ_NSEG]; + /* Buffer split segment descriptions - sizes, offsets, pools. */ + uint32_t rxseg_n; /* Number of split segment descriptions. */ uint32_t tunnel; /* Tunnel information. */ uint64_t flow_meta_mask; int32_t flow_meta_offset; @@ -304,6 +307,10 @@ struct mlx5_txq_ctrl { int mlx5_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, unsigned int socket, const struct rte_eth_rxconf *conf, struct rte_mempool *mp); +int mlx5_rxseg_queue_setup + (struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, + unsigned int socket, const struct rte_eth_rxconf *conf, + const struct rte_eth_rxseg *rx_seg, uint16_t n_seg); int mlx5_rx_hairpin_queue_setup (struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, const struct rte_eth_hairpin_conf *hairpin_conf); @@ -316,7 +323,8 @@ int mlx5_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, unsigned int socket, const struct rte_eth_rxconf *conf, - struct rte_mempool *mp); + const struct rte_eth_rxseg *rx_seg, + uint16_t n_seg); struct mlx5_rxq_ctrl *mlx5_rxq_hairpin_new (struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, const struct rte_eth_hairpin_conf *hairpin_conf); From patchwork Mon Oct 12 16:19:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 80381 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A4E8A04B6; Mon, 12 Oct 2020 18:20:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B2B471D939; Mon, 12 Oct 2020 18:19:57 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 45DA91D927 for ; Mon, 12 Oct 2020 18:19:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@nvidia.com) with SMTP; 12 Oct 2020 19:19:46 +0300 Received: from nvidia.com (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 09CGJjKO017440; Mon, 12 Oct 2020 19:19:45 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: thomasm@monjalon.net, stephen@networkplumber.org, ferruh.yigit@intel.com, olivier.matz@6wind.com, jerinjacobk@gmail.com, maxime.coquelin@redhat.com, david.marchand@redhat.com, arybchenko@solarflare.com Date: Mon, 12 Oct 2020 16:19:43 +0000 Message-Id: <1602519585-5194-8-git-send-email-viacheslavo@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> References: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> Subject: [dpdk-dev] [PATCH v3 7/9] net/mlx5: configure Rx queue to support split X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The scatter-gather elements should be configured accordingly to support the buffer split feature. The application provides the desired settings for the segments at the beginning of the packets and PMD pads the buffer chain (if needed) with attributes of last specified segment to accommodate the packet of maximal length. There are some limitations are implied. The MPRQ feature should be disengaged if split is requested, due to MPRQ neither supports pushing data to the dedicated pools nor follows the flexible buffer sizes. The vectorized rx_burst routines does not support the scattering (these ones are extremely simplified and work over the single segment only) and can't handle split as well. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_rxq.c | 94 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 80 insertions(+), 14 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 42818d8..4ec4677 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1417,7 +1417,8 @@ struct mlx5_rxq_ctrl * uint64_t offloads = conf->offloads | dev->data->dev_conf.rxmode.offloads; unsigned int lro_on_queue = !!(offloads & DEV_RX_OFFLOAD_TCP_LRO); - const int mprq_en = mlx5_check_mprq_support(dev) > 0 && n_seg == 1; + const int mprq_en = mlx5_check_mprq_support(dev) > 0 && n_seg == 1 && + !rx_seg[0].offset && !rx_seg[0].length; unsigned int max_rx_pkt_len = lro_on_queue ? dev->data->dev_conf.rxmode.max_lro_pkt_size : dev->data->dev_conf.rxmode.max_rx_pkt_len; @@ -1425,22 +1426,87 @@ struct mlx5_rxq_ctrl * RTE_PKTMBUF_HEADROOM; unsigned int max_lro_size = 0; unsigned int first_mb_free_size = mb_len - RTE_PKTMBUF_HEADROOM; + const struct rte_eth_rxseg *qs_seg = rx_seg; + unsigned int tail_len; - if (non_scatter_min_mbuf_size > mb_len && !(offloads & - DEV_RX_OFFLOAD_SCATTER)) { + tmpl = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, sizeof(*tmpl) + + desc_n * sizeof(struct rte_mbuf *), 0, socket); + if (!tmpl) { + rte_errno = ENOMEM; + return NULL; + } + MLX5_ASSERT(n_seg && n_seg <= MLX5_MAX_RXQ_NSEG); + /* + * Build the array of actual buffer offsets and lengths. + * Pad with the buffers from the last memory pool if + * needed to handle max size packets, replace zero length + * with the buffer length from the pool. + */ + tail_len = max_rx_pkt_len; + do { + struct rte_eth_rxseg *hw_seg = + &tmpl->rxq.rxseg[tmpl->rxq.rxseg_n]; + uint32_t buf_len = rte_pktmbuf_data_room_size(qs_seg->mp); + uint32_t offset, seg_len; + + /* + * For the buffers beyond descriptions offset is zero, + * the first buffer contains head room. + */ + offset = (tmpl->rxq.rxseg_n >= n_seg ? 0 : qs_seg->offset) + + (tmpl->rxq.rxseg_n ? 0 : RTE_PKTMBUF_HEADROOM); + /* + * For the buffers beyond descriptions the length is + * pool buffer length, zero lengths are replaced with + * pool buffer length either. + */ + seg_len = tmpl->rxq.rxseg_n >= n_seg ? buf_len : + qs_seg->length ? qs_seg->length : (buf_len - offset); + /* Check is done in long int, now overflows. */ + if (buf_len < seg_len + offset) { + DRV_LOG(ERR, "port %u Rx queue %u: Split offset/length " + "%u/%u can't be satisfied", + dev->data->port_id, idx, + qs_seg->length, qs_seg->offset); + rte_errno = EINVAL; + goto error; + } + if (seg_len > tail_len) + seg_len = buf_len - offset; + if (++tmpl->rxq.rxseg_n > MLX5_MAX_RXQ_NSEG) { + DRV_LOG(ERR, + "port %u too many SGEs (%u) needed to handle" + " requested maximum packet size %u, the maximum" + " supported are %u", dev->data->port_id, + tmpl->rxq.rxseg_n, max_rx_pkt_len, + MLX5_MAX_RXQ_NSEG); + rte_errno = ENOTSUP; + goto error; + } + /* Build the actual scattering element in the queue object. */ + hw_seg->mp = qs_seg->mp; + MLX5_ASSERT(offset <= UINT16_MAX); + MLX5_ASSERT(seg_len <= UINT16_MAX); + hw_seg->offset = (uint16_t)offset; + hw_seg->length = (uint16_t)seg_len; + /* + * Advance the segment descriptor, the padding is the based + * on the attributes of the last descriptor. + */ + if (tmpl->rxq.rxseg_n < n_seg) + qs_seg++; + tail_len -= RTE_MIN(tail_len, seg_len); + } while (tail_len || !rte_is_power_of_2(tmpl->rxq.rxseg_n)); + MLX5_ASSERT(tmpl->rxq.rxseg_n && + tmpl->rxq.rxseg_n <= MLX5_MAX_RXQ_NSEG); + if (tmpl->rxq.rxseg_n > 1 && !(offloads & DEV_RX_OFFLOAD_SCATTER)) { DRV_LOG(ERR, "port %u Rx queue %u: Scatter offload is not" " configured and no enough mbuf space(%u) to contain " "the maximum RX packet length(%u) with head-room(%u)", dev->data->port_id, idx, mb_len, max_rx_pkt_len, RTE_PKTMBUF_HEADROOM); rte_errno = ENOSPC; - return NULL; - } - tmpl = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, sizeof(*tmpl) + - desc_n * sizeof(struct rte_mbuf *), 0, socket); - if (!tmpl) { - rte_errno = ENOMEM; - return NULL; + goto error; } tmpl->type = MLX5_RXQ_TYPE_STANDARD; if (mlx5_mr_btree_init(&tmpl->rxq.mr_ctrl.cache_bh, @@ -1467,7 +1533,7 @@ struct mlx5_rxq_ctrl * * - The number of descs is more than the number of strides. * - max_rx_pkt_len plus overhead is less than the max size * of a stride or mprq_stride_size is specified by a user. - * Need to nake sure that there are enough stides to encap + * Need to make sure that there are enough stides to encap * the maximum packet size in case mprq_stride_size is set. * Otherwise, enable Rx scatter if necessary. */ @@ -1497,11 +1563,11 @@ struct mlx5_rxq_ctrl * " strd_num_n = %u, strd_sz_n = %u", dev->data->port_id, idx, tmpl->rxq.strd_num_n, tmpl->rxq.strd_sz_n); - } else if (max_rx_pkt_len <= first_mb_free_size) { + } else if (tmpl->rxq.rxseg_n == 1) { + MLX5_ASSERT(max_rx_pkt_len <= first_mb_free_size); tmpl->rxq.sges_n = 0; max_lro_size = max_rx_pkt_len; } else if (offloads & DEV_RX_OFFLOAD_SCATTER) { - unsigned int size = non_scatter_min_mbuf_size; unsigned int sges_n; if (lro_on_queue && first_mb_free_size < @@ -1516,7 +1582,7 @@ struct mlx5_rxq_ctrl * * Determine the number of SGEs needed for a full packet * and round it to the next power of two. */ - sges_n = log2above((size / mb_len) + !!(size % mb_len)); + sges_n = log2above(tmpl->rxq.rxseg_n); if (sges_n > MLX5_MAX_LOG_RQ_SEGS) { DRV_LOG(ERR, "port %u too many SGEs (%u) needed to handle" From patchwork Mon Oct 12 16:19:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 80383 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BBD3CA04B6; Mon, 12 Oct 2020 18:20:52 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 49ED11D947; Mon, 12 Oct 2020 18:20:01 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 40F371D925 for ; Mon, 12 Oct 2020 18:19:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@nvidia.com) with SMTP; 12 Oct 2020 19:19:46 +0300 Received: from nvidia.com (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 09CGJjKP017440; Mon, 12 Oct 2020 19:19:46 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: thomasm@monjalon.net, stephen@networkplumber.org, ferruh.yigit@intel.com, olivier.matz@6wind.com, jerinjacobk@gmail.com, maxime.coquelin@redhat.com, david.marchand@redhat.com, arybchenko@solarflare.com Date: Mon, 12 Oct 2020 16:19:44 +0000 Message-Id: <1602519585-5194-9-git-send-email-viacheslavo@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> References: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> Subject: [dpdk-dev] [PATCH v3 8/9] net/mlx5: register multiple pool for Rx queue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The split feature for receiving packets was added to the mlx5 PMD, now Rx queue can receive the data to the buffers belonging to the different pools and the memory of all the involved pool must be registered for DMA operations in order to allow hardware to store the data. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_mr.c | 3 +++ drivers/net/mlx5/mlx5_trigger.c | 20 ++++++++++++-------- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index dbcf0aa..c308ecc 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -536,6 +536,9 @@ struct mr_update_mp_data { .ret = 0, }; + DRV_LOG(DEBUG, "Port %u Rx queue registering mp %s " + "having %u chunks.", dev->data->port_id, + mp->name, mp->nb_mem_chunks); rte_mempool_mem_iter(mp, mlx5_mr_update_mp_cb, &data); if (data.ret < 0 && rte_errno == ENXIO) { /* Mempool may have externally allocated memory. */ diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index e72e5fb..643e10f 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -145,18 +145,22 @@ dev->data->port_id, priv->sh->device_attr.max_sge); for (i = 0; i != priv->rxqs_n; ++i) { struct mlx5_rxq_ctrl *rxq_ctrl = mlx5_rxq_get(dev, i); - struct rte_mempool *mp; if (!rxq_ctrl) continue; if (rxq_ctrl->type == MLX5_RXQ_TYPE_STANDARD) { - /* Pre-register Rx mempool. */ - mp = mlx5_rxq_mprq_enabled(&rxq_ctrl->rxq) ? - rxq_ctrl->rxq.mprq_mp : rxq_ctrl->rxq.mp; - DRV_LOG(DEBUG, "Port %u Rx queue %u registering mp %s" - " having %u chunks.", dev->data->port_id, - rxq_ctrl->rxq.idx, mp->name, mp->nb_mem_chunks); - mlx5_mr_update_mp(dev, &rxq_ctrl->rxq.mr_ctrl, mp); + /* Pre-register Rx mempools. */ + if (mlx5_rxq_mprq_enabled(&rxq_ctrl->rxq)) { + mlx5_mr_update_mp(dev, &rxq_ctrl->rxq.mr_ctrl, + rxq_ctrl->rxq.mprq_mp); + } else { + uint32_t s; + + for (s = 0; s < rxq_ctrl->rxq.rxseg_n; s++) + mlx5_mr_update_mp + (dev, &rxq_ctrl->rxq.mr_ctrl, + rxq_ctrl->rxq.rxseg[s].mp); + } ret = rxq_alloc_elts(rxq_ctrl); if (ret) goto error; From patchwork Mon Oct 12 16:19:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 80384 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8D26FA04B6; Mon, 12 Oct 2020 18:21:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 024131D94E; Mon, 12 Oct 2020 18:20:03 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 4DB761D933 for ; Mon, 12 Oct 2020 18:19:52 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@nvidia.com) with SMTP; 12 Oct 2020 19:19:46 +0300 Received: from nvidia.com (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 09CGJjKQ017440; Mon, 12 Oct 2020 19:19:46 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: thomasm@monjalon.net, stephen@networkplumber.org, ferruh.yigit@intel.com, olivier.matz@6wind.com, jerinjacobk@gmail.com, maxime.coquelin@redhat.com, david.marchand@redhat.com, arybchenko@solarflare.com Date: Mon, 12 Oct 2020 16:19:45 +0000 Message-Id: <1602519585-5194-10-git-send-email-viacheslavo@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> References: <1602519585-5194-1-git-send-email-viacheslavo@nvidia.com> Subject: [dpdk-dev] [PATCH v3 9/9] net/mlx5: update Rx datapath to support split X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Only the regular rx_burst routine is updated to support split, because the vectorized ones does not support scatter and MPRQ does not support split at all. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_rxq.c | 11 +++++------ drivers/net/mlx5/mlx5_rxtx.c | 3 ++- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 4ec4677..2ebb265 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -210,9 +210,10 @@ /* Iterate on segments. */ for (i = 0; (i != elts_n); ++i) { + struct rte_eth_rxseg *seg = &rxq_ctrl->rxq.rxseg[i % sges_n]; struct rte_mbuf *buf; - buf = rte_pktmbuf_alloc(rxq_ctrl->rxq.mp); + buf = rte_pktmbuf_alloc(seg->mp); if (buf == NULL) { DRV_LOG(ERR, "port %u empty mbuf pool", PORT_ID(rxq_ctrl->priv)); @@ -225,12 +226,10 @@ MLX5_ASSERT(rte_pktmbuf_data_len(buf) == 0); MLX5_ASSERT(rte_pktmbuf_pkt_len(buf) == 0); MLX5_ASSERT(!buf->next); - /* Only the first segment keeps headroom. */ - if (i % sges_n) - SET_DATA_OFF(buf, 0); + SET_DATA_OFF(buf, seg->offset); PORT(buf) = rxq_ctrl->rxq.port_id; - DATA_LEN(buf) = rte_pktmbuf_tailroom(buf); - PKT_LEN(buf) = DATA_LEN(buf); + DATA_LEN(buf) = seg->length; + PKT_LEN(buf) = seg->length; NB_SEGS(buf) = 1; (*rxq_ctrl->rxq.elts)[i] = buf; } diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index b530ff4..dd84249 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -1334,7 +1334,8 @@ enum mlx5_txcmp_code { rte_prefetch0(seg); rte_prefetch0(cqe); rte_prefetch0(wqe); - rep = rte_mbuf_raw_alloc(rxq->mp); + /* Allocate the buf from the same pool. */ + rep = rte_mbuf_raw_alloc(seg->pool); if (unlikely(rep == NULL)) { ++rxq->stats.rx_nombuf; if (!pkt) {