Message ID | 1563786795-14027-1-git-send-email-matan@mellanox.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D8F311BDF1; Mon, 22 Jul 2019 11:13:33 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 6225F1BC07 for <dev@dpdk.org>; Mon, 22 Jul 2019 11:13:24 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE2 (envelope-from matan@mellanox.com) with ESMTPS (AES256-SHA encrypted); 22 Jul 2019 12:13:22 +0300 Received: from pegasus07.mtr.labs.mlnx (pegasus07.mtr.labs.mlnx [10.210.16.112]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x6M9DMjP010084; Mon, 22 Jul 2019 12:13:22 +0300 From: Matan Azrad <matan@mellanox.com> To: Shahaf Shuler <shahafs@mellanox.com>, Yongseok Koh <yskoh@mellanox.com>, Viacheslav Ovsiienko <viacheslavo@mellanox.com> Cc: dev@dpdk.org, Dekel Peled <dekelp@mellanox.com> Date: Mon, 22 Jul 2019 09:12:47 +0000 Message-Id: <1563786795-14027-1-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 Subject: [dpdk-dev] [PATCH 00/28] net/mlx5: support LRO X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
net/mlx5: support LRO
|
|
Message
Matan Azrad
July 22, 2019, 9:12 a.m. UTC
Introduction: LRO (Large Receive Offload) is intended to reduce host CPU overhead when processing Rx TCP packets. LRO works by aggregating multiple incoming packets from a single stream into a larger buffer, before they are passed higher up the networking stack. Thus reducing the number of packets that have to be processed. Use: MLX5 PMD will query the HCA capabilities on initialization to check if LRO is supported and can be used. LRO in MLX5 PMD is intended for use by applications using a relatively small number of flows. LRO support can be enabled only per port. In each LRO session, packets of the same flow will be coalesced until one of the following occur: * Buffer size limit is exceeded. * Session timeout is exceeded. * Packet from a different flow is received on the same queue. When LRO session ends the coalesced packet is passed to the PMD, which will update the header fields before passing the packet to the application. For efficient memory utilization, the MPRQ mechanism is used. Support of Non-LRO flows will not be impacted. Existing API: Offload capability DEV_RX_OFFLOAD_TCP_LRO will be used to indicate device supports LRO. testpmd command-line option "-enable-lro" will be used to request LRO feature enable on application start. testpmd rx_offload "tcp_lro" on or off will be used to request LRO feature enable or disable during application runtime. Offload flag PKT_RX_LRO will be used. This flag can be set in Rx mbuf to indicate this is a LRO coalesced packet. New API: PMD configuration parameter lro_timeout_usec will be added. This parameter can be used by application to select LRO session timeout (in microseconds). If this value is not specified, the minimal value supported by device will be used. Known limitations: mbuf head-room is zero for any packet if LRO is configured in the port. Keep CRC offload cannot be supported with LRO. CQE compression is not supported with LRO. Dekel Peled (23): net/mlx5: remove redundant item from union net/mlx5: add LRO APIs and initial settings net/mlx5: support LRO caps query using devx API net/mlx5: glue func for queue query using new API net/mlx5: glue function for action using new API net/mlx5: check conditions to enable LRO net/mlx5: support Tx interface query using new API net/mlx5: update Tx queue create for LRO net/mlx5: create advanced RxQ object using new API net/mlx5: modify advanced RxQ object using new API net/mlx5: create advanced Rx object using new API net/mlx5: create advanced RxQ table using new API net/mlx5: allocate door-bells using new API net/mlx5: rename RxQ verbs to general RxQ object net/mlx5: rename verbs indirection table to obj net/mlx5: rename hash RxQ verbs to general net/mlx5: update queue state modify function net/mlx5: store protection domain number on create net/mlx5: func to create Rx verbs completion queue net/mlx5: function to create Rx verbs work queue net/mlx5: create advanced RxQ using new API net/mlx5: support LRO with single RxQ object doc: update MLX5 doc and release notes with LRO Matan Azrad (5): net/mlx5: replace the external mbuf shared memory net/mlx5: update LRO fields in completion entry net/mlx5: handle LRO packets in Rx queue net/mlx5: zero the LRO mbuf headroom net/mlx5: adjust the maximum LRO message size doc/guides/nics/features/mlx5.ini | 1 + doc/guides/nics/mlx5.rst | 14 + doc/guides/rel_notes/release_19_08.rst | 2 +- drivers/net/mlx5/Makefile | 5 + drivers/net/mlx5/meson.build | 2 + drivers/net/mlx5/mlx5.c | 223 ++++++- drivers/net/mlx5/mlx5.h | 160 ++++- drivers/net/mlx5/mlx5_devx_cmds.c | 326 +++++++++ drivers/net/mlx5/mlx5_ethdev.c | 14 +- drivers/net/mlx5/mlx5_flow.h | 6 + drivers/net/mlx5/mlx5_flow_dv.c | 28 +- drivers/net/mlx5/mlx5_flow_verbs.c | 3 +- drivers/net/mlx5/mlx5_glue.c | 33 + drivers/net/mlx5/mlx5_glue.h | 6 +- drivers/net/mlx5/mlx5_prm.h | 379 ++++++++++- drivers/net/mlx5/mlx5_rxq.c | 1135 ++++++++++++++++++++++---------- drivers/net/mlx5/mlx5_rxtx.c | 167 ++++- drivers/net/mlx5/mlx5_rxtx.h | 80 ++- drivers/net/mlx5/mlx5_rxtx_vec.h | 6 +- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 16 +- drivers/net/mlx5/mlx5_trigger.c | 12 +- drivers/net/mlx5/mlx5_txq.c | 27 +- drivers/net/mlx5/mlx5_vlan.c | 32 +- 23 files changed, 2197 insertions(+), 480 deletions(-)
Comments
Hi, > -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Matan Azrad > Sent: Monday, July 22, 2019 12:13 PM > To: Shahaf Shuler <shahafs@mellanox.com>; Yongseok Koh > <yskoh@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com> > Cc: dev@dpdk.org; Dekel Peled <dekelp@mellanox.com> > Subject: [dpdk-dev] [PATCH 00/28] net/mlx5: support LRO > > Introduction: > LRO (Large Receive Offload) is intended to reduce host CPU overhead when > processing Rx TCP packets. > LRO works by aggregating multiple incoming packets from a single stream > into a larger buffer, before they are passed higher up the networking stack. > Thus reducing the number of packets that have to be processed. > > Use: > MLX5 PMD will query the HCA capabilities on initialization to check if LRO is > supported and can be used. > LRO in MLX5 PMD is intended for use by applications using a relatively small > number of flows. > LRO support can be enabled only per port. > In each LRO session, packets of the same flow will be coalesced until one of > the following occur: > * Buffer size limit is exceeded. > * Session timeout is exceeded. > * Packet from a different flow is received on the same queue. > > When LRO session ends the coalesced packet is passed to the PMD, which > will update the header fields before passing the packet to the application. > For efficient memory utilization, the MPRQ mechanism is used. > Support of Non-LRO flows will not be impacted. > > Existing API: > Offload capability DEV_RX_OFFLOAD_TCP_LRO will be used to indicate > device supports LRO. > testpmd command-line option "-enable-lro" will be used to request LRO > feature enable on application start. > testpmd rx_offload "tcp_lro" on or off will be used to request LRO feature > enable or disable during application runtime. > Offload flag PKT_RX_LRO will be used. This flag can be set in Rx mbuf to > indicate this is a LRO coalesced packet. > > New API: > PMD configuration parameter lro_timeout_usec will be added. > This parameter can be used by application to select LRO session timeout (in > microseconds). > If this value is not specified, the minimal value supported by device will be > used. > > Known limitations: > mbuf head-room is zero for any packet if LRO is configured in the port. > Keep CRC offload cannot be supported with LRO. > CQE compression is not supported with LRO. > > Dekel Peled (23): > net/mlx5: remove redundant item from union > net/mlx5: add LRO APIs and initial settings > net/mlx5: support LRO caps query using devx API > net/mlx5: glue func for queue query using new API > net/mlx5: glue function for action using new API > net/mlx5: check conditions to enable LRO > net/mlx5: support Tx interface query using new API > net/mlx5: update Tx queue create for LRO > net/mlx5: create advanced RxQ object using new API > net/mlx5: modify advanced RxQ object using new API > net/mlx5: create advanced Rx object using new API > net/mlx5: create advanced RxQ table using new API > net/mlx5: allocate door-bells using new API > net/mlx5: rename RxQ verbs to general RxQ object > net/mlx5: rename verbs indirection table to obj > net/mlx5: rename hash RxQ verbs to general > net/mlx5: update queue state modify function > net/mlx5: store protection domain number on create > net/mlx5: func to create Rx verbs completion queue > net/mlx5: function to create Rx verbs work queue > net/mlx5: create advanced RxQ using new API > net/mlx5: support LRO with single RxQ object > doc: update MLX5 doc and release notes with LRO > > Matan Azrad (5): > net/mlx5: replace the external mbuf shared memory > net/mlx5: update LRO fields in completion entry > net/mlx5: handle LRO packets in Rx queue > net/mlx5: zero the LRO mbuf headroom > net/mlx5: adjust the maximum LRO message size > > doc/guides/nics/features/mlx5.ini | 1 + > doc/guides/nics/mlx5.rst | 14 + > doc/guides/rel_notes/release_19_08.rst | 2 +- > drivers/net/mlx5/Makefile | 5 + > drivers/net/mlx5/meson.build | 2 + > drivers/net/mlx5/mlx5.c | 223 ++++++- > drivers/net/mlx5/mlx5.h | 160 ++++- > drivers/net/mlx5/mlx5_devx_cmds.c | 326 +++++++++ > drivers/net/mlx5/mlx5_ethdev.c | 14 +- > drivers/net/mlx5/mlx5_flow.h | 6 + > drivers/net/mlx5/mlx5_flow_dv.c | 28 +- > drivers/net/mlx5/mlx5_flow_verbs.c | 3 +- > drivers/net/mlx5/mlx5_glue.c | 33 + > drivers/net/mlx5/mlx5_glue.h | 6 +- > drivers/net/mlx5/mlx5_prm.h | 379 ++++++++++- > drivers/net/mlx5/mlx5_rxq.c | 1135 ++++++++++++++++++++++------- > --- > drivers/net/mlx5/mlx5_rxtx.c | 167 ++++- > drivers/net/mlx5/mlx5_rxtx.h | 80 ++- > drivers/net/mlx5/mlx5_rxtx_vec.h | 6 +- > drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 16 +- > drivers/net/mlx5/mlx5_trigger.c | 12 +- > drivers/net/mlx5/mlx5_txq.c | 27 +- > drivers/net/mlx5/mlx5_vlan.c | 32 +- > 23 files changed, 2197 insertions(+), 480 deletions(-) > > -- > 1.8.3.1 Series applied to next-net-mlx, Kindest regards, Raslan Darawsheh
On 7/22/2019 10:12 AM, Matan Azrad wrote: > Introduction: > LRO (Large Receive Offload) is intended to reduce host CPU overhead when processing Rx TCP packets. > LRO works by aggregating multiple incoming packets from a single stream into a larger buffer, before they are passed higher up the networking stack. Thus reducing the number of packets that have to be processed. > > Use: > MLX5 PMD will query the HCA capabilities on initialization to check if LRO is supported and can be used. > LRO in MLX5 PMD is intended for use by applications using a relatively small number of flows. > LRO support can be enabled only per port. > In each LRO session, packets of the same flow will be coalesced until one of the following occur: > * Buffer size limit is exceeded. > * Session timeout is exceeded. > * Packet from a different flow is received on the same queue. > > When LRO session ends the coalesced packet is passed to the PMD, which will update the header fields before passing the packet to the application. > For efficient memory utilization, the MPRQ mechanism is used. > Support of Non-LRO flows will not be impacted. > > Existing API: > Offload capability DEV_RX_OFFLOAD_TCP_LRO will be used to indicate device supports LRO. > testpmd command-line option "-enable-lro" will be used to request LRO feature enable on application start. > testpmd rx_offload "tcp_lro" on or off will be used to request LRO feature enable or disable during application runtime. > Offload flag PKT_RX_LRO will be used. This flag can be set in Rx mbuf to indicate this is a LRO coalesced packet. > > New API: > PMD configuration parameter lro_timeout_usec will be added. > This parameter can be used by application to select LRO session timeout (in microseconds). > If this value is not specified, the minimal value supported by device will be used. > > Known limitations: > mbuf head-room is zero for any packet if LRO is configured in the port. > Keep CRC offload cannot be supported with LRO. > CQE compression is not supported with LRO. > > Dekel Peled (23): > net/mlx5: remove redundant item from union > net/mlx5: add LRO APIs and initial settings > net/mlx5: support LRO caps query using devx API > net/mlx5: glue func for queue query using new API > net/mlx5: glue function for action using new API > net/mlx5: check conditions to enable LRO > net/mlx5: support Tx interface query using new API > net/mlx5: update Tx queue create for LRO > net/mlx5: create advanced RxQ object using new API > net/mlx5: modify advanced RxQ object using new API > net/mlx5: create advanced Rx object using new API > net/mlx5: create advanced RxQ table using new API > net/mlx5: allocate door-bells using new API > net/mlx5: rename RxQ verbs to general RxQ object > net/mlx5: rename verbs indirection table to obj > net/mlx5: rename hash RxQ verbs to general > net/mlx5: update queue state modify function > net/mlx5: store protection domain number on create > net/mlx5: func to create Rx verbs completion queue > net/mlx5: function to create Rx verbs work queue > net/mlx5: create advanced RxQ using new API > net/mlx5: support LRO with single RxQ object > doc: update MLX5 doc and release notes with LRO > > Matan Azrad (5): > net/mlx5: replace the external mbuf shared memory > net/mlx5: update LRO fields in completion entry > net/mlx5: handle LRO packets in Rx queue > net/mlx5: zero the LRO mbuf headroom > net/mlx5: adjust the maximum LRO message size I am getting build error on patch by patch build, didn't dig to figure out which exact patch to fail, please investigate. And this patchset, adding a new feature, is sent on last day of the rc2, and merged in the same day, do you guys really sure it has been reviewed well? There are two group of build errors [1] & [2], both of them are observed with both gcc and clang. [1] .../drivers/net/mlx5/mlx5_rxq.c:2150:7: error: variable 'qp' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]... if (!tir) {... ^~~~... .../drivers/net/mlx5/mlx5_rxq.c:2191:6: note: uninitialized use occurs here... if (qp)... ^~... .../drivers/net/mlx5/mlx5_rxq.c:2150:3: note: remove the 'if' if its condition is always false... if (!tir) {... ^~~~~~~~~~~... .../drivers/net/mlx5/mlx5_rxq.c:2046:19: note: initialize the variable 'qp' to silence this warning... struct ibv_qp *qp;... ^... = NULL... 1 error generated.... [2] .../drivers/net/mlx5/mlx5.c:1450:17: error: incomplete definition of type 'struct mlx5dv_devx_umem' if (page->umem->umem_id == umem_id) ~~~~~~~~~~^ .../drivers/net/mlx5/mlx5_glue.h:64:8: note: forward declaration of 'struct mlx5dv_devx_umem' struct mlx5dv_devx_umem; ^ 1 error generated.
Hi Ferrue From: Ferruh Yigit > On 7/22/2019 10:12 AM, Matan Azrad wrote: > > Introduction: > > LRO (Large Receive Offload) is intended to reduce host CPU overhead > when processing Rx TCP packets. > > LRO works by aggregating multiple incoming packets from a single stream > into a larger buffer, before they are passed higher up the networking stack. > Thus reducing the number of packets that have to be processed. > > > > Use: > > MLX5 PMD will query the HCA capabilities on initialization to check if LRO is > supported and can be used. > > LRO in MLX5 PMD is intended for use by applications using a relatively small > number of flows. > > LRO support can be enabled only per port. > > In each LRO session, packets of the same flow will be coalesced until one of > the following occur: > > * Buffer size limit is exceeded. > > * Session timeout is exceeded. > > * Packet from a different flow is received on the same queue. > > > > When LRO session ends the coalesced packet is passed to the PMD, which > will update the header fields before passing the packet to the application. > > For efficient memory utilization, the MPRQ mechanism is used. > > Support of Non-LRO flows will not be impacted. > > > > Existing API: > > Offload capability DEV_RX_OFFLOAD_TCP_LRO will be used to indicate > device supports LRO. > > testpmd command-line option "-enable-lro" will be used to request LRO > feature enable on application start. > > testpmd rx_offload "tcp_lro" on or off will be used to request LRO feature > enable or disable during application runtime. > > Offload flag PKT_RX_LRO will be used. This flag can be set in Rx mbuf to > indicate this is a LRO coalesced packet. > > > > New API: > > PMD configuration parameter lro_timeout_usec will be added. > > This parameter can be used by application to select LRO session timeout (in > microseconds). > > If this value is not specified, the minimal value supported by device will be > used. > > > > Known limitations: > > mbuf head-room is zero for any packet if LRO is configured in the port. > > Keep CRC offload cannot be supported with LRO. > > CQE compression is not supported with LRO. > > > > Dekel Peled (23): > > net/mlx5: remove redundant item from union > > net/mlx5: add LRO APIs and initial settings > > net/mlx5: support LRO caps query using devx API > > net/mlx5: glue func for queue query using new API > > net/mlx5: glue function for action using new API > > net/mlx5: check conditions to enable LRO > > net/mlx5: support Tx interface query using new API > > net/mlx5: update Tx queue create for LRO > > net/mlx5: create advanced RxQ object using new API > > net/mlx5: modify advanced RxQ object using new API > > net/mlx5: create advanced Rx object using new API > > net/mlx5: create advanced RxQ table using new API > > net/mlx5: allocate door-bells using new API > > net/mlx5: rename RxQ verbs to general RxQ object > > net/mlx5: rename verbs indirection table to obj > > net/mlx5: rename hash RxQ verbs to general > > net/mlx5: update queue state modify function > > net/mlx5: store protection domain number on create > > net/mlx5: func to create Rx verbs completion queue > > net/mlx5: function to create Rx verbs work queue > > net/mlx5: create advanced RxQ using new API > > net/mlx5: support LRO with single RxQ object > > doc: update MLX5 doc and release notes with LRO > > > > Matan Azrad (5): > > net/mlx5: replace the external mbuf shared memory > > net/mlx5: update LRO fields in completion entry > > net/mlx5: handle LRO packets in Rx queue > > net/mlx5: zero the LRO mbuf headroom > > net/mlx5: adjust the maximum LRO message size > > I am getting build error on patch by patch build, didn't dig to figure out which > exact patch to fail, please investigate. > > And this patchset, adding a new feature, is sent on last day of the rc2, and > merged in the same day, do you guys really sure it has been reviewed well? Yes, most of the patches were reviewed by 2 guys, all the others by 1. It is in testing for all the week, so it is good enough for RC2. May be some fixes\enhancements in RC3. > There are two group of build errors [1] & [2], both of them are observed with > both gcc and clang. > > > [1] > .../drivers/net/mlx5/mlx5_rxq.c:2150:7: error: variable 'qp' is used > uninitialized whenever 'if' condition is true [-Werror,-Wsometimes- > uninitialized]... > if (!tir) {... > ^~~~... > .../drivers/net/mlx5/mlx5_rxq.c:2191:6: note: uninitialized use occurs here... > if (qp)... > ^~... > .../drivers/net/mlx5/mlx5_rxq.c:2150:3: note: remove the 'if' if its condition is > always false... > if (!tir) {... > ^~~~~~~~~~~... > .../drivers/net/mlx5/mlx5_rxq.c:2046:19: note: initialize the variable 'qp' to > silence this warning... > struct ibv_qp *qp;... > ^... > = NULL... > 1 error generated.... > > > [2] > .../drivers/net/mlx5/mlx5.c:1450:17: error: incomplete definition of type > 'struct mlx5dv_devx_umem' > if (page->umem->umem_id == umem_id) > ~~~~~~~~~~^ > .../drivers/net/mlx5/mlx5_glue.h:64:8: note: forward declaration of 'struct > mlx5dv_devx_umem' > struct mlx5dv_devx_umem; > ^ > 1 error generated. Will address it, now... Thanks