Message ID | 20240312075238.3319480-1-huangdengdui@huawei.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7116243C8E; Tue, 12 Mar 2024 08:52:42 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 591E040C35; Tue, 12 Mar 2024 08:52:42 +0100 (CET) Received: from szxga06-in.huawei.com (szxga06-in.huawei.com [45.249.212.32]) by mails.dpdk.org (Postfix) with ESMTP id C6A5C406FF for <dev@dpdk.org>; Tue, 12 Mar 2024 08:52:40 +0100 (CET) Received: from mail.maildlp.com (unknown [172.19.162.112]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4Tv5Sr1ZRXz1vwY4; Tue, 12 Mar 2024 15:51:56 +0800 (CST) Received: from dggpeml500011.china.huawei.com (unknown [7.185.36.84]) by mail.maildlp.com (Postfix) with ESMTPS id F14FC140485; Tue, 12 Mar 2024 15:52:38 +0800 (CST) Received: from localhost.huawei.com (10.50.165.33) by dggpeml500011.china.huawei.com (7.185.36.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Tue, 12 Mar 2024 15:52:38 +0800 From: Dengdui Huang <huangdengdui@huawei.com> To: <dev@dpdk.org> CC: <ferruh.yigit@amd.com>, <aman.deep.singh@intel.com>, <yuying.zhang@intel.com>, <thomas@monjalon.net>, <andrew.rybchenko@oktetlabs.ru>, <liuyonglong@huawei.com>, <fengchengwen@huawei.com>, <haijie1@huawei.com>, <lihuisong@huawei.com> Subject: [PATCH 0/3] support setting lanes Date: Tue, 12 Mar 2024 15:52:35 +0800 Message-ID: <20240312075238.3319480-1-huangdengdui@huawei.com> X-Mailer: git-send-email 2.33.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.50.165.33] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpeml500011.china.huawei.com (7.185.36.84) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org |
Series |
support setting lanes
|
|
Message
huangdengdui
March 12, 2024, 7:52 a.m. UTC
Some speeds can be achieved with different number of lanes. For example, 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. When use different lanes, the port cannot be up. This patch set add support setting lanes for ethdev In addition, modify the testpmd and hns3 driver to adapt to it. Dengdui Huang (3): ethdev: support setting lanes net/hns3: support setting lanes app/testpmd: support setting lanes app/test-pmd/cmdline.c | 110 ++++---- app/test-pmd/config.c | 60 +++-- doc/guides/rel_notes/release_24_03.rst | 11 +- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 3 +- drivers/net/bnxt/bnxt_ethdev.c | 3 +- drivers/net/hns3/hns3_cmd.h | 15 +- drivers/net/hns3/hns3_common.c | 3 +- drivers/net/hns3/hns3_ethdev.c | 112 ++++---- drivers/net/hns3/hns3_ethdev.h | 2 + lib/ethdev/ethdev_driver.h | 1 - lib/ethdev/ethdev_linux_ethtool.c | 101 +++++++- lib/ethdev/ethdev_private.h | 4 + lib/ethdev/ethdev_trace.h | 4 +- lib/ethdev/meson.build | 2 + lib/ethdev/rte_ethdev.c | 272 +++++++++++++++++++- lib/ethdev/rte_ethdev.h | 99 +++++-- lib/ethdev/version.map | 7 + 17 files changed, 635 insertions(+), 174 deletions(-)
Comments
12/03/2024 08:52, Dengdui Huang: > Some speeds can be achieved with different number of lanes. For example, > 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > When use different lanes, the port cannot be up. I'm not sure what you are referring to. I suppose it is not PCI lanes. Please could you link to an explanation of how a port is split in lanes? Which hardware does this?
On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > 12/03/2024 08:52, Dengdui Huang: > > Some speeds can be achieved with different number of lanes. For example, > > 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > > When use different lanes, the port cannot be up. > > I'm not sure what you are referring to. > I suppose it is not PCI lanes. > Please could you link to an explanation of how a port is split in lanes? > Which hardware does this? > > > This is a snapshot of 100Gb that the latest BCM576xx supports. 100Gb (NRZ: 25G per lane, 4 lanes) link speed 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed 100Gb (PAM4-112: 100G per lane, 1 lane) link speed Let the user feed in lanes=< integer value> and the NIC driver decides the matching combination speed x lanes that works. In future if a new speed is implemented with more than 8 lanes, there wouldn't be a need to touch this speed command. Using separate lane command would be a better alternative to support already shipped products and only new drivers would consider this lanes configuration, if applicable.
On Mon, 18 Mar 2024 14:26:33 -0700 Damodharam Ammepalli <damodharam.ammepalli@broadcom.com> wrote: > On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > > > 12/03/2024 08:52, Dengdui Huang: > > > Some speeds can be achieved with different number of lanes. For example, > > > 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > > > When use different lanes, the port cannot be up. > > > > I'm not sure what you are referring to. > > I suppose it is not PCI lanes. > > Please could you link to an explanation of how a port is split in lanes? > > Which hardware does this? > > > > > > > This is a snapshot of 100Gb that the latest BCM576xx supports. > 100Gb (NRZ: 25G per lane, 4 lanes) link speed > 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed > 100Gb (PAM4-112: 100G per lane, 1 lane) link speed > > Let the user feed in lanes=< integer value> and the NIC driver decides > the matching combination speed x lanes that works. In future if a new speed > is implemented with more than 8 lanes, there wouldn't be a need > to touch this speed command. Using separate lane command would > be a better alternative to support already shipped products and only new > drivers would consider this lanes configuration, if applicable. > The DPDK does not need more driver specific knobs. Shouldn't the PMD be able to auto negotiate the speed? What does Linux do?
On Mon, Mar 18, 2024 at 2:42 PM Stephen Hemminger <stephen@networkplumber.org> wrote: > > On Mon, 18 Mar 2024 14:26:33 -0700 > Damodharam Ammepalli <damodharam.ammepalli@broadcom.com> wrote: > > > On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > > > > > 12/03/2024 08:52, Dengdui Huang: > > > > Some speeds can be achieved with different number of lanes. For example, > > > > 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > > > > When use different lanes, the port cannot be up. > > > > > > I'm not sure what you are referring to. > > > I suppose it is not PCI lanes. > > > Please could you link to an explanation of how a port is split in lanes? > > > Which hardware does this? > > > > > > > > > > > This is a snapshot of 100Gb that the latest BCM576xx supports. > > 100Gb (NRZ: 25G per lane, 4 lanes) link speed > > 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed > > 100Gb (PAM4-112: 100G per lane, 1 lane) link speed > > > > Let the user feed in lanes=< integer value> and the NIC driver decides > > the matching combination speed x lanes that works. In future if a new speed > > is implemented with more than 8 lanes, there wouldn't be a need > > to touch this speed command. Using separate lane command would > > be a better alternative to support already shipped products and only new > > drivers would consider this lanes configuration, if applicable. > > > > The DPDK does not need more driver specific knobs. > Shouldn't the PMD be able to auto negotiate the speed? Yes. Its possible to auto negotiate. And that's the default. Even for the lane count, a default number can be arrived at. > What does Linux do? ethtool has been extended a while ago to allow configuring the number of lanes along with speed and other settings. But as usual, autoneg is possible.
18/03/2024 22:26, Damodharam Ammepalli: > On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > > > 12/03/2024 08:52, Dengdui Huang: > > > Some speeds can be achieved with different number of lanes. For example, > > > 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > > > When use different lanes, the port cannot be up. > > > > I'm not sure what you are referring to. > > I suppose it is not PCI lanes. > > Please could you link to an explanation of how a port is split in lanes? > > Which hardware does this? > > This is a snapshot of 100Gb that the latest BCM576xx supports. > 100Gb (NRZ: 25G per lane, 4 lanes) link speed > 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed > 100Gb (PAM4-112: 100G per lane, 1 lane) link speed > > Let the user feed in lanes=< integer value> and the NIC driver decides > the matching combination speed x lanes that works. In future if a new speed > is implemented with more than 8 lanes, there wouldn't be a need > to touch this speed command. Using separate lane command would > be a better alternative to support already shipped products and only new > drivers would consider this lanes configuration, if applicable. Sorry it does not provide enough explanations. What is a lane? How does it work? Is it only for Broadcom devices? Do you know other devices?
On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote: > On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: >> >> 12/03/2024 08:52, Dengdui Huang: >>> Some speeds can be achieved with different number of lanes. For example, >>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. >>> When use different lanes, the port cannot be up. >> >> I'm not sure what you are referring to. >> I suppose it is not PCI lanes. >> Please could you link to an explanation of how a port is split in lanes? >> Which hardware does this? >> >> >> > This is a snapshot of 100Gb that the latest BCM576xx supports. > 100Gb (NRZ: 25G per lane, 4 lanes) link speed > 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed > 100Gb (PAM4-112: 100G per lane, 1 lane) link speed > > Let the user feed in lanes=< integer value> and the NIC driver decides > the matching combination speed x lanes that works. In future if a new speed > is implemented with more than 8 lanes, there wouldn't be a need > to touch this speed command. Using separate lane command would > be a better alternative to support already shipped products and only new > drivers would consider this lanes configuration, if applicable. > As far as I understand, lane is related to the physical layer of the NIC, there are multiple copies of transmitter, receiver, modulator HW block and each set called as a 'lane' and multiple lanes work together to achieve desired speed. (please correct me if this is wrong). Why not just configuring the speed is not enough? Why user needs to know the detail and configuration of the lanes? Will it work if driver/device configure the "speed x lane" internally for the requested speed? Is there a benefit to force specific lane count for a specific speed (like power optimization, just a wild guess)? And +1 for auto-negotiation if possible.
On 2024/3/20 20:31, Ferruh Yigit wrote: > On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote: >> On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: >>> >>> 12/03/2024 08:52, Dengdui Huang: >>>> Some speeds can be achieved with different number of lanes. For example, >>>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. >>>> When use different lanes, the port cannot be up. >>> >>> I'm not sure what you are referring to. >>> I suppose it is not PCI lanes. >>> Please could you link to an explanation of how a port is split in lanes? >>> Which hardware does this? >>> >>> >>> >> This is a snapshot of 100Gb that the latest BCM576xx supports. >> 100Gb (NRZ: 25G per lane, 4 lanes) link speed >> 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed >> 100Gb (PAM4-112: 100G per lane, 1 lane) link speed >> >> Let the user feed in lanes=< integer value> and the NIC driver decides >> the matching combination speed x lanes that works. In future if a new speed >> is implemented with more than 8 lanes, there wouldn't be a need >> to touch this speed command. Using separate lane command would >> be a better alternative to support already shipped products and only new >> drivers would consider this lanes configuration, if applicable. >> > > As far as I understand, lane is related to the physical layer of the > NIC, there are multiple copies of transmitter, receiver, modulator HW > block and each set called as a 'lane' and multiple lanes work together > to achieve desired speed. (please correct me if this is wrong). > > Why not just configuring the speed is not enough? Why user needs to know > the detail and configuration of the lanes? > Will it work if driver/device configure the "speed x lane" internally > for the requested speed? > > Is there a benefit to force specific lane count for a specific speed > (like power optimization, just a wild guess)? > > > And +1 for auto-negotiation if possible. As you said above,,multiple lanes work together to achieve desired speed. For example, the following solutions can be used to implement 100G: 1、Combines four 25G lanes 2、Combines two 50G lanes 3、A single 100G lane It is assumed that two ports are interconnected and the two ports support the foregoing three solutions. But, we just configured the speed to 100G and one port uses four 25G lanes by default and the other port uses two 50G lanes by default, the port cannot be up. In this case, we need to configure the two ports to use the same solutions (for example, uses two 50G lanes) so that the ports can be up. ethtool has supported lanes configuration a while ago.[1] [1] https://lore.kernel.org/netdev/20201010154119.3537085-1-idosch@idosch.org/T/
21/03/2024 03:02, huangdengdui: > > On 2024/3/20 20:31, Ferruh Yigit wrote: > > On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote: > >> On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: > >>> > >>> 12/03/2024 08:52, Dengdui Huang: > >>>> Some speeds can be achieved with different number of lanes. For example, > >>>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > >>>> When use different lanes, the port cannot be up. > >>> > >>> I'm not sure what you are referring to. > >>> I suppose it is not PCI lanes. > >>> Please could you link to an explanation of how a port is split in lanes? > >>> Which hardware does this? > >>> > >> This is a snapshot of 100Gb that the latest BCM576xx supports. > >> 100Gb (NRZ: 25G per lane, 4 lanes) link speed > >> 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed > >> 100Gb (PAM4-112: 100G per lane, 1 lane) link speed > >> > >> Let the user feed in lanes=< integer value> and the NIC driver decides > >> the matching combination speed x lanes that works. In future if a new speed > >> is implemented with more than 8 lanes, there wouldn't be a need > >> to touch this speed command. Using separate lane command would > >> be a better alternative to support already shipped products and only new > >> drivers would consider this lanes configuration, if applicable. > >> > > > > As far as I understand, lane is related to the physical layer of the > > NIC, there are multiple copies of transmitter, receiver, modulator HW > > block and each set called as a 'lane' and multiple lanes work together > > to achieve desired speed. (please correct me if this is wrong). > > > > Why not just configuring the speed is not enough? Why user needs to know > > the detail and configuration of the lanes? > > Will it work if driver/device configure the "speed x lane" internally > > for the requested speed? > > > > Is there a benefit to force specific lane count for a specific speed > > (like power optimization, just a wild guess)? > > > > > > And +1 for auto-negotiation if possible. > > As you said above,,multiple lanes work together to achieve desired speed. > For example, the following solutions can be used to implement 100G: > 1、Combines four 25G lanes > 2、Combines two 50G lanes > 3、A single 100G lane > > It is assumed that two ports are interconnected and the two ports support > the foregoing three solutions. But, we just configured the speed to 100G and > one port uses four 25G lanes by default and the other port uses two 50G lanes > by default, the port cannot be up. In this case, we need to configure the > two ports to use the same solutions (for example, uses two 50G lanes) > so that the ports can be up. Why this config is not OK? How do we know? Really I have a very bad feeling about this feature.
On 2024/3/21 16:28, Thomas Monjalon wrote: > 21/03/2024 03:02, huangdengdui: >> >> On 2024/3/20 20:31, Ferruh Yigit wrote: >>> On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote: >>>> On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: >>>>> >>>>> 12/03/2024 08:52, Dengdui Huang: >>>>>> Some speeds can be achieved with different number of lanes. For example, >>>>>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. >>>>>> When use different lanes, the port cannot be up. >>>>> >>>>> I'm not sure what you are referring to. >>>>> I suppose it is not PCI lanes. >>>>> Please could you link to an explanation of how a port is split in lanes? >>>>> Which hardware does this? >>>>> >>>> This is a snapshot of 100Gb that the latest BCM576xx supports. >>>> 100Gb (NRZ: 25G per lane, 4 lanes) link speed >>>> 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed >>>> 100Gb (PAM4-112: 100G per lane, 1 lane) link speed >>>> >>>> Let the user feed in lanes=< integer value> and the NIC driver decides >>>> the matching combination speed x lanes that works. In future if a new speed >>>> is implemented with more than 8 lanes, there wouldn't be a need >>>> to touch this speed command. Using separate lane command would >>>> be a better alternative to support already shipped products and only new >>>> drivers would consider this lanes configuration, if applicable. >>>> >>> >>> As far as I understand, lane is related to the physical layer of the >>> NIC, there are multiple copies of transmitter, receiver, modulator HW >>> block and each set called as a 'lane' and multiple lanes work together >>> to achieve desired speed. (please correct me if this is wrong). >>> >>> Why not just configuring the speed is not enough? Why user needs to know >>> the detail and configuration of the lanes? >>> Will it work if driver/device configure the "speed x lane" internally >>> for the requested speed? >>> >>> Is there a benefit to force specific lane count for a specific speed >>> (like power optimization, just a wild guess)? >>> >>> >>> And +1 for auto-negotiation if possible. >> >> As you said above,,multiple lanes work together to achieve desired speed. >> For example, the following solutions can be used to implement 100G: >> 1、Combines four 25G lanes >> 2、Combines two 50G lanes >> 3、A single 100G lane >> >> It is assumed that two ports are interconnected and the two ports support >> the foregoing three solutions. But, we just configured the speed to 100G and >> one port uses four 25G lanes by default and the other port uses two 50G lanes >> by default, the port cannot be up. In this case, we need to configure the >> two ports to use the same solutions (for example, uses two 50G lanes) >> so that the ports can be up. > > Why this config is not OK? How do we know? > Really I have a very bad feeling about this feature. > > Sorry, I don't quite understand your question. Are you asking why cannot be up when one port uses four 25G lanes and the other port uses two 50G lanes? 100GBASE-SR2 (two 50G lanes) and 100GBASE-SR4 (four 25G lanes) have different standards at the physical layer.[1] So it's not possible to communicate. Configuring lanes can help the driver choose the same standard. [1] https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9844436
On Fri, Mar 22, 2024 at 7:58 AM huangdengdui <huangdengdui@huawei.com> wrote: > > > > On 2024/3/21 16:28, Thomas Monjalon wrote: > > 21/03/2024 03:02, huangdengdui: > >> > >> On 2024/3/20 20:31, Ferruh Yigit wrote: > >>> On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote: > >>>> On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: > >>>>> > >>>>> 12/03/2024 08:52, Dengdui Huang: > >>>>>> Some speeds can be achieved with different number of lanes. For example, > >>>>>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > >>>>>> When use different lanes, the port cannot be up. > >>>>> > >>>>> I'm not sure what you are referring to. > >>>>> I suppose it is not PCI lanes. > >>>>> Please could you link to an explanation of how a port is split in lanes? > >>>>> Which hardware does this? > >>>>> > >>>> This is a snapshot of 100Gb that the latest BCM576xx supports. > >>>> 100Gb (NRZ: 25G per lane, 4 lanes) link speed > >>>> 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed > >>>> 100Gb (PAM4-112: 100G per lane, 1 lane) link speed > >>>> > >>>> Let the user feed in lanes=< integer value> and the NIC driver decides > >>>> the matching combination speed x lanes that works. In future if a new speed > >>>> is implemented with more than 8 lanes, there wouldn't be a need > >>>> to touch this speed command. Using separate lane command would > >>>> be a better alternative to support already shipped products and only new > >>>> drivers would consider this lanes configuration, if applicable. > >>>> > >>> > >>> As far as I understand, lane is related to the physical layer of the > >>> NIC, there are multiple copies of transmitter, receiver, modulator HW > >>> block and each set called as a 'lane' and multiple lanes work together > >>> to achieve desired speed. (please correct me if this is wrong). > >>> > >>> Why not just configuring the speed is not enough? Why user needs to know > >>> the detail and configuration of the lanes? > >>> Will it work if driver/device configure the "speed x lane" internally > >>> for the requested speed? > >>> > >>> Is there a benefit to force specific lane count for a specific speed > >>> (like power optimization, just a wild guess)? > >>> > >>> > >>> And +1 for auto-negotiation if possible. > >> > >> As you said above,,multiple lanes work together to achieve desired speed. > >> For example, the following solutions can be used to implement 100G: > >> 1、Combines four 25G lanes > >> 2、Combines two 50G lanes > >> 3、A single 100G lane > >> > >> It is assumed that two ports are interconnected and the two ports support > >> the foregoing three solutions. But, we just configured the speed to 100G and > >> one port uses four 25G lanes by default and the other port uses two 50G lanes > >> by default, the port cannot be up. In this case, we need to configure the > >> two ports to use the same solutions (for example, uses two 50G lanes) > >> so that the ports can be up. > > > > Why this config is not OK? How do we know? > > Really I have a very bad feeling about this feature. > > > > > Sorry, I don't quite understand your question. > Are you asking why cannot be up when one port uses four 25G lanes and the other port uses two 50G lanes? > > 100GBASE-SR2 (two 50G lanes) and 100GBASE-SR4 (four 25G lanes) have different standards at the physical layer.[1] > So it's not possible to communicate. Configuring lanes can help the driver choose the same standard. Typically, low-level drivers like FW configure this. For example, If FW configures, 100G port as 100GBASE-SR2 then two ethdev(port 0 and port1) will show up. Now, assume if we expose this API and Can end user configure port 1 as 25G lines if so, a) What happens to port0 and it states? b) Will port2, port3 will show up after issuing this API(As end user configured 25Gx4 for 100G)? Will application needs to hotplug to get use ports. > https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9844436
On Thu, Mar 21, 2024 at 9:39 PM Jerin Jacob <jerinjacobk@gmail.com> wrote: > > On Fri, Mar 22, 2024 at 7:58 AM huangdengdui <huangdengdui@huawei.com> wrote: > > > > > > > > On 2024/3/21 16:28, Thomas Monjalon wrote: > > > 21/03/2024 03:02, huangdengdui: > > >> > > >> On 2024/3/20 20:31, Ferruh Yigit wrote: > > >>> On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote: > > >>>> On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > >>>>> > > >>>>> 12/03/2024 08:52, Dengdui Huang: > > >>>>>> Some speeds can be achieved with different number of lanes. For example, > > >>>>>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > > >>>>>> When use different lanes, the port cannot be up. > > >>>>> > > >>>>> I'm not sure what you are referring to. > > >>>>> I suppose it is not PCI lanes. > > >>>>> Please could you link to an explanation of how a port is split in lanes? > > >>>>> Which hardware does this? > > >>>>> > > >>>> This is a snapshot of 100Gb that the latest BCM576xx supports. > > >>>> 100Gb (NRZ: 25G per lane, 4 lanes) link speed > > >>>> 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed > > >>>> 100Gb (PAM4-112: 100G per lane, 1 lane) link speed > > >>>> > > >>>> Let the user feed in lanes=< integer value> and the NIC driver decides > > >>>> the matching combination speed x lanes that works. In future if a new speed > > >>>> is implemented with more than 8 lanes, there wouldn't be a need > > >>>> to touch this speed command. Using separate lane command would > > >>>> be a better alternative to support already shipped products and only new > > >>>> drivers would consider this lanes configuration, if applicable. > > >>>> > > >>> > > >>> As far as I understand, lane is related to the physical layer of the > > >>> NIC, there are multiple copies of transmitter, receiver, modulator HW > > >>> block and each set called as a 'lane' and multiple lanes work together > > >>> to achieve desired speed. (please correct me if this is wrong). > > >>> > > >>> Why not just configuring the speed is not enough? Why user needs to know > > >>> the detail and configuration of the lanes? > > >>> Will it work if driver/device configure the "speed x lane" internally > > >>> for the requested speed? > > >>> > > >>> Is there a benefit to force specific lane count for a specific speed > > >>> (like power optimization, just a wild guess)? > > >>> > > >>> > > >>> And +1 for auto-negotiation if possible. > > >> > > >> As you said above,,multiple lanes work together to achieve desired speed. > > >> For example, the following solutions can be used to implement 100G: > > >> 1、Combines four 25G lanes > > >> 2、Combines two 50G lanes > > >> 3、A single 100G lane > > >> > > >> It is assumed that two ports are interconnected and the two ports support > > >> the foregoing three solutions. But, we just configured the speed to 100G and > > >> one port uses four 25G lanes by default and the other port uses two 50G lanes > > >> by default, the port cannot be up. In this case, we need to configure the > > >> two ports to use the same solutions (for example, uses two 50G lanes) > > >> so that the ports can be up. > > > > > > Why this config is not OK? How do we know? > > > Really I have a very bad feeling about this feature. > > > > > > > > Sorry, I don't quite understand your question. > > Are you asking why cannot be up when one port uses four 25G lanes and the other port uses two 50G lanes? > > > > 100GBASE-SR2 (two 50G lanes) and 100GBASE-SR4 (four 25G lanes) have different standards at the physical layer.[1] > > So it's not possible to communicate. Configuring lanes can help the driver choose the same standard. > > Typically, low-level drivers like FW configure this. > > For example, If FW configures, 100G port as 100GBASE-SR2 then two > ethdev(port 0 and port1) will show up. > Now, assume if we expose this API and Can end user configure port 1 as > 25G lines if so, > a) What happens to port0 and it states? There should be no impact to port0. > b) Will port2, port3 will show up after issuing this API(As end user > configured 25Gx4 for 100G)? Will application needs to hotplug to get > use ports. No. The port count does not change. Nor does the number of PCI functions seen by the host. Unless designed otherwise. Changing the lane count does not change anything in physical terms. What changes is the modulation or the signaling scheme. The number of lanes which can be supported is determined by the PHY itself and the cables used and needs to be negotiated appropriately with the remote partner - which is just like using forced Ethernet Speed instead of auto-negotiated speeds. I tried to search for some links, but it has not been easy. So let me try to put something down. Sorry for the long mail. In the above example, if the 100G port is configured for 2 lanes, the SerDes in the PHY is configured such that each lane communicates at 50G with the remote partner. If the port is configured for 4 lanes, the internal SerDes is programmed to use a lower 25G signaling scheme. Each lane provides a parallel data transmission path and are typically implemented using multiple pairs of copper wires or optical fibers. When the lane configuration of the Ethernet port is changed, the port speed does not change. But the PHY FW uses a suitable modulation scheme to achieve the total aggregate port speed. The choice of modulation schemes include NRZ and PAM4. Typically 100G uses four lanes at 25 Gbps each and happens to be NRZ. For higher Ethernet speeds, the shift has been to use PAM4 signaling which allows twice the transmission rates compared to NRZ signaling. So the same 100G port can be configured to send signals to the remote partner using two lanes at 50G + 50G, or just 1 lane operating at 100G. It gets interesting as the speeds increase. 200G can be achieved with 4 lanes at 50G or 2 lanes at 100G. PAM4 offers improved bandwidth efficiency by encoding two bits per symbol compared to NRZ's one bit per symbol. The Ethernet standards committee I think has already completed work on 200G speed per lane. BTW - PAM4 stands for 4-Level Pulse Amplitude Modulation. NRZ stands for Non-Return-to-Zero. Thanks Ajit > > > https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9844436
On Fri, Mar 22, 2024 at 10:56 AM Ajit Khaparde <ajit.khaparde@broadcom.com> wrote: > > On Thu, Mar 21, 2024 at 9:39 PM Jerin Jacob <jerinjacobk@gmail.com> wrote: > > > > On Fri, Mar 22, 2024 at 7:58 AM huangdengdui <huangdengdui@huawei.com> wrote: > > > > > > > > > > > > On 2024/3/21 16:28, Thomas Monjalon wrote: > > > > 21/03/2024 03:02, huangdengdui: > > > >> > > > >> On 2024/3/20 20:31, Ferruh Yigit wrote: > > > >>> On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote: > > > >>>> On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > > >>>>> > > > >>>>> 12/03/2024 08:52, Dengdui Huang: > > > >>>>>> Some speeds can be achieved with different number of lanes. For example, > > > >>>>>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > > > >>>>>> When use different lanes, the port cannot be up. > > > >>>>> > > > >>>>> I'm not sure what you are referring to. > > > >>>>> I suppose it is not PCI lanes. > > > >>>>> Please could you link to an explanation of how a port is split in lanes? > > > >>>>> Which hardware does this? > > > >>>>> > > > >>>> This is a snapshot of 100Gb that the latest BCM576xx supports. > > > >>>> 100Gb (NRZ: 25G per lane, 4 lanes) link speed > > > >>>> 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed > > > >>>> 100Gb (PAM4-112: 100G per lane, 1 lane) link speed > > > >>>> > > > >>>> Let the user feed in lanes=< integer value> and the NIC driver decides > > > >>>> the matching combination speed x lanes that works. In future if a new speed > > > >>>> is implemented with more than 8 lanes, there wouldn't be a need > > > >>>> to touch this speed command. Using separate lane command would > > > >>>> be a better alternative to support already shipped products and only new > > > >>>> drivers would consider this lanes configuration, if applicable. > > > >>>> > > > >>> > > > >>> As far as I understand, lane is related to the physical layer of the > > > >>> NIC, there are multiple copies of transmitter, receiver, modulator HW > > > >>> block and each set called as a 'lane' and multiple lanes work together > > > >>> to achieve desired speed. (please correct me if this is wrong). > > > >>> > > > >>> Why not just configuring the speed is not enough? Why user needs to know > > > >>> the detail and configuration of the lanes? > > > >>> Will it work if driver/device configure the "speed x lane" internally > > > >>> for the requested speed? > > > >>> > > > >>> Is there a benefit to force specific lane count for a specific speed > > > >>> (like power optimization, just a wild guess)? > > > >>> > > > >>> > > > >>> And +1 for auto-negotiation if possible. > > > >> > > > >> As you said above,,multiple lanes work together to achieve desired speed. > > > >> For example, the following solutions can be used to implement 100G: > > > >> 1、Combines four 25G lanes > > > >> 2、Combines two 50G lanes > > > >> 3、A single 100G lane > > > >> > > > >> It is assumed that two ports are interconnected and the two ports support > > > >> the foregoing three solutions. But, we just configured the speed to 100G and > > > >> one port uses four 25G lanes by default and the other port uses two 50G lanes > > > >> by default, the port cannot be up. In this case, we need to configure the > > > >> two ports to use the same solutions (for example, uses two 50G lanes) > > > >> so that the ports can be up. > > > > > > > > Why this config is not OK? How do we know? > > > > Really I have a very bad feeling about this feature. > > > > > > > > > > > Sorry, I don't quite understand your question. > > > Are you asking why cannot be up when one port uses four 25G lanes and the other port uses two 50G lanes? > > > > > > 100GBASE-SR2 (two 50G lanes) and 100GBASE-SR4 (four 25G lanes) have different standards at the physical layer.[1] > > > So it's not possible to communicate. Configuring lanes can help the driver choose the same standard. > > > > Typically, low-level drivers like FW configure this. > > > > For example, If FW configures, 100G port as 100GBASE-SR2 then two > > ethdev(port 0 and port1) will show up. > > Now, assume if we expose this API and Can end user configure port 1 as > > 25G lines if so, > > a) What happens to port0 and it states? > There should be no impact to port0. > > > b) Will port2, port3 will show up after issuing this API(As end user > > configured 25Gx4 for 100G)? Will application needs to hotplug to get > > use ports. > No. The port count does not change. Nor does the number of PCI > functions seen by the host. Unless designed otherwise. > > Changing the lane count does not change anything in physical terms. > What changes is the modulation or the signaling scheme. > The number of lanes which can be supported is determined by > the PHY itself and the cables used and needs to be negotiated appropriately > with the remote partner - which is just like using forced Ethernet Speed > instead of auto-negotiated speeds. OK. It looks like platform independent then. At least cnxk driver, End user cannot simplify change the line config parameters while traffic is active also, it looks like other drivers need to have SerDes training with remote partner while reconfiguring it. At least on cnxk platform, 25Gx4 on 100G will show as 4 ethdev devices. Having said that, If other NICs support this feature without disturbing current port states, I don't have an objection to this API.
22/03/2024 06:51, Jerin Jacob: > On Fri, Mar 22, 2024 at 10:56 AM Ajit Khaparde > <ajit.khaparde@broadcom.com> wrote: > > > > On Thu, Mar 21, 2024 at 9:39 PM Jerin Jacob <jerinjacobk@gmail.com> wrote: > > > > > > On Fri, Mar 22, 2024 at 7:58 AM huangdengdui <huangdengdui@huawei.com> wrote: > > > > > > > > > > > > > > > > On 2024/3/21 16:28, Thomas Monjalon wrote: > > > > > 21/03/2024 03:02, huangdengdui: > > > > >> > > > > >> On 2024/3/20 20:31, Ferruh Yigit wrote: > > > > >>> On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote: > > > > >>>> On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > > > >>>>> > > > > >>>>> 12/03/2024 08:52, Dengdui Huang: > > > > >>>>>> Some speeds can be achieved with different number of lanes. For example, > > > > >>>>>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps. > > > > >>>>>> When use different lanes, the port cannot be up. > > > > >>>>> > > > > >>>>> I'm not sure what you are referring to. > > > > >>>>> I suppose it is not PCI lanes. > > > > >>>>> Please could you link to an explanation of how a port is split in lanes? > > > > >>>>> Which hardware does this? > > > > >>>>> > > > > >>>> This is a snapshot of 100Gb that the latest BCM576xx supports. > > > > >>>> 100Gb (NRZ: 25G per lane, 4 lanes) link speed > > > > >>>> 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed > > > > >>>> 100Gb (PAM4-112: 100G per lane, 1 lane) link speed > > > > >>>> > > > > >>>> Let the user feed in lanes=< integer value> and the NIC driver decides > > > > >>>> the matching combination speed x lanes that works. In future if a new speed > > > > >>>> is implemented with more than 8 lanes, there wouldn't be a need > > > > >>>> to touch this speed command. Using separate lane command would > > > > >>>> be a better alternative to support already shipped products and only new > > > > >>>> drivers would consider this lanes configuration, if applicable. > > > > >>>> > > > > >>> > > > > >>> As far as I understand, lane is related to the physical layer of the > > > > >>> NIC, there are multiple copies of transmitter, receiver, modulator HW > > > > >>> block and each set called as a 'lane' and multiple lanes work together > > > > >>> to achieve desired speed. (please correct me if this is wrong). > > > > >>> > > > > >>> Why not just configuring the speed is not enough? Why user needs to know > > > > >>> the detail and configuration of the lanes? > > > > >>> Will it work if driver/device configure the "speed x lane" internally > > > > >>> for the requested speed? > > > > >>> > > > > >>> Is there a benefit to force specific lane count for a specific speed > > > > >>> (like power optimization, just a wild guess)? > > > > >>> > > > > >>> > > > > >>> And +1 for auto-negotiation if possible. > > > > >> > > > > >> As you said above,,multiple lanes work together to achieve desired speed. > > > > >> For example, the following solutions can be used to implement 100G: > > > > >> 1、Combines four 25G lanes > > > > >> 2、Combines two 50G lanes > > > > >> 3、A single 100G lane > > > > >> > > > > >> It is assumed that two ports are interconnected and the two ports support > > > > >> the foregoing three solutions. But, we just configured the speed to 100G and > > > > >> one port uses four 25G lanes by default and the other port uses two 50G lanes > > > > >> by default, the port cannot be up. In this case, we need to configure the > > > > >> two ports to use the same solutions (for example, uses two 50G lanes) > > > > >> so that the ports can be up. > > > > > > > > > > Why this config is not OK? How do we know? > > > > > Really I have a very bad feeling about this feature. > > > > > > > > > > > > > > Sorry, I don't quite understand your question. > > > > Are you asking why cannot be up when one port uses four 25G lanes and the other port uses two 50G lanes? > > > > > > > > 100GBASE-SR2 (two 50G lanes) and 100GBASE-SR4 (four 25G lanes) have different standards at the physical layer.[1] > > > > So it's not possible to communicate. Configuring lanes can help the driver choose the same standard. > > > > > > Typically, low-level drivers like FW configure this. > > > > > > For example, If FW configures, 100G port as 100GBASE-SR2 then two > > > ethdev(port 0 and port1) will show up. > > > Now, assume if we expose this API and Can end user configure port 1 as > > > 25G lines if so, > > > a) What happens to port0 and it states? > > There should be no impact to port0. > > > > > b) Will port2, port3 will show up after issuing this API(As end user > > > configured 25Gx4 for 100G)? Will application needs to hotplug to get > > > use ports. > > No. The port count does not change. Nor does the number of PCI > > functions seen by the host. Unless designed otherwise. > > > > Changing the lane count does not change anything in physical terms. > > What changes is the modulation or the signaling scheme. > > The number of lanes which can be supported is determined by > > the PHY itself and the cables used and needs to be negotiated appropriately > > with the remote partner - which is just like using forced Ethernet Speed > > instead of auto-negotiated speeds. Thanks for the explanation Ajit. > OK. It looks like platform independent then. At least cnxk driver, End > user cannot simplify change the line config parameters > while traffic is active also, it looks like other drivers need to have > SerDes training with remote partner while reconfiguring it. > > At least on cnxk platform, 25Gx4 on 100G will show as 4 ethdev devices. That's a strange behaviour. Why showing 4 ports which are not independent? > Having said that, If other NICs support this feature without > disturbing current port states, I don't have an objection to this API.
On Fri, Mar 22, 2024 at 7:21 PM Thomas Monjalon <thomas@monjalon.net> wrote: > > 22/03/2024 06:51, Jerin Jacob: > > On Fri, Mar 22, 2024 at 10:56 AM Ajit Khaparde > > <ajit.khaparde@broadcom.com> wrote: > > > > > > On Thu, Mar 21, 2024 at 9:39 PM Jerin Jacob <jerinjacobk@gmail.com> wrote: > > > > > > > > On Fri, Mar 22, 2024 at 7:58 AM huangdengdui <huangdengdui@huawei.com> wrote: > > > > > > > > > > > > > > > > > > > For example, If FW configures, 100G port as 100GBASE-SR2 then two > > > > ethdev(port 0 and port1) will show up. > > > > Now, assume if we expose this API and Can end user configure port 1 as > > > > 25G lines if so, > > > > a) What happens to port0 and it states? > > > There should be no impact to port0. > > > > > > > b) Will port2, port3 will show up after issuing this API(As end user > > > > configured 25Gx4 for 100G)? Will application needs to hotplug to get > > > > use ports. > > > No. The port count does not change. Nor does the number of PCI > > > functions seen by the host. Unless designed otherwise. > > > > > > Changing the lane count does not change anything in physical terms. > > > What changes is the modulation or the signaling scheme. > > > The number of lanes which can be supported is determined by > > > the PHY itself and the cables used and needs to be negotiated appropriately > > > with the remote partner - which is just like using forced Ethernet Speed > > > instead of auto-negotiated speeds. > > Thanks for the explanation Ajit. > > > OK. It looks like platform independent then. At least cnxk driver, End > > user cannot simplify change the line config parameters > > while traffic is active also, it looks like other drivers need to have > > SerDes training with remote partner while reconfiguring it. > > > > At least on cnxk platform, 25Gx4 on 100G will show as 4 ethdev devices. > > That's a strange behaviour. > Why showing 4 ports which are not independent? I checked SerDes + NIC configuration again. It supports both modes. Show up as One port vs four ports. > > > Having said that, If other NICs support this feature without > > disturbing current port states, I don't have an objection to this API. > > >