| Message ID | 1476451768-21278-1-git-send-email-bernard.iremonger@intel.com (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Delegated to: | Bruce Richardson |
| Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 4F61A6CC4; Fri, 14 Oct 2016 15:29:35 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 5D2BA1396 for <dev@dpdk.org>; Fri, 14 Oct 2016 15:29:33 +0200 (CEST) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP; 14 Oct 2016 06:29:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,493,1473145200"; d="scan'208";a="19650883" Received: from sivswdev01.ir.intel.com (HELO localhost.localdomain) ([10.237.217.45]) by orsmga005.jf.intel.com with ESMTP; 14 Oct 2016 06:29:31 -0700 From: Bernard Iremonger <bernard.iremonger@intel.com> To: dev@dpdk.org, rahul.r.shah@intel.com, wenzhuo.lu@intel.com Cc: Bernard Iremonger <bernard.iremonger@intel.com> Date: Fri, 14 Oct 2016 14:29:28 +0100 Message-Id: <1476451768-21278-1-git-send-email-bernard.iremonger@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: <1472202487-20119-1-git-send-email-bernard.iremonger@intel.com> References: <1472202487-20119-1-git-send-email-bernard.iremonger@intel.com> Subject: [dpdk-dev] [PATCH v2] net/ixgbe: support multiqueue mode VMDq DCB with SRIOV X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Iremonger, Bernard
Oct. 14, 2016, 1:29 p.m. UTC
modify ixgbe_dcb_tx_hw_config function. modify ixgbe_dev_mq_rx_configure function. modify ixgbe_configure_dcb function. Changes in v2: Rebased to DPDK v16.11-rc1 Signed-off-by: Rahul R Shah <rahul.r.shah@intel.com> Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com> --- drivers/net/ixgbe/ixgbe_ethdev.c | 9 ++++----- drivers/net/ixgbe/ixgbe_rxtx.c | 37 +++++++++++++++++++++---------------- 2 files changed, 25 insertions(+), 21 deletions(-)
Comments
Hi Bernard, Rahul, > -----Original Message----- > From: Iremonger, Bernard > Sent: Friday, October 14, 2016 9:29 PM > To: dev@dpdk.org; Shah, Rahul R; Lu, Wenzhuo > Cc: Iremonger, Bernard > Subject: [PATCH v2] net/ixgbe: support multiqueue mode VMDq DCB with SRIOV > > modify ixgbe_dcb_tx_hw_config function. > modify ixgbe_dev_mq_rx_configure function. > modify ixgbe_configure_dcb function. Would you like to add more details about why we need to modify these functions and what has been done? Thanks. > > Changes in v2: > Rebased to DPDK v16.11-rc1 > > Signed-off-by: Rahul R Shah <rahul.r.shah@intel.com> > Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com> > --- > drivers/net/ixgbe/ixgbe_ethdev.c | 9 ++++----- > drivers/net/ixgbe/ixgbe_rxtx.c | 37 +++++++++++++++++++++---------------- > 2 files changed, 25 insertions(+), 21 deletions(-) > > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c > index 4ca5747..114698d 100644 > --- a/drivers/net/ixgbe/ixgbe_ethdev.c > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > @@ -1977,6 +1977,8 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) > /* check multi-queue mode */ > switch (dev_conf->rxmode.mq_mode) { > case ETH_MQ_RX_VMDQ_DCB: Should we store the mq mode here? > + PMD_INIT_LOG(INFO, "ETH_MQ_RX_VMDQ_DCB > mode supported in SRIOV"); > + break; > case ETH_MQ_RX_VMDQ_DCB_RSS: > /* DCB/RSS VMDQ in SRIOV mode, not implement yet > */ > PMD_INIT_LOG(ERR, "SRIOV active," > @@ -2012,11 +2014,8 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) > > switch (dev_conf->txmode.mq_mode) { > case ETH_MQ_TX_VMDQ_DCB: > - /* DCB VMDQ in SRIOV mode, not implement yet */ > - PMD_INIT_LOG(ERR, "SRIOV is active," > - " unsupported VMDQ mq_mode > tx %d.", > - dev_conf->txmode.mq_mode); > - return -EINVAL; > + PMD_INIT_LOG(INFO, "ETH_MQ_TX_VMDQ_DCB mode > supported in SRIOV"); > + break; > default: /* ETH_MQ_TX_VMDQ_ONLY or ETH_MQ_TX_NONE > */ > dev->data->dev_conf.txmode.mq_mode = > ETH_MQ_TX_VMDQ_ONLY; > break; > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c index > 2ce8234..bb13889 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx.c > +++ b/drivers/net/ixgbe/ixgbe_rxtx.c > @@ -1,7 +1,7 @@ > /*- > * BSD LICENSE > * > - * Copyright(c) 2010-2015 Intel Corporation. All rights reserved. > + * Copyright(c) 2010-2016 Intel Corporation. All rights reserved. > * Copyright 2014 6WIND S.A. > * All rights reserved. > * > @@ -3313,15 +3313,16 @@ ixgbe_vmdq_dcb_configure(struct rte_eth_dev > *dev) > > /** > * ixgbe_dcb_config_tx_hw_config - Configure general DCB TX parameters > - * @hw: pointer to hardware structure > + * @dev: pointer to eth_dev structure > * @dcb_config: pointer to ixgbe_dcb_config structure > */ > static void > -ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, > +ixgbe_dcb_tx_hw_config(struct rte_eth_dev *dev, > struct ixgbe_dcb_config *dcb_config) { > uint32_t reg; > uint32_t q; > + struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data- > >dev_private); > > PMD_INIT_FUNC_TRACE(); > if (hw->mac.type != ixgbe_mac_82598EB) { @@ -3339,11 +3340,17 @@ > ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, > if (dcb_config->vt_mode) > reg |= IXGBE_MTQC_VT_ENA; > IXGBE_WRITE_REG(hw, IXGBE_MTQC, reg); > - > - /* Disable drop for all queues */ > - for (q = 0; q < 128; q++) > - IXGBE_WRITE_REG(hw, IXGBE_QDE, > - (IXGBE_QDE_WRITE | (q << > IXGBE_QDE_IDX_SHIFT))); > + if (RTE_ETH_DEV_SRIOV(dev).active == 0) { > + /* Disable drop for all queues in VMDQ mode*/ > + for (q = 0; q < 128; q++) > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > + (IXGBE_QDE_WRITE | (q << > IXGBE_QDE_IDX_SHIFT) | IXGBE_QDE_ENABLE)); > + } else { > + /* Enable drop for all queues in SRIOV mode */ > + for (q = 0; q < 128; q++) > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > + (IXGBE_QDE_WRITE | (q << > IXGBE_QDE_IDX_SHIFT))); > + } I think it has nothing to do with mq mode. Do I miss something? > > /* Enable the Tx desc arbiter */ > reg = IXGBE_READ_REG(hw, IXGBE_RTTDCS); @@ -3378,7 > +3385,7 @@ ixgbe_vmdq_dcb_hw_tx_config(struct rte_eth_dev *dev, > vmdq_tx_conf->nb_queue_pools == ETH_16_POOLS ? > 0xFFFF : 0xFFFFFFFF); > > /*Configure general DCB TX parameters*/ > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > } > > static void > @@ -3661,7 +3668,7 @@ ixgbe_dcb_hw_configure(struct rte_eth_dev *dev, > /*get DCB TX configuration parameters from rte_eth_conf*/ > ixgbe_dcb_tx_config(dev, dcb_config); > /*Configure general DCB TX parameters*/ > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > break; > default: > PMD_INIT_LOG(ERR, "Incorrect DCB TX mode configuration"); > @@ -3810,9 +3817,6 @@ void ixgbe_configure_dcb(struct rte_eth_dev *dev) > (dev_conf->rxmode.mq_mode != ETH_MQ_RX_DCB_RSS)) > return; > > - if (dev->data->nb_rx_queues != ETH_DCB_NUM_QUEUES) > - return; I remember it's a limitation of implementation. The reason is the resource allocation. Why could we remove it now? > - > /** Configure DCB hardware **/ > ixgbe_dcb_hw_configure(dev, dcb_cfg); > } > @@ -4082,12 +4086,13 @@ ixgbe_dev_mq_rx_configure(struct rte_eth_dev > *dev) > case ETH_MQ_RX_VMDQ_RSS: > ixgbe_config_vf_rss(dev); > break; > - > - /* FIXME if support DCB/RSS together with VMDq & SRIOV */ > case ETH_MQ_RX_VMDQ_DCB: > + ixgbe_vmdq_dcb_configure(dev); > + break; > + /* FIXME if support DCB/RSS together with VMDq & SRIOV */ > case ETH_MQ_RX_VMDQ_DCB_RSS: > PMD_INIT_LOG(ERR, > - "Could not support DCB with VMDq & SRIOV"); > + "Could not support DCB/RSS with VMDq & > SRIOV"); > return -1; > default: > ixgbe_config_vf_default(dev); > -- > 2.10.1
Changes in v3: rebase to latest master. update commit message for ixgbe patch add testpmd patch. Changes in v2: rebase to latest master. Bernard Iremonger (2): net/ixgbe: support multiqueue mode VMDq DCB with SRIOV app/test_pmd: fix DCB configuration app/test-pmd/testpmd.c | 4 ++-- drivers/net/ixgbe/ixgbe_ethdev.c | 9 ++++----- drivers/net/ixgbe/ixgbe_rxtx.c | 37 +++++++++++++++++++++---------------- 3 files changed, 27 insertions(+), 23 deletions(-)
Hi Wenzhuo, <snip> > > Subject: [PATCH v2] net/ixgbe: support multiqueue mode VMDq DCB with > > SRIOV > > > > modify ixgbe_dcb_tx_hw_config function. > > modify ixgbe_dev_mq_rx_configure function. > > modify ixgbe_configure_dcb function. > Would you like to add more details about why we need to modify these > functions and what has been done? Thanks. I have updated the commit message for this patch with more information. I have also added a testpmd patch and sent a v3 patchset. > > Changes in v2: > > Rebased to DPDK v16.11-rc1 > > > > Signed-off-by: Rahul R Shah <rahul.r.shah@intel.com> > > Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com> > > --- > > drivers/net/ixgbe/ixgbe_ethdev.c | 9 ++++----- > > drivers/net/ixgbe/ixgbe_rxtx.c | 37 +++++++++++++++++++++------------ > ---- > > 2 files changed, 25 insertions(+), 21 deletions(-) > > > > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c > > b/drivers/net/ixgbe/ixgbe_ethdev.c > > index 4ca5747..114698d 100644 > > --- a/drivers/net/ixgbe/ixgbe_ethdev.c > > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > > @@ -1977,6 +1977,8 @@ ixgbe_check_mq_mode(struct rte_eth_dev > *dev) > > /* check multi-queue mode */ > > switch (dev_conf->rxmode.mq_mode) { > > case ETH_MQ_RX_VMDQ_DCB: > Should we store the mq mode here? Yes, it probably should be stored here. > > + PMD_INIT_LOG(INFO, "ETH_MQ_RX_VMDQ_DCB > > mode supported in SRIOV"); > > + break; > > case ETH_MQ_RX_VMDQ_DCB_RSS: > > /* DCB/RSS VMDQ in SRIOV mode, not implement > yet */ > > PMD_INIT_LOG(ERR, "SRIOV active," > > @@ -2012,11 +2014,8 @@ ixgbe_check_mq_mode(struct rte_eth_dev > *dev) > > > > switch (dev_conf->txmode.mq_mode) { > > case ETH_MQ_TX_VMDQ_DCB: > > - /* DCB VMDQ in SRIOV mode, not implement yet */ > > - PMD_INIT_LOG(ERR, "SRIOV is active," > > - " unsupported VMDQ mq_mode > > tx %d.", > > - dev_conf->txmode.mq_mode); > > - return -EINVAL; > > + PMD_INIT_LOG(INFO, "ETH_MQ_TX_VMDQ_DCB > mode > > supported in SRIOV"); > > + break; > > default: /* ETH_MQ_TX_VMDQ_ONLY or > ETH_MQ_TX_NONE */ > > dev->data->dev_conf.txmode.mq_mode = > ETH_MQ_TX_VMDQ_ONLY; > > break; > > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c > > b/drivers/net/ixgbe/ixgbe_rxtx.c index > > 2ce8234..bb13889 100644 > > --- a/drivers/net/ixgbe/ixgbe_rxtx.c > > +++ b/drivers/net/ixgbe/ixgbe_rxtx.c > > @@ -1,7 +1,7 @@ > > /*- > > * BSD LICENSE > > * > > - * Copyright(c) 2010-2015 Intel Corporation. All rights reserved. > > + * Copyright(c) 2010-2016 Intel Corporation. All rights reserved. > > * Copyright 2014 6WIND S.A. > > * All rights reserved. > > * > > @@ -3313,15 +3313,16 @@ ixgbe_vmdq_dcb_configure(struct > rte_eth_dev > > *dev) > > > > /** > > * ixgbe_dcb_config_tx_hw_config - Configure general DCB TX > > parameters > > - * @hw: pointer to hardware structure > > + * @dev: pointer to eth_dev structure > > * @dcb_config: pointer to ixgbe_dcb_config structure > > */ > > static void > > -ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, > > +ixgbe_dcb_tx_hw_config(struct rte_eth_dev *dev, > > struct ixgbe_dcb_config *dcb_config) { > > uint32_t reg; > > uint32_t q; > > + struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data- > > >dev_private); > > > > PMD_INIT_FUNC_TRACE(); > > if (hw->mac.type != ixgbe_mac_82598EB) { @@ -3339,11 +3340,17 > @@ > > ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, > > if (dcb_config->vt_mode) > > reg |= IXGBE_MTQC_VT_ENA; > > IXGBE_WRITE_REG(hw, IXGBE_MTQC, reg); > > - > > - /* Disable drop for all queues */ > > - for (q = 0; q < 128; q++) > > - IXGBE_WRITE_REG(hw, IXGBE_QDE, > > - (IXGBE_QDE_WRITE | (q << > > IXGBE_QDE_IDX_SHIFT))); > > + if (RTE_ETH_DEV_SRIOV(dev).active == 0) { > > + /* Disable drop for all queues in VMDQ mode*/ > > + for (q = 0; q < 128; q++) > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > + (IXGBE_QDE_WRITE | (q << > > IXGBE_QDE_IDX_SHIFT) | IXGBE_QDE_ENABLE)); > > + } else { > > + /* Enable drop for all queues in SRIOV mode */ > > + for (q = 0; q < 128; q++) > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > + (IXGBE_QDE_WRITE | (q << > > IXGBE_QDE_IDX_SHIFT))); > > + } > I think it has nothing to do with mq mode. Do I miss something? Behavior is different when SRIOV is enabled. > > > > > /* Enable the Tx desc arbiter */ > > reg = IXGBE_READ_REG(hw, IXGBE_RTTDCS); @@ -3378,7 > > +3385,7 @@ ixgbe_vmdq_dcb_hw_tx_config(struct rte_eth_dev *dev, > > vmdq_tx_conf->nb_queue_pools == ETH_16_POOLS > ? > > 0xFFFF : 0xFFFFFFFF); > > > > /*Configure general DCB TX parameters*/ > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > } > > > > static void > > @@ -3661,7 +3668,7 @@ ixgbe_dcb_hw_configure(struct rte_eth_dev > *dev, > > /*get DCB TX configuration parameters from rte_eth_conf*/ > > ixgbe_dcb_tx_config(dev, dcb_config); > > /*Configure general DCB TX parameters*/ > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > break; > > default: > > PMD_INIT_LOG(ERR, "Incorrect DCB TX mode > configuration"); @@ > > -3810,9 +3817,6 @@ void ixgbe_configure_dcb(struct rte_eth_dev *dev) > > (dev_conf->rxmode.mq_mode != ETH_MQ_RX_DCB_RSS)) > > return; > > > > - if (dev->data->nb_rx_queues != ETH_DCB_NUM_QUEUES) > > - return; > I remember it's a limitation of implementation. The reason is the resource > allocation. Why could we remove it now? ETH_DCB_NUM_QUEUES is 128, nb_rx_queues may not be 128. > > > - > > /** Configure DCB hardware **/ > > ixgbe_dcb_hw_configure(dev, dcb_cfg); } @@ -4082,12 +4086,13 > @@ > > ixgbe_dev_mq_rx_configure(struct rte_eth_dev > > *dev) > > case ETH_MQ_RX_VMDQ_RSS: > > ixgbe_config_vf_rss(dev); > > break; > > - > > - /* FIXME if support DCB/RSS together with VMDq & SRIOV */ > > case ETH_MQ_RX_VMDQ_DCB: > > + ixgbe_vmdq_dcb_configure(dev); > > + break; > > + /* FIXME if support DCB/RSS together with VMDq & SRIOV */ > > case ETH_MQ_RX_VMDQ_DCB_RSS: > > PMD_INIT_LOG(ERR, > > - "Could not support DCB with VMDq & > SRIOV"); > > + "Could not support DCB/RSS with VMDq & > > SRIOV"); > > return -1; > > default: > > ixgbe_config_vf_default(dev); > > -- > > 2.10.1 Regards, Bernard.
Changes in v4: changes to ixgbe patch following comments. Changes in v3: rebase to latest master. update commit message for ixgbe patch add testpmd patch. Changes in v2: rebase to latest master. Bernard Iremonger (2): net/ixgbe: support multiqueue mode VMDq DCB with SRIOV app/test_pmd: fix DCB configuration app/test-pmd/testpmd.c | 4 ++-- drivers/net/ixgbe/ixgbe_ethdev.c | 11 ++++++----- drivers/net/ixgbe/ixgbe_rxtx.c | 36 ++++++++++++++++++++++-------------- 3 files changed, 30 insertions(+), 21 deletions(-)
Hi Bernard, > -----Original Message----- > From: Iremonger, Bernard > Sent: Tuesday, October 18, 2016 12:27 AM > To: Lu, Wenzhuo; dev@dpdk.org; Shah, Rahul R > Subject: RE: [PATCH v2] net/ixgbe: support multiqueue mode VMDq DCB with > SRIOV > > Hi Wenzhuo, > > > > if (hw->mac.type != ixgbe_mac_82598EB) { @@ -3339,11 +3340,17 > > @@ > > > ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, > > > if (dcb_config->vt_mode) > > > reg |= IXGBE_MTQC_VT_ENA; > > > IXGBE_WRITE_REG(hw, IXGBE_MTQC, reg); > > > - > > > - /* Disable drop for all queues */ > > > - for (q = 0; q < 128; q++) > > > - IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > - (IXGBE_QDE_WRITE | (q << > > > IXGBE_QDE_IDX_SHIFT))); > > > + if (RTE_ETH_DEV_SRIOV(dev).active == 0) { > > > + /* Disable drop for all queues in VMDQ mode*/ > > > + for (q = 0; q < 128; q++) > > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > + (IXGBE_QDE_WRITE | (q << > > > IXGBE_QDE_IDX_SHIFT) | IXGBE_QDE_ENABLE)); > > > + } else { > > > + /* Enable drop for all queues in SRIOV mode */ > > > + for (q = 0; q < 128; q++) > > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > + (IXGBE_QDE_WRITE | (q << > > > IXGBE_QDE_IDX_SHIFT))); > > > + } > > I think it has nothing to do with mq mode. Do I miss something? > > Behavior is different when SRIOV is enabled. I don't understand why the behavior is different. To my opinion, the drop has nothing to do with the mode. We can enable or disable it. The old behavior is disabling it by default. Now you change it to disabling it by default in NO-SRIOV mode, but enabling it in SRIOV mode. What I don't get is the reason. > > > > > > > > > /* Enable the Tx desc arbiter */ > > > reg = IXGBE_READ_REG(hw, IXGBE_RTTDCS); @@ -3378,7 > > > +3385,7 @@ ixgbe_vmdq_dcb_hw_tx_config(struct rte_eth_dev *dev, > > > vmdq_tx_conf->nb_queue_pools == ETH_16_POOLS > > ? > > > 0xFFFF : 0xFFFFFFFF); > > > > > > /*Configure general DCB TX parameters*/ > > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > > } > > > > > > static void > > > @@ -3661,7 +3668,7 @@ ixgbe_dcb_hw_configure(struct rte_eth_dev > > *dev, > > > /*get DCB TX configuration parameters from rte_eth_conf*/ > > > ixgbe_dcb_tx_config(dev, dcb_config); > > > /*Configure general DCB TX parameters*/ > > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > > break; > > > default: > > > PMD_INIT_LOG(ERR, "Incorrect DCB TX mode > > configuration"); @@ > > > -3810,9 +3817,6 @@ void ixgbe_configure_dcb(struct rte_eth_dev *dev) > > > (dev_conf->rxmode.mq_mode != ETH_MQ_RX_DCB_RSS)) > > > return; > > > > > > - if (dev->data->nb_rx_queues != ETH_DCB_NUM_QUEUES) > > > - return; > > I remember it's a limitation of implementation. The reason is the > > resource allocation. Why could we remove it now? > > ETH_DCB_NUM_QUEUES is 128, nb_rx_queues may not be 128. I think it's a limitation to force the queue number to be ETH_DCB_NUM_QUEUES. Just to confirm it, have you try to set rx queue number to something different from 128, like 64, 32...
Hi Wenzhuo, <snip> > > Subject: RE: [PATCH v2] net/ixgbe: support multiqueue mode VMDq DCB > > with SRIOV > > > > Hi Wenzhuo, > > > > > > if (hw->mac.type != ixgbe_mac_82598EB) { @@ -3339,11 +3340,17 > > > @@ > > > > ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, > > > > if (dcb_config->vt_mode) > > > > reg |= IXGBE_MTQC_VT_ENA; > > > > IXGBE_WRITE_REG(hw, IXGBE_MTQC, reg); > > > > - > > > > - /* Disable drop for all queues */ > > > > - for (q = 0; q < 128; q++) > > > > - IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > > - (IXGBE_QDE_WRITE | (q << > > > > IXGBE_QDE_IDX_SHIFT))); > > > > + if (RTE_ETH_DEV_SRIOV(dev).active == 0) { > > > > + /* Disable drop for all queues in VMDQ mode*/ > > > > + for (q = 0; q < 128; q++) > > > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > > + (IXGBE_QDE_WRITE | (q << > > > > IXGBE_QDE_IDX_SHIFT) | IXGBE_QDE_ENABLE)); > > > > + } else { > > > > + /* Enable drop for all queues in SRIOV mode */ > > > > + for (q = 0; q < 128; q++) > > > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > > + (IXGBE_QDE_WRITE | (q << > > > > IXGBE_QDE_IDX_SHIFT))); > > > > + } > > > I think it has nothing to do with mq mode. Do I miss something? > > > > Behavior is different when SRIOV is enabled. > I don't understand why the behavior is different. To my opinion, the drop > has nothing to do with the mode. We can enable or disable it. > The old behavior is disabling it by default. Now you change it to disabling it by > default in NO-SRIOV mode, but enabling it in SRIOV mode. > What I don't get is the reason. Please refer to section 4.6.11.3.1 page 180 of the 82599-10-gbe-controller-datasheet.pdf The last paragraph on page 180 states: "Queue Drop Enable (PFQDE) - In SR-IO the QDE bit should be set to 1b in the PFQDE register for all queues. In VMDq mode, the QDE bit should be set to 0b for all queues." > > > > /* Enable the Tx desc arbiter */ > > > > reg = IXGBE_READ_REG(hw, IXGBE_RTTDCS); @@ -3378,7 > > > > +3385,7 @@ ixgbe_vmdq_dcb_hw_tx_config(struct rte_eth_dev *dev, > > > > vmdq_tx_conf->nb_queue_pools == ETH_16_POOLS > > > ? > > > > 0xFFFF : 0xFFFFFFFF); > > > > > > > > /*Configure general DCB TX parameters*/ > > > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > > > } > > > > > > > > static void > > > > @@ -3661,7 +3668,7 @@ ixgbe_dcb_hw_configure(struct rte_eth_dev > > > *dev, > > > > /*get DCB TX configuration parameters from rte_eth_conf*/ > > > > ixgbe_dcb_tx_config(dev, dcb_config); > > > > /*Configure general DCB TX parameters*/ > > > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > > > break; > > > > default: > > > > PMD_INIT_LOG(ERR, "Incorrect DCB TX mode > > > configuration"); @@ > > > > -3810,9 +3817,6 @@ void ixgbe_configure_dcb(struct rte_eth_dev > *dev) > > > > (dev_conf->rxmode.mq_mode != ETH_MQ_RX_DCB_RSS)) > > > > return; > > > > > > > > - if (dev->data->nb_rx_queues != ETH_DCB_NUM_QUEUES) > > > > - return; > > > I remember it's a limitation of implementation. The reason is the > > > resource allocation. Why could we remove it now? > > > > ETH_DCB_NUM_QUEUES is 128, nb_rx_queues may not be 128. > I think it's a limitation to force the queue number to be > ETH_DCB_NUM_QUEUES. > Just to confirm it, have you try to set rx queue number to something > different from 128, like 64, 32... In my test scenario the nb_rx_queues is 1. Regards, Bernard
Hi Bernard, > -----Original Message----- > From: Iremonger, Bernard > Sent: Tuesday, October 18, 2016 4:58 PM > To: Lu, Wenzhuo; dev@dpdk.org; Shah, Rahul R > Subject: RE: [PATCH v2] net/ixgbe: support multiqueue mode VMDq DCB with > SRIOV > > Hi Wenzhuo, > > <snip> > > > > Subject: RE: [PATCH v2] net/ixgbe: support multiqueue mode VMDq DCB > > > with SRIOV > > > > > > Hi Wenzhuo, > > > > > > > > if (hw->mac.type != ixgbe_mac_82598EB) { @@ -3339,11 +3340,17 > > > > @@ > > > > > ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, > > > > > if (dcb_config->vt_mode) > > > > > reg |= IXGBE_MTQC_VT_ENA; > > > > > IXGBE_WRITE_REG(hw, IXGBE_MTQC, reg); > > > > > - > > > > > - /* Disable drop for all queues */ > > > > > - for (q = 0; q < 128; q++) > > > > > - IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > > > - (IXGBE_QDE_WRITE | (q << > > > > > IXGBE_QDE_IDX_SHIFT))); > > > > > + if (RTE_ETH_DEV_SRIOV(dev).active == 0) { > > > > > + /* Disable drop for all queues in VMDQ mode*/ > > > > > + for (q = 0; q < 128; q++) > > > > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > > > + (IXGBE_QDE_WRITE | > (q << > > > > > IXGBE_QDE_IDX_SHIFT) | IXGBE_QDE_ENABLE)); This code is used to enable drop, but the comments say 'disable drop'. > > > > > + } else { > > > > > + /* Enable drop for all queues in SRIOV mode */ > > > > > + for (q = 0; q < 128; q++) > > > > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > > > + (IXGBE_QDE_WRITE | > (q << > > > > > IXGBE_QDE_IDX_SHIFT))); This code is used to disable drop, but the comments say 'enable drop'. I've checked the V4, the same problem too. I think the problem is the comments, right? > > > > > + } > > > > I think it has nothing to do with mq mode. Do I miss something? > > > > > > Behavior is different when SRIOV is enabled. > > I don't understand why the behavior is different. To my opinion, the > > drop has nothing to do with the mode. We can enable or disable it. > > The old behavior is disabling it by default. Now you change it to > > disabling it by default in NO-SRIOV mode, but enabling it in SRIOV mode. > > What I don't get is the reason. > > Please refer to section 4.6.11.3.1 page 180 of the 82599-10-gbe-controller- > datasheet.pdf > The last paragraph on page 180 states: > > "Queue Drop Enable (PFQDE) - In SR-IO the QDE bit should be set to 1b in the > PFQDE register for all queues. > In VMDq mode, the QDE bit should be set to 0b for all queues." Got it, thanks a lot :) > > > > > > > /* Enable the Tx desc arbiter */ > > > > > reg = IXGBE_READ_REG(hw, IXGBE_RTTDCS); @@ -3378,7 > > > > > +3385,7 @@ ixgbe_vmdq_dcb_hw_tx_config(struct rte_eth_dev *dev, > > > > > vmdq_tx_conf->nb_queue_pools == ETH_16_POOLS > > > > ? > > > > > 0xFFFF : 0xFFFFFFFF); > > > > > > > > > > /*Configure general DCB TX parameters*/ > > > > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > > > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > > > > } > > > > > > > > > > static void > > > > > @@ -3661,7 +3668,7 @@ ixgbe_dcb_hw_configure(struct rte_eth_dev > > > > *dev, > > > > > /*get DCB TX configuration parameters from rte_eth_conf*/ > > > > > ixgbe_dcb_tx_config(dev, dcb_config); > > > > > /*Configure general DCB TX parameters*/ > > > > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > > > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > > > > break; > > > > > default: > > > > > PMD_INIT_LOG(ERR, "Incorrect DCB TX mode > > > > configuration"); @@ > > > > > -3810,9 +3817,6 @@ void ixgbe_configure_dcb(struct rte_eth_dev > > *dev) > > > > > (dev_conf->rxmode.mq_mode != ETH_MQ_RX_DCB_RSS)) > > > > > return; > > > > > > > > > > - if (dev->data->nb_rx_queues != ETH_DCB_NUM_QUEUES) > > > > > - return; > > > > I remember it's a limitation of implementation. The reason is the > > > > resource allocation. Why could we remove it now? > > > > > > ETH_DCB_NUM_QUEUES is 128, nb_rx_queues may not be 128. > > I think it's a limitation to force the queue number to be > > ETH_DCB_NUM_QUEUES. > > Just to confirm it, have you try to set rx queue number to something > > different from 128, like 64, 32... > > In my test scenario the nb_rx_queues is 1. Glad to know that, thanks :) > > Regards, > > Bernard
Hi Wenzhuo, <snip> > > > > Subject: RE: [PATCH v2] net/ixgbe: support multiqueue mode VMDq > > > > DCB with SRIOV > > > > > > > > Hi Wenzhuo, > > > > > > > > > > if (hw->mac.type != ixgbe_mac_82598EB) { @@ -3339,11 > > > > > > +3340,17 > > > > > @@ > > > > > > ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, > > > > > > if (dcb_config->vt_mode) > > > > > > reg |= IXGBE_MTQC_VT_ENA; > > > > > > IXGBE_WRITE_REG(hw, IXGBE_MTQC, reg); > > > > > > - > > > > > > - /* Disable drop for all queues */ > > > > > > - for (q = 0; q < 128; q++) > > > > > > - IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > > > > - (IXGBE_QDE_WRITE | (q << > > > > > > IXGBE_QDE_IDX_SHIFT))); > > > > > > + if (RTE_ETH_DEV_SRIOV(dev).active == 0) { > > > > > > + /* Disable drop for all queues in VMDQ > mode*/ > > > > > > + for (q = 0; q < 128; q++) > > > > > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > > > > + (IXGBE_QDE_WRITE | > > (q << > > > > > > IXGBE_QDE_IDX_SHIFT) | IXGBE_QDE_ENABLE)); > This code is used to enable drop, but the comments say 'disable drop'. The comment is correct but the code is not. > > > > > > + } else { > > > > > > + /* Enable drop for all queues in SRIOV mode > */ > > > > > > + for (q = 0; q < 128; q++) > > > > > > + IXGBE_WRITE_REG(hw, IXGBE_QDE, > > > > > > + (IXGBE_QDE_WRITE | > > (q << > > > > > > IXGBE_QDE_IDX_SHIFT))); > This code is used to disable drop, but the comments say 'enable drop'. The comment is correct but the code is not. > I've checked the V4, the same problem too. I think the problem is the > comments, right? No, the comments are correct but the code is reversed. I will send a v5. > > > > > > + } > > > > > I think it has nothing to do with mq mode. Do I miss something? > > > > > > > > Behavior is different when SRIOV is enabled. > > > I don't understand why the behavior is different. To my opinion, the > > > drop has nothing to do with the mode. We can enable or disable it. > > > The old behavior is disabling it by default. Now you change it to > > > disabling it by default in NO-SRIOV mode, but enabling it in SRIOV mode. > > > What I don't get is the reason. > > > > Please refer to section 4.6.11.3.1 page 180 of the > > 82599-10-gbe-controller- datasheet.pdf The last paragraph on page 180 > > states: > > > > "Queue Drop Enable (PFQDE) - In SR-IO the QDE bit should be set to 1b > > in the PFQDE register for all queues. > > In VMDq mode, the QDE bit should be set to 0b for all queues." > Got it, thanks a lot :) > > > > > > > > > > > /* Enable the Tx desc arbiter */ > > > > > > reg = IXGBE_READ_REG(hw, IXGBE_RTTDCS); @@ - > 3378,7 > > > > > > +3385,7 @@ ixgbe_vmdq_dcb_hw_tx_config(struct rte_eth_dev > > > > > > +*dev, > > > > > > vmdq_tx_conf->nb_queue_pools == > ETH_16_POOLS > > > > > ? > > > > > > 0xFFFF : 0xFFFFFFFF); > > > > > > > > > > > > /*Configure general DCB TX parameters*/ > > > > > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > > > > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > > > > > } > > > > > > > > > > > > static void > > > > > > @@ -3661,7 +3668,7 @@ ixgbe_dcb_hw_configure(struct > > > > > > rte_eth_dev > > > > > *dev, > > > > > > /*get DCB TX configuration parameters from > rte_eth_conf*/ > > > > > > ixgbe_dcb_tx_config(dev, dcb_config); > > > > > > /*Configure general DCB TX parameters*/ > > > > > > - ixgbe_dcb_tx_hw_config(hw, dcb_config); > > > > > > + ixgbe_dcb_tx_hw_config(dev, dcb_config); > > > > > > break; > > > > > > default: > > > > > > PMD_INIT_LOG(ERR, "Incorrect DCB TX mode > > > > > configuration"); @@ > > > > > > -3810,9 +3817,6 @@ void ixgbe_configure_dcb(struct rte_eth_dev > > > *dev) > > > > > > (dev_conf->rxmode.mq_mode != ETH_MQ_RX_DCB_RSS)) > > > > > > return; > > > > > > > > > > > > - if (dev->data->nb_rx_queues != ETH_DCB_NUM_QUEUES) > > > > > > - return; > > > > > I remember it's a limitation of implementation. The reason is > > > > > the resource allocation. Why could we remove it now? > > > > > > > > ETH_DCB_NUM_QUEUES is 128, nb_rx_queues may not be 128. > > > I think it's a limitation to force the queue number to be > > > ETH_DCB_NUM_QUEUES. > > > Just to confirm it, have you try to set rx queue number to something > > > different from 128, like 64, 32... > > > > In my test scenario the nb_rx_queues is 1. > Glad to know that, thanks :) > Regards, Bernard
Changes in v5: fix enable/disable of the QDE bit in the PFQDE register. Changes in v4: changes to ixgbe patch following comments. Changes in v3: rebase to latest master. update commit message for ixgbe patch add testpmd patch. Changes in v2: rebase to latest master. Bernard Iremonger (2): net/ixgbe: support multiqueue mode VMDq DCB with SRIOV app/test_pmd: fix DCB configuration app/test-pmd/testpmd.c | 4 ++-- drivers/net/ixgbe/ixgbe_ethdev.c | 11 ++++++----- drivers/net/ixgbe/ixgbe_rxtx.c | 35 ++++++++++++++++++++++------------- 3 files changed, 30 insertions(+), 20 deletions(-)
Hi, > -----Original Message----- > From: Iremonger, Bernard > Sent: Wednesday, October 19, 2016 6:21 PM > To: dev@dpdk.org; Shah, Rahul R; Lu, Wenzhuo > Cc: Iremonger, Bernard > Subject: [PATCH v5 0/2] net/ixgbe: VMDq DCB with SRIOV > > Changes in v5: > fix enable/disable of the QDE bit in the PFQDE register. > > Changes in v4: > changes to ixgbe patch following comments. > > Changes in v3: > rebase to latest master. > update commit message for ixgbe patch > add testpmd patch. > > Changes in v2: > rebase to latest master. > > Bernard Iremonger (2): > net/ixgbe: support multiqueue mode VMDq DCB with SRIOV > app/test_pmd: fix DCB configuration > > app/test-pmd/testpmd.c | 4 ++-- > drivers/net/ixgbe/ixgbe_ethdev.c | 11 ++++++----- > drivers/net/ixgbe/ixgbe_rxtx.c | 35 ++++++++++++++++++++++------------- > 3 files changed, 30 insertions(+), 20 deletions(-) > > -- > 2.10.1 Series-Acked-by: Wenzhuo Lu <Wenzhuo.lu@intel.com>
Changes in v6: rebase to latest master. revise commit messages. Changes in v5: fix enable/disable of the QDE bit in the PFQDE register. Changes in v4: changes to ixgbe patch following comments. Changes in v3: rebase to latest master. update commit message for ixgbe patch add testpmd patch. Changes in v2: rebase to latest master. Bernard Iremonger (2): net/ixgbe: support multiqueue mode VMDq DCB with SRIOV app/test_pmd: fix DCB configuration app/test-pmd/testpmd.c | 4 ++-- drivers/net/ixgbe/ixgbe_ethdev.c | 11 ++++++----- drivers/net/ixgbe/ixgbe_rxtx.c | 35 ++++++++++++++++++++++------------- 3 files changed, 30 insertions(+), 20 deletions(-)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c index 4ca5747..114698d 100644 --- a/drivers/net/ixgbe/ixgbe_ethdev.c +++ b/drivers/net/ixgbe/ixgbe_ethdev.c @@ -1977,6 +1977,8 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) /* check multi-queue mode */ switch (dev_conf->rxmode.mq_mode) { case ETH_MQ_RX_VMDQ_DCB: + PMD_INIT_LOG(INFO, "ETH_MQ_RX_VMDQ_DCB mode supported in SRIOV"); + break; case ETH_MQ_RX_VMDQ_DCB_RSS: /* DCB/RSS VMDQ in SRIOV mode, not implement yet */ PMD_INIT_LOG(ERR, "SRIOV active," @@ -2012,11 +2014,8 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) switch (dev_conf->txmode.mq_mode) { case ETH_MQ_TX_VMDQ_DCB: - /* DCB VMDQ in SRIOV mode, not implement yet */ - PMD_INIT_LOG(ERR, "SRIOV is active," - " unsupported VMDQ mq_mode tx %d.", - dev_conf->txmode.mq_mode); - return -EINVAL; + PMD_INIT_LOG(INFO, "ETH_MQ_TX_VMDQ_DCB mode supported in SRIOV"); + break; default: /* ETH_MQ_TX_VMDQ_ONLY or ETH_MQ_TX_NONE */ dev->data->dev_conf.txmode.mq_mode = ETH_MQ_TX_VMDQ_ONLY; break; diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c index 2ce8234..bb13889 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx.c +++ b/drivers/net/ixgbe/ixgbe_rxtx.c @@ -1,7 +1,7 @@ /*- * BSD LICENSE * - * Copyright(c) 2010-2015 Intel Corporation. All rights reserved. + * Copyright(c) 2010-2016 Intel Corporation. All rights reserved. * Copyright 2014 6WIND S.A. * All rights reserved. * @@ -3313,15 +3313,16 @@ ixgbe_vmdq_dcb_configure(struct rte_eth_dev *dev) /** * ixgbe_dcb_config_tx_hw_config - Configure general DCB TX parameters - * @hw: pointer to hardware structure + * @dev: pointer to eth_dev structure * @dcb_config: pointer to ixgbe_dcb_config structure */ static void -ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, +ixgbe_dcb_tx_hw_config(struct rte_eth_dev *dev, struct ixgbe_dcb_config *dcb_config) { uint32_t reg; uint32_t q; + struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private); PMD_INIT_FUNC_TRACE(); if (hw->mac.type != ixgbe_mac_82598EB) { @@ -3339,11 +3340,17 @@ ixgbe_dcb_tx_hw_config(struct ixgbe_hw *hw, if (dcb_config->vt_mode) reg |= IXGBE_MTQC_VT_ENA; IXGBE_WRITE_REG(hw, IXGBE_MTQC, reg); - - /* Disable drop for all queues */ - for (q = 0; q < 128; q++) - IXGBE_WRITE_REG(hw, IXGBE_QDE, - (IXGBE_QDE_WRITE | (q << IXGBE_QDE_IDX_SHIFT))); + if (RTE_ETH_DEV_SRIOV(dev).active == 0) { + /* Disable drop for all queues in VMDQ mode*/ + for (q = 0; q < 128; q++) + IXGBE_WRITE_REG(hw, IXGBE_QDE, + (IXGBE_QDE_WRITE | (q << IXGBE_QDE_IDX_SHIFT) | IXGBE_QDE_ENABLE)); + } else { + /* Enable drop for all queues in SRIOV mode */ + for (q = 0; q < 128; q++) + IXGBE_WRITE_REG(hw, IXGBE_QDE, + (IXGBE_QDE_WRITE | (q << IXGBE_QDE_IDX_SHIFT))); + } /* Enable the Tx desc arbiter */ reg = IXGBE_READ_REG(hw, IXGBE_RTTDCS); @@ -3378,7 +3385,7 @@ ixgbe_vmdq_dcb_hw_tx_config(struct rte_eth_dev *dev, vmdq_tx_conf->nb_queue_pools == ETH_16_POOLS ? 0xFFFF : 0xFFFFFFFF); /*Configure general DCB TX parameters*/ - ixgbe_dcb_tx_hw_config(hw, dcb_config); + ixgbe_dcb_tx_hw_config(dev, dcb_config); } static void @@ -3661,7 +3668,7 @@ ixgbe_dcb_hw_configure(struct rte_eth_dev *dev, /*get DCB TX configuration parameters from rte_eth_conf*/ ixgbe_dcb_tx_config(dev, dcb_config); /*Configure general DCB TX parameters*/ - ixgbe_dcb_tx_hw_config(hw, dcb_config); + ixgbe_dcb_tx_hw_config(dev, dcb_config); break; default: PMD_INIT_LOG(ERR, "Incorrect DCB TX mode configuration"); @@ -3810,9 +3817,6 @@ void ixgbe_configure_dcb(struct rte_eth_dev *dev) (dev_conf->rxmode.mq_mode != ETH_MQ_RX_DCB_RSS)) return; - if (dev->data->nb_rx_queues != ETH_DCB_NUM_QUEUES) - return; - /** Configure DCB hardware **/ ixgbe_dcb_hw_configure(dev, dcb_cfg); } @@ -4082,12 +4086,13 @@ ixgbe_dev_mq_rx_configure(struct rte_eth_dev *dev) case ETH_MQ_RX_VMDQ_RSS: ixgbe_config_vf_rss(dev); break; - - /* FIXME if support DCB/RSS together with VMDq & SRIOV */ case ETH_MQ_RX_VMDQ_DCB: + ixgbe_vmdq_dcb_configure(dev); + break; + /* FIXME if support DCB/RSS together with VMDq & SRIOV */ case ETH_MQ_RX_VMDQ_DCB_RSS: PMD_INIT_LOG(ERR, - "Could not support DCB with VMDq & SRIOV"); + "Could not support DCB/RSS with VMDq & SRIOV"); return -1; default: ixgbe_config_vf_default(dev);