From patchwork Wed Dec 19 13:37:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 49096 X-Patchwork-Delegate: qi.z.zhang@intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9A0661BB29; Wed, 19 Dec 2018 07:02:29 +0100 (CET) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 3FFCD1B6C3 for ; Wed, 19 Dec 2018 07:02:22 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Dec 2018 22:02:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,371,1539673200"; d="scan'208";a="110558277" Received: from dpdk-server.sh.intel.com ([10.67.110.170]) by fmsmga008.fm.intel.com with ESMTP; 18 Dec 2018 22:02:19 -0800 From: Zhirun Yan To: dev@dpdk.org, qi.z.zhang@intel.com Cc: Zhirun Yan , Haiyue Wang Date: Wed, 19 Dec 2018 13:37:41 +0000 Message-Id: <20181219133743.57423-2-zhirun.yan@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181219133743.57423-1-zhirun.yan@intel.com> References: <20181219130856.19862-4-zhirun.yan@intel.com> <20181219133743.57423-1-zhirun.yan@intel.com> Subject: [dpdk-dev] [PATCH v8 1/3] net/i40e: support VF request more queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Before this patch, VF gets a default number of queues from the PF. This patch enables VF to request a different number. When VF configures more queues, it will send VIRTCHNL_OP_REQUEST_QUEUES to PF to request more queues, if success, PF will reset the VF. User can run "port stop all", "port config port_id rxq/txq queue_num" and "port start all" to reconfigure queue number. Signed-off-by: Zhirun Yan Signed-off-by: Haiyue Wang --- drivers/net/i40e/i40e_ethdev_vf.c | 81 +++++++++++++++++++++++++++---- 1 file changed, 71 insertions(+), 10 deletions(-) diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c index 05dc6596b..22d39a586 100644 --- a/drivers/net/i40e/i40e_ethdev_vf.c +++ b/drivers/net/i40e/i40e_ethdev_vf.c @@ -359,6 +359,28 @@ i40evf_execute_vf_cmd(struct rte_eth_dev *dev, struct vf_cmd_info *args) } while (i++ < MAX_TRY_TIMES); _clear_cmd(vf); break; + case VIRTCHNL_OP_REQUEST_QUEUES: + /** + * ignore async reply, only wait for system message, + * vf_reset = true if get VIRTCHNL_EVENT_RESET_IMPENDING, + * if not, means request queues failed. + */ + err = -1; + do { + ret = i40evf_read_pfmsg(dev, &info); + vf->cmd_retval = info.result; + if (ret == I40EVF_MSG_SYS && vf->vf_reset) { + err = 0; + break; + } else if (ret == I40EVF_MSG_ERR || + ret == I40EVF_MSG_CMD) { + break; + } + rte_delay_ms(ASQ_DELAY_MS); + /* If don't read msg or read sys event, continue */ + } while (i++ < MAX_TRY_TIMES); + _clear_cmd(vf); + break; default: /* for other adminq in running time, waiting the cmd done flag */ @@ -1012,6 +1034,28 @@ i40evf_add_vlan(struct rte_eth_dev *dev, uint16_t vlanid) return err; } +static int +i40evf_request_queues(struct rte_eth_dev *dev, uint16_t num) +{ + struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private); + struct virtchnl_vf_res_request vfres; + struct vf_cmd_info args; + int err; + + vfres.num_queue_pairs = num; + + args.ops = VIRTCHNL_OP_REQUEST_QUEUES; + args.in_args = (u8 *)&vfres; + args.in_args_size = sizeof(vfres); + args.out_buffer = vf->aq_resp; + args.out_size = I40E_AQ_BUF_SZ; + err = i40evf_execute_vf_cmd(dev, &args); + if (err) + PMD_DRV_LOG(ERR, "fail to execute command OP_REQUEST_QUEUES"); + + return err; +} + static int i40evf_del_vlan(struct rte_eth_dev *dev, uint16_t vlanid) { @@ -1109,14 +1153,14 @@ i40evf_reset_vf(struct i40e_hw *hw) return -1; } /** - * After issuing vf reset command to pf, pf won't necessarily - * reset vf, it depends on what state it exactly is. If it's not - * initialized yet, it won't have vf reset since it's in a certain - * state. If not, it will try to reset. Even vf is reset, pf will - * set I40E_VFGEN_RSTAT to COMPLETE first, then wait 10ms and set - * it to ACTIVE. In this duration, vf may not catch the moment that - * COMPLETE is set. So, for vf, we'll try to wait a long time. - */ + * After issuing vf reset command to pf, pf won't necessarily + * reset vf, it depends on what state it exactly is. If it's not + * initialized yet, it won't have vf reset since it's in a certain + * state. If not, it will try to reset. Even vf is reset, pf will + * set I40E_VFGEN_RSTAT to COMPLETE first, then wait 10ms and set + * it to ACTIVE. In this duration, vf may not catch the moment that + * COMPLETE is set. So, for vf, we'll try to wait a long time. + */ rte_delay_ms(200); ret = i40evf_check_vf_reset_done(hw); @@ -1516,8 +1560,11 @@ RTE_PMD_REGISTER_KMOD_DEP(net_i40e_vf, "* igb_uio | vfio-pci"); static int i40evf_dev_configure(struct rte_eth_dev *dev) { + struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private); struct i40e_adapter *ad = I40E_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); + uint16_t num_queue_pairs = RTE_MAX(dev->data->nb_rx_queues, + dev->data->nb_tx_queues); /* Initialize to TRUE. If any of Rx queues doesn't meet the bulk * allocation or vector Rx preconditions we will reset it. @@ -1527,6 +1574,20 @@ i40evf_dev_configure(struct rte_eth_dev *dev) ad->tx_simple_allowed = true; ad->tx_vec_allowed = true; + if (num_queue_pairs > vf->vsi_res->num_queue_pairs) { + int ret = 0; + + PMD_DRV_LOG(INFO, "change queue pairs from %u to %u", + vf->vsi_res->num_queue_pairs, num_queue_pairs); + ret = i40evf_request_queues(dev, num_queue_pairs); + if (ret != 0) + return ret; + + ret = i40evf_dev_reset(dev); + if (ret != 0) + return ret; + } + return i40evf_init_vlan(dev); } @@ -2145,8 +2206,8 @@ i40evf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) { struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private); - dev_info->max_rx_queues = vf->vsi_res->num_queue_pairs; - dev_info->max_tx_queues = vf->vsi_res->num_queue_pairs; + dev_info->max_rx_queues = I40E_MAX_QP_NUM_PER_VF; + dev_info->max_tx_queues = I40E_MAX_QP_NUM_PER_VF; dev_info->min_rx_bufsize = I40E_BUF_SIZE_MIN; dev_info->max_rx_pktlen = I40E_FRAME_SIZE_MAX; dev_info->hash_key_size = (I40E_VFQF_HKEY_MAX_INDEX + 1) * sizeof(uint32_t); From patchwork Wed Dec 19 13:37:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 49095 X-Patchwork-Delegate: qi.z.zhang@intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0C63A1B81C; Wed, 19 Dec 2018 07:02:28 +0100 (CET) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 26B721B6B6 for ; Wed, 19 Dec 2018 07:02:23 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Dec 2018 22:02:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,371,1539673200"; d="scan'208";a="110558283" Received: from dpdk-server.sh.intel.com ([10.67.110.170]) by fmsmga008.fm.intel.com with ESMTP; 18 Dec 2018 22:02:20 -0800 From: Zhirun Yan To: dev@dpdk.org, qi.z.zhang@intel.com Cc: Zhirun Yan , Haiyue Wang Date: Wed, 19 Dec 2018 13:37:42 +0000 Message-Id: <20181219133743.57423-3-zhirun.yan@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181219133743.57423-1-zhirun.yan@intel.com> References: <20181219130856.19862-4-zhirun.yan@intel.com> <20181219133743.57423-1-zhirun.yan@intel.com> Subject: [dpdk-dev] [PATCH v8 2/3] net/i40e: support PF respond VF request more queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch respond the VIRTCHNL_OP_REQUEST_QUEUES msg from VF, and process to allocated more queues for the requested VF. If successful, PF will notify VF to reset. If unsuccessful, PF will send message to inform VF. Signed-off-by: Zhirun Yan Signed-off-by: Haiyue Wang --- drivers/net/i40e/i40e_pf.c | 68 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/drivers/net/i40e/i40e_pf.c b/drivers/net/i40e/i40e_pf.c index dd3962d38..092e0d3e9 100644 --- a/drivers/net/i40e/i40e_pf.c +++ b/drivers/net/i40e/i40e_pf.c @@ -1218,6 +1218,69 @@ i40e_notify_vf_link_status(struct rte_eth_dev *dev, struct i40e_pf_vf *vf) I40E_SUCCESS, (uint8_t *)&event, sizeof(event)); } +/** + * i40e_vc_notify_vf_reset + * @vf: pointer to the VF structure + * + * indicate a pending reset to the given VF + **/ +static void +i40e_vc_notify_vf_reset(struct i40e_pf_vf *vf) +{ + struct i40e_hw *hw = I40E_PF_TO_HW(vf->pf); + struct virtchnl_pf_event pfe; + int abs_vf_id; + uint16_t vf_id = vf->vf_idx; + + abs_vf_id = vf_id + hw->func_caps.vf_base_id; + pfe.event = VIRTCHNL_EVENT_RESET_IMPENDING; + pfe.severity = PF_EVENT_SEVERITY_CERTAIN_DOOM; + i40e_aq_send_msg_to_vf(hw, abs_vf_id, VIRTCHNL_OP_EVENT, 0, (u8 *)&pfe, + sizeof(struct virtchnl_pf_event), NULL); +} + +static int +i40e_pf_host_process_cmd_request_queues(struct i40e_pf_vf *vf, uint8_t *msg) +{ + struct virtchnl_vf_res_request *vfres = + (struct virtchnl_vf_res_request *)msg; + struct i40e_pf *pf; + uint32_t req_pairs = vfres->num_queue_pairs; + uint32_t cur_pairs = vf->vsi->nb_used_qps; + + pf = vf->pf; + + if (req_pairs <= 0) { + PMD_DRV_LOG(ERR, + "VF %d tried to request %d queues. Ignoring.\n", + vf->vf_idx, + I40E_MAX_QP_NUM_PER_VF); + } else if (req_pairs > I40E_MAX_QP_NUM_PER_VF) { + PMD_DRV_LOG(ERR, + "VF %d tried to request more than %d queues.\n", + vf->vf_idx, + I40E_MAX_QP_NUM_PER_VF); + vfres->num_queue_pairs = I40E_MAX_QP_NUM_PER_VF; + } else if (req_pairs > cur_pairs + pf->qp_pool.num_free) { + PMD_DRV_LOG(ERR, + "VF %d requested %d more queues, but noly %d left\n", + vf->vf_idx, + req_pairs - cur_pairs, + pf->qp_pool.num_free); + vfres->num_queue_pairs = pf->qp_pool.num_free + cur_pairs; + } else { + i40e_vc_notify_vf_reset(vf); + vf->vsi->nb_qps = req_pairs; + pf->vf_nb_qps = req_pairs; + i40e_pf_host_process_cmd_reset_vf(vf); + + return 0; + } + + return i40e_pf_host_send_msg_to_vf(vf, VIRTCHNL_OP_REQUEST_QUEUES, 0, + (u8 *)vfres, sizeof(*vfres)); +} + void i40e_pf_host_handle_vf_msg(struct rte_eth_dev *dev, uint16_t abs_vf_id, uint32_t opcode, @@ -1351,6 +1414,11 @@ i40e_pf_host_handle_vf_msg(struct rte_eth_dev *dev, PMD_DRV_LOG(INFO, "OP_CONFIG_RSS_KEY received"); i40e_pf_host_process_cmd_set_rss_key(vf, msg, msglen, b_op); break; + case VIRTCHNL_OP_REQUEST_QUEUES: + PMD_DRV_LOG(INFO, "OP_REQUEST_QUEUES received"); + i40e_pf_host_process_cmd_request_queues(vf, msg); + break; + /* Don't add command supported below, which will * return an error code. */ From patchwork Wed Dec 19 13:37:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 49097 X-Patchwork-Delegate: qi.z.zhang@intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4C5BA1BBD1; Wed, 19 Dec 2018 07:02:31 +0100 (CET) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id DA1271B6B6 for ; Wed, 19 Dec 2018 07:02:23 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Dec 2018 22:02:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,371,1539673200"; d="scan'208";a="110558287" Received: from dpdk-server.sh.intel.com ([10.67.110.170]) by fmsmga008.fm.intel.com with ESMTP; 18 Dec 2018 22:02:21 -0800 From: Zhirun Yan To: dev@dpdk.org, qi.z.zhang@intel.com Cc: Zhirun Yan Date: Wed, 19 Dec 2018 13:37:43 +0000 Message-Id: <20181219133743.57423-4-zhirun.yan@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181219133743.57423-1-zhirun.yan@intel.com> References: <20181219130856.19862-4-zhirun.yan@intel.com> <20181219133743.57423-1-zhirun.yan@intel.com> Subject: [dpdk-dev] [PATCH v8 3/3] doc: update queue number per vf for i40e X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Updated the doc and release notes on the support for requesting more queues. Signed-off-by: Zhirun Yan --- doc/guides/nics/i40e.rst | 16 +++++++++------- doc/guides/rel_notes/release_19_02.rst | 8 ++++++++ 2 files changed, 17 insertions(+), 7 deletions(-) diff --git a/doc/guides/nics/i40e.rst b/doc/guides/nics/i40e.rst index bfacbd117..70143d6a0 100644 --- a/doc/guides/nics/i40e.rst +++ b/doc/guides/nics/i40e.rst @@ -129,13 +129,15 @@ Please note that enabling debugging options may affect system performance. Runtime Config Options ~~~~~~~~~~~~~~~~~~~~~~ -- ``Number of Queues per VF`` (default ``4``) - - The number of queue per VF is determined by its host PF. If the PCI address - of an i40e PF is aaaa:bb.cc, the number of queues per VF can be configured - with EAL parameter like -w aaaa:bb.cc,queue-num-per-vf=n. The value n can be - 1, 2, 4, 8 or 16. If no such parameter is configured, the number of queues - per VF is 4 by default. +- ``Reserved number of Queues per VF`` (default ``4``) + + The number of reserved queue per VF is determined by its host PF. If the + PCI address of an i40e PF is aaaa:bb.cc, the number of reserved queues per + VF can be configured with EAL parameter like -w aaaa:bb.cc,queue-num-per-vf=n. + The value n can be 1, 2, 4, 8 or 16. If no such parameter is configured, the + number of reserved queues per VF is 4 by default. If VF request more than + reserved queues per VF, PF will able to allocate max to 16 queues after a VF + reset. - ``Support multiple driver`` (default ``disable``) diff --git a/doc/guides/rel_notes/release_19_02.rst b/doc/guides/rel_notes/release_19_02.rst index 8deb68b9a..524e731b2 100644 --- a/doc/guides/rel_notes/release_19_02.rst +++ b/doc/guides/rel_notes/release_19_02.rst @@ -60,6 +60,14 @@ New Features * Added the handler to get firmware version string. * Added support for multicast filtering. +* **Added dynamic queues allocation support for i40e VF.** + + Previously, available queues of VF is reserved by PF at initialize stage. + Now both DPDK PF and Kernel PF (>=2.1.14) will support dynamic queue + allocation. At runtime, when VF request more queue number exceed the initial + reserved amount, PF can allocate up to 16 queues as the request after a VF + reset. + Removed Items -------------