From patchwork Tue Sep 17 09:09:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 59323 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F34411C00F; Tue, 17 Sep 2019 11:23:52 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id A95191C00E for ; Tue, 17 Sep 2019 11:23:51 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Sep 2019 02:23:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,515,1559545200"; d="scan'208";a="387501522" Received: from dpdk-dipei.sh.intel.com ([10.67.110.224]) by fmsmga006.fm.intel.com with ESMTP; 17 Sep 2019 02:23:49 -0700 From: Andy Pei To: dev@dpdk.org Cc: rosen.xu@intel.com, xiaolong.ye@intel.com, tiwei.bie@intel.com, xiao.w.wang@intel.com Date: Tue, 17 Sep 2019 17:09:46 +0800 Message-Id: <1568711388-257817-1-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 Subject: [dpdk-dev] [PATCH v2 1/3] net/ifcvf: add multiqueue configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is in preparation for multiqueue enabling for vDPA devices. Signed-off-by: Xiaolong Ye Signed-off-by: Andy Pei --- drivers/net/ifc/base/ifcvf.c | 1 + drivers/net/ifc/base/ifcvf.h | 2 ++ 2 files changed, 3 insertions(+) diff --git a/drivers/net/ifc/base/ifcvf.c b/drivers/net/ifc/base/ifcvf.c index 3c0b2df..6462281 100644 --- a/drivers/net/ifc/base/ifcvf.c +++ b/drivers/net/ifc/base/ifcvf.c @@ -82,6 +82,7 @@ } hw->lm_cfg = hw->mem_resource[4].addr; + hw->mq_cfg = hw->mem_resource[4].addr + IFCVF_MQ_OFFSET; if (hw->common_cfg == NULL || hw->notify_base == NULL || hw->isr == NULL || hw->dev_cfg == NULL) { diff --git a/drivers/net/ifc/base/ifcvf.h b/drivers/net/ifc/base/ifcvf.h index 9be2770..a4cb1a4 100644 --- a/drivers/net/ifc/base/ifcvf.h +++ b/drivers/net/ifc/base/ifcvf.h @@ -38,6 +38,7 @@ #define IFCVF_LM_CFG_SIZE 0x40 #define IFCVF_LM_RING_STATE_OFFSET 0x20 +#define IFCVF_MQ_OFFSET 0x28 #define IFCVF_LM_LOGGING_CTRL 0x0 @@ -127,6 +128,7 @@ struct ifcvf_hw { u16 *notify_base; u16 *notify_addr[IFCVF_MAX_QUEUES * 2]; u8 *lm_cfg; + u8 *mq_cfg; struct vring_info vring[IFCVF_MAX_QUEUES * 2]; u8 nr_vring; struct ifcvf_pci_mem_resource mem_resource[IFCVF_PCI_MAX_RESOURCE]; From patchwork Tue Sep 17 09:09:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 59324 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6FC6D1C06B; Tue, 17 Sep 2019 11:23:57 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id 5E7D71C069 for ; Tue, 17 Sep 2019 11:23:55 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Sep 2019 02:23:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,515,1559545200"; d="scan'208";a="387501548" Received: from dpdk-dipei.sh.intel.com ([10.67.110.224]) by fmsmga006.fm.intel.com with ESMTP; 17 Sep 2019 02:23:53 -0700 From: Andy Pei To: dev@dpdk.org Cc: rosen.xu@intel.com, xiaolong.ye@intel.com, tiwei.bie@intel.com, xiao.w.wang@intel.com Date: Tue, 17 Sep 2019 17:09:47 +0800 Message-Id: <1568711388-257817-2-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1568711388-257817-1-git-send-email-andy.pei@intel.com> References: <1568711388-257817-1-git-send-email-andy.pei@intel.com> Subject: [dpdk-dev] [PATCH v2 2/3] vhost: call vDPA callback at the end of vring enable handler X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" vDPA's set_vring_state callback would need to know the virtqueues' enable status to configure the hardware. Signed-off-by: Xiaolong Ye Signed-off-by: Andy Pei --- v2: add nr_active_vring as a parameter to ops function set_vring_state in case of callback in set_vring_state() and avoid exposing new API. lib/librte_vhost/rte_vdpa.h | 4 ++-- lib/librte_vhost/vhost_user.c | 27 +++++++++++++++++++++++++-- 2 files changed, 27 insertions(+), 4 deletions(-) diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h index 9a3deb3..6e55d4d 100644 --- a/lib/librte_vhost/rte_vdpa.h +++ b/lib/librte_vhost/rte_vdpa.h @@ -54,8 +54,8 @@ struct rte_vdpa_dev_ops { int (*dev_conf)(int vid); int (*dev_close)(int vid); - /** Enable/disable this vring */ - int (*set_vring_state)(int vid, int vring, int state); + /** Enable/disable vring queue pairs */ + int (*set_vring_state)(int vid, int nr_active_vring); /** Set features when changed */ int (*set_features)(int vid); diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c index 0b72648..4d7de44 100644 --- a/lib/librte_vhost/vhost_user.c +++ b/lib/librte_vhost/vhost_user.c @@ -1325,6 +1325,25 @@ static int vhost_user_set_vring_err(struct virtio_net **pdev __rte_unused, return RTE_VHOST_MSG_RESULT_REPLY; } +static uint16_t +vhost_get_active_vring_num(int vid) +{ + struct virtio_net *dev = get_device(vid); + struct vhost_virtqueue *vq; + uint16_t qid; + + if (dev == NULL) + return 0; + + for (qid = 0; qid < dev->nr_vring; qid++) { + vq = dev->virtqueue[qid]; + if (!vq->enabled) + break; + } + + return qid; +} + /* * when virtio queues are ready to work, qemu will send us to * enable the virtio queue pair. @@ -1339,6 +1358,7 @@ static int vhost_user_set_vring_err(struct virtio_net **pdev __rte_unused, int index = (int)msg->payload.state.index; struct rte_vdpa_device *vdpa_dev; int did = -1; + int nr_active_vring; RTE_LOG(INFO, VHOST_CONFIG, "set queue enable: %d to qp idx: %d\n", @@ -1346,8 +1366,6 @@ static int vhost_user_set_vring_err(struct virtio_net **pdev __rte_unused, did = dev->vdpa_dev_id; vdpa_dev = rte_vdpa_get_device(did); - if (vdpa_dev && vdpa_dev->ops->set_vring_state) - vdpa_dev->ops->set_vring_state(dev->vid, index, enable); if (dev->notify_ops->vring_state_changed) dev->notify_ops->vring_state_changed(dev->vid, @@ -1359,6 +1377,11 @@ static int vhost_user_set_vring_err(struct virtio_net **pdev __rte_unused, dev->virtqueue[index]->enabled = enable; + if (vdpa_dev && vdpa_dev->ops->set_vring_state) { + nr_active_vring = vhost_get_active_vring_num(dev->vid); + vdpa_dev->ops->set_vring_state(dev->vid, nr_active_vring); + } + return RTE_VHOST_MSG_RESULT_OK; } From patchwork Tue Sep 17 09:09:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 59325 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 60A5A1C08D; Tue, 17 Sep 2019 11:23:59 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id 1A6C41C07C for ; Tue, 17 Sep 2019 11:23:57 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Sep 2019 02:23:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,515,1559545200"; d="scan'208";a="387501561" Received: from dpdk-dipei.sh.intel.com ([10.67.110.224]) by fmsmga006.fm.intel.com with ESMTP; 17 Sep 2019 02:23:56 -0700 From: Andy Pei To: dev@dpdk.org Cc: rosen.xu@intel.com, xiaolong.ye@intel.com, tiwei.bie@intel.com, xiao.w.wang@intel.com Date: Tue, 17 Sep 2019 17:09:48 +0800 Message-Id: <1568711388-257817-3-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1568711388-257817-1-git-send-email-andy.pei@intel.com> References: <1568711388-257817-1-git-send-email-andy.pei@intel.com> Subject: [dpdk-dev] [PATCH v2 3/3] net/ifcvf: enable mutliqueue support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Enable mutliqueue support for ifcvf devices by setting the mutliqueue configuration space. Signed-off-by: Xiaolong Ye Signed-off-by: Andy Pei --- drivers/net/ifc/base/ifcvf.c | 8 ++++++++ drivers/net/ifc/base/ifcvf.h | 5 ++++- drivers/net/ifc/ifcvf_vdpa.c | 30 ++++++++++++++++++++++++++++-- 3 files changed, 40 insertions(+), 3 deletions(-) diff --git a/drivers/net/ifc/base/ifcvf.c b/drivers/net/ifc/base/ifcvf.c index 6462281..ac09537 100644 --- a/drivers/net/ifc/base/ifcvf.c +++ b/drivers/net/ifc/base/ifcvf.c @@ -310,6 +310,14 @@ *(u32 *)(lm_cfg + IFCVF_LM_LOGGING_CTRL) = IFCVF_LM_DISABLE; } +void ifcvf_enable_multiqueue(struct ifcvf_hw *hw, u16 nr_queue_pair) +{ + u8 *mq_cfg; + + mq_cfg = hw->mq_cfg; + *(u32 *)mq_cfg = nr_queue_pair; +} + void ifcvf_notify_queue(struct ifcvf_hw *hw, u16 qid) { diff --git a/drivers/net/ifc/base/ifcvf.h b/drivers/net/ifc/base/ifcvf.h index a4cb1a4..3eb0bf3 100644 --- a/drivers/net/ifc/base/ifcvf.h +++ b/drivers/net/ifc/base/ifcvf.h @@ -12,7 +12,7 @@ #define IFCVF_SUBSYS_VENDOR_ID 0x8086 #define IFCVF_SUBSYS_DEVICE_ID 0x001A -#define IFCVF_MAX_QUEUES 1 +#define IFCVF_MAX_QUEUES 32 #define VIRTIO_F_IOMMU_PLATFORM 33 /* Common configuration */ @@ -153,6 +153,9 @@ struct ifcvf_hw { ifcvf_disable_logging(struct ifcvf_hw *hw); void +ifcvf_enable_multiqueue(struct ifcvf_hw *hw, u16 nr_queue_pair); + +void ifcvf_notify_queue(struct ifcvf_hw *hw, u16 qid); u8 diff --git a/drivers/net/ifc/ifcvf_vdpa.c b/drivers/net/ifc/ifcvf_vdpa.c index 8de9ef1..b3f71c4 100644 --- a/drivers/net/ifc/ifcvf_vdpa.c +++ b/drivers/net/ifc/ifcvf_vdpa.c @@ -994,6 +994,30 @@ struct internal_list { } static int +ifcvf_set_vring_state(int vid, int nr_active_vring) +{ + int did, nr_queue_pair; + struct internal_list *list; + + if (nr_active_vring == 0) { + DRV_LOG(ERR, "No enabled vring"); + return -1; + } + nr_queue_pair = (nr_active_vring + 1) / 2; + + did = rte_vhost_get_vdpa_device_id(vid); + list = find_internal_resource_by_did(did); + if (list == NULL) { + DRV_LOG(ERR, "Invalid device id: %d", did); + return -1; + } + + ifcvf_enable_multiqueue(&list->internal->hw, nr_queue_pair); + + return 0; +} + +static int ifcvf_get_notify_area(int vid, int qid, uint64_t *offset, uint64_t *size) { int did; @@ -1062,7 +1086,9 @@ struct internal_list { 1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ | \ 1ULL << VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD | \ 1ULL << VHOST_USER_PROTOCOL_F_HOST_NOTIFIER | \ - 1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD) + 1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD | \ + 1ULL << VHOST_USER_PROTOCOL_F_MQ) + static int ifcvf_get_protocol_features(int did __rte_unused, uint64_t *features) { @@ -1076,7 +1102,7 @@ struct internal_list { .get_protocol_features = ifcvf_get_protocol_features, .dev_conf = ifcvf_dev_config, .dev_close = ifcvf_dev_close, - .set_vring_state = NULL, + .set_vring_state = ifcvf_set_vring_state, .set_features = ifcvf_set_features, .migration_done = NULL, .get_vfio_group_fd = ifcvf_get_vfio_group_fd,