From patchwork Fri Sep 6 03:20:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 58706 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0B6A81F0D4; Fri, 6 Sep 2019 05:33:33 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 2A3051F0B2 for ; Fri, 6 Sep 2019 05:33:30 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Sep 2019 20:33:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,472,1559545200"; d="scan'208";a="383123521" Received: from dpdk-dipei.sh.intel.com ([10.67.110.224]) by fmsmga005.fm.intel.com with ESMTP; 05 Sep 2019 20:33:28 -0700 From: Andy Pei To: dev@dpdk.org Cc: rosen.xu@intel.com, xiaolong.ye@intel.com, tiwei.bie@intel.com, xiao.w.wang@intel.com Date: Fri, 6 Sep 2019 11:20:48 +0800 Message-Id: <1567740051-367172-1-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 Subject: [dpdk-dev] [PATCH 1/4] vhost: introduce new API to get the active vring number X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" It's useful for hardware vhost backend (like vDPA devices) to set multiqueue configuration accordingly. Signed-off-by: Xiaolong Ye Signed-off-by: Andy Pei --- lib/librte_vhost/rte_vhost.h | 12 ++++++++++++ lib/librte_vhost/vhost.c | 19 +++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h index 7fb1729..28811b0 100644 --- a/lib/librte_vhost/rte_vhost.h +++ b/lib/librte_vhost/rte_vhost.h @@ -525,6 +525,18 @@ int rte_vhost_driver_callback_register(const char *path, uint16_t rte_vhost_get_vring_num(int vid); /** + * Get the number of active vrings of the device. + * + * @param vid + * vhost device ID + * + * @return + * The number of active vrings, 0 on failure + */ +uint16_t +rte_vhost_get_active_vring_num(int vid); + +/** * Get the virtio net device's ifname, which is the vhost-user socket * file path. * diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c index 981837b..c714818 100644 --- a/lib/librte_vhost/vhost.c +++ b/lib/librte_vhost/vhost.c @@ -674,6 +674,25 @@ return dev->nr_vring; } +uint16_t +rte_vhost_get_active_vring_num(int vid) +{ + struct virtio_net *dev = get_device(vid); + struct vhost_virtqueue *vq; + uint16_t qid; + + if (dev == NULL) + return 0; + + for (qid = 0; qid < dev->nr_vring; qid++) { + vq = dev->virtqueue[qid]; + if (!vq->enabled) + break; + } + + return qid; +} + int rte_vhost_get_ifname(int vid, char *buf, size_t len) { From patchwork Fri Sep 6 03:20:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 58707 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5F1491F0C9; Fri, 6 Sep 2019 05:33:35 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 932491F0B2 for ; Fri, 6 Sep 2019 05:33:32 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Sep 2019 20:33:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,472,1559545200"; d="scan'208";a="383123535" Received: from dpdk-dipei.sh.intel.com ([10.67.110.224]) by fmsmga005.fm.intel.com with ESMTP; 05 Sep 2019 20:33:30 -0700 From: Andy Pei To: dev@dpdk.org Cc: rosen.xu@intel.com, xiaolong.ye@intel.com, tiwei.bie@intel.com, xiao.w.wang@intel.com Date: Fri, 6 Sep 2019 11:20:49 +0800 Message-Id: <1567740051-367172-2-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1567740051-367172-1-git-send-email-andy.pei@intel.com> References: <1567740051-367172-1-git-send-email-andy.pei@intel.com> Subject: [dpdk-dev] [PATCH 2/4] net/ifcvf: add multiqueue configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is in preparation for multiqueue enabling for vDPA devices. Signed-off-by: Xiaolong Ye Signed-off-by: Andy Pei --- drivers/net/ifc/base/ifcvf.c | 1 + drivers/net/ifc/base/ifcvf.h | 2 ++ 2 files changed, 3 insertions(+) diff --git a/drivers/net/ifc/base/ifcvf.c b/drivers/net/ifc/base/ifcvf.c index 3c0b2df..6462281 100644 --- a/drivers/net/ifc/base/ifcvf.c +++ b/drivers/net/ifc/base/ifcvf.c @@ -82,6 +82,7 @@ } hw->lm_cfg = hw->mem_resource[4].addr; + hw->mq_cfg = hw->mem_resource[4].addr + IFCVF_MQ_OFFSET; if (hw->common_cfg == NULL || hw->notify_base == NULL || hw->isr == NULL || hw->dev_cfg == NULL) { diff --git a/drivers/net/ifc/base/ifcvf.h b/drivers/net/ifc/base/ifcvf.h index 9be2770..a4cb1a4 100644 --- a/drivers/net/ifc/base/ifcvf.h +++ b/drivers/net/ifc/base/ifcvf.h @@ -38,6 +38,7 @@ #define IFCVF_LM_CFG_SIZE 0x40 #define IFCVF_LM_RING_STATE_OFFSET 0x20 +#define IFCVF_MQ_OFFSET 0x28 #define IFCVF_LM_LOGGING_CTRL 0x0 @@ -127,6 +128,7 @@ struct ifcvf_hw { u16 *notify_base; u16 *notify_addr[IFCVF_MAX_QUEUES * 2]; u8 *lm_cfg; + u8 *mq_cfg; struct vring_info vring[IFCVF_MAX_QUEUES * 2]; u8 nr_vring; struct ifcvf_pci_mem_resource mem_resource[IFCVF_PCI_MAX_RESOURCE]; From patchwork Fri Sep 6 03:20:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 58708 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 742B21F0E2; Fri, 6 Sep 2019 05:33:37 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 284841F0D9 for ; Fri, 6 Sep 2019 05:33:34 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Sep 2019 20:33:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,472,1559545200"; d="scan'208";a="383123553" Received: from dpdk-dipei.sh.intel.com ([10.67.110.224]) by fmsmga005.fm.intel.com with ESMTP; 05 Sep 2019 20:33:33 -0700 From: Andy Pei To: dev@dpdk.org Cc: rosen.xu@intel.com, xiaolong.ye@intel.com, tiwei.bie@intel.com, xiao.w.wang@intel.com Date: Fri, 6 Sep 2019 11:20:50 +0800 Message-Id: <1567740051-367172-3-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1567740051-367172-1-git-send-email-andy.pei@intel.com> References: <1567740051-367172-1-git-send-email-andy.pei@intel.com> Subject: [dpdk-dev] [PATCH 3/4] vhost: call vDPA callback at the end of vring enable handler X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" vDPA's set_vring_state callback would need to know the virtqueues' enable status to configure the hardware. Signed-off-by: Xiaolong Ye Signed-off-by: Andy Pei --- lib/librte_vhost/rte_vdpa.h | 4 ++-- lib/librte_vhost/vhost_user.c | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h index 9a3deb3..19cf897 100644 --- a/lib/librte_vhost/rte_vdpa.h +++ b/lib/librte_vhost/rte_vdpa.h @@ -54,8 +54,8 @@ struct rte_vdpa_dev_ops { int (*dev_conf)(int vid); int (*dev_close)(int vid); - /** Enable/disable this vring */ - int (*set_vring_state)(int vid, int vring, int state); + /** Enable/disable vring queue pairs */ + int (*set_vring_state)(int vid); /** Set features when changed */ int (*set_features)(int vid); diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c index 0b72648..21028cc 100644 --- a/lib/librte_vhost/vhost_user.c +++ b/lib/librte_vhost/vhost_user.c @@ -1346,8 +1346,6 @@ static int vhost_user_set_vring_err(struct virtio_net **pdev __rte_unused, did = dev->vdpa_dev_id; vdpa_dev = rte_vdpa_get_device(did); - if (vdpa_dev && vdpa_dev->ops->set_vring_state) - vdpa_dev->ops->set_vring_state(dev->vid, index, enable); if (dev->notify_ops->vring_state_changed) dev->notify_ops->vring_state_changed(dev->vid, @@ -1359,6 +1357,9 @@ static int vhost_user_set_vring_err(struct virtio_net **pdev __rte_unused, dev->virtqueue[index]->enabled = enable; + if (vdpa_dev && vdpa_dev->ops->set_vring_state) + vdpa_dev->ops->set_vring_state(dev->vid); + return RTE_VHOST_MSG_RESULT_OK; } From patchwork Fri Sep 6 03:20:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 58709 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B3BC11F0E8; Fri, 6 Sep 2019 05:33:39 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id A2E2C1F0E4 for ; Fri, 6 Sep 2019 05:33:37 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Sep 2019 20:33:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,472,1559545200"; d="scan'208";a="383123568" Received: from dpdk-dipei.sh.intel.com ([10.67.110.224]) by fmsmga005.fm.intel.com with ESMTP; 05 Sep 2019 20:33:35 -0700 From: Andy Pei To: dev@dpdk.org Cc: rosen.xu@intel.com, xiaolong.ye@intel.com, tiwei.bie@intel.com, xiao.w.wang@intel.com Date: Fri, 6 Sep 2019 11:20:51 +0800 Message-Id: <1567740051-367172-4-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1567740051-367172-1-git-send-email-andy.pei@intel.com> References: <1567740051-367172-1-git-send-email-andy.pei@intel.com> Subject: [dpdk-dev] [PATCH 4/4] net/ifcvf: enable mutliqueue support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Enable mutliqueue support for ifcvf devices by setting the mutliqueue configuration space. Signed-off-by: Xiaolong Ye Signed-off-by: Andy Pei --- drivers/net/ifc/base/ifcvf.c | 8 ++++++++ drivers/net/ifc/base/ifcvf.h | 5 ++++- drivers/net/ifc/ifcvf_vdpa.c | 31 +++++++++++++++++++++++++++++-- 3 files changed, 41 insertions(+), 3 deletions(-) diff --git a/drivers/net/ifc/base/ifcvf.c b/drivers/net/ifc/base/ifcvf.c index 6462281..ac09537 100644 --- a/drivers/net/ifc/base/ifcvf.c +++ b/drivers/net/ifc/base/ifcvf.c @@ -310,6 +310,14 @@ *(u32 *)(lm_cfg + IFCVF_LM_LOGGING_CTRL) = IFCVF_LM_DISABLE; } +void ifcvf_enable_multiqueue(struct ifcvf_hw *hw, u16 nr_queue_pair) +{ + u8 *mq_cfg; + + mq_cfg = hw->mq_cfg; + *(u32 *)mq_cfg = nr_queue_pair; +} + void ifcvf_notify_queue(struct ifcvf_hw *hw, u16 qid) { diff --git a/drivers/net/ifc/base/ifcvf.h b/drivers/net/ifc/base/ifcvf.h index a4cb1a4..3eb0bf3 100644 --- a/drivers/net/ifc/base/ifcvf.h +++ b/drivers/net/ifc/base/ifcvf.h @@ -12,7 +12,7 @@ #define IFCVF_SUBSYS_VENDOR_ID 0x8086 #define IFCVF_SUBSYS_DEVICE_ID 0x001A -#define IFCVF_MAX_QUEUES 1 +#define IFCVF_MAX_QUEUES 32 #define VIRTIO_F_IOMMU_PLATFORM 33 /* Common configuration */ @@ -153,6 +153,9 @@ struct ifcvf_hw { ifcvf_disable_logging(struct ifcvf_hw *hw); void +ifcvf_enable_multiqueue(struct ifcvf_hw *hw, u16 nr_queue_pair); + +void ifcvf_notify_queue(struct ifcvf_hw *hw, u16 qid); u8 diff --git a/drivers/net/ifc/ifcvf_vdpa.c b/drivers/net/ifc/ifcvf_vdpa.c index 8de9ef1..be71f92 100644 --- a/drivers/net/ifc/ifcvf_vdpa.c +++ b/drivers/net/ifc/ifcvf_vdpa.c @@ -994,6 +994,31 @@ struct internal_list { } static int +ifcvf_set_vring_state(int vid) +{ + int did, nr_active_vring, nr_queue_pair; + struct internal_list *list; + + nr_active_vring = rte_vhost_get_active_vring_num(vid); + if (nr_active_vring == 0) { + DRV_LOG(ERR, "No enabled vring"); + return -1; + } + nr_queue_pair = (nr_active_vring + 1) / 2; + + did = rte_vhost_get_vdpa_device_id(vid); + list = find_internal_resource_by_did(did); + if (list == NULL) { + DRV_LOG(ERR, "Invalid device id: %d", did); + return -1; + } + + ifcvf_enable_multiqueue(&list->internal->hw, nr_queue_pair); + + return 0; +} + +static int ifcvf_get_notify_area(int vid, int qid, uint64_t *offset, uint64_t *size) { int did; @@ -1062,7 +1087,9 @@ struct internal_list { 1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ | \ 1ULL << VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD | \ 1ULL << VHOST_USER_PROTOCOL_F_HOST_NOTIFIER | \ - 1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD) + 1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD | \ + 1ULL << VHOST_USER_PROTOCOL_F_MQ) + static int ifcvf_get_protocol_features(int did __rte_unused, uint64_t *features) { @@ -1076,7 +1103,7 @@ struct internal_list { .get_protocol_features = ifcvf_get_protocol_features, .dev_conf = ifcvf_dev_config, .dev_close = ifcvf_dev_close, - .set_vring_state = NULL, + .set_vring_state = ifcvf_set_vring_state, .set_features = ifcvf_set_features, .migration_done = NULL, .get_vfio_group_fd = ifcvf_get_vfio_group_fd,