From patchwork Tue Jul 4 09:49:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 26354 X-Patchwork-Delegate: yuanhan.liu@linux.intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 43DBB7CBE; Tue, 4 Jul 2017 11:50:42 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 39C447CB1 for ; Tue, 4 Jul 2017 11:50:40 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 88E7679701; Tue, 4 Jul 2017 09:50:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 88E7679701 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=maxime.coquelin@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 88E7679701 Received: from localhost.localdomain (ovpn-112-47.ams2.redhat.com [10.36.112.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id BCC9D63F8C; Tue, 4 Jul 2017 09:50:36 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, Yuanhan Liu Cc: mst@redhat.com, vkaplans@redhat.com, jasowang@redhat.com, jfreiman@redhat.com, Maxime Coquelin Date: Tue, 4 Jul 2017 11:49:19 +0200 Message-Id: <20170704094922.11405-17-maxime.coquelin@redhat.com> In-Reply-To: <20170704094922.11405-1-maxime.coquelin@redhat.com> References: <20170704094922.11405-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 04 Jul 2017 09:50:39 +0000 (UTC) Subject: [dpdk-dev] [RFC 16/19] vhost-user: translate ring addresses when IOMMU enabled X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When IOMMU is enabled, the ring addresses set by the VHOST_USER_SET_VRING_ADDR requests are guest's IO virtual addresses, whereas Qemu virtual addresses when IOMMU is disabled. When enabled and the required translation is not in the IOTLB cache, an IOTLB miss request is sent, but being called by the vhost-user socket handling thread, the function does not wait for the requested IOTLB update. The function will be called again on the next IOTLB update message reception if matching the vring addresses. Signed-off-by: Maxime Coquelin --- lib/librte_vhost/vhost_user.c | 43 +++++++++++++++++++++++++++++++++---------- 1 file changed, 33 insertions(+), 10 deletions(-) diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c index d9712e9..8230578 100644 --- a/lib/librte_vhost/vhost_user.c +++ b/lib/librte_vhost/vhost_user.c @@ -290,10 +290,7 @@ numa_realloc(struct virtio_net *dev, int index __rte_unused) } #endif -/* - * Converts QEMU virtual address to Vhost virtual address. This function is - * used to convert the ring addresses to our address space. - */ +/* Converts QEMU virtual address to Vhost virtual address. */ static uint64_t qva_to_vva(struct virtio_net *dev, uint64_t qva) { @@ -314,6 +311,29 @@ qva_to_vva(struct virtio_net *dev, uint64_t qva) return 0; } + +/* + * Converts ring address to Vhost virtual address. + * If IOMMU is enabled, the ring address is a guest IO virtual address, + * else it is a QEMU virtual address. + */ +static uint64_t +ring_addr_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, + uint64_t ra, uint64_t size) +{ + if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM)) { + uint64_t vva; + + vva = vhost_user_iotlb_find(vq, ra, &size, VHOST_ACCESS_RW); + if (!vva) + vhost_user_iotlb_miss(dev, ra, VHOST_ACCESS_RW); + + return vva; + } + + return qva_to_vva(dev, ra); +} + /* * The virtio device sends us the desc, used and avail ring addresses. * This function then converts these to our address space. @@ -346,8 +366,11 @@ static struct virtio_net *translate_ring_addresses(struct virtio_net *dev, struct vhost_vring_addr *addr = &vq->ring_addrs; /* The addresses are converted from QEMU virtual to Vhost virtual. */ - vq->desc = (struct vring_desc *)(uintptr_t)qva_to_vva(dev, - addr->desc_user_addr); + if (vq->desc && vq->avail && vq->used) + return dev; + + vq->desc = (struct vring_desc *)(uintptr_t)ring_addr_to_vva(dev, + vq, addr->desc_user_addr, sizeof(struct vring_desc)); if (vq->desc == 0) { RTE_LOG(ERR, VHOST_CONFIG, "(%d) failed to find desc ring address.\n", @@ -358,8 +381,8 @@ static struct virtio_net *translate_ring_addresses(struct virtio_net *dev, dev = numa_realloc(dev, vq_index); vq = dev->virtqueue[vq_index]; - vq->avail = (struct vring_avail *)(uintptr_t)qva_to_vva(dev, - addr->avail_user_addr); + vq->avail = (struct vring_avail *)(uintptr_t)ring_addr_to_vva(dev, + vq, addr->avail_user_addr, sizeof(struct vring_avail)); if (vq->avail == 0) { RTE_LOG(ERR, VHOST_CONFIG, "(%d) failed to find avail ring address.\n", @@ -367,8 +390,8 @@ static struct virtio_net *translate_ring_addresses(struct virtio_net *dev, return NULL; } - vq->used = (struct vring_used *)(uintptr_t)qva_to_vva(dev, - addr->used_user_addr); + vq->used = (struct vring_used *)(uintptr_t)ring_addr_to_vva(dev, + vq, addr->used_user_addr, sizeof(struct vring_used)); if (vq->used == 0) { RTE_LOG(ERR, VHOST_CONFIG, "(%d) failed to find used ring address.\n",