From patchwork Tue Jun 6 08:18:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128166 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4BEB142C3E; Tue, 6 Jun 2023 10:19:08 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AEE4041144; Tue, 6 Jun 2023 10:19:05 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id C42A4410F6 for ; Tue, 6 Jun 2023 10:19:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039544; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kdk7A4dPJ3aSyRCNhnDx2xvj/ieo1OEJTR+LhLYwHNo=; b=fbfml4eAcIMw2RkvAT1GgW3fJXZUCgaVHCOuJbXXmoZzEjscq8pxLhaxgG/397D3RcVnoS qGSgKYANhXCCYOWFLpJK2YIfqxO50Ufnx1QhcRXcoJjN7rn7kqC+7A60wu5qUlwlFMAf2c IcRqG6KBvM79RkISz1McVxh6lyGAgHI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-223-X418fs4gNv2SXC-C3EtJPg-1; Tue, 06 Jun 2023 04:19:01 -0400 X-MC-Unique: X418fs4gNv2SXC-C3EtJPg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8587D1C06927; Tue, 6 Jun 2023 08:19:00 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id E095840B4CD6; Tue, 6 Jun 2023 08:18:57 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin , stable@dpdk.org Subject: [PATCH v5 01/26] vhost: fix IOTLB entries overlap check with previous entry Date: Tue, 6 Jun 2023 10:18:27 +0200 Message-Id: <20230606081852.71003-2-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Commit 22b6d0ac691a ("vhost: fix madvise IOTLB entries pages overlap check") fixed the check to ensure the entry to be removed does not overlap with the next one in the IOTLB cache before marking it as DONTDUMP with madvise(). This is not enough, because the same issue is present when comparing with the previous entry in the cache, where the end address of the previous entry should be used, not the start one. Fixes: dea092d0addb ("vhost: fix madvise arguments alignment") Cc: stable@dpdk.org Signed-off-by: Maxime Coquelin Acked-by: Mike Pattrick Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 3f45bc6061..870c8acb88 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -178,8 +178,8 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtque mask = ~(alignment - 1); /* Don't disable coredump if the previous node is in the same page */ - if (prev_node == NULL || - (node->uaddr & mask) != (prev_node->uaddr & mask)) { + if (prev_node == NULL || (node->uaddr & mask) != + ((prev_node->uaddr + prev_node->size - 1) & mask)) { next_node = RTE_TAILQ_NEXT(node, next); /* Don't disable coredump if the next node is in the same page */ if (next_node == NULL || ((node->uaddr + node->size - 1) & mask) != @@ -283,8 +283,8 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq mask = ~(alignment-1); /* Don't disable coredump if the previous node is in the same page */ - if (prev_node == NULL || - (node->uaddr & mask) != (prev_node->uaddr & mask)) { + if (prev_node == NULL || (node->uaddr & mask) != + ((prev_node->uaddr + prev_node->size - 1) & mask)) { next_node = RTE_TAILQ_NEXT(node, next); /* Don't disable coredump if the next node is in the same page */ if (next_node == NULL || ((node->uaddr + node->size - 1) & mask) != From patchwork Tue Jun 6 08:18:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128167 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3730A42C3E; Tue, 6 Jun 2023 10:19:14 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 04E4E42C76; Tue, 6 Jun 2023 10:19:07 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 6BD25410F6 for ; Tue, 6 Jun 2023 10:19:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039545; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nghe3pVt71XrFPUd8X5i89nN3A/M9CrhzX8c+jD0QKw=; b=cshSh48/9rTFdL/ESTOG66WE8hMURJdTe6p8lF8/enB4+c+jE1jXFRLhTSYCqFTfLxOQNg PlVAA6lbU0vVg6pW+05mqzrpmMnyLJ+Ax2sjE+NS1ESb4O/KaHe5KiDjkYPM/LlrtBsUsJ /COzTEcHk83iEGqNB3+YDj1TH0zSM3o= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-353-fOLKiAtpMqWUKBiOOa5XTA-1; Tue, 06 Jun 2023 04:19:03 -0400 X-MC-Unique: fOLKiAtpMqWUKBiOOa5XTA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 528E680120A; Tue, 6 Jun 2023 08:19:03 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id CBF6040B4CD6; Tue, 6 Jun 2023 08:19:00 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 02/26] vhost: add helper of IOTLB entries coredump Date: Tue, 6 Jun 2023 10:18:28 +0200 Message-Id: <20230606081852.71003-3-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch reworks IOTLB code to extract madvise-related bits into dedicated helper. This refactoring improves code sharing. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 77 +++++++++++++++++++++++++---------------------- 1 file changed, 41 insertions(+), 36 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 870c8acb88..51d45de446 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -23,6 +23,34 @@ struct vhost_iotlb_entry { #define IOTLB_CACHE_SIZE 2048 +static void +vhost_user_iotlb_set_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node) +{ + uint64_t align; + + align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); + + mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, true, align); +} + +static void +vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node, + struct vhost_iotlb_entry *prev, struct vhost_iotlb_entry *next) +{ + uint64_t align, mask; + + align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); + mask = ~(align - 1); + + /* Don't disable coredump if the previous node is in the same page */ + if (prev == NULL || (node->uaddr & mask) != ((prev->uaddr + prev->size - 1) & mask)) { + /* Don't disable coredump if the next node is in the same page */ + if (next == NULL || + ((node->uaddr + node->size - 1) & mask) != (next->uaddr & mask)) + mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, false, align); + } +} + static struct vhost_iotlb_entry * vhost_user_iotlb_pool_get(struct vhost_virtqueue *vq) { @@ -149,8 +177,8 @@ vhost_user_iotlb_cache_remove_all(struct virtio_net *dev, struct vhost_virtqueue rte_rwlock_write_lock(&vq->iotlb_lock); RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, false, - hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr)); + vhost_user_iotlb_clear_dump(dev, node, NULL, NULL); + TAILQ_REMOVE(&vq->iotlb_list, node, next); vhost_user_iotlb_pool_put(vq, node); } @@ -164,7 +192,6 @@ static void vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtqueue *vq) { struct vhost_iotlb_entry *node, *temp_node, *prev_node = NULL; - uint64_t alignment, mask; int entry_idx; rte_rwlock_write_lock(&vq->iotlb_lock); @@ -173,20 +200,10 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtque RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { if (!entry_idx) { - struct vhost_iotlb_entry *next_node; - alignment = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); - mask = ~(alignment - 1); - - /* Don't disable coredump if the previous node is in the same page */ - if (prev_node == NULL || (node->uaddr & mask) != - ((prev_node->uaddr + prev_node->size - 1) & mask)) { - next_node = RTE_TAILQ_NEXT(node, next); - /* Don't disable coredump if the next node is in the same page */ - if (next_node == NULL || ((node->uaddr + node->size - 1) & mask) != - (next_node->uaddr & mask)) - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, - false, alignment); - } + struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); + + vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); + TAILQ_REMOVE(&vq->iotlb_list, node, next); vhost_user_iotlb_pool_put(vq, node); vq->iotlb_cache_nr--; @@ -240,16 +257,16 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, struct vhost_virtqueue *vq vhost_user_iotlb_pool_put(vq, new_node); goto unlock; } else if (node->iova > new_node->iova) { - mem_set_dump((void *)(uintptr_t)new_node->uaddr, new_node->size, true, - hua_to_alignment(dev->mem, (void *)(uintptr_t)new_node->uaddr)); + vhost_user_iotlb_set_dump(dev, new_node); + TAILQ_INSERT_BEFORE(node, new_node, next); vq->iotlb_cache_nr++; goto unlock; } } - mem_set_dump((void *)(uintptr_t)new_node->uaddr, new_node->size, true, - hua_to_alignment(dev->mem, (void *)(uintptr_t)new_node->uaddr)); + vhost_user_iotlb_set_dump(dev, new_node); + TAILQ_INSERT_TAIL(&vq->iotlb_list, new_node, next); vq->iotlb_cache_nr++; @@ -265,7 +282,6 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq uint64_t iova, uint64_t size) { struct vhost_iotlb_entry *node, *temp_node, *prev_node = NULL; - uint64_t alignment, mask; if (unlikely(!size)) return; @@ -278,20 +294,9 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq break; if (iova < node->iova + node->size) { - struct vhost_iotlb_entry *next_node; - alignment = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); - mask = ~(alignment-1); - - /* Don't disable coredump if the previous node is in the same page */ - if (prev_node == NULL || (node->uaddr & mask) != - ((prev_node->uaddr + prev_node->size - 1) & mask)) { - next_node = RTE_TAILQ_NEXT(node, next); - /* Don't disable coredump if the next node is in the same page */ - if (next_node == NULL || ((node->uaddr + node->size - 1) & mask) != - (next_node->uaddr & mask)) - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, - false, alignment); - } + struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); + + vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); TAILQ_REMOVE(&vq->iotlb_list, node, next); vhost_user_iotlb_pool_put(vq, node); From patchwork Tue Jun 6 08:18:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128168 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1EA8B42C3E; Tue, 6 Jun 2023 10:19:23 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BF26342D36; Tue, 6 Jun 2023 10:19:09 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 69EA942D32 for ; Tue, 6 Jun 2023 10:19:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039547; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xtWU2swtOfVhpOdXEIx7YzgGQtuh2JboI/uCu+jZY8U=; b=gAEE/sthfDdeYVHGSj/f0AOVn8EtCLbE0eJjtzLSfvOTbkeLqN6dCLrPu65xpzZF2C6FZ1 p4VQFWPEr0SvwO2jHvpsXbqKZYL1AC7bmRiZa50FfnSlsEx2Ljm4A6ATyB6STGgBwT3N1d tt+ev23hUE//WqBjNSv1HhrXQWxEEqE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-76-mc4VQHDZM3e3kWh00YHMdQ-1; Tue, 06 Jun 2023 04:19:06 -0400 X-MC-Unique: mc4VQHDZM3e3kWh00YHMdQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 22D638030CD; Tue, 6 Jun 2023 08:19:06 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 986AF40B4CD6; Tue, 6 Jun 2023 08:19:03 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 03/26] vhost: add helper for IOTLB entries shared page check Date: Tue, 6 Jun 2023 10:18:29 +0200 Message-Id: <20230606081852.71003-4-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduces a helper to check whether two IOTLB entries share a page. Signed-off-by: Maxime Coquelin Acked-by: Mike Pattrick Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 51d45de446..4ef038adff 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -23,6 +23,23 @@ struct vhost_iotlb_entry { #define IOTLB_CACHE_SIZE 2048 +static bool +vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b, + uint64_t align) +{ + uint64_t a_end, b_start; + + if (a == NULL || b == NULL) + return false; + + /* Assumes entry a lower than entry b */ + RTE_ASSERT(a->uaddr < b->uaddr); + a_end = RTE_ALIGN_CEIL(a->uaddr + a->size, align); + b_start = RTE_ALIGN_FLOOR(b->uaddr, align); + + return a_end > b_start; +} + static void vhost_user_iotlb_set_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node) { @@ -37,16 +54,14 @@ static void vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node, struct vhost_iotlb_entry *prev, struct vhost_iotlb_entry *next) { - uint64_t align, mask; + uint64_t align; align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); - mask = ~(align - 1); /* Don't disable coredump if the previous node is in the same page */ - if (prev == NULL || (node->uaddr & mask) != ((prev->uaddr + prev->size - 1) & mask)) { + if (!vhost_user_iotlb_share_page(prev, node, align)) { /* Don't disable coredump if the next node is in the same page */ - if (next == NULL || - ((node->uaddr + node->size - 1) & mask) != (next->uaddr & mask)) + if (!vhost_user_iotlb_share_page(node, next, align)) mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, false, align); } } From patchwork Tue Jun 6 08:18:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128169 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2980342C3E; Tue, 6 Jun 2023 10:19:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2E52942D2C; Tue, 6 Jun 2023 10:19:15 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 7A14642D2C for ; Tue, 6 Jun 2023 10:19:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039553; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3viU0ucDifS7vbgVuPxlajnSbkTJ5nflHILVn2DSYNM=; b=gEvdNUPcm3EE5Luy+l98gwJJYizTDNMt/oP2CmNPtlzAALFeNx9s341fXAhwqEADIDztjh V8gEqmpxlcZXv0lmXUPqcaRfJatK+NwTmJmSt/b4JPvm3HDavRHGbVNMWEYRPEHYn449hG PKUmighl68nYHvWvvphmQZqAGDVCUko= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-583-89fWjfcBP0ylxMWeT9bFaQ-1; Tue, 06 Jun 2023 04:19:09 -0400 X-MC-Unique: 89fWjfcBP0ylxMWeT9bFaQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1561085A5BB; Tue, 6 Jun 2023 08:19:09 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 667A040CF8F6; Tue, 6 Jun 2023 08:19:06 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin , stable@dpdk.org Subject: [PATCH v5 04/26] vhost: don't dump unneeded pages with IOTLB Date: Tue, 6 Jun 2023 10:18:30 +0200 Message-Id: <20230606081852.71003-5-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On IOTLB entry removal, previous fixes took care of not marking pages shared with other IOTLB entries as DONTDUMP. However, if an IOTLB entry is spanned on multiple pages, the other pages were kept as DODUMP while they might not have been shared with other entries, increasing needlessly the coredump size. This patch addresses this issue by excluding only the shared pages from madvise's DONTDUMP. Fixes: dea092d0addb ("vhost: fix madvise arguments alignment") Cc: stable@dpdk.org Signed-off-by: Maxime Coquelin Acked-by: Mike Pattrick Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 4ef038adff..95d67ac832 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -54,16 +54,23 @@ static void vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node, struct vhost_iotlb_entry *prev, struct vhost_iotlb_entry *next) { - uint64_t align; + uint64_t align, start, end; + + start = node->uaddr; + end = node->uaddr + node->size; align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); - /* Don't disable coredump if the previous node is in the same page */ - if (!vhost_user_iotlb_share_page(prev, node, align)) { - /* Don't disable coredump if the next node is in the same page */ - if (!vhost_user_iotlb_share_page(node, next, align)) - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, false, align); - } + /* Skip first page if shared with previous entry. */ + if (vhost_user_iotlb_share_page(prev, node, align)) + start = RTE_ALIGN_CEIL(start, align); + + /* Skip last page if shared with next entry. */ + if (vhost_user_iotlb_share_page(node, next, align)) + end = RTE_ALIGN_FLOOR(end, align); + + if (end > start) + mem_set_dump((void *)(uintptr_t)start, end - start, false, align); } static struct vhost_iotlb_entry * From patchwork Tue Jun 6 08:18:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128170 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 41F7E42C3E; Tue, 6 Jun 2023 10:19:37 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 437FB42D3E; Tue, 6 Jun 2023 10:19:18 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 2F05342D3E for ; Tue, 6 Jun 2023 10:19:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039555; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=REctACCwjaix+IZinDTY/dmf8cyAct9mknHtWiNZYgU=; b=bVFUmi/OldXZpQUsv7XPmv9ErNmdrs4b47F+qB8pvqLqGGHYTBXz06X8rtObsRLYYVAhLf Mto0rA0PwGYHYsW4VwPnj+Ju6pHMg8yk720/oIWxmQ835SsrVN8rvNb1zkMfQVGllw/ftK aoEqXYmPNLwxPfWJ1Z4h3lFw08CkPOc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-149-6GbnGgr7NUCmBjD_Wd8hRA-1; Tue, 06 Jun 2023 04:19:12 -0400 X-MC-Unique: 6GbnGgr7NUCmBjD_Wd8hRA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0AE4E8032FF; Tue, 6 Jun 2023 08:19:12 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5B45540CF8F6; Tue, 6 Jun 2023 08:19:09 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 05/26] vhost: change to single IOTLB cache per device Date: Tue, 6 Jun 2023 10:18:31 +0200 Message-Id: <20230606081852.71003-6-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch simplifies IOTLB implementation and improves IOTLB memory consumption by having a single IOTLB cache per device, instead of having one per queue. In order to not impact performance, it keeps an IOTLB lock per virtqueue, so that there is no contention between multiple queue trying to acquire it. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 212 +++++++++++++++++++---------------------- lib/vhost/iotlb.h | 43 ++++++--- lib/vhost/vhost.c | 18 ++-- lib/vhost/vhost.h | 16 ++-- lib/vhost/vhost_user.c | 23 +++-- 5 files changed, 159 insertions(+), 153 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 95d67ac832..6d49bf6b30 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -74,86 +74,81 @@ vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *no } static struct vhost_iotlb_entry * -vhost_user_iotlb_pool_get(struct vhost_virtqueue *vq) +vhost_user_iotlb_pool_get(struct virtio_net *dev) { struct vhost_iotlb_entry *node; - rte_spinlock_lock(&vq->iotlb_free_lock); - node = SLIST_FIRST(&vq->iotlb_free_list); + rte_spinlock_lock(&dev->iotlb_free_lock); + node = SLIST_FIRST(&dev->iotlb_free_list); if (node != NULL) - SLIST_REMOVE_HEAD(&vq->iotlb_free_list, next_free); - rte_spinlock_unlock(&vq->iotlb_free_lock); + SLIST_REMOVE_HEAD(&dev->iotlb_free_list, next_free); + rte_spinlock_unlock(&dev->iotlb_free_lock); return node; } static void -vhost_user_iotlb_pool_put(struct vhost_virtqueue *vq, - struct vhost_iotlb_entry *node) +vhost_user_iotlb_pool_put(struct virtio_net *dev, struct vhost_iotlb_entry *node) { - rte_spinlock_lock(&vq->iotlb_free_lock); - SLIST_INSERT_HEAD(&vq->iotlb_free_list, node, next_free); - rte_spinlock_unlock(&vq->iotlb_free_lock); + rte_spinlock_lock(&dev->iotlb_free_lock); + SLIST_INSERT_HEAD(&dev->iotlb_free_list, node, next_free); + rte_spinlock_unlock(&dev->iotlb_free_lock); } static void -vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtqueue *vq); +vhost_user_iotlb_cache_random_evict(struct virtio_net *dev); static void -vhost_user_iotlb_pending_remove_all(struct vhost_virtqueue *vq) +vhost_user_iotlb_pending_remove_all(struct virtio_net *dev) { struct vhost_iotlb_entry *node, *temp_node; - rte_rwlock_write_lock(&vq->iotlb_pending_lock); + rte_rwlock_write_lock(&dev->iotlb_pending_lock); - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_pending_list, next, temp_node) { - TAILQ_REMOVE(&vq->iotlb_pending_list, node, next); - vhost_user_iotlb_pool_put(vq, node); + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_pending_list, next, temp_node) { + TAILQ_REMOVE(&dev->iotlb_pending_list, node, next); + vhost_user_iotlb_pool_put(dev, node); } - rte_rwlock_write_unlock(&vq->iotlb_pending_lock); + rte_rwlock_write_unlock(&dev->iotlb_pending_lock); } bool -vhost_user_iotlb_pending_miss(struct vhost_virtqueue *vq, uint64_t iova, - uint8_t perm) +vhost_user_iotlb_pending_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm) { struct vhost_iotlb_entry *node; bool found = false; - rte_rwlock_read_lock(&vq->iotlb_pending_lock); + rte_rwlock_read_lock(&dev->iotlb_pending_lock); - TAILQ_FOREACH(node, &vq->iotlb_pending_list, next) { + TAILQ_FOREACH(node, &dev->iotlb_pending_list, next) { if ((node->iova == iova) && (node->perm == perm)) { found = true; break; } } - rte_rwlock_read_unlock(&vq->iotlb_pending_lock); + rte_rwlock_read_unlock(&dev->iotlb_pending_lock); return found; } void -vhost_user_iotlb_pending_insert(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint8_t perm) +vhost_user_iotlb_pending_insert(struct virtio_net *dev, uint64_t iova, uint8_t perm) { struct vhost_iotlb_entry *node; - node = vhost_user_iotlb_pool_get(vq); + node = vhost_user_iotlb_pool_get(dev); if (node == NULL) { VHOST_LOG_CONFIG(dev->ifname, DEBUG, - "IOTLB pool for vq %"PRIu32" empty, clear entries for pending insertion\n", - vq->index); - if (!TAILQ_EMPTY(&vq->iotlb_pending_list)) - vhost_user_iotlb_pending_remove_all(vq); + "IOTLB pool empty, clear entries for pending insertion\n"); + if (!TAILQ_EMPTY(&dev->iotlb_pending_list)) + vhost_user_iotlb_pending_remove_all(dev); else - vhost_user_iotlb_cache_random_evict(dev, vq); - node = vhost_user_iotlb_pool_get(vq); + vhost_user_iotlb_cache_random_evict(dev); + node = vhost_user_iotlb_pool_get(dev); if (node == NULL) { VHOST_LOG_CONFIG(dev->ifname, ERR, - "IOTLB pool vq %"PRIu32" still empty, pending insertion failure\n", - vq->index); + "IOTLB pool still empty, pending insertion failure\n"); return; } } @@ -161,22 +156,21 @@ vhost_user_iotlb_pending_insert(struct virtio_net *dev, struct vhost_virtqueue * node->iova = iova; node->perm = perm; - rte_rwlock_write_lock(&vq->iotlb_pending_lock); + rte_rwlock_write_lock(&dev->iotlb_pending_lock); - TAILQ_INSERT_TAIL(&vq->iotlb_pending_list, node, next); + TAILQ_INSERT_TAIL(&dev->iotlb_pending_list, node, next); - rte_rwlock_write_unlock(&vq->iotlb_pending_lock); + rte_rwlock_write_unlock(&dev->iotlb_pending_lock); } void -vhost_user_iotlb_pending_remove(struct vhost_virtqueue *vq, - uint64_t iova, uint64_t size, uint8_t perm) +vhost_user_iotlb_pending_remove(struct virtio_net *dev, uint64_t iova, uint64_t size, uint8_t perm) { struct vhost_iotlb_entry *node, *temp_node; - rte_rwlock_write_lock(&vq->iotlb_pending_lock); + rte_rwlock_write_lock(&dev->iotlb_pending_lock); - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_pending_list, next, + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_pending_list, next, temp_node) { if (node->iova < iova) continue; @@ -184,81 +178,78 @@ vhost_user_iotlb_pending_remove(struct vhost_virtqueue *vq, continue; if ((node->perm & perm) != node->perm) continue; - TAILQ_REMOVE(&vq->iotlb_pending_list, node, next); - vhost_user_iotlb_pool_put(vq, node); + TAILQ_REMOVE(&dev->iotlb_pending_list, node, next); + vhost_user_iotlb_pool_put(dev, node); } - rte_rwlock_write_unlock(&vq->iotlb_pending_lock); + rte_rwlock_write_unlock(&dev->iotlb_pending_lock); } static void -vhost_user_iotlb_cache_remove_all(struct virtio_net *dev, struct vhost_virtqueue *vq) +vhost_user_iotlb_cache_remove_all(struct virtio_net *dev) { struct vhost_iotlb_entry *node, *temp_node; - rte_rwlock_write_lock(&vq->iotlb_lock); + vhost_user_iotlb_wr_lock_all(dev); - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_list, next, temp_node) { vhost_user_iotlb_clear_dump(dev, node, NULL, NULL); - TAILQ_REMOVE(&vq->iotlb_list, node, next); - vhost_user_iotlb_pool_put(vq, node); + TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_pool_put(dev, node); } - vq->iotlb_cache_nr = 0; + dev->iotlb_cache_nr = 0; - rte_rwlock_write_unlock(&vq->iotlb_lock); + vhost_user_iotlb_wr_unlock_all(dev); } static void -vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtqueue *vq) +vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) { struct vhost_iotlb_entry *node, *temp_node, *prev_node = NULL; int entry_idx; - rte_rwlock_write_lock(&vq->iotlb_lock); + vhost_user_iotlb_wr_lock_all(dev); - entry_idx = rte_rand() % vq->iotlb_cache_nr; + entry_idx = rte_rand() % dev->iotlb_cache_nr; - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_list, next, temp_node) { if (!entry_idx) { struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); - TAILQ_REMOVE(&vq->iotlb_list, node, next); - vhost_user_iotlb_pool_put(vq, node); - vq->iotlb_cache_nr--; + TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_pool_put(dev, node); + dev->iotlb_cache_nr--; break; } prev_node = node; entry_idx--; } - rte_rwlock_write_unlock(&vq->iotlb_lock); + vhost_user_iotlb_wr_unlock_all(dev); } void -vhost_user_iotlb_cache_insert(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint64_t uaddr, +vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, uint64_t size, uint8_t perm) { struct vhost_iotlb_entry *node, *new_node; - new_node = vhost_user_iotlb_pool_get(vq); + new_node = vhost_user_iotlb_pool_get(dev); if (new_node == NULL) { VHOST_LOG_CONFIG(dev->ifname, DEBUG, - "IOTLB pool vq %"PRIu32" empty, clear entries for cache insertion\n", - vq->index); - if (!TAILQ_EMPTY(&vq->iotlb_list)) - vhost_user_iotlb_cache_random_evict(dev, vq); + "IOTLB pool empty, clear entries for cache insertion\n"); + if (!TAILQ_EMPTY(&dev->iotlb_list)) + vhost_user_iotlb_cache_random_evict(dev); else - vhost_user_iotlb_pending_remove_all(vq); - new_node = vhost_user_iotlb_pool_get(vq); + vhost_user_iotlb_pending_remove_all(dev); + new_node = vhost_user_iotlb_pool_get(dev); if (new_node == NULL) { VHOST_LOG_CONFIG(dev->ifname, ERR, - "IOTLB pool vq %"PRIu32" still empty, cache insertion failed\n", - vq->index); + "IOTLB pool still empty, cache insertion failed\n"); return; } } @@ -268,49 +259,47 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, struct vhost_virtqueue *vq new_node->size = size; new_node->perm = perm; - rte_rwlock_write_lock(&vq->iotlb_lock); + vhost_user_iotlb_wr_lock_all(dev); - TAILQ_FOREACH(node, &vq->iotlb_list, next) { + TAILQ_FOREACH(node, &dev->iotlb_list, next) { /* * Entries must be invalidated before being updated. * So if iova already in list, assume identical. */ if (node->iova == new_node->iova) { - vhost_user_iotlb_pool_put(vq, new_node); + vhost_user_iotlb_pool_put(dev, new_node); goto unlock; } else if (node->iova > new_node->iova) { vhost_user_iotlb_set_dump(dev, new_node); TAILQ_INSERT_BEFORE(node, new_node, next); - vq->iotlb_cache_nr++; + dev->iotlb_cache_nr++; goto unlock; } } vhost_user_iotlb_set_dump(dev, new_node); - TAILQ_INSERT_TAIL(&vq->iotlb_list, new_node, next); - vq->iotlb_cache_nr++; + TAILQ_INSERT_TAIL(&dev->iotlb_list, new_node, next); + dev->iotlb_cache_nr++; unlock: - vhost_user_iotlb_pending_remove(vq, iova, size, perm); - - rte_rwlock_write_unlock(&vq->iotlb_lock); + vhost_user_iotlb_pending_remove(dev, iova, size, perm); + vhost_user_iotlb_wr_unlock_all(dev); } void -vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint64_t size) +vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t size) { struct vhost_iotlb_entry *node, *temp_node, *prev_node = NULL; if (unlikely(!size)) return; - rte_rwlock_write_lock(&vq->iotlb_lock); + vhost_user_iotlb_wr_lock_all(dev); - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_list, next, temp_node) { /* Sorted list */ if (unlikely(iova + size < node->iova)) break; @@ -320,19 +309,19 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); - TAILQ_REMOVE(&vq->iotlb_list, node, next); - vhost_user_iotlb_pool_put(vq, node); - vq->iotlb_cache_nr--; - } else + TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_pool_put(dev, node); + dev->iotlb_cache_nr--; + } else { prev_node = node; + } } - rte_rwlock_write_unlock(&vq->iotlb_lock); + vhost_user_iotlb_wr_unlock_all(dev); } uint64_t -vhost_user_iotlb_cache_find(struct vhost_virtqueue *vq, uint64_t iova, - uint64_t *size, uint8_t perm) +vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *size, uint8_t perm) { struct vhost_iotlb_entry *node; uint64_t offset, vva = 0, mapped = 0; @@ -340,7 +329,7 @@ vhost_user_iotlb_cache_find(struct vhost_virtqueue *vq, uint64_t iova, if (unlikely(!*size)) goto out; - TAILQ_FOREACH(node, &vq->iotlb_list, next) { + TAILQ_FOREACH(node, &dev->iotlb_list, next) { /* List sorted by iova */ if (unlikely(iova < node->iova)) break; @@ -373,60 +362,57 @@ vhost_user_iotlb_cache_find(struct vhost_virtqueue *vq, uint64_t iova, } void -vhost_user_iotlb_flush_all(struct virtio_net *dev, struct vhost_virtqueue *vq) +vhost_user_iotlb_flush_all(struct virtio_net *dev) { - vhost_user_iotlb_cache_remove_all(dev, vq); - vhost_user_iotlb_pending_remove_all(vq); + vhost_user_iotlb_cache_remove_all(dev); + vhost_user_iotlb_pending_remove_all(dev); } int -vhost_user_iotlb_init(struct virtio_net *dev, struct vhost_virtqueue *vq) +vhost_user_iotlb_init(struct virtio_net *dev) { unsigned int i; int socket = 0; - if (vq->iotlb_pool) { + if (dev->iotlb_pool) { /* * The cache has already been initialized, * just drop all cached and pending entries. */ - vhost_user_iotlb_flush_all(dev, vq); - rte_free(vq->iotlb_pool); + vhost_user_iotlb_flush_all(dev); + rte_free(dev->iotlb_pool); } #ifdef RTE_LIBRTE_VHOST_NUMA - if (get_mempolicy(&socket, NULL, 0, vq, MPOL_F_NODE | MPOL_F_ADDR) != 0) + if (get_mempolicy(&socket, NULL, 0, dev, MPOL_F_NODE | MPOL_F_ADDR) != 0) socket = 0; #endif - rte_spinlock_init(&vq->iotlb_free_lock); - rte_rwlock_init(&vq->iotlb_lock); - rte_rwlock_init(&vq->iotlb_pending_lock); + rte_spinlock_init(&dev->iotlb_free_lock); + rte_rwlock_init(&dev->iotlb_pending_lock); - SLIST_INIT(&vq->iotlb_free_list); - TAILQ_INIT(&vq->iotlb_list); - TAILQ_INIT(&vq->iotlb_pending_list); + SLIST_INIT(&dev->iotlb_free_list); + TAILQ_INIT(&dev->iotlb_list); + TAILQ_INIT(&dev->iotlb_pending_list); if (dev->flags & VIRTIO_DEV_SUPPORT_IOMMU) { - vq->iotlb_pool = rte_calloc_socket("iotlb", IOTLB_CACHE_SIZE, + dev->iotlb_pool = rte_calloc_socket("iotlb", IOTLB_CACHE_SIZE, sizeof(struct vhost_iotlb_entry), 0, socket); - if (!vq->iotlb_pool) { - VHOST_LOG_CONFIG(dev->ifname, ERR, - "Failed to create IOTLB cache pool for vq %"PRIu32"\n", - vq->index); + if (!dev->iotlb_pool) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to create IOTLB cache pool\n"); return -1; } for (i = 0; i < IOTLB_CACHE_SIZE; i++) - vhost_user_iotlb_pool_put(vq, &vq->iotlb_pool[i]); + vhost_user_iotlb_pool_put(dev, &dev->iotlb_pool[i]); } - vq->iotlb_cache_nr = 0; + dev->iotlb_cache_nr = 0; return 0; } void -vhost_user_iotlb_destroy(struct vhost_virtqueue *vq) +vhost_user_iotlb_destroy(struct virtio_net *dev) { - rte_free(vq->iotlb_pool); + rte_free(dev->iotlb_pool); } diff --git a/lib/vhost/iotlb.h b/lib/vhost/iotlb.h index 73b5465b41..3490b9e6be 100644 --- a/lib/vhost/iotlb.h +++ b/lib/vhost/iotlb.h @@ -37,20 +37,37 @@ vhost_user_iotlb_wr_unlock(struct vhost_virtqueue *vq) rte_rwlock_write_unlock(&vq->iotlb_lock); } -void vhost_user_iotlb_cache_insert(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint64_t uaddr, +static __rte_always_inline void +vhost_user_iotlb_wr_lock_all(struct virtio_net *dev) + __rte_no_thread_safety_analysis +{ + uint32_t i; + + for (i = 0; i < dev->nr_vring; i++) + rte_rwlock_write_lock(&dev->virtqueue[i]->iotlb_lock); +} + +static __rte_always_inline void +vhost_user_iotlb_wr_unlock_all(struct virtio_net *dev) + __rte_no_thread_safety_analysis +{ + uint32_t i; + + for (i = 0; i < dev->nr_vring; i++) + rte_rwlock_write_unlock(&dev->virtqueue[i]->iotlb_lock); +} + +void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, uint64_t size, uint8_t perm); -void vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint64_t size); -uint64_t vhost_user_iotlb_cache_find(struct vhost_virtqueue *vq, uint64_t iova, +void vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t size); +uint64_t vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *size, uint8_t perm); -bool vhost_user_iotlb_pending_miss(struct vhost_virtqueue *vq, uint64_t iova, - uint8_t perm); -void vhost_user_iotlb_pending_insert(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint8_t perm); -void vhost_user_iotlb_pending_remove(struct vhost_virtqueue *vq, uint64_t iova, +bool vhost_user_iotlb_pending_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm); +void vhost_user_iotlb_pending_insert(struct virtio_net *dev, uint64_t iova, uint8_t perm); +void vhost_user_iotlb_pending_remove(struct virtio_net *dev, uint64_t iova, uint64_t size, uint8_t perm); -void vhost_user_iotlb_flush_all(struct virtio_net *dev, struct vhost_virtqueue *vq); -int vhost_user_iotlb_init(struct virtio_net *dev, struct vhost_virtqueue *vq); -void vhost_user_iotlb_destroy(struct vhost_virtqueue *vq); +void vhost_user_iotlb_flush_all(struct virtio_net *dev); +int vhost_user_iotlb_init(struct virtio_net *dev); +void vhost_user_iotlb_destroy(struct virtio_net *dev); + #endif /* _VHOST_IOTLB_H_ */ diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index 79e88f986e..3ddd2a963f 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -67,7 +67,7 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, tmp_size = *size; - vva = vhost_user_iotlb_cache_find(vq, iova, &tmp_size, perm); + vva = vhost_user_iotlb_cache_find(dev, iova, &tmp_size, perm); if (tmp_size == *size) { if (dev->flags & VIRTIO_DEV_STATS_ENABLED) vq->stats.iotlb_hits++; @@ -79,7 +79,7 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, iova += tmp_size; - if (!vhost_user_iotlb_pending_miss(vq, iova, perm)) { + if (!vhost_user_iotlb_pending_miss(dev, iova, perm)) { /* * iotlb_lock is read-locked for a full burst, * but it only protects the iotlb cache. @@ -89,12 +89,12 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, */ vhost_user_iotlb_rd_unlock(vq); - vhost_user_iotlb_pending_insert(dev, vq, iova, perm); + vhost_user_iotlb_pending_insert(dev, iova, perm); if (vhost_user_iotlb_miss(dev, iova, perm)) { VHOST_LOG_DATA(dev->ifname, ERR, "IOTLB miss req failed for IOVA 0x%" PRIx64 "\n", iova); - vhost_user_iotlb_pending_remove(vq, iova, 1, perm); + vhost_user_iotlb_pending_remove(dev, iova, 1, perm); } vhost_user_iotlb_rd_lock(vq); @@ -401,7 +401,6 @@ free_vq(struct virtio_net *dev, struct vhost_virtqueue *vq) vhost_free_async_mem(vq); rte_rwlock_write_unlock(&vq->access_lock); rte_free(vq->batch_copy_elems); - vhost_user_iotlb_destroy(vq); rte_free(vq->log_cache); rte_free(vq); } @@ -579,7 +578,7 @@ vring_invalidate(struct virtio_net *dev __rte_unused, struct vhost_virtqueue *vq } static void -init_vring_queue(struct virtio_net *dev, struct vhost_virtqueue *vq, +init_vring_queue(struct virtio_net *dev __rte_unused, struct vhost_virtqueue *vq, uint32_t vring_idx) { int numa_node = SOCKET_ID_ANY; @@ -599,8 +598,6 @@ init_vring_queue(struct virtio_net *dev, struct vhost_virtqueue *vq, } #endif vq->numa_node = numa_node; - - vhost_user_iotlb_init(dev, vq); } static void @@ -635,6 +632,7 @@ alloc_vring_queue(struct virtio_net *dev, uint32_t vring_idx) dev->virtqueue[i] = vq; init_vring_queue(dev, vq, i); rte_rwlock_init(&vq->access_lock); + rte_rwlock_init(&vq->iotlb_lock); vq->avail_wrap_counter = 1; vq->used_wrap_counter = 1; vq->signalled_used_valid = false; @@ -799,6 +797,10 @@ vhost_setup_virtio_net(int vid, bool enable, bool compliant_ol_flags, bool stats dev->flags |= VIRTIO_DEV_SUPPORT_IOMMU; else dev->flags &= ~VIRTIO_DEV_SUPPORT_IOMMU; + + if (vhost_user_iotlb_init(dev) < 0) + VHOST_LOG_CONFIG("device", ERR, "failed to init IOTLB\n"); + } void diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index eaf3b0d392..ee952de175 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -305,13 +305,6 @@ struct vhost_virtqueue { struct log_cache_entry *log_cache; rte_rwlock_t iotlb_lock; - rte_rwlock_t iotlb_pending_lock; - struct vhost_iotlb_entry *iotlb_pool; - TAILQ_HEAD(, vhost_iotlb_entry) iotlb_list; - TAILQ_HEAD(, vhost_iotlb_entry) iotlb_pending_list; - int iotlb_cache_nr; - rte_spinlock_t iotlb_free_lock; - SLIST_HEAD(, vhost_iotlb_entry) iotlb_free_list; /* Used to notify the guest (trigger interrupt) */ int callfd; @@ -486,6 +479,15 @@ struct virtio_net { int extbuf; int linearbuf; struct vhost_virtqueue *virtqueue[VHOST_MAX_QUEUE_PAIRS * 2]; + + rte_rwlock_t iotlb_pending_lock; + struct vhost_iotlb_entry *iotlb_pool; + TAILQ_HEAD(, vhost_iotlb_entry) iotlb_list; + TAILQ_HEAD(, vhost_iotlb_entry) iotlb_pending_list; + int iotlb_cache_nr; + rte_spinlock_t iotlb_free_lock; + SLIST_HEAD(, vhost_iotlb_entry) iotlb_free_list; + struct inflight_mem_info *inflight_info; #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ) char ifname[IF_NAME_SZ]; diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index c9454ce3d9..f2fe7ebc93 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -237,6 +237,8 @@ vhost_backend_cleanup(struct virtio_net *dev) } dev->postcopy_listening = 0; + + vhost_user_iotlb_destroy(dev); } static void @@ -539,7 +541,6 @@ numa_realloc(struct virtio_net **pdev, struct vhost_virtqueue **pvq) if (vq != dev->virtqueue[vq->index]) { VHOST_LOG_CONFIG(dev->ifname, INFO, "reallocated virtqueue on node %d\n", node); dev->virtqueue[vq->index] = vq; - vhost_user_iotlb_init(dev, vq); } if (vq_is_packed(dev)) { @@ -664,6 +665,8 @@ numa_realloc(struct virtio_net **pdev, struct vhost_virtqueue **pvq) return; } dev->guest_pages = gp; + + vhost_user_iotlb_init(dev); } #else static void @@ -1360,8 +1363,7 @@ vhost_user_set_mem_table(struct virtio_net **pdev, /* Flush IOTLB cache as previous HVAs are now invalid */ if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM)) - for (i = 0; i < dev->nr_vring; i++) - vhost_user_iotlb_flush_all(dev, dev->virtqueue[i]); + vhost_user_iotlb_flush_all(dev); free_mem_region(dev); rte_free(dev->mem); @@ -2194,7 +2196,7 @@ vhost_user_get_vring_base(struct virtio_net **pdev, ctx->msg.size = sizeof(ctx->msg.payload.state); ctx->fd_num = 0; - vhost_user_iotlb_flush_all(dev, vq); + vhost_user_iotlb_flush_all(dev); vring_invalidate(dev, vq); @@ -2639,15 +2641,14 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, if (!vva) return RTE_VHOST_MSG_RESULT_ERR; + vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, len, imsg->perm); + for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; if (!vq) continue; - vhost_user_iotlb_cache_insert(dev, vq, imsg->iova, vva, - len, imsg->perm); - if (is_vring_iotlb(dev, vq, imsg)) { rte_rwlock_write_lock(&vq->access_lock); translate_ring_addresses(&dev, &vq); @@ -2657,15 +2658,14 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, } break; case VHOST_IOTLB_INVALIDATE: + vhost_user_iotlb_cache_remove(dev, imsg->iova, imsg->size); + for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; if (!vq) continue; - vhost_user_iotlb_cache_remove(dev, vq, imsg->iova, - imsg->size); - if (is_vring_iotlb(dev, vq, imsg)) { rte_rwlock_write_lock(&vq->access_lock); vring_invalidate(dev, vq); @@ -2674,8 +2674,7 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, } break; default: - VHOST_LOG_CONFIG(dev->ifname, ERR, - "invalid IOTLB message type (%d)\n", + VHOST_LOG_CONFIG(dev->ifname, ERR, "invalid IOTLB message type (%d)\n", imsg->type); return RTE_VHOST_MSG_RESULT_ERR; } From patchwork Tue Jun 6 08:18:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128171 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D225742C3E; Tue, 6 Jun 2023 10:19:45 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B5BA442D35; Tue, 6 Jun 2023 10:19:20 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 2909D40697 for ; Tue, 6 Jun 2023 10:19:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pRtT2U6/p62Goq2GE+Yawodl2EdFsJLJutV0Bqwp8lg=; b=UI4rV3z0/OD8jIpCi5LbCDirCMqKU13rfNqry6TD7LqvQY4HLozAwnfOKn5tHHjPbb1zTO 3+ig8lafFVxxCrFPVRSIkXH5ODrS+pi4guqAVEuorb1cQ/ccDxqGfo18aTKqftq9KAhXbb +jNr6sOyo15bhYxPD0I4JDoE+k/4IyY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-70-A5w1xqoqMqGJ0jr36D-spQ-1; Tue, 06 Jun 2023 04:19:15 -0400 X-MC-Unique: A5w1xqoqMqGJ0jr36D-spQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1364485A5BE; Tue, 6 Jun 2023 08:19:15 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5AAAC40CF8F6; Tue, 6 Jun 2023 08:19:12 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 06/26] vhost: add offset field to IOTLB entries Date: Tue, 6 Jun 2023 10:18:32 +0200 Message-Id: <20230606081852.71003-7-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch is a preliminary work to prepare for VDUSE support, for which we need to keep track of the mmaped base address and offset in order to be able to unmap it later when IOTLB entry is invalidated. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 30 ++++++++++++++++++------------ lib/vhost/iotlb.h | 2 +- lib/vhost/vhost_user.c | 2 +- 3 files changed, 20 insertions(+), 14 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 6d49bf6b30..aa5100e6e7 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -17,6 +17,7 @@ struct vhost_iotlb_entry { uint64_t iova; uint64_t uaddr; + uint64_t uoffset; uint64_t size; uint8_t perm; }; @@ -27,15 +28,18 @@ static bool vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b, uint64_t align) { - uint64_t a_end, b_start; + uint64_t a_start, a_end, b_start; if (a == NULL || b == NULL) return false; + a_start = a->uaddr + a->uoffset; + b_start = b->uaddr + b->uoffset; + /* Assumes entry a lower than entry b */ - RTE_ASSERT(a->uaddr < b->uaddr); - a_end = RTE_ALIGN_CEIL(a->uaddr + a->size, align); - b_start = RTE_ALIGN_FLOOR(b->uaddr, align); + RTE_ASSERT(a_start < b_start); + a_end = RTE_ALIGN_CEIL(a_start + a->size, align); + b_start = RTE_ALIGN_FLOOR(b_start, align); return a_end > b_start; } @@ -43,11 +47,12 @@ vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entr static void vhost_user_iotlb_set_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node) { - uint64_t align; + uint64_t align, start; - align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); + start = node->uaddr + node->uoffset; + align = hua_to_alignment(dev->mem, (void *)(uintptr_t)start); - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, true, align); + mem_set_dump((void *)(uintptr_t)start, node->size, true, align); } static void @@ -56,10 +61,10 @@ vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *no { uint64_t align, start, end; - start = node->uaddr; - end = node->uaddr + node->size; + start = node->uaddr + node->uoffset; + end = start + node->size; - align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); + align = hua_to_alignment(dev->mem, (void *)(uintptr_t)start); /* Skip first page if shared with previous entry. */ if (vhost_user_iotlb_share_page(prev, node, align)) @@ -234,7 +239,7 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, - uint64_t size, uint8_t perm) + uint64_t uoffset, uint64_t size, uint8_t perm) { struct vhost_iotlb_entry *node, *new_node; @@ -256,6 +261,7 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t ua new_node->iova = iova; new_node->uaddr = uaddr; + new_node->uoffset = uoffset; new_node->size = size; new_node->perm = perm; @@ -344,7 +350,7 @@ vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *siz offset = iova - node->iova; if (!vva) - vva = node->uaddr + offset; + vva = node->uaddr + node->uoffset + offset; mapped += node->size - offset; iova = node->iova + node->size; diff --git a/lib/vhost/iotlb.h b/lib/vhost/iotlb.h index 3490b9e6be..bee36c5903 100644 --- a/lib/vhost/iotlb.h +++ b/lib/vhost/iotlb.h @@ -58,7 +58,7 @@ vhost_user_iotlb_wr_unlock_all(struct virtio_net *dev) } void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, - uint64_t size, uint8_t perm); + uint64_t uoffset, uint64_t size, uint8_t perm); void vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t size); uint64_t vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *size, uint8_t perm); diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index f2fe7ebc93..7f88a8754f 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -2641,7 +2641,7 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, if (!vva) return RTE_VHOST_MSG_RESULT_ERR; - vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, len, imsg->perm); + vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, 0, len, imsg->perm); for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; From patchwork Tue Jun 6 08:18:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128172 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8191842C3E; Tue, 6 Jun 2023 10:19:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DD56542D50; Tue, 6 Jun 2023 10:19:22 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id D54AE42D50 for ; Tue, 6 Jun 2023 10:19:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sGEJh1nMhfTRX7+OJKRQZyCAdFbDueQwqgvaVGPX80o=; b=Zd6AQVtiF9ZMmxCZmxteFv4iJUB6gzjjzC/N0PNz1Q0kr5QiovBDqAH7cLEP5xK/urlBTx uiTopZk372zXwN4lH4M11OrYoqSN7QCYDtsk7BwdTu9bR9C7IdDF7rF27xIPjL2U+58Erq IcAwP3fat7+UxlJEu0R3nabeoyZdlmc= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-538-h-sPwbsQNVGupfzEinhmwQ-1; Tue, 06 Jun 2023 04:19:18 -0400 X-MC-Unique: h-sPwbsQNVGupfzEinhmwQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D2A051C06901; Tue, 6 Jun 2023 08:19:17 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 589AB40CF8F6; Tue, 6 Jun 2023 08:19:15 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 07/26] vhost: add page size info to IOTLB entry Date: Tue, 6 Jun 2023 10:18:33 +0200 Message-Id: <20230606081852.71003-8-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org VDUSE will close the file descriptor after having mapped the shared memory, so it will not be possible to get the page size afterwards. This patch adds an new page_shift field to the IOTLB entry, so that the information will be passed at IOTLB cache insertion time. The information is stored as a bit shift value so that IOTLB entry keeps fitting in a single cacheline. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 46 ++++++++++++++++++++---------------------- lib/vhost/iotlb.h | 2 +- lib/vhost/vhost.h | 1 - lib/vhost/vhost_user.c | 8 +++++--- 4 files changed, 28 insertions(+), 29 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index aa5100e6e7..87986f2489 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -19,14 +19,14 @@ struct vhost_iotlb_entry { uint64_t uaddr; uint64_t uoffset; uint64_t size; + uint8_t page_shift; uint8_t perm; }; #define IOTLB_CACHE_SIZE 2048 static bool -vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b, - uint64_t align) +vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b) { uint64_t a_start, a_end, b_start; @@ -38,44 +38,41 @@ vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entr /* Assumes entry a lower than entry b */ RTE_ASSERT(a_start < b_start); - a_end = RTE_ALIGN_CEIL(a_start + a->size, align); - b_start = RTE_ALIGN_FLOOR(b_start, align); + a_end = RTE_ALIGN_CEIL(a_start + a->size, RTE_BIT64(a->page_shift)); + b_start = RTE_ALIGN_FLOOR(b_start, RTE_BIT64(b->page_shift)); return a_end > b_start; } static void -vhost_user_iotlb_set_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node) +vhost_user_iotlb_set_dump(struct vhost_iotlb_entry *node) { - uint64_t align, start; + uint64_t start; start = node->uaddr + node->uoffset; - align = hua_to_alignment(dev->mem, (void *)(uintptr_t)start); - - mem_set_dump((void *)(uintptr_t)start, node->size, true, align); + mem_set_dump((void *)(uintptr_t)start, node->size, true, RTE_BIT64(node->page_shift)); } static void -vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node, +vhost_user_iotlb_clear_dump(struct vhost_iotlb_entry *node, struct vhost_iotlb_entry *prev, struct vhost_iotlb_entry *next) { - uint64_t align, start, end; + uint64_t start, end; start = node->uaddr + node->uoffset; end = start + node->size; - align = hua_to_alignment(dev->mem, (void *)(uintptr_t)start); - /* Skip first page if shared with previous entry. */ - if (vhost_user_iotlb_share_page(prev, node, align)) - start = RTE_ALIGN_CEIL(start, align); + if (vhost_user_iotlb_share_page(prev, node)) + start = RTE_ALIGN_CEIL(start, RTE_BIT64(node->page_shift)); /* Skip last page if shared with next entry. */ - if (vhost_user_iotlb_share_page(node, next, align)) - end = RTE_ALIGN_FLOOR(end, align); + if (vhost_user_iotlb_share_page(node, next)) + end = RTE_ALIGN_FLOOR(end, RTE_BIT64(node->page_shift)); if (end > start) - mem_set_dump((void *)(uintptr_t)start, end - start, false, align); + mem_set_dump((void *)(uintptr_t)start, end - start, false, + RTE_BIT64(node->page_shift)); } static struct vhost_iotlb_entry * @@ -198,7 +195,7 @@ vhost_user_iotlb_cache_remove_all(struct virtio_net *dev) vhost_user_iotlb_wr_lock_all(dev); RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_list, next, temp_node) { - vhost_user_iotlb_clear_dump(dev, node, NULL, NULL); + vhost_user_iotlb_clear_dump(node, NULL, NULL); TAILQ_REMOVE(&dev->iotlb_list, node, next); vhost_user_iotlb_pool_put(dev, node); @@ -223,7 +220,7 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) if (!entry_idx) { struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); - vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); + vhost_user_iotlb_clear_dump(node, prev_node, next_node); TAILQ_REMOVE(&dev->iotlb_list, node, next); vhost_user_iotlb_pool_put(dev, node); @@ -239,7 +236,7 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, - uint64_t uoffset, uint64_t size, uint8_t perm) + uint64_t uoffset, uint64_t size, uint64_t page_size, uint8_t perm) { struct vhost_iotlb_entry *node, *new_node; @@ -263,6 +260,7 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t ua new_node->uaddr = uaddr; new_node->uoffset = uoffset; new_node->size = size; + new_node->page_shift = __builtin_ctzll(page_size); new_node->perm = perm; vhost_user_iotlb_wr_lock_all(dev); @@ -276,7 +274,7 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t ua vhost_user_iotlb_pool_put(dev, new_node); goto unlock; } else if (node->iova > new_node->iova) { - vhost_user_iotlb_set_dump(dev, new_node); + vhost_user_iotlb_set_dump(new_node); TAILQ_INSERT_BEFORE(node, new_node, next); dev->iotlb_cache_nr++; @@ -284,7 +282,7 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t ua } } - vhost_user_iotlb_set_dump(dev, new_node); + vhost_user_iotlb_set_dump(new_node); TAILQ_INSERT_TAIL(&dev->iotlb_list, new_node, next); dev->iotlb_cache_nr++; @@ -313,7 +311,7 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t si if (iova < node->iova + node->size) { struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); - vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); + vhost_user_iotlb_clear_dump(node, prev_node, next_node); TAILQ_REMOVE(&dev->iotlb_list, node, next); vhost_user_iotlb_pool_put(dev, node); diff --git a/lib/vhost/iotlb.h b/lib/vhost/iotlb.h index bee36c5903..81ca04df21 100644 --- a/lib/vhost/iotlb.h +++ b/lib/vhost/iotlb.h @@ -58,7 +58,7 @@ vhost_user_iotlb_wr_unlock_all(struct virtio_net *dev) } void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, - uint64_t uoffset, uint64_t size, uint8_t perm); + uint64_t uoffset, uint64_t size, uint64_t page_size, uint8_t perm); void vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t size); uint64_t vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *size, uint8_t perm); diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index ee952de175..de84c115b7 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -1032,7 +1032,6 @@ mbuf_is_consumed(struct rte_mbuf *m) return true; } -uint64_t hua_to_alignment(struct rte_vhost_memory *mem, void *ptr); void mem_set_dump(void *ptr, size_t size, bool enable, uint64_t alignment); /* Versioned functions */ diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 7f88a8754f..98d8b8ac79 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -743,7 +743,7 @@ log_addr_to_gpa(struct virtio_net *dev, struct vhost_virtqueue *vq) return log_gpa; } -uint64_t +static uint64_t hua_to_alignment(struct rte_vhost_memory *mem, void *ptr) { struct rte_vhost_mem_region *r; @@ -2632,7 +2632,7 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, struct virtio_net *dev = *pdev; struct vhost_iotlb_msg *imsg = &ctx->msg.payload.iotlb; uint16_t i; - uint64_t vva, len; + uint64_t vva, len, pg_sz; switch (imsg->type) { case VHOST_IOTLB_UPDATE: @@ -2641,7 +2641,9 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, if (!vva) return RTE_VHOST_MSG_RESULT_ERR; - vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, 0, len, imsg->perm); + pg_sz = hua_to_alignment(dev->mem, (void *)(uintptr_t)vva); + + vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, 0, len, pg_sz, imsg->perm); for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; From patchwork Tue Jun 6 08:18:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128173 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4590B42C3E; Tue, 6 Jun 2023 10:19:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2CB0042D59; Tue, 6 Jun 2023 10:19:25 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 6BA1342D56 for ; Tue, 6 Jun 2023 10:19:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YVqmLHMAAAMHCW9ouzdoXrU9qsZSPl5hdtEviSt13uc=; b=L3LwiC0EUH5SdvITV0TWnz2dP7ETB82uz4PwyvdbRVkJijEUrk5SqAsk3ZydOPwZXOBIxN 15NEh/2sY8THd5d357B4dkFiWULU3yBUIgZ2UROjuwyP/UeTfVFX1kfK0xcH7UdSXZCR2P J25lSea4/4dCzpCsnT7VKjP3VP3bl3c= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-639-NWszhd2ePECIrNEaetaAKQ-1; Tue, 06 Jun 2023 04:19:21 -0400 X-MC-Unique: NWszhd2ePECIrNEaetaAKQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9DFD6101A53B; Tue, 6 Jun 2023 08:19:20 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 243F240CF8F6; Tue, 6 Jun 2023 08:19:17 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 08/26] vhost: retry translating IOVA after IOTLB miss Date: Tue, 6 Jun 2023 10:18:34 +0200 Message-Id: <20230606081852.71003-9-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Vhost-user backend IOTLB misses and updates are asynchronous, so IOVA address translation function just fails after having sent an IOTLB miss update if needed entry was not in the IOTLB cache. This is not the case for VDUSE, for which the needed IOTLB update is returned directly when sending an IOTLB miss. This patch retry again finding the needed entry in the IOTLB cache after having sent an IOTLB miss. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vhost.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index 3ddd2a963f..7e1af487c1 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -100,6 +100,12 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, vhost_user_iotlb_rd_lock(vq); } + tmp_size = *size; + /* Retry in case of VDUSE, as it is synchronous */ + vva = vhost_user_iotlb_cache_find(dev, iova, &tmp_size, perm); + if (tmp_size == *size) + return vva; + return 0; } From patchwork Tue Jun 6 08:18:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128175 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F00C842C3E; Tue, 6 Jun 2023 10:20:09 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5243442D48; Tue, 6 Jun 2023 10:19:32 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 14D2642D3C for ; Tue, 6 Jun 2023 10:19:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039570; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RAJbhtRtDIaak0TIE345YBErRVtJxh6iqZrYiMJd/lE=; b=DJyL8hEjaOId8ZCYeDYZ4hKq66eh3ibBb4nY83OYx7dSeAEVMFwqhyuprd6wsSqHPTPInO 3tO4N2xQoRDw1FY1Cg5CpDLm69xKd2jOKuZK4rVLXPdGIgNHKXv91mEYT3ZjxQldZPTqFC qMUJw1Svpfx2I7hKiS8jXjOSw60TPuc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-549-mxqHh8MAPe6aKNi2XPcldQ-1; Tue, 06 Jun 2023 04:19:24 -0400 X-MC-Unique: mxqHh8MAPe6aKNi2XPcldQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 75466858F1C; Tue, 6 Jun 2023 08:19:23 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id EE76F40CF8F8; Tue, 6 Jun 2023 08:19:20 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 09/26] vhost: introduce backend ops Date: Tue, 6 Jun 2023 10:18:35 +0200 Message-Id: <20230606081852.71003-10-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduces backend ops struct, that will enable calling backend specifics callbacks (Vhost-user, VDUSE), in shared code. This is an empty shell for now, it will be filled in later patches. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/socket.c | 2 +- lib/vhost/vhost.c | 8 +++++++- lib/vhost/vhost.h | 10 +++++++++- lib/vhost/vhost_user.c | 8 ++++++++ lib/vhost/vhost_user.h | 1 + 5 files changed, 26 insertions(+), 3 deletions(-) diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c index cb0218b7bc..407d0011c3 100644 --- a/lib/vhost/socket.c +++ b/lib/vhost/socket.c @@ -223,7 +223,7 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket) return; } - vid = vhost_new_device(); + vid = vhost_user_new_device(); if (vid == -1) { goto err; } diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index 7e1af487c1..d054772bf8 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -680,11 +680,16 @@ reset_device(struct virtio_net *dev) * there is a new virtio device being attached). */ int -vhost_new_device(void) +vhost_new_device(struct vhost_backend_ops *ops) { struct virtio_net *dev; int i; + if (ops == NULL) { + VHOST_LOG_CONFIG("device", ERR, "missing backend ops.\n"); + return -1; + } + pthread_mutex_lock(&vhost_dev_lock); for (i = 0; i < RTE_MAX_VHOST_DEVICE; i++) { if (vhost_devices[i] == NULL) @@ -712,6 +717,7 @@ vhost_new_device(void) dev->backend_req_fd = -1; dev->postcopy_ufd = -1; rte_spinlock_init(&dev->backend_req_lock); + dev->backend_ops = ops; return i; } diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index de84c115b7..966b6dd67a 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -89,6 +89,12 @@ for (iter = val; iter < num; iter++) #endif +/** + * Structure that contains backend-specific ops. + */ +struct vhost_backend_ops { +}; + /** * Structure contains buffer address, length and descriptor index * from vring to do scatter RX. @@ -516,6 +522,8 @@ struct virtio_net { void *extern_data; /* pre and post vhost user message handlers for the device */ struct rte_vhost_user_extern_ops extern_ops; + + struct vhost_backend_ops *backend_ops; } __rte_cache_aligned; static inline void @@ -815,7 +823,7 @@ get_device(int vid) return dev; } -int vhost_new_device(void); +int vhost_new_device(struct vhost_backend_ops *ops); void cleanup_device(struct virtio_net *dev, int destroy); void reset_device(struct virtio_net *dev); void vhost_destroy_device(int); diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 98d8b8ac79..2cd86686a5 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3464,3 +3464,11 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) return ret; } + +static struct vhost_backend_ops vhost_user_backend_ops; + +int +vhost_user_new_device(void) +{ + return vhost_new_device(&vhost_user_backend_ops); +} diff --git a/lib/vhost/vhost_user.h b/lib/vhost/vhost_user.h index a0987a58f9..61456049c8 100644 --- a/lib/vhost/vhost_user.h +++ b/lib/vhost/vhost_user.h @@ -185,5 +185,6 @@ int vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm); int read_fd_message(char *ifname, int sockfd, char *buf, int buflen, int *fds, int max_fds, int *fd_num); int send_fd_message(char *ifname, int sockfd, char *buf, int buflen, int *fds, int fd_num); +int vhost_user_new_device(void); #endif From patchwork Tue Jun 6 08:18:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128174 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B043942C3E; Tue, 6 Jun 2023 10:20:03 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5CFED42BFE; Tue, 6 Jun 2023 10:19:29 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 3D8B542D29 for ; Tue, 6 Jun 2023 10:19:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039567; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E1CAwqEqZ0aCaZX14KXIsKgiIikY251g0neJCz5ALdQ=; b=GNVz0AMZayyf0UosQuUuiQJ7Tg1gI9ywxjLtZQ7t0rRg4Ts1CTNactX6r60BQdeFq4+DVn CX9jtmF6RN25IFwxUJVJWXIc1kthcFh65zx+c0QdGclxNsAmUXt1IAsC6LYUaV7Auk2pcF nHieQGF77k/5n/W/qJoZyUlzpADTK+Y= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-591-3ypC2WLJOc2PT40KsnjnxQ-1; Tue, 06 Jun 2023 04:19:27 -0400 X-MC-Unique: 3ypC2WLJOc2PT40KsnjnxQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 42EF41C06919; Tue, 6 Jun 2023 08:19:26 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBCE240CF8F6; Tue, 6 Jun 2023 08:19:23 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 10/26] vhost: add IOTLB cache entry removal callback Date: Tue, 6 Jun 2023 10:18:36 +0200 Message-Id: <20230606081852.71003-11-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org VDUSE will need to munmap() the IOTLB entry on removal from the cache, as it performs mmap() before insertion. This patch introduces a callback that VDUSE layer will implement to achieve this. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 12 ++++++++++++ lib/vhost/vhost.h | 3 +++ 2 files changed, 15 insertions(+) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 87986f2489..424121cc00 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -25,6 +25,15 @@ struct vhost_iotlb_entry { #define IOTLB_CACHE_SIZE 2048 +static void +vhost_user_iotlb_remove_notify(struct virtio_net *dev, struct vhost_iotlb_entry *entry) +{ + if (dev->backend_ops->iotlb_remove_notify == NULL) + return; + + dev->backend_ops->iotlb_remove_notify(entry->uaddr, entry->uoffset, entry->size); +} + static bool vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b) { @@ -198,6 +207,7 @@ vhost_user_iotlb_cache_remove_all(struct virtio_net *dev) vhost_user_iotlb_clear_dump(node, NULL, NULL); TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_remove_notify(dev, node); vhost_user_iotlb_pool_put(dev, node); } @@ -223,6 +233,7 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) vhost_user_iotlb_clear_dump(node, prev_node, next_node); TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_remove_notify(dev, node); vhost_user_iotlb_pool_put(dev, node); dev->iotlb_cache_nr--; break; @@ -314,6 +325,7 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t si vhost_user_iotlb_clear_dump(node, prev_node, next_node); TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_remove_notify(dev, node); vhost_user_iotlb_pool_put(dev, node); dev->iotlb_cache_nr--; } else { diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 966b6dd67a..69df8b14c0 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -89,10 +89,13 @@ for (iter = val; iter < num; iter++) #endif +typedef void (*vhost_iotlb_remove_notify)(uint64_t addr, uint64_t off, uint64_t size); + /** * Structure that contains backend-specific ops. */ struct vhost_backend_ops { + vhost_iotlb_remove_notify iotlb_remove_notify; }; /** From patchwork Tue Jun 6 08:18:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128176 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1565742C3E; Tue, 6 Jun 2023 10:20:18 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 98E4642D70; Tue, 6 Jun 2023 10:19:34 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 3C6EA42D63 for ; Tue, 6 Jun 2023 10:19:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039572; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MzQdgogwAu1MPoz6zTxH904RRXYwVsGzzVYeHbwQb+8=; b=Fx7BeFZeGiGo12h7cH1GPRsX8AIN2ScpE1y0y113IYwBcej8xr6H6oKcGKgcMO1b6nh/fw SWAw+4b3Uq1xm32sFUoiOYNu7HvuOa48F95aYtKx4Ob1xWLi1pHi6HXCcjE+C2F2TB5+Yi 0OihWMpsEIjx2jXrjDPRuXBi/FKm8l0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-303-Rw2uo-obP0-waBDi-ro0rQ-1; Tue, 06 Jun 2023 04:19:29 -0400 X-MC-Unique: Rw2uo-obP0-waBDi-ro0rQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0EEC0185A78B; Tue, 6 Jun 2023 08:19:29 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 88B9040CF8F6; Tue, 6 Jun 2023 08:19:26 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 11/26] vhost: add helper for IOTLB misses Date: Tue, 6 Jun 2023 10:18:37 +0200 Message-Id: <20230606081852.71003-12-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds a helper for sending IOTLB misses as VDUSE will use an ioctl while Vhost-user use a dedicated Vhost-user backend request. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vhost.c | 13 ++++++++++++- lib/vhost/vhost.h | 4 ++++ lib/vhost/vhost_user.c | 6 ++++-- lib/vhost/vhost_user.h | 1 - 4 files changed, 20 insertions(+), 4 deletions(-) diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index d054772bf8..f77f30d674 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -56,6 +56,12 @@ static const struct vhost_vq_stats_name_off vhost_vq_stat_strings[] = { #define VHOST_NB_VQ_STATS RTE_DIM(vhost_vq_stat_strings) +static int +vhost_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm) +{ + return dev->backend_ops->iotlb_miss(dev, iova, perm); +} + uint64_t __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, uint64_t iova, uint64_t *size, uint8_t perm) @@ -90,7 +96,7 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, vhost_user_iotlb_rd_unlock(vq); vhost_user_iotlb_pending_insert(dev, iova, perm); - if (vhost_user_iotlb_miss(dev, iova, perm)) { + if (vhost_iotlb_miss(dev, iova, perm)) { VHOST_LOG_DATA(dev->ifname, ERR, "IOTLB miss req failed for IOVA 0x%" PRIx64 "\n", iova); @@ -690,6 +696,11 @@ vhost_new_device(struct vhost_backend_ops *ops) return -1; } + if (ops->iotlb_miss == NULL) { + VHOST_LOG_CONFIG("device", ERR, "missing IOTLB miss backend op.\n"); + return -1; + } + pthread_mutex_lock(&vhost_dev_lock); for (i = 0; i < RTE_MAX_VHOST_DEVICE; i++) { if (vhost_devices[i] == NULL) diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 69df8b14c0..25255aaea8 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -89,13 +89,17 @@ for (iter = val; iter < num; iter++) #endif +struct virtio_net; typedef void (*vhost_iotlb_remove_notify)(uint64_t addr, uint64_t off, uint64_t size); +typedef int (*vhost_iotlb_miss_cb)(struct virtio_net *dev, uint64_t iova, uint8_t perm); + /** * Structure that contains backend-specific ops. */ struct vhost_backend_ops { vhost_iotlb_remove_notify iotlb_remove_notify; + vhost_iotlb_miss_cb iotlb_miss; }; /** diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 2cd86686a5..30ad63aba0 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3305,7 +3305,7 @@ vhost_user_msg_handler(int vid, int fd) return ret; } -int +static int vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm) { int ret; @@ -3465,7 +3465,9 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) return ret; } -static struct vhost_backend_ops vhost_user_backend_ops; +static struct vhost_backend_ops vhost_user_backend_ops = { + .iotlb_miss = vhost_user_iotlb_miss, +}; int vhost_user_new_device(void) diff --git a/lib/vhost/vhost_user.h b/lib/vhost/vhost_user.h index 61456049c8..1ffeca92f3 100644 --- a/lib/vhost/vhost_user.h +++ b/lib/vhost/vhost_user.h @@ -179,7 +179,6 @@ struct __rte_packed vhu_msg_context { /* vhost_user.c */ int vhost_user_msg_handler(int vid, int fd); -int vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm); /* socket.c */ int read_fd_message(char *ifname, int sockfd, char *buf, int buflen, int *fds, int max_fds, From patchwork Tue Jun 6 08:18:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128177 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F2ADF42C3E; Tue, 6 Jun 2023 10:20:23 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ADA1542D10; Tue, 6 Jun 2023 10:19:36 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 1918B42D75 for ; Tue, 6 Jun 2023 10:19:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039574; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BOFgfKVmUk5DBDxg658RC36571AMByiB0QftAZCajvA=; b=VZPMWCI+FaIvw7e2zbLeHWpk4Fb+wRWo2nHhzBevDfoYkEJfvbNSArIhN7+zH9ItCYMtNp StZmufRCufkXNasXKSGRM2FY/VPdt1HGgCSrgfmxGKbygibv0PPYYQJijM0WEj6IDzl3av nK/PXj5JZ/pMuPTB9nxIcNCMt4hqqh8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-635-TBUHM5wvMmGbktpqc1dPUw-1; Tue, 06 Jun 2023 04:19:33 -0400 X-MC-Unique: TBUHM5wvMmGbktpqc1dPUw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 036033825BB4; Tue, 6 Jun 2023 08:19:33 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5593040CF8F6; Tue, 6 Jun 2023 08:19:29 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 12/26] vhost: add helper for interrupt injection Date: Tue, 6 Jun 2023 10:18:38 +0200 Message-Id: <20230606081852.71003-13-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Vhost-user uses eventfd to inject IRQs, but VDUSE uses an ioctl. This patch prepares vhost_vring_call_split() and vhost_vring_call_packed() to support VDUSE by introducing a new helper. It also adds a new counter for guest notification failures, which could happen in case of uninitialized call file descriptor for example. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vhost.c | 25 +++++++++++++------------ lib/vhost/vhost.h | 19 +++++++++---------- lib/vhost/vhost_user.c | 10 ++++++++++ 3 files changed, 32 insertions(+), 22 deletions(-) diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index f77f30d674..eb6309b681 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -701,6 +701,11 @@ vhost_new_device(struct vhost_backend_ops *ops) return -1; } + if (ops->inject_irq == NULL) { + VHOST_LOG_CONFIG("device", ERR, "missing IRQ injection backend op.\n"); + return -1; + } + pthread_mutex_lock(&vhost_dev_lock); for (i = 0; i < RTE_MAX_VHOST_DEVICE; i++) { if (vhost_devices[i] == NULL) @@ -1511,20 +1516,16 @@ rte_vhost_notify_guest(int vid, uint16_t queue_id) rte_rwlock_read_lock(&vq->access_lock); - if (vq->callfd >= 0) { - int ret = eventfd_write(vq->callfd, (eventfd_t)1); - - if (ret) { - if (dev->flags & VIRTIO_DEV_STATS_ENABLED) - __atomic_fetch_add(&vq->stats.guest_notifications_error, + if (dev->backend_ops->inject_irq(dev, vq)) { + if (dev->flags & VIRTIO_DEV_STATS_ENABLED) + __atomic_fetch_add(&vq->stats.guest_notifications_error, 1, __ATOMIC_RELAXED); - } else { - if (dev->flags & VIRTIO_DEV_STATS_ENABLED) - __atomic_fetch_add(&vq->stats.guest_notifications, + } else { + if (dev->flags & VIRTIO_DEV_STATS_ENABLED) + __atomic_fetch_add(&vq->stats.guest_notifications, 1, __ATOMIC_RELAXED); - if (dev->notify_ops->guest_notified) - dev->notify_ops->guest_notified(dev->vid); - } + if (dev->notify_ops->guest_notified) + dev->notify_ops->guest_notified(dev->vid); } rte_rwlock_read_unlock(&vq->access_lock); diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 25255aaea8..ea2798b0bf 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -90,16 +90,20 @@ #endif struct virtio_net; +struct vhost_virtqueue; + typedef void (*vhost_iotlb_remove_notify)(uint64_t addr, uint64_t off, uint64_t size); typedef int (*vhost_iotlb_miss_cb)(struct virtio_net *dev, uint64_t iova, uint8_t perm); +typedef int (*vhost_vring_inject_irq_cb)(struct virtio_net *dev, struct vhost_virtqueue *vq); /** * Structure that contains backend-specific ops. */ struct vhost_backend_ops { vhost_iotlb_remove_notify iotlb_remove_notify; vhost_iotlb_miss_cb iotlb_miss; + vhost_vring_inject_irq_cb inject_irq; }; /** @@ -906,8 +910,6 @@ vhost_need_event(uint16_t event_idx, uint16_t new_idx, uint16_t old) static __rte_always_inline void vhost_vring_inject_irq(struct virtio_net *dev, struct vhost_virtqueue *vq) { - int ret; - if (dev->notify_ops->guest_notify && dev->notify_ops->guest_notify(dev->vid, vq->index)) { if (dev->flags & VIRTIO_DEV_STATS_ENABLED) @@ -916,8 +918,7 @@ vhost_vring_inject_irq(struct virtio_net *dev, struct vhost_virtqueue *vq) return; } - ret = eventfd_write(vq->callfd, (eventfd_t) 1); - if (ret) { + if (dev->backend_ops->inject_irq(dev, vq)) { if (dev->flags & VIRTIO_DEV_STATS_ENABLED) __atomic_fetch_add(&vq->stats.guest_notifications_error, 1, __ATOMIC_RELAXED); @@ -950,14 +951,12 @@ vhost_vring_call_split(struct virtio_net *dev, struct vhost_virtqueue *vq) "%s: used_event_idx=%d, old=%d, new=%d\n", __func__, vhost_used_event(vq), old, new); - if ((vhost_need_event(vhost_used_event(vq), new, old) || - unlikely(!signalled_used_valid)) && - vq->callfd >= 0) + if (vhost_need_event(vhost_used_event(vq), new, old) || + unlikely(!signalled_used_valid)) vhost_vring_inject_irq(dev, vq); } else { /* Kick the guest if necessary. */ - if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT) && - (vq->callfd >= 0)) + if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT)) vhost_vring_inject_irq(dev, vq); } } @@ -1009,7 +1008,7 @@ vhost_vring_call_packed(struct virtio_net *dev, struct vhost_virtqueue *vq) if (vhost_need_event(off, new, old)) kick = true; kick: - if (kick && vq->callfd >= 0) + if (kick) vhost_vring_inject_irq(dev, vq); } diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 30ad63aba0..901a80bbaa 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3465,8 +3465,18 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) return ret; } +static int +vhost_user_inject_irq(struct virtio_net *dev __rte_unused, struct vhost_virtqueue *vq) +{ + if (vq->callfd < 0) + return -1; + + return eventfd_write(vq->callfd, (eventfd_t)1); +} + static struct vhost_backend_ops vhost_user_backend_ops = { .iotlb_miss = vhost_user_iotlb_miss, + .inject_irq = vhost_user_inject_irq, }; int From patchwork Tue Jun 6 08:18:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128178 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7F51A42C3E; Tue, 6 Jun 2023 10:20:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A96A742D20; Tue, 6 Jun 2023 10:19:41 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id D2B8642D20 for ; Tue, 6 Jun 2023 10:19:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039579; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eQsdqTKoZFsA+LFxY9lC29OGa2HfDgN3GjZo5acYI5s=; b=BHzPuMd7ylxsJHsR/Iee4W9hccq9WXx7dnkBDzLPUU5sC99kcG77M32z5tCQcIX69Lckzk JWGhO9NicoDeOvS1w9eIvuGJi5zznEzo1henbwHq0LWuS4QG9mAZkqV2dS84S5hl1BhBTg EtRpc8fl9yloy65RZZ9dOJiCbivye5Q= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-474-21fSeHURNjuc0RzWc6956g-1; Tue, 06 Jun 2023 04:19:36 -0400 X-MC-Unique: 21fSeHURNjuc0RzWc6956g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CEC8A280AA20; Tue, 6 Jun 2023 08:19:35 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 53D0440CF8F6; Tue, 6 Jun 2023 08:19:33 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 13/26] vhost: add API to set max queue pairs Date: Tue, 6 Jun 2023 10:18:39 +0200 Message-Id: <20230606081852.71003-14-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduces a new rte_vhost_driver_set_max_queues API as preliminary work for multiqueue support with VDUSE. Indeed, with VDUSE we need to pre-allocate the vrings at device creation time, so we need such API not to allocate the 128 queue pairs supported by the Vhost library. Calling the API is optional, 128 queue pairs remaining the default. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- doc/guides/prog_guide/vhost_lib.rst | 4 +++ doc/guides/rel_notes/release_23_07.rst | 5 ++++ lib/vhost/rte_vhost.h | 17 ++++++++++++ lib/vhost/socket.c | 36 ++++++++++++++++++++++++-- lib/vhost/version.map | 1 + 5 files changed, 61 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst index 0f964d7a4a..0c2b4d020a 100644 --- a/doc/guides/prog_guide/vhost_lib.rst +++ b/doc/guides/prog_guide/vhost_lib.rst @@ -339,6 +339,10 @@ The following is an overview of some key Vhost API functions: Inject the offloaded interrupt received by the 'guest_notify' callback, into the vhost device's queue. +* ``rte_vhost_driver_set_max_queue_num(const char *path, uint32_t max_queue_pairs)`` + + Set the maximum number of queue pairs supported by the device. + Vhost-user Implementations -------------------------- diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index 3eed8ac561..7034fb664c 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -62,6 +62,11 @@ New Features rte_vhost_notify_guest(), is added to raise the interrupt outside of the fast path. +* **Added Vhost API to set maximum queue pairs supported.** + + Introduced ``rte_vhost_driver_set_max_queue_num()`` to be able to limit the + maximum number of supported queue pairs, required for VDUSE support. + Removed Items ------------- diff --git a/lib/vhost/rte_vhost.h b/lib/vhost/rte_vhost.h index 7a10bc36cf..7844c9f142 100644 --- a/lib/vhost/rte_vhost.h +++ b/lib/vhost/rte_vhost.h @@ -609,6 +609,23 @@ rte_vhost_driver_get_protocol_features(const char *path, int rte_vhost_driver_get_queue_num(const char *path, uint32_t *queue_num); +/** + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice. + * + * Set the maximum number of queue pairs supported by the device. + * + * @param path + * The vhost-user socket file path + * @param max_queue_pairs + * The maximum number of queue pairs + * @return + * 0 on success, -1 on failure + */ +__rte_experimental +int +rte_vhost_driver_set_max_queue_num(const char *path, uint32_t max_queue_pairs); + /** * Get the feature bits after negotiation * diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c index 407d0011c3..29f7a8cece 100644 --- a/lib/vhost/socket.c +++ b/lib/vhost/socket.c @@ -57,6 +57,8 @@ struct vhost_user_socket { uint64_t protocol_features; + uint32_t max_queue_pairs; + struct rte_vdpa_device *vdpa_dev; struct rte_vhost_device_ops const *notify_ops; @@ -823,7 +825,7 @@ rte_vhost_driver_get_queue_num(const char *path, uint32_t *queue_num) vdpa_dev = vsocket->vdpa_dev; if (!vdpa_dev) { - *queue_num = VHOST_MAX_QUEUE_PAIRS; + *queue_num = vsocket->max_queue_pairs; goto unlock_exit; } @@ -833,7 +835,36 @@ rte_vhost_driver_get_queue_num(const char *path, uint32_t *queue_num) goto unlock_exit; } - *queue_num = RTE_MIN((uint32_t)VHOST_MAX_QUEUE_PAIRS, vdpa_queue_num); + *queue_num = RTE_MIN(vsocket->max_queue_pairs, vdpa_queue_num); + +unlock_exit: + pthread_mutex_unlock(&vhost_user.mutex); + return ret; +} + +int +rte_vhost_driver_set_max_queue_num(const char *path, uint32_t max_queue_pairs) +{ + struct vhost_user_socket *vsocket; + int ret = 0; + + VHOST_LOG_CONFIG(path, INFO, "Setting max queue pairs to %u\n", max_queue_pairs); + + if (max_queue_pairs > VHOST_MAX_QUEUE_PAIRS) { + VHOST_LOG_CONFIG(path, ERR, "Library only supports up to %u queue pairs\n", + VHOST_MAX_QUEUE_PAIRS); + return -1; + } + + pthread_mutex_lock(&vhost_user.mutex); + vsocket = find_vhost_user_socket(path); + if (!vsocket) { + VHOST_LOG_CONFIG(path, ERR, "socket file is not registered yet.\n"); + ret = -1; + goto unlock_exit; + } + + vsocket->max_queue_pairs = max_queue_pairs; unlock_exit: pthread_mutex_unlock(&vhost_user.mutex); @@ -889,6 +920,7 @@ rte_vhost_driver_register(const char *path, uint64_t flags) goto out_free; } vsocket->vdpa_dev = NULL; + vsocket->max_queue_pairs = VHOST_MAX_QUEUE_PAIRS; vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT; vsocket->linearbuf = flags & RTE_VHOST_USER_LINEARBUF_SUPPORT; vsocket->async_copy = flags & RTE_VHOST_USER_ASYNC_COPY; diff --git a/lib/vhost/version.map b/lib/vhost/version.map index 7bcbfd12cf..5051c29022 100644 --- a/lib/vhost/version.map +++ b/lib/vhost/version.map @@ -107,6 +107,7 @@ EXPERIMENTAL { # added in 23.07 rte_vhost_notify_guest; + rte_vhost_driver_set_max_queue_num; }; INTERNAL { From patchwork Tue Jun 6 08:18:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128179 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BFB3942C3E; Tue, 6 Jun 2023 10:20:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AC4C942D1D; Tue, 6 Jun 2023 10:19:44 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id A683D42D1D for ; Tue, 6 Jun 2023 10:19:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039583; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HKy6hSXmK5VsHBV/QWhIAznNiUc9xtI8d5uBiU8cXMs=; b=HN9dL+WddHt5EPgfi8IKoN3jfGb15YK3XwDapogX9CZSKFtMvnlBZgVnKpt6+ZuvwXpSDi 5CIZYe8lNIsuxQFkFPABHYqn2xOzhnGSCt246pNR+a84cYn0dni7BCHsV5Mj6mZFnExnJp TkqHR+AuDzJqqilCrsHWkw6zh3Ptwk4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-85-mr0rF523PYaJjBSrrUQ06w-1; Tue, 06 Jun 2023 04:19:39 -0400 X-MC-Unique: mr0rF523PYaJjBSrrUQ06w-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9BFFF801585; Tue, 6 Jun 2023 08:19:38 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2045540CF8F6; Tue, 6 Jun 2023 08:19:35 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 14/26] net/vhost: use API to set max queue pairs Date: Tue, 6 Jun 2023 10:18:40 +0200 Message-Id: <20230606081852.71003-15-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In order to support multiqueue with VDUSE, we need to be able to limit the maximum number of queue pairs, to avoid unnecessary memory consumption since the maximum number of queue pairs need to be allocated at device creation time, as opposed to Vhost-user which allocate only when the frontend initialize them. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- drivers/net/vhost/rte_eth_vhost.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c index 62ef955ebc..8d37ec9775 100644 --- a/drivers/net/vhost/rte_eth_vhost.c +++ b/drivers/net/vhost/rte_eth_vhost.c @@ -1013,6 +1013,9 @@ vhost_driver_setup(struct rte_eth_dev *eth_dev) goto drv_unreg; } + if (rte_vhost_driver_set_max_queue_num(internal->iface_name, internal->max_queues)) + goto drv_unreg; + if (rte_vhost_driver_callback_register(internal->iface_name, &vhost_ops) < 0) { VHOST_LOG(ERR, "Can't register callbacks\n"); From patchwork Tue Jun 6 08:18:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128180 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 911AD42C3E; Tue, 6 Jun 2023 10:20:41 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BD3C542D85; Tue, 6 Jun 2023 10:19:46 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 69EA842D56 for ; Tue, 6 Jun 2023 10:19:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039585; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YPQYGUKXd3T5G0nzSO7UYxof2jO85vBggzbd+oY0ZrI=; b=In4UD6wxCkzkPN8CiaYpoaXwpjoMjI6bK9czYfby42bQ/ScHMOZEqgCZaVrkK8lJgMZFY1 TlCmIPUsnJVhywqWdKgRCZ2Klwf2Um3pXTkSjhoE8Vt2O1TUwScqN8tHOckpSfqs589MfT iLHcOQ6mrioOrp3mOQW4LnFGabt3yps= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-155-hsdVi_GmP3y7aueGXhFQuw-1; Tue, 06 Jun 2023 04:19:42 -0400 X-MC-Unique: hsdVi_GmP3y7aueGXhFQuw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 73642801182; Tue, 6 Jun 2023 08:19:41 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id ECF2140CF8F6; Tue, 6 Jun 2023 08:19:38 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 15/26] vhost: add control virtqueue support Date: Tue, 6 Jun 2023 10:18:41 +0200 Message-Id: <20230606081852.71003-16-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In order to support multi-queue with VDUSE, having control queue support is required. This patch adds control queue implementation, it will be used later when adding VDUSE support. Only split ring layout is supported for now, packed ring support will be added later. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/meson.build | 1 + lib/vhost/vhost.h | 2 + lib/vhost/virtio_net_ctrl.c | 286 ++++++++++++++++++++++++++++++++++++ lib/vhost/virtio_net_ctrl.h | 10 ++ 4 files changed, 299 insertions(+) create mode 100644 lib/vhost/virtio_net_ctrl.c create mode 100644 lib/vhost/virtio_net_ctrl.h diff --git a/lib/vhost/meson.build b/lib/vhost/meson.build index 05679447db..d211a0bd37 100644 --- a/lib/vhost/meson.build +++ b/lib/vhost/meson.build @@ -27,6 +27,7 @@ sources = files( 'vhost_crypto.c', 'vhost_user.c', 'virtio_net.c', + 'virtio_net_ctrl.c', ) headers = files( 'rte_vdpa.h', diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index ea2798b0bf..04267a3ac5 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -527,6 +527,8 @@ struct virtio_net { int postcopy_ufd; int postcopy_listening; + struct vhost_virtqueue *cvq; + struct rte_vdpa_device *vdpa_dev; /* context data for the external message handlers */ diff --git a/lib/vhost/virtio_net_ctrl.c b/lib/vhost/virtio_net_ctrl.c new file mode 100644 index 0000000000..6b583a0ce6 --- /dev/null +++ b/lib/vhost/virtio_net_ctrl.c @@ -0,0 +1,286 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#include +#include +#include + +#include "iotlb.h" +#include "vhost.h" +#include "virtio_net_ctrl.h" + +struct virtio_net_ctrl { + uint8_t class; + uint8_t command; + uint8_t command_data[]; +}; + +struct virtio_net_ctrl_elem { + struct virtio_net_ctrl *ctrl_req; + uint16_t head_idx; + uint16_t n_descs; + uint8_t *desc_ack; +}; + +static int +virtio_net_ctrl_pop(struct virtio_net *dev, struct vhost_virtqueue *cvq, + struct virtio_net_ctrl_elem *ctrl_elem) + __rte_shared_locks_required(&cvq->iotlb_lock) +{ + uint16_t avail_idx, desc_idx, n_descs = 0; + uint64_t desc_len, desc_addr, desc_iova, data_len = 0; + uint8_t *ctrl_req; + struct vring_desc *descs; + + avail_idx = __atomic_load_n(&cvq->avail->idx, __ATOMIC_ACQUIRE); + if (avail_idx == cvq->last_avail_idx) { + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "Control queue empty\n"); + return 0; + } + + desc_idx = cvq->avail->ring[cvq->last_avail_idx]; + if (desc_idx >= cvq->size) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Out of range desc index, dropping\n"); + goto err; + } + + ctrl_elem->head_idx = desc_idx; + + if (cvq->desc[desc_idx].flags & VRING_DESC_F_INDIRECT) { + desc_len = cvq->desc[desc_idx].len; + desc_iova = cvq->desc[desc_idx].addr; + + descs = (struct vring_desc *)(uintptr_t)vhost_iova_to_vva(dev, cvq, + desc_iova, &desc_len, VHOST_ACCESS_RO); + if (!descs || desc_len != cvq->desc[desc_idx].len) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to map ctrl indirect descs\n"); + goto err; + } + + desc_idx = 0; + } else { + descs = cvq->desc; + } + + while (1) { + desc_len = descs[desc_idx].len; + desc_iova = descs[desc_idx].addr; + + n_descs++; + + if (descs[desc_idx].flags & VRING_DESC_F_WRITE) { + if (ctrl_elem->desc_ack) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Unexpected ctrl chain layout\n"); + goto err; + } + + if (desc_len != sizeof(uint8_t)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Invalid ack size for ctrl req, dropping\n"); + goto err; + } + + ctrl_elem->desc_ack = (uint8_t *)(uintptr_t)vhost_iova_to_vva(dev, cvq, + desc_iova, &desc_len, VHOST_ACCESS_WO); + if (!ctrl_elem->desc_ack || desc_len != sizeof(uint8_t)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to map ctrl ack descriptor\n"); + goto err; + } + } else { + if (ctrl_elem->desc_ack) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Unexpected ctrl chain layout\n"); + goto err; + } + + data_len += desc_len; + } + + if (!(descs[desc_idx].flags & VRING_DESC_F_NEXT)) + break; + + desc_idx = descs[desc_idx].next; + } + + desc_idx = ctrl_elem->head_idx; + + if (cvq->desc[desc_idx].flags & VRING_DESC_F_INDIRECT) + ctrl_elem->n_descs = 1; + else + ctrl_elem->n_descs = n_descs; + + if (!ctrl_elem->desc_ack) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Missing ctrl ack descriptor\n"); + goto err; + } + + if (data_len < sizeof(ctrl_elem->ctrl_req->class) + sizeof(ctrl_elem->ctrl_req->command)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Invalid control header size\n"); + goto err; + } + + ctrl_elem->ctrl_req = malloc(data_len); + if (!ctrl_elem->ctrl_req) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to alloc ctrl request\n"); + goto err; + } + + ctrl_req = (uint8_t *)ctrl_elem->ctrl_req; + + if (cvq->desc[desc_idx].flags & VRING_DESC_F_INDIRECT) { + desc_len = cvq->desc[desc_idx].len; + desc_iova = cvq->desc[desc_idx].addr; + + descs = (struct vring_desc *)(uintptr_t)vhost_iova_to_vva(dev, cvq, + desc_iova, &desc_len, VHOST_ACCESS_RO); + if (!descs || desc_len != cvq->desc[desc_idx].len) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to map ctrl indirect descs\n"); + goto free_err; + } + + desc_idx = 0; + } else { + descs = cvq->desc; + } + + while (!(descs[desc_idx].flags & VRING_DESC_F_WRITE)) { + desc_len = descs[desc_idx].len; + desc_iova = descs[desc_idx].addr; + + desc_addr = vhost_iova_to_vva(dev, cvq, desc_iova, &desc_len, VHOST_ACCESS_RO); + if (!desc_addr || desc_len < descs[desc_idx].len) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to map ctrl descriptor\n"); + goto free_err; + } + + memcpy(ctrl_req, (void *)(uintptr_t)desc_addr, desc_len); + ctrl_req += desc_len; + + if (!(descs[desc_idx].flags & VRING_DESC_F_NEXT)) + break; + + desc_idx = descs[desc_idx].next; + } + + cvq->last_avail_idx++; + if (cvq->last_avail_idx >= cvq->size) + cvq->last_avail_idx -= cvq->size; + + if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX)) + vhost_avail_event(cvq) = cvq->last_avail_idx; + + return 1; + +free_err: + free(ctrl_elem->ctrl_req); +err: + cvq->last_avail_idx++; + if (cvq->last_avail_idx >= cvq->size) + cvq->last_avail_idx -= cvq->size; + + if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX)) + vhost_avail_event(cvq) = cvq->last_avail_idx; + + return -1; +} + +static uint8_t +virtio_net_ctrl_handle_req(struct virtio_net *dev, struct virtio_net_ctrl *ctrl_req) +{ + uint8_t ret = VIRTIO_NET_ERR; + + if (ctrl_req->class == VIRTIO_NET_CTRL_MQ && + ctrl_req->command == VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET) { + uint16_t queue_pairs; + uint32_t i; + + queue_pairs = *(uint16_t *)(uintptr_t)ctrl_req->command_data; + VHOST_LOG_CONFIG(dev->ifname, INFO, "Ctrl req: MQ %u queue pairs\n", queue_pairs); + ret = VIRTIO_NET_OK; + + for (i = 0; i < dev->nr_vring; i++) { + struct vhost_virtqueue *vq = dev->virtqueue[i]; + bool enable; + + if (vq == dev->cvq) + continue; + + if (i < queue_pairs * 2) + enable = true; + else + enable = false; + + vq->enabled = enable; + if (dev->notify_ops->vring_state_changed) + dev->notify_ops->vring_state_changed(dev->vid, i, enable); + } + } + + return ret; +} + +static int +virtio_net_ctrl_push(struct virtio_net *dev, struct virtio_net_ctrl_elem *ctrl_elem) +{ + struct vhost_virtqueue *cvq = dev->cvq; + struct vring_used_elem *used_elem; + + used_elem = &cvq->used->ring[cvq->last_used_idx]; + used_elem->id = ctrl_elem->head_idx; + used_elem->len = ctrl_elem->n_descs; + + cvq->last_used_idx++; + if (cvq->last_used_idx >= cvq->size) + cvq->last_used_idx -= cvq->size; + + __atomic_store_n(&cvq->used->idx, cvq->last_used_idx, __ATOMIC_RELEASE); + + vhost_vring_call_split(dev, dev->cvq); + + free(ctrl_elem->ctrl_req); + + return 0; +} + +int +virtio_net_ctrl_handle(struct virtio_net *dev) +{ + int ret = 0; + + if (dev->features & (1ULL << VIRTIO_F_RING_PACKED)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Packed ring not supported yet\n"); + return -1; + } + + if (!dev->cvq) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "missing control queue\n"); + return -1; + } + + rte_rwlock_read_lock(&dev->cvq->access_lock); + vhost_user_iotlb_rd_lock(dev->cvq); + + while (1) { + struct virtio_net_ctrl_elem ctrl_elem; + + memset(&ctrl_elem, 0, sizeof(struct virtio_net_ctrl_elem)); + + ret = virtio_net_ctrl_pop(dev, dev->cvq, &ctrl_elem); + if (ret <= 0) + break; + + *ctrl_elem.desc_ack = virtio_net_ctrl_handle_req(dev, ctrl_elem.ctrl_req); + + ret = virtio_net_ctrl_push(dev, &ctrl_elem); + if (ret < 0) + break; + } + + vhost_user_iotlb_rd_unlock(dev->cvq); + rte_rwlock_read_unlock(&dev->cvq->access_lock); + + return ret; +} diff --git a/lib/vhost/virtio_net_ctrl.h b/lib/vhost/virtio_net_ctrl.h new file mode 100644 index 0000000000..9a90f4b9da --- /dev/null +++ b/lib/vhost/virtio_net_ctrl.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#ifndef _VIRTIO_NET_CTRL_H +#define _VIRTIO_NET_CTRL_H + +int virtio_net_ctrl_handle(struct virtio_net *dev); + +#endif From patchwork Tue Jun 6 08:18:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128181 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4CFF542C3E; Tue, 6 Jun 2023 10:20:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2A26942D3C; Tue, 6 Jun 2023 10:19:49 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 48C3442D56 for ; Tue, 6 Jun 2023 10:19:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039585; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Srz5SyFp3lpfq/AC319jSE9Sw5G0wmsLmdf65yy9f0g=; b=CJKmgQSkqGD3EFOGPOL8ph0rM2e+Mf+oZse7RNjkLM68q51MpYV/kBnOa2NC9jgpW9BczS 9r4X0ZCWHMI9Hchyvhqf/dqiy5RzQyycKT5HN7VP3OG6z6Fs1kzYA9v/rbYR2xvRM0vkZ+ SQv9+Z+z7yFlz6NL32bzPB5FFZPcXtg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-98-9ndNdiRLMcii-QtNqsUPTA-1; Tue, 06 Jun 2023 04:19:44 -0400 X-MC-Unique: 9ndNdiRLMcii-QtNqsUPTA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 40D33185A78B; Tue, 6 Jun 2023 08:19:44 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA18940CF8F7; Tue, 6 Jun 2023 08:19:41 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 16/26] vhost: add VDUSE device creation and destruction Date: Tue, 6 Jun 2023 10:18:42 +0200 Message-Id: <20230606081852.71003-17-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds initial support for VDUSE, which includes the device creation and destruction. It does not include the virtqueues configuration, so this is not functionnal at this point. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/meson.build | 4 + lib/vhost/socket.c | 34 ++++--- lib/vhost/vduse.c | 201 ++++++++++++++++++++++++++++++++++++++++++ lib/vhost/vduse.h | 33 +++++++ lib/vhost/vhost.h | 2 + 5 files changed, 262 insertions(+), 12 deletions(-) create mode 100644 lib/vhost/vduse.c create mode 100644 lib/vhost/vduse.h diff --git a/lib/vhost/meson.build b/lib/vhost/meson.build index d211a0bd37..13e0382c92 100644 --- a/lib/vhost/meson.build +++ b/lib/vhost/meson.build @@ -29,6 +29,10 @@ sources = files( 'virtio_net.c', 'virtio_net_ctrl.c', ) +if cc.has_header('linux/vduse.h') + sources += files('vduse.c') + cflags += '-DVHOST_HAS_VDUSE' +endif headers = files( 'rte_vdpa.h', 'rte_vhost.h', diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c index 29f7a8cece..19a7469e45 100644 --- a/lib/vhost/socket.c +++ b/lib/vhost/socket.c @@ -19,6 +19,7 @@ #include #include "fd_man.h" +#include "vduse.h" #include "vhost.h" #include "vhost_user.h" @@ -36,6 +37,7 @@ struct vhost_user_socket { int socket_fd; struct sockaddr_un un; bool is_server; + bool is_vduse; bool reconnect; bool iommu_support; bool use_builtin_virtio_net; @@ -991,18 +993,21 @@ rte_vhost_driver_register(const char *path, uint64_t flags) #endif } - if ((flags & RTE_VHOST_USER_CLIENT) != 0) { - vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT); - if (vsocket->reconnect && reconn_tid == 0) { - if (vhost_user_reconnect_init() != 0) - goto out_mutex; - } + if (!strncmp("/dev/vduse/", path, strlen("/dev/vduse/"))) { + vsocket->is_vduse = true; } else { - vsocket->is_server = true; - } - ret = create_unix_socket(vsocket); - if (ret < 0) { - goto out_mutex; + if ((flags & RTE_VHOST_USER_CLIENT) != 0) { + vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT); + if (vsocket->reconnect && reconn_tid == 0) { + if (vhost_user_reconnect_init() != 0) + goto out_mutex; + } + } else { + vsocket->is_server = true; + } + ret = create_unix_socket(vsocket); + if (ret < 0) + goto out_mutex; } vhost_user.vsockets[vhost_user.vsocket_cnt++] = vsocket; @@ -1067,7 +1072,9 @@ rte_vhost_driver_unregister(const char *path) if (strcmp(vsocket->path, path)) continue; - if (vsocket->is_server) { + if (vsocket->is_vduse) { + vduse_device_destroy(path); + } else if (vsocket->is_server) { /* * If r/wcb is executing, release vhost_user's * mutex lock, and try again since the r/wcb @@ -1218,6 +1225,9 @@ rte_vhost_driver_start(const char *path) if (!vsocket) return -1; + if (vsocket->is_vduse) + return vduse_device_create(path); + if (fdset_tid == 0) { /** * create a pipe which will be waited by poll and notified to diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c new file mode 100644 index 0000000000..d67818bfb5 --- /dev/null +++ b/lib/vhost/vduse.c @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#include +#include +#include +#include + + +#include +#include + +#include +#include + +#include + +#include "vduse.h" +#include "vhost.h" + +#define VHOST_VDUSE_API_VERSION 0 +#define VDUSE_CTRL_PATH "/dev/vduse/control" + +#define VDUSE_NET_SUPPORTED_FEATURES ((1ULL << VIRTIO_NET_F_MRG_RXBUF) | \ + (1ULL << VIRTIO_F_ANY_LAYOUT) | \ + (1ULL << VIRTIO_F_VERSION_1) | \ + (1ULL << VIRTIO_NET_F_GSO) | \ + (1ULL << VIRTIO_NET_F_HOST_TSO4) | \ + (1ULL << VIRTIO_NET_F_HOST_TSO6) | \ + (1ULL << VIRTIO_NET_F_HOST_UFO) | \ + (1ULL << VIRTIO_NET_F_HOST_ECN) | \ + (1ULL << VIRTIO_NET_F_CSUM) | \ + (1ULL << VIRTIO_NET_F_GUEST_CSUM) | \ + (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \ + (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \ + (1ULL << VIRTIO_NET_F_GUEST_UFO) | \ + (1ULL << VIRTIO_NET_F_GUEST_ECN) | \ + (1ULL << VIRTIO_RING_F_INDIRECT_DESC) | \ + (1ULL << VIRTIO_F_IN_ORDER) | \ + (1ULL << VIRTIO_F_IOMMU_PLATFORM)) + +static struct vhost_backend_ops vduse_backend_ops = { +}; + +int +vduse_device_create(const char *path) +{ + int control_fd, dev_fd, vid, ret; + uint32_t i; + struct virtio_net *dev; + uint64_t ver = VHOST_VDUSE_API_VERSION; + struct vduse_dev_config *dev_config = NULL; + const char *name = path + strlen("/dev/vduse/"); + + control_fd = open(VDUSE_CTRL_PATH, O_RDWR); + if (control_fd < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to open %s: %s\n", + VDUSE_CTRL_PATH, strerror(errno)); + return -1; + } + + if (ioctl(control_fd, VDUSE_SET_API_VERSION, &ver)) { + VHOST_LOG_CONFIG(name, ERR, "Failed to set API version: %" PRIu64 ": %s\n", + ver, strerror(errno)); + ret = -1; + goto out_ctrl_close; + } + + dev_config = malloc(offsetof(struct vduse_dev_config, config)); + if (!dev_config) { + VHOST_LOG_CONFIG(name, ERR, "Failed to allocate VDUSE config\n"); + ret = -1; + goto out_ctrl_close; + } + + memset(dev_config, 0, sizeof(struct vduse_dev_config)); + + strncpy(dev_config->name, name, VDUSE_NAME_MAX - 1); + dev_config->device_id = VIRTIO_ID_NET; + dev_config->vendor_id = 0; + dev_config->features = VDUSE_NET_SUPPORTED_FEATURES; + dev_config->vq_num = 2; + dev_config->vq_align = sysconf(_SC_PAGE_SIZE); + dev_config->config_size = 0; + + ret = ioctl(control_fd, VDUSE_CREATE_DEV, dev_config); + if (ret < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to create VDUSE device: %s\n", + strerror(errno)); + goto out_free; + } + + dev_fd = open(path, O_RDWR); + if (dev_fd < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to open device %s: %s\n", + path, strerror(errno)); + ret = -1; + goto out_dev_close; + } + + ret = fcntl(dev_fd, F_SETFL, O_NONBLOCK); + if (ret < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to set chardev as non-blocking: %s\n", + strerror(errno)); + goto out_dev_close; + } + + vid = vhost_new_device(&vduse_backend_ops); + if (vid < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to create new Vhost device\n"); + ret = -1; + goto out_dev_close; + } + + dev = get_device(vid); + if (!dev) { + ret = -1; + goto out_dev_close; + } + + strncpy(dev->ifname, path, IF_NAME_SZ - 1); + dev->vduse_ctrl_fd = control_fd; + dev->vduse_dev_fd = dev_fd; + vhost_setup_virtio_net(dev->vid, true, true, true, true); + + for (i = 0; i < 2; i++) { + struct vduse_vq_config vq_cfg = { 0 }; + + ret = alloc_vring_queue(dev, i); + if (ret) { + VHOST_LOG_CONFIG(name, ERR, "Failed to alloc vring %d metadata\n", i); + goto out_dev_destroy; + } + + vq_cfg.index = i; + vq_cfg.max_size = 1024; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP, &vq_cfg); + if (ret) { + VHOST_LOG_CONFIG(name, ERR, "Failed to set-up VQ %d\n", i); + goto out_dev_destroy; + } + } + + free(dev_config); + + return 0; + +out_dev_destroy: + vhost_destroy_device(vid); +out_dev_close: + if (dev_fd >= 0) + close(dev_fd); + ioctl(control_fd, VDUSE_DESTROY_DEV, name); +out_free: + free(dev_config); +out_ctrl_close: + close(control_fd); + + return ret; +} + +int +vduse_device_destroy(const char *path) +{ + const char *name = path + strlen("/dev/vduse/"); + struct virtio_net *dev; + int vid, ret; + + for (vid = 0; vid < RTE_MAX_VHOST_DEVICE; vid++) { + dev = vhost_devices[vid]; + + if (dev == NULL) + continue; + + if (!strcmp(path, dev->ifname)) + break; + } + + if (vid == RTE_MAX_VHOST_DEVICE) + return -1; + + if (dev->vduse_dev_fd >= 0) { + close(dev->vduse_dev_fd); + dev->vduse_dev_fd = -1; + } + + if (dev->vduse_ctrl_fd >= 0) { + ret = ioctl(dev->vduse_ctrl_fd, VDUSE_DESTROY_DEV, name); + if (ret) + VHOST_LOG_CONFIG(name, ERR, "Failed to destroy VDUSE device: %s\n", + strerror(errno)); + close(dev->vduse_ctrl_fd); + dev->vduse_ctrl_fd = -1; + } + + vhost_destroy_device(vid); + + return 0; +} diff --git a/lib/vhost/vduse.h b/lib/vhost/vduse.h new file mode 100644 index 0000000000..a15e5d4c16 --- /dev/null +++ b/lib/vhost/vduse.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#ifndef _VDUSE_H +#define _VDUSE_H + +#include "vhost.h" + +#ifdef VHOST_HAS_VDUSE + +int vduse_device_create(const char *path); +int vduse_device_destroy(const char *path); + +#else + +static inline int +vduse_device_create(const char *path) +{ + VHOST_LOG_CONFIG(path, ERR, "VDUSE support disabled at build time\n"); + return -1; +} + +static inline int +vduse_device_destroy(const char *path) +{ + VHOST_LOG_CONFIG(path, ERR, "VDUSE support disabled at build time\n"); + return -1; +} + +#endif /* VHOST_HAS_VDUSE */ + +#endif /* _VDUSE_H */ diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 04267a3ac5..9ecede0f30 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -526,6 +526,8 @@ struct virtio_net { int postcopy_ufd; int postcopy_listening; + int vduse_ctrl_fd; + int vduse_dev_fd; struct vhost_virtqueue *cvq; From patchwork Tue Jun 6 08:18:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128182 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0F65542C3E; Tue, 6 Jun 2023 10:20:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 201B342D38; Tue, 6 Jun 2023 10:19:52 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 16C8F42D9C for ; Tue, 6 Jun 2023 10:19:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039590; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bCjb9bPrkWjcMRuH9nhbBJ1yUcyb/M8upG8Hsux6BOI=; b=G0d/dce/59hTxGvd9DQqbLilXz5ZzK9JEW0l3KFDC0sHJd4Bylt435rQNtl+BwLjtyrBa/ jkICJQf3qHc9O9lUdTN3eOZx+QLjdaIULAmice07DGoO14QPOqJp7FMavejjvnp2uUAB9L YbPOwac3kpOWI+aCFYbem15llX1TL+E= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-241-5ru44dCHMYShD35044SXYg-1; Tue, 06 Jun 2023 04:19:47 -0400 X-MC-Unique: 5ru44dCHMYShD35044SXYg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0EAE63C0BE3D; Tue, 6 Jun 2023 08:19:47 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 867C340CF8F6; Tue, 6 Jun 2023 08:19:44 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 17/26] vhost: add VDUSE callback for IOTLB miss Date: Tue, 6 Jun 2023 10:18:43 +0200 Message-Id: <20230606081852.71003-18-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch implements the VDUSE callback for IOTLB misses, which is done by using the VDUSE VDUSE_IOTLB_GET_FD ioctl. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index d67818bfb5..f72c7bf6ab 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -13,9 +13,11 @@ #include #include +#include #include +#include "iotlb.h" #include "vduse.h" #include "vhost.h" @@ -40,7 +42,63 @@ (1ULL << VIRTIO_F_IN_ORDER) | \ (1ULL << VIRTIO_F_IOMMU_PLATFORM)) +static int +vduse_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm __rte_unused) +{ + struct vduse_iotlb_entry entry; + uint64_t size, page_size; + struct stat stat; + void *mmap_addr; + int fd, ret; + + entry.start = iova; + entry.last = iova + 1; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_IOTLB_GET_FD, &entry); + if (ret < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get IOTLB entry for 0x%" PRIx64 "\n", + iova); + return -1; + } + + fd = ret; + + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "New IOTLB entry:\n"); + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "\tIOVA: %" PRIx64 " - %" PRIx64 "\n", + (uint64_t)entry.start, (uint64_t)entry.last); + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "\toffset: %" PRIx64 "\n", (uint64_t)entry.offset); + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "\tfd: %d\n", fd); + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "\tperm: %x\n", entry.perm); + + size = entry.last - entry.start + 1; + mmap_addr = mmap(0, size + entry.offset, entry.perm, MAP_SHARED, fd, 0); + if (!mmap_addr) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to mmap IOTLB entry for 0x%" PRIx64 "\n", iova); + ret = -1; + goto close_fd; + } + + ret = fstat(fd, &stat); + if (ret < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get page size.\n"); + munmap(mmap_addr, entry.offset + size); + goto close_fd; + } + page_size = (uint64_t)stat.st_blksize; + + vhost_user_iotlb_cache_insert(dev, entry.start, (uint64_t)(uintptr_t)mmap_addr, + entry.offset, size, page_size, entry.perm); + + ret = 0; +close_fd: + close(fd); + + return ret; +} + static struct vhost_backend_ops vduse_backend_ops = { + .iotlb_miss = vduse_iotlb_miss, }; int From patchwork Tue Jun 6 08:18:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128183 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 132F642C3E; Tue, 6 Jun 2023 10:21:01 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3CED142D31; Tue, 6 Jun 2023 10:19:55 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id B6D9D40697 for ; Tue, 6 Jun 2023 10:19:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039593; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KaBO5Glxw4sHd0jMxJSKcFBrQo1PSSFcgrz7LOJWFWo=; b=dYxwNJjvtGxaQOrqwpLztjUczE4KtxCUXwWvMzqOm3qLYbXtFVa+ZyQHsFWPNimoaBHoNf Ncm4EE4AB0TKEyiViLrSQ31VEQgJD/8mSdd8SgQM8C235oGidWNE+7NjQwp4dUbUJh/Cas H+O9SyzaVAscKSyyaU8kxxXC+fTJM3U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-90-pT2z2hY7N5qAJ-3ubMci9Q-1; Tue, 06 Jun 2023 04:19:50 -0400 X-MC-Unique: pT2z2hY7N5qAJ-3ubMci9Q-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CDBF385A5B5; Tue, 6 Jun 2023 08:19:49 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 535F040CF8F6; Tue, 6 Jun 2023 08:19:47 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 18/26] vhost: add VDUSE callback for IOTLB entry removal Date: Tue, 6 Jun 2023 10:18:44 +0200 Message-Id: <20230606081852.71003-19-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch implements the VDUSE callback for IOTLB entry removal, where it unmaps the pages from the invalidated IOTLB entry. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index f72c7bf6ab..58c1b384a8 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -42,6 +42,12 @@ (1ULL << VIRTIO_F_IN_ORDER) | \ (1ULL << VIRTIO_F_IOMMU_PLATFORM)) +static void +vduse_iotlb_remove_notify(uint64_t addr, uint64_t offset, uint64_t size) +{ + munmap((void *)(uintptr_t)addr, offset + size); +} + static int vduse_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm __rte_unused) { @@ -99,6 +105,7 @@ vduse_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm __rte_unuse static struct vhost_backend_ops vduse_backend_ops = { .iotlb_miss = vduse_iotlb_miss, + .iotlb_remove_notify = vduse_iotlb_remove_notify, }; int From patchwork Tue Jun 6 08:18:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128184 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1BD2F42C3E; Tue, 6 Jun 2023 10:21:07 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3EE5B42D93; Tue, 6 Jun 2023 10:19:58 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 990BC42DA5 for ; Tue, 6 Jun 2023 10:19:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039596; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aYGAMnOnshRq1wFFVMK1JfUyFWNr5tDK5GwYNj+Lrw8=; b=VW+E3vCxCOJeW0hvk3CnVEZwz89cIXFEbmnU7mW38F9zhVVupkKcwc2OihMD9iaaeU0t23 Pd5W9OQ1pdMymHMQz2ONqxQTDj6GmdTmxYXnmOkCOT2I0prYAcyxt95wPqG7IDSkyXe2an pEjJt1Lo5Jrr0X1smJFZZXyDX9ObQt8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-568-8i4Iaij9NmWnca_9zZ1wNA-1; Tue, 06 Jun 2023 04:19:53 -0400 X-MC-Unique: 8i4Iaij9NmWnca_9zZ1wNA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A3D693C0BE3F; Tue, 6 Jun 2023 08:19:52 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 299B340CF8F7; Tue, 6 Jun 2023 08:19:50 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 19/26] vhost: add VDUSE callback for IRQ injection Date: Tue, 6 Jun 2023 10:18:45 +0200 Message-Id: <20230606081852.71003-20-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch implements the VDUSE callback for kicking virtqueues. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 58c1b384a8..d39e39b9dc 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -42,6 +42,12 @@ (1ULL << VIRTIO_F_IN_ORDER) | \ (1ULL << VIRTIO_F_IOMMU_PLATFORM)) +static int +vduse_inject_irq(struct virtio_net *dev, struct vhost_virtqueue *vq) +{ + return ioctl(dev->vduse_dev_fd, VDUSE_VQ_INJECT_IRQ, &vq->index); +} + static void vduse_iotlb_remove_notify(uint64_t addr, uint64_t offset, uint64_t size) { @@ -106,6 +112,7 @@ vduse_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm __rte_unuse static struct vhost_backend_ops vduse_backend_ops = { .iotlb_miss = vduse_iotlb_miss, .iotlb_remove_notify = vduse_iotlb_remove_notify, + .inject_irq = vduse_inject_irq, }; int From patchwork Tue Jun 6 08:18:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128185 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D65E42C3E; Tue, 6 Jun 2023 10:21:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 62F6342D44; Tue, 6 Jun 2023 10:20:03 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 5807542DAA for ; Tue, 6 Jun 2023 10:20:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039601; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/NaxkaDcGVWBez8isP823L/8Qgqwaypg58E+DhHMgoE=; b=FbMtOdbw83zv3vT5TeqLlW9ViYZ7nbtrLXJplLicsEQT0OaeHpwjRL25/pMk/G/uCw0LVz cB35M3RV+LXdlURutHN1WgtDM+qT+IiTvzJzSS4UvyI2njnsMQx/P+zauASEBvvW/BRmpz C50y/p3ubYKEZZo/7FN7oFU+yG2374I= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-638-IA_CEAlqNDOEe_-GtGaQmg-1; Tue, 06 Jun 2023 04:19:56 -0400 X-MC-Unique: IA_CEAlqNDOEe_-GtGaQmg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C32883825BB4; Tue, 6 Jun 2023 08:19:55 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0034F40CF8F6; Tue, 6 Jun 2023 08:19:52 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 20/26] vhost: add VDUSE events handler Date: Tue, 6 Jun 2023 10:18:46 +0200 Message-Id: <20230606081852.71003-21-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch makes use of Vhost lib's FD manager to install a handler for VDUSE events occurring on the VDUSE device FD. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 101 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index d39e39b9dc..92c515cff2 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -17,6 +17,7 @@ #include +#include "fd_man.h" #include "iotlb.h" #include "vduse.h" #include "vhost.h" @@ -42,6 +43,31 @@ (1ULL << VIRTIO_F_IN_ORDER) | \ (1ULL << VIRTIO_F_IOMMU_PLATFORM)) +struct vduse { + struct fdset fdset; +}; + +static struct vduse vduse = { + .fdset = { + .fd = { [0 ... MAX_FDS - 1] = {-1, NULL, NULL, NULL, 0} }, + .fd_mutex = PTHREAD_MUTEX_INITIALIZER, + .fd_pooling_mutex = PTHREAD_MUTEX_INITIALIZER, + .num = 0 + }, +}; + +static bool vduse_events_thread; + +static const char * const vduse_reqs_str[] = { + "VDUSE_GET_VQ_STATE", + "VDUSE_SET_STATUS", + "VDUSE_UPDATE_IOTLB", +}; + +#define vduse_req_id_to_str(id) \ + (id < RTE_DIM(vduse_reqs_str) ? \ + vduse_reqs_str[id] : "Unknown") + static int vduse_inject_irq(struct virtio_net *dev, struct vhost_virtqueue *vq) { @@ -115,16 +141,80 @@ static struct vhost_backend_ops vduse_backend_ops = { .inject_irq = vduse_inject_irq, }; +static void +vduse_events_handler(int fd, void *arg, int *remove __rte_unused) +{ + struct virtio_net *dev = arg; + struct vduse_dev_request req; + struct vduse_dev_response resp; + int ret; + + memset(&resp, 0, sizeof(resp)); + + ret = read(fd, &req, sizeof(req)); + if (ret < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to read request: %s\n", + strerror(errno)); + return; + } else if (ret < (int)sizeof(req)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Incomplete to read request %d\n", ret); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, INFO, "New request: %s (%u)\n", + vduse_req_id_to_str(req.type), req.type); + + switch (req.type) { + default: + resp.result = VDUSE_REQ_RESULT_FAILED; + break; + } + + resp.request_id = req.request_id; + + ret = write(dev->vduse_dev_fd, &resp, sizeof(resp)); + if (ret != sizeof(resp)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to write response %s\n", + strerror(errno)); + return; + } + VHOST_LOG_CONFIG(dev->ifname, INFO, "Request %s (%u) handled successfully\n", + vduse_req_id_to_str(req.type), req.type); +} + int vduse_device_create(const char *path) { int control_fd, dev_fd, vid, ret; + pthread_t fdset_tid; uint32_t i; struct virtio_net *dev; uint64_t ver = VHOST_VDUSE_API_VERSION; struct vduse_dev_config *dev_config = NULL; const char *name = path + strlen("/dev/vduse/"); + /* If first device, create events dispatcher thread */ + if (vduse_events_thread == false) { + /** + * create a pipe which will be waited by poll and notified to + * rebuild the wait list of poll. + */ + if (fdset_pipe_init(&vduse.fdset) < 0) { + VHOST_LOG_CONFIG(path, ERR, "failed to create pipe for vduse fdset\n"); + return -1; + } + + ret = rte_ctrl_thread_create(&fdset_tid, "vduse-events", NULL, + fdset_event_dispatch, &vduse.fdset); + if (ret != 0) { + VHOST_LOG_CONFIG(path, ERR, "failed to create vduse fdset handling thread\n"); + fdset_pipe_uninit(&vduse.fdset); + return -1; + } + + vduse_events_thread = true; + } + control_fd = open(VDUSE_CTRL_PATH, O_RDWR); if (control_fd < 0) { VHOST_LOG_CONFIG(name, ERR, "Failed to open %s: %s\n", @@ -215,6 +305,14 @@ vduse_device_create(const char *path) } } + ret = fdset_add(&vduse.fdset, dev->vduse_dev_fd, vduse_events_handler, NULL, dev); + if (ret) { + VHOST_LOG_CONFIG(name, ERR, "Failed to add fd %d to vduse fdset\n", + dev->vduse_dev_fd); + goto out_dev_destroy; + } + fdset_pipe_notify(&vduse.fdset); + free(dev_config); return 0; @@ -253,6 +351,9 @@ vduse_device_destroy(const char *path) if (vid == RTE_MAX_VHOST_DEVICE) return -1; + fdset_del(&vduse.fdset, dev->vduse_dev_fd); + fdset_pipe_notify(&vduse.fdset); + if (dev->vduse_dev_fd >= 0) { close(dev->vduse_dev_fd); dev->vduse_dev_fd = -1; From patchwork Tue Jun 6 08:18:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128186 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 230ED42C3E; Tue, 6 Jun 2023 10:21:20 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8C3CD42DAD; Tue, 6 Jun 2023 10:20:04 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 8635D42DAB for ; Tue, 6 Jun 2023 10:20:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039602; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/mMbfPtlgp+8d/URc01bG4SThKEEoWRRLfDp81aBQ+U=; b=efFoPCYniGHLqvHiA85R62GV6Y8MCVXKRZdb9B1NlxBdKCwp8/vhua+N6O+CMvs6o2QjL8 SJ0CpGjDHlqH/VCWyvss8+Hh3qWWHg9LqywL6c3KMH0jG4rCOe5zcLcplIVHHknipRXiXV RuWqCAzsAIdyXc85aK/Vww1mXwCPfWg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-249-_fY5PEWfNRuX7AxgzEQHbA-1; Tue, 06 Jun 2023 04:19:59 -0400 X-MC-Unique: _fY5PEWfNRuX7AxgzEQHbA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 90B14811E86; Tue, 6 Jun 2023 08:19:58 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 15E5A40CF8F6; Tue, 6 Jun 2023 08:19:55 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 21/26] vhost: add support for virtqueue state get event Date: Tue, 6 Jun 2023 10:18:47 +0200 Message-Id: <20230606081852.71003-22-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds support for VDUSE_GET_VQ_STATE event handling, which consists in providing the backend last available index for the specified virtqueue. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 92c515cff2..7e36c50b6c 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -147,6 +147,7 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) struct virtio_net *dev = arg; struct vduse_dev_request req; struct vduse_dev_response resp; + struct vhost_virtqueue *vq; int ret; memset(&resp, 0, sizeof(resp)); @@ -165,6 +166,13 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) vduse_req_id_to_str(req.type), req.type); switch (req.type) { + case VDUSE_GET_VQ_STATE: + vq = dev->virtqueue[req.vq_state.index]; + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tvq index: %u, avail_index: %u\n", + req.vq_state.index, vq->last_avail_idx); + resp.vq_state.split.avail_index = vq->last_avail_idx; + resp.result = VDUSE_REQ_RESULT_OK; + break; default: resp.result = VDUSE_REQ_RESULT_FAILED; break; From patchwork Tue Jun 6 08:18:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128187 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1B3E642C3E; Tue, 6 Jun 2023 10:21:28 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F3D5342C4D; Tue, 6 Jun 2023 10:20:07 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id AF3B342DBC for ; Tue, 6 Jun 2023 10:20:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039605; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+AtyqhGEYrXQHh7YOVU0PAAJ8QpP4RzlHjniUfhJWNo=; b=N/59K5BytvjlEOUxKyawXxL/lgtln12wnlYFyJxxo0IIMeV1JtNBE/JKkS1cmM00IFJ7Qu Ph1g9aAU/DXHCxK8LPVJJhF+zJs6fjhRChumu6x/2+HsRFZvKZ5G6VQYxlL8K6ouDCC29P LGX3/ZMcbg5UwwhBlNX9JYQY4rFYEm0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-531-ppfujOXJNz6iOyC0vrknDA-1; Tue, 06 Jun 2023 04:20:02 -0400 X-MC-Unique: ppfujOXJNz6iOyC0vrknDA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5C5C8802E58; Tue, 6 Jun 2023 08:20:01 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6E1740CF8F6; Tue, 6 Jun 2023 08:19:58 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 22/26] vhost: add support for VDUSE status set event Date: Tue, 6 Jun 2023 10:18:48 +0200 Message-Id: <20230606081852.71003-23-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds support for VDUSE_SET_STATUS event handling, which consists in updating the Virtio device status set by the Virtio driver. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 7e36c50b6c..3bf65d4b8b 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -173,6 +173,12 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) resp.vq_state.split.avail_index = vq->last_avail_idx; resp.result = VDUSE_REQ_RESULT_OK; break; + case VDUSE_SET_STATUS: + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", + req.s.status); + dev->status = req.s.status; + resp.result = VDUSE_REQ_RESULT_OK; + break; default: resp.result = VDUSE_REQ_RESULT_FAILED; break; From patchwork Tue Jun 6 08:18:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128188 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A80D142C3E; Tue, 6 Jun 2023 10:21:32 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1474842DC2; Tue, 6 Jun 2023 10:20:09 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id E656040A84 for ; Tue, 6 Jun 2023 10:20:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039606; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yyWxx4cOr/d1MsHouVd421UBe4fkggg/wLUy230+ehE=; b=FkXbKTL6rg6bJa3xp2kdekQbIm5Qywvc5r5hLWbjw3mEYLNkBFn7mQVXbQIfysUBdp9m9a U9uPxCZXmhFAirAK1jjnOg0h1JSpUtBMI8K7/XoZuHvxW/+EZPmuvDypISvCc0keiprTDm sTxp9b5iVjr1DRTgF0IJb7iygL2wJB4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-32-XUcrfOgnNXWlm-qFG4saWA-1; Tue, 06 Jun 2023 04:20:04 -0400 X-MC-Unique: XUcrfOgnNXWlm-qFG4saWA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3124E3825BC1; Tue, 6 Jun 2023 08:20:04 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id AC68440CF8F6; Tue, 6 Jun 2023 08:20:01 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 23/26] vhost: add support for VDUSE IOTLB update event Date: Tue, 6 Jun 2023 10:18:49 +0200 Message-Id: <20230606081852.71003-24-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds support for VDUSE_UPDATE_IOTLB event handling, which consists in invaliding IOTLB entries for the range specified in the request. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 3bf65d4b8b..110654ec68 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -179,6 +179,13 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) dev->status = req.s.status; resp.result = VDUSE_REQ_RESULT_OK; break; + case VDUSE_UPDATE_IOTLB: + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tIOVA range: %" PRIx64 " - %" PRIx64 "\n", + (uint64_t)req.iova.start, (uint64_t)req.iova.last); + vhost_user_iotlb_cache_remove(dev, req.iova.start, + req.iova.last - req.iova.start + 1); + resp.result = VDUSE_REQ_RESULT_OK; + break; default: resp.result = VDUSE_REQ_RESULT_FAILED; break; From patchwork Tue Jun 6 08:18:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128189 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 394D742C3E; Tue, 6 Jun 2023 10:21:38 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1EDFE42DB8; Tue, 6 Jun 2023 10:20:11 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 2DF8242DC8 for ; Tue, 6 Jun 2023 10:20:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039608; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KTps7kn3VgS+82NSzdeTAojFnN3TFT0QtD8ACEztNSY=; b=Pkio9lZqSu6Hs8b6NWrdLuMrz8L3O8kjAbd18t2yezk1u7pymNNjNfOmG6vHHACoRgJ92A o7zs4Y6fW90D/sjvd+13cumq+prF1rylo0h4cqVxur7TYNbLNJYgwmNRisc/zbNs6yYwZu txi+Sxh8YoYiSnkabuIvKW4VBUrT/ig= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-304-RO984r_VMvS_9BVLCerzQw-1; Tue, 06 Jun 2023 04:20:07 -0400 X-MC-Unique: RO984r_VMvS_9BVLCerzQw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0CCB9811E88; Tue, 6 Jun 2023 08:20:07 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 82B7540CF8F8; Tue, 6 Jun 2023 08:20:04 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 24/26] vhost: add VDUSE device startup Date: Tue, 6 Jun 2023 10:18:50 +0200 Message-Id: <20230606081852.71003-25-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds the device and its virtqueues initialization once the Virtio driver has set the DRIVER_OK in the Virtio status register. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 110654ec68..ff4c54c321 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -141,6 +141,128 @@ static struct vhost_backend_ops vduse_backend_ops = { .inject_irq = vduse_inject_irq, }; +static void +vduse_vring_setup(struct virtio_net *dev, unsigned int index) +{ + struct vhost_virtqueue *vq = dev->virtqueue[index]; + struct vhost_vring_addr *ra = &vq->ring_addrs; + struct vduse_vq_info vq_info; + struct vduse_vq_eventfd vq_efd; + int ret; + + vq_info.index = index; + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_GET_INFO, &vq_info); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get VQ %u info: %s\n", + index, strerror(errno)); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, INFO, "VQ %u info:\n", index); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnum: %u\n", vq_info.num); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdesc_addr: %llx\n", vq_info.desc_addr); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdriver_addr: %llx\n", vq_info.driver_addr); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdevice_addr: %llx\n", vq_info.device_addr); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tavail_idx: %u\n", vq_info.split.avail_index); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tready: %u\n", vq_info.ready); + + vq->last_avail_idx = vq_info.split.avail_index; + vq->size = vq_info.num; + vq->ready = true; + vq->enabled = vq_info.ready; + ra->desc_user_addr = vq_info.desc_addr; + ra->avail_user_addr = vq_info.driver_addr; + ra->used_user_addr = vq_info.device_addr; + + vq->kickfd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (vq->kickfd < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to init kickfd for VQ %u: %s\n", + index, strerror(errno)); + vq->kickfd = VIRTIO_INVALID_EVENTFD; + return; + } + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tkick fd: %d\n", vq->kickfd); + + vq->shadow_used_split = rte_malloc_socket(NULL, + vq->size * sizeof(struct vring_used_elem), + RTE_CACHE_LINE_SIZE, 0); + vq->batch_copy_elems = rte_malloc_socket(NULL, + vq->size * sizeof(struct batch_copy_elem), + RTE_CACHE_LINE_SIZE, 0); + + vhost_user_iotlb_rd_lock(vq); + if (vring_translate(dev, vq)) + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to translate vring %d addresses\n", + index); + + if (vhost_enable_guest_notification(dev, vq, 0)) + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to disable guest notifications on vring %d\n", + index); + vhost_user_iotlb_rd_unlock(vq); + + vq_efd.index = index; + vq_efd.fd = vq->kickfd; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to setup kickfd for VQ %u: %s\n", + index, strerror(errno)); + close(vq->kickfd); + vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + return; + } +} + +static void +vduse_device_start(struct virtio_net *dev) +{ + unsigned int i, ret; + + VHOST_LOG_CONFIG(dev->ifname, INFO, "Starting device...\n"); + + dev->notify_ops = vhost_driver_callback_get(dev->ifname); + if (!dev->notify_ops) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to get callback ops for driver\n"); + return; + } + + ret = ioctl(dev->vduse_dev_fd, VDUSE_DEV_GET_FEATURES, &dev->features); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get features: %s\n", + strerror(errno)); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, INFO, "Negotiated Virtio features: 0x%" PRIx64 "\n", + dev->features); + + if (dev->features & + ((1ULL << VIRTIO_NET_F_MRG_RXBUF) | + (1ULL << VIRTIO_F_VERSION_1) | + (1ULL << VIRTIO_F_RING_PACKED))) { + dev->vhost_hlen = sizeof(struct virtio_net_hdr_mrg_rxbuf); + } else { + dev->vhost_hlen = sizeof(struct virtio_net_hdr); + } + + for (i = 0; i < dev->nr_vring; i++) + vduse_vring_setup(dev, i); + + dev->flags |= VIRTIO_DEV_READY; + + if (dev->notify_ops->new_device(dev->vid) == 0) + dev->flags |= VIRTIO_DEV_RUNNING; + + for (i = 0; i < dev->nr_vring; i++) { + struct vhost_virtqueue *vq = dev->virtqueue[i]; + + if (dev->notify_ops->vring_state_changed) + dev->notify_ops->vring_state_changed(dev->vid, i, vq->enabled); + } +} + static void vduse_events_handler(int fd, void *arg, int *remove __rte_unused) { @@ -199,6 +321,10 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) strerror(errno)); return; } + + if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) + vduse_device_start(dev); + VHOST_LOG_CONFIG(dev->ifname, INFO, "Request %s (%u) handled successfully\n", vduse_req_id_to_str(req.type), req.type); } From patchwork Tue Jun 6 08:18:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128190 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F39C142C3E; Tue, 6 Jun 2023 10:21:43 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 37BB242D12; Tue, 6 Jun 2023 10:20:15 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id EB99842D12 for ; Tue, 6 Jun 2023 10:20:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9wSFaqy1dpAkxCn6+WZ5vv73qTwFfrYbgnDwd4Ikh54=; b=Z8sKtTpjxKE8MmqcMIM7J/Nu4M4h9Uj46J6P92O1CPDQciCdGJ4YCyjIw79jWBl9zWijQp qeuU7YFpTZvWpkWERU6ebudIbpaJiW8YDO+PLVevzKFCnmLnJFbH4852xfLW/aqZkxXXDz KB6n51LhDVMzvKZH4aVqarYGu6MOFao= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-653-xedKmvSHOL20Zeny_GTw7g-1; Tue, 06 Jun 2023 04:20:10 -0400 X-MC-Unique: xedKmvSHOL20Zeny_GTw7g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C88E6801182; Tue, 6 Jun 2023 08:20:09 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4E49340CF8F6; Tue, 6 Jun 2023 08:20:07 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 25/26] vhost: add multiqueue support to VDUSE Date: Tue, 6 Jun 2023 10:18:51 +0200 Message-Id: <20230606081852.71003-26-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch enables control queue support in order to support multiqueue. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 83 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 76 insertions(+), 7 deletions(-) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index ff4c54c321..d3759077ff 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -21,6 +21,7 @@ #include "iotlb.h" #include "vduse.h" #include "vhost.h" +#include "virtio_net_ctrl.h" #define VHOST_VDUSE_API_VERSION 0 #define VDUSE_CTRL_PATH "/dev/vduse/control" @@ -41,7 +42,9 @@ (1ULL << VIRTIO_NET_F_GUEST_ECN) | \ (1ULL << VIRTIO_RING_F_INDIRECT_DESC) | \ (1ULL << VIRTIO_F_IN_ORDER) | \ - (1ULL << VIRTIO_F_IOMMU_PLATFORM)) + (1ULL << VIRTIO_F_IOMMU_PLATFORM) | \ + (1ULL << VIRTIO_NET_F_CTRL_VQ) | \ + (1ULL << VIRTIO_NET_F_MQ)) struct vduse { struct fdset fdset; @@ -141,6 +144,25 @@ static struct vhost_backend_ops vduse_backend_ops = { .inject_irq = vduse_inject_irq, }; +static void +vduse_control_queue_event(int fd, void *arg, int *remove __rte_unused) +{ + struct virtio_net *dev = arg; + uint64_t buf; + int ret; + + ret = read(fd, &buf, sizeof(buf)); + if (ret < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to read control queue event: %s\n", + strerror(errno)); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "Control queue kicked\n"); + if (virtio_net_ctrl_handle(dev)) + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to handle ctrl request\n"); +} + static void vduse_vring_setup(struct virtio_net *dev, unsigned int index) { @@ -212,6 +234,22 @@ vduse_vring_setup(struct virtio_net *dev, unsigned int index) vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; return; } + + if (vq == dev->cvq) { + ret = fdset_add(&vduse.fdset, vq->kickfd, vduse_control_queue_event, NULL, dev); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to setup kickfd handler for VQ %u: %s\n", + index, strerror(errno)); + vq_efd.fd = VDUSE_EVENTFD_DEASSIGN; + ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); + close(vq->kickfd); + vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + } + fdset_pipe_notify(&vduse.fdset); + vhost_enable_guest_notification(dev, vq, 1); + VHOST_LOG_CONFIG(dev->ifname, INFO, "Ctrl queue event handler installed\n"); + } } static void @@ -258,6 +296,9 @@ vduse_device_start(struct virtio_net *dev) for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; + if (vq == dev->cvq) + continue; + if (dev->notify_ops->vring_state_changed) dev->notify_ops->vring_state_changed(dev->vid, i, vq->enabled); } @@ -334,9 +375,11 @@ vduse_device_create(const char *path) { int control_fd, dev_fd, vid, ret; pthread_t fdset_tid; - uint32_t i; + uint32_t i, max_queue_pairs, total_queues; struct virtio_net *dev; + struct virtio_net_config vnet_config = { 0 }; uint64_t ver = VHOST_VDUSE_API_VERSION; + uint64_t features = VDUSE_NET_SUPPORTED_FEATURES; struct vduse_dev_config *dev_config = NULL; const char *name = path + strlen("/dev/vduse/"); @@ -376,22 +419,39 @@ vduse_device_create(const char *path) goto out_ctrl_close; } - dev_config = malloc(offsetof(struct vduse_dev_config, config)); + dev_config = malloc(offsetof(struct vduse_dev_config, config) + + sizeof(vnet_config)); if (!dev_config) { VHOST_LOG_CONFIG(name, ERR, "Failed to allocate VDUSE config\n"); ret = -1; goto out_ctrl_close; } + ret = rte_vhost_driver_get_queue_num(path, &max_queue_pairs); + if (ret < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to get max queue pairs\n"); + goto out_free; + } + + VHOST_LOG_CONFIG(path, INFO, "VDUSE max queue pairs: %u\n", max_queue_pairs); + total_queues = max_queue_pairs * 2; + + if (max_queue_pairs == 1) + features &= ~(RTE_BIT64(VIRTIO_NET_F_CTRL_VQ) | RTE_BIT64(VIRTIO_NET_F_MQ)); + else + total_queues += 1; /* Includes ctrl queue */ + + vnet_config.max_virtqueue_pairs = max_queue_pairs; memset(dev_config, 0, sizeof(struct vduse_dev_config)); strncpy(dev_config->name, name, VDUSE_NAME_MAX - 1); dev_config->device_id = VIRTIO_ID_NET; dev_config->vendor_id = 0; - dev_config->features = VDUSE_NET_SUPPORTED_FEATURES; - dev_config->vq_num = 2; + dev_config->features = features; + dev_config->vq_num = total_queues; dev_config->vq_align = sysconf(_SC_PAGE_SIZE); - dev_config->config_size = 0; + dev_config->config_size = sizeof(struct virtio_net_config); + memcpy(dev_config->config, &vnet_config, sizeof(vnet_config)); ret = ioctl(control_fd, VDUSE_CREATE_DEV, dev_config); if (ret < 0) { @@ -433,7 +493,7 @@ vduse_device_create(const char *path) dev->vduse_dev_fd = dev_fd; vhost_setup_virtio_net(dev->vid, true, true, true, true); - for (i = 0; i < 2; i++) { + for (i = 0; i < total_queues; i++) { struct vduse_vq_config vq_cfg = { 0 }; ret = alloc_vring_queue(dev, i); @@ -452,6 +512,8 @@ vduse_device_create(const char *path) } } + dev->cvq = dev->virtqueue[max_queue_pairs * 2]; + ret = fdset_add(&vduse.fdset, dev->vduse_dev_fd, vduse_events_handler, NULL, dev); if (ret) { VHOST_LOG_CONFIG(name, ERR, "Failed to add fd %d to vduse fdset\n", @@ -498,6 +560,13 @@ vduse_device_destroy(const char *path) if (vid == RTE_MAX_VHOST_DEVICE) return -1; + if (dev->cvq && dev->cvq->kickfd >= 0) { + fdset_del(&vduse.fdset, dev->cvq->kickfd); + fdset_pipe_notify(&vduse.fdset); + close(dev->cvq->kickfd); + dev->cvq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + } + fdset_del(&vduse.fdset, dev->vduse_dev_fd); fdset_pipe_notify(&vduse.fdset); From patchwork Tue Jun 6 08:18:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128191 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2A77B42C3E; Tue, 6 Jun 2023 10:21:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5892D42C76; Tue, 6 Jun 2023 10:20:18 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id C9D2B41141 for ; Tue, 6 Jun 2023 10:20:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039616; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IIyMCcgg6MKZkCLy0ze0X6ty1Yk1sayPpDrb/zVwqQw=; b=G+Z7bqKhUrtMyh8UQ9pT4I+S+A/q1WG+37iZFEPNELkmEil2n60m0+uohw8IB3kWOy322+ VJ20U5IapecLKESRtC2OHGxepJqvg6ZrygWbRHgbF9OmEWKsw5MHRXAjKdBrvrn/SiqkHe Q1tXGsLNwqVcnflJw1Eu8gYrEWPPFR4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-470-ynkhvemNNpu3CZeDebpe0w-1; Tue, 06 Jun 2023 04:20:13 -0400 X-MC-Unique: ynkhvemNNpu3CZeDebpe0w-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 95A42811E86; Tue, 6 Jun 2023 08:20:12 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1A21840CF8F6; Tue, 6 Jun 2023 08:20:09 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 26/26] vhost: add VDUSE device stop Date: Tue, 6 Jun 2023 10:18:52 +0200 Message-Id: <20230606081852.71003-27-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds VDUSE device stop and cleanup of its virtqueues. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- doc/guides/rel_notes/release_23_07.rst | 7 +++ lib/vhost/vduse.c | 72 +++++++++++++++++++++++--- 2 files changed, 71 insertions(+), 8 deletions(-) diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index 7034fb664c..0f44c859da 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -67,6 +67,13 @@ New Features Introduced ``rte_vhost_driver_set_max_queue_num()`` to be able to limit the maximum number of supported queue pairs, required for VDUSE support. +* **Added VDUSE support into Vhost library.** + + VDUSE aims at implementing vDPA devices in userspace. It can be used as an + alternative to Vhost-user when using Vhost-vDPA, but also enable providing a + virtio-net netdev to the host when using Virtio-vDPA driver. A limitation in + this release is the lack of reconnection support. + Removed Items ------------- diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index d3759077ff..a509daf80c 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -252,6 +252,44 @@ vduse_vring_setup(struct virtio_net *dev, unsigned int index) } } +static void +vduse_vring_cleanup(struct virtio_net *dev, unsigned int index) +{ + struct vhost_virtqueue *vq = dev->virtqueue[index]; + struct vduse_vq_eventfd vq_efd; + int ret; + + if (vq == dev->cvq && vq->kickfd >= 0) { + fdset_del(&vduse.fdset, vq->kickfd); + fdset_pipe_notify(&vduse.fdset); + } + + vq_efd.index = index; + vq_efd.fd = VDUSE_EVENTFD_DEASSIGN; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); + if (ret) + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to cleanup kickfd for VQ %u: %s\n", + index, strerror(errno)); + + close(vq->kickfd); + vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + + vring_invalidate(dev, vq); + + rte_free(vq->batch_copy_elems); + vq->batch_copy_elems = NULL; + + rte_free(vq->shadow_used_split); + vq->shadow_used_split = NULL; + + vq->enabled = false; + vq->ready = false; + vq->size = 0; + vq->last_used_idx = 0; + vq->last_avail_idx = 0; +} + static void vduse_device_start(struct virtio_net *dev) { @@ -304,6 +342,23 @@ vduse_device_start(struct virtio_net *dev) } } +static void +vduse_device_stop(struct virtio_net *dev) +{ + unsigned int i; + + VHOST_LOG_CONFIG(dev->ifname, INFO, "Stopping device...\n"); + + vhost_destroy_device_notify(dev); + + dev->flags &= ~VIRTIO_DEV_READY; + + for (i = 0; i < dev->nr_vring; i++) + vduse_vring_cleanup(dev, i); + + vhost_user_iotlb_flush_all(dev); +} + static void vduse_events_handler(int fd, void *arg, int *remove __rte_unused) { @@ -311,6 +366,7 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) struct vduse_dev_request req; struct vduse_dev_response resp; struct vhost_virtqueue *vq; + uint8_t old_status = dev->status; int ret; memset(&resp, 0, sizeof(resp)); @@ -339,6 +395,7 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) case VDUSE_SET_STATUS: VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", req.s.status); + old_status = dev->status; dev->status = req.s.status; resp.result = VDUSE_REQ_RESULT_OK; break; @@ -363,8 +420,12 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) return; } - if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) - vduse_device_start(dev); + if ((old_status ^ dev->status) & VIRTIO_DEVICE_STATUS_DRIVER_OK) { + if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) + vduse_device_start(dev); + else + vduse_device_stop(dev); + } VHOST_LOG_CONFIG(dev->ifname, INFO, "Request %s (%u) handled successfully\n", vduse_req_id_to_str(req.type), req.type); @@ -560,12 +621,7 @@ vduse_device_destroy(const char *path) if (vid == RTE_MAX_VHOST_DEVICE) return -1; - if (dev->cvq && dev->cvq->kickfd >= 0) { - fdset_del(&vduse.fdset, dev->cvq->kickfd); - fdset_pipe_notify(&vduse.fdset); - close(dev->cvq->kickfd); - dev->cvq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; - } + vduse_device_stop(dev); fdset_del(&vduse.fdset, dev->vduse_dev_fd); fdset_pipe_notify(&vduse.fdset);