From patchwork Fri Jun 18 13:57:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 94503 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2285EA0C46; Fri, 18 Jun 2021 15:57:56 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2A9D5410F3; Fri, 18 Jun 2021 15:57:54 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 3922740142 for ; Fri, 18 Jun 2021 15:57:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624024671; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p5ifyoguWwTJ4/PHtxZC50++jvmsTMeiRvv64l3peaQ=; b=XHc839l/xz6pbKgSuTjhxgO3ottakERhxZwkiJQiUSEW4zl/kbw1HwBLIk4i6KVrPySI2x eoNp5A3BoitRn7qBvrreyAA+WeyjUu51qV+ZueQQaZIp/WZGKHgwZupdhFfmHTYT5KzBod kONKjlqid2d0eqf8iLcQSnJPqWgP4rY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-290-LioqU7UxOg6SZK6249eMZQ-1; Fri, 18 Jun 2021 09:57:47 -0400 X-MC-Unique: LioqU7UxOg6SZK6249eMZQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7ED4C8030D6; Fri, 18 Jun 2021 13:57:46 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.36.110.21]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3CA1D60BE5; Fri, 18 Jun 2021 13:57:45 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, david.marchand@redhat.com, chenbo.xia@intel.com Cc: Maxime Coquelin , stable@dpdk.org Date: Fri, 18 Jun 2021 15:57:32 +0200 Message-Id: <20210618135735.253770-2-maxime.coquelin@redhat.com> In-Reply-To: <20210618135735.253770-1-maxime.coquelin@redhat.com> References: <20210618135735.253770-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v5 1/4] vhost: fix missing memory table NUMA realloc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When the guest allocates virtqueues on a different NUMA node than the one the Vhost metadata are allocated, both the Vhost device struct and the virtqueues struct are reallocated. However, reallocating the Vhost memory table was missing, which likely causes at least one cross-NUMA accesses for every burst of packets. This patch reallocates this table on the same NUMA node as the other metadata. Fixes: 552e8fd3d2b4 ("vhost: simplify memory regions handling") Cc: stable@dpdk.org Reported-by: David Marchand Signed-off-by: Maxime Coquelin --- lib/vhost/vhost_user.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 8f0eba6412..b5a84f3dcd 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -473,8 +473,8 @@ vhost_user_set_vring_num(struct virtio_net **pdev, } /* - * Reallocate virtio_dev and vhost_virtqueue data structure to make them on the - * same numa node as the memory of vring descriptor. + * Reallocate virtio_dev, vhost_virtqueue and related data structures to + * make them on the same numa node as the memory of vring descriptor. */ #ifdef RTE_LIBRTE_VHOST_NUMA static struct virtio_net* @@ -557,6 +557,9 @@ numa_realloc(struct virtio_net *dev, int index) goto out; } if (oldnode != newnode) { + struct rte_vhost_memory *old_mem; + ssize_t mem_size; + VHOST_LOG_CONFIG(INFO, "reallocate dev from %d to %d node\n", oldnode, newnode); @@ -568,6 +571,18 @@ numa_realloc(struct virtio_net *dev, int index) memcpy(dev, old_dev, sizeof(*dev)); rte_free(old_dev); + + mem_size = sizeof(struct rte_vhost_memory) + + sizeof(struct rte_vhost_mem_region) * dev->mem->nregions; + old_mem = dev->mem; + dev->mem = rte_malloc_socket(NULL, mem_size, 0, newnode); + if (!dev->mem) { + dev->mem = old_mem; + goto out; + } + + memcpy(dev->mem, old_mem, mem_size); + rte_free(old_mem); } out: From patchwork Fri Jun 18 13:57:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 94506 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D23D8A0C46; Fri, 18 Jun 2021 15:58:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9BE4D41109; Fri, 18 Jun 2021 15:57:57 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id F13D4410ED for ; Fri, 18 Jun 2021 15:57:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624024673; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8L7NENvF8N4RNbft6LIfuCRbUH/slJJvIDfO/KyhL5Y=; b=YZblCUQuAd5+gowit3du72+LvibcbPsjghv9LOYw6gQMoZRzbvi0OkRn/NnEpSzmoKoCY5 eYoceqD+HedWyqa7mgwpxXZ+wToKONbXClb00UqNO3eAa2I5+8orORtOB6L9am4828QbJV r9cSc4NwduAjfsrHlm1OpGfDux01IJk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-209-59TenZl_NNCHW1KOdSfr2g-1; Fri, 18 Jun 2021 09:57:49 -0400 X-MC-Unique: 59TenZl_NNCHW1KOdSfr2g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0F0321084F5A; Fri, 18 Jun 2021 13:57:48 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.36.110.21]) by smtp.corp.redhat.com (Postfix) with ESMTP id CA20060BE5; Fri, 18 Jun 2021 13:57:46 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, david.marchand@redhat.com, chenbo.xia@intel.com Cc: Maxime Coquelin , stable@dpdk.org Date: Fri, 18 Jun 2021 15:57:33 +0200 Message-Id: <20210618135735.253770-3-maxime.coquelin@redhat.com> In-Reply-To: <20210618135735.253770-1-maxime.coquelin@redhat.com> References: <20210618135735.253770-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v5 2/4] vhost: fix missing guest pages table NUMA realloc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When the guest allocates virtqueues on a different NUMA node than the one the Vhost metadata are allocated, both the Vhost device struct and the virtqueues struct are reallocated. However, reallocating the guest pages table was missing, which likely causes at least one cross-NUMA accesses for every burst of packets. This patch reallocates this table on the same NUMA node as the other metadata. Fixes: e246896178e6 ("vhost: get guest/host physical address mappings") Cc: stable@dpdk.org Signed-off-by: Maxime Coquelin --- lib/vhost/vhost_user.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index b5a84f3dcd..5fb055ea2e 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -558,7 +558,8 @@ numa_realloc(struct virtio_net *dev, int index) } if (oldnode != newnode) { struct rte_vhost_memory *old_mem; - ssize_t mem_size; + struct guest_page *old_gp; + ssize_t mem_size, gp_size; VHOST_LOG_CONFIG(INFO, "reallocate dev from %d to %d node\n", @@ -583,6 +584,17 @@ numa_realloc(struct virtio_net *dev, int index) memcpy(dev->mem, old_mem, mem_size); rte_free(old_mem); + + gp_size = dev->max_guest_pages * sizeof(*dev->guest_pages); + old_gp = dev->guest_pages; + dev->guest_pages = rte_malloc_socket(NULL, gp_size, RTE_CACHE_LINE_SIZE, newnode); + if (!dev->guest_pages) { + dev->guest_pages = old_gp; + goto out; + } + + memcpy(dev->guest_pages, old_gp, gp_size); + rte_free(old_gp); } out: From patchwork Fri Jun 18 13:57:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 94504 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 349B7A0C46; Fri, 18 Jun 2021 15:58:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 54CB1410F8; Fri, 18 Jun 2021 15:57:55 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 9BD7E40142 for ; Fri, 18 Jun 2021 15:57:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624024672; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4nA/YdQzawgs4cnJ0LJABUiVp395zGgXIjBbifv+O5A=; b=cO3Lr1ohcQG5I4FmQ1hRMw+zxQVtUNcw6vdUYjXmVNckV/nBApjhAbAgBt/WmiWweUeflS 6avZQ54onVARRj+fsAFXoKnzrUFYm/LEykBHA1ybAwuwgDyi1MN7/t1h5jQkLOuel4JIdC Uy+Kygy1NnqxN/N5lwWkKTiQokyxyCA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-200-yT_4IoIGMQGmdfAnSUjFUA-1; Fri, 18 Jun 2021 09:57:50 -0400 X-MC-Unique: yT_4IoIGMQGmdfAnSUjFUA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 75488804310; Fri, 18 Jun 2021 13:57:49 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.36.110.21]) by smtp.corp.redhat.com (Postfix) with ESMTP id 63DF560BE5; Fri, 18 Jun 2021 13:57:48 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, david.marchand@redhat.com, chenbo.xia@intel.com Cc: Maxime Coquelin Date: Fri, 18 Jun 2021 15:57:34 +0200 Message-Id: <20210618135735.253770-4-maxime.coquelin@redhat.com> In-Reply-To: <20210618135735.253770-1-maxime.coquelin@redhat.com> References: <20210618135735.253770-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v5 3/4] vhost: fix missing cache logging NUMA realloc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When the guest allocates virtqueues on a different NUMA node than the one the Vhost metadata are allocated, both the Vhost device struct and the virtqueues struct are reallocated. However, reallocating the log cache on the new NUMA node was not done. This patch fixes this by reallocating it if it has been allocated already, which means a live-migration is on-going. Fixes: 1818a63147fb ("vhost: move dirty logging cache out of virtqueue") Signed-off-by: Maxime Coquelin --- lib/vhost/vhost_user.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 5fb055ea2e..82adf80fe5 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -545,6 +545,16 @@ numa_realloc(struct virtio_net *dev, int index) vq->batch_copy_elems = new_batch_copy_elems; } + if (vq->log_cache) { + struct log_cache_entry *log_cache; + + log_cache = rte_realloc_socket(vq->log_cache, + sizeof(struct log_cache_entry) * VHOST_LOG_CACHE_NR, + 0, newnode); + if (log_cache) + vq->log_cache = log_cache; + } + rte_free(old_vq); } From patchwork Fri Jun 18 13:57:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 94505 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 60A61A0C46; Fri, 18 Jun 2021 15:58:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8B8AF41100; Fri, 18 Jun 2021 15:57:56 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id E1786410EC for ; Fri, 18 Jun 2021 15:57:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624024673; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a92PJGDpE6NOX935b5es9O86+faHrPTzj480GRujjQs=; b=JOcI6/6Dg8oUWodJWJWH+DuzFADc+TYwZNbQWRctnwE6VIEZezSdCUxoNtHppjIJI7dkzM 0xsawgGDf/NJ/3jH3v/s/CmOpj2BcoJNj6l+U/koDrzAugeHFRPvqiaZihfkg64O6TvemZ 3Z540euYyPXc8YGY6DGVc1ubZIRcHyk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-226-QZrgGgSxNxO71xAaLqpEOw-1; Fri, 18 Jun 2021 09:57:51 -0400 X-MC-Unique: QZrgGgSxNxO71xAaLqpEOw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 08638824F8D; Fri, 18 Jun 2021 13:57:51 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.36.110.21]) by smtp.corp.redhat.com (Postfix) with ESMTP id C2A1260BE5; Fri, 18 Jun 2021 13:57:49 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, david.marchand@redhat.com, chenbo.xia@intel.com Cc: Maxime Coquelin , stable@dpdk.org Date: Fri, 18 Jun 2021 15:57:35 +0200 Message-Id: <20210618135735.253770-5-maxime.coquelin@redhat.com> In-Reply-To: <20210618135735.253770-1-maxime.coquelin@redhat.com> References: <20210618135735.253770-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH v5 4/4] vhost: fix NUMA reallocation with multiqueue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since the Vhost-user device initialization has been reworked, enabling the application to start using the device as soon as the first queue pair is ready, NUMA reallocation no more happened on queue pairs other than the first one since numa_realloc() was returning early if the device was running. This patch fixes this issue by only preventing the device metadata to be allocated if the device is running. For the virtqueues, a vring state change notification is sent to notify the application of its disablement. Since the callback is supposed to be blocking, it is safe to reallocate it afterwards. Fixes: d0fcc38f5fa4 ("vhost: improve device readiness notifications") Cc: stable@dpdk.org Signed-off-by: Maxime Coquelin --- lib/vhost/vhost_user.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 82adf80fe5..51b96a0716 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -488,12 +488,16 @@ numa_realloc(struct virtio_net *dev, int index) struct batch_copy_elem *new_batch_copy_elems; int ret; - if (dev->flags & VIRTIO_DEV_RUNNING) - return dev; - old_dev = dev; vq = old_vq = dev->virtqueue[index]; + /* + * If VQ is ready, it is too late to reallocate, it certainly already + * happened anyway on VHOST_USER_SET_VRING_ADRR. + */ + if (vq->ready) + return dev; + ret = get_mempolicy(&newnode, NULL, 0, old_vq->desc, MPOL_F_NODE | MPOL_F_ADDR); @@ -558,6 +562,9 @@ numa_realloc(struct virtio_net *dev, int index) rte_free(old_vq); } + if (dev->flags & VIRTIO_DEV_RUNNING) + goto out; + /* check if we need to reallocate dev */ ret = get_mempolicy(&oldnode, NULL, 0, old_dev, MPOL_F_NODE | MPOL_F_ADDR);