From patchwork Fri Apr 20 08:39:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 38608 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 06CDA7CFF; Fri, 20 Apr 2018 10:39:40 +0200 (CEST) Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id 09B187CEF for ; Fri, 20 Apr 2018 10:39:38 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 80098406F97B; Fri, 20 Apr 2018 08:39:37 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-43.ams2.redhat.com [10.36.112.43]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A93D10B0080; Fri, 20 Apr 2018 08:39:34 +0000 (UTC) From: Maxime Coquelin To: jianfeng.tan@intel.com, tiwei.bie@intel.com, dev@dpdk.org Cc: Maxime Coquelin Date: Fri, 20 Apr 2018 10:39:21 +0200 Message-Id: <20180420083921.28025-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 20 Apr 2018 08:39:37 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 20 Apr 2018 08:39:37 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'maxime.coquelin@redhat.com' RCPT:'' Subject: [dpdk-dev] [PATCH] Revert "vhost: avoid concurrency when logging dirty pages" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This reverts commit 394313fff39d0f994325c47f7eab39daf5dc9e11. While the patch did solve concurrency issue, it induces more pages copies as some clean pages are marked as dirty for performance reasons. Moreover, as there is no more contention doing the logging, the rate of packets than can be processed is higher, leading to even more pages to be dirtied. It has been reported that with more than one queue pair, and with a relatively low packet rate (1Mpps), the live migration never converges until the flow is stopped. While a better solution is found, it is better to reset to the old behaviour, i.e. using atomic operation for dirty pages logging. Signed-off-by: Maxime Coquelin --- lib/librte_vhost/vhost.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h index c9b64461d..4dbed9ad2 100644 --- a/lib/librte_vhost/vhost.h +++ b/lib/librte_vhost/vhost.h @@ -304,14 +304,18 @@ struct virtio_net { #define VHOST_LOG_PAGE 4096 /* - * Mark all pages belonging to the same dirty log bitmap byte - * as dirty. The goal is to avoid concurrency between different - * threads doing atomic read-modify-writes on the same byte. + * Atomically set a bit in memory. */ +static __rte_always_inline void +vhost_set_bit(unsigned int nr, volatile uint8_t *addr) +{ + __sync_fetch_and_or_8(addr, (1U << nr)); +} + static __rte_always_inline void vhost_log_page(uint8_t *log_base, uint64_t page) { - log_base[page / 8] = 0xff; + vhost_set_bit(page % 8, &log_base[page / 8]); } static __rte_always_inline void