From patchwork Mon Mar 4 10:35:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 137879 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4D91A43B8E; Mon, 4 Mar 2024 11:36:11 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3AD3540687; Mon, 4 Mar 2024 11:36:11 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 371BC40685 for ; Mon, 4 Mar 2024 11:36:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1709548568; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R+rOyM77UbD8Zhwf9T9JIaM4IzQhV/H29/LyD5G4U4U=; b=BEU6O3lqjCSxX1alsJkCGmuyCnjtDLuVlEIQKQfqquu8PZQ3bLLCSlHgS4nhWACbkgyV69 ZXsy6+AkPfV9PNzZsAPqrArzC1LM3JDGWOA1FK0Nh1rUDYLdEDndTENtjwf+VZoz+70Gq2 IUSp1m1rZqAdvtmGmHviVs7woSTltIc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-158-q3LvNn1jMr6nmR_7GtdYTQ-1; Mon, 04 Mar 2024 05:36:05 -0500 X-MC-Unique: q3LvNn1jMr6nmR_7GtdYTQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 48A7483B8E5; Mon, 4 Mar 2024 10:36:05 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.225.66]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2952F492BC7; Mon, 4 Mar 2024 10:36:04 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: Maxime Coquelin , stable@dpdk.org, Chenbo Xia Subject: [PATCH v2] vhost: fix VDUSE device destruction failure Date: Mon, 4 Mar 2024 11:35:58 +0100 Message-ID: <20240304103558.1500695-1-david.marchand@redhat.com> In-Reply-To: <20240229122502.2572343-2-maxime.coquelin@redhat.com> References: <20240229122502.2572343-2-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Maxime Coquelin VDUSE_DESTROY_DEVICE ioctl can fail because the device's chardev is not released despite close syscall having been called. It happens because the events handler thread is still polling the file descriptor. fdset_pipe_notify() is not enough because it does not ensure the notification has been handled by the event thread, it just returns once the notification is sent. To fix this, this patch introduces a synchronization mechanism based on pthread's condition, so that fdset_pipe_notify_sync() only returns once the pipe's read callback has been executed. Fixes: 51d018fdac4e ("vhost: add VDUSE events handler") Cc: stable@dpdk.org Signed-off-by: Maxime Coquelin Signed-off-by: David Marchand Reviewed-by: Maxime Coquelin --- Changes since v1: - sync'd only when in VDUSE destruction path, - added explicit init of sync_mutex, --- lib/vhost/fd_man.c | 23 +++++++++++++++++++++-- lib/vhost/fd_man.h | 6 ++++++ lib/vhost/socket.c | 1 + lib/vhost/vduse.c | 3 ++- 4 files changed, 30 insertions(+), 3 deletions(-) diff --git a/lib/vhost/fd_man.c b/lib/vhost/fd_man.c index 79a8d2c006..481e6b900a 100644 --- a/lib/vhost/fd_man.c +++ b/lib/vhost/fd_man.c @@ -309,10 +309,11 @@ fdset_event_dispatch(void *arg) } static void -fdset_pipe_read_cb(int readfd, void *dat __rte_unused, +fdset_pipe_read_cb(int readfd, void *dat, int *remove __rte_unused) { char charbuf[16]; + struct fdset *fdset = dat; int r = read(readfd, charbuf, sizeof(charbuf)); /* * Just an optimization, we don't care if read() failed @@ -320,6 +321,11 @@ fdset_pipe_read_cb(int readfd, void *dat __rte_unused, * compiler happy */ RTE_SET_USED(r); + + pthread_mutex_lock(&fdset->sync_mutex); + fdset->sync = true; + pthread_cond_broadcast(&fdset->sync_cond); + pthread_mutex_unlock(&fdset->sync_mutex); } void @@ -342,7 +348,7 @@ fdset_pipe_init(struct fdset *fdset) } ret = fdset_add(fdset, fdset->u.readfd, - fdset_pipe_read_cb, NULL, NULL); + fdset_pipe_read_cb, NULL, fdset); if (ret < 0) { VHOST_FDMAN_LOG(ERR, @@ -366,5 +372,18 @@ fdset_pipe_notify(struct fdset *fdset) * compiler happy */ RTE_SET_USED(r); +} + +void +fdset_pipe_notify_sync(struct fdset *fdset) +{ + pthread_mutex_lock(&fdset->sync_mutex); + + fdset->sync = false; + fdset_pipe_notify(fdset); + + while (!fdset->sync) + pthread_cond_wait(&fdset->sync_cond, &fdset->sync_mutex); + pthread_mutex_unlock(&fdset->sync_mutex); } diff --git a/lib/vhost/fd_man.h b/lib/vhost/fd_man.h index 6315904c8e..7816fb11ac 100644 --- a/lib/vhost/fd_man.h +++ b/lib/vhost/fd_man.h @@ -6,6 +6,7 @@ #define _FD_MAN_H_ #include #include +#include #define MAX_FDS 1024 @@ -35,6 +36,10 @@ struct fdset { int writefd; }; } u; + + pthread_mutex_t sync_mutex; + pthread_cond_t sync_cond; + bool sync; }; @@ -53,5 +58,6 @@ int fdset_pipe_init(struct fdset *fdset); void fdset_pipe_uninit(struct fdset *fdset); void fdset_pipe_notify(struct fdset *fdset); +void fdset_pipe_notify_sync(struct fdset *fdset); #endif diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c index a2fdac30a4..96b3ab5595 100644 --- a/lib/vhost/socket.c +++ b/lib/vhost/socket.c @@ -93,6 +93,7 @@ static struct vhost_user vhost_user = { .fd = { [0 ... MAX_FDS - 1] = {-1, NULL, NULL, NULL, 0} }, .fd_mutex = PTHREAD_MUTEX_INITIALIZER, .fd_pooling_mutex = PTHREAD_MUTEX_INITIALIZER, + .sync_mutex = PTHREAD_MUTEX_INITIALIZER, .num = 0 }, .vsocket_cnt = 0, diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index d462428d2c..e0c6991b69 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -36,6 +36,7 @@ static struct vduse vduse = { .fd = { [0 ... MAX_FDS - 1] = {-1, NULL, NULL, NULL, 0} }, .fd_mutex = PTHREAD_MUTEX_INITIALIZER, .fd_pooling_mutex = PTHREAD_MUTEX_INITIALIZER, + .sync_mutex = PTHREAD_MUTEX_INITIALIZER, .num = 0 }, }; @@ -618,7 +619,7 @@ vduse_device_destroy(const char *path) vduse_device_stop(dev); fdset_del(&vduse.fdset, dev->vduse_dev_fd); - fdset_pipe_notify(&vduse.fdset); + fdset_pipe_notify_sync(&vduse.fdset); if (dev->vduse_dev_fd >= 0) { close(dev->vduse_dev_fd);