From patchwork Mon Mar 14 07:54:00 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tetsuya Mukawa X-Patchwork-Id: 11477 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 2D5944A65; Mon, 14 Mar 2016 08:54:06 +0100 (CET) Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com [209.85.220.53]) by dpdk.org (Postfix) with ESMTP id E47AF2C1A for ; Mon, 14 Mar 2016 08:54:03 +0100 (CET) Received: by mail-pa0-f53.google.com with SMTP id tt10so152116434pab.3 for ; Mon, 14 Mar 2016 00:54:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igel-co-jp.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=sXvfImagK7n3hsdnAO1H2POdvTCyYZ0SMhpJ3T4NFAA=; b=Kde3i7BeeNTPM61nUO9YtDe0yWl9tesPJ9TthoYmBTSPZgOYIJtth3x5Qcyz/IfPU8 I7E/WdiuXuSOXMALF6etT2DGdcTzJ4aGuyH/jpvi58sTdm1Ebo/9h83ZI851L750/fw1 lXmhWmUTCO5H+YjLMlanCgmQxg+l0Tom87U/xNTRYbzQmEPJ8EA6KxM9shUxJ0Cg3q9j dilsa1L1+DnIpgJPGQszi7DVzrH/+A5hgyT+x2l7MrgwnNfGSVf+k9xpxJ1rYL9udlBU 7Y8r6L2p5sgOGxitc9MJi9tpEiW8DWc8VqShPABNu6arFQD5TfcEUxGppz6g3oU0B4JQ mPoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=sXvfImagK7n3hsdnAO1H2POdvTCyYZ0SMhpJ3T4NFAA=; b=dzelSr/k7L92FaMegOJFE/fB4bMfDfcnsGtrluoQr6jzEPND6QMRAMqGDy3qPvIWAc bKULry1QyQeQT+jrxJ+sjeDN6sDrqb07svEtpeGc+t3hW+hcL2nezQrbSexpMe3Omb+S 3VTntnS9et84KiW0PRymkfTsefL0knCGl4Sd8sGMagtLJxPoEspTH+SJD67J6DJi12+z zIVhez2v6t8gvXmixnBeCZS5y+QwGXQIsm/S6vCYPc39RNt0GU+W68pbLH4wGuiofz5x pRlIhwmTPdM4EkEpMOPbQxPQ++TiIeLDLZ864tr6NBqZLlBzV2KWNmZeqWq/mx23kUzl jfqA== X-Gm-Message-State: AD7BkJLEiK59BFDBi9GX2ZQm3fgijTQJsSyMp/ulGXU/ZfFUlbJJ04iksrjFKFuRZ/EyZg== X-Received: by 10.66.190.168 with SMTP id gr8mr35911597pac.23.1457942043310; Mon, 14 Mar 2016 00:54:03 -0700 (PDT) Received: from [10.16.129.101] (napt.igel.co.jp. [219.106.231.132]) by smtp.googlemail.com with ESMTPSA id ey7sm29838711pab.47.2016.03.14.00.54.01 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 14 Mar 2016 00:54:02 -0700 (PDT) To: Yuanhan Liu References: <1457590462-3680-1-git-send-email-mukawa@igel.co.jp> <1457593565-16240-1-git-send-email-mukawa@igel.co.jp> <20160311071907.GI979@yliu-dev.sh.intel.com> <56E619C6.6060809@igel.co.jp> <20160314020858.GJ979@yliu-dev.sh.intel.com> Cc: dev@dpdk.org, jianfeng.tan@intel.com, huawei.xie@intel.com From: Tetsuya Mukawa Message-ID: <56E66E18.4000700@igel.co.jp> Date: Mon, 14 Mar 2016 16:54:00 +0900 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <20160314020858.GJ979@yliu-dev.sh.intel.com> Subject: Re: [dpdk-dev] [PATCH v2] vhost: Fix default value of kickfd and callfd X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 2016/03/14 11:08, Yuanhan Liu wrote: > On Mon, Mar 14, 2016 at 10:54:14AM +0900, Tetsuya Mukawa wrote: >> On 2016/03/11 16:19, Yuanhan Liu wrote: >>> On Thu, Mar 10, 2016 at 04:06:05PM +0900, Tetsuya Mukawa wrote: >>>> Currently, default values of kickfd and callfd are -1. >>>> If the values are -1, current code guesses kickfd and callfd haven't >>>> been initialized yet. And vhost library will guess the virtqueue isn't >>>> ready for processing. >>>> But callfd and kickfd will be set as -1 when "--enable-kvm" >>>> isn't specified in QEMU command line. It means we cannot treat -1 as >>>> uninitialized state. The patch changes default values to -2. And the >>>> patch defines -2 as VIRTIO_UNINITIALIZED_EVENTFD. >>> This looks more like a workaround to me. >> >> Hi Yuanhan, >> >> Sorry for late reply. >> I have checked QEMU documentation, and found below. >> >> ---------- >> * VHOST_USER_SET_VRING_CALL >> >> Id: 14 >> Equivalent ioctl: VHOST_SET_VRING_CALL >> Master payload: u64 >> >> Set the event file descriptor to signal when buffers are used. It >> is passed in the ancillary data. >> Bits (0-7) of the payload contain the vring index. Bit 8 is the >> invalid FD flag. >> ---------- >> >> VHOST_USER_SET_VRING_KICK has almost same description. >> I will check this invalid flag, and if it works for our case, then will >> use it. >> How about it? > Yeah, that indeed sounds much better. I've checked current dpdk code. It seems we've already checked invalid flag like below. if (pmsg->payload.u64 & VHOST_USER_VRING_NOFD_MASK) file.fd = -1; else file.fd = pmsg->fds[0]; So how about adding below macros or enum? #define VIRTIO_UNINITIALIZED_EVENTFD (-2) #define VIRTIO_INVALID_EVENTFD (-1) I am still not sure whether using enum is better or not. But here is one of example patch. What do you think? > --yliu > >>> Besides, this patch would make >>> following fail: >>> >>> eventfd_write(vq->callfd, (eventfd_t)1); >> It's my fault. >> I thought above case should be blocked by virtio spec itself. So just >> leave it. >> But eventfd is came from vhost spec, so VRING_AVAIL_F_NO_INTERRUPT might >> be set even when kickfd and callfd are -1. >> Thanks for checking it. >> >> Tetsuya >> >>> --yliu diff --git a/lib/librte_vhost/rte_virtio_net.h b/lib/librte_vhost/rte_virtio_net.h index 7d1fde2..2a7566d 100644 --- a/lib/librte_vhost/rte_virtio_net.h +++ b/lib/librte_vhost/rte_virtio_net.h @@ -89,6 +89,8 @@ struct vhost_virtqueue { uint16_t vhost_hlen; /**< Vhost header length (varies depending on RX merge buffers. */ volatile uint16_t last_used_idx; /**< Last index used on the available ring */ volatile uint16_t last_used_idx_res; /**< Used for multiple devices reserving buffers. */ +#define VIRTIO_UNINITIALIZED_EVENTFD (-2) +#define VIRTIO_INVALID_EVENTFD (-1) int callfd; /**< Used to notify the guest (trigger interrupt). */ int kickfd; /**< Currently unused as polling mode is enabled. */ int enabled; diff --git a/lib/librte_vhost/vhost_rxtx.c b/lib/librte_vhost/vhost_rxtx.c index 9d23eb1..29d002d 100644 --- a/lib/librte_vhost/vhost_rxtx.c +++ b/lib/librte_vhost/vhost_rxtx.c @@ -333,7 +333,8 @@ virtio_dev_rx(struct virtio_net *dev, uint16_t queue_id, rte_mb(); /* Kick the guest if necessary. */ - if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT)) + if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT) + && (vq->callfd >= 0)) eventfd_write(vq->callfd, (eventfd_t)1); return count; } @@ -654,7 +655,8 @@ merge_rx_exit: rte_mb(); /* Kick the guest if necessary. */ - if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT)) + if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT) + && (vq->callfd >= 0)) eventfd_write(vq->callfd, (eventfd_t)1); } @@ -1048,7 +1050,8 @@ rte_vhost_dequeue_burst(struct virtio_net *dev, uint16_t queue_id, sizeof(vq->used->idx)); /* Kick guest if required. */ - if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT)) + if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT) + && (vq->callfd >= 0)) eventfd_write(vq->callfd, (eventfd_t)1); out: diff --git a/lib/librte_vhost/vhost_user/virtio-net-user.c b/lib/librte_vhost/vhost_user/virtio-net-user.c index 65b5652..f5248bc 100644 --- a/lib/librte_vhost/vhost_user/virtio-net-user.c +++ b/lib/librte_vhost/vhost_user/virtio-net-user.c @@ -225,8 +225,8 @@ static int vq_is_ready(struct vhost_virtqueue *vq) { return vq && vq->desc && - vq->kickfd != -1 && - vq->callfd != -1; + vq->kickfd != VIRTIO_UNINITIALIZED_EVENTFD && + vq->callfd != VIRTIO_UNINITIALIZED_EVENTFD; } static int @@ -258,7 +258,7 @@ user_set_vring_call(struct vhost_device_ctx ctx, struct VhostUserMsg *pmsg) file.index = pmsg->payload.u64 & VHOST_USER_VRING_IDX_MASK; if (pmsg->payload.u64 & VHOST_USER_VRING_NOFD_MASK) - file.fd = -1; + file.fd = VIRTIO_INVALID_EVENTFD; else file.fd = pmsg->fds[0]; RTE_LOG(INFO, VHOST_CONFIG, @@ -279,7 +279,7 @@ user_set_vring_kick(struct vhost_device_ctx ctx, struct VhostUserMsg *pmsg) file.index = pmsg->payload.u64 & VHOST_USER_VRING_IDX_MASK; if (pmsg->payload.u64 & VHOST_USER_VRING_NOFD_MASK) - file.fd = -1; + file.fd = VIRTIO_INVALID_EVENTFD; else file.fd = pmsg->fds[0]; RTE_LOG(INFO, VHOST_CONFIG, @@ -316,10 +316,10 @@ user_get_vring_base(struct vhost_device_ctx ctx, * sent and only sent in vhost_vring_stop. * TODO: cleanup the vring, it isn't usable since here. */ - if (dev->virtqueue[state->index]->kickfd >= 0) { + if (dev->virtqueue[state->index]->kickfd >= 0) close(dev->virtqueue[state->index]->kickfd); - dev->virtqueue[state->index]->kickfd = -1; - } + + dev->virtqueue[state->index]->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; return 0; } diff --git a/lib/librte_vhost/virtio-net.c b/lib/librte_vhost/virtio-net.c index fe1a77e..3498e9d 100644 --- a/lib/librte_vhost/virtio-net.c +++ b/lib/librte_vhost/virtio-net.c @@ -263,8 +263,8 @@ init_vring_queue(struct vhost_virtqueue *vq, int qp_idx) { memset(vq, 0, sizeof(struct vhost_virtqueue)); - vq->kickfd = -1; - vq->callfd = -1; + vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + vq->callfd = VIRTIO_UNINITIALIZED_EVENTFD; /* Backends are set to -1 indicating an inactive device. */ vq->backend = -1;