From patchwork Wed Dec 5 09:49:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 48537 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BFC341B1E3; Wed, 5 Dec 2018 10:50:34 +0100 (CET) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 4BAEB1B160; Wed, 5 Dec 2018 10:50:33 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AAB6C659AC; Wed, 5 Dec 2018 09:50:32 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-38.ams2.redhat.com [10.36.112.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E12C5D757; Wed, 5 Dec 2018 09:50:26 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, jfreimann@redhat.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com Cc: Maxime Coquelin , stable@dpdk.org Date: Wed, 5 Dec 2018 10:49:53 +0100 Message-Id: <20181205094957.1938-2-maxime.coquelin@redhat.com> In-Reply-To: <20181205094957.1938-1-maxime.coquelin@redhat.com> References: <20181205094957.1938-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 05 Dec 2018 09:50:32 +0000 (UTC) Subject: [dpdk-dev] [PATCH 1/5] vhost: enforce avail index and desc read ordering X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" A read barrier is required to ensure the ordering between available index and the descriptor reads is enforced. Fixes: 4796ad63ba1f ("examples/vhost: import userspace vhost application") Cc: stable@dpdk.org Reported-by: Jason Wang Signed-off-by: Maxime Coquelin Acked-by: Ilya Maximets --- lib/librte_vhost/virtio_net.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index 5e1a1a727..f11ebb54f 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -791,6 +791,12 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); avail_head = *((volatile uint16_t *)&vq->avail->idx); + /* + * The ordering between avail index and + * desc reads needs to be enforced. + */ + rte_smp_rmb(); + for (pkt_idx = 0; pkt_idx < count; pkt_idx++) { uint32_t pkt_len = pkts[pkt_idx]->pkt_len + dev->vhost_hlen; uint16_t nr_vec = 0; @@ -1373,6 +1379,12 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, if (free_entries == 0) return 0; + /* + * The ordering between avail index and + * desc reads needs to be enforced. + */ + rte_smp_rmb(); + VHOST_LOG_DEBUG(VHOST_DATA, "(%d) %s\n", dev->vid, __func__); count = RTE_MIN(count, MAX_PKT_BURST); From patchwork Wed Dec 5 09:49:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 48538 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 598F71B1EE; Wed, 5 Dec 2018 10:50:37 +0100 (CET) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 20AD71B1EB; Wed, 5 Dec 2018 10:50:36 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 735C1C058CB2; Wed, 5 Dec 2018 09:50:35 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-38.ams2.redhat.com [10.36.112.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B79E5D780; Wed, 5 Dec 2018 09:50:32 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, jfreimann@redhat.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com Cc: Maxime Coquelin , stable@dpdk.org Date: Wed, 5 Dec 2018 10:49:54 +0100 Message-Id: <20181205094957.1938-3-maxime.coquelin@redhat.com> In-Reply-To: <20181205094957.1938-1-maxime.coquelin@redhat.com> References: <20181205094957.1938-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 05 Dec 2018 09:50:35 +0000 (UTC) Subject: [dpdk-dev] [PATCH 2/5] vhost: enforce desc flags and content read ordering X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" A read barrier is required to ensure that the ordering between descriptor's flags and content reads is enforced. Fixes: 2f3225a7d69b ("vhost: add vector filling support for packed ring") Cc: stable@dpdk.org Reported-by: Jason Wang Signed-off-by: Maxime Coquelin Acked-by: Ilya Maximets --- lib/librte_vhost/virtio_net.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index f11ebb54f..68b72e7a5 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -520,6 +520,12 @@ fill_vec_buf_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, if (unlikely(!desc_is_avail(&descs[avail_idx], wrap_counter))) return -1; + /* + * The ordering between desc flags and desc + * content reads need to be enforced. + */ + rte_smp_rmb(); + *desc_count = 0; *len = 0; From patchwork Wed Dec 5 09:49:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 48539 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7B31A1B1F2; Wed, 5 Dec 2018 10:50:39 +0100 (CET) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 60B701B1F2 for ; Wed, 5 Dec 2018 10:50:38 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C3BD730944E4; Wed, 5 Dec 2018 09:50:37 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-38.ams2.redhat.com [10.36.112.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id CA7485D757; Wed, 5 Dec 2018 09:50:35 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, jfreimann@redhat.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com Cc: Maxime Coquelin Date: Wed, 5 Dec 2018 10:49:55 +0100 Message-Id: <20181205094957.1938-4-maxime.coquelin@redhat.com> In-Reply-To: <20181205094957.1938-1-maxime.coquelin@redhat.com> References: <20181205094957.1938-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Wed, 05 Dec 2018 09:50:37 +0000 (UTC) Subject: [dpdk-dev] [PATCH 3/5] vhost: prefetch descriptor after the read barrier X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch moves the prefetch after the available index is read to avoid prefetching a descriptor not available yet. Signed-off-by: Maxime Coquelin --- lib/librte_vhost/virtio_net.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index 68b72e7a5..0a860ca72 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -794,7 +794,6 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, struct buf_vector buf_vec[BUF_VECTOR_MAX]; uint16_t avail_head; - rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); avail_head = *((volatile uint16_t *)&vq->avail->idx); /* @@ -803,6 +802,8 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, */ rte_smp_rmb(); + rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); + for (pkt_idx = 0; pkt_idx < count; pkt_idx++) { uint32_t pkt_len = pkts[pkt_idx]->pkt_len + dev->vhost_hlen; uint16_t nr_vec = 0; @@ -1378,8 +1379,6 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, } } - rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); - free_entries = *((volatile uint16_t *)&vq->avail->idx) - vq->last_avail_idx; if (free_entries == 0) @@ -1391,6 +1390,8 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, */ rte_smp_rmb(); + rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); + VHOST_LOG_DEBUG(VHOST_DATA, "(%d) %s\n", dev->vid, __func__); count = RTE_MIN(count, MAX_PKT_BURST); From patchwork Wed Dec 5 09:49:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 48540 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B02841B1FC; Wed, 5 Dec 2018 10:50:42 +0100 (CET) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 14E341B160 for ; Wed, 5 Dec 2018 10:50:41 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 76376300272E; Wed, 5 Dec 2018 09:50:40 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-38.ams2.redhat.com [10.36.112.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1C9F05D780; Wed, 5 Dec 2018 09:50:37 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, jfreimann@redhat.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com Cc: Maxime Coquelin Date: Wed, 5 Dec 2018 10:49:56 +0100 Message-Id: <20181205094957.1938-5-maxime.coquelin@redhat.com> In-Reply-To: <20181205094957.1938-1-maxime.coquelin@redhat.com> References: <20181205094957.1938-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Wed, 05 Dec 2018 09:50:40 +0000 (UTC) Subject: [dpdk-dev] [PATCH 4/5] vhost: remove useless prefetch for packed ring descriptor X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This prefetch does not show any performance improvement. Signed-off-by: Maxime Coquelin --- lib/librte_vhost/virtio_net.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index 0a860ca72..679ce388b 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -1476,8 +1476,6 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, { uint16_t i; - rte_prefetch0(&vq->desc_packed[vq->last_avail_idx]); - if (unlikely(dev->dequeue_zero_copy)) { struct zcopy_mbuf *zmbuf, *next; From patchwork Wed Dec 5 09:49:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 48541 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0D11D1B20A; Wed, 5 Dec 2018 10:50:57 +0100 (CET) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id C341A1B1F9 for ; Wed, 5 Dec 2018 10:50:55 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1FBF288E51; Wed, 5 Dec 2018 09:50:55 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-38.ams2.redhat.com [10.36.112.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id 163325D760; Wed, 5 Dec 2018 09:50:40 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, jfreimann@redhat.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com Cc: Maxime Coquelin Date: Wed, 5 Dec 2018 10:49:57 +0100 Message-Id: <20181205094957.1938-6-maxime.coquelin@redhat.com> In-Reply-To: <20181205094957.1938-1-maxime.coquelin@redhat.com> References: <20181205094957.1938-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 05 Dec 2018 09:50:55 +0000 (UTC) Subject: [dpdk-dev] [PATCH 5/5] vhost: remove useless casts to volatile X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Cast to volatile is done when reading avail index and writing the used index. This would not be necessary if proper barriers are used. Now that the read barrier has been added, we can remove these cast to volatile. Signed-off-by: Maxime Coquelin --- lib/librte_vhost/virtio_net.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index 679ce388b..eab1a5b4c 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -114,7 +114,7 @@ flush_shadow_used_ring_split(struct virtio_net *dev, struct vhost_virtqueue *vq) vhost_log_cache_sync(dev, vq); - *(volatile uint16_t *)&vq->used->idx += vq->shadow_used_idx; + vq->used->idx += vq->shadow_used_idx; vq->shadow_used_idx = 0; vhost_log_used_vring(dev, vq, offsetof(struct vring_used, idx), sizeof(vq->used->idx)); @@ -794,7 +794,7 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, struct buf_vector buf_vec[BUF_VECTOR_MAX]; uint16_t avail_head; - avail_head = *((volatile uint16_t *)&vq->avail->idx); + avail_head = vq->avail->idx; /* * The ordering between avail index and @@ -1379,8 +1379,7 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, } } - free_entries = *((volatile uint16_t *)&vq->avail->idx) - - vq->last_avail_idx; + free_entries = vq->avail->idx - vq->last_avail_idx; if (free_entries == 0) return 0;