From patchwork Wed Dec 19 08:21:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 49108 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1919F1BC14; Wed, 19 Dec 2018 09:21:29 +0100 (CET) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id ADCF41BACE; Wed, 19 Dec 2018 09:21:27 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DE241C02835A; Wed, 19 Dec 2018 08:21:26 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-26.ams2.redhat.com [10.36.112.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id BB22767E68; Wed, 19 Dec 2018 08:21:24 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, i.maximets@samsung.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com, mst@redhat.com Cc: stable@dpdk.org, Maxime Coquelin Date: Wed, 19 Dec 2018 09:21:10 +0100 Message-Id: <20181219082113.24455-2-maxime.coquelin@redhat.com> In-Reply-To: <20181219082113.24455-1-maxime.coquelin@redhat.com> References: <20181219082113.24455-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 19 Dec 2018 08:21:26 +0000 (UTC) Subject: [dpdk-dev] [PATCH v2 1/4] vhost: enforce avail index and desc read ordering X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" A read barrier is required to ensure the ordering between available index and the descriptor reads is enforced. 1. read avail_head = avail->idx 2. read cur_idx = last_avail_idx if (cur_idx != avail_head) { 3. read idx = avail->ring[cur_idx] 4. read desc[idx] } There is a control dependency between step 1 and steps 3 & 4, 3 could be speculatively executed before 1, which could result in 'idx' to not being updated yet. Fixes: 4796ad63ba1f ("examples/vhost: import userspace vhost application") Cc: stable@dpdk.org Reported-by: Jason Wang Signed-off-by: Maxime Coquelin Acked-by: Ilya Maximets Acked-by: Michael S. Tsirkin --- lib/librte_vhost/virtio_net.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index 8c657a101..7f37bbbed 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -752,6 +752,12 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); avail_head = *((volatile uint16_t *)&vq->avail->idx); + /* + * The ordering between avail index and + * desc reads needs to be enforced. + */ + rte_smp_rmb(); + for (pkt_idx = 0; pkt_idx < count; pkt_idx++) { uint32_t pkt_len = pkts[pkt_idx]->pkt_len + dev->vhost_hlen; uint16_t nr_vec = 0; @@ -1334,6 +1340,12 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, if (free_entries == 0) return 0; + /* + * The ordering between avail index and + * desc reads needs to be enforced. + */ + rte_smp_rmb(); + VHOST_LOG_DEBUG(VHOST_DATA, "(%d) %s\n", dev->vid, __func__); count = RTE_MIN(count, MAX_PKT_BURST); From patchwork Wed Dec 19 08:21:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 49109 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8E9011BC04; Wed, 19 Dec 2018 09:21:32 +0100 (CET) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 8803A1BBFC; Wed, 19 Dec 2018 09:21:30 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D7F08C00C10B; Wed, 19 Dec 2018 08:21:29 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-26.ams2.redhat.com [10.36.112.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 44A9D6BF73; Wed, 19 Dec 2018 08:21:27 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, i.maximets@samsung.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com, mst@redhat.com Cc: stable@dpdk.org, Maxime Coquelin Date: Wed, 19 Dec 2018 09:21:11 +0100 Message-Id: <20181219082113.24455-3-maxime.coquelin@redhat.com> In-Reply-To: <20181219082113.24455-1-maxime.coquelin@redhat.com> References: <20181219082113.24455-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 19 Dec 2018 08:21:29 +0000 (UTC) Subject: [dpdk-dev] [PATCH v2 2/4] vhost: enforce desc flags and content read ordering X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" A read barrier is required to ensure that the ordering between descriptor's flags and content reads is enforced. 1. read flags = desc->flags if (flags & AVAIL_BIT) 2. read desc->id There is a control dependency between steps 1 and step 2. 2 could be speculatively executed before 1, which could result in 'id' to not be updated yet. Fixes: 2f3225a7d69b ("vhost: add vector filling support for packed ring") Cc: stable@dpdk.org Reported-by: Jason Wang Signed-off-by: Maxime Coquelin Acked-by: Ilya Maximets --- lib/librte_vhost/virtio_net.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index 7f37bbbed..a9b1bc55f 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -481,6 +481,12 @@ fill_vec_buf_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, if (unlikely(!desc_is_avail(&descs[avail_idx], wrap_counter))) return -1; + /* + * The ordering between desc flags and desc + * content reads need to be enforced. + */ + rte_smp_rmb(); + *desc_count = 0; *len = 0; From patchwork Wed Dec 19 08:21:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 49110 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3223A1BC7B; Wed, 19 Dec 2018 09:21:35 +0100 (CET) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id DE73A1BC73; Wed, 19 Dec 2018 09:21:33 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3DD764E920; Wed, 19 Dec 2018 08:21:33 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-26.ams2.redhat.com [10.36.112.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 42D0B67E68; Wed, 19 Dec 2018 08:21:30 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, i.maximets@samsung.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com, mst@redhat.com Cc: stable@dpdk.org, Maxime Coquelin Date: Wed, 19 Dec 2018 09:21:12 +0100 Message-Id: <20181219082113.24455-4-maxime.coquelin@redhat.com> In-Reply-To: <20181219082113.24455-1-maxime.coquelin@redhat.com> References: <20181219082113.24455-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 19 Dec 2018 08:21:33 +0000 (UTC) Subject: [dpdk-dev] [PATCH v2 3/4] vhost: prefetch descriptor after the read barrier X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch moves the prefetch after the available index is read to avoid prefetching a descriptor not available yet. Signed-off-by: Maxime Coquelin --- lib/librte_vhost/virtio_net.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index a9b1bc55f..a9be633de 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -755,7 +755,6 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, struct buf_vector buf_vec[BUF_VECTOR_MAX]; uint16_t avail_head; - rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); avail_head = *((volatile uint16_t *)&vq->avail->idx); /* @@ -764,6 +763,8 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, */ rte_smp_rmb(); + rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); + for (pkt_idx = 0; pkt_idx < count; pkt_idx++) { uint32_t pkt_len = pkts[pkt_idx]->pkt_len + dev->vhost_hlen; uint16_t nr_vec = 0; @@ -1339,8 +1340,6 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, } } - rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); - free_entries = *((volatile uint16_t *)&vq->avail->idx) - vq->last_avail_idx; if (free_entries == 0) @@ -1352,6 +1351,8 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, */ rte_smp_rmb(); + rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); + VHOST_LOG_DEBUG(VHOST_DATA, "(%d) %s\n", dev->vid, __func__); count = RTE_MIN(count, MAX_PKT_BURST); From patchwork Wed Dec 19 08:21:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 49111 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 942A31BC82; Wed, 19 Dec 2018 09:21:45 +0100 (CET) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id AD30B1BC05; Wed, 19 Dec 2018 09:21:43 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 12AF65F722; Wed, 19 Dec 2018 08:21:43 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-26.ams2.redhat.com [10.36.112.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id A587960DB7; Wed, 19 Dec 2018 08:21:33 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, i.maximets@samsung.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com, mst@redhat.com Cc: stable@dpdk.org, Maxime Coquelin Date: Wed, 19 Dec 2018 09:21:13 +0100 Message-Id: <20181219082113.24455-5-maxime.coquelin@redhat.com> In-Reply-To: <20181219082113.24455-1-maxime.coquelin@redhat.com> References: <20181219082113.24455-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 19 Dec 2018 08:21:43 +0000 (UTC) Subject: [dpdk-dev] [PATCH v2 4/4] vhost: remove useless prefetch for packed ring descriptor X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This prefetch does not show any performance improvement. Signed-off-by: Maxime Coquelin --- lib/librte_vhost/virtio_net.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index a9be633de..4d201a3fd 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -1437,8 +1437,6 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, { uint16_t i; - rte_prefetch0(&vq->desc_packed[vq->last_avail_idx]); - if (unlikely(dev->dequeue_zero_copy)) { struct zcopy_mbuf *zmbuf, *next;