diff mbox series

vhost: fix external mbuf creation

Message ID 20201007125318.9850-1-olivier.matz@6wind.com (mailing list archive)
State Accepted, archived
Delegated to: Maxime Coquelin
Headers
Series vhost: fix external mbuf creation |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/iol-testing success Testing PASS
ci/Intel-compilation success Compilation OK
ci/iol-intel-Performance success Performance Testing PASS
ci/travis-robot success Travis build: passed
ci/iol-mellanox-Performance success Performance Testing PASS

Commit Message

Olivier Matz Oct. 7, 2020, 12:53 p.m. UTC 
  In virtio_dev_extbuf_alloc(), the shinfo structure used to store
the reference counter and the free callback of the external buffer
is by default stored inside the mbuf data.

This is wrong because the mbuf (and its data) can be freed before
the external buffer, for instance in the following situation:

  pkt2 = rte_pktmbuf_alloc(mp);
  rte_pktmbuf_attach(pkt2, pkt);
  rte_pktmbuf_free(pkt);

After this, pkt is freed, but it still contains shinfo, which is
referenced by pkt2.

Fix this by always storing the shinfo beside the external buffer.

Fixes: c3ff0ac70acb ("vhost: improve performance by supporting large buffer")
Cc: stable@dpdk.org

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---

Hi,

I found this issue by code review while discussing about this
patchset [1]. I did not tested the patch, but as I'm only removing
one path among the two, I don't expect that it breaks anything.

[1] http://inbox.dpdk.org/dev/20200730120900.108232-1-yang_y_yi@163.com/

Regards,
Olivier

 lib/librte_vhost/virtio_net.c | 30 ++++++++----------------------
 1 file changed, 8 insertions(+), 22 deletions(-)

Comments

Maxime Coquelin Oct. 9, 2020, 7:15 a.m. UTC | #1 
On 10/7/20 2:53 PM, Olivier Matz wrote:
> In virtio_dev_extbuf_alloc(), the shinfo structure used to store
> the reference counter and the free callback of the external buffer
> is by default stored inside the mbuf data.
> 
> This is wrong because the mbuf (and its data) can be freed before
> the external buffer, for instance in the following situation:
> 
>   pkt2 = rte_pktmbuf_alloc(mp);
>   rte_pktmbuf_attach(pkt2, pkt);
>   rte_pktmbuf_free(pkt);
> 
> After this, pkt is freed, but it still contains shinfo, which is
> referenced by pkt2.
> 
> Fix this by always storing the shinfo beside the external buffer.
> 
> Fixes: c3ff0ac70acb ("vhost: improve performance by supporting large buffer")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
> 
> Hi,
> 
> I found this issue by code review while discussing about this
> patchset [1]. I did not tested the patch, but as I'm only removing
> one path among the two, I don't expect that it breaks anything.
> 
> [1] http://inbox.dpdk.org/dev/20200730120900.108232-1-yang_y_yi@163.com/
> 
> Regards,
> Olivier
> 
>  lib/librte_vhost/virtio_net.c | 30 ++++++++----------------------
>  1 file changed, 8 insertions(+), 22 deletions(-)

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime
Maxime Coquelin Oct. 9, 2020, 7:24 a.m. UTC | #2 
On 10/7/20 2:53 PM, Olivier Matz wrote:
> In virtio_dev_extbuf_alloc(), the shinfo structure used to store
> the reference counter and the free callback of the external buffer
> is by default stored inside the mbuf data.
> 
> This is wrong because the mbuf (and its data) can be freed before
> the external buffer, for instance in the following situation:
> 
>   pkt2 = rte_pktmbuf_alloc(mp);
>   rte_pktmbuf_attach(pkt2, pkt);
>   rte_pktmbuf_free(pkt);
> 
> After this, pkt is freed, but it still contains shinfo, which is
> referenced by pkt2.
> 
> Fix this by always storing the shinfo beside the external buffer.
> 
> Fixes: c3ff0ac70acb ("vhost: improve performance by supporting large buffer")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
> 
> Hi,
> 
> I found this issue by code review while discussing about this
> patchset [1]. I did not tested the patch, but as I'm only removing
> one path among the two, I don't expect that it breaks anything.
> 
> [1] http://inbox.dpdk.org/dev/20200730120900.108232-1-yang_y_yi@163.com/
> 
> Regards,
> Olivier
> 
>  lib/librte_vhost/virtio_net.c | 30 ++++++++----------------------
>  1 file changed, 8 insertions(+), 22 deletions(-)

Applied to dpdk-next-virtio/main.

Thanks,
Maxime
yang_y_yi Oct. 9, 2020, 8:47 a.m. UTC | #3 
Tested-by: Yi Yang <yangyi01@inspur.com>





















At 2020-10-07 20:53:18, "Olivier Matz" <olivier.matz@6wind.com> wrote:
>In virtio_dev_extbuf_alloc(), the shinfo structure used to store
>the reference counter and the free callback of the external buffer
>is by default stored inside the mbuf data.
>
>This is wrong because the mbuf (and its data) can be freed before
>the external buffer, for instance in the following situation:
>
>  pkt2 = rte_pktmbuf_alloc(mp);
>  rte_pktmbuf_attach(pkt2, pkt);
>  rte_pktmbuf_free(pkt);
>
>After this, pkt is freed, but it still contains shinfo, which is
>referenced by pkt2.
>
>Fix this by always storing the shinfo beside the external buffer.
>
>Fixes: c3ff0ac70acb ("vhost: improve performance by supporting large buffer")
>Cc: stable@dpdk.org
>
>Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
>---
>
>Hi,
>
>I found this issue by code review while discussing about this
>patchset [1]. I did not tested the patch, but as I'm only removing
>one path among the two, I don't expect that it breaks anything.
>
>[1] http://inbox.dpdk.org/dev/20200730120900.108232-1-yang_y_yi@163.com/
>
>Regards,
>Olivier
>
> lib/librte_vhost/virtio_net.c | 30 ++++++++----------------------
> 1 file changed, 8 insertions(+), 22 deletions(-)
>
>diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c
>index 0a0bea1a5a..008f5ceb04 100644
>--- a/lib/librte_vhost/virtio_net.c
>+++ b/lib/librte_vhost/virtio_net.c
>@@ -2098,16 +2098,8 @@ virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint32_t size)
> 	rte_iova_t iova;
> 	void *buf;
> 
>-	/* Try to use pkt buffer to store shinfo to reduce the amount of memory
>-	 * required, otherwise store shinfo in the new buffer.
>-	 */
>-	if (rte_pktmbuf_tailroom(pkt) >= sizeof(*shinfo))
>-		shinfo = rte_pktmbuf_mtod(pkt,
>-					  struct rte_mbuf_ext_shared_info *);
>-	else {
>-		total_len += sizeof(*shinfo) + sizeof(uintptr_t);
>-		total_len = RTE_ALIGN_CEIL(total_len, sizeof(uintptr_t));
>-	}
>+	total_len += sizeof(*shinfo) + sizeof(uintptr_t);
>+	total_len = RTE_ALIGN_CEIL(total_len, sizeof(uintptr_t));
> 
> 	if (unlikely(total_len > UINT16_MAX))
> 		return -ENOSPC;
>@@ -2118,18 +2110,12 @@ virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint32_t size)
> 		return -ENOMEM;
> 
> 	/* Initialize shinfo */
>-	if (shinfo) {
>-		shinfo->free_cb = virtio_dev_extbuf_free;
>-		shinfo->fcb_opaque = buf;
>-		rte_mbuf_ext_refcnt_set(shinfo, 1);
>-	} else {
>-		shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf, &buf_len,
>-					      virtio_dev_extbuf_free, buf);
>-		if (unlikely(shinfo == NULL)) {
>-			rte_free(buf);
>-			VHOST_LOG_DATA(ERR, "Failed to init shinfo\n");
>-			return -1;
>-		}
>+	shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf, &buf_len,
>+						virtio_dev_extbuf_free, buf);
>+	if (unlikely(shinfo == NULL)) {
>+		rte_free(buf);
>+		VHOST_LOG_DATA(ERR, "Failed to init shinfo\n");
>+		return -1;
> 	}
> 
> 	iova = rte_malloc_virt2iova(buf);
>-- 
>2.25.1
yang_y_yi Oct. 9, 2020, 9:04 a.m. UTC | #4 
Thanks Oliver for posting this patch, it clearly explain his concern, I totally agree this change. I tested it by OVS DPDK, it works well.

At 2020-10-07 20:53:18, "Olivier Matz" <olivier.matz@6wind.com> wrote:
>In virtio_dev_extbuf_alloc(), the shinfo structure used to store
>the reference counter and the free callback of the external buffer
>is by default stored inside the mbuf data.
>
>This is wrong because the mbuf (and its data) can be freed before
>the external buffer, for instance in the following situation:
>
>  pkt2 = rte_pktmbuf_alloc(mp);
>  rte_pktmbuf_attach(pkt2, pkt);
>  rte_pktmbuf_free(pkt);
>
>After this, pkt is freed, but it still contains shinfo, which is
>referenced by pkt2.
>
>Fix this by always storing the shinfo beside the external buffer.
>
>Fixes: c3ff0ac70acb ("vhost: improve performance by supporting large buffer")
>Cc: stable@dpdk.org
>
>Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
>---
>
>Hi,
>
>I found this issue by code review while discussing about this
>patchset [1]. I did not tested the patch, but as I'm only removing
>one path among the two, I don't expect that it breaks anything.
>
>[1] http://inbox.dpdk.org/dev/20200730120900.108232-1-yang_y_yi@163.com/
>
>Regards,
>Olivier
>
> lib/librte_vhost/virtio_net.c | 30 ++++++++----------------------
> 1 file changed, 8 insertions(+), 22 deletions(-)
>
>diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c
>index 0a0bea1a5a..008f5ceb04 100644
>--- a/lib/librte_vhost/virtio_net.c
>+++ b/lib/librte_vhost/virtio_net.c
>@@ -2098,16 +2098,8 @@ virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint32_t size)
> 	rte_iova_t iova;
> 	void *buf;
> 
>-	/* Try to use pkt buffer to store shinfo to reduce the amount of memory
>-	 * required, otherwise store shinfo in the new buffer.
>-	 */
>-	if (rte_pktmbuf_tailroom(pkt) >= sizeof(*shinfo))
>-		shinfo = rte_pktmbuf_mtod(pkt,
>-					  struct rte_mbuf_ext_shared_info *);
>-	else {
>-		total_len += sizeof(*shinfo) + sizeof(uintptr_t);
>-		total_len = RTE_ALIGN_CEIL(total_len, sizeof(uintptr_t));
>-	}
>+	total_len += sizeof(*shinfo) + sizeof(uintptr_t);
>+	total_len = RTE_ALIGN_CEIL(total_len, sizeof(uintptr_t));
> 
> 	if (unlikely(total_len > UINT16_MAX))
> 		return -ENOSPC;
>@@ -2118,18 +2110,12 @@ virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint32_t size)
> 		return -ENOMEM;
> 
> 	/* Initialize shinfo */
>-	if (shinfo) {
>-		shinfo->free_cb = virtio_dev_extbuf_free;
>-		shinfo->fcb_opaque = buf;
>-		rte_mbuf_ext_refcnt_set(shinfo, 1);
>-	} else {
>-		shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf, &buf_len,
>-					      virtio_dev_extbuf_free, buf);
>-		if (unlikely(shinfo == NULL)) {
>-			rte_free(buf);
>-			VHOST_LOG_DATA(ERR, "Failed to init shinfo\n");
>-			return -1;
>-		}
>+	shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf, &buf_len,
>+						virtio_dev_extbuf_free, buf);
>+	if (unlikely(shinfo == NULL)) {
>+		rte_free(buf);
>+		VHOST_LOG_DATA(ERR, "Failed to init shinfo\n");
>+		return -1;
> 	}
> 
> 	iova = rte_malloc_virt2iova(buf);
>-- 
>2.25.1
diff mbox series

Patch

diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c
index 0a0bea1a5a..008f5ceb04 100644
--- a/lib/librte_vhost/virtio_net.c
+++ b/lib/librte_vhost/virtio_net.c
@@ -2098,16 +2098,8 @@  virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint32_t size)
 	rte_iova_t iova;
 	void *buf;
 
-	/* Try to use pkt buffer to store shinfo to reduce the amount of memory
-	 * required, otherwise store shinfo in the new buffer.
-	 */
-	if (rte_pktmbuf_tailroom(pkt) >= sizeof(*shinfo))
-		shinfo = rte_pktmbuf_mtod(pkt,
-					  struct rte_mbuf_ext_shared_info *);
-	else {
-		total_len += sizeof(*shinfo) + sizeof(uintptr_t);
-		total_len = RTE_ALIGN_CEIL(total_len, sizeof(uintptr_t));
-	}
+	total_len += sizeof(*shinfo) + sizeof(uintptr_t);
+	total_len = RTE_ALIGN_CEIL(total_len, sizeof(uintptr_t));
 
 	if (unlikely(total_len > UINT16_MAX))
 		return -ENOSPC;
@@ -2118,18 +2110,12 @@  virtio_dev_extbuf_alloc(struct rte_mbuf *pkt, uint32_t size)
 		return -ENOMEM;
 
 	/* Initialize shinfo */
-	if (shinfo) {
-		shinfo->free_cb = virtio_dev_extbuf_free;
-		shinfo->fcb_opaque = buf;
-		rte_mbuf_ext_refcnt_set(shinfo, 1);
-	} else {
-		shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf, &buf_len,
-					      virtio_dev_extbuf_free, buf);
-		if (unlikely(shinfo == NULL)) {
-			rte_free(buf);
-			VHOST_LOG_DATA(ERR, "Failed to init shinfo\n");
-			return -1;
-		}
+	shinfo = rte_pktmbuf_ext_shinfo_init_helper(buf, &buf_len,
+						virtio_dev_extbuf_free, buf);
+	if (unlikely(shinfo == NULL)) {
+		rte_free(buf);
+		VHOST_LOG_DATA(ERR, "Failed to init shinfo\n");
+		return -1;
 	}
 
 	iova = rte_malloc_virt2iova(buf);