[dpdk-dev,v2,3/3] vhost: fix vq realloc at numa_realloc

  vq is allocated on pairs, hence we should do pair reallocation
at numa_realloc() as well, otherwise an error like following
occurs while do numa reallocation:

    VHOST_CONFIG: reallocate vq from 0 to 1 node
    PANIC in rte_free():
    Fatal error: Invalid memory

The reason we don't catch it is because numa_realloc() will
not take effect when RTE_LIBRTE_VHOST_NUMA is not enabled,
which is the default case.

Fixes: e049ca6d10e0 ("vhost-user: prepare multiple queue setup")

Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Acked-by: Huawei Xie <huawei.xie@intel.com>
---
 lib/librte_vhost/virtio-net.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
  

> 
> vq is allocated on pairs, hence we should do pair reallocation
> at numa_realloc() as well, otherwise an error like following
> occurs while do numa reallocation:
> 
>     VHOST_CONFIG: reallocate vq from 0 to 1 node
>     PANIC in rte_free():
>     Fatal error: Invalid memory
> 
> The reason we don't catch it is because numa_realloc() will
> not take effect when RTE_LIBRTE_VHOST_NUMA is not enabled,
> which is the default case.
> 
> Fixes: e049ca6d10e0 ("vhost-user: prepare multiple queue setup")
> 
> Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
> Acked-by: Huawei Xie <huawei.xie@intel.com>
> ---
>  lib/librte_vhost/virtio-net.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/librte_vhost/virtio-net.c b/lib/librte_vhost/virtio-net.c
> index 1566c93..7469312 100644
> --- a/lib/librte_vhost/virtio-net.c
> +++ b/lib/librte_vhost/virtio-net.c
> @@ -445,6 +445,13 @@ numa_realloc(struct virtio_net *dev, int index)
>  	struct vhost_virtqueue *old_vq, *vq;
>  	int ret;
> 
> +	/*
> +	 * vq is allocated on pairs, we should try to do realloc
> +	 * on first queue of one queue pair only.
> +	 */
> +	if (index % VIRTIO_QNUM != 0)
> +		return dev;
> +
>  	old_dev = dev;
>  	vq = old_vq = dev->virtqueue[index];
> 
> @@ -461,11 +468,12 @@ numa_realloc(struct virtio_net *dev, int index)
>  	if (oldnode != newnode) {
>  		RTE_LOG(INFO, VHOST_CONFIG,
>  			"reallocate vq from %d to %d node\n", oldnode,
> newnode);
> -		vq = rte_malloc_socket(NULL, sizeof(*vq), 0, newnode);
> +		vq = rte_malloc_socket(NULL, sizeof(*vq) * VIRTIO_QNUM,
> 0,
> +				       newnode);
>  		if (!vq)
>  			return dev;
> 
> -		memcpy(vq, old_vq, sizeof(*vq));
> +		memcpy(vq, old_vq, sizeof(*vq) * VIRTIO_QNUM);
>  		rte_free(old_vq);
>  	}
> 
> @@ -491,6 +499,7 @@ numa_realloc(struct virtio_net *dev, int index)
> 
>  out:
>  	dev->virtqueue[index] = vq;
> +	dev->virtqueue[index + 1] = vq + 1;
>  	vhost_devices[dev->device_fh] = dev;
> 
>  	return dev;
> --
> 1.9.0

I encountered the " PANIC in rte_free():" error when using RTE_LIBRTE_VHOST_NUMA too, and applying this series resolved the issue. Thanks for the patches.

Tested-by: Ciara Loftus <ciara.loftus@intel.com>

Thanks,
Ciara
  

On Mon, Mar 07, 2016 at 01:49:26PM +0000, Loftus, Ciara wrote:
> 
> I encountered the " PANIC in rte_free():" error when using RTE_LIBRTE_VHOST_NUMA too, and applying this series resolved the issue. Thanks for the patches.
> 
> Tested-by: Ciara Loftus <ciara.loftus@intel.com>

Thanks for testing. A new version (just about rebase) will hopefully be
sent out in one or two days.

	--yliu