[v6,3/7] vhost: fix missing cache logging NUMA realloc
Checks
Commit Message
When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.
However, reallocating the log cache on the new NUMA node was
not done. This patch fixes this by reallocating it if it has
been allocated already, which means a live-migration is
on-going.
Fixes: 1818a63147fb ("vhost: move dirty logging cache out of virtqueue")
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
lib/vhost/vhost_user.c | 10 ++++++++++
1 file changed, 10 insertions(+)
Comments
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Friday, June 18, 2021 10:04 PM
> To: dev@dpdk.org; david.marchand@redhat.com; Xia, Chenbo <chenbo.xia@intel.com>
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH v6 3/7] vhost: fix missing cache logging NUMA realloc
>
> When the guest allocates virtqueues on a different NUMA node
> than the one the Vhost metadata are allocated, both the Vhost
> device struct and the virtqueues struct are reallocated.
>
> However, reallocating the log cache on the new NUMA node was
> not done. This patch fixes this by reallocating it if it has
> been allocated already, which means a live-migration is
> on-going.
>
> Fixes: 1818a63147fb ("vhost: move dirty logging cache out of virtqueue")
This commit is of 21.05, although LTS maintainers don't maintain non-LTS stable
releases now, I guess it's still better to add 'cc stable tag' in case anyone
volunteers to do that?
Thanks,
Chenbo
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> lib/vhost/vhost_user.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
> index 5fb055ea2e..82adf80fe5 100644
> --- a/lib/vhost/vhost_user.c
> +++ b/lib/vhost/vhost_user.c
> @@ -545,6 +545,16 @@ numa_realloc(struct virtio_net *dev, int index)
> vq->batch_copy_elems = new_batch_copy_elems;
> }
>
> + if (vq->log_cache) {
> + struct log_cache_entry *log_cache;
> +
> + log_cache = rte_realloc_socket(vq->log_cache,
> + sizeof(struct log_cache_entry) *
> VHOST_LOG_CACHE_NR,
> + 0, newnode);
> + if (log_cache)
> + vq->log_cache = log_cache;
> + }
> +
> rte_free(old_vq);
> }
>
> --
> 2.31.1
On 6/25/21 4:50 AM, Xia, Chenbo wrote:
> Hi Maxime,
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Friday, June 18, 2021 10:04 PM
>> To: dev@dpdk.org; david.marchand@redhat.com; Xia, Chenbo <chenbo.xia@intel.com>
>> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Subject: [PATCH v6 3/7] vhost: fix missing cache logging NUMA realloc
>>
>> When the guest allocates virtqueues on a different NUMA node
>> than the one the Vhost metadata are allocated, both the Vhost
>> device struct and the virtqueues struct are reallocated.
>>
>> However, reallocating the log cache on the new NUMA node was
>> not done. This patch fixes this by reallocating it if it has
>> been allocated already, which means a live-migration is
>> on-going.
>>
>> Fixes: 1818a63147fb ("vhost: move dirty logging cache out of virtqueue")
>
> This commit is of 21.05, although LTS maintainers don't maintain non-LTS stable
> releases now, I guess it's still better to add 'cc stable tag' in case anyone
> volunteers to do that?
I don't think that's what we do usually.
If someone wants to maintain v21.05 in the future, he can just look for
the Fixes tag in the git history.
Thanks,
Maxime
> Thanks,
> Chenbo
>
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> lib/vhost/vhost_user.c | 10 ++++++++++
>> 1 file changed, 10 insertions(+)
>>
>> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
>> index 5fb055ea2e..82adf80fe5 100644
>> --- a/lib/vhost/vhost_user.c
>> +++ b/lib/vhost/vhost_user.c
>> @@ -545,6 +545,16 @@ numa_realloc(struct virtio_net *dev, int index)
>> vq->batch_copy_elems = new_batch_copy_elems;
>> }
>>
>> + if (vq->log_cache) {
>> + struct log_cache_entry *log_cache;
>> +
>> + log_cache = rte_realloc_socket(vq->log_cache,
>> + sizeof(struct log_cache_entry) *
>> VHOST_LOG_CACHE_NR,
>> + 0, newnode);
>> + if (log_cache)
>> + vq->log_cache = log_cache;
>> + }
>> +
>> rte_free(old_vq);
>> }
>>
>> --
>> 2.31.1
>
Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Tuesday, June 29, 2021 10:39 PM
> To: Xia, Chenbo <chenbo.xia@intel.com>; dev@dpdk.org;
> david.marchand@redhat.com
> Subject: Re: [PATCH v6 3/7] vhost: fix missing cache logging NUMA realloc
>
>
>
> On 6/25/21 4:50 AM, Xia, Chenbo wrote:
> > Hi Maxime,
> >
> >> -----Original Message-----
> >> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> >> Sent: Friday, June 18, 2021 10:04 PM
> >> To: dev@dpdk.org; david.marchand@redhat.com; Xia, Chenbo
> <chenbo.xia@intel.com>
> >> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> >> Subject: [PATCH v6 3/7] vhost: fix missing cache logging NUMA realloc
> >>
> >> When the guest allocates virtqueues on a different NUMA node
> >> than the one the Vhost metadata are allocated, both the Vhost
> >> device struct and the virtqueues struct are reallocated.
> >>
> >> However, reallocating the log cache on the new NUMA node was
> >> not done. This patch fixes this by reallocating it if it has
> >> been allocated already, which means a live-migration is
> >> on-going.
> >>
> >> Fixes: 1818a63147fb ("vhost: move dirty logging cache out of virtqueue")
> >
> > This commit is of 21.05, although LTS maintainers don't maintain non-LTS
> stable
> > releases now, I guess it's still better to add 'cc stable tag' in case
> anyone
> > volunteers to do that?
>
>
> I don't think that's what we do usually.
> If someone wants to maintain v21.05 in the future, he can just look for
> the Fixes tag in the git history.
>
> Thanks,
> Maxime
I asked Thomas and Ferruh this question to make sure we are all aligned. Seems
they think we'd better add it in this case. Thomas's two reasons:
- we don't know in advance whether a branch will be maintained
- it helps those maintaining a private stable branch
And my understanding is adding both fix tag and stable tag makes it clearer for
stable release maintainers (They can just ignore 'only fix tag' case). And anyway
they need to check the fix commit ID.
Anyway, I could add it with some small changes David asked for if you don’t plan
a new version. Do you?
Thanks,
Chenbo
>
> > Thanks,
> > Chenbo
> >
> >>
> >> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> >> ---
> >> lib/vhost/vhost_user.c | 10 ++++++++++
> >> 1 file changed, 10 insertions(+)
> >>
> >> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
> >> index 5fb055ea2e..82adf80fe5 100644
> >> --- a/lib/vhost/vhost_user.c
> >> +++ b/lib/vhost/vhost_user.c
> >> @@ -545,6 +545,16 @@ numa_realloc(struct virtio_net *dev, int index)
> >> vq->batch_copy_elems = new_batch_copy_elems;
> >> }
> >>
> >> + if (vq->log_cache) {
> >> + struct log_cache_entry *log_cache;
> >> +
> >> + log_cache = rte_realloc_socket(vq->log_cache,
> >> + sizeof(struct log_cache_entry) *
> >> VHOST_LOG_CACHE_NR,
> >> + 0, newnode);
> >> + if (log_cache)
> >> + vq->log_cache = log_cache;
> >> + }
> >> +
> >> rte_free(old_vq);
> >> }
> >>
> >> --
> >> 2.31.1
> >
@@ -545,6 +545,16 @@ numa_realloc(struct virtio_net *dev, int index)
vq->batch_copy_elems = new_batch_copy_elems;
}
+ if (vq->log_cache) {
+ struct log_cache_entry *log_cache;
+
+ log_cache = rte_realloc_socket(vq->log_cache,
+ sizeof(struct log_cache_entry) * VHOST_LOG_CACHE_NR,
+ 0, newnode);
+ if (log_cache)
+ vq->log_cache = log_cache;
+ }
+
rte_free(old_vq);
}