[dpdk-dev,v3,1/6] net/ixgbevf: unregister irq handler when other interrupts not allowed.
Checks
Commit Message
From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
When bind the ixgbe VF (e.g 82599 card) to igb_uio and enable the
rx-interrupt, there will be more than one epoll_wait on intr_handle.fd.
One is in "eal-intr-thread" thread, and the others are in the thread
which call the "rte_epoll_wait". The problem is that sometimes
"eal-intr-thread" thread will process the rx interrupt, and then
rte_epoll_wait can't get the event anymore, and the packets may be lost.
The patch unregister the status interrupt handler in "eal-intr-thread"
thread and the ixgbe pf is in the same case.
Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>
---
drivers/net/ixgbe/ixgbe_ethdev.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
Comments
> -----Original Message-----
> From: xiangxia.m.yue@gmail.com [mailto:xiangxia.m.yue@gmail.com]
> Sent: Sunday, January 14, 2018 6:04 PM
> To: Dai, Wei <wei.dai@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Tonghao Zhang <xiangxia.m.yue@gmail.com>
> Subject: [PATCH v3 1/6] net/ixgbevf: unregister irq handler when other
> interrupts not allowed.
>
> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>
> When bind the ixgbe VF (e.g 82599 card) to igb_uio and enable the
> rx-interrupt, there will be more than one epoll_wait on intr_handle.fd.
> One is in "eal-intr-thread" thread, and the others are in the thread which call
> the "rte_epoll_wait". The problem is that sometimes "eal-intr-thread" thread
> will process the rx interrupt, and then rte_epoll_wait can't get the event
> anymore, and the packets may be lost.
>
> The patch unregister the status interrupt handler in "eal-intr-thread"
> thread and the ixgbe pf is in the same case.
>
> Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> Acked-by: Beilei Xing <beilei.xing@intel.com>
In my feedback to your v2 version of this patch, I mentioned miscellaneous interrupt
need to be handled in eal-intr-thread.
Your patch removes the handler of PF->VF reset event and leads to missing of
this event.
My suggestion is still using VFIO-PCI instead of igb_uio for rx-queue interrupt.
> ---
> drivers/net/ixgbe/ixgbe_ethdev.c | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> b/drivers/net/ixgbe/ixgbe_ethdev.c
> index ff19a56..0e056a2 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -5078,6 +5078,15 @@ static int
> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
> }
> ixgbevf_configure_msix(dev);
>
> + if (!rte_intr_allow_others(intr_handle)) {
> + rte_intr_callback_unregister(intr_handle,
> + ixgbevf_dev_interrupt_handler,
> + dev);
> + if (dev->data->dev_conf.intr_conf.lsc != 0)
> + PMD_INIT_LOG(INFO, "lsc won't enable because of"
> + " no intr multiplex");
> + }
> +
> /* When a VF port is bound to VFIO-PCI, only miscellaneous interrupt
> * is mapped to VFIO vector 0 in eth_ixgbevf_dev_init( ).
> * If previous VFIO interrupt mapping setting in eth_ixgbevf_dev_init( )
> @@ -5120,6 +5129,12 @@ static int
> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
>
> ixgbe_dev_clear_queues(dev);
>
> + if (!rte_intr_allow_others(intr_handle))
> + /* resume to the default handler */
> + rte_intr_callback_register(intr_handle,
> + ixgbevf_dev_interrupt_handler,
> + (void *)dev);
> +
> /* Clean datapath event and queue/vec mapping */
> rte_intr_efd_disable(intr_handle);
> if (intr_handle->intr_vec != NULL) {
> --
> 1.8.3.1
On Mon, Jan 15, 2018 at 12:57 PM, Dai, Wei <wei.dai@intel.com> wrote:
>> -----Original Message-----
>> From: xiangxia.m.yue@gmail.com [mailto:xiangxia.m.yue@gmail.com]
>> Sent: Sunday, January 14, 2018 6:04 PM
>> To: Dai, Wei <wei.dai@intel.com>; Xing, Beilei <beilei.xing@intel.com>
>> Cc: dev@dpdk.org; Tonghao Zhang <xiangxia.m.yue@gmail.com>
>> Subject: [PATCH v3 1/6] net/ixgbevf: unregister irq handler when other
>> interrupts not allowed.
>>
>> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>>
>> When bind the ixgbe VF (e.g 82599 card) to igb_uio and enable the
>> rx-interrupt, there will be more than one epoll_wait on intr_handle.fd.
>> One is in "eal-intr-thread" thread, and the others are in the thread which call
>> the "rte_epoll_wait". The problem is that sometimes "eal-intr-thread" thread
>> will process the rx interrupt, and then rte_epoll_wait can't get the event
>> anymore, and the packets may be lost.
>>
>> The patch unregister the status interrupt handler in "eal-intr-thread"
>> thread and the ixgbe pf is in the same case.
>>
>> Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>> Acked-by: Beilei Xing <beilei.xing@intel.com>
> In my feedback to your v2 version of this patch, I mentioned miscellaneous interrupt
> need to be handled in eal-intr-thread.
> Your patch removes the handler of PF->VF reset event and leads to missing of
> this event.
> My suggestion is still using VFIO-PCI instead of igb_uio for rx-queue interrupt.
I got it, Thanks for your review.
>
>> ---
>> drivers/net/ixgbe/ixgbe_ethdev.c | 15 +++++++++++++++
>> 1 file changed, 15 insertions(+)
>>
>> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
>> b/drivers/net/ixgbe/ixgbe_ethdev.c
>> index ff19a56..0e056a2 100644
>> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
>> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
>> @@ -5078,6 +5078,15 @@ static int
>> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
>> }
>> ixgbevf_configure_msix(dev);
>>
>> + if (!rte_intr_allow_others(intr_handle)) {
>> + rte_intr_callback_unregister(intr_handle,
>> + ixgbevf_dev_interrupt_handler,
>> + dev);
>> + if (dev->data->dev_conf.intr_conf.lsc != 0)
>> + PMD_INIT_LOG(INFO, "lsc won't enable because of"
>> + " no intr multiplex");
>> + }
>> +
>> /* When a VF port is bound to VFIO-PCI, only miscellaneous interrupt
>> * is mapped to VFIO vector 0 in eth_ixgbevf_dev_init( ).
>> * If previous VFIO interrupt mapping setting in eth_ixgbevf_dev_init( )
>> @@ -5120,6 +5129,12 @@ static int
>> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
>>
>> ixgbe_dev_clear_queues(dev);
>>
>> + if (!rte_intr_allow_others(intr_handle))
>> + /* resume to the default handler */
>> + rte_intr_callback_register(intr_handle,
>> + ixgbevf_dev_interrupt_handler,
>> + (void *)dev);
>> +
>> /* Clean datapath event and queue/vec mapping */
>> rte_intr_efd_disable(intr_handle);
>> if (intr_handle->intr_vec != NULL) {
>> --
>> 1.8.3.1
>
On Mon, Jan 15, 2018 at 4:57 PM, Tonghao Zhang <xiangxia.m.yue@gmail.com> wrote:
> On Mon, Jan 15, 2018 at 12:57 PM, Dai, Wei <wei.dai@intel.com> wrote:
>>> -----Original Message-----
>>> From: xiangxia.m.yue@gmail.com [mailto:xiangxia.m.yue@gmail.com]
>>> Sent: Sunday, January 14, 2018 6:04 PM
>>> To: Dai, Wei <wei.dai@intel.com>; Xing, Beilei <beilei.xing@intel.com>
>>> Cc: dev@dpdk.org; Tonghao Zhang <xiangxia.m.yue@gmail.com>
>>> Subject: [PATCH v3 1/6] net/ixgbevf: unregister irq handler when other
>>> interrupts not allowed.
>>>
>>> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>>>
>>> When bind the ixgbe VF (e.g 82599 card) to igb_uio and enable the
>>> rx-interrupt, there will be more than one epoll_wait on intr_handle.fd.
>>> One is in "eal-intr-thread" thread, and the others are in the thread which call
>>> the "rte_epoll_wait". The problem is that sometimes "eal-intr-thread" thread
>>> will process the rx interrupt, and then rte_epoll_wait can't get the event
>>> anymore, and the packets may be lost.
>>>
>>> The patch unregister the status interrupt handler in "eal-intr-thread"
>>> thread and the ixgbe pf is in the same case.
>>>
>>> Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>>> Acked-by: Beilei Xing <beilei.xing@intel.com>
>> In my feedback to your v2 version of this patch, I mentioned miscellaneous interrupt
>> need to be handled in eal-intr-thread.
>> Your patch removes the handler of PF->VF reset event and leads to missing of
>> this event.
>> My suggestion is still using VFIO-PCI instead of igb_uio for rx-queue interrupt.
> I got it, Thanks for your review.
Hi wei, beilei,
The patches in the patchset have no dependency. Can you help me to
review other patches ?
>>
>>> ---
>>> drivers/net/ixgbe/ixgbe_ethdev.c | 15 +++++++++++++++
>>> 1 file changed, 15 insertions(+)
>>>
>>> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
>>> b/drivers/net/ixgbe/ixgbe_ethdev.c
>>> index ff19a56..0e056a2 100644
>>> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
>>> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
>>> @@ -5078,6 +5078,15 @@ static int
>>> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
>>> }
>>> ixgbevf_configure_msix(dev);
>>>
>>> + if (!rte_intr_allow_others(intr_handle)) {
>>> + rte_intr_callback_unregister(intr_handle,
>>> + ixgbevf_dev_interrupt_handler,
>>> + dev);
>>> + if (dev->data->dev_conf.intr_conf.lsc != 0)
>>> + PMD_INIT_LOG(INFO, "lsc won't enable because of"
>>> + " no intr multiplex");
>>> + }
>>> +
>>> /* When a VF port is bound to VFIO-PCI, only miscellaneous interrupt
>>> * is mapped to VFIO vector 0 in eth_ixgbevf_dev_init( ).
>>> * If previous VFIO interrupt mapping setting in eth_ixgbevf_dev_init( )
>>> @@ -5120,6 +5129,12 @@ static int
>>> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
>>>
>>> ixgbe_dev_clear_queues(dev);
>>>
>>> + if (!rte_intr_allow_others(intr_handle))
>>> + /* resume to the default handler */
>>> + rte_intr_callback_register(intr_handle,
>>> + ixgbevf_dev_interrupt_handler,
>>> + (void *)dev);
>>> +
>>> /* Clean datapath event and queue/vec mapping */
>>> rte_intr_efd_disable(intr_handle);
>>> if (intr_handle->intr_vec != NULL) {
>>> --
>>> 1.8.3.1
>>
Hi Tonghao, Xiangxia
In general, I have several comments for your patch sets.
1. As there are 6 patches in your patch set, so a cover letter is needed.
2. You have send several version of patch sets, so '--in-reply-to' is needed to
make sure all can see the history version together. Please follow the guidance
on dpdk.org to contribute patches.
3. I saw one ACKs for one of your patches. You still need to get ACKs for all of
your patch set. Please address the comments, and ask maintainers to help directly.
4. As today is the integration deadline, I don't know if canbe applied in 18.02. Anyway
I will apply it as early as possibel whenever everything is ready and reasonable.
Thank you very much for your contribution!
Regards,
Helin
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of
> xiangxia.m.yue@gmail.com
> Sent: Sunday, January 14, 2018 6:04 PM
> To: Dai, Wei; Xing, Beilei
> Cc: dev@dpdk.org; Tonghao Zhang
> Subject: [dpdk-dev] [PATCH v3 1/6] net/ixgbevf: unregister irq handler when
> other interrupts not allowed.
>
> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>
> When bind the ixgbe VF (e.g 82599 card) to igb_uio and enable the rx-interrupt,
> there will be more than one epoll_wait on intr_handle.fd.
> One is in "eal-intr-thread" thread, and the others are in the thread which call
> the "rte_epoll_wait". The problem is that sometimes "eal-intr-thread" thread
> will process the rx interrupt, and then rte_epoll_wait can't get the event
> anymore, and the packets may be lost.
>
> The patch unregister the status interrupt handler in "eal-intr-thread"
> thread and the ixgbe pf is in the same case.
>
> Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> Acked-by: Beilei Xing <beilei.xing@intel.com>
> ---
> drivers/net/ixgbe/ixgbe_ethdev.c | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> b/drivers/net/ixgbe/ixgbe_ethdev.c
> index ff19a56..0e056a2 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -5078,6 +5078,15 @@ static int
> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
> }
> ixgbevf_configure_msix(dev);
>
> + if (!rte_intr_allow_others(intr_handle)) {
> + rte_intr_callback_unregister(intr_handle,
> + ixgbevf_dev_interrupt_handler,
> + dev);
> + if (dev->data->dev_conf.intr_conf.lsc != 0)
> + PMD_INIT_LOG(INFO, "lsc won't enable because of"
> + " no intr multiplex");
> + }
> +
> /* When a VF port is bound to VFIO-PCI, only miscellaneous interrupt
> * is mapped to VFIO vector 0 in eth_ixgbevf_dev_init( ).
> * If previous VFIO interrupt mapping setting in eth_ixgbevf_dev_init( )
> @@ -5120,6 +5129,12 @@ static int
> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
>
> ixgbe_dev_clear_queues(dev);
>
> + if (!rte_intr_allow_others(intr_handle))
> + /* resume to the default handler */
> + rte_intr_callback_register(intr_handle,
> + ixgbevf_dev_interrupt_handler,
> + (void *)dev);
> +
> /* Clean datapath event and queue/vec mapping */
> rte_intr_efd_disable(intr_handle);
> if (intr_handle->intr_vec != NULL) {
> --
> 1.8.3.1
@@ -5078,6 +5078,15 @@ static int ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
}
ixgbevf_configure_msix(dev);
+ if (!rte_intr_allow_others(intr_handle)) {
+ rte_intr_callback_unregister(intr_handle,
+ ixgbevf_dev_interrupt_handler,
+ dev);
+ if (dev->data->dev_conf.intr_conf.lsc != 0)
+ PMD_INIT_LOG(INFO, "lsc won't enable because of"
+ " no intr multiplex");
+ }
+
/* When a VF port is bound to VFIO-PCI, only miscellaneous interrupt
* is mapped to VFIO vector 0 in eth_ixgbevf_dev_init( ).
* If previous VFIO interrupt mapping setting in eth_ixgbevf_dev_init( )
@@ -5120,6 +5129,12 @@ static int ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
ixgbe_dev_clear_queues(dev);
+ if (!rte_intr_allow_others(intr_handle))
+ /* resume to the default handler */
+ rte_intr_callback_register(intr_handle,
+ ixgbevf_dev_interrupt_handler,
+ (void *)dev);
+
/* Clean datapath event and queue/vec mapping */
rte_intr_efd_disable(intr_handle);
if (intr_handle->intr_vec != NULL) {