[dpdk-dev,v2,1/4] net/ixgbevf: unregister irq handler when other interrupts not allowed.
Checks
Commit Message
From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
When bind the ixgbe VF (e.g 82599 card) to igb_uio and enable the
rx-interrupt, there will be more than one epoll_wait on intr_handle.fd.
One is in "eal-intr-thread" thread, and the others are in the thread
which call the "rte_epoll_wait". The problem is that sometimes
"eal-intr-thread" thread will process the rx interrupt, and then
rte_epoll_wait can't get the event anymore, and the packets may be lost.
The patch unregister the status interrupt handler in "eal-intr-thread"
thread and the ixgbe pf is in the same case.
Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>
---
drivers/net/ixgbe/ixgbe_ethdev.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
Comments
Hi, Tonghao & Beilei
The issue reported by Tonghao is caused by shortage of igb_uio.
If you want to use Rx queue interrupt in your DPDK application,
I suggest use VFIO-PCI to bind NIC port instead of igb_uio.
Currently igb_uio only support single event fd. This fd is triggered by
both miscellaneous and RX queue interrupt in ixgbe VF.
But same event fd is added to epoll fd of eal-intr-thread
and also epoll fd of normal thread for Rx task, this cause the issue.
If VFIO-PCI is used, different event fd for misc and Rx queue are created.
By the way, your patch also lead to missing of PF to VF reset interrupt.
So, I am sorry that I can't agree it now.
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of
> xiangxia.m.yue@gmail.com
> Sent: Friday, January 12, 2018 12:40 AM
> To: Xing, Beilei <beilei.xing@intel.com>; dev@dpdk.org
> Cc: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> Subject: [dpdk-dev] [PATCH v2 1/4] net/ixgbevf: unregister irq handler when
> other interrupts not allowed.
>
> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>
> When bind the ixgbe VF (e.g 82599 card) to igb_uio and enable the
> rx-interrupt, there will be more than one epoll_wait on intr_handle.fd.
> One is in "eal-intr-thread" thread, and the others are in the thread which call
> the "rte_epoll_wait". The problem is that sometimes "eal-intr-thread" thread
> will process the rx interrupt, and then rte_epoll_wait can't get the event
> anymore, and the packets may be lost.
>
> The patch unregister the status interrupt handler in "eal-intr-thread"
> thread and the ixgbe pf is in the same case.
>
> Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> Acked-by: Beilei Xing <beilei.xing@intel.com>
> ---
> drivers/net/ixgbe/ixgbe_ethdev.c | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> b/drivers/net/ixgbe/ixgbe_ethdev.c
> index ff19a56..0e056a2 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -5078,6 +5078,15 @@ static int
> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
> }
> ixgbevf_configure_msix(dev);
>
> + if (!rte_intr_allow_others(intr_handle)) {
> + rte_intr_callback_unregister(intr_handle,
> + ixgbevf_dev_interrupt_handler,
> + dev);
> + if (dev->data->dev_conf.intr_conf.lsc != 0)
> + PMD_INIT_LOG(INFO, "lsc won't enable because of"
> + " no intr multiplex");
> + }
> +
> /* When a VF port is bound to VFIO-PCI, only miscellaneous interrupt
> * is mapped to VFIO vector 0 in eth_ixgbevf_dev_init( ).
> * If previous VFIO interrupt mapping setting in eth_ixgbevf_dev_init( )
> @@ -5120,6 +5129,12 @@ static int
> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
>
> ixgbe_dev_clear_queues(dev);
>
> + if (!rte_intr_allow_others(intr_handle))
> + /* resume to the default handler */
> + rte_intr_callback_register(intr_handle,
> + ixgbevf_dev_interrupt_handler,
> + (void *)dev);
> +
> /* Clean datapath event and queue/vec mapping */
> rte_intr_efd_disable(intr_handle);
> if (intr_handle->intr_vec != NULL) {
> --
> 1.8.3.1
On Fri, Jan 12, 2018 at 4:39 PM, Dai, Wei <wei.dai@intel.com> wrote:
> Hi, Tonghao & Beilei
>
> The issue reported by Tonghao is caused by shortage of igb_uio.
> If you want to use Rx queue interrupt in your DPDK application,
> I suggest use VFIO-PCI to bind NIC port instead of igb_uio.
The performance of igb_uio is better than vfio-pic. So igb_uio is
a better solution. And the PF binded with igb_uio can use the rx queue
interrupt,
because it unregister the irq handler in eal-intr-thread via
rte_intr_callback_unregister.
Then the PF will also not handler the irq (mailbox) from VF, right ?
To support the feature of rx queue interrupt , the PF and VF binded
with igb_uio may unregister the other irq.
> Currently igb_uio only support single event fd. This fd is triggered by
> both miscellaneous and RX queue interrupt in ixgbe VF.
> But same event fd is added to epoll fd of eal-intr-thread
> and also epoll fd of normal thread for Rx task, this cause the issue.
Yes, it's right
> If VFIO-PCI is used, different event fd for misc and Rx queue are created.
> By the way, your patch also lead to missing of PF to VF reset interrupt.
> So, I am sorry that I can't agree it now.
>
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of
>> xiangxia.m.yue@gmail.com
>> Sent: Friday, January 12, 2018 12:40 AM
>> To: Xing, Beilei <beilei.xing@intel.com>; dev@dpdk.org
>> Cc: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>> Subject: [dpdk-dev] [PATCH v2 1/4] net/ixgbevf: unregister irq handler when
>> other interrupts not allowed.
>>
>> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>>
>> When bind the ixgbe VF (e.g 82599 card) to igb_uio and enable the
>> rx-interrupt, there will be more than one epoll_wait on intr_handle.fd.
>> One is in "eal-intr-thread" thread, and the others are in the thread which call
>> the "rte_epoll_wait". The problem is that sometimes "eal-intr-thread" thread
>> will process the rx interrupt, and then rte_epoll_wait can't get the event
>> anymore, and the packets may be lost.
>>
>> The patch unregister the status interrupt handler in "eal-intr-thread"
>> thread and the ixgbe pf is in the same case.
>>
>> Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>> Acked-by: Beilei Xing <beilei.xing@intel.com>
>> ---
>> drivers/net/ixgbe/ixgbe_ethdev.c | 15 +++++++++++++++
>> 1 file changed, 15 insertions(+)
>>
>> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
>> b/drivers/net/ixgbe/ixgbe_ethdev.c
>> index ff19a56..0e056a2 100644
>> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
>> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
>> @@ -5078,6 +5078,15 @@ static int
>> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
>> }
>> ixgbevf_configure_msix(dev);
>>
>> + if (!rte_intr_allow_others(intr_handle)) {
>> + rte_intr_callback_unregister(intr_handle,
>> + ixgbevf_dev_interrupt_handler,
>> + dev);
>> + if (dev->data->dev_conf.intr_conf.lsc != 0)
>> + PMD_INIT_LOG(INFO, "lsc won't enable because of"
>> + " no intr multiplex");
>> + }
>> +
>> /* When a VF port is bound to VFIO-PCI, only miscellaneous interrupt
>> * is mapped to VFIO vector 0 in eth_ixgbevf_dev_init( ).
>> * If previous VFIO interrupt mapping setting in eth_ixgbevf_dev_init( )
>> @@ -5120,6 +5129,12 @@ static int
>> ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
>>
>> ixgbe_dev_clear_queues(dev);
>>
>> + if (!rte_intr_allow_others(intr_handle))
>> + /* resume to the default handler */
>> + rte_intr_callback_register(intr_handle,
>> + ixgbevf_dev_interrupt_handler,
>> + (void *)dev);
>> +
>> /* Clean datapath event and queue/vec mapping */
>> rte_intr_efd_disable(intr_handle);
>> if (intr_handle->intr_vec != NULL) {
>> --
>> 1.8.3.1
>
@@ -5078,6 +5078,15 @@ static int ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
}
ixgbevf_configure_msix(dev);
+ if (!rte_intr_allow_others(intr_handle)) {
+ rte_intr_callback_unregister(intr_handle,
+ ixgbevf_dev_interrupt_handler,
+ dev);
+ if (dev->data->dev_conf.intr_conf.lsc != 0)
+ PMD_INIT_LOG(INFO, "lsc won't enable because of"
+ " no intr multiplex");
+ }
+
/* When a VF port is bound to VFIO-PCI, only miscellaneous interrupt
* is mapped to VFIO vector 0 in eth_ixgbevf_dev_init( ).
* If previous VFIO interrupt mapping setting in eth_ixgbevf_dev_init( )
@@ -5120,6 +5129,12 @@ static int ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
ixgbe_dev_clear_queues(dev);
+ if (!rte_intr_allow_others(intr_handle))
+ /* resume to the default handler */
+ rte_intr_callback_register(intr_handle,
+ ixgbevf_dev_interrupt_handler,
+ (void *)dev);
+
/* Clean datapath event and queue/vec mapping */
rte_intr_efd_disable(intr_handle);
if (intr_handle->intr_vec != NULL) {