[dpdk-dev,v4] kni: fix possible rx_q mbuf leaks and speed up alloc_q release
Checks
Commit Message
rx_q fifo can only be released by kernel thread. There may be
mbuf leaks in rx_q because kernel threads are randomly stopped.
When the kni is released and netdev is unregisterd, convert the
physical address mbufs in rx_q to the virtual address in free_q.
By the way, alloc_q can be processed together to speed up the
release rate in userspace.
In my test, it is improved from 300-500ms with a mempool that has
131072 mbufs to 10ms(regardless of the specifications).
Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
Suggested-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
v4:
* Add improve performance description.
---
kernel/linux/kni/kni_dev.h | 1 +
kernel/linux/kni/kni_misc.c | 2 ++
kernel/linux/kni/kni_net.c | 40 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 43 insertions(+), 0 deletions(-)
Comments
On 4/19/2018 4:12 AM, Yangchao Zhou wrote:
> rx_q fifo can only be released by kernel thread. There may be
> mbuf leaks in rx_q because kernel threads are randomly stopped.
>
> When the kni is released and netdev is unregisterd, convert the
> physical address mbufs in rx_q to the virtual address in free_q.
> By the way, alloc_q can be processed together to speed up the
> release rate in userspace.
>
> In my test, it is improved from 300-500ms with a mempool that has
> 131072 mbufs to 10ms(regardless of the specifications).
>
> Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
> Suggested-by: Ferruh Yigit <ferruh.yigit@intel.com>
> ---
> v4:
> * Add improve performance description.
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
04/05/2018 14:46, Ferruh Yigit:
> On 4/19/2018 4:12 AM, Yangchao Zhou wrote:
> > rx_q fifo can only be released by kernel thread. There may be
> > mbuf leaks in rx_q because kernel threads are randomly stopped.
> >
> > When the kni is released and netdev is unregisterd, convert the
> > physical address mbufs in rx_q to the virtual address in free_q.
> > By the way, alloc_q can be processed together to speed up the
> > release rate in userspace.
> >
> > In my test, it is improved from 300-500ms with a mempool that has
> > 131072 mbufs to 10ms(regardless of the specifications).
> >
> > Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
> > Suggested-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > ---
> > v4:
> > * Add improve performance description.
>
> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Applied, thanks
@@ -92,6 +92,7 @@ struct kni_dev {
void *alloc_va[MBUF_BURST_SZ];
};
+void kni_net_release_fifo_phy(struct kni_dev *kni);
void kni_net_rx(struct kni_dev *kni);
void kni_net_init(struct net_device *dev);
void kni_net_config_lo_mode(char *lo_str);
@@ -192,6 +192,8 @@ struct kni_net {
free_netdev(dev->net_dev);
}
+ kni_net_release_fifo_phy(dev);
+
return 0;
}
@@ -163,6 +163,46 @@
return (ret == 0) ? req.result : ret;
}
+static void
+kni_fifo_trans_pa2va(struct kni_dev *kni,
+ struct rte_kni_fifo *src_pa, struct rte_kni_fifo *dst_va)
+{
+ uint32_t ret, i, num_dst, num_rx;
+ void *kva;
+ do {
+ num_dst = kni_fifo_free_count(dst_va);
+ if (num_dst == 0)
+ return;
+
+ num_rx = min_t(uint32_t, num_dst, MBUF_BURST_SZ);
+
+ num_rx = kni_fifo_get(src_pa, kni->pa, num_rx);
+ if (num_rx == 0)
+ return;
+
+ for (i = 0; i < num_rx; i++) {
+ kva = pa2kva(kni->pa[i]);
+ kni->va[i] = pa2va(kni->pa[i], kva);
+ }
+
+ ret = kni_fifo_put(dst_va, kni->va, num_rx);
+ if (ret != num_rx) {
+ /* Failing should not happen */
+ pr_err("Fail to enqueue entries into dst_va\n");
+ return;
+ }
+ } while (1);
+}
+
+/* Try to release mbufs when kni release */
+void kni_net_release_fifo_phy(struct kni_dev *kni)
+{
+ /* release rx_q first, because it can't release in userspace */
+ kni_fifo_trans_pa2va(kni, kni->rx_q, kni->free_q);
+ /* release alloc_q for speeding up kni release in userspace */
+ kni_fifo_trans_pa2va(kni, kni->alloc_q, kni->free_q);
+}
+
/*
* Configuration changes (passed on by ifconfig)
*/