From patchwork Thu Apr 19 03:12:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt X-Patchwork-Id: 38449 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 081EF5F4F; Thu, 19 Apr 2018 05:14:01 +0200 (CEST) Received: from mail-pl0-f66.google.com (mail-pl0-f66.google.com [209.85.160.66]) by dpdk.org (Postfix) with ESMTP id 6E7695F48; Thu, 19 Apr 2018 05:13:59 +0200 (CEST) Received: by mail-pl0-f66.google.com with SMTP id e7-v6so2346392plt.6; Wed, 18 Apr 2018 20:13:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Hk5IHh1i5VjUCnlzH40Zy84pwm7B+p497pQhGoJhPr0=; b=en/YXp/ZJBASHZrhOGGFf+8FPqfDdPoz1DiWdjb5zmWqeIomLgnd4drQLP5mDIjI1P 3HCpc5VfbpPwYoRsuTVI7RMpi9+6whrrpxybMnLOT2CXGu3+rb5kBzfWxMl4/HOrKzO4 Xty2dCZ/+ct6ANKD8/MVaVdQED62EdgJ+BehBkkFo14DR0Go0qGEy+BZpiWV6glHIwte 6cK7UuNWlvwvgaYqdmnjZgMENkk/4a3/l+qxAMv1Wtk4rMxhPD6Jm71Ihg4wFXsfoqNZ AbmE95i1gmRSpETZZK1m/EdZ1C+qB0qfd5qujhMmPhOpaj6Wy1cS9B5Q/XnlNNiCld+k 1gQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Hk5IHh1i5VjUCnlzH40Zy84pwm7B+p497pQhGoJhPr0=; b=bCvXUhA2xl7JyD+JbwuMAEudGq//G8hM+awu7D0qgNG3zm7zxYDxEHpFY8iZENDy+y hSS1PNuAVloufYVzVo9O6zfNoUbTKUhnK5jEYPuyxZc8ghQS1Xft8Qp3p9i3eLRxS3p6 ZXnMrCm8OIrMaSQQQDWnXUZZrIkwQpcOFYAYRE/r5wvHkbYfx+6WmMhHzYsJnAkVGkr/ wh73PeV/JXi6a/bgMkneTEumwpnDDI2aZNnPZ3r0vbUhV8rv93PTg3J6vC/rUaL0K5k8 J0IbdyKRZwJb42z5ME0yYsIfB9xsp1XuOY7mad5xmhm4GbumZrkpXpPle8ELT9IivKK8 f/ag== X-Gm-Message-State: ALQs6tBGs01aTtZ3UlmwHEbRrEmU+MBkUoLs+ShsZ41E6DRgmx5EWLVX 7DvCDRR4M5xCJxHuNJfDS62How== X-Google-Smtp-Source: AIpwx497jIwV6FqLrp3WS56lVNg0vgnjkOWEGZJ2a7AbdE9DgDls4+1B9MojR+m/3OmMv61R5/+XBg== X-Received: by 2002:a17:902:4d46:: with SMTP id o6-v6mr4454207plh.243.1524107638277; Wed, 18 Apr 2018 20:13:58 -0700 (PDT) Received: from localhost.localdomain (176.122.150.126.16clouds.com. [176.122.150.126]) by smtp.gmail.com with ESMTPSA id j125sm6374440pfg.188.2018.04.18.20.13.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Apr 2018 20:13:57 -0700 (PDT) From: Yangchao Zhou To: dev@dpdk.org Cc: stable@dpdk.org, ferruh.yigit@intel.com, thomas@monjalon.net Date: Thu, 19 Apr 2018 11:12:20 +0800 Message-Id: <1524107540-11544-1-git-send-email-zhouyates@gmail.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <9a57896f-a2db-7f4b-f5c4-cf04970f1636@intel.com> References: <9a57896f-a2db-7f4b-f5c4-cf04970f1636@intel.com> Subject: [dpdk-dev] [PATCH v4] kni: fix possible rx_q mbuf leaks and speed up alloc_q release X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" rx_q fifo can only be released by kernel thread. There may be mbuf leaks in rx_q because kernel threads are randomly stopped. When the kni is released and netdev is unregisterd, convert the physical address mbufs in rx_q to the virtual address in free_q. By the way, alloc_q can be processed together to speed up the release rate in userspace. In my test, it is improved from 300-500ms with a mempool that has 131072 mbufs to 10ms(regardless of the specifications). Signed-off-by: Yangchao Zhou Suggested-by: Ferruh Yigit Acked-by: Ferruh Yigit --- v4: * Add improve performance description. --- kernel/linux/kni/kni_dev.h | 1 + kernel/linux/kni/kni_misc.c | 2 ++ kernel/linux/kni/kni_net.c | 40 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 0 deletions(-) diff --git a/kernel/linux/kni/kni_dev.h b/kernel/linux/kni/kni_dev.h index c9393d8..6275ef2 100644 --- a/kernel/linux/kni/kni_dev.h +++ b/kernel/linux/kni/kni_dev.h @@ -92,6 +92,7 @@ struct kni_dev { void *alloc_va[MBUF_BURST_SZ]; }; +void kni_net_release_fifo_phy(struct kni_dev *kni); void kni_net_rx(struct kni_dev *kni); void kni_net_init(struct net_device *dev); void kni_net_config_lo_mode(char *lo_str); diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c index 01574ec..fa69f8e 100644 --- a/kernel/linux/kni/kni_misc.c +++ b/kernel/linux/kni/kni_misc.c @@ -192,6 +192,8 @@ struct kni_net { free_netdev(dev->net_dev); } + kni_net_release_fifo_phy(dev); + return 0; } diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c index 9f9b798..7fcfa10 100644 --- a/kernel/linux/kni/kni_net.c +++ b/kernel/linux/kni/kni_net.c @@ -163,6 +163,46 @@ return (ret == 0) ? req.result : ret; } +static void +kni_fifo_trans_pa2va(struct kni_dev *kni, + struct rte_kni_fifo *src_pa, struct rte_kni_fifo *dst_va) +{ + uint32_t ret, i, num_dst, num_rx; + void *kva; + do { + num_dst = kni_fifo_free_count(dst_va); + if (num_dst == 0) + return; + + num_rx = min_t(uint32_t, num_dst, MBUF_BURST_SZ); + + num_rx = kni_fifo_get(src_pa, kni->pa, num_rx); + if (num_rx == 0) + return; + + for (i = 0; i < num_rx; i++) { + kva = pa2kva(kni->pa[i]); + kni->va[i] = pa2va(kni->pa[i], kva); + } + + ret = kni_fifo_put(dst_va, kni->va, num_rx); + if (ret != num_rx) { + /* Failing should not happen */ + pr_err("Fail to enqueue entries into dst_va\n"); + return; + } + } while (1); +} + +/* Try to release mbufs when kni release */ +void kni_net_release_fifo_phy(struct kni_dev *kni) +{ + /* release rx_q first, because it can't release in userspace */ + kni_fifo_trans_pa2va(kni, kni->rx_q, kni->free_q); + /* release alloc_q for speeding up kni release in userspace */ + kni_fifo_trans_pa2va(kni, kni->alloc_q, kni->free_q); +} + /* * Configuration changes (passed on by ifconfig) */