From patchwork Tue Jan 28 16:16:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Junxiao Shi X-Patchwork-Id: 65270 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C1664A04B3; Tue, 28 Jan 2020 17:42:33 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 13FD91D53C; Tue, 28 Jan 2020 17:42:33 +0100 (CET) Received: from mails2n0-route0.email.arizona.edu (mails2n0-route0.email.arizona.edu [128.196.130.122]) by dpdk.org (Postfix) with ESMTP id 546611D537 for ; Tue, 28 Jan 2020 17:42:28 +0100 (CET) IronPort-SDR: NFigLkQ9HKEAnAql04LUQNo4jkHifqyzHXQBUCejmef9+/PTCr2t21XS/MZ3b4wNjJ5z/0kXK/ QdxaYOCDh/bQ== IronPort-PHdr: 9a23:JL3ycx0YIqvYW+E3smDT+DRfVm0co7zxezQtwd8ZseITKfad9pjvdHbS+e9qxAeQG9mCt7Qf16GG7uigATVGvc/a9ihaMdRlbFwssY0uhQsuAcqIWwXQDcXBSGgEJvlET0Jv5HqhMEJYS47UblzWpWCuv3ZJQk2sfQV6Kf7oFYHMks+5y/69+4HJYwVPmTGxfa5+IA+5oAnMucQam4VvJ6k+xhbKonZDZuBayX91KV6JkBvw+8m98IR//yhMvv4q6tJNX7j9c6kkV7JTES4oM3oy5M3ltBnDSRWA634BWWgIkRRGHhbI4gjiUpj+riX1uOx92DKHPcLtVrA7RS6i76ZwRxD2jioMKiM0/3vWisx0i6JbvQ6hqhliyIPafI2ZKPxzdb7bcNgHR2ROQ9xRWjRDDYOyb4UBAfcPM/hDooXhu1cCsQeyCRWwCO/zzDJDm3/43bc90+QkCQzI2BEgH9QBsXTVttr1M7oZX/21zKnJ0D7OaPxW2Tbh6IXTdxAuu/CMXbZtfcbLyUgvEB7Fjk+LqYz4OTOV0OQMv3KH4OpnUOKikmgqoBx/rDiow8cjkIjJhoQNx1/Z7SV23YU1Kce/SE5hbt6pFoZbuSKCN4ZuX88vR39ktDwnxrEYo5K2fSwHxI4nyhLHdvCKd5aE7gj+WOqPOzt1hXZodKiiixux8EWtzPD3WNOu31ZQtCVFl8HBtnUK1xPO9MeKUuB9/kK92TaX0ADT9/1ELVg0laXFL54hxaY9lp0LsUTfHy/2nV/5gLWKeUU/4OSn8f7nYrTnp5+bMY97kBvyPbozlcyiGeg4Mw4OUHaH+emkybHu+VP1TK9XgvEoiKXUsZTXKd4bq6KhBg9ayIcj6xKxDze819QYmGEKLElFeBKGjYjpJlfOIf/5DfilhFShiytkx+vcMrL7GJXNL3nCkLDnfbln8UJT1hc8zc1H65JOFr4BOO7zWlP2tNHAFR85Lgq0w+DnCdV5zIweX3+AAq+CPaPdtF+H/OMvI+2WaIAJvzb9LuAv5+Tygn8hhV8dYa6p0IMKZ3+mAPRrOFiWYWLxjdcCC2sKuRA+TOO5wGGFBDJeeV6+R64/oDYhB9GIF4DGE6KwhqyA1SDzJZ1XLjRJC0ykFH7pc9zCSvARcjmPJMlw1DcDUO7yGMcayRiyuVqimPJcJe3O93hAuA== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DBAACQYzBejAVFDMBlHAEBAQEBBwEBEQEEBAEBgWoEAQELAYMUVjEqjReGKQFUBosmkVIBAQEMAQETEAkBAgEBhmgkNwYOAgMNAQEBBAEBAQEBBQICARMBAQEmhWcMgjsigwIGJ1IQLCUPSBmDJgGCViUPrlYzikIGgTgBjB0aeYEHhGKCZAQahzUEl3qXXoJDh0KPEJp8l0SSKQIEBgUCFAGBaCKBWTMaCBsVgydQGA2OKReIZIVfIQEyAoEEAQGNKAEB X-IPAS-Result: A2DBAACQYzBejAVFDMBlHAEBAQEBBwEBEQEEBAEBgWoEAQELAYMUVjEqjReGKQFUBosmkVIBAQEMAQETEAkBAgEBhmgkNwYOAgMNAQEBBAEBAQEBBQICARMBAQEmhWcMgjsigwIGJ1IQLCUPSBmDJgGCViUPrlYzikIGgTgBjB0aeYEHhGKCZAQahzUEl3qXXoJDh0KPEJp8l0SSKQIEBgUCFAGBaCKBWTMaCBsVgydQGA2OKReIZIVfIQEyAoEEAQGNKAEB X-IronPort-AV: E=Sophos;i="5.70,374,1574146800"; d="scan'208";a="452496390" Received: from optima.cs.arizona.edu ([192.12.69.5]) by mails2n0out.email.arizona.edu with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jan 2020 09:42:26 -0700 Received: from lectura.cs.arizona.edu (lectura.cs.arizona.edu [192.12.69.186]) by optima.cs.arizona.edu (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id 00SGgQvO030484 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 28 Jan 2020 09:42:26 -0700 Received: from lectura.cs.arizona.edu (localhost [127.0.0.1]) by lectura.cs.arizona.edu (8.14.4/8.14.4/Debian-4.1ubuntu1.1) with ESMTP id 00SGgPlu032153; Tue, 28 Jan 2020 09:42:25 -0700 Received: (from shijunxiao@localhost) by lectura.cs.arizona.edu (8.14.4/8.14.4/Submit) id 00SGgPfT032152; Tue, 28 Jan 2020 09:42:25 -0700 Message-Id: <202001281642.00SGgPfT032152@lectura.cs.arizona.edu> In-Reply-To: <201909101917.x8AJH281027807@lectura.cs.arizona.edu> References: <201909101917.x8AJH281027807@lectura.cs.arizona.edu> From: Junxiao Shi Date: Tue, 28 Jan 2020 09:16:44 -0700 To: dev@dpdk.org Cc: sunnylandh@gmail.com Subject: [dpdk-dev] [RFC PATCH v2] kni: properly translate pa2va for cloned mbuf X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, KNI kernel module uses the difference between m->buf_addr and m->buf_iova to calculate userspace virtual address from physical address. This works for direct mbufs, but does not work for indirect (cloned) mbufs that come from another mempool. Transmitting a cloned mbuf may cause segmentation fault in userspace. Now, userspace KNI library writes the virtual address of each mbuf in m->userdata field, and KNI kernel module uses this field to restore virtual address before putting mbuf into free_q. This approach works for both direct and indirect mbufs. NOTE TO REVIEWER - DO NOT MERGE The idea of this change is at https://bugs.dpdk.org/show_bug.cgi?id=183#c4 Test case is at https://bugs.dpdk.org/show_bug.cgi?id=183#c5 I only modified kni_net_rx_normal function. If this approach is acceptable, I will modify kni_net_rx_lo_fifo, kni_net_rx_lo_fifo_skb, and kni_fifo_trans_pa2va(rx_q) as well. Bugzilla ID: 183 Signed-off-by: Junxiao Shi --- kernel/linux/kni/kni_net.c | 4 ++-- lib/librte_eal/linux/eal/include/rte_kni_common.h | 3 ++- lib/librte_kni/rte_kni.c | 8 ++++++-- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c index 97fe85b..d783545 100644 --- a/kernel/linux/kni/kni_net.c +++ b/kernel/linux/kni/kni_net.c @@ -377,7 +377,7 @@ kni_net_rx_normal(struct kni_dev *kni) kva = get_kva(kni, kni->pa[i]); len = kva->pkt_len; data_kva = get_data_kva(kni, kva); - kni->va[i] = pa2va(kni->pa[i], kva); + kni->va[i] = kva->va; skb = netdev_alloc_skb(dev, len); if (!skb) { @@ -403,7 +403,7 @@ kni_net_rx_normal(struct kni_dev *kni) kva = pa2kva(kva->next); data_kva = kva2data_kva(kva); /* Convert physical address to virtual address */ - prev_kva->next = pa2va(prev_kva->next, kva); + prev_kva->next = kva->va; } } diff --git a/lib/librte_eal/linux/eal/include/rte_kni_common.h b/lib/librte_eal/linux/eal/include/rte_kni_common.h index 7313ef5..c694a1d 100644 --- a/lib/librte_eal/linux/eal/include/rte_kni_common.h +++ b/lib/librte_eal/linux/eal/include/rte_kni_common.h @@ -86,7 +86,8 @@ struct rte_kni_mbuf { uint16_t data_len; /**< Amount of data in segment buffer. */ /* fields on second cache line */ - char pad3[8] __attribute__((__aligned__(RTE_CACHE_LINE_MIN_SIZE))); + void *va __attribute__((__aligned__(RTE_CACHE_LINE_MIN_SIZE))); + /**< Virtual address of this mbuf in userspace (overwrites userdata). */ void *pool; void *next; /**< Physical address of next mbuf in kernel. */ }; diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index e388751..463485f 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -359,13 +359,15 @@ va2pa(struct rte_mbuf *m) static void * va2pa_all(struct rte_mbuf *mbuf) { - void *phy_mbuf = va2pa(mbuf); + void *phy_mbuf = (void*)rte_mempool_virt2iova(mbuf); struct rte_mbuf *next = mbuf->next; while (next) { - mbuf->next = va2pa(next); + mbuf->userdata = mbuf; + mbuf->next = (void*)rte_mempool_virt2iova(next); mbuf = next; next = mbuf->next; } + mbuf->userdata = mbuf; return phy_mbuf; } @@ -652,6 +654,8 @@ kni_allocate_mbufs(struct rte_kni *kni) offsetof(struct rte_kni_mbuf, buf_addr)); RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, next) != offsetof(struct rte_kni_mbuf, next)); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, userdata) != + offsetof(struct rte_kni_mbuf, va)); RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != offsetof(struct rte_kni_mbuf, data_off)); RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=