Message ID | 1535642492-21831-1-git-send-email-alejandro.lucero@netronome.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BBEF14D27; Thu, 30 Aug 2018 17:22:29 +0200 (CEST) Received: from netronome.com (host-79-78-33-110.static.as9105.net [79.78.33.110]) by dpdk.org (Postfix) with ESMTP id 7D1E94CBB; Thu, 30 Aug 2018 17:22:27 +0200 (CEST) Received: from netronome.com (localhost [127.0.0.1]) by netronome.com (8.14.4/8.14.4/Debian-4.1ubuntu1) with ESMTP id w7UFLY7d021907; Thu, 30 Aug 2018 16:21:34 +0100 Received: (from alucero@localhost) by netronome.com (8.14.4/8.14.4/Submit) id w7UFLYh6021906; Thu, 30 Aug 2018 16:21:34 +0100 From: Alejandro Lucero <alejandro.lucero@netronome.com> To: dev@dpdk.org Cc: stable@dpdk.org Date: Thu, 30 Aug 2018 16:21:27 +0100 Message-Id: <1535642492-21831-1-git-send-email-alejandro.lucero@netronome.com> X-Mailer: git-send-email 1.9.1 Subject: [dpdk-dev] [PATH 0/5] use IOVAs check based on DMA mask X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
use IOVAs check based on DMA mask
|
|
Message
Alejandro Lucero
Aug. 30, 2018, 3:21 p.m. UTC
I sent a patchset about this to be applied on 17.11 stable. The memory code has had main changes since that version, so here it is the patchset adjusted to current master repo. This patchset adds, mainly, a check for ensuring IOVAs are within a restricted range due to addressing limitations with some devices. There are two known cases: NFP and IOMMU VT-d emulation. With this check IOVAs out of range are detected and PMDs can abort initialization. For the VT-d case, IOVA VA mode is allowed as long as IOVAs are within the supported range, avoiding to forbid IOVA VA by default. For the addressing limitations known cases, there are just 40(NFP) or 39(VT-d) bits for handling IOVAs. When using IOVA PA, those limitations imply 1TB(NFP) or 512M(VT-d) as upper limits, which is likely enough for most systems. With machines using more memory, the added check will ensure IOVAs within the range. With IOVA VA, and because the way the Linux kernel serves mmap calls in 64 bits systems, 39 or 40 bits are not enough. It is possible to give an address hint with a lower starting address than the default one used by the kernel, and then ensuring the mmap uses that hint or hint plus some offset. With 64 bits systems, the process virtual address space is large enoguh for doing the hugepages mmaping within the supported range when those addressing limitations exist. This patchset also adds a change for using such a hint making the use of IOVA VA a more than likely possibility when there are those addressing limitations. The check is not done by default but just when it is required. This patchset adds the check for NFP initialization and for setting the IOVA mode is an emulated VT-d is detected. Also, because the recent patchset adding dynamic memory allocation, the check is also invoked for ensuring the new memsegs are within the required range. This patchset could be applied to stable 18.05.
Comments
There is a problem with the patch due to a last-hour change regarding MAX_DMA_MASK_BITS. It turns out initial X86_VA_WIDTH definition was to 47 bits, but it is not enough in some systems leading to IOVA VA not available. Because MAX_DMA_MASK_BITS was just used for a sanity check about the mask length, it is harmless to increment the value to 63 for 64-bits systems, but I was really clumsy doing the change and it does not compile now. I've already set the patches as not applicable at patchwork and I will send another version fixing that issue. On Thu, Aug 30, 2018 at 5:21 PM, Alejandro Lucero < alejandro.lucero@netronome.com> wrote: > I sent a patchset about this to be applied on 17.11 stable. The memory > code has had main changes since that version, so here it is the patchset > adjusted to current master repo. > > This patchset adds, mainly, a check for ensuring IOVAs are within a > restricted range due to addressing limitations with some devices. There > are two known cases: NFP and IOMMU VT-d emulation. > > With this check IOVAs out of range are detected and PMDs can abort > initialization. For the VT-d case, IOVA VA mode is allowed as long as > IOVAs are within the supported range, avoiding to forbid IOVA VA by > default. > > For the addressing limitations known cases, there are just 40(NFP) or > 39(VT-d) bits for handling IOVAs. When using IOVA PA, those limitations > imply 1TB(NFP) or 512M(VT-d) as upper limits, which is likely enough for > most systems. With machines using more memory, the added check will > ensure IOVAs within the range. > > With IOVA VA, and because the way the Linux kernel serves mmap calls > in 64 bits systems, 39 or 40 bits are not enough. It is possible to > give an address hint with a lower starting address than the default one > used by the kernel, and then ensuring the mmap uses that hint or hint plus > some offset. With 64 bits systems, the process virtual address space is > large enoguh for doing the hugepages mmaping within the supported range > when those addressing limitations exist. This patchset also adds a change > for using such a hint making the use of IOVA VA a more than likely > possibility when there are those addressing limitations. > > The check is not done by default but just when it is required. This > patchset adds the check for NFP initialization and for setting the IOVA > mode is an emulated VT-d is detected. Also, because the recent patchset > adding dynamic memory allocation, the check is also invoked for ensuring > the new memsegs are within the required range. > > This patchset could be applied to stable 18.05. > >