From patchwork Mon Jul 2 17:27:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Lucero X-Patchwork-Id: 42127 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CBC181B5AE; Mon, 2 Jul 2018 19:27:22 +0200 (CEST) Received: from netronome.com (host-79-78-33-110.static.as9105.net [79.78.33.110]) by dpdk.org (Postfix) with ESMTP id 3C6D81B567; Mon, 2 Jul 2018 19:27:12 +0200 (CEST) Received: from netronome.com (localhost [127.0.0.1]) by netronome.com (8.14.4/8.14.4/Debian-4.1ubuntu1) with ESMTP id w62HR8I2032360; Mon, 2 Jul 2018 18:27:08 +0100 Received: (from alucero@localhost) by netronome.com (8.14.4/8.14.4/Submit) id w62HR8p6032359; Mon, 2 Jul 2018 18:27:08 +0100 From: Alejandro Lucero To: dev@dpdk.org Cc: stable@dpdk.org, anatoly.burakov@intel.com, maxime.coquelin@redhat.com Date: Mon, 2 Jul 2018 18:27:01 +0100 Message-Id: <1530552423-32301-5-git-send-email-alejandro.lucero@netronome.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1530552423-32301-1-git-send-email-alejandro.lucero@netronome.com> References: <1530552423-32301-1-git-send-email-alejandro.lucero@netronome.com> Subject: [dpdk-dev] [PATCH 4/6] mem: use address hint for mapping hugepages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Linux kernel uses a really high address as starting address for serving mmaps calls. If there exists addressing limitations and IOVA mode is VA, this starting address is likely too high for those devices. However, it is possible to use a lower address in the process virtual address space as with 64 bits there is a lot of available space. This patch adds an address hint as starting address for 64 bits systems. Signed-off-by: Alejandro Lucero --- lib/librte_eal/linuxapp/eal/eal_memory.c | 50 ++++++++++++++++++++++++++------ 1 file changed, 41 insertions(+), 9 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c index 17c20d4..0a134c4 100644 --- a/lib/librte_eal/linuxapp/eal/eal_memory.c +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c @@ -88,6 +88,23 @@ static uint64_t baseaddr_offset; +#ifdef RTE_ARCH_64 +/* + * Linux kernel uses a really high address as starting address for serving + * mmaps calls. If there exists addressing limitations and IOVA mode is VA, + * this starting address is likely too high for those devices. However, it + * is possible to use a lower address in the process virtual address space + * as with 64 bits there is a lot of available space. + * + * Current known limitations are 39 or 40 bits. Setting the starting address + * at 4GB implies there are 508GB or 1020GB for mapping the available + * hugepages. This is likely enough for most systems, although a device with + * addressing limitations should call rte_dev_check_dma_mask for ensuring all + * memory is within supported range. + */ +static uint64_t baseaddr = 0x100000000; +#endif + static bool phys_addrs_available = true; #define RANDOMIZE_VA_SPACE_FILE "/proc/sys/kernel/randomize_va_space" @@ -260,16 +277,10 @@ static void * get_virtual_area(size_t *size, size_t hugepage_sz) { - void *addr; + void *addr, *addr_hint; int fd; long aligned_addr; - if (internal_config.base_virtaddr != 0) { - addr = (void*) (uintptr_t) (internal_config.base_virtaddr + - baseaddr_offset); - } - else addr = NULL; - RTE_LOG(DEBUG, EAL, "Ask a virtual area of 0x%zx bytes\n", *size); fd = open("/dev/zero", O_RDONLY); @@ -278,7 +289,22 @@ return NULL; } do { - addr = mmap(addr, + if (internal_config.base_virtaddr != 0) { + addr_hint = (void *) (uintptr_t) + (internal_config.base_virtaddr + + baseaddr_offset); + } +#ifdef RTE_ARCH_64 + else { + addr_hint = (void *) (uintptr_t) (baseaddr + + baseaddr_offset); + } +#else + else { + addr_hint = NULL; + } +#endif + addr = mmap(addr_hint, (*size) + hugepage_sz, PROT_READ, #ifdef RTE_ARCH_PPC_64 MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, @@ -286,8 +312,14 @@ MAP_PRIVATE, #endif fd, 0); - if (addr == MAP_FAILED) + if (addr == MAP_FAILED) { + /* map failed. Let's try with less memory */ *size -= hugepage_sz; + } else if (addr_hint && addr != addr_hint) { + /* map not using hint. Let's try with another offset */ + addr = MAP_FAILED; + baseaddr_offset += 0x100000000; + } } while (addr == MAP_FAILED && *size > 0); if (addr == MAP_FAILED) {