From patchwork Tue Apr 24 10:19:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatoly Burakov X-Patchwork-Id: 38792 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1E0C62A66; Tue, 24 Apr 2018 12:19:36 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id F0DFA2B92 for ; Tue, 24 Apr 2018 12:19:31 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Apr 2018 03:19:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,322,1520924400"; d="scan'208";a="218945518" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by orsmga005.jf.intel.com with ESMTP; 24 Apr 2018 03:19:25 -0700 Received: from sivswdev01.ir.intel.com (sivswdev01.ir.intel.com [10.237.217.45]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id w3OAJPHI001390; Tue, 24 Apr 2018 11:19:25 +0100 Received: from sivswdev01.ir.intel.com (localhost [127.0.0.1]) by sivswdev01.ir.intel.com with ESMTP id w3OAJPdk024936; Tue, 24 Apr 2018 11:19:25 +0100 Received: (from aburakov@localhost) by sivswdev01.ir.intel.com with LOCAL id w3OAJPUR024931; Tue, 24 Apr 2018 11:19:25 +0100 From: Anatoly Burakov To: dev@dpdk.org Cc: reshma.pattan@intel.com Date: Tue, 24 Apr 2018 11:19:24 +0100 Message-Id: X-Mailer: git-send-email 1.7.0.7 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v3 3/3] mem: improve autodetection of hugepage counts on 32-bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" For non-legacy mode, we are preallocating space for hugepages, so we know in advance which pages we will be able to allocate, and which we won't. However, the init procedure was using hugepage counts gathered from sysfs and paid no attention to hugepage sizes that were actually available for reservation, and failed on attempts to reserve unavailable pages. Fix this by limiting total page counts by number of pages actually preallocated. Also, VA preallocate procedure only looks at mountpoints that are available, and expects pages to exist if a mountpoint exists. That might not necessarily be the case, so also check if there are hugepages available for a particular page size on a particular NUMA node. Signed-off-by: Anatoly Burakov Tested-by: Jananee Parthasarathy --- Notes: v3: - Added a check to see if pages are actually available before reserving VA space for them. Only applies to 32-bit non-legacy. lib/librte_eal/common/eal_common_memory.c | 4 ++++ lib/librte_eal/linuxapp/eal/eal_memory.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/lib/librte_eal/common/eal_common_memory.c b/lib/librte_eal/common/eal_common_memory.c index d819abe..991979c 100644 --- a/lib/librte_eal/common/eal_common_memory.c +++ b/lib/librte_eal/common/eal_common_memory.c @@ -329,6 +329,10 @@ memseg_primary_init_32(void) hpi = &internal_config.hugepage_info[hpi_idx]; hugepage_sz = hpi->hugepage_sz; + /* check if pages are actually available */ + if (hpi->num_pages[socket_id] == 0) + continue; + max_segs = RTE_MAX_MEMSEG_PER_TYPE; max_pagesz_mem = max_socket_mem - cur_socket_mem; diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c index fadc1de..6f2cdf8 100644 --- a/lib/librte_eal/linuxapp/eal/eal_memory.c +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c @@ -1603,6 +1603,18 @@ eal_legacy_hugepage_init(void) return -1; } +static int __rte_unused +hugepage_count_walk(const struct rte_memseg_list *msl, void *arg) +{ + struct hugepage_info *hpi = arg; + + if (msl->page_sz != hpi->hugepage_sz) + return 0; + + hpi->num_pages[msl->socket_id] += msl->memseg_arr.len; + return 0; +} + static int eal_hugepage_init(void) { @@ -1617,10 +1629,29 @@ eal_hugepage_init(void) for (hp_sz_idx = 0; hp_sz_idx < (int) internal_config.num_hugepage_sizes; hp_sz_idx++) { +#ifndef RTE_ARCH_64 + struct hugepage_info dummy; + unsigned int i; +#endif /* also initialize used_hp hugepage sizes in used_hp */ struct hugepage_info *hpi; hpi = &internal_config.hugepage_info[hp_sz_idx]; used_hp[hp_sz_idx].hugepage_sz = hpi->hugepage_sz; + +#ifndef RTE_ARCH_64 + /* for 32-bit, limit number of pages on socket to whatever we've + * preallocated, as we cannot allocate more. + */ + memset(&dummy, 0, sizeof(dummy)); + dummy.hugepage_sz = hpi->hugepage_sz; + if (rte_memseg_list_walk(hugepage_count_walk, &dummy) < 0) + return -1; + + for (i = 0; i < RTE_DIM(dummy.num_pages); i++) { + hpi->num_pages[i] = RTE_MIN(hpi->num_pages[i], + dummy.num_pages[i]); + } +#endif } /* make a copy of socket_mem, needed for balanced allocation. */