From patchwork Tue Oct 20 17:04:50 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "shesha Sreenivasamurthy (shesha)" X-Patchwork-Id: 7787 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 342AB8E7A; Tue, 20 Oct 2015 19:05:13 +0200 (CEST) Received: from alln-iport-3.cisco.com (alln-iport-3.cisco.com [173.37.142.90]) by dpdk.org (Postfix) with ESMTP id 16DB45A5E for ; Tue, 20 Oct 2015 19:05:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=27047; q=dns/txt; s=iport; t=1445360711; x=1446570311; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=TpoeHtd1PGcD/Lz5vJxO+YIlThOjkIzNXDp4pXrdacg=; b=WqeghayrRdolVsbKHPDrTuitFAkxtgtdfzwbp7QimbGfayZGrNqR7Um6 JXWZwGdioRmy1Lr2zJSyAynrs2JJt7LcC0Bj9Bw7HEvJBVE1V8PVyaNuy KIZAzJK5hdAK3gSWWlryuxC1gGSQz7a24jduZ/Jwp2EkG8SYtg+wI7xiv w=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0D4AQD4cyZW/5xdJa1dgmlNgUMGuXiEIQENgVqGHgKBOzgUAQEBAQEBAX8LhC4BAQQnUhACAUAHBzIUEQIEDgWIMMQMAQEBAQEBAQECAQEBAQEBAQEBAQEYim+GEweELgWHO4cHh2IBjR6cIAEfAQFCghEdgVVyhB4kH4EGAQEB X-IronPort-AV: E=Sophos;i="5.17,707,1437436800"; d="scan'208,217";a="199416642" Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by alln-iport-3.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Oct 2015 17:05:09 +0000 Received: from XCH-ALN-004.cisco.com (xch-aln-004.cisco.com [173.36.7.14]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id t9KH59lj003345 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 20 Oct 2015 17:05:09 GMT Received: from xch-rcd-004.cisco.com (173.37.102.14) by XCH-ALN-004.cisco.com (173.36.7.14) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Tue, 20 Oct 2015 12:04:50 -0500 Received: from xch-rcd-004.cisco.com ([173.37.102.14]) by XCH-RCD-004.cisco.com ([173.37.102.14]) with mapi id 15.00.1104.000; Tue, 20 Oct 2015 12:04:50 -0500 From: "shesha Sreenivasamurthy (shesha)" To: "dev@dpdk.org" Thread-Topic: [PATCH v2] mem: command line option to delete hugepage backing files Thread-Index: AQHRC1jVXvzHm9nhFUW3n7KImjqHHJ50erYA Date: Tue, 20 Oct 2015 17:04:50 +0000 Message-ID: References: <1445335331-9050-1-git-send-email-shesha@cisco.com> In-Reply-To: <1445335331-9050-1-git-send-email-shesha@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.24.228.81] MIME-Version: 1.0 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] [PATCH v2] mem: command line option to delete hugepage backing files X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When an application using huge-pages crash or exists, the hugetlbfs backing files are not cleaned up. This is a patch to clean those files. There are multi-process DPDK applications that may be benefited by those backing files. Therefore, I have made that configurable so that the application that does not need those backing files can remove them, thus not changing the current default behavior. The application itself can clean it up, however the rationale behind DPDK cleaning it up is, DPDK created it and therefore, it is better it unlinks it. Signed-off-by: Shesha Sreenivasamurthy > --- lib/librte_eal/common/eal_common_options.c | 12 +++++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/linuxapp/eal/eal_memory.c | 39 ++++++++++++++++++++++++++++++ 4 files changed, 54 insertions(+) goto fail; } + /* free the hugepage backing files */ + if (internal_config.hugepage_unlink && + unlink_hugepage_files(tmp_hp, used_hp, + internal_config.num_hugepage_sizes) < 0) { + RTE_LOG(ERR, EAL, "Unlinking hugepage backing files failed!\n"); + goto fail; + } + /* free the temporary hugepage table */ free(tmp_hp); tmp_hp = NULL; -- 1.9.1 diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 1f459ac..5fe6374 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -79,6 +79,7 @@ eal_long_options[] = { {OPT_MASTER_LCORE, 1, NULL, OPT_MASTER_LCORE_NUM }, {OPT_NO_HPET, 0, NULL, OPT_NO_HPET_NUM }, {OPT_NO_HUGE, 0, NULL, OPT_NO_HUGE_NUM }, + {OPT_HUGE_UNLINK, 0, NULL, OPT_HUGE_UNLINK_NUM }, {OPT_NO_PCI, 0, NULL, OPT_NO_PCI_NUM }, {OPT_NO_SHCONF, 0, NULL, OPT_NO_SHCONF_NUM }, {OPT_PCI_BLACKLIST, 1, NULL, OPT_PCI_BLACKLIST_NUM }, @@ -722,6 +723,10 @@ eal_parse_common_option(int opt, const char *optarg, conf->no_hugetlbfs = 1; break; + case OPT_HUGE_UNLINK_NUM: + conf->hugepage_unlink = 1; + break; + case OPT_NO_PCI_NUM: conf->no_pci = 1; break; @@ -856,6 +861,12 @@ eal_check_common_options(struct internal_config *internal_cfg) return -1; } + if (internal_cfg->no_hugetlbfs && internal_cfg->hugepage_unlink) { + RTE_LOG(ERR, EAL, "Option --"OPT_HUGE_UNLINK" cannot " + "be specified together with --"OPT_NO_HUGE"\n"); + return -1; + } + if (rte_eal_devargs_type_count(RTE_DEVTYPE_WHITELISTED_PCI) != 0 && rte_eal_devargs_type_count(RTE_DEVTYPE_BLACKLISTED_PCI) != 0) { RTE_LOG(ERR, EAL, "Options blacklist (-b) and whitelist (-w) " @@ -906,6 +917,7 @@ eal_common_usage(void) " -h, --help This help\n" "\nEAL options for DEBUG use only:\n" " --"OPT_NO_HUGE" Use malloc instead of hugetlbfs\n" + " --"OPT_HUGE_UNLINK" Unlink hugepage backing file after initalization\n" " --"OPT_NO_PCI" Disable PCI\n" " --"OPT_NO_HPET" Disable HPET\n" " --"OPT_NO_SHCONF" No shared config (mmap'd files)\n" diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h index e2ecb0d..84b075f 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -64,6 +64,7 @@ struct internal_config { volatile unsigned force_nchannel; /**< force number of channels */ volatile unsigned force_nrank; /**< force number of ranks */ volatile unsigned no_hugetlbfs; /**< true to disable hugetlbfs */ + volatile unsigned hugepage_unlink; /** < true to unlink backing files */ volatile unsigned xen_dom0_support; /**< support app running on Xen Dom0*/ volatile unsigned no_pci; /**< true to disable PCI */ volatile unsigned no_hpet; /**< true to disable HPET */ diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index f6714d9..745f38c 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -63,6 +63,8 @@ enum { OPT_PROC_TYPE_NUM, #define OPT_NO_HPET "no-hpet" OPT_NO_HPET_NUM, +#define OPT_HUGE_UNLINK "huge-unlink" + OPT_HUGE_UNLINK_NUM, #define OPT_NO_HUGE "no-huge" OPT_NO_HUGE_NUM, #define OPT_NO_PCI "no-pci" diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c index ac2745e..2b86428 100644 --- a/lib/librte_eal/linuxapp/eal/eal_memory.c +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c @@ -786,6 +786,37 @@ copy_hugepages_to_shared_mem(struct hugepage_file * dst, int dest_size, return 0; } +static int +unlink_hugepage_files(struct hugepage_file *hugepg_tbl, + struct hugepage_info *hpi, + unsigned num_hp_info) +{ + unsigned socket, size; + int page, nrpages = 0; + + /* get total number of hugepages */ + for (size = 0; size < num_hp_info; size++) + for (socket = 0; socket < RTE_MAX_NUMA_NODES; socket++) + nrpages += internal_config.hugepage_info[size].num_pages[socket]; + + for (size = 0; size < num_hp_info; size++) { + for (socket = 0; socket < RTE_MAX_NUMA_NODES; socket++) { + for (page = 0; page < nrpages; page++) { + struct hugepage_file *hp = &hugepg_tbl[page]; + if ((hp->size == hpi[size].hugepage_sz) && + (hp->socket_id == (int) socket) && + hp->final_va != NULL) { + if (unlink(hp->filepath)) { + RTE_LOG(WARNING, EAL, "%s(): Removing %s failed: %s\n", + __func__, hp->filepath, strerror(errno)); + } + } + } /* foreach page */ + } /* foreach socket */ + } /* foreach pagesize */ + return 0; +} + /* * unmaps hugepages that are not going to be used. since we originally allocate * ALL hugepages (not just those we need), additional unmapping needs to be done. @@ -1289,6 +1320,14 @@ rte_eal_hugepage_init(void)