From patchwork Thu Nov 15 15:47:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Burakov, Anatoly" X-Patchwork-Id: 48129 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 342685587; Thu, 15 Nov 2018 16:47:36 +0100 (CET) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 96C883238 for ; Thu, 15 Nov 2018 16:47:25 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Nov 2018 07:47:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,236,1539673200"; d="scan'208";a="281360514" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by fmsmga006.fm.intel.com with ESMTP; 15 Nov 2018 07:47:22 -0800 Received: from sivswdev01.ir.intel.com (sivswdev01.ir.intel.com [10.237.217.45]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id wAFFlM17024815; Thu, 15 Nov 2018 15:47:22 GMT Received: from sivswdev01.ir.intel.com (localhost [127.0.0.1]) by sivswdev01.ir.intel.com with ESMTP id wAFFlMeD028157; Thu, 15 Nov 2018 15:47:22 GMT Received: (from aburakov@localhost) by sivswdev01.ir.intel.com with LOCAL id wAFFlMVO028153; Thu, 15 Nov 2018 15:47:22 GMT From: Anatoly Burakov To: dev@dpdk.org Cc: john.mcnamara@intel.com, bruce.richardson@intel.com, pablo.de.lara.guarch@intel.com, david.hunt@intel.com, mohammad.abdul.awal@intel.com, thomas@monjalon.net, ferruh.yigit@intel.com Date: Thu, 15 Nov 2018 15:47:21 +0000 Message-Id: <58e36e0a9bc7f9cb3d5a288feed1230984e72ee6.1542291869.git.anatoly.burakov@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [RFC v2 9/9] usertools/lib: add GRUB utility library for hugepage config X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This library is highly experimental and can kill kittens, but its main purpose is to automatically set up GRUB command-line to allocate a given number of hugepages at boot time. It works in a similar way HugeUtil library does, but instead of committing changes to fstab or runtime configuration, it commits its changes to GRUB default command-line and updates all GRUB entries afterwards. I got it to a state where it's safe to use on my system, but see the part above about killing kittens - you have been warned :) No example scripts will currently be provided. Signed-off-by: Anatoly Burakov --- usertools/DPDKConfigLib/GrubHugeUtil.py | 175 ++++++++++++++++++++++++ 1 file changed, 175 insertions(+) create mode 100755 usertools/DPDKConfigLib/GrubHugeUtil.py diff --git a/usertools/DPDKConfigLib/GrubHugeUtil.py b/usertools/DPDKConfigLib/GrubHugeUtil.py new file mode 100755 index 000000000..4b8e349b8 --- /dev/null +++ b/usertools/DPDKConfigLib/GrubHugeUtil.py @@ -0,0 +1,175 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + + +from .PlatformInfo import * +from .Util import * +import re +import os + +__KERNEL_TRANSPARENT_HP = "/sys/kernel/mm/transparent_hugepage/enabled" +_GRUB_CMDLINE_PARAM_NAME = "GRUB_CMDLINE_LINUX_DEFAULT" + +# local copy of platform info +info = PlatformInfo() + +def _find_linux_default_cmdline(): + with open("/etc/default/grub") as f: + for line in f: + line = line.strip() + if line.startswith(_GRUB_CMDLINE_PARAM_NAME): + return line + else: + raise RuntimeError("Invalid GRUB default configuration format") + + +def _parse_linux_default_cmdline(line): + # get value to the right of equals sign, strip whitespace and quotes, + # split into separate keys and make a list of values + _, cmdline = kv_split(line, "=") + # remove quotes + if cmdline[0] == cmdline[-1] == '"': + cmdline = cmdline[1:-1] + + return [kv_split(v, "=") for v in cmdline.split()] + + +def _generate_linux_default_cmdline(cmdline): + lines = [] + cmdline_idx = -1 + with open("/etc/default/grub") as f: + for idx, line in enumerate(f): + line = line.strip() + lines.extend([line]) + if line.startswith(_GRUB_CMDLINE_PARAM_NAME): + cmdline_idx = idx + if cmdline_idx == -1: + raise RuntimeError("Invalid GRUB default configuration format") + + # write the lines back, replacing one we want + with open("/etc/default/grub", "w") as f: + for idx, line in enumerate(lines): + if idx == cmdline_idx: + line = cmdline + f.write(line + "\n") + + +def _find_transparent_hugepage(): + if not os.path.exists(__KERNEL_TRANSPARENT_HP): + return None + value = read_file(__KERNEL_TRANSPARENT_HP) + m = re.search(r"\[([a-z]+)\]", value) + if not m: + raise RuntimeError("BUG: Bad regular expression") + return m.group(1) + + +class GrubHugepageConfig: + def __init__(self): + self.update() + + def update(self): + self.reset() + + hugepage_sizes = info.hugepage_sizes_supported + if len(hugepage_sizes) == 0: + raise RuntimeError("Hugepages appear to be unsupported") + cmdline = _find_linux_default_cmdline() + values = _parse_linux_default_cmdline(cmdline) + + # parse values in the list + self.default_hugepagesz = info.default_hugepage_size + self.transparent_hugepage = _find_transparent_hugepage() + sizes = [] + nrs = [] + for k, v in values: + if k == "default_hugepagesz": + self.default_hugepagesz = human_readable_to_kilobytes(v) + elif k == "transparent_hugepage": + self.transparent_hugepage = v + elif k == "hugepagesz": + sizes.append(human_readable_to_kilobytes(v)) + elif k == "hugepages": + nrs.append(v) + if len(sizes) != len(nrs): + raise RuntimeError("GRUB hugepage configuration is wrong") + detected_hugepages = dict(zip(sizes, map(int, nrs))) + self.nr_hugepages = {size: detected_hugepages.get(size, 0) + for size in hugepage_sizes} + + def commit(self): + # perform sanity checks - we can't afford invalid data making it into + # bootloader config, as that might render user's machine unbootable, so + # tread really really carefully + + # first, check if user didn't add any unexpected hugepage sizes + configured_sizes = set(self.nr_hugepages.keys()) + supported_sizes = set(info.hugepage_sizes_supported) + + if configured_sizes != supported_sizes: + diff = configured_sizes.difference(supported_sizes) + raise ValueError("Unsupported hugepage sizes: %s" % + [kilobytes_to_human_readable(s) for s in diff]) + + # check if default hugepage is one of the supported ones + if self.default_hugepagesz is not None and\ + self.default_hugepagesz not in configured_sizes: + s = kilobytes_to_human_readable(self.default_hugepagesz) + raise ValueError("Unsupported default hugepage size: %i" % s) + + # transparent hugepages support was added in recent kernels, so check + # if user is trying to set this + if _find_transparent_hugepage() is None and \ + self.transparent_hugepage is not None: + raise ValueError("Transparent hugepages are not unsupported") + + # OK, parameters look to be valid - let's roll + + # read and parse current cmdline + cmdline = _find_linux_default_cmdline() + + values = _parse_linux_default_cmdline(cmdline) + + # clear out old data + klist = ["transparent_hugepage", "default_hugepagesz", + "hugepage", "hugepagesz"] + # iterate over a copy so that we could delete items + for k, v in values[:]: + if k in klist: + values.remove((k, v)) + + # generate new cmdline + cmdline = " ".join([("%s=%s" % (k, v)) if v is not None else k + for k, v in values]) + + # now, populate cmdline with new data + new_items = [] + for sz, nr in self.nr_hugepages.items(): + sz = kilobytes_to_human_readable(sz) + new_items += "hugepagesz=%s hugepages=%i" % (sz, nr) + if self.default_hugepagesz is not None: + new_items += "default_hugepagesz=%i" % self.default_hugepagesz + if self.transparent_hugepage is not None: + new_items += "transparent_hugepage=%s" % self.transparent_hugepage + + cmdline = "%s %s" % (cmdline, " ".join(new_items)) + + # strip any extraneous whitespace we may have added + cmdline = re.sub(r"\s\s+", " ", cmdline).strip() + + # now, put everything back together + cmdline = '%s="%s"' % (_GRUB_CMDLINE_PARAM_NAME, cmdline) + + # write it to config + _generate_linux_default_cmdline(cmdline) + + # finally, update GRUB + if not run(["update-grub"]): + raise RuntimeError("Failed to update GRUB") + self.update() + + def reset(self): + self.nr_hugepages = {} # pagesz: number + self.default_hugepagesz = None + self.transparent_hugepage = None